Tracking application exceptions with logger


, , , , , ,

If you have service which is running in production and you want to be sure that everything is working as expected, and that service doesn’t have nice alerting system integrated you can use logging system instead and create simple alerting around it as first aid solution. My implementation is based on log4j backend. Idea is to create custom Appender which will get all logging events ( if it’s possible filtered to some level ) and report them to maintainer or support ( via email or some other way of communication ).

import org.apache.log4j.AppenderSkeleton;
import org.apache.log4j.Level;
import org.apache.log4j.spi.LoggingEvent;
public class AlerterAppender extends AppenderSkeleton {
    protected void append(LoggingEvent loggingEvent) {
        // level should be set in log4j properties or other backend configuration
        // but just in case we will check it once more
        if (loggingEvent.getLevel() == Level.ERROR) {
            StringBuilder sb = new StringBuilder();
            sb.append("Error: ").append(loggingEvent.getMessage());
            sb.append(" at thread: " + loggingEvent.getThreadName()).append("\n");
            String[] stack = loggingEvent.getThrowableStrRep();
            if (stack != null) {
                for (String call: stack) {

            // TODO send email

    public void close() {
        // empty

    public boolean requiresLayout() {
        return false;

After you have your custom appender in place you only need to add it in log4j configuration:

log4j.rootLogger=INFO, ..., Alerter

Outputting LZO compressed and indexed data from Map Reduce tasks


, , , , , , ,

Hadoop distribution currently doesn’t support outputting of LZO compressed data which will be indexed at same time. This feature is extremely useful if data you create with one Map Reduce task is used as input of another. With some changes of default TextOutputFormat I managed to accomplish this. Idea is to create specific LzoOutputFormat class which will be set as output format of task. Example is bellow

LzoIndexedOutputFormat class:

public class LzoIndexedOutputFormat<K, V> extends TextOutputFormat<K, V> {
    public RecordWriter<K, V> getRecordWriter(TaskAttemptContext job) throws IOException, InterruptedException {
        // copied from TextOutputFormat class in Hadoop source code
        Configuration conf = job.getConfiguration();
        String keyValueSeparator =  conf.get("mapred.textoutputformat.separator", "\t");

        LzopCodec lzopCodec = new LzopCodec();

        Path filePath = getDefaultWorkFile(job, lzopCodec.getDefaultExtension());
        FileSystem fs = filePath.getFileSystem(conf);

        Path indexFilePath = new Path(filePath.toString() + ".index");

        FSDataOutputStream fileOut = fs.create(filePath, false);
        FSDataOutputStream fileIndexOut = fs.create(indexFilePath, false);

        OutputStream finalStream = lzopCodec.createIndexedOutputStream(fileOut, new DataOutputStream(fileIndexOut));

        return new LineRecordWriter<K, V>(new DataOutputStream(finalStream), keyValueSeparator);

To use this newly created OutputFormat use:

Job job = new Job(new Configuration(), "lzo job");


Helper class for NSNotificationCenter


, , , ,

Last weekend I tried to modify one of apps to use NSNotificationCenter instead of complicated custom code to notify more than one object about change in data. For this purpose I wrote simple helper class.


#import <Foundation/Foundation.h>
#import "NotificationDelegate.h"

@interface NotificationHelper : NSObject

+ (void)pushNotification:(NSString*)notification WithObject:(id)object;
+ (void)registerForNotification:(NSString*)notification WithDelegate:(id)delegate;
+ (void)unregisterForNotification:(id)delegate;



#import "NotificationHelper.h"

@implementation NotificationHelper

+ (void)pushNotification:(NSString*)notification WithObject:(id)object
     [[NSNotificationCenter defaultCenter] postNotificationName:notification object:object];

+ (void)registerForNotification:(NSString*)notification WithDelegate:(id)delegate
     [[NSNotificationCenter defaultCenter] addObserver:delegate selector:@selector(reactOnNotification:) 
           name:notification object:nil];

+ (void)unregisterForNotification:(id)delegate
     [[NSNotificationCenter defaultCenter] removeObserver:delegate];


and NotificationDelegate protocol:

#import <Foundation/Foundation.h>

@protocol NotificationDelegate

- (void)reactOnNotification:(NSNotification*)notification;


Usage is pretty simple. All receivers should implement NotificationDelegate and register for notifications with call to NotificationHelper. Code which should be executed when message is recieved goes inside reactOnNotification:(NSNotification*)notification. Message sender should to call NotificationHelper’s pushNotification when they have message.

I pushed sample application on my github.


First steps with Jade, Node template engine


, , , , , , ,

During experimenting with Jade I tried to create as simple as possible server / client code for serving of simple HTML file generated with Jade. Result are two examples that I will try to explain briefly, one using server side to generate HTML and second using client side to generate HTML.

Client side example uses Node as server of Jade template files which will be obtained by Ajax on client side, compiled, rendered and added to HTML. For server I used modified version of Simple static file HTTP server with Node.js. Only change is one extra if for files with jade extension which will look inside jade sub directory.

    } else if (filename.match(".jade$")) {
        contentType = "text/plain";
        pathToRead = "jade/" + filename;

Magic happens on client side during load of html:

    $(document).ready(function() {
        $.ajax({url: "index.jade", success:function(data) {
            var fn = jade.compile(data);
            var html = fn({});

Server side example uses Node to generate HTML which will be pushed back to client in rendered HTML form. Code for sever is different from one used in first example. Basically, we assume that each HTML file have corresponding Jade template, try to fetch that template, parse it, render and serve back to client as HTML.

var fs = require("fs");
var url = require("url");
var jade = require("jade");
var connect = require("connect");

connect.createServer(function(req, res){
    var request = url.parse(req.url, false);
    var filename = request.pathname.slice(1);

    if (request.pathname == '/') {
        filename = 'index.html';

	console.log("Serving request: " + request.pathname + " => " + filename);

    var jadeFilename = "jade/" + filename.slice(0, filename.lastIndexOf(".")) + ".jade";

    console.log("Serving jade file: " + jadeFilename);

	try {
	} catch (e) {

	fs.readFile(jadeFilename, function(err, data) {
		if (err) {

        res.writeHead(200, {"Content-Type": "text/html"});

        var fn = jade.compile(data);
        var html = fn({});


You can get code for examples from my GitHub repo. To start any of these use node base/server.js inside example directory.

Drawing line to UIImage using CoreGraphics [ iOS ]


, , , , , ,

How to draw single line into UIImage using Core Graphics framework? Recently I needed this, so here is code ( everything is straightforward -> no explanation is needed 🙂 ):

    NSLog(@"Creating image");

    CGSize size = CGSizeMake(240.0f, 240.0f);
    CGContextRef context = UIGraphicsGetCurrentContext();

    CGContextSetStrokeColorWithColor(context, [[UIColor blackColor] CGColor]);
    CGContextSetFillColorWithColor(context, [[UIColor whiteColor] CGColor]);

    CGContextFillRect(context, CGRectMake(0.0f, 0.0f, 240.0f, 240.0f));

    CGContextSetLineWidth(context, 5.0f);
    CGContextMoveToPoint(context, 100.0f, 100.0f);
    CGContextAddLineToPoint(context, 150.0f, 150.0f);

    UIImage* result = UIGraphicsGetImageFromCurrentImageContext();

    imageView.image = result;
    [imageView setNeedsDisplay];

    NSLog(@"Image creation finished");

Example can be found on my GitHub


Hadoop: How to get active jobs in cluster


, , ,

Recently I was making Hadoop alerting infrastructure and I needed something to track active jobs in cluster.

So, for start you need instance of JobClient. JobClient is wrapper around JobTracker RPC, basically
under the hood JobClient creates JobSubmissionProtocol instance:

    public void init() throws IOException {
        String tracker = conf.get("mapred.job.tracker", "local");
        if ("local".equals(tracker)) {
          this.jobSubmitClient = new LocalJobRunner(conf);
        } else {
          this.jobSubmitClient = (JobSubmissionProtocol) 
                         JobTracker.getAddress(conf), conf);

Let’s code:

import org.apache.hadoop.mapred.JobClient;

initialize JobClient instance:

JobClient jobClient = new JobClient(new InetSocketAddress(jobTrackerHost, jobTrackerPort), new Configuration());

where jobTrackerHost and jobTrackerPort are host name and port where Job Tracker is running …

To get list of currently active jobs in cluster all you have to do is:

JobStatus[] activeJobs = jobClient.jobsToComplete();

This will give you list of all active records, JobStatus is pretty useful. You can get jobId, username that was used to summit job, start time…

I created sample code which will every n seconds output number of active jobs and their info, look on my GitHub.

Hadoop “Could not complete file …” issue


, , , , ,

Recently I run into problem with Hadoop, causing DFSClient to stop responding and run into infinite while displaying “Could not complete file …” message. Hadoop version is 0.20.1, svn version 810220, but it seems to me from code that this issue can occur on newer version too.

NameNode logs are showing that file was created, blocks where assigned to it and there is no complete message in logs. In DataNode logs there is Exception which looks like network connectivity issue.

I found this issue on web HDFS-148 and it seems that I have same problem. My biggest problem is that I cannot replicate issue, it happens once in let’s say month.

After some digging in code I found part that causes me trouble:

    private void completeFile() throws IOException {
      long localstart = System.currentTimeMillis();
      boolean fileComplete = false;
      while (!fileComplete) {
        fileComplete = namenode.complete(src, clientName);
        if (!fileComplete) {
          if (!clientRunning ||
                (hdfsTimeout > 0 &&
                 localstart + hdfsTimeout < System.currentTimeMillis())) {
              String msg = "Unable to close file because dfsclient " +
                            " was unable to contact the HDFS servers." +
                            " clientRunning " + clientRunning +
                            " hdfsTimeout " + hdfsTimeout;
              throw new IOException(msg);
          try {
            if (System.currentTimeMillis() - localstart > 5000) {
    "Could not complete file " + src + " retrying...");
          } catch (InterruptedException ie) {

So, as I can conclude from logs, DFSClient entered this while loop and is constantly outputting:"Could not complete file " + src + " retrying...");

From some reason file is never completed ( name node doesn’t have complete call in logs, probably some network issue ), but completeFile should throw IOException when this is fulfilled:

if (!clientRunning || (hdfsTimeout > 0 && localstart + hdfsTimeout < System.currentTimeMillis()))

By default hdfsTimeout is set to -1 and client is running so this piece of code that throws exception is never executed. Code that sets hdfsTimeout in Client looks like:

  final public static int getTimeout(Configuration conf) {
    if (!conf.getBoolean("", true)) {
      return getPingInterval(conf);
    return -1;

I tried to look more about setting ping to false, found this HADOOP-6099. I will try to play with disabling ping but it’s hard because I can’t recreate issue.

Node.js on Amazon EC2


, , , , ,

What’s fastest way to start up EC2 instance and run Node.js on it which will serve simple http requests?

So, you created Amazon AWS account and wanted to give it a try. As Amazon promised you will have micro instance for one year free of charge, let’s use it 🙂

I found various posts how to run Node.js with already prepared image or some third party EBS ( Elastic Block Store ) images but I wanted to do it from scratch. Here are steps:

0. Log in to AWS Management Console: Here
1. Fire up micro instance, I’m using “Lunch Classic Wizard” with Basic 64-bit Amazon Linux AMI.
2. Setup key pair if you don’t have it already. Download to computer. You will need them to access instance after start up is finished
3. Under “Configure firewall”, be sure to add ssh and http ports exceptions. For ssh, tcp port 22 with source: and for http, tcp port 80 with source I added range 8000 – 9000 ( just in case I need it later ). Note: for https different port is used than for http.
4. Lunch instance

After instance is lunched you will see instance details under “Instances”. Copy public DNS we will use it to attach to instance ( something like )

Note: There is possibility to assign static IP address to this instance. This feature is free as I understood as long it’s linked to active instance. If instance is down and given IP is kept you need to pay. Smart way to convince you to run your instance 24×7 :). For more details see Amazon AWS pricing and “Elastic IPs” under AWS Console.

So to continue our journey, you have active instance, so let’s use it:

5. ssh to your instance ssh -i //ec2.pem ( you will probably need to change certificate permissions to 700 )
6. do sudo su to get the feeling 🙂

You have root access to you machine, let’s install Node.js and run it.

7. Install required packages

yum install gcc gcc-c++ openssl-devel make

8. install Node.js

tar -zxf node-v0.6.5.tar.gz
cd node-v0.6.5
sudo make install
sudo chown -R ec2-user /usr/local
curl | sh

After this steps you should have node installed. To check it out you can download some simple node server ( I will use one from one of earlier posts Simple static file HTTP server with Node.js )

9. install git

yum install git

10. become ec2-user ( use exit command if you are root )
11. checkout code to home dir

git clone git://
cd JavaScriptPlayground/nodejs_static_file_server

12. run Node.js

node server.js

You can test everything by going with you browser to

Note: if you want to run Node.js on default HTTP port ( 80 ) you need to do so as root and of course to change server.js code to start server on port 80

13. change server.js


14. run Node.js as root

sudo /usr/local/bin/node server.js

Scrolling view to show fields hidden by keyboard on iOS


, , , , , , , , ,


During some JSON framework testing I found interesting problem, on screen keyboard was hiding one of text fields. Trying to figure out how to fix this problem I found two approaches, basically both are using same technique, move view up when keyboard is shown and return it to original position when keyboard is dismissed. First, cleaner approach, uses UIScrollView to do moving also recommended in Text, Web, and Editing Programming Guide for iOS under “Moving Content That Is Located Under the Keyboard”. Second approach is to use plain UIView and do moving manually. For production purpose I will definitely use first, but now I wanted to give a chance to second.


I will create simple project that will contain four text fields and one text view. All five elements will report to single view controller via UITextFieldDelegate and UITextViewDelegate. On textFieldDidBeginEditing and textViewDidBeginEditing view is moved if needed. When textFieldDidEndEditing and textViewDidEndEditing are triggered view will be moved to original position. On screenshots given bellow implemented functionality is shown. On first screen all five elements are shown, on second screen keyboard is active and text field with text: “test 4” is edited.

screenscreen with keyboard shown


Showing textFieldDidBeginEditing:

- (void)textViewDidBeginEditing:(UITextView *)textView

    if (textView.frame.origin.y + textView.frame.size.height > 480 - 216) {
        double offset = 480 - 216 - textView.frame.origin.y - textView.frame.size.height - 20;
        CGRect rect = CGRectMake(0, offset, self.view.frame.size.width, self.view.frame.size.height);

        [UIView beginAnimations:nil context:NULL];
        [UIView setAnimationDuration:0.3];

        self.view.frame = rect;

        [UIView commitAnimations];

And textViewDidEndEditing:

- (void)textViewDidEndEditing:(UITextView *)textView

    CGRect rect = CGRectMake(0, 20, self.view.frame.size.width, self.view.frame.size.height);

    [UIView beginAnimations:nil context:NULL];
    [UIView setAnimationDuration:0.3];

    self.view.frame = rect;

    [UIView commitAnimations];     

Also, hideKeyboard is used to hide keyboard when “Done” is pressed ( all text fields are calling this selector on “Did End On Exit”.

- (IBAction)hideKeyboard:(id)sender
    [sender resignFirstResponder];

Source code is available on my GitHub under iOS Playground repo. Project is called OnKeyboardViewResize.

HTTPS client for iOS


, , , , , ,

Some time ago I blogged about client/server implementation for two way SSL ( both client and server are authenticated with certs ) on top of nodeJS, ( blog post ). Now, I’m trying to connect iOS client to nodeJS HTTPS server.

I will use NSURLConnection for requests. Also, client authentication is only possible with asynchronized requests, simplified sendSynchronousRequest doesn’t support delegates. I will load p12 generated certificate from App resources for authentication ( maybe better idea is to use Keychain for “production” applications but here I just want to test things )

Example Xcode project is available on HttpSSLClient – GitHub

So, when request is made to HTTPS server which requires authentication with certificate delegate’s connection:didReceiveAuthenticationChallenge is called. In this method we need to obtain certificate ( for code simplicity I will load cert each time from resource ) and present it to sender which will use that certificate against server.

Note: I’m playing here with self signed certificates and code didn’t work until I added this ( thanks to guy that found this ). Also, for simplicity I’m returning YES without any validation here.

- (BOOL)connection:(NSURLConnection *)connection canAuthenticateAgainstProtectionSpace:(NSURLProtectionSpace *)protectionSpace
    return YES;

Back on track, to load certificate I’m using:

NSString *path = [[NSBundle mainBundle] pathForResource:@"userA" ofType:@"p12"];
NSData *p12data = [NSData dataWithContentsOfFile:path];
CFDataRef inP12data = (__bridge CFDataRef)p12data;
SecIdentityRef myIdentity;
SecTrustRef myTrust;
OSStatus status = extractIdentityAndTrust(inP12data, &myIdentity, &myTrust);
SecCertificateRef myCertificate;
SecIdentityCopyCertificate(myIdentity, &myCertificate);
const void *certs[] = { myCertificate };
CFArrayRef certsArray = CFArrayCreate(NULL, certs, 1, NULL);

extractIdentityAndTrust function is copied from Apple Certificate, Key, and Trust Services Programming Guide with slightly modifications:

OSStatus extractIdentityAndTrust(CFDataRef inP12data, SecIdentityRef *identity, SecTrustRef *trust)
    OSStatus securityError = errSecSuccess;
    CFStringRef password = CFSTR("userA");
    const void *keys[] = { kSecImportExportPassphrase };
    const void *values[] = { password };
    CFDictionaryRef options = CFDictionaryCreate(NULL, keys, values, 1, NULL, NULL);
    CFArrayRef items = CFArrayCreate(NULL, 0, 0, NULL);
    securityError = SecPKCS12Import(inP12data, options, &items);
    if (securityError == 0) {
        CFDictionaryRef myIdentityAndTrust = CFArrayGetValueAtIndex(items, 0);
        const void *tempIdentity = NULL;
        tempIdentity = CFDictionaryGetValue(myIdentityAndTrust, kSecImportItemIdentity);
        *identity = (SecIdentityRef)tempIdentity;
        const void *tempTrust = NULL;
        tempTrust = CFDictionaryGetValue(myIdentityAndTrust, kSecImportItemTrust);
        *trust = (SecTrustRef)tempTrust;
    if (options) {
    return securityError;

After certificate is loaded we need NSURLCredential that will be sent to certificate challenger

NSURLCredential *credential = [NSURLCredential credentialWithIdentity:myIdentity certificates:(__bridge NSArray*)certsArray persistence:NSURLCredentialPersistencePermanent];
[[challenge sender] useCredential:credential forAuthenticationChallenge:challenge];

For debug I’m using nodeJS server and certificate for userA is taken from that Git repo.


NSURLConnection class
Apple URL loading guide
Apple Certificate, Key, and Trust Services Programming Guide
NSURLConnection with Self-Signed Certificates
How to use Client Certificate Authentication in iOS App
How to connect with client certificate using a WebView in Cocoa?