Using Hazelcast ( In-Memory Data Grid )

Hazelcast is open source  in memory data grid that allows architects and developers easily design and develop faster, highly scalable and reliable applications.

Features

  • Distributed implementations of java.util.{Queue, Set, List, Map}.
  • Distributed Topic for publish/subscribe messaging.
  • Distributed listeners and events.
  • Dynamic clustering.
  • Dynamic fail-over.

Hazelcast is implemented in Java , it  uses Java NIO to implement non blocking I/O and it can dynamically discover and join the server cluster using IP Multicast .

Some use cases for  Hazelcast .

  • Designing highly scalable data store shared by multiple processes , multiple applications, or multiple servers.
  • Designing highly available distributed cache for your web application.
  • Designing distributed pub/sub.
  • Designing distributed queues.

Similar to Redis, Hazelcast can also be used to store session data in web application which can survive web server restarts and crashes. This setup also enables nice horizontal scalability of web application as shown below.

web-arch

As Event Router

One of the other uses case is to use it as distributed publish/subscribe server to  allows different application on the  same machine or across network to exchange messages in a loosely coupled fashion.

Following picture shows the architecture of such as system.  Clients ( publisher/subscribers ) connect to Event Router server to send/receive messages.

event-router

Server application source code

import com.hazelcast.core.*;
import com.hazelcast.config.Config;
import java.util.Map;
import java.util.Queue;
import java.io.*;
import java.util.*;
public class DemoRouter {	
    
    public static void main(String[] args) {
        
        Config cfg = new Config();		
        HazelcastInstance instance = Hazelcast.newHazelcastInstance(cfg);
        
        BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
        try {
            in.readLine();
        }catch(Exception exp) { }
        
        System.out.println("Exiting ..");		
    }
}

Subscriber client application source code

import com.hazelcast.core.*;
import com.hazelcast.client.config.*;
import com.hazelcast.client.*;
import java.util.Map;
import java.util.Queue;
import java.io.*;
import java.util.*;

public class SubscribeClient {

    public static void main(String[] args) {
        
        ClientConfig clientConfig = new ClientConfig();
        clientConfig.addAddress("127.0.0.1:5701");
        
        HazelcastInstance hz = HazelcastClient.newHazelcastClient(clientConfig);
        ITopic topic = hz.getTopic("default");
        topic.addMessageListener(new MessageListener<String>() {		
            public void onMessage(Message<String> message) {			
                String myEvent = message.getMessageObject();
                System.out.println("Message received = " + myEvent.toString());
            }
        });		
        BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
        try {
            in.readLine();
        }catch(Exception exp) { }
        System.out.println("Exiting ..");	
    }
}

Publisher  client application source code

import com.hazelcast.core.*;
import com.hazelcast.client.*;
import com.hazelcast.client.config.*;
import java.util.Map;
import java.util.Queue;
import java.io.*;
import java.util.*;

public class PublishClient {

    public static void main(String[] args) {
        ClientConfig clientConfig = new ClientConfig();
        clientConfig.addAddress("127.0.0.1:5701");
        
        HazelcastInstance hz = HazelcastClient.newHazelcastClient(clientConfig);
        ITopic topic = hz.getTopic("default");
        
        topic.publish("Hello from Client1");
        topic.publish("Hello from Client2");
        topic.publish("Hello from Client3");
        BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
        try {
            in.readLine();
        } catch (Exception exp) {
        }
        System.out.println("Exiting ..");
    }
}

Resource

1. Hazelcast documentation 

2. Using Hazelcast screencast

Advertisements

Build Media Center PC using Raspberry Pi

Raspberry Pi is a credit card sized computer built using Broadcom SOC (system on chip) created by Raspberry Pi foundation. It costs $35 for model B board which has 700 MHz ARM CPU, 512 MB RAM, 2 USB2 ports, 1 HDMI port ,1 RCA video port,Ethernet and audio port. It has SD card for installing Linux distribution.

Here is screenshot of Raspberry Pi board

rasppi

There are number of Linux distribution that target Raspberry Pi , including distribution to run Media Center software.

In this post I will talked about how I built my media center pc using Raspberry Pi and OpenELEC ( open embedded Linux entertainment center ) Linux distribution.

OpenELEC is a small Linux distribution with complete XBMC media center software. Compared to Raspbmc which is another Linux based media center distribution , OpenELEC is very small around 80MB in size. Because OpenELEC distribution  packages only software necessary for media center it is small and fast compared to Raspbmc.

Here is the screenshot of my Raspberry Pi board with all necessary connection .(HDMI connection to monitor, micro usb power connection, Ethernet connection and USB keyboard and mouse).

2012-12-27 19.55.18

Creating bootable SD image

Before powering up the board , you need download OpenELEC distribution from the OpenELEC website. Make sure that you download image which as RPi-arm name in it,  example : OpenELEC-RPi.arm-devel-20121124031454-r12577.tar.bz2

To create bootable OpenELEC image on SD card you can Windows/Mac/Linux OS workstation with SD card reader. Here are the steps to create SD image on Ubuntu 12.10 .

1. Unzip and un-tar OpenELEC to some folder ,

2. Insert 2GB blank SD card to your machine.

3. Execute create_sdcard shell script from OpenELEC folder with device path of the SD card.

Example

linux-vm:~/OpenELEC-RPi.arm$ ./create_sdcard  /dev/sdb

4. Once step 3  completes without any errors, SD card will have bootable OpenELEC image.

5. Now insert SD card to Raspberry Pi board and power up.

6. You will see the boot screen with OpenELEC and Raspberry Pi logo.

2012-12-28 11.34.03

7. After while you see familiar XBMC media center interface.

2012-12-28 11.35.00

8. Now you navigate the menu using keyboard and add your data source where you stored your media collection.

OpenELEC supports UPnP, DLNA,NFS,CIFS,SMB,FTP,SFTP and others file sharing protocols. Using these you should be able to browse the media collection stored on your PC , MAC, Linux or NAS box.

You can also connect your USB2 hard drive to Raspberry Pi and browse media collection from OpenELEC.  OpenELEC supports hard drive formatted using NTFS ,FAT/FAT32 and other Linux file system formats.

Once you have contents you play any audio/video/photo formats.

OpenELEC also supports Apple AirPlay as target. You can enable this from settings menu. Once enabled , you can stream photo and video from you IOS device( iPhone ,iPad ,iPod touch and MAC computers )  to OpenELEC connected device ( TV or monitor )

In the next blog post I will explore installing Java 8 on Raspberry Pi and running Java FX application.

Resource

1. Raspberry Pi : Main Raspberry Pi website.

2. OpenELEC : Open Embedded  Linux media center website.

3. OpenELEC on Raspberry Pi :  Instructions to install OpenELEC on Raspberry Pi

4. XBMC : XBMC wiki page .

5. Raspbmc :  Raspberry Pi media center distribution based with XBMC software.

6. Run Java Application on Raspberry Pi. Oracle TechNet article for installing Java on Raspberry Pi

7. Raspberry Pi accessories :  Collection of Raspberry Pi compatible accessories.

BTrace : Troubleshooting tool for Java applications

BTrace allows users to dynamically instrument running Java application using special Java annotations. Annotations act as directive to indicate where tracing code should be inserted in the target application.
For example you can insert code (tracing/debug) print all arguments to a method or print information whenever particular method is called along with caller information using stack trace.
All this can be done at runtime without requiring any source code changes to target application or restarting target application.
Here is how it works

At high level you write Java source file using the framework provided by the BTrace which allows you to print arguments ,trace method calls. When user executes external BTrace annotated Java code using BTrace client, it creates and invokes an agent in the target JVM ( through the Attach API ) and then it uses client to communicate with that agent to perform instrumentation.

This technique  is similar to aspect oriented programming  using aspectj where you write aspects  in external file and use aspectj compiler to weave byte code.Advantage of using BTrace is , you are changing byte code in running application.

How to Use BTrace

BTrace provides command line tool and plugin to Visual VM tool ( Visual VM is part of JDK release ).

In this blog post I will show you how to install and use BTrace plugin in Visual VM.

Visual VM is part of JDK since 6.0. You can also download from java.net.

Launch Visual VM and select “Tools\Plugins” option in Menu bar.

In the plugins dialog , select “Available Plugins” tab. One of the option listed is “BTrace workbench”. If it is not listed enter , “BTrace” in search box.

Note:  To get the available list of plugins ,make sure that you are connected to internet and proxy configuration is proper.

image

In the plugin dialog select “BTrace workbench” and install. Restart the Visual VM application so that newly installed plugin is loaded.

Once you restart the Visual VM , it will show all the running java application on your machine. In my case I am running “Notepad” demo application which ships with JDK demos ( JDK_HOME\demos\jfc\Notepad ).

Select the Notepad application , right click and select “Trace application…” from the menu, as show below.

image

Once “Trace application” option is selected following dialog pops up to allow user to enter java code trace “Notepad” application.

image

Now lets trace all the files opened for reading/writing by “Notepad” application. Remove the existing code from the window and copy/paste following code.

This is the sample code from code from BTrace library , which allows to trace file open for read/write.

/* BTrace Script Template */
import com.sun.btrace.annotations.*;
import static com.sun.btrace.BTraceUtils.*;

import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;

/**
 * This sample prints all files opened for read/write
 * by a Java process. Note that if you pass FileDescriptor
 * to File{Input/Output}Stream or File{Reader/Writer}, 
 * that is not tracked by this script.
 */
@BTrace public class FileTracker {
    @TLS private static String name;

    @OnMethod(
        clazz="java.io.FileInputStream",
        method="<init>"
    )
    public static void onNewFileInputStream(@Self FileInputStream self, 
                                            File f) {
        name = Strings.str(f);
    }

    @OnMethod(
        clazz="java.io.FileInputStream",
        method="<init>",
        type="void (java.io.File)",
        location=@Location(Kind.RETURN)
    )
    public static void onNewFileInputStreamReturn() {
        if (name != null) {
            println(Strings.strcat("opened for read ", name));
            name = null;
        }
    }

    @OnMethod(
        clazz="java.io.FileOutputStream",
        method="<init>"
    )
    public static void onNewFileOutputStream(@Self FileOutputStream self, 
                                              File f, boolean b) {
        name = str(f);
    }

    @OnMethod(
        clazz="java.io.FileOutputStream",
        method="<init>",
        type="void (java.io.File, boolean)",
        location=@Location(Kind.RETURN)
    )
    public static void OnNewFileOutputStreamReturn() {
        if (name != null) {
            println(Strings.strcat("opened for write ", name));
            name = null;
        }
    }
}

Select “Start” option . Now try to open any file in Notepad application or create new file , you will see trace message printed in the output windows shown below.

image

More samples available in BTrace User guide

Using Command Line version

BTrace also ships with command line tool which allows you debug/trace target application. From the command line you can use btrace client tool to trace Java application

btrace <target-java-pid> FileTracker.java

BTrace ships with number of sample application that shows various BTrace features. Download the sample application from the btrace website

Resouces

1. BTrace website

2. BTrace user guide ( user guide as number samples which demonstrate capbaility of BTrace ).

3. BTrace developers guide

4. BTrace wiki.

Change Java Logger level without application restart

In any application development logging is fundamental activity which helps developer to debug application issues during development & in production.

Java provides built in Logger API for logging messages to report error or any additional information about the running application.These messages can be of SEVERE ,WARING,INFO,CONFIG,FINE,FINER and FINEST Level. Usually application is configured to log message only of type WARNING and above ( which include SEVERE ) to avoid performance issues of logging lot of messages to log destination ( FILE or Socket ) at other levels .

If you want to change the log level to detail ( such as FINE and above ) , it requires changing the logging.properties file and application restart.

Many times we want to enable FINE logging of running application without application restart so that we can debug issues which are difficult to reproduce.

Using Java Management Extension API & JVM tool interface it is possible to change the running application Logger level  without application restart. This technique is used in JConsole application distributed with JDK.

But often it will be useful to have command line based interface so that it will be faster and does not need UI toolkit on production machine(headless system ).

Following sample code shows , how to change the logger level without application restart.

Note: This solution requires JDK tools ( does not work with JRE ) as it depends on tools.jar distributed with JDK. It is tested using                JDK 1.6.0_26.

This utility needs process id of the running application ( which can be got from jps utility ), name of the logger and level to change.

Steps:

1. Connect to running Java Application  using ‘Virtual Machine’ attach API

2. Get the JMX connector URL, if JMX argent is not loaded load JMX agent.

3. Connect to Management Bean server on the remote Java Virtual Machine.

4. Get access to LoggingMXBean instance .

5. Using MXBean instance change the logger level of the given logger.

For other logging toolkits(log4net … ) you can use either built-in facility ( if it exists ) or polling for configuration file change , if you are using JDK 7  you can also use directory changed notification API to monitor configuration change and reload the configuration for the toolkit.

import java.io.File;
import java.io.IOException;
import java.lang.management.ManagementFactory;
import java.util.logging.LogManager;
import java.util.logging.LoggingMXBean;

import javax.management.MBeanServerConnection;
import javax.management.remote.JMXConnector;
import javax.management.remote.JMXConnectorFactory;
import javax.management.remote.JMXServiceURL;

import com.sun.tools.attach.AttachNotSupportedException;
import com.sun.tools.attach.VirtualMachine;
import com.sun.tools.attach.VirtualMachineDescriptor;

/**
 * Sample requires JDK installation,because it uses 
 * VirtualMachine attach api present in tools.jar.
 * Following examples shows how to use this api. For example ,
 * you have java application which using java logger to log
 * information at various logger levels.By default application 
 * is shipped to log at certain level. 
 * Assuming that it is WARING level. If you want to change the
 * logger level to INFO ,without application restart do the 
 * following steps
 * 1.Find tha process id of the running Virtual machine. 
 * 	You can use jps.exe -lvm ( ships with jdk ). 
 * 2.Run following command 
 * ChangeLogLevel -cp $JAVA_HOME\lib\tools.jar:. <process_id> <logger_name> <logger_level>
 * ChangeLogLevel -cp $JAVA_HOME\lib\tools.jar:. 1234 xyz.perf.log FINE
 */
public class ChangeLogLevel {

	private static final String JMX_CONNECTOR_ADDRESS = 
			"com.sun.management.jmxremote.localConnectorAddress";

	
	 // List the existing Java VM on this machine.
	 
	public static void listExistingVMs() {
		try {
			for (VirtualMachineDescriptor vd : VirtualMachine.list()) {
				System.out.println(vd.displayName() + " - " + vd.id());
			}
		} catch (Exception e) {
			e.printStackTrace();
		}

	}

	
	 // Get JMX URL for given process id
	 
	private static JMXServiceURL getURLForPid(String pid) 
			throws Exception {

		// attach to the target application
		VirtualMachine vm = VirtualMachine.attach(pid);

		// get the connector address
		String connectorAddress = vm.getAgentProperties().getProperty(
				JMX_CONNECTOR_ADDRESS);

		// no connector address, start the JMX agent
		if (connectorAddress == null) {
			String agent = vm.getSystemProperties().
					getProperty("java.home")
					+ File.separator + "lib" + File.separator
					+ "management-agent.jar";
			vm.loadAgent(agent);
			// agent is started, get the connector address
			connectorAddress = vm.getAgentProperties().getProperty(
					JMX_CONNECTOR_ADDRESS);
			if (connectorAddress == null) {
				throw new Exception("Fail to get jmx address");
			}
		}
		return new JMXServiceURL(connectorAddress);
	}

	
	 // Changes the log level of the given logger
	 
	public static void changeLevel(MBeanServerConnection remoteConnection,
			String loggerName, String loggerLevel) throws IOException {

		LoggingMXBean logBean = ManagementFactory.newPlatformMXBeanProxy(
				remoteConnection, LogManager.LOGGING_MXBEAN_NAME,
				LoggingMXBean.class);

		String currentLogLevel = logBean.getLoggerLevel(loggerName);
		System.out.println("Current log level for " + loggerName + " is "
				+ currentLogLevel);
		logBean.setLoggerLevel(loggerName, loggerLevel);
		String changedLogLevel = logBean.getLoggerLevel(loggerName);
		System.out.println("After the change,log level for " + loggerName
				+ " is " + changedLogLevel);
	}

	public static void main(String[] args) throws IOException {

		System.out.println("List of Running Java VMs");
		ChangeLogLevel.listExistingVMs();

		if (args.length < 3) {
			System.out
					.println("Usage:ChangeLogLevel <pid> <logger_name> <log_level>");
			System.out
					.println("Example:ChangeLogLevel 2121 com.test.xyz INFO");
			System.out.println("Example : ");
			System.exit(0);
		}

		JMXServiceURL url = null;
		try {
			url = getURLForPid(args[0]);
		} catch (Exception e) {
			e.printStackTrace();
		}

		// Connect to target vm
		JMXConnector connector = null;
		try {
			connector = JMXConnectorFactory.connect(url);

			// Get an MBeanServerConnection on the remote VM.
			MBeanServerConnection remote = connector.
					getMBeanServerConnection();
			changeLevel(remote, args[1], args[2]);

		} catch (Exception e) {
			e.printStackTrace();
		} finally {
			if (connector != null) {
				connector.close();
			}
		}

	}

}

How to current get executing method name & caller of the method in Java

Recently I was investigating issue in our internal software , where I needed to get the current executing method name ( similar to c++ __FUNCTION___ )  and caller name.

It is possible get this information using getting access to stack trace in java. see the example below.

public class DebugInfo {
  /**
   *
   * This function returns the method name of the calling functions parent
   * function
   *
   * @return
   */
  public static String getCallingMethodName() {

    StackTraceElement stack[] = Thread.currentThread().getStackTrace();

    StackTraceElement element = stack[3];

    return element.toString();
  }
  /**
   *
   * This function returns the method name of the calling function
   *
   * @return
   */
  public static String myMethodName() {

    StackTraceElement stack[] = Thread.currentThread().getStackTrace();

    StackTraceElement element = stack[2];

    return element.toString();
  }
}

Java2html
Usage:
public class Foo
{
public void bar()
{
   // print the current method name
   System.out.println( DebugInfo.myMethodName());
   // print the current calling method name
   System.out.println( DebugInfo.getCallingMethodName());
}
public static void main(String [] args )
{
    Foo f = new Foo();
            f.bar();
}
}