Integrating Java and Erlang

Building scalable, distributed and reliable systems is trivial in Erlang. This can be a hard pill to swallow for many enterprise developers. Erlang is a dynamically typed functional language and Java is a statically typed object-oriented language. Erlang is an agitator to traditional enterprise development because it excels so well at concurrency, uptimes of five nines or more, and "hot deployment" of code. But there are valid reasons for why someone may not want to dive in head first. How many CIOs want to lose their investments in Java? Who wants to leave behind all the great libraries produced by the Java community?

This article is for a lot of people: language enthusiasts, software fashion victims, or anyone who wants to create serious business value bridging Erlang with Java. We'll start with an introductory tour of Erlang by building a simple client server application. Following this we'll reverse engineer the application in pure Java using Jinterface, an open source Java library distributed with Ericsson's Open Telecom Platform. Then we'll wrap things up with a few words about hybrid systems development. Download and install Erlang and Java if you wish to try this at home.
Concurrent Programming with a Simple Client Server Module

Erlang is a freak of nature; it has concurrency in its DNA. Concurrency is a native construct, all the way from the syntax down to guts of the Erlang virtual machine. This is fundamentally different than the traditional approaches to concurrency: expensive third party products, complex APIs for distribution, and java.lang.Thread. Let's begin our overview of Erlang with a little concurrent programming.

In an effort to stay focused on the core concepts we will avoid the deep end for now and keep it simple ... by calculating the sum of two numbers. The logic needed to perform addition will be contained within an Erlang module and exposed via an Erlang process. Afterwards we'll consume this service from a client process. With a little stretch of the imagination I'm sure most of you can envision the possibilities, like distributing a financial algorithm across 32 cores - or 32 cores on 32 machines.

An Erlang module consists of annotations and functions. For the purposes of this article, think of an Erlang module as a Java package. Our module starts with two annotations in a single file named mathserver.erl.

-module(mathserver).
-export([start/0, add/2]).

The first annotation declares the module name. The second annotation declares which functions are exported by the module; this is similar to using the keyword "public" in Java. The exported functions of this module are start and add, which we'll implement shortly. Did you notice that each line ends with a period? Functions and annotations in Erlang are terminated with a period, not a semicolon. Now let's create our first function, the server entry point for the mathserver module.

start() ->
   Pid = spawn(fun() -> loop() end),
register(mathserver, Pid).

The start function creates an Erlang process with one of the Erlang concurrency primitives, the spawn function. The term "process" can be confusing to newcomers, who often (understandably) misinterpret it as an operating system process. An Erlang process is more like a Java Thread, only extremely lightweight. Feel free to spawn thousands if you need to. Each process has a private heap. This design insures no shared state among processes. Processes are therefore easily parallelized and messages are passed by value, as opposed to being passed by reference. All intra-process communication in Erlang is asynchronous and all processes run in parallel. Read a little about the Actor Model for a more conceptual perspective on Erlang processes.

The argument passed to the spawn function, fun() -> loop () end , is an anonymous function. It represents the behavior of the to-be-spawned process. This is similar to a closure in Javascript, or a lambda in Ruby; in Erlang, it is called a fun. This fun wraps a named function, called loop, which we'll get to later.

Each process has a process identifier, or pid. The spawn function returns the pid of the spawned process. A pid can be conceptually thought of as a mailing address for a process. It is the means by which messages are passed to each private mailbox of a process. There is a one to one to one relationship between a pid, process and mailbox.

The start function ends using the built-in register function to register the pid as "mathserver". Once registered, we can forget about tracking the pid value and simply address the spawned process via this alias constant.

Erlang is more restrictive than Java when it comes to variables. For example, did you notice the pid variable starts with a capital letter? This is true for all variables in Erlang. Erlang also enforces "no side effects" with single assignment, so the value bound to the Pid variable is constant. In Java this is a choice made by the developer, using the keyword final.

Remember the anonymous function we passed to spawn? That function wrapped the loop function, which represents the actual behavior of the spawned process.

loop() ->
   receive
      {From, {add, First, Second}} ->
        From ! {mathserver, First + Second},
        loop()
   end.

The loop function uses another one of the Erlang concurrency primitives, the receive statement. When a message is sent to the mailbox of an Erlang process, it is pulled out of the mailbox and matched against each pattern in the receive block. Like a lot of other functional programming languages Erlang makes heavy use of pattern matching. Feel free to temporarily think of receive statements as switch blocks, and receive patterns as case labels (Erlang also has a case statement). This particular receive statement only matches for a single tuple, { From , { add , First , Second }}.

Tuples are fixed ordered lists of data and they are common in Erlang. The first element of this tuple is the pid of the sending process, bound to the variable From. This acts as a reply-to mailing address. The second element of this tuple is another tuple, consisting of an atom and the two terms to be added, First and Second. What is an atom? For the purposes of this article, let's just think of atoms as constants that are never garbage collected.

After the incoming message is matched against this tuple pattern a tuple response message is sent back to the client process via the reply-to pid.

From ! {mathserver, First + Second}

This illustrates a third Erlang concurrency primitive: the send operator. Messaging can be challenging on most programming platforms: Erlang has it down to a single character. When we see "Pid ! Msg" in Erlang, it means "send Msg to Pid". This allows us to sum both terms and send a response back to the math client with one line of code, fire and forget. The response is a tuple consisting of two elements, the mathserver atom and the sum.

Some of you might have laughed when you read the loop function: it ends with recursion. How reliable can this application be when it's only a matter of time before the runtime produces the Erlang equivalent of a java.lang.StackOverflowError? You don't have to worry about this. When the compiler encounters tail recursion it will optimize the Erlang bytecode so that the function can run indefinitely without consuming stack space.

Our last module function is the add function. This function is used to send and receive messages with the mathserver process. It uses many of the concepts and constructs previously covered: the send operator, tuples, the receive statement and pattern matching.

add(First, Second) ->
   mathserver ! {self(), {add, First, Second}},
   receive
      {mathserver, Reply} -> Reply
   end.

Look at the first statement. It creates a tuple message and asynchronously passes this to the mathserver process via the send operator.

mathserver ! {self(), {add, First, Second}}

Pay attention to two things here. First, to the right of the send operator we are not using the server pid. Instead we are sending the message to the mathserver process using the registered atom, mathserver. This is why we registered the server process back in the start function. Second, the self function has been introduced. The self function is a built-in function used to obtain the pid of the current process. For lack of a better analogy, think of the keyword "this" in Java. It is important the client sends its own pid as part of the message, otherwise the server would not know who to reply to. The add function ends with a receive statement used to match the reply message tuple sent by the server. When a message matching { mathserver , Reply } is pulled from the client mailbox, the second element of the message is returned. This completes the module.

-module(mathserver).
-export([start/0, add/2]).

start() ->
   Pid = spawn(fun() -> loop() end),
   register(mathserver, Pid).

loop() ->
   receive
      {From, {add, First, Second}} ->
        From ! {mathserver, First + Second},
        loop()
   end.

add(First, Second) ->
   mathserver ! {self(), {add, First, Second}},
   receive
      {mathserver, Reply} -> Reply
   end.

Using the mathserver Module in the Erlang Emulator

Those of you familiar with BeanShell, irb or jirb will find yourselves at home with erl. Let's run a few commands.

$ erl -name servernode -setcookie cookie
1> node().
'[e-mail]'
2> pwd().
/work/tss/article
ok
3> ls().
mathserver.erl
ok
4> c(mathserver).
{ok,mathserver}
5> ls().
mathserver.beam          mathserver.erl
6> mathserver:start().
true
7> mathserver:add(1,2).
3

What did that do? The shell command created an Erlang node called servernode. The built-in node function verifies this on line one. Lines two and three tell us the present working directory and what we have in it. Line four compiles and loads the mathserver module. The directory listing on line five reveals that the compile command has created a new file, mathserver.beam. Think of this as a .class file in Java. Line six starts the mathserver, spawning the server process. Finally we test our service with the add function.

It is now trivial to consume this service from any process on any node from any host as long as we specify the same cookie. On another machine, we can do this:

$ erl -name clientnode -setcookie cookie
1> node().
'[e-mail]'
2> rpc:call([e-mail], mathserver, add, [1,2]).
3

In this shell we create a node named clientnode on the host dbyrne.net and use a function from the built-in rpc module. The call function takes four arguments: the fully qualified name of the target node, the name of the module containing the to-be-invoked function, the remote function name, and the arguments to be passed to the remote function.

Jinterface, Getting the Best of Both Worlds

Jinterface allows us to create a cluster of Erlang and/or Java nodes. It is distributed with many other useful libraries in the Open Telecom Platform. The OtpErlang.jar file has no dependencies and can be found under <ERLANG_INSTALL_DIR>/erlx.x.x/lib/jinterface-1.x/priv. All source code is open and licensed under the Erlang Public License, a child of the Mozilla Public License.

Remember when we remotely invoked the mathserver service from the clientnode node? Let's do this from Java by porting the clientnode process to a ClientNode class.

OtpSelf cNode = new OtpSelf("clientnode", "cookie");
OtpPeer sNode = new OtpPeer("[e-mail]");
OtpConnection connection = cNode.connect(sNode);

First we create an OtpSelf instance. The OtpSelf constructor takes two arguments: the node name and the cookie. Does this remind you of one of the command lines typed earlier? The node name and cookie values are identical.

$ erl -name clientnode -setcookie cookie

The second line of code creates a representation of the remote server node with an OtpPeer instance. The OtpPeer constructor takes the fully qualified node name of the server. Does this node name look familiar?

$ erl -name servernode -setcookie cookie
1> node().
'[e-mail]'

Remember when we made a remote procedure call in Erlang?

   > rpc:call([e-mail], mathserver, add, [1,2]).

Now that a connection has been established it is time to do this in Java.

OtpErlangObject[] args = new OtpErlangObject[]{
new OtpErlangLong(1), new OtpErlangLong(2)};
connection.sendRPC("mathserver", "add", args);
OtpErlangLong sum = (OtpErlangLong) connection.receiveRPC();
assertEquals(3, sum.intValue());

The sendRPC method arguments are a one to one conceptual match with the rpc:call function arguments. We specify the name of the module containing the to-be-invoked function, the remote function name, and the arguments to be passed to the remote function. Finally, we use a static JUnit method to verify whether or not the Erlang server process is passing the correct message back to the ClientNode instance - the same way it did for the clientnode node. Here is the ClientNode class in its entirety.

import com.ericsson.otp.erlang.*;

public class ClientNode {

   public static void main (String[] _args) throws Exception{

OtpSelf cNode = new OtpSelf("clientnode", "cookie");
OtpPeer sNode = new OtpPeer("[e-mail]");
OtpConnection connection = cNode.connect(sNode);

OtpErlangObject[] args = new OtpErlangObject[]{
new OtpErlangLong(1), new OtpErlangLong(2)};
connection.sendRPC("mathserver", "add", args);
OtpErlangLong sum = (OtpErlangLong) connection.receiveRPC();
assertEquals(3, sum.intValue());

   }

}

Jinterface nodes can do more than just talk to Erlang nodes. They can communicate with each other as well. The servernode process can actually be reimplemented as a ServerNode class, allowing us to perform asynchronous messaging without a single line of Erlang. To do this the ServerNode must first tell the world it is "open for business".

OtpSelf sNode = new OtpSelf("servernode", "cookie");
sNode.publishPort();
OtpConnection connection = sNode.accept();

The ServerNode class begins by creating an OtpSelf instance – using the same node name and cookie as before. The node then publishes its port to the Erlang Port Mapper Daemon. This registers the node name and port, making it available to a remote client process. When the port is published it is important to immediately invoke the accept method. Forgetting to accept a connection after publishing the port would be the programmatic equivalent of false advertising. Once we've obtained a connection it is time to start processing messages.

OtpErlangTuple terms = (OtpErlangTuple) connection.receive();
OtpErlangLong first = (OtpErlangLong) terms.elementAt(0);
OtpErlangLong second = (OtpErlangLong) terms.elementAt(1);
long sum = first.longValue() + second.longValue();
connection.send(connection.peer().node(), new OtpErlangLong(sum));

The receive method of a Jinterface connection blocks until it receives a tuple from the ClientNode. Remember the receive statements back in Erlang mathserver module? Once a message is received the sum is calculated and sent back to the client. Here is the ServerNode in its entirety.

import com.ericsson.otp.erlang.*;

public class ServerNode {

   public static void main (String[] _args) throws Exception{

      OtpSelf sNode = new OtpSelf("servernode", "cookie");
      sNode.publishPort();
      OtpConnection connection = sNode.accept();

      while(true) try {

        OtpErlangTuple terms = (OtpErlangTuple) connection.receive();
        OtpErlangLong first = (OtpErlangLong) terms.elementAt(0);
        OtpErlangLong second = (OtpErlangLong) terms.elementAt(1);
        long sum = first.longValue() + second.longValue();
        connection.send(connection.peer().node(),new OtpErlangLong(sum));

      }catch(OtpErlangExit e) { break; }

      sNode.unPublishPort();
      connection.close();

   }

}

Here is the ClientNode modified. The sendRPC method is no longer used to send an OtpErlangObject array. Instead the send method is used to send an OtpErlangTuple.

import com.ericsson.otp.erlang.*;

public class ClientNode {

   public static void main (String[] _args) throws Exception{

OtpSelf cNode = new OtpSelf("clientnode", "cookie");
OtpPeer sNode = new OtpPeer("[e-mail]");
OtpConnection connection = cNode.connect(sNode);

OtpErlangObject[] args = new OtpErlangObject[]{
new OtpErlangLong(1), new OtpErlangLong(2)};
connection.send(sNode.node(), new OtpErlangTuple(args));
OtpErlangLong received = (OtpErlangLong) connection.receive();
assertEquals(3, received.intValue());

   }

}

Conclusion

Java and Erlang are not mutually exclusive, they complement each other. I personally have learned to embrace both because very few complex business problems can be modeled exclusively from an object oriented or functional paradigm. The solutions to these problems can be sequential or concurrent. Jinterface can cleanly divide (and conquer) a system into parts suitable for Java and parts suitable for Erlang.