JSON Message Passing for Distributed Computing with NodeJS

NodeJS is a runtime environment built using Chrome's javascript runtime that can be used to create scalable server-side network applications. NodeJS has become a popular tool for serving web applications using the built in http server / client module that makes it easy to route, process and respond to http requests.

There is a similar module called 'net' built into NodeJS that provides an asynchronous network wrapper for communication through the TCP/IP protocol that simplifies reading and writing network streams. In combination with a few supporting modules that facilitate passing JSON objects and creating queues, NodeJS can become a powerful tool for distributed computation. We will walk through the process of creating simple server and client application complete with JSON message passing that can be used to distribute computation across an arbitrary number of nodes.

Client.js

Begin by creating a script file that will become our client application called client.js. We will need to import a few modules that will help us along the way. One of these does not come packaged with NodeJS but can be easily installed using the Node package manager (npm) from the console:
 

npm install json-socket

Select all Open in new window

Open up client.js and import some of the packages that will help us along the way:
 

var net = require('net'),
                          JsonSocket = require('json-socket');

Select all Open in new window

The first line imports NodeJS' net module that wraps all of the network functionality into a convenient, easy to use package. json-socket allows JSON to be easily passed around using TCP. Without this module we would have to 'chunk' our message manually which is a headache you will be happy to avoid.

Create an array of server names that you will be running your servers on. In this example we will use localhost so the server and client can be tested on the same machine, but you will want to substitute network computer names or IP addresses when you deploy to multiple servers.
 

var servers = ['localhost'];

Select all Open in new window

Next, we'll create a function that takes a socket (instance created with net.Socket()), decorates it using the json-socket so it easily handles JSON data, and tells it all the events we want it to listen for. Whenever the server responds, we'll handle the results with a closure:
 

var initSubscriptions = function(socket) {
                         socket.on('connect', function() {
                            socket.sendMessage({name: 'James'});
                      
                            //handles a message returned from the server
                            socket.on('message', function(message) {
                               console.log('Server responds: ' + JSON.stringify(message));
                            });
                            
                            //'close' is emitted whenever a server closes
                            socket.on('close', function() {
                               console.log('Server Disconnected!');
                            });
                         });
                      };

Select all Open in new window

Now we simply loop through all of our server names, open a socket for them and finally create subscriptions using our initSubscriptions function
 

var nodes = [];
                      
                      servers.forEach(function(nodeName) {
                         var socket = new JsonSocket(new net.Socket());
                         socket.connect(42424, nodeName);
                         initSubscriptions(socket);
                         nodes.push(socket);
                      });

Select all Open in new window

We'll have the client and server communicate through port 42424 in this example, though you can easily modify this so that it listens on a free port of your choosing. The nodes array will hold the actual socket instances that represent the TCP layer through which we communicate with our servers.

Here we simply take out the first socket and send a message through it, but notice that you could, for example, compute indexes into this array of nodes to distribute messages among nodes.

Server.js

The server simply listens for messages, pulls out the name from the message and sends back "Hello ". So lets get started.
 

var net = require('net'),
                          JsonSocket = require('json-socket');
                      
                      var server = net.createServer();
                      server.listen(42424);
                      
                      server.on('connection', function(socket) {
                         console.log('client connected');
                         socket = new JsonSocket(socket);
                         socket.on('message', function(message) {
                            socket.sendMessage('Hello ' + message.name);
                         });
                      });

Select all Open in new window

Much of this should look familiar. We import the net and json-socket classes and use net.createServer() to create a net TCP server. We then make that server listen on the port we defined in our client class. When a server detects a connection we provide a callback function that takes the socket as a parameter. We then decorate that socket using json-socket and finally create a listener that waits for a message, takes out the name and returns it with 'Hello'.

Run the server with nodejs server.js and the client with nodejs client.js and watch the message passing begin!

For more information on json-socket check here: https://www.npmjs.org/package/json-socket