Learn how to a build a cloud-first strategyRegister Now

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 217
  • Last Modified:

Developing or adapting a cross-platform tcp/ip protocol for transfering XML

I'm writing a program that needs to transfer accrued sales data from various slave nodes to a master node, over an ethernet LAN. The master application software is written in C++/MFC for Windows and the slaves application software is written in C++/Qtopia for Linux. I think that the most effective way of doing this is by accrueing a single day's sales data into a single XML file, transfering this XML file to the master, have the master parse this and store it in its database and then delete its local copy. The slave keeps its original XML file, should it be needed again.

My questions are:

1. Is my basic approach a sensible way of doing it? Convenience of implementation is important. I'm hoping to use someone else's nice Apache/BSD licenced API.

2. How should I transfer the XML file? Obviously, I favour something efficient, with a low-overhead. I don't like the look of SOAP. Using http as a "transport" protocol probably incurs an unnecessary overhead, and is aesthetically ugly. There has to be a better way. That said, we aren't talking about transfering very large files here, and they'll be transfered at full 100mb ethernet speed, so efficiency isn't an all-important consideration.
0
sternocera
Asked:
sternocera
  • 5
  • 5
1 Solution
 
itsmeandnobodyelseCommented:
Normally you would use XML to transfer between different applications. If you are responsible for both server and client you may consider to transfering binary structures which is much more efficient and fast.

I would use plain sockets for transfering which has the least overhead.

Regards, Alex
0
 
itsmeandnobodyelseCommented:
If transfering binary structures between Windows and Linux and back you have to consider alignment issues. Either put the struct members in order (double, float, int, short, char) so that each member begins and ends at a 32bit segment (maybe using fillers) or use preprocessor commands (platform dependent) to have the same alignment.

0
 
sternoceraAuthor Commented:
Alex,

Writing to a socket sounds complicated. I have no winsock/berkeley sockets experience. Surely there is a simpler solution, such as a cross-platform API?

If I was to use XML, what method of transfering the file would you recommend?

Thanks,
Sternocera
0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 
itsmeandnobodyelseCommented:
>>>> Writing to a socket sounds complicated.
Actually it isn't.

Assuming you have a socket connected you would write a struct passed like

   bool writeToSocket(MyStruct& st, SOCKET s)
   {
          if (!send(s, (char*)&st, sizeof(MyStruct))
               return false;
          return true;
   }

Reading is a little more difficult cause it would block if nothing is to read. You either set the socket to non-blocking and handle the error WSAEWOULDBLOCK (or EWOULDBLOCK at Linux).

   bool readFromSocket(MyStruct& st, SOCKET s, int timeout)
   {
         int noblock = timeout > 0? 1 : 0;
         ioctlsocket(s, FIONBIO, &noblock);  // set socket to blocking/no blocking
                                                                 // depending on timeout
         }
         int expired = 0;
         int bytesread = 0;
         while (expired <= timeout && bytesread <= sizeof(MyStruct))
         {
                 int rc = recv(s, ((char*)st) + bytesread, sizeof(MyStruct) - bytesread);
                 if (rc == SOCKET_ERROR)
                 {
                       if (GetLastError() == WSAEWOULDBLOCK)
                       {
                             int timeslice = timeout/10;
                             Sleep(timeslice);
                             expired += timeslice + 1;
                             continue;
                       }
                        // some other error
                        return false;
                 }
                 else if (rc > 0)
                 {
                        bytesread += rc;
                 }
         }
   }

In LINUX the part beginning with GetLastError is a little different:

                       if (errno == EWOULDBLOCK)
                       {
                             int timeslice = timeout/10;
                             sleep(timeslice);
                             expired += timeslice + 1;
                             continue;
                       }

Regards, Alex
0
 
sternoceraAuthor Commented:
Alex,

I'm looking at hessianCPP, but I'm not yet sure if that particular hessian implementation supports TCP transfers.

Writing to a raw socket is simple enough - I've done things like playing with netcat to get a webserver to process a HTTP GET and return html. However, creating a robust, functional protocol, with error handling, merely to transfer a fairly simple data structure seems like repetition of effort. I just know someone, somewhere has already done most of the work for me.

Another API that someone mentioned was JAXRPC, but thats Java only,

Thanks,
Sternocera
0
 
sternoceraAuthor Commented:
It turns out hessiancpp only supports the client side of the Caucho Hessian protocol. Therefore, back to the drawing board on that one.
0
 
itsmeandnobodyelseCommented:
>>>> However, creating a robust, functional protocol,
>>>> with error handling, merely to transfer a fairly simple
>>>> data structure seems like repetition of effort.
Hmm. You may know better but I made the experience that 'using' a thirdparty library goes with lack of documentation and is less simple than using plain sockets. Error handling can be quite simple cause beside of disconnecting issues errors cannot be handled other than to cancel the current operation and try again.

In case you are still interested, here the code to setup a server socket:


The server would create a socket, bind it to a port and listen for clients.

    // get socket
    SOCKET sock = socket(AF_INET, SOCK_STREAM, 0);

    // check if socket creation failed
    if (sock == 0)
    {
         // error
    }

    unsigned int ip = inet_addr("192.168.0.1");
    // Berkeley socket address structure
    struct  sockaddr_in host;                                          
    // server address            
    // fill awsome sockaddr_in structure
    memset (&host, 0, sizeof (host));                                
    host.sin_family = AF_INET;                                            
    host.sin_port = htons (41111);  // use any odd port between 2000 and 65535  
    host.sin_addr.s_addr = htonl (ip); // that is little to big endian

    // bind the socket
    if (bind(sock, (struct sockaddr*) &host, sizeof(host)) < 0)
    {      
        // error
    }
    if (listen(sock, 5) < 0)
    {
        // error
    }

After that the server runs a loop where it calls accept for incoming clients. To prevent from blocking it could be made similar than the read or call select with timeout like the following:

int selectRead(SOCKET sockHost, int msecWait)
{
    int     result = 0;
    // fd_set manages an array of sockets
    fd_set          readSockets;
    struct timeval  tval;

    // set host sockat as first and only element
    FD_ZERO(&readSockets);
    FD_SET(sockHost, &readSockets);

    return NET_SUCCESS;
    // init timeout
    tval.tv_sec  = msecWait / 1000;
    tval.tv_usec = (msecWait % 1000) * 1000;

    result = select(sockHost + 1, &readSockets, NULL, NULL, &tval);

    if (result == 0)
        return 1;
    else if (result == SOCKET_ERROR)
        return -1;

    return 0;
}

    // loop to accept clients

    // loop will not end until again flag is set to False or an error occurs  
    bool again = true;
    while (again)
    {
        // check socket for incoming data with timeout
        int result = selectRead(timeout);

        // check if job has been cancelled in the meantime
        if (result == -1)
              return;

        // timeout is the most probable return code
        if (result == 1)
            continue;

        // if we come here, a client tries to connect                  
        // we accept and notify the server
        struct sockaddr_in sockAddr = { 0 };
        int lenAddr = sizeof(sockAddr);

        SOCKET sockClient = accept(sock, (struct sockaddr*) &sockAdr, &lenAddr);

        if (sockClient != 0)
        {
              // here we normally start a thread that handles communication with the client
        }

The client's code is much simpler. They create a socket, fill the sockaddr_in structure with IP address and port (of the server to connect), and call connect.

   // connect to socket
   if ((result = connect(sock, (struct sockaddr*) &host, sizeof(host))) < 0)
   {

Normally you would do all in a thread so that the communication was done in background. You would use some kind of a queue to transfer buffers between the threads,

Regards, Alex
0
 
sternoceraAuthor Commented:
Alex,

Interesting stuff. I think that whatever I do, I'll create a dedicated daemon to transfer the sales data that is seperate from my main application. It listens for connections. When it gets one, the client can be told the most recent xml file (they're named incremented numbers, as if from a db sequence). The client can also request an arbitrary specified XML file.  As it is part of an embedded application, this is acceptable. This approach is more modular then creating a new thread.

Perhaps you're right - a socket is the way to go. Thank you,
Sternocera
0
 
itsmeandnobodyelseCommented:
>>>> This approach is more modular then creating a new thread.
You may consider to creating a thread for each client connection or your server would need to both handling new clients and the communication to connected clients, e. g. like that


     while (true)     // run an infinite loop
     {
            if (readSelect(hostsock, 10) == 0)  // little timeout
            {
                   Client* pc = new Client;
                   pc->sock = accept(hostsock, ...);
                   clients.push_back(pc);
            }
            // loop all connected clients  
            for (int i = 0; i < clients.size(); ++i)
            {
                   Client* pc = clients[i];
                   if (readSelect(pc->sock, 10) == 0)  // little timeout
                   {
                          ReadMessage msg = { 0 };
                          int rc = readFromSocket(msg, sizeof(msg), pc->sock);
                          // maybe send an answer
                          // maybe delete client from list
                   }
            }
     }

if doing the second part in a own thread, the code will be more straight -forward.
0
 
sternoceraAuthor Commented:
I meant that it would be more sensible to create this as a seperate daemon, because it operates quite independently of my main application. This way, I can treat it as a seperate entity, which is logical and convenient,
Thanks
0

Featured Post

Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

  • 5
  • 5
Tackle projects and never again get stuck behind a technical roadblock.
Join Now