• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 232
  • Last Modified:

Decoding data retrieval instructions

I am am attempting to write a program in Java that grabs specific data from a remote data server over the internet.  I was provided an "example program" using pseudocode that was supposed to help me write a working class/program to grab such data for my use.  However, I have found the "example program" to be quite confusing, and wonder if anyone reading this thread will be able to see something I haven't.  The instructions are as follows (with my comments in round brackets):

1.) connect to the remote data server (host: xxxxxxxxx, port 8800)

2.) write 16 bytes of subscription data in network format (??) to the remote data server.  Data is non-zero for the type of data that is being requested by the remote client.  (Earlier in the doucument, several subscription options are present.  Each option takes two bytes.  I'm assuming that to "Enable" an option, pass 2 non-zero bytes in the 16 byte subscription header.  However, I have no idea how to send such data in byte format using Java.)

3.) read configuration data from the remote data server and convert from network format (what do they mean by network format?).  The amount of data received varies based on the subscription (this makes sense, though).

I will spare you the rest of the instructions.  I am stuck on #1 as it is, so if I can figure out these first 3 steps, I will be in much better shape (if not, I will post the rest of the steps later).
0
rnicholus
Asked:
rnicholus
  • 16
  • 16
  • 12
2 Solutions
 
aozarovCommented:
http://www.javaalmanac.com/cgi-bin/search/find.pl?words=Socket (read Creating a Client Socket, Writing Text to a Socket, ...)
0
 
CEHJCommented:
You need to get the basics first:

http://java.sun.com/docs/books/tutorial/networking/
0
 
rnicholusAuthor Commented:
I see examples that will help me connect to the server and read response data from it.  However, I am still unsure as to writing data to the server.  For example, step 2 tells me to "write 16 bytes of subscription data in network format to the RDS.  Each option that makes up the subscription header is described as 2 bytes.  An enabled option is non-zero, a disabled option is zero.  Lets say the possible options are as follows:

NAME              TYPE                  SIZE (bytes)
opt 1               integer                      2
opt 2               integer                      2
opt 3               integer                      2
opt 4               integer                      2
opt 5               integer                      2
opt 6               integer                      2
opt 7               integer                      2
opt 8               integer                      2

...and let's say I want to enabled options 1, 2, and 3, but disable options 4-8.  How do I format the input?  What is meant by "network format"?
0
Cloud Class® Course: Ruby Fundamentals

This course will introduce you to Ruby, as well as teach you about classes, methods, variables, data structures, loops, enumerable methods, and finishing touches.

 
rnicholusAuthor Commented:
I think I meant to say "how do i format the output (to the RDS)" in my last post.
0
 
aozarovCommented:
you create a byte array with 16 bytes and then for each opt index i you set the array in index i to the value of the integer.
Then you write the byte array using the write(byte_array) method
0
 
aozarovCommented:
Be aware that Java integer is 32 bits and not 16 bits (It assumes that your contained value are always < 2 ^ 16)
0
 
CEHJCommented:
Not sure why it requires 2 bytes. An 'option mask' could be sent in one. To use your example

int[] options = { 1, 2, 3 };

int choice = 0;
for(int i = 0;i < options.length;i++) {
      choice |= 1 << (options[i] - 1);
}

...

out.write(choice);
0
 
rnicholusAuthor Commented:
If I create a byte array with 16 bytes, this array will have 16 index values, correct?  However, there are only 8 options.  Does this mean that option 1 corresponds to 0 and 1 in the byte array, and option 1 corresponds to 2 and 3 in the byte array, and so on?
0
 
aozarovCommented:
byte[] array = new byte[16];
int opts_index = 0;
for(int i = 0; i < array.length; i+=2)
array[i] = opts[opts_index++];
0
 
aozarovCommented:
You got it right.
0
 
rnicholusAuthor Commented:
So, does this method make sense?...


byte subscription[] = new byte[16];

//enable options 1-3
subscription[0] = (byte)1;
subscription[2] = (byte)1;
subscription[4] = (byte)1;
0
 
CEHJCommented:
Yes, although that's not efficient, although of course that's not your fault ;-)
0
 
rnicholusAuthor Commented:
Is there a more efficient way to do this with Java?
0
 
aozarovCommented:
Sorry I made mistake above.

byte[] array = new byte[16];
int opts_index = 0;
for(int i = 0; i < array.length; i+=2)
{
array[i] = (opts[opts_index] >> 8) & 0xff;
array[i + 1] = opts[opts_index++] & 0xff;
}
0
 
aozarovCommented:
>> So, does this method make sense?...
That will make sense only if your values (per option) are always smaller then 256.
Otherwise you will need to use my "fixed" version.
0
 
CEHJCommented:
>> Is there a more efficient way to do this with Java?

Yes - i posted it above, but it doesn't conform to your spec
0
 
rnicholusAuthor Commented:
How so?
0
 
CEHJCommented:
>> How so?

Because the spec is asking you to provide the request as 16 bytes. My code would provide it as one
0
 
rnicholusAuthor Commented:
Which Java class has an overloaded write method that takes an array of bytes as its parameter?  I looked at bufferedwriter and didn't find such a method.
0
 
CEHJCommented:
Any stream class can write byte[]
0
 
CEHJCommented:
That is the default for streams. You should not use Writers as you are writing 'binary' data
0
 
aozarovCommented:
You can write directly to the OutputStream given by calling socket.getOutputStream()
0
 
rnicholusAuthor Commented:
This is becoming very confusing for me.  If you could only see the document outlining access to data on this server.  It is maddening.  

To write the subscription data to the RDS, I have constructed the following code:
      
                public void run()
      {
            try
            {
                  byte subscription[] = new byte[16];
                  byte configuration[] = new byte[70];
                  subscription[0] = (byte)1;

                  InetAddress addr = InetAddress.getByName("xxxxxxxxx");
                  int port = 8800;

                  Socket socket = new Socket(addr, port);

                  ByteArrayOutputStream wr = new ByteArrayOutputStream();

                  wr.write(subscription, 0, 16);
                  wr.writeTo(socket.getOutputStream());
                  wr.flush();
            }
            catch (Exception ex) { ex.printStackTrace(); }
      }

Now, after I write the subscription data to the RDS, it is supposed to "transmit static configuration data to the remote client".  I am enabling option 1, as you can see in my code.  With this in mind, according to the instructions, the server should return, in bytes, the # of (we'll call them data sets) which takes up 2 bytes, and then the number of datasets * 68 bytes.  How can I easily read this reponse?  Is my code above correct for sending the subscription request?
0
 
CEHJCommented:
All you need is:

Socket socket = new Socket(addr, port);
OutputStream out = socket.getOutputStream();
out.write(subscription);
out.close();
0
 
CEHJCommented:
(For the sending part)

For the receiving part you need to read them *into* something or write them into something. What are you meant to do?
0
 
rnicholusAuthor Commented:
Ok, now I have the following code:

                  byte subscription[] = new byte[16];
                  subscription[0] = (byte)1;

                  InetAddress addr = InetAddress.getByName("xxxxxx");
                  int port = 8800;

                  Socket socket = new Socket(addr, port);
                  OutputStream out = socket.getOutputStream();
                  out.write(subscription);
                  out.close();
                  socket.getInputStream().read(links);


I'm assuming I need to keep the socket open in order to read the response.  As I stated before, the response seems to have the length in bytes as so:

# of datasets (2 bytes)
# of datasets * 68 bytes

This means that the total # of bytes returned is 2 + (# of datasets * 68).  The problem is, I don't know how many bytes are being returned until I read the first two bytes.  Each dataset (68 bytes) contains specific information regarding the dataset, but that isn't important yet.  What is the best way (or any way, for that matter) to retrieve this reponse based on the description I have just provided?
0
 
CEHJCommented:
You can do

DataInputStream in = new DataInputStream(socket.getInputStream());
int numDatasets = in.readInt();
byte[] data = new byte[numDatasets * 68];
in.read(data);

0
 
rnicholusAuthor Commented:
Actually, I don't think that will work.  This data source treats an integer as 16 bits, not 32.  So, your 2nd line will actually read 4 bytes, instead of the two required...
0
 
CEHJCommented:
Dead right! Well spotted. It should be

int numDatasets = in.readShort();
0
 
aozarovCommented:
To get the size (if it is two bytes):

DataInputStream in = new DataInputStream(socket.getInputStream());
int size = in.read() & 0xff;
size = size << 8 + (in.read() & 0xff);
....
0
 
aozarovCommented:
change: size = size << 8 + (in.read() & 0xff);
to size = (size << 8) + (in.read() & 0xff);
0
 
rnicholusAuthor Commented:
this returns the size, in bytes, of the response?

Now, once the int numDatasets line attempts to execute, it takes several minutes for my program to read the response.  Is this most likely due to, say, a bandwidth issue on the remote server (since my connection is fine) or could this be something else?

Here is my code thusfar:

                  byte subscription[] = new byte[16];
                  byte links[] = new byte[2];
                  subscription[0] = (byte)1;

                  InetAddress addr = InetAddress.getByName("xxxxxx");
                  int port = 8800;

                  Socket socket = new Socket(addr, port);
                  OutputStream out = socket.getOutputStream();
                  out.write(subscription);
                  //out.close();

                  DataInputStream in = new DataInputStream(socket.getInputStream());
                  int numDatasets = in.readShort();
                  byte[] data = new byte[numDatasets * 68];
                  in.read(data);

                  socket.close();
0
 
CEHJCommented:
>> Is this most likely due to, say, a bandwidth issue on the remote server

Could be. Nothing wrong with the code, and you say your connection is OK with other servers...
0
 
aozarovCommented:
change -> in.read(data) to in.readFully(data)
0
 
rnicholusAuthor Commented:
It is their server.  It's popping up right away now.  

aozarov:
Why did you reccomend that change?  What is the difference?  (Pardon my ignorance).
0
 
aozarovCommented:
read(data) does not guarenty to block until all data (based on the size of the array) is availale.
You might not get all the data after this call and this is why the method returns int (to indicate how many bytes were actually read).
where as readFully(array) will block until that array is full.
0
 
rnicholusAuthor Commented:
Ok, another question.  Suppose I have an array of bytes and I want to take the values from the index range 30-33 (4 bytes) and convert that range to its integer value.  How would I do this?  The same goes for, say, an index range of 50-51 (2 bytes).
0
 
CEHJCommented:
DataInputStream in = new DataInputStream(new ByteArrayInputStream(yourArray));
in.skipBytes(30);
int i = in.readInt(); // or readX
0
 
rnicholusAuthor Commented:
Thanks you all very much for your help on this issue.  I know have a much better understanding of this type of data retrieval.  
0
 
CEHJCommented:
No problem
0
 
CEHJCommented:
:-)
0
 
rnicholusAuthor Commented:
I mean, i NOW have a much better understanding.  Here is how and why I split up the points:

- CEHJ provided the initial solution I used to write the subscription data

- azarov provided a note about the readFully method, which proved to be very important, as I found that I was working with a partially-filled byte array without it (when I attempted to print the contents of the array without the readFully in place).
0
 
rnicholusAuthor Commented:
Sorry, I forgot to add that I split the points in half for the above reasons.  I really need to read over my messages before I send them.
0
 
CEHJCommented:
That's OK
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

  • 16
  • 16
  • 12
Tackle projects and never again get stuck behind a technical roadblock.
Join Now