?
Solved

Decoding data retrieval instructions

Posted on 2005-04-18
44
Medium Priority
?
215 Views
Last Modified: 2010-03-31
I am am attempting to write a program in Java that grabs specific data from a remote data server over the internet.  I was provided an "example program" using pseudocode that was supposed to help me write a working class/program to grab such data for my use.  However, I have found the "example program" to be quite confusing, and wonder if anyone reading this thread will be able to see something I haven't.  The instructions are as follows (with my comments in round brackets):

1.) connect to the remote data server (host: xxxxxxxxx, port 8800)

2.) write 16 bytes of subscription data in network format (??) to the remote data server.  Data is non-zero for the type of data that is being requested by the remote client.  (Earlier in the doucument, several subscription options are present.  Each option takes two bytes.  I'm assuming that to "Enable" an option, pass 2 non-zero bytes in the 16 byte subscription header.  However, I have no idea how to send such data in byte format using Java.)

3.) read configuration data from the remote data server and convert from network format (what do they mean by network format?).  The amount of data received varies based on the subscription (this makes sense, though).

I will spare you the rest of the instructions.  I am stuck on #1 as it is, so if I can figure out these first 3 steps, I will be in much better shape (if not, I will post the rest of the steps later).
0
Comment
Question by:rnicholus
  • 16
  • 16
  • 12
44 Comments
 
LVL 15

Expert Comment

by:aozarov
ID: 13807341
http://www.javaalmanac.com/cgi-bin/search/find.pl?words=Socket (read Creating a Client Socket, Writing Text to a Socket, ...)
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 13807351
You need to get the basics first:

http://java.sun.com/docs/books/tutorial/networking/
0
 

Author Comment

by:rnicholus
ID: 13807474
I see examples that will help me connect to the server and read response data from it.  However, I am still unsure as to writing data to the server.  For example, step 2 tells me to "write 16 bytes of subscription data in network format to the RDS.  Each option that makes up the subscription header is described as 2 bytes.  An enabled option is non-zero, a disabled option is zero.  Lets say the possible options are as follows:

NAME              TYPE                  SIZE (bytes)
opt 1               integer                      2
opt 2               integer                      2
opt 3               integer                      2
opt 4               integer                      2
opt 5               integer                      2
opt 6               integer                      2
opt 7               integer                      2
opt 8               integer                      2

...and let's say I want to enabled options 1, 2, and 3, but disable options 4-8.  How do I format the input?  What is meant by "network format"?
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:rnicholus
ID: 13807516
I think I meant to say "how do i format the output (to the RDS)" in my last post.
0
 
LVL 15

Expert Comment

by:aozarov
ID: 13807593
you create a byte array with 16 bytes and then for each opt index i you set the array in index i to the value of the integer.
Then you write the byte array using the write(byte_array) method
0
 
LVL 15

Expert Comment

by:aozarov
ID: 13807604
Be aware that Java integer is 32 bits and not 16 bits (It assumes that your contained value are always < 2 ^ 16)
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 13807607
Not sure why it requires 2 bytes. An 'option mask' could be sent in one. To use your example

int[] options = { 1, 2, 3 };

int choice = 0;
for(int i = 0;i < options.length;i++) {
      choice |= 1 << (options[i] - 1);
}

...

out.write(choice);
0
 

Author Comment

by:rnicholus
ID: 13807619
If I create a byte array with 16 bytes, this array will have 16 index values, correct?  However, there are only 8 options.  Does this mean that option 1 corresponds to 0 and 1 in the byte array, and option 1 corresponds to 2 and 3 in the byte array, and so on?
0
 
LVL 15

Expert Comment

by:aozarov
ID: 13807676
byte[] array = new byte[16];
int opts_index = 0;
for(int i = 0; i < array.length; i+=2)
array[i] = opts[opts_index++];
0
 
LVL 15

Expert Comment

by:aozarov
ID: 13807685
You got it right.
0
 

Author Comment

by:rnicholus
ID: 13807738
So, does this method make sense?...


byte subscription[] = new byte[16];

//enable options 1-3
subscription[0] = (byte)1;
subscription[2] = (byte)1;
subscription[4] = (byte)1;
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 13807766
Yes, although that's not efficient, although of course that's not your fault ;-)
0
 

Author Comment

by:rnicholus
ID: 13807773
Is there a more efficient way to do this with Java?
0
 
LVL 15

Expert Comment

by:aozarov
ID: 13807776
Sorry I made mistake above.

byte[] array = new byte[16];
int opts_index = 0;
for(int i = 0; i < array.length; i+=2)
{
array[i] = (opts[opts_index] >> 8) & 0xff;
array[i + 1] = opts[opts_index++] & 0xff;
}
0
 
LVL 15

Expert Comment

by:aozarov
ID: 13807788
>> So, does this method make sense?...
That will make sense only if your values (per option) are always smaller then 256.
Otherwise you will need to use my "fixed" version.
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 13807797
>> Is there a more efficient way to do this with Java?

Yes - i posted it above, but it doesn't conform to your spec
0
 

Author Comment

by:rnicholus
ID: 13807847
How so?
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 13807877
>> How so?

Because the spec is asking you to provide the request as 16 bytes. My code would provide it as one
0
 

Author Comment

by:rnicholus
ID: 13807888
Which Java class has an overloaded write method that takes an array of bytes as its parameter?  I looked at bufferedwriter and didn't find such a method.
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 13808054
Any stream class can write byte[]
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 13808063
That is the default for streams. You should not use Writers as you are writing 'binary' data
0
 
LVL 15

Expert Comment

by:aozarov
ID: 13808196
You can write directly to the OutputStream given by calling socket.getOutputStream()
0
 

Author Comment

by:rnicholus
ID: 13808430
This is becoming very confusing for me.  If you could only see the document outlining access to data on this server.  It is maddening.  

To write the subscription data to the RDS, I have constructed the following code:
      
                public void run()
      {
            try
            {
                  byte subscription[] = new byte[16];
                  byte configuration[] = new byte[70];
                  subscription[0] = (byte)1;

                  InetAddress addr = InetAddress.getByName("xxxxxxxxx");
                  int port = 8800;

                  Socket socket = new Socket(addr, port);

                  ByteArrayOutputStream wr = new ByteArrayOutputStream();

                  wr.write(subscription, 0, 16);
                  wr.writeTo(socket.getOutputStream());
                  wr.flush();
            }
            catch (Exception ex) { ex.printStackTrace(); }
      }

Now, after I write the subscription data to the RDS, it is supposed to "transmit static configuration data to the remote client".  I am enabling option 1, as you can see in my code.  With this in mind, according to the instructions, the server should return, in bytes, the # of (we'll call them data sets) which takes up 2 bytes, and then the number of datasets * 68 bytes.  How can I easily read this reponse?  Is my code above correct for sending the subscription request?
0
 
LVL 86

Accepted Solution

by:
CEHJ earned 1000 total points
ID: 13808499
All you need is:

Socket socket = new Socket(addr, port);
OutputStream out = socket.getOutputStream();
out.write(subscription);
out.close();
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 13808524
(For the sending part)

For the receiving part you need to read them *into* something or write them into something. What are you meant to do?
0
 

Author Comment

by:rnicholus
ID: 13808592
Ok, now I have the following code:

                  byte subscription[] = new byte[16];
                  subscription[0] = (byte)1;

                  InetAddress addr = InetAddress.getByName("xxxxxx");
                  int port = 8800;

                  Socket socket = new Socket(addr, port);
                  OutputStream out = socket.getOutputStream();
                  out.write(subscription);
                  out.close();
                  socket.getInputStream().read(links);


I'm assuming I need to keep the socket open in order to read the response.  As I stated before, the response seems to have the length in bytes as so:

# of datasets (2 bytes)
# of datasets * 68 bytes

This means that the total # of bytes returned is 2 + (# of datasets * 68).  The problem is, I don't know how many bytes are being returned until I read the first two bytes.  Each dataset (68 bytes) contains specific information regarding the dataset, but that isn't important yet.  What is the best way (or any way, for that matter) to retrieve this reponse based on the description I have just provided?
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 13808659
You can do

DataInputStream in = new DataInputStream(socket.getInputStream());
int numDatasets = in.readInt();
byte[] data = new byte[numDatasets * 68];
in.read(data);

0
 

Author Comment

by:rnicholus
ID: 13808738
Actually, I don't think that will work.  This data source treats an integer as 16 bits, not 32.  So, your 2nd line will actually read 4 bytes, instead of the two required...
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 13808868
Dead right! Well spotted. It should be

int numDatasets = in.readShort();
0
 
LVL 15

Expert Comment

by:aozarov
ID: 13808894
To get the size (if it is two bytes):

DataInputStream in = new DataInputStream(socket.getInputStream());
int size = in.read() & 0xff;
size = size << 8 + (in.read() & 0xff);
....
0
 
LVL 15

Expert Comment

by:aozarov
ID: 13808898
change: size = size << 8 + (in.read() & 0xff);
to size = (size << 8) + (in.read() & 0xff);
0
 

Author Comment

by:rnicholus
ID: 13809042
this returns the size, in bytes, of the response?

Now, once the int numDatasets line attempts to execute, it takes several minutes for my program to read the response.  Is this most likely due to, say, a bandwidth issue on the remote server (since my connection is fine) or could this be something else?

Here is my code thusfar:

                  byte subscription[] = new byte[16];
                  byte links[] = new byte[2];
                  subscription[0] = (byte)1;

                  InetAddress addr = InetAddress.getByName("xxxxxx");
                  int port = 8800;

                  Socket socket = new Socket(addr, port);
                  OutputStream out = socket.getOutputStream();
                  out.write(subscription);
                  //out.close();

                  DataInputStream in = new DataInputStream(socket.getInputStream());
                  int numDatasets = in.readShort();
                  byte[] data = new byte[numDatasets * 68];
                  in.read(data);

                  socket.close();
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 13809086
>> Is this most likely due to, say, a bandwidth issue on the remote server

Could be. Nothing wrong with the code, and you say your connection is OK with other servers...
0
 
LVL 15

Expert Comment

by:aozarov
ID: 13809094
change -> in.read(data) to in.readFully(data)
0
 

Author Comment

by:rnicholus
ID: 13809118
It is their server.  It's popping up right away now.  

aozarov:
Why did you reccomend that change?  What is the difference?  (Pardon my ignorance).
0
 
LVL 15

Assisted Solution

by:aozarov
aozarov earned 1000 total points
ID: 13809181
read(data) does not guarenty to block until all data (based on the size of the array) is availale.
You might not get all the data after this call and this is why the method returns int (to indicate how many bytes were actually read).
where as readFully(array) will block until that array is full.
0
 

Author Comment

by:rnicholus
ID: 13809866
Ok, another question.  Suppose I have an array of bytes and I want to take the values from the index range 30-33 (4 bytes) and convert that range to its integer value.  How would I do this?  The same goes for, say, an index range of 50-51 (2 bytes).
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 13809946
DataInputStream in = new DataInputStream(new ByteArrayInputStream(yourArray));
in.skipBytes(30);
int i = in.readInt(); // or readX
0
 

Author Comment

by:rnicholus
ID: 13810716
Thanks you all very much for your help on this issue.  I know have a much better understanding of this type of data retrieval.  
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 13810724
No problem
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 13810739
:-)
0
 

Author Comment

by:rnicholus
ID: 13810745
I mean, i NOW have a much better understanding.  Here is how and why I split up the points:

- CEHJ provided the initial solution I used to write the subscription data

- azarov provided a note about the readFully method, which proved to be very important, as I found that I was working with a partially-filled byte array without it (when I attempted to print the contents of the array without the readFully in place).
0
 

Author Comment

by:rnicholus
ID: 13810749
Sorry, I forgot to add that I split the points in half for the above reasons.  I really need to read over my messages before I send them.
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 13810758
That's OK
0

Featured Post

Keep up with what's happening at Experts Exchange!

Sign up to receive Decoded, a new monthly digest with product updates, feature release info, continuing education opportunities, and more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction Java can be integrated with native programs using an interface called JNI(Java Native Interface). Native programs are programs which can directly run on the processor. JNI is simply a naming and calling convention so that the JVM (Java…
Java Flight Recorder and Java Mission Control together create a complete tool chain to continuously collect low level and detailed runtime information enabling after-the-fact incident analysis. Java Flight Recorder is a profiling and event collectio…
The viewer will learn how to implement Singleton Design Pattern in Java.
This video teaches viewers about errors in exception handling.
Suggested Courses
Course of the Month16 days, 11 hours left to enroll

864 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question