Solved

block I/O and reliability

Posted on 2003-11-08
18
260 Views
Last Modified: 2010-05-18
Hi, I use nio (buffers and channels) to transfer data over the network and was wondering if it is a totally reliable transfer (i.e. can data that sent using a channel get lost on the way?).   In my chat program, which seem to work alright, sometimes when I send a long string message such as "I want to eat an ice cream" the server sends back a string "I want to eat an ice cream \n n ice cream".   The last substring of the original message is sent along and after the original.   This does not happen all the time though, for example if I send "Hello", it returns "Hello" and that hasn't failed yet.   So I just wanted to know if this is possibly due to the nature of nio stuff and that I need to come up with a way to improve this communication protocol or because I encoded and decoded using "US-ASCII" and need to use other character set or if this is some logic error I made in my program.   Any suggestions?
0
Comment
Question by:skyblue01
  • 9
  • 8
18 Comments
 
LVL 15

Expert Comment

by:JakobA
Comment Utility
sounds like you are not flushing the buffer after telling it to send. ie: purely a programming error, have another look at the interface to the bufferedwriter class or whatever you are using.
0
 
LVL 92

Expert Comment

by:objects
Comment Utility
Post the code you are using to read and write the buffer.
I think I posted some code in one of your previous questions to avoid that problem.
0
 

Author Comment

by:skyblue01
Comment Utility
Sorry for the late response.   I still can't figure out what's the problem.   I'll post what I'm doing when dealing with buffers and channel.   Let me know what I'm doing wrong here.   Thanks.  
----------------------------------------------------------------
    // The processMessage method gets called when the user types something in the GUI
    // Client sends the message to Server here
    private void processMessage(String message) {
        try {
            buffer.clear();

            buffer = encoder.encode(CharBuffer.wrap(message));

            int numWritten = sc.write(buffer); // sends it to the server
            System.out.println("numWritten: " + numWritten); // TEST

            tf.setText(""); // clears out text input field
        } catch(IOException ie) { System.out.println(ie); }
    }

    // upon receiving a message, for each incoming connection, Server starts ServerThread
    // run method runs in a separate thread and read the message from the channel
    public void run() {
        try {
            while (true) {
                buffer.clear();
                int r = socketChannel.read(buffer);

                if (r == -1)
                    break;

                buffer.flip();

                CharBuffer charBuffer = decoder.decode(buffer);
                String message = charBuffer.toString();
                charBuffer.clear(); // TEST

                System.out.println("Sending " + message); // tells the world

                server.sendToAll(message); // and have the server send it to all clients
            }
        } catch(EOFException ie) {
        } catch(IOException ie) { // this doesn't need an error message

        ie.printStackTrace(); // this does; tells the world
        } finally {
            server.removeConnection( socketChannel ); // the connection is closed for one reason or another, so have the server deal with it
        }
    }

    // Server then sends the received message to all the Client participating in the ongoing chat session
    // sendToAll method sends a message to all clients
    void sendToAll(String message) {

        synchronized(outputChannels) {

            for (ListIterator l = getOutputChannels(); l.hasNext();) { // for each client

                SocketChannel scBack = (SocketChannel) l.next(); // gets the output channel -- SocketChannel sc?

                try {
                    buffer.clear();
                    buffer = encoder.encode(CharBuffer.wrap(message));

                    scBack.write(buffer); // and sends the message
                } catch(IOException ie) { System.out.println(ie); }
            }
        }
    }

    // finally, each Client starts a background thread to receive all messages sent from Server
    // The run method is ran by background thread and shows messages from other window
    public void run() {
        try {
            System.out.println("Testing... Client.run()"); // TEST

            while (true) {
                buffer.clear();

                int r = sc.read(buffer); // there's nothing to read from!!

                if (r == -1)
                    break;

                buffer.flip();

                CharBuffer charBuffer = decoder.decode(buffer);
                String message = charBuffer.toString();
                charBuffer.clear(); // TEST
                System.out.println("received message: " + message); // TEST

                ta.append(message + "\n"); // prints it to our text window
            }
        } catch(IOException ie) { System.out.println(ie); }
    }
---------------------------------------------------------------
0
 
LVL 92

Expert Comment

by:objects
Comment Utility
Whats buffer defined as?
0
 
LVL 92

Accepted Solution

by:
objects earned 250 total points
Comment Utility
Won't you get a problem if the client sends a message that is larger than the buffer size on the server.
Similarily for the client recieving the message.

You also assume that the call to read() will read the entire message.
0
 

Author Comment

by:skyblue01
Comment Utility
I have:

ByteBuffer buffer = ByteBuffer.allocate(1024);
0
 

Author Comment

by:skyblue01
Comment Utility
Buffer size doesn't seem to matter... no matter how much I increase the buffer size, Client still receives the original plus the cut-off message.  

I'll check out if the read() is not reading the entire message...
0
 
LVL 92

Expert Comment

by:objects
Comment Utility
>   buffer.clear();
>   buffer = encoder.encode(CharBuffer.wrap(message));

On a side note, there is no point clear the buffer in the above case as you create a new buffer to send.
0
 

Author Comment

by:skyblue01
Comment Utility
As in the java api, read() should read up to the end of what's in the buffer:

read()

An attempt is made to read up to r bytes from the channel, where r is the number of bytes remaining in the buffer, that is, dst.remaining(), at the moment this method is invoked.

so it's got to be something else...
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 
LVL 92

Expert Comment

by:objects
Comment Utility
"An attempt is made to read up to r bytes ..", this doesn't guarantee that the buffer will be filled.

Also from the javadoc:
"A read operation might not fill the buffer, and in fact it might not read any bytes at all...."

And your client and server buffer sizes are different so how do you know when one message ends and another begins.

Try printing out the number of bytes being written at one end, and the number being read by each read at the other to see whats going on.
0
 
LVL 92

Expert Comment

by:objects
Comment Utility
> Buffer size doesn't seem to matter

Maybe not for this problem, but I think it will cause a different problem.
0
 

Author Comment

by:skyblue01
Comment Utility
Yeah, I have two Client windows open and the one not sending message receives the right message.   The one that sends receives twice, one original, and the second one a cut-off so it's maybe a problem with something else.   I'll go back again and check.  
0
 

Author Comment

by:skyblue01
Comment Utility
objects, if you're still around, I thought if you could take a look at my code.   I've tested and everything works fine until the last line, scBack.write(buffer);.   Then there's something else that this line is dependent on but I don't know why.   What I figured out from my tests are:

suppose two clients, c1 and c2 are connected to the server.   each channel is stored in a linked list called outputChannels.   for each incoming connection, the server starts a thread, and whenever a message is received from that channel, it invokes sendToAll(str) method.   it is this method that's causing the problem.   suppose "hello" is sent from c2.   then c1 and c2 successfully receives "hello".   then suppose c2 sends "dddddddddddd kkkkkkkkkkk".   then c1 receives "dddddddddddd kkkkkkkkkkk", but c2 receives two messages, "dddddddddddd kkkkkkkkkkk" and "dd kkkkkkkkkkk".   I can't figure out why this is happening because my linked list hold only two elements, namely socketchannel1 connected from c1 to server and socketchannel2 connected from c2 to server.   in the sendToAll(String message), message passed to this method is printed just as it is, and after encoding this to be written into the channel, I tested to see if this buffer is what i want it to be written to the channel so i decrypted, and it prints exactly what i want.   so i don't know why c2 received two messages because the background thread that runs in Clinet class to receive any incoming messages is only started once.   also, what i figured was that if i type in at c2 the messages "hello" and "dddddddddddd kkkkkkkkkkk" in that order, what i said above happens but if i type in "dddddddddddd kkkkkkkkkkk" first, then it is successfully sent to c1 and c2.   please let me know if you have any suggestion, or if you have time, i'd be greatly appreciated if you can take a look at what i have.   in that case, i'll increase the point value as well since that might take more time...
0
 
LVL 92

Expert Comment

by:objects
Comment Utility
> but c2 receives two messages, "dddddddddddd kkkkkkkkkkk" and "dd kkkkkkkkkkk".

See my above comments.
There is no way for you client to know the size of the message.
It simply reads whatever bytes are avaible into the buffer and assumes that is the next message.
0
 

Author Comment

by:skyblue01
Comment Utility
SocketChannel does not seem to provide a method for such issue (i.e. a way to know if all bytes are read into a buffer).   is there any way (or method) that can be used to counter this problem?
0
 
LVL 92

Expert Comment

by:objects
Comment Utility
No it's just reading a stream of bytes, same as standard I/O. You needs to handle these kinds of issues yourself. By providing some means to first determine message seperation (eg. fixed size messages, or include message length), and then to ensure you have read the entire message.
0
 

Author Comment

by:skyblue01
Comment Utility
k.   i'll try that.
0
 
LVL 92

Expert Comment

by:objects
Comment Utility
0

Featured Post

6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
Java 1603 Error 2 42
Fibonacci challenge 11 82
Tools or ways to handle development of complex web applications 4 73
Java and GPO 11 46
For customizing the look of your lightweight component and making it look lucid like it was made of glass. Or: how to make your component more Apple-ish ;) This tip assumes your component to be of rectangular shape and completely opaque. (COD…
INTRODUCTION Working with files is a moderately common task in Java.  For most projects hard coding the file names, using parameters in configuration files, or using command-line arguments is sufficient.   However, when your application has vi…
Viewers learn about the “for” loop and how it works in Java. By comparing it to the while loop learned before, viewers can make the transition easily. You will learn about the formatting of the for loop as we write a program that prints even numbers…
Viewers will learn about basic arrays, how to declare them, and how to use them. Introduction and definition: Declare an array and cover the syntax of declaring them: Initialize every index in the created array: Example/Features of a basic arr…

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now