Link to home
Start Free TrialLog in
Avatar of skyblue01
skyblue01

asked on

block I/O and reliability

Hi, I use nio (buffers and channels) to transfer data over the network and was wondering if it is a totally reliable transfer (i.e. can data that sent using a channel get lost on the way?).   In my chat program, which seem to work alright, sometimes when I send a long string message such as "I want to eat an ice cream" the server sends back a string "I want to eat an ice cream \n n ice cream".   The last substring of the original message is sent along and after the original.   This does not happen all the time though, for example if I send "Hello", it returns "Hello" and that hasn't failed yet.   So I just wanted to know if this is possibly due to the nature of nio stuff and that I need to come up with a way to improve this communication protocol or because I encoded and decoded using "US-ASCII" and need to use other character set or if this is some logic error I made in my program.   Any suggestions?
Avatar of JakobA
JakobA

sounds like you are not flushing the buffer after telling it to send. ie: purely a programming error, have another look at the interface to the bufferedwriter class or whatever you are using.
Avatar of Mick Barry
Post the code you are using to read and write the buffer.
I think I posted some code in one of your previous questions to avoid that problem.
Avatar of skyblue01

ASKER

Sorry for the late response.   I still can't figure out what's the problem.   I'll post what I'm doing when dealing with buffers and channel.   Let me know what I'm doing wrong here.   Thanks.  
----------------------------------------------------------------
    // The processMessage method gets called when the user types something in the GUI
    // Client sends the message to Server here
    private void processMessage(String message) {
        try {
            buffer.clear();

            buffer = encoder.encode(CharBuffer.wrap(message));

            int numWritten = sc.write(buffer); // sends it to the server
            System.out.println("numWritten: " + numWritten); // TEST

            tf.setText(""); // clears out text input field
        } catch(IOException ie) { System.out.println(ie); }
    }

    // upon receiving a message, for each incoming connection, Server starts ServerThread
    // run method runs in a separate thread and read the message from the channel
    public void run() {
        try {
            while (true) {
                buffer.clear();
                int r = socketChannel.read(buffer);

                if (r == -1)
                    break;

                buffer.flip();

                CharBuffer charBuffer = decoder.decode(buffer);
                String message = charBuffer.toString();
                charBuffer.clear(); // TEST

                System.out.println("Sending " + message); // tells the world

                server.sendToAll(message); // and have the server send it to all clients
            }
        } catch(EOFException ie) {
        } catch(IOException ie) { // this doesn't need an error message

        ie.printStackTrace(); // this does; tells the world
        } finally {
            server.removeConnection( socketChannel ); // the connection is closed for one reason or another, so have the server deal with it
        }
    }

    // Server then sends the received message to all the Client participating in the ongoing chat session
    // sendToAll method sends a message to all clients
    void sendToAll(String message) {

        synchronized(outputChannels) {

            for (ListIterator l = getOutputChannels(); l.hasNext();) { // for each client

                SocketChannel scBack = (SocketChannel) l.next(); // gets the output channel -- SocketChannel sc?

                try {
                    buffer.clear();
                    buffer = encoder.encode(CharBuffer.wrap(message));

                    scBack.write(buffer); // and sends the message
                } catch(IOException ie) { System.out.println(ie); }
            }
        }
    }

    // finally, each Client starts a background thread to receive all messages sent from Server
    // The run method is ran by background thread and shows messages from other window
    public void run() {
        try {
            System.out.println("Testing... Client.run()"); // TEST

            while (true) {
                buffer.clear();

                int r = sc.read(buffer); // there's nothing to read from!!

                if (r == -1)
                    break;

                buffer.flip();

                CharBuffer charBuffer = decoder.decode(buffer);
                String message = charBuffer.toString();
                charBuffer.clear(); // TEST
                System.out.println("received message: " + message); // TEST

                ta.append(message + "\n"); // prints it to our text window
            }
        } catch(IOException ie) { System.out.println(ie); }
    }
---------------------------------------------------------------
Whats buffer defined as?
ASKER CERTIFIED SOLUTION
Avatar of Mick Barry
Mick Barry
Flag of Australia image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
I have:

ByteBuffer buffer = ByteBuffer.allocate(1024);
Buffer size doesn't seem to matter... no matter how much I increase the buffer size, Client still receives the original plus the cut-off message.  

I'll check out if the read() is not reading the entire message...
>   buffer.clear();
>   buffer = encoder.encode(CharBuffer.wrap(message));

On a side note, there is no point clear the buffer in the above case as you create a new buffer to send.
As in the java api, read() should read up to the end of what's in the buffer:

read()

An attempt is made to read up to r bytes from the channel, where r is the number of bytes remaining in the buffer, that is, dst.remaining(), at the moment this method is invoked.

so it's got to be something else...
"An attempt is made to read up to r bytes ..", this doesn't guarantee that the buffer will be filled.

Also from the javadoc:
"A read operation might not fill the buffer, and in fact it might not read any bytes at all...."

And your client and server buffer sizes are different so how do you know when one message ends and another begins.

Try printing out the number of bytes being written at one end, and the number being read by each read at the other to see whats going on.
> Buffer size doesn't seem to matter

Maybe not for this problem, but I think it will cause a different problem.
Yeah, I have two Client windows open and the one not sending message receives the right message.   The one that sends receives twice, one original, and the second one a cut-off so it's maybe a problem with something else.   I'll go back again and check.  
objects, if you're still around, I thought if you could take a look at my code.   I've tested and everything works fine until the last line, scBack.write(buffer);.   Then there's something else that this line is dependent on but I don't know why.   What I figured out from my tests are:

suppose two clients, c1 and c2 are connected to the server.   each channel is stored in a linked list called outputChannels.   for each incoming connection, the server starts a thread, and whenever a message is received from that channel, it invokes sendToAll(str) method.   it is this method that's causing the problem.   suppose "hello" is sent from c2.   then c1 and c2 successfully receives "hello".   then suppose c2 sends "dddddddddddd kkkkkkkkkkk".   then c1 receives "dddddddddddd kkkkkkkkkkk", but c2 receives two messages, "dddddddddddd kkkkkkkkkkk" and "dd kkkkkkkkkkk".   I can't figure out why this is happening because my linked list hold only two elements, namely socketchannel1 connected from c1 to server and socketchannel2 connected from c2 to server.   in the sendToAll(String message), message passed to this method is printed just as it is, and after encoding this to be written into the channel, I tested to see if this buffer is what i want it to be written to the channel so i decrypted, and it prints exactly what i want.   so i don't know why c2 received two messages because the background thread that runs in Clinet class to receive any incoming messages is only started once.   also, what i figured was that if i type in at c2 the messages "hello" and "dddddddddddd kkkkkkkkkkk" in that order, what i said above happens but if i type in "dddddddddddd kkkkkkkkkkk" first, then it is successfully sent to c1 and c2.   please let me know if you have any suggestion, or if you have time, i'd be greatly appreciated if you can take a look at what i have.   in that case, i'll increase the point value as well since that might take more time...
> but c2 receives two messages, "dddddddddddd kkkkkkkkkkk" and "dd kkkkkkkkkkk".

See my above comments.
There is no way for you client to know the size of the message.
It simply reads whatever bytes are avaible into the buffer and assumes that is the next message.
SocketChannel does not seem to provide a method for such issue (i.e. a way to know if all bytes are read into a buffer).   is there any way (or method) that can be used to counter this problem?
No it's just reading a stream of bytes, same as standard I/O. You needs to handle these kinds of issues yourself. By providing some means to first determine message seperation (eg. fixed size messages, or include message length), and then to ensure you have read the entire message.
k.   i'll try that.