Solved

Appending EOF character to output stream

Posted on 2004-09-03
62
5,720 Views
Last Modified: 2013-12-03
I have a client/server running, and my server sends back data to the client one line at a time. Now, the client doesn't know when to stop because it is goign to read until the eof character is reached.

This is my client code,

while ((readline = in.readLine ()) != null) {
                        
   System.out.println(readline);      

}

in is a BufferedReader object

On the server, i use,

out.println (data);

out is a PrintWriter object

But the readline doesn't return since I have added an end of stream marker. Can you please tell me how to append an end of file marker to my output stream so that the loop above would return?

Thanks

AJ
0
Comment
Question by:aratani
  • 23
  • 13
  • 9
  • +4
62 Comments
 
LVL 86

Expert Comment

by:CEHJ
ID: 11974962
Just arrange to send a rarely used character
0
 
LVL 4

Author Comment

by:aratani
ID: 11974985
That is not the solution I am looking for though. I would like it to send the proper character that would cause the stream to end, and my loop to stop and the condition,

readline == null

to be true.

Anymore suggestions?

AJ

0
 
LVL 86

Expert Comment

by:CEHJ
ID: 11975012
There isn't a 'proper' character really. This is protocol-specific. e.g. mail servers send a '.' on its own. You should do something similar
0
 
LVL 4

Author Comment

by:aratani
ID: 11975082
What is the representation for an EOF character in Java? You mean to say there is no way a server can signal to you that it is an eof? How come then this is what is suggested in the Java tutorial from Sun?

while ((readline = in.readLine ()) != null) {
                   
   System.out.println(readline);    

}

Probably, because there has to be something that results in the loop ending. I would like to know what I can append to my data that would cause the loop to end?

Any suggestions?

AJ
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 11975083
... and IRC uses a QUIT and SQUIT message
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 11975105
>>there has to be something that results in the loop ending

You can make it end by closing the stream, which will close the socket. Otherwise the loop will simply block, waiting for more to read
0
 
LVL 4

Author Comment

by:aratani
ID: 11975131
Nope I don't want to close my socket.
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 11975208
Why, btw do you want to signal eof? Are you sending a text file, or what?
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 11975241
btw just to fill in a bit, signals such as you mention only occur at the TCP/IP level, e.g. when you're disconnecting, otherwise the application protocol takes care of signalling things
0
 
LVL 4

Author Comment

by:aratani
ID: 11975296
i'm sending it a stream of data and it has to read it on the other side in that loop. I want to send something so that this loop can exit,

while ((readline = in.readLine ()) != null) {
                   
   System.out.println(readline);    

}

So, i need to know what to send for this loop to exit.

AJ
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 11975377
Send it a specific String:

if ("QUIT".equals(readline)) {
    break;
}
0
 
LVL 4

Author Comment

by:aratani
ID: 11975479
I believe I have already said before that I don't want that approach. If there is any other person who has an idea that can answer my question please post a message.

Thanks

AJ
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 11975550
And i'm afraid, i have already said, there isn't one. TCP/IP sockets don't work like that.
0
 
LVL 24

Expert Comment

by:sciuriware
ID: 11975623
Well, the whole idea of an end-of-file code is virtual: it is always something chosen
or simply a close of communication.

The only, 50-year old, code I know is EOD on the spot where you expect something else.
(EOD == 4)

Because difficulties with binary data, they always send a file length block before a binary file.
A text file should have EOD as only character on a line.


;JOOP!
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 11975658
You could of course use non-blocking io, then you wouldn't have a loop as such to worry about terminating
0
 
LVL 24

Assisted Solution

by:sciuriware
sciuriware earned 100 total points
ID: 11975664
P.S.:
// String s;  k is an inputStream,

     if((s = k.readLine()) == null || s.length == 1 && charAt(0) == 4)
     {
          // the end
0
 
LVL 21

Expert Comment

by:MogalManic
ID: 11975692
EOF characters are operating system specific.  
 - DOS uses CTRL-Z
 - Unix uses CTRL-D
 - Mac uses ?

Since you are sending and recieving a file across, potentially, multiple platforms, you need some sort of protocol.  For example HTTP sends the filesize before file is downloaded, and/or closes the stream.
0
 
LVL 7

Expert Comment

by:JugglerW
ID: 11977161
Just close the stream on server side and your readLine will return null.
0
 
LVL 92

Expert Comment

by:objects
ID: 11977549
>  I would like it to send the proper character that would cause the stream to end

> Nope I don't want to close my socket.

Can you explain why you want the stream to end, but you don't want to close the socket?
If the stream has ended why would you want to keep it open?

If you just want to exit the loop, then send an empty string (assuming that is not valid data).

while ((readline = in.readLine ()) != null && readLine.length()>0) {
                   
   System.out.println(readline);    

}
0
 
LVL 30

Assisted Solution

by:mayankeagle
mayankeagle earned 100 total points
ID: 11980057
Try wrapping your data in an XML:

<File>
  <Data>.... the entire data to be transmitted goes here ....</Data>
</File>

At the other side, when you get a </File>, you know that transmission is over. I'm assuming it can't be a part of the data ;-)

while ( ( readLine = in.readLine () ) != null && ! readLine.equalsIgnoreCase ( "</File>" ) )
0
 
LVL 30

Expert Comment

by:mayankeagle
ID: 11980060
Or maybe that you could first send the total number of lines that you would be transmitting, and then send the lines. The other side would loop only as many times as the number of lines.
0
 
LVL 30

Expert Comment

by:mayankeagle
ID: 11980064
int iNumberOfLines = Integer.parseInt ( in.readLine () ) ; // get the number of lines first

for ( int i = 0 ; i < iNumberOfLines ; i ++ )
{
  readLine = in.readLine () ;
  // process 'readLine'

} // end for
0
 
LVL 24

Expert Comment

by:sciuriware
ID: 11980929
Protest, EOD is not INSIDE a file, but practised to end terminal input,
and it's not UNIX but mainframe's.
Btw, I think mayankeagle's XML a great idea! Was it ever applied somewhere?

;JOOP!
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 11981907
>>Btw, I think mayankeagle's XML a great idea!

But how does its central idea or 'active ingredient' about termination:

>>readLine.equalsIgnoreCase ( "</File>" )

differ from what's already been suggested?
0
 
LVL 24

Expert Comment

by:sciuriware
ID: 11983493
CEHJ, in essence you are right, the principle was mentioned first by you.
But, as I indicated in another place, the XML-idea has in it that the final rule matches the opening rule,
which will be more acceptable to the questioner and his/her customers.
That's what I meant by "hitting a perception".

My humble opinion: you deserve at least split points for this; aratani will decide.
;JOOP!
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 11983598
>>which will be more acceptable to the questioner and his/her customers.

(Possibly - as long as s/he doesn't mind completely altering the protocol ;-))
0
 
LVL 92

Expert Comment

by:objects
ID: 11986355
> CEHJ, in essence you are right, the principle was mentioned first by you.

And what you suggested was to pick 'rarely used character' which is hardly practical in a real world situation.
0
 
LVL 30

Expert Comment

by:mayankeagle
ID: 11987030
>> differ from what's already been suggested?

Just that XML is more standard than using just some dummy character like # or any other ;-)

But I hope the number-of-lines idea was not propsed by anyone else ;-)
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 11988452
>>And what you suggested was to pick 'rarely used character' which is hardly practical in a real world situation

Why not? If it's character-based, and it seems to be, there are plenty of rarely-used characters
0
 
LVL 30

Expert Comment

by:mayankeagle
ID: 11988546
I guess the approach of sending the number of lines first is more extensible.... you need not use any special character for that. Some fool tester (bad pun) might try to break the system by putting the special character as a part of the data as well ;-)
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 11988565
>>I guess the approach of sending the number of lines first is more extensible

I doubt it - first off, if you were sending a file, you'd have to count them first wouldn't you? ;-)
0
Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

 
LVL 92

Expert Comment

by:objects
ID: 11988637
> Why not?

Cause it will fail.
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 11988660
ROTFL. It seems to work fine for the millions of POP client/servers that there are around and that uses the rarely-used ... full stop ;-)
0
 
LVL 30

Expert Comment

by:mayankeagle
ID: 11988948
>> first off, if you were sending a file, you'd have to count them first wouldn't you? ;-)

I see that they are being sent as Strings as per the code posted by the asker (because it has readLine ()). Which means that, you can always count them and send the count first. You can read the Strings from the file and store into an array or a Collection, and then send them after sending the size of the collection first.
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 11989130
>>You can read the Strings from the file and store into an array or a Collection, and then send them after sending the size of the collection first.

Effectively that would mean reading twice - once from the source and once from the collection
0
 
LVL 30

Expert Comment

by:mayankeagle
ID: 11989251
Doesn't really matter much as long as its not a very very real-time application where every loop counts. Reading from the Collection will not be very costly in terms of performance as its an in-memory thing.
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 11989286
>>Reading from the Collection will not be very costly in terms of performance

It'll be more costly than reading the source once and printing directly with the PrintWriter, which is all that's required ;-)
0
 
LVL 30

Expert Comment

by:mayankeagle
ID: 11989350
>> It'll be more costly than reading the source once

Since we're reading Strings one by one and sending them, there has to be a loop. How would reading from the file in a loop be better than reading from a Collection? I agree that in the case of sending the number of lines first, there are two reads: once from the file into the Collection, then from the Collection to the socket. But what I mean to say is that reading from the Collection will perhaps not be much of a performance-overhead unless every loop counts. We might compromise a little on performance if the solution is a little extensible in that you need not worry about the end-character being a part of the data.
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 11989387
>>in that you need not worry about the end-character being a part of the data.

Why should you need to worry about a special character on a line on its own?
0
 
LVL 30

Expert Comment

by:mayankeagle
ID: 11989404
>> Why should you need to worry about a special character on a line on its own?

Not sure if I got that. What I meant was: you don't need to worry about the special-character being in the data (what if it is?)

Special-character: #

Data:

ABCD
EFGH
#
IJKL

- simple case, where the special character coincidentally happens to be a part of the data being sent. I agree that the choice of the special character should be such that it would not likely be a part of the data, but it is not a full-proof or a fool-proof solution ;-)
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 11989424
>>but it is not a full-proof or a fool-proof solution ;-)

Well, as i've said, it works perfectly well in the POP protocol ;-)
0
 
LVL 30

Expert Comment

by:mayankeagle
ID: 11989435
Yes, I know it works well in many protocols. Was just suggesting a solution which I thought was more robust though slightly degrading in performance. Let's see what aratani's case is and what he has to say.
0
 
LVL 92

Expert Comment

by:objects
ID: 11992758
> Well, as i've said, it works perfectly well in the POP protocol ;-)

I suggest you read the RFC ;)
0
 
LVL 7

Accepted Solution

by:
JugglerW earned 200 total points
ID: 11994958
To end this endless discussion :-)

The question was simply to implement stream handling in such a way that the end of stream can be tested with:

while ((readline = in.readLine ()) != null) {                    
   System.out.println(readline);    
}

aratani said that a special character is not the solution!
As I've alread posted some days ago: Just close the output stream on server side and it works.
Here a small class that demonstrates this. Just run:

============= Save as EOFTest.java =================
import java.io.*;
import java.net.*;


public class EOFTest
{
    public static final String[] text = new String[]
    {
        "This is line 1",
        "This is line 2",
        "Another line",
        "The last line",
    };
   
    public static void main(String[] args)
    {
      new Server().start();
       new Client().start();      
    }
   
    static class Client extends Thread
    {
        public void run()
        {
            try {
      Socket              socket  = new Socket( "localhost", 12345 );
                BufferedReader reader = new BufferedReader( new InputStreamReader( socket.getInputStream() ) );
                String               line;

                // Read until readLine returns null
                while ( (line = reader.readLine() ) != null )
                    System.out.println("Client: Got line: " + line );
                System.out.println( "Client: End of InputStream reached.");                    
      } catch (Exception e) { e.printStackTrace(); }
           
            System.out.println("Client terminated");
        }
    }

    static class Server extends Thread
    {
        public void run()
        {
            try {
                ServerSocket server = new ServerSocket( 12345 );
      Socket       client = server.accept();
                PrintWriter  writer = new PrintWriter(client.getOutputStream() );
               
                for ( int idx = 0; idx < text.length; idx++ )
                    writer.println( text[ idx] );
                writer.close();
                System.out.println("Server: " + text.length + " lines written and output stream closed");                                            
            } catch (IOException e) { e.printStackTrace(); }
            System.out.println("Server terminated");            
        }
    }
}
============= End of EOFTest.java =================

Output is:

Server: 4 lines written and output stream closed
Server terminated
Client: Got line: This is line 1
Client: Got line: This is line 2
Client: Got line: Another line
Client: Got line: The last line
Client: End of InputStream reached.
Client terminated
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 11995155
>>Just close the output stream on server side and it works.

And s/he's also said that this is not an option ;-)
0
 
LVL 4

Author Comment

by:aratani
ID: 11997304
Hey guys,

Yeah thanks for all the responses. CEHJ is right, I didn't want to close the stream either. What I wanted to do was to have a connection between a client and a server (there is a thread for the server handling the client requests), and that they exchange information with each other. When the information sent from the server is over, then it needs to send something that would cause that loop, while ( (line = reader.readLine() ) != null ) to get over but keep the connection open so that the client can request more data and doesn't need to reopen the socket to get more data. I assume using JugglerW's method, the socket will close and the client would need to get another socket connection to send more data, and that the connection won't remain open.

I have this problem since I have to develop the server and the client has been developed by somebody else. I guess what has come about is that there is no real character or output I can send that would cause that loop to terminate (without closing the stream or the socket).

Thanks guys for your suggestions,

AJ
0
 
LVL 86

Assisted Solution

by:CEHJ
CEHJ earned 100 total points
ID: 11997479
Yes, you can't really develop a client/server application where each side has been written independently, as you need a protocol.
0
 
LVL 92

Expert Comment

by:objects
ID: 12001150
> I have this problem since I have to develop the server and the client has been developed by somebody else.
> I guess what has come about is that there is no real character or output I can send that would
> cause that loop to terminate (without closing the stream or the socket).

You need to decide with the person writing the client what should trigger a blank line, ie something that would *never* occur during normal communication.
eg. if a blank line is not valid data then it could be used to indicate to break the loop.
0
 
LVL 30

Expert Comment

by:mayankeagle
ID: 12003407
>> the client has been developed by somebody else

If you had told that initially, it would've helped evade a lot of discussion on this page ;-) is it possible to modify the client now? If yes, you can try one of the several approaches suggested, like decide upon a stop-character or a blank String or an XML structure or sending the number of lines first, etc
0
 
LVL 4

Author Comment

by:aratani
ID: 12006801
Thanks for all your suggestions. I wonder if anyone can tell me how do HTTP Servers work then. When you send a request for a webpage, how does the client know how much data to read.

When I designed some code in PHP for requesting data from an HTTP Server, I used the following code,

while (!feof ($socket)) {

  //Read the data
}

fclose ($socket);

So, that means that there is something that is sent from the HTTP server that causes that loop to exit.

Also, the problem with us is that the client is being developed in VB and the server in Java. So, we used a common protocol like TCP/IP to communicate.

Thanks

AJ
0
 
LVL 92

Expert Comment

by:objects
ID: 12006849
it either specified the size in the response, or it reads until eof.
0
 
LVL 7

Expert Comment

by:JugglerW
ID: 12008289
A HTTP 1.0 server normally closes the connection after sending a response (like my example above).
In the HTTP header you normally have a header field content-length which gives the client the length of content in bytes.
The header and content are divided by an empty line.

With HTTP 1.1 thing get more complicated. Beside content-length you may have chunked transfer encoding and keep alive connections.
For details see the HTTP specification (http://www.w3.org/Protocols/rfc2616/rfc2616.html).
But be warned: its not very entertaining to read.

0
 
LVL 86

Expert Comment

by:CEHJ
ID: 12009269
Not sure why the 'accepted answer' was the accepted answer, but anyway - thanks ;-)
0
 
LVL 30

Expert Comment

by:mayankeagle
ID: 12012819
HTTP often reads till it gets EOF. You could put it to something like this:

(Its a code written in C#. I used it while making a C# .NET HTTP server and tried to make it communicate with a Java HTTP client. The syntax is more or less same as Java - the idea is basically to make you cognizant with what the HTTP protocol is like and to make you understand the algorithm.)

// create a new memory-stream
MemoryStream memoryStream = new MemoryStream () ;

do
{
      int b = networkStream.ReadByte () ; // read a byte

      if ( b == 0x0d ) // if the byte is a '\r'
      {
            // append the '\r'
            memoryStream.WriteByte ( ( byte ) b ) ;
            // read the next-byte ('\n')
            b = networkStream.ReadByte () ;
            // append the '\n'
            memoryStream.WriteByte ( ( byte ) b ) ;
            break ; // break out of the loop

      } // end if

      while ( b != 0x0d ) // while the byte read is not '\r'
      {
            // append the byte
            memoryStream.WriteByte ( ( byte ) b ) ;
            b = networkStream.ReadByte () ; // read the next byte

      } // end while

      memoryStream.WriteByte ( ( byte ) b ) ; // append '\r'
      b = networkStream.ReadByte () ; // read the next-byte ('\n')
      memoryStream.WriteByte ( ( byte ) b ) ; // append '\n'

} // while there is data
while ( networkStream.DataAvailable ) ; // end do-while

// convert the bytes to the string message
string message = Encoding.UTF8.GetString ( memoryStream.GetBuffer () ) ;
memoryStream.Close () ; // close the network stream
return message ; // return the message

Somewhere HTTP messages often end with 2 end of lines, e.g., a typical HTTP message could be:

"POST / HTTP/1.1
Content-type: text/html
Content-length: 10
ABCDEFGHIJ

"
0
 
LVL 7

Expert Comment

by:JugglerW
ID: 12013619
@CEHJ:

> Not sure why the 'accepted answer' was the accepted answer, but anyway - thanks ;-)

If you read again the question it was:

> Can you please tell me how to append an end of file marker to my output stream so that the loop above would return?

And the correct answer is: You can't! Because the only way to terminate the given loop is to close the stream.

All other answers are valuable in itself and give advice how to change the given code to become better, but didn't answer the question correctly. :-)
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 12013652
>>
If you read again the question it was:

> Can you please tell me how to append an end of file marker to my output stream so that the loop above would return?

And the correct answer is: You can't!
>>

(which is why i'd already said that ;-))
0
 
LVL 24

Expert Comment

by:sciuriware
ID: 12015148
Please CEHJ, don't critisise all the decisions questioners make in all those questions ......

;JOOP!
0
 
LVL 4

Author Comment

by:aratani
ID: 12016733
Man, you guys take this too seriously. :).

Anyway, thanks for all your suggestions ...

AJ
0
 
LVL 24

Expert Comment

by:sciuriware
ID: 12016761
Well, CEHJ is nearly JAVA GENIUS (1 000 000 points) ! Expect celebrations!

;JOOP!
0
 
LVL 30

Expert Comment

by:mayankeagle
ID: 12016910
Nearly? He already was.... since long (I mean - overall points, not yearly).
0
 
LVL 4

Author Comment

by:aratani
ID: 12016977
Yeah, he is second highest in the Java rankings.

AJ
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 12018969
:-)
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

This was posted to the Netbeans forum a Feb, 2010 and I also sent it to Verisign. Who didn't help much in my struggles to get my application signed. ------------------------- Start The idea here is to target your cell phones with the correct…
In this post we will learn how to connect and configure Android Device (Smartphone etc.) with Android Studio. After that we will run a simple Hello World Program.
Viewers will learn about the regular for loop in Java and how to use it. Definition: Break the for loop down into 3 parts: Syntax when using for loops: Example using a for loop:
This video teaches viewers about errors in exception handling.

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now