Java Async Socket

use wait() and notify()....

yes, insteaf of using async (which is hard to program and debug even in C) socket, use another thread for socket IO will solve your problem.  Besides, most of the time it is the readiing that will block rather than write() (for small size, it will return once it is copied to the kernel's output buffer)

>>>Is there a way to find out if a call to Socket.write will block before calling write, or have write return immediately if it would block?

No way to do that in pure Java as it encapsulates the relevant implementation details and if the underlying platform does provide a way to determine some functionality ,you have to call it explicitly by using JNI .
Remember it is 'portable' or whatever that means :)

The acceptable solution as 'faster' said use another thread for socket IO.
In
http://www.oreilly.com/catalog/jthreads2/

they have given a fantastic & complete example of AsynchInputStream by which we can easily extend that functionality to a class called as AsynchOutputStream,

else if you want a simpler & direct version of Socket Reader/Writer I can provide it to you today or on Tuesday. I am tired of reading in many books about spin off I/O in it's own thread and nobody except these guys have come across a full blooded implementation of HOW to do it actually in code.

btw ,the source code examples are freely downloadable ...

The program is currently written using threads.  

We have thousands of clients and we are pumping a lot of data out.    This presents two problems with a thread implementation:

1) All VMs we have tried bog down as you go over 2000 threads.

2) We use a queue with a wait/notify but with some of the slower clients the data is coming in much faster then they are reading it.  Therefore, the queues eventually run out of memory.

We could throw data out but it doesnt solve the thread limitation.  

An async socket would really nice.  A callback when the socket is free would be even nicer.  Sun are you listening?

The program is currently written using threads.  

We have thousands of clients and we are pumping a lot of data out.    This presents two problems with a thread implementation:

1) All VMs we have tried bog down as you go over 2000 threads.

2) We use a queue with a wait/notify but with some of the slower clients the data is coming in much faster then they are reading it.  Therefore, the queues eventually run out of memory.

We could throw data out but it doesnt solve the thread limitation.  

An async socket would really nice.  A callback when the socket is free would be even nicer.  Sun are you listening?

Sorry I am listening but will post something later on ,it's 10pm here at night.
:)

solution for your first problem?

(1)All VMs we have tried bog down as you go over 2000 threads.

read this,I dont know where I got it from,used a search engine.
'Connection scaling in Java '
https://www.experts-exchange.com/jsp/qShow.jsp?ta=java&qid=10294884 

The solution is to Use a ThreadPool, readymade code is available at the  above site,I storngly suggest you buy this book,I have it by my side always

As i said you can download code from the Examples section or if you face a difficulty I just now put the Zip file at this URL so you can download it

either of them I dont know which will work
http://www.geocities.com/SiliconValley/Cable/6751/Code/javathreads.zip
http://www.geocities.com/mbormann0/Code/javathreads.zip

feel free to download anything there that you like :)

(2)We use a queue with a wait/notify but with some of the slower clients the data is coming in much faster then they are reading it.  Therefore, the queues eventually run out of memory.

Yes you require a AsyncOutputStream class,but it will still not solve the problem of Memory.

Are your clients Java ? When you write say 100 bytes do you actually remove this written data from your queues at server? Dont you use a cache at the client side? this is over the Internet is it not?

>>>An async socket would really nice.
for what ? Input or Output ,if it's the former it's already available if the latter ,write it yourself?

>>>A callback when the socket is free would be even nicer.
can you clarify what do you mean by this?

See you on Monday :)

We feel thread pooling will help with the problem but not solve the problem.

The data is being sent via HTTP tunneling over the internet.  We don't have much choice.  Therefore, using a queue or cache other than TCP OS buffer on the client side is not easy.

It's so simple to do C++.  I was hoping there was a way do it in Java. Oh well.

Aha so there's a firewall involved ,also I have to guess what you are aiming at but my gut feel is that since your clients are slow and they are NOT in your control,it will look in first place

this is a general purpose server you are having which is pumping data and then your original question >>>if a call to Socket.write will block before calling write

has to be changed and phrased somewhat differently.

Can you provide answer to these specific questions?

(1)>>>Therefore, the queues eventually run out of memory.
talking abt TCP /IP stack buffers,I think this is ambiguous since they are on client side ,there maybe be a corruption of data since they can't keep up with teh incoming data,so I think depending on implementations on some it will corrupt ,on some it will run out of memory(I dont know abt the second,but I think mostly it will be the first,correct me here).

So my question is since you made this statement
>>>We use a queue with a wait/notify

The clients MUST be Java? So store the data as a cache as a StringBuffer in some thread at client side ,inside a class which has only 2 methods and that Buffer ,and read that data in another methid ,the 2 must be synchronized.

Cache the data at the Program level and not at TCP/IP stack.

(2)You have been giving conflicting signals about this client ,are they in your control or not ?

The queues I was referring to are used to pass messages between the main thread and the writer thread.  Therefore, if the main thread is producing data faster than the writer thread is writing it the queue between the two will grow and grow and grow.

The wait/notify is at the server level and handles the queues mentioned above.

We are in control of the client.  It is in java.  We do buffer/queue at the application level.  

Tunneling over the internet in some cases just isn't a fast enough transport for us.

You can define a very flexible producer-consumer scheme that would put to sleep the producer once the queue is full (you define the treshold, not just leave it grow uncontrolled) and wake him up once the it is being consumed again.

I do not really se a problem in this aspect.

Cheers,
  Nik

It all boils down to the fact that we DO NOT WANT TO BLOCK.  If we are blocking then the data we are blocking on is already out of date.  Therefore, there is no reason to continue to send it or send it in the first place.  

For example, say you have an applet that displays a stock quote.  The user is only interested in the most recent stock quote.  The stock quote changes 100 times a sec but your bandwidth only allows for 50 updates a second.  Why bother sending a quote that will be out a date if you know you are going to block?  You are better off waiting until you can send a whole quote without delay/block.

So I go back to my original question...

Is there a way to find out if a call to Socket.write() will block before calling write, OR have write() return immediately if it would block?

I am looking for the Java equilvalent of WSAWOULDBLOCK combined with WSAAsyncSelect in windows.

This is really easy in C++/Windows and so far I haven't found a way of doing it in Java.

Thanks for all your help and suggestions so far.
Milo

Even in C, where you do have async socket, don't want to block does not sound to be a practical choice.

If you quote does change so often, then it simply does not make sense to send it every time, how the user is supposed to react to so frequent change?  So most likely even if a quote is delayed for 1-2 second, it is still useful and should be delivered.  If your concern is to save bandwidth, then simply do a buffering on the sending side.

Besides, no block only means your kernel buffer has room, it does not mean it will actually be sent right away and even if it is sent, when it can reach the client is something you never know for sure (normally until you get the response from the other party)

2-3 coments above, about the outdated information:

When information is outdated, you do not send to the client. You can then skip sending information into the queue instead of blocking, as the client does not consume at the same rate you produce anyway. You can measure the consumption rate and adjust your skipping speed dynamically (or size of the queue too).

Cheers,
  Nik

Thanks for confirming my thoughts that there is no way of having a truly async socket in pure Java.

I appreciate all the design suggestions from everyone.  Unfortunately, we have tried most of them in the past.  They come really close but none of them are optimal.

Milo

Looks like we are not the only people who needed this....

JSR-000051 New I/O APIs for the Java Platform
(Review closes February 3, 2000)
 http://java.sun.com/aboutJava/communityprocess/jsr/jsr_051_ioapis.html

Sun might add async I/O to Java 2.