HttpUrlConnection leaves connections in CLOSED_WAIT state

Our java application fetches pages from various websites.  One of our customers is finding that their machines have a very large number of TCP/IP connections in CLOSED_WAIT states that are remaining for long periods of time.   We have not been able to reproduce this -- any thoughts?

I have attached our basic processing code (with non-relevant code removed).  We are running on Microsoft Windows Server 2003 and using Java 6 and the java.net.HttpURLConnection class.   For each connection, we close the input stream after reading it, but do not disconnect, so that Java can re-use the connection.  This normally produces many connections with TIMED_WAIT states, but only an occasional short-lived CLOSED_WAIT state.








URLConnection urlConnection = url.openConnection();
    urlConnection.setConnectTimeout(timeout);
    urlConnection.setReadTimeout(timeout);

    if (isPost()){
      ((HttpURLConnection)urlConnection).setDoOutput(true);
    }
    urlConnection.connect();   
    
    try{
      try{
        contentsStream = urlConnection.getInputStream();
      }
      catch(IOException e){
        if (urlConnection instanceof HttpURLConnection){
          contentsStream = ((HttpURLConnection)urlConnection).getErrorStream();
        }
        if (contentsStream == null){
          throw e;
        }
      }
  
      String contentEncodingHeader = getResponseHeader().getOne("Content-Encoding");
      if (contentEncodingHeader != null){
        if (contentEncodingHeader.equalsIgnoreCase("gzip")){
          contentsStream = new GZIPInputStream(contentsStream);
        }
        else if (contentEncodingHeader.equalsIgnoreCase("deflate")){
          contentsStream = new InflaterInputStream(contentsStream, new Inflater(true));
        }
        else{
          throw new UnsupportedContentEncodingException(getRequest(), contentEncodingHeader);
        }
      }
      
      while (bytesRead = contentsStream.read(current)) >= 0) {
      }
    }
    finally{
        if (contentsStream != null){
          contentsStream.close();
        }
        //Don't disconnect, as we want to let Java re-use the connection if possible.
        //disconnect(urlConnection);
    }

Open in new window

gh5Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Ajay-SinghCommented:
It typically happens when the application (here java) has closed the socket and OS didn't reclaim it.
0
gh5Author Commented:
What conditions would cause the OS not to reclaim the socket.  Is there something we should be doing in the Java application to prevent this?  

Our customer is hypothesizing that we are not properly responding to the CLOSE request in the site response, but I believe our code is correct and that Java handles this okay.  One flaw I did notice upon review of our code is that our try/finally block is not large enough.  If an error happens after connect() but before we explicitly read the input stream - we will not close the URLConnection's input stream.  (I accidentally removed code from the example above that checks the response code after connecting.  I believe either the connect() itself or checking the response code will open up the input stream.)  Can this cause this type of problem?
0
VenabiliCommented:
In most cases this means that you have keep-alive that are not getting closed - kinda known issue in the way the Sun implementation deals with the keep-alive- had seen it happening now and then and so far never found a real answer so a few workarounds and things to look at below:

1. Try System.gc() after the closing of the connection... (As per http://forums.sun.com/thread.jspa?threadID=5218501 - and I recall using it at least once before)
2. See also http://lists.w3.org/Archives/Public/www-jigsaw/2001MarApr/0020.html - it is old and deals with a somewhat different class but the problem is the same.

Maybe this is related as well:
http://mail-archives.apache.org/mod_mbox/ws-axis-user/200311.mbox/%3CD3365BA42AA4D611A5100008023D06D206F5B683@exchny49.ny.ssmb.com%3E or http://stackoverflow.com/questions/1936872/how-to-keep-multiple-java-httpconnections-open-to-same-destination (different problem but same core issue - see all the comments there)

Alternatively you might try to set the keep-alive differently but not sure how hard that would be and I never managed to make it work anyway

The only thing that really worked for me was changing the library (using HttpClient). And from what I see around - people seem to be doing the same
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
manavilaiCommented:
are you really using a pool to get your URLConnections.
CLOSED_WAIT is happening because the connection has been closed by the remote machine but still kept open by your code.you might need to see the life of the connection and close it as appropriate.
0
gh5Author Commented:
We are still looking into this issue.  After more auditing, the code may be working perfectly well.  What the customer is reporting is not happening consistently and we have not able to see it when auditing the system.  Instead, we see that the CLOSE_WAIT usually resolve themselves relatively quickly.  There may be other configuration issues with the customers setup that is interfering with the TCP/IP process and therefore occasionally causing this issue.

Thanks for the responses.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Windows Networking

From novice to tech pro — start learning today.