When to close the web service client connection in a scaled environment

as per our other thread on

https://www.experts-exchange.com/questions/27630729/Web-service-unclosed-connections.html

I would recommend creating and disposing the clients promptly as required...

(dont include the declaration in the class or the intialiser in the ctor - rather store the credentials in private variables in the class)

check out the code below..

Note - i have only implemented the put method wrapper in my code example as its hard work writing without the service reference but I'm sure you can follow the path to fill in the gaps..



namespace MetaSource.SPF.Storage
{
    public class SPFStorageSPF// : ISPFStorage
    {
        protected static class Params
        {
            public static string Site;
            public static string UserName;
            public static string Password;
            public static string BasePath;
        }

        public SPFStorageSPF(string site, string userName, string password, string basePath)
        {//ms coding guideline suggest input params should be camel-case
            Params.BasePath = basePath;
            Params.UserName = userName;
            Params.Password = password;
            Params.Site = site;  
        }
        protected SPFWebService.File.Service1 CreateClientConnection()
        {
            return CreateClientConnection(60000);
        }
        protected SPFWebService.File.Service1 CreateClientConnection(int timeOut)
        {
            SPFWebService.File.Service1 tret =   new SPFWebService.File.Service1();
            tret.Url =Params. Site;
            tret.Credentials = new NetworkCredential(Params.UserName, Params.Password);

            tret.Timeout = timeOut;
        }
        #region ISPFStorage Members

        public bool Put(
            out string FilePath,
            ref string FileName,
            string FileExtension,
            bool Image,
            string ClientCode,
            int ProjectID,
            int DocumentID,
            int FileID,
            byte[] FileData)
        {
            string errorMessage;

            SPFWebService.File.Service1 fileWebService = CreateClientConnection();

            bool tRet =     fileWebService.Put(
                Params.BasePath ,
                ClientCode,
                ProjectID.ToString(),
                ref FileName,
                FileExtension,
                Image,
                DocumentID,
                FileID,
                FileData,
                out FilePath, out errorMessage);

            fileWebService.Close();

            return tRet;
        }

       
        #endregion
    }
}

and there really should be a try block around the actual call too..

Hope that is what you were looking for.

lojk:


Calling:

.Close();

for the webservice client object does not affect TIME_WAIT on the server.

I want to close the TCP socket when the webservice is not in use.  Close() does not help.

They are discussing it in detail on this thread.

http://stackoverflow.com/questions/2014887/socket-close-doesnt-really-close-tcp-socket-c

Without calling the close though, nothing would have worked. WCF is way above the TCP layer and the top answer on that page makes a whole lot of sense to me.

I've seen some mentions for Socket.Shutdown(  )

Can a webservice make this call somehow?

I saw that too - got a  feeling the underlying socket is accesible from the client might even be as simple as
fileWebService.close ()
fileWebService.Socket.Shutdown()
but currently in restaurant tapping on my mobile. If you haven't sussd it by the time I get home a little later will hop on PC and check it out for you.

I couldnt remember where i had seen it - i think it was actually on an a WinMobile (XML-WS for .NET2) reference to a service but this page pretty much confirms you cant and dont in fact need to because it will reuse the closed connections socket fairly intelligently (i.e. in the scalable manner that you seek). I think i remember reading something about 'why you should move to WCF from XML-WS' when i actually did and scalability was listed as a major factor.

http://social.msdn.microsoft.com/forums/en-US/wcf/thread/a50c63ee-ba41-4f50-9979-038fa16b7b9f

In our new code above we are creating and closing the client channel in line with the preffered MS example method - have you actually tried hitting it with a few connections (e.g. a get then a put, then a get then a put from one client host) with the new close method implementation in place to see if it has reduced the total number of open TCP ports on the server and/or waiting for four minutes to see if they are just dropped from the netstat(?) trace in line with my earlier link's top answer?

I have not tried your code yet.

How would my **much simpler** "Hello World" code need to be modified in order to NOT leave sockets open unnecessarily?

Please provide your modified code based on the following (all you need is to create a console app that references the web service.

Here is the code behind for the console part:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace ConsoleApplicationTomTest
{
    class Program
    {
        static void Main(string[] args)
        {

            ConsoleApplicationTomTest.ServiceReference1.Service1Client sc = new ServiceReference1.Service1Client();

            
            OutputToConsole("Remote Address", sc.InnerChannel.RemoteAddress.ToString());
            OutputToConsole("Remote Address is Anonymous", sc.InnerChannel.RemoteAddress.IsAnonymous.ToString());
            OutputToConsole("Via", sc.InnerChannel.Via.ToString());

            sc.InnerChannel.OperationTimeout = TimeSpan.FromSeconds(10);

            string a = "";

            OutputToConsole(sc.State.ToString());

            for (int i = 0; i < 2; i++)
            { 
                a = sc.GetData(1);
                OutputToConsole(sc.State.ToString());
                OutputToConsole(a);
            }

            OutputToConsole(sc.State.ToString());
            sc.Close();
            OutputToConsole(sc.State.ToString());
                       
            Console.ReadLine();
        }

        private static void OutputToConsole(string outputmessage)
        {
            Console.WriteLine(outputmessage);
        }

        private static void OutputToConsole(string intro, string outputmessage)
        {
            Console.WriteLine(intro + ":  " + outputmessage);
        }
    }
}

Select allOpen in new window

Here is the WebService interace (just the boilerplate code .NET auto creates)

using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.ServiceModel;
using System.ServiceModel.Web;
using System.Text;

namespace WcfService1
{
    // NOTE: You can use the "Rename" command on the "Refactor" menu to change the interface name "IService1" in both code and config file together.
    [ServiceContract]
    public interface IService1
    {

        [OperationContract]
        string GetData(int value);

        [OperationContract]
        CompositeType GetDataUsingDataContract(CompositeType composite);

        // TODO: Add your service operations here
    }


    // Use a data contract as illustrated in the sample below to add composite types to service operations.
    [DataContract]
    public class CompositeType
    {
        bool boolValue = true;
        string stringValue = "Hello ";

        [DataMember]
        public bool BoolValue
        {
            get { return boolValue; }
            set { boolValue = value; }
        }

        [DataMember]
        public string StringValue
        {
            get { return stringValue; }
            set { stringValue = value; }
        }
    }
}

Select allOpen in new window

And then the fulfillment of the contract:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.ServiceModel;
using System.ServiceModel.Web;
using System.Text;

namespace WcfService1
{
    // NOTE: You can use the "Rename" command on the "Refactor" menu to change the class name "Service1" in code, svc and config file together.
    public class Service1 : IService1
    {
        public string GetData(int value)
        {
            return string.Format("You entered: {0}", value);
        }

        public CompositeType GetDataUsingDataContract(CompositeType composite)
        {
            if (composite == null)
            {
                throw new ArgumentNullException("composite");
            }
            if (composite.BoolValue)
            {
                composite.StringValue += "Suffix";
            }
            return composite;
        }
    }
}

Select allOpen in new window

All looks good to me you are opening and closing the chanell correctly but there is a touch of a flaw in your test method.

I reckon that every time you run the Main void it will leave a connection hanging around in TIME_WAIT as each client proces that ran would be unaware of any other connections but that is not how it would work in a (looks like a silverlight/wp7 you are creating?) real app (or my code sample).

If you wrap the contents of the Main Void with an additional for n loop - the theory (and the MS page in particular) says that it should still only leave one TCP connection open even though it has created and closed n client connections.

Is that the case?

Do you mean like this?

        static void Main(string[] args)
        {

            ConsoleApplicationTomTest.ServiceReference1.Service1Client sc = new ServiceReference1.Service1Client();

            
            OutputToConsole("Remote Address", sc.InnerChannel.RemoteAddress.ToString());
            OutputToConsole("Remote Address is Anonymous", sc.InnerChannel.RemoteAddress.IsAnonymous.ToString());
            OutputToConsole("Via", sc.InnerChannel.Via.ToString());

            sc.InnerChannel.OperationTimeout = TimeSpan.FromSeconds(10);

            string a = "";

            OutputToConsole(sc.State.ToString());


[subtitle]          [b]  for (int j = 0; j < 5; j++)
            {
                for (int i = 0; i < 5; i++)
                {
                    a = sc.GetData(1);
                    OutputToConsole(sc.State.ToString());
                    OutputToConsole(a);
                }
            }[/b]
[/subtitle]
            OutputToConsole(sc.State.ToString());
            sc.Close();
            OutputToConsole(sc.State.ToString());
                       
            Console.ReadLine();
        }

Select allOpen in new window

Sorry, I was trying to bold the nested for loops and it did not work.

netstat results AFTER closing the console app:

...

...


 [firefox.exe]
  TCP    127.0.0.1:64640        127.0.0.1:6999         TIME_WAIT       0
  TCP    127.0.0.1:64642        127.0.0.1:6999         TIME_WAIT       0
  TCP    127.0.0.1:64645        127.0.0.1:6999         TIME_WAIT       0
  TCP    127.0.0.1:64650        127.0.0.1:6999         TIME_WAIT       0
  TCP    127.0.0.1:64651        127.0.0.1:6999         TIME_WAIT       0
  TCP    127.0.0.1:64654        127.0.0.1:6999         TIME_WAIT       0
  TCP    127.0.0.1:64666        127.0.0.1:6999         TIME_WAIT       0

//HERE ARE THE ORPHANED SOCKETS ... THE WEBSERVICE CALL  IS ALREADY DONE ...NO NEED FOR THESE SOCKETS TO BE OPEN...

  TCP    [::1]:64680            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64681            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64682            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64683            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64684            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64685            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64686            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64687            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64688            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64689            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64691            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64692            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64693            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64694            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64695            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64696            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64697            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64698            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64699            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64700            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64701            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64702            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:65119            [::1]:64678            TIME_WAIT       0
  TCP    [::1]:65119            [::1]:64679            TIME_WAIT       0
  TCP    [::1]:65119            [::1]:64690            TIME_WAIT       0

Select allOpen in new window

More like this - create multiple client instances from one process (like my original sample above would..

        static void Main(string[] args)
        {

 for (int c = 0; c < 3; cj++)
{
            ConsoleApplicationTomTest.ServiceReference1.Service1Client sc = new ServiceReference1.Service1Client();

   OutputToConsole("Test Iteration", c);
         
           
            OutputToConsole("Remote Address", sc.InnerChannel.RemoteAddress.ToString());
            OutputToConsole("Remote Address is Anonymous", sc.InnerChannel.RemoteAddress.IsAnonymous.ToString());
            OutputToConsole("Via", sc.InnerChannel.Via.ToString());

            sc.InnerChannel.OperationTimeout = TimeSpan.FromSeconds(10);

            string a = "";

            OutputToConsole(sc.State.ToString());


           for (int j = 0; j < 5; j++)
            {
                for (int i = 0; i < 5; i++)
                {
                    a = sc.GetData(1);
                    OutputToConsole(sc.State.ToString());
                    OutputToConsole(a);
                }
            }
            OutputToConsole(sc.State.ToString());
            sc.Close();
            OutputToConsole(sc.State.ToString());
                       

}
            Console.ReadLine();
        }

Results after console app is closed (15 open sockets as expected):

  TCP    [::1]:64781            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64782            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64783            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64784            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64785            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64786            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64787            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64788            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64789            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64790            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64791            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64792            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64793            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:64794            [::1]:65119            TIME_WAIT       0
  TCP    [::1]:65119            [::1]:64780            TIME_WAIT       0

Select allOpen in new window

Here is my current console app C# code (updated):

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace ConsoleApplicationTomTest
{
    class Program
    {
        static void Main(string[] args)
        {

            for (int c = 0; c < 3; c++)
            {
                ConsoleApplicationTomTest.ServiceReference1.Service1Client sc = new ServiceReference1.Service1Client();

                OutputToConsole("Test Iteration", c.ToString());


                OutputToConsole("Remote Address", sc.InnerChannel.RemoteAddress.ToString());
                OutputToConsole("Remote Address is Anonymous", sc.InnerChannel.RemoteAddress.IsAnonymous.ToString());
                OutputToConsole("Via", sc.InnerChannel.Via.ToString());

                sc.InnerChannel.OperationTimeout = TimeSpan.FromSeconds(10);

                string a = "";

                OutputToConsole(sc.State.ToString());


                    for (int i = 0; i < 5; i++)
                    {
                        a = sc.GetData(1);
                        OutputToConsole(sc.State.ToString());
                        OutputToConsole(a);
                    }
               
                OutputToConsole(sc.State.ToString());
                sc.Close();
                OutputToConsole(sc.State.ToString());


                Console.ReadLine();
            }

        }

        private static void OutputToConsole(string outputmessage)
        {
            Console.WriteLine(outputmessage);
        }

        private static void OutputToConsole(string intro, string outputmessage)
        {
            Console.WriteLine(intro + ":  " + outputmessage);
        }
    }
}

Select allOpen in new window

I think what i am ultimately saying here is that i dont see how you can influence this.

We/you are creating, using and closing (now) the connections exactly as microsoft say, the TCP connections are staying open even after the intialising process has exited  - I dont know what else to suggest other than trying a Simple XML-WS but i dont see how it can differ significantly - it just seems to be behaviour by design.

The only thing thing i can offer is that if you read that ms forum page they link to a method to abort the client (i guess, attempting to drop the TCP connection more ruthlessly).

http://blogs.msdn.com/b/drnick/archive/2007/05/04/the-try-catch-abort-pattern.aspx

Maybe adding an Abort after the Close will do what you want - I'm just not sure its what you need and you may be confusing the issue now of TCP sockets closing when it makes sense to the Network Stack with TCP sockets never being closed until the .Net4 garbage collector swallowed up the orphaned services reference instances that were never being closed correctly.

I had a simliar issue to this originally when i first starting using WCF services but since ensuring that all connections are closed i dont see my 'no more connections available'  error message or whatever it said server side when it hit the MaxConnLimit.

Maybe there is a WCF Config setting in the binding to disable App/Connection Pooling because my gut feeling says that that is what this is (as does the first MS Forum Page) but disabling that seems counter intuitive to me.

>>>>you may be confusing the issue now of TCP sockets closing when it makes sense to the Network Stack with TCP sockets never being closed until the .Net4 garbage collector swallowed up the orphaned services reference instances that were never being closed correctly.

==========

That is the problem.  It's not just ONE console app running - but hundreds - each making a call to a webservice method x hundreds of calls per second (potentially).


Now, this is only my third day at this job and this is my first project.  I have not worked in many environments where scalability was part of my responsibilities.  

But it has been made clear to me that, while it seems to be NORMAL for the server OS to close the sockets when it is ready and "safe" to do so -- we cannot afford to wait for them to close this way.  That is the core of the issue.  We must close them as soon as the web method is finished and the socket is not really needed anymore.

Why can't webservice close the socket that was opened because it was in use?  Said another way - the only reason the socket is open is because the webservice was called.  If the webservice method knows that it is finshed, then we **should** be able to tell the OS "you can close this paricular socket now - I am not using it anymore"


I can't believe this is not possible.

Here is that original MS Social Forum page - It is David-So MSFT's comment that just rings the truest to me..

http://social.msdn.microsoft.com/forums/en-US/wcf/thread/a50c63ee-ba41-4f50-9979-038fa16b7b9f


Here's the MSDN Library Page for accessing Services using a client referred to by that page
http://msdn.microsoft.com/en-us/library/ms733912.aspx

"I can't believe this is not possible."

I dont think you have a choice - the original question did not indicate this level of service consumption. Scalability also falls on more than just the application tier - if the level of traffic is as you say then surely you need to look at load balancing that service onto multiple servers - handling hundreds of connections per second would be ambitious for any service, from a file system as well as a network stack perspective.

perhaps using a DNS round-robin to spread the load to multiple servers is your next option alternatively if you need to be so direct/aggressive with your control of the TCP channels you may have no other choice but to move to a native TCP client/server connection using a normal socket implementation and forgoing all the WCF/IIS fluff.

if you need to be so direct/aggressive with your control of the TCP channels you may have no other choice but to move to a native TCP client/server connection using a normal socket implementation and forgoing all the WCF/IIS fluff.

I may have overstated this a tad.

All I know is that my boss says that we are running out of available sockets.

Their current fix was to double the amount of available sockets to 10,000 instead of 5000 (minus the 1024 reserved sockets for system use).  So far, this has been working, but it is bandaids and duct tape.

Setting aside "normal OS behavior" for a minute ---  the reality is that we DO have thousands of sockets in a TIME_WAIT status at any one time.  And for no good reason other than the server OS has not seen fit to close them -- even though the reason they were opened in the first place (to facilitate the communication of the webservice method with the console app) was over LONG AGO.

My boss says "I understand it, and it is not acceptable to me to wait until the server OS closes the sockets.  Find a way to close them sooner... the ideal would be to close them as soon as they serve no useful purpose to the client."


That is my task.


Keep your thoughts coming on alternatives.


Must TCP IP be used?

What about HTTP?

But I guess HTTP sits on top of TCP/IP   (right?)   so that puts us back where we were.

Thank you for your help.  I am sorry if I offended you in any way.

Not offended buddy - but your last comment was bordering on cheeky.

You are getting paid for this - I'm not ;-)

How was my last comment bordering on cheeky?