Multiplexed Socket.Select() problem

In my program, I have a server socket that listens for new connections. When new connection requests arrive, a socket is created for the connection and I add the socket to an arraylist. To process data on these sockets, I execute a multiplex Select() command and pass the arraylist. I am basically looking for all sockets that have data on them waiting to be read. The problem is when I have an arraylist of 65 sockets or more. When I do a Socket.Select on an arraylist of 65 sockets, I get an "index out of bound" error message. I have included my program down below. For testing purposes, I have a client that spawns 100 thread. Each thread is trying to connect and send some data to my component.

using System;
using System.Collections;
using System.Net;
using System.Net.Sockets;
using System.Text;

class SelectTcpSrvr
{
     public static void Main()
     {
          ArrayList sockList = new ArrayList();
          ArrayList copyList = new ArrayList();
          Socket main = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);

          IPEndPoint iep = new IPEndPoint(IPAddress.Any, 9050);
          byte[] data = new byte[1024];
          string stringData;
          int recv;

          main.Bind(iep);
          main.Listen(1000);

          try
          {
               while (true)
               {
                    if (main.Poll(1000000,SelectMode.SelectRead))
                    {
                         Socket client1 = main.Accept();
                         IPEndPoint iep1 = (IPEndPoint)client1.RemoteEndPoint;
                         Console.WriteLine("Connected to {0}", iep1.ToString());
                         sockList.Add(client1);
                    }

                    if (sockList.Count > 0)
                    {
                         System.Threading.Thread.Sleep(500000);
                         copyList = new ArrayList(sockList);
                         Console.WriteLine("Monitoring {0} sockets...", copyList.Count);
                         Socket.Select(copyList, null, null, 10000);
                         foreach(Socket client in copyList)
                         {
                              data = new byte[1024];
                              recv = client.Receive(data);
                              stringData = Encoding.ASCII.GetString(data, 0, recv);
                              Console.WriteLine("Received: {0}", stringData);
                              if (recv == 0)
                              {
                                   iep = (IPEndPoint)client.RemoteEndPoint;
                                   Console.WriteLine("Client {0} disconnected.", iep.ToString());
                                   client.Close();
                                   sockList.Remove(client);
                                   if (sockList.Count == 0)
                                   {
                                        Console.WriteLine("Last client disconnected, bye");
                                        return;
                                   }
                              }
                         }
                    }
               }
          }
          catch (SocketException ex)
          {}
          finally
          {
               main.Close();
          }
     }
}
osalehupsAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

jrocnuckCommented:
I don't have any C# experience, but in my experience with select, don't you need to know some information that is returned from the select call?

if it times out you don't want to receive from any of the sockets...
and if a socket becomes available to receive on, don't you want to know which index into the array that you need to receive from?

also, in the Receive call, don' t you need to tell it the max buffer size?
0
osalehupsAuthor Commented:
The maximum number of sockets that a Windows Sockets application can use is determined at compile time by the manifest constant FD_SETSIZE. This value is used in constructing the FD_SET structures used in select (which I am trying to use). The default value in Winsock2.h is 64. This is why my application fails at 64. If an application is designed to be capable of working with more than 64 sockets, the implementor should define the manifest FD_SETSIZE in every source file before including Winsock2.h. One way of doing this may be to include the definition within the compiler options in the makefile. For example, you could add "-DFD_SETSIZE=128" as an option to the compiler command line for Microsoft C. The thing is how do you do that in c#?

0
jrocnuckCommented:
good question... no easy google search has given any hint yet.
0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

osalehupsAuthor Commented:
that is why it is a 500 point question ;-)
0
jrocnuckCommented:
This is an excerpt that from "Network Programming For Microsoft Windows 2nd Edition" by Anthony Jones and Jim Ohlund that you may want to consider:

The advantage of using select is the capability to multiplex connections and I/O on many sockets from a single thread. This prevents the explosion of threads associated with blocking sockets and multiple connections. The disadvantage is the maximum number of sockets that may be added to the fd_set structures. By default, the maximum is defined as FD_SETSIZE, which is defined in WINSOCK2.H as 64. To increase this limit, an application might define FD_SETSIZE to something large. This define must appear before including WINSOCK2.H. Also, the underlying provider imposes an arbitrary maximum fd_set size, which typically is 1024 but is not guaranteed to be. Finally, for a large FD_SETSIZE, consider the performance hit of setting 1000 sockets before calling select followed by checking whether each of those 1000 sockets is set after the call returns.


I'll continue to see if I find the equivalent for C#
0
jrocnuckCommented:
This book also has a .NET Sockets in C# chapter this is an excerpt regarding the usage of Select in C# that may be of interest:  (at least now I understand that the list after the Select only has the sockets needing to be read.. I like that feature)


Select I/O
With the limitations of blocking I/O for managing multiple sockets, .NET Sockets features a Select method that is similar to the Select Winsock 1 API that allows managing multiple socket I/O from one execution thread. Essentially, you can provide Select with a list of sockets to test for readability, writeabilty, and OOB data. The following code fragment demonstrates how to test sockets for readability:

// Assume we have 2 connected sockets - Socket1 and Socket2
Socket[] ReadList = new Socket[2];
ReadList[0] = Socket1;
ReadList[1] = Socket2;

Socket.Select(ReadList, null, null, 100000);

// When Select returns either by our timeout
// value 100000 or if data is pending on one
// of our sockets, the ReadList will contain
// only those sockets that need to be read.
for (int i = 0; i < ReadList.Length; i++)
{
      byte [] Buffer = new byte[1024];

      // Receive data from the returned socket;
      ReadList[i].Receive(Buffer);                                                            
}
Although Select can manage multiple sockets from a single thread, we highly recommend using our next model—asynchronous—especially if you are developing a high-performance server. For more information about using the Winsock select API, see Chapter 5.

Asynchronous I/O
The asynchronous model in .NET sockets is the best way to manage I/O from one or more sockets. It is the most efficient model of the three because its design is similar to the I/O completion ports model (described in Chapter 5) and on Windows NT–based systems it will use the completion port I/O model internally. Because of this, you can potentially develop a high-performance, scalable Winsock server in C# and even possibly in Visual Basic. For a complete discussion of how to develop a high-performance, scalable Winsock application, see Chapter 6.

In Table 13-2 we described several methods that may be used to process I/O asynchronously: BeginAccept, EndAccept, BeginConnect, EndConnect, BeginReceive, EndReceive, BeginSend, EndSend, BeginSendTo, and EndSendTo. Notice how each one of these methods has a “BeginXXX”-“EndXXX” pair for each of the major I/O-bound socket methods—Accept, Connect, Receive, Send, and SendTo.

To call one of the I/O socket methods asynchronously, you must call the “BeginXXX” method counterpart and supply a delegate (or callback) method in the “BeginXXX” call. When the “BeginXXX” call completes, your calling thread may continue processing other things while your supplied delegate method internally waits for I/O to complete. When the socket has completed your I/O operation, your delegate method is called to process the completed I/O results. Inside your delegate method, you can retrieve the completed I/O results using the EndXXX counterpart method.

For example, let's describe how to process a Receive call asynchronously. We chose Receive because it is one of the most common methods that can cause your application to block when you wait for data to arrive on a socket. To call Receive asynchronously, you must call BeginReceive, which is defined as

public IAsyncResult BeginReceive(
   byte[] buffer,
   int offset,
   int size,
   SocketFlags socketFlags,
   AsyncCallback callback,
   object state
);
Most of the parameters are similar to the Winsock recv API except for the callback and state parameters. The callback parameter accepts a delegate method that is used to handle the completed results of the asynchronous BeginReceive. The delegate method must have the following form:

public delegate void AsyncCallback(
   IAsyncResult ar
);
The ar parameter is an input parameter that receives an IAsyncResult object, which you can pass to the EndReceive counterpart method (alternatively, you can use the IAsyncResult object that is returned from the originating BeginReceive call). Also, IAsyncResult contains an important member variable, AsyncState, which contains per-I/O data that was originally passed in the state parameter of the originating BeginReceive call. Typically, you will use this per-I/O data object to pass buffer and socket information that is related to the receive call.

Once your delegate method is called after BeginReceive has completed, you should call EndReceive to retrieve the results of the asynchronous Receive, which is defined as

public int EndReceive(
   IAsyncResult asyncResult
);
EndReceive returns the number of bytes received in the buffer that was originally passed to BeginReceive. Once you have the completed results, you can begin processing the data received in the buffer.

 
 When you call BeginReceive, BeginReceiveFrom, BeginSend, or BeginSendTo, you are not allowed to access the supplied buffer until your delegate method has been called indicating that the asynchronous method has completed.
 
 

The following code fragment demonstrates how to call Receive asynchronously using BeginReceive and EndReceive. On the companion CD we have provided a sample called TCPServer that demonstrates how to call Accept, Receive, and Send asynchronously on a TCP socket.

// Assume we have a connected socket named MySocket

PerIOData PData;
PData.s = MySocket;

public AsyncCallback AsyncReceiveCallback = new
AsyncCallback(ProcessReceiveResults);

MySocket.BeginReceive(PData.Buffer, 0, PData.Buffer.Length,
      SocketFlags.None, AsyncReceiveCallback, PData);



public static void ProcessReceiveResults(IAsyncResult ar)
{
      PerIOData PData = (PerIOData) ar.AsyncState;

      int BytesReceived = PData.s.EndReceive(ar);

   // Do something about your received results
   . . .
}

public class PerIOData
{
   // Put whatever data you need here for the delegate method.
   // Most applications will probably define the data buffer
   // here for the received data.
   byte [] Buffer = new byte[4096];
   Socket s;
   . . .
}

0
jrocnuckCommented:
Your code looks like it came from the sample in this book:
http://www.sybex.com/sybexbooks.nsf/Booklist/4176



0
jrocnuckCommented:
That book's author is Rich Blum and he posts avidly on
microsoft.public.dotnet.languages.csharp

for example, check these:
http://groups.google.com/groups?q=rich+blum+csharp+socket&hl=en&lr=&ie=UTF-8&oe=UTF-8&start=40&sa=N

your best bet would be to probably go there and post and hope that he answers the question.  Being that he's written this book, he's probably had quite a bit more experience with both the limitations on #s of sockets and the combination of C#.
0
osalehupsAuthor Commented:
well, if I get an answer from Richard Blum i will give you the points since you suggected it. But I need an answer first. thx
0
jrocnuckCommented:
I just saw this example that looks to use 10000 sockets.  I don't know if it works, but you might find it interesting and worth a try

http://www.csharphelp.com/archives3/archive486.html

0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
jrocnuckCommented:
Also, although the documentation says it's only for older sockets, there is the WSAStartup function that takes the WSAdata structure for C/C++ and it has a member iMaxSockets.  My guess is that it really doesn't do anything, but it's another source of investigation.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Programming Languages-Other

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.