codejunky
asked on
Page File/ memory usage
I have a windows server 2003 that houses an application taht is very thread and socket intensive. the application, when under normal running conditions, has approximately 600 threads and 600 listeners running. MY question relates to the memory/ page file usage on the server. According to Task Manager, My available memory is usually around 780M and the system has 1G installed. The page file usage is around 900m - 1G. I would like to know why this is. I know it directly relates to the application, but I am not sure how and I can find nothing on the WWW to help.
Also, the application in question was develeoped in house by me using C#, if that helps in anyway.
Thanks in advance.
Also, the application in question was develeoped in house by me using C#, if that helps in anyway.
Thanks in advance.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
"my memory usage is remaining low"
How are you measuring this? Task manager by chance? Task manager doesn't show you actual memory usage it shows you your working set size. http://www.itwriting.com/dotnetmem.php explains more on the difference between the two. You want to use perfmon to check this (look at private bytes).
How are you measuring this? Task manager by chance? Task manager doesn't show you actual memory usage it shows you your working set size. http://www.itwriting.com/dotnetmem.php explains more on the difference between the two. You want to use perfmon to check this (look at private bytes).
ASKER
Oh, and I am using async sockets.
if you are creating every listener in its own thread how are you using async sockets? Perhaps there is a terminology problem here?
ASKER
basically I have a listener class that creates it's own thread for the listner to wait in. It is based on this example I found on the web.
http://www.csharphelp.com/archives3/archive486.html
http://www.csharphelp.com/archives3/archive486.html
ok but how many "Listeners" do you have? You only need one per port you are listening on, are you listening to 600 ports?
btw: that code is a fairly bad example there are some issues with it :)
Cheers,
Greg
Cheers,
Greg
ASKER
Yes, I am listening on 600 different ports and growing.
I would welcome any examples that are better... This was my first attempt at a socket applcation and it was a rather indepth project. since I first used the example, I have changed probably 90% of the code, but the thread creation has remained the same. I may not be understanding something , but it seems like the only way to handle multiple listeners.
I would welcome any examples that are better... This was my first attempt at a socket applcation and it was a rather indepth project. since I first used the example, I have changed probably 90% of the code, but the thread creation has remained the same. I may not be understanding something , but it seems like the only way to handle multiple listeners.
why are you listenning on 600 ports? you only need to listen to 1 port generally then when you accept the socket it gets accepted on another port. I just put up a small example of a purely async server to illustrate some buffer management work I did.
http://codebetter.com/blogs/gregyoung/archive/2007/06/18/async-sockets-and-buffer-management.aspx original discussion
http://codebetter.com/blogs/gregyoung/archive/2007/07/20/async-sockets-and-buffer-management-ctd.aspx (example)
There are no managed threads used in the example at all (only IOCP) and it supports 10k users easily.
http://codebetter.com/blogs/gregyoung/archive/2007/06/18/async-sockets-and-buffer-management.aspx original discussion
http://codebetter.com/blogs/gregyoung/archive/2007/07/20/async-sockets-and-buffer-management-ctd.aspx (example)
There are no managed threads used in the example at all (only IOCP) and it supports 10k users easily.
ASKER
the multiple ports is a design necessity. I will check out the links above. I appreciate your help. Your answers have explained what I was originally asking. I was completely unaware of 1M stack per thread. I am now trying to find a way to change that. Thanks alot for your help.
you say you are listenning on 600 ports ... can i ask why? this is very unusual.
ASKER
I really like to be different...
Actually, it's an application conceptually similar to what they are running at gotomypc.com.
Actually, it's an application conceptually similar to what they are running at gotomypc.com.
Yes i understand but you onlyneed one listener, it accepts sockets to other ports
ASKER
How exactly to do I create listener that will answere calls on both port 11000 and 12000 at the same time?
ASKER
Maybe I should have said create a single listener bound to 2 ports simultaneously
well for that you create 2 using your current methodology ... but you can do it without threads ... the example I gave above does listen on a port without any threads.
I can understand 2 or even 10 but 600 seems a bit over the top :)
I can understand 2 or even 10 but 600 seems a bit over the top :)
ASKER
It probably wouldn't if I could explain the applicaiton in more detail, but I am not at liberty to speak very intensively about the inner workings without some sort of confidentiality in place. I am really not trying to be difficult, I am just limited regarding what I can and can't say.
Thanks,
Chad
Thanks,
Chad
No worries. Either way check out my code, it doesn't need a thread for a listenner. I am copying/pasting the relevant portion here for you.
class SimpleServer
{
private IPEndPoint m_LocalEndPoint;
private Socket m_Listener;
private List<Connection> m_ActiveConnections = new List<Connection>();
public SimpleServer(IPEndPoint _LocalEndPoint)
{
if (_LocalEndPoint == null) { throw new ArgumentNullException("_Lo calEndPoin t"); }
m_LocalEndPoint = _LocalEndPoint;
}
public void BeginListenning()
{
if (m_Listener != null) { throw new Exception("Already listenning"); }
m_Listener = new Socket(AddressFamily.Inter Network, SocketType.Stream, ProtocolType.Tcp);
Console.WriteLine("Begin Listenning on " + m_LocalEndPoint.ToString() );
try
{
m_Listener.Bind(m_LocalEnd Point);
m_Listener.Listen(100);
m_Listener.BeginAccept(new AsyncCallback(AcceptCallba ck), m_Listener);
}
catch (Exception Ex)
{
Console.WriteLine("BeginLi stenning" + Ex.ToString());
}
}
private void AcceptCallback(IAsyncResul t ar)
{
Socket Listener = (Socket)ar.AsyncState;
Socket Handler = Listener.EndAccept(ar);
Listener.BeginAccept(new AsyncCallback(AcceptCallba ck), Listener);
Console.WriteLine("Receive d Connection From " + Handler.RemoteEndPoint.ToS tring());
Connection c = new Connection(Handler);
c.ConnectionLost += new ConnectionLostHandler(Clie nt_Connect ionLost);
lock (m_ActiveConnections)
{
m_ActiveConnections.Add(c) ;
}
}
void Client_ConnectionLost(Conn ection _Connection)
{
Console.WriteLine("Lost connection from " + _Connection.RemoteEndPoint .ToString( ));
lock (m_ActiveConnections)
{
m_ActiveConnections.Remove (_Connecti on);
}
}
}
Then it is called in the form:
class Program
{
static void Main(string[] args)
{
IPEndPoint p = new IPEndPoint(IPAddress.Parse ("127.0.0. 1"), 12345);
SimpleServer server = new SimpleServer(p);
server.BeginListenning();
Console.WriteLine("Server up and listenning hit enter to end program");
Console.ReadLine();
}
}
class SimpleServer
{
private IPEndPoint m_LocalEndPoint;
private Socket m_Listener;
private List<Connection> m_ActiveConnections = new List<Connection>();
public SimpleServer(IPEndPoint _LocalEndPoint)
{
if (_LocalEndPoint == null) { throw new ArgumentNullException("_Lo
m_LocalEndPoint = _LocalEndPoint;
}
public void BeginListenning()
{
if (m_Listener != null) { throw new Exception("Already listenning"); }
m_Listener = new Socket(AddressFamily.Inter
Console.WriteLine("Begin Listenning on " + m_LocalEndPoint.ToString()
try
{
m_Listener.Bind(m_LocalEnd
m_Listener.Listen(100);
m_Listener.BeginAccept(new
}
catch (Exception Ex)
{
Console.WriteLine("BeginLi
}
}
private void AcceptCallback(IAsyncResul
{
Socket Listener = (Socket)ar.AsyncState;
Socket Handler = Listener.EndAccept(ar);
Listener.BeginAccept(new AsyncCallback(AcceptCallba
Console.WriteLine("Receive
Connection c = new Connection(Handler);
c.ConnectionLost += new ConnectionLostHandler(Clie
lock (m_ActiveConnections)
{
m_ActiveConnections.Add(c)
}
}
void Client_ConnectionLost(Conn
{
Console.WriteLine("Lost connection from " + _Connection.RemoteEndPoint
lock (m_ActiveConnections)
{
m_ActiveConnections.Remove
}
}
}
Then it is called in the form:
class Program
{
static void Main(string[] args)
{
IPEndPoint p = new IPEndPoint(IPAddress.Parse
SimpleServer server = new SimpleServer(p);
server.BeginListenning();
Console.WriteLine("Server up and listenning hit enter to end program");
Console.ReadLine();
}
}
ASKER
that makes more sense than the infinte loop in my example. thanks for the advice. that should significantly reduce the memory foot print. not that it is overly necessary( I am only putting around 900 listeneres on a single server and memory is cheaper than time.), but it definately seems like a better implementation. since you illustrated the 1M stack size, I have reduced the stack size to 512k for these threads and that has helped alot form a size perspective.
ASKER
well, that was quick. I changed the code and the private bytes footprint with 600 active connections is 70M as opposed to 600M before. I appreciate your help.
Chad
Chad
:)
glad to help. see the rest of that post on buffer management as well (makes a big difference)
glad to help. see the rest of that post on buffer management as well (makes a big difference)
ASKER