memory problems resulting in 'not enough system memory to execute query' and unavailability of server


We are experiencing frequently errors in the sqlserver error log:

Failed Virtual Allocate Bytes: FAIL_VIRTUAL_RESERVE 1048576

11/04/2008 06:19:27,spid59,Unknown,MEMORYBROKER_FOR_RESERVE <nl/>
11/04/2008 06:19:27,spid59,Unknown,MEMORYBROKER_FOR_STEAL <nl/>
11/04/2008 06:19:27,spid59,Unknown,MEMORYBROKER_FOR_CACHE <nl/>
11/04/2008 06:19:27,spid59,Unknown,Big Gateway <nl/>
11/04/2008 06:19:27,spid59,Unknown,Medium Gateway <nl/>
11/04/2008 06:19:27,spid59,Unknown,Small Gateway <nl/>
11/04/2008 06:19:27,spid59,Unknown,Optimization Queue <nl/>
11/04/2008 06:19:27,spid59,Unknown,Small Query Memory Objects:  Grants=0 Waiting=0 Maximum=6732 Available=6732 Limit=6732
11/04/2008 06:19:27,spid59,Unknown,Query Memory Objects:  Next Request=0 Waiting For=0 Cost=0 Timeout=0 Wait Time=0 Last Target=134655
11/04/2008 06:19:27,spid59,Unknown,Query Memory Objects:  Grants=0 Waiting=0 Maximum=127923 Available=127923 Limit=127923
11/04/2008 06:19:27,spid59,Unknown,Global Memory Objects: <nl/>
11/04/2008 06:19:27,spid59,Unknown,Process physical/virtual memory pressure: 0/0 System physical memory pressure: 0
11/04/2008 06:19:27,spid59,Unknown,Procedure Cache:  TotalProcs=3 TotalPages=21 InUsePages=2
11/04/2008 06:19:27,spid59,Unknown,Buffer Counts:  Committed=178879 Target=179200 Hashed=173734<nl/>
11/04/2008 06:19:27,spid59,Unknown,Buffer Distribution:  Stolen=3042 Free=559 Cached=1544<nl/>
11/04/2008 06:19:27,spid59,Unknown,OBJECTSTORE_LOCK_MANAGER (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,OBJECTSTORE_SERVICE_BROKER (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,OBJECTSTORE_SNI_PACKET (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,OBJECTSTORE_LBSS (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,USERSTORE_SXC (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,USERSTORE_OBJPERM (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,USERSTORE_TOKENPERM (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,USERSTORE_DBMETADATA (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,USERSTORE_SCHEMAMGR (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,CACHESTORE_SYSTEMROWSET (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,CACHESTORE_EVENTS (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,CACHESTORE_BROKERTO (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,CACHESTORE_BROKERREADONLY (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,CACHESTORE_BROKERRSB (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,CACHESTORE_BROKERUSERCERTLOOKUP (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,CACHESTORE_BROKERDSH (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,CACHESTORE_BROKERKEK (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,CACHESTORE_BROKERTBLACS (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,CACHESTORE_STACKFRAMES (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,CACHESTORE_XMLDBATTRIBUTE (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,CACHESTORE_XMLDBELEMENT (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,CACHESTORE_XMLDBTYPE (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,CACHESTORE_VIEWDEFINITIONS (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,CACHESTORE_NOTIF (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,CACHESTORE_TEMPTABLES (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,CACHESTORE_XPROC (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,CACHESTORE_PHDR (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,CACHESTORE_SQLCP (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,CACHESTORE_OBJCP (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,MEMORYCLERK_SQLSERVICEBROKERTRANSPORT (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,MEMORYCLERK_SOSNODE (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,MEMORYCLERK_HOST (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,MEMORYCLERK_SQLXP (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,MEMORYCLERK_FULLTEXT (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,MEMORYCLERK_SNI (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,MEMORYCLERK_SQLHTTP (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,MEMORYCLERK_SQLSERVICEBROKER (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,MEMORYCLERK_SQLCLR (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,MEMORYCLERK_SQLCONNECTIONPOOL (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,MEMORYCLERK_SQLSTORENG (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,MEMORYCLERK_SQLUTILITIES (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,MEMORYCLERK_SQLOPTIMIZER (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,MEMORYCLERK_SQLBUFFERPOOL (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,MEMORYCLERK_SQLGENERAL (Total) <nl/>
11/04/2008 06:19:27,spid59,Unknown,Memory node Id = 0 <nl/>
11/04/2008 06:19:27,spid59,Unknown,Memory Manager <nl/>
11/04/2008 06:19:27,spid59,Unknown,Failed Virtual Allocate Bytes: FAIL_VIRTUAL_RESERVE 1048576

After this error we keep getting more and more sql server memory wait states.  
Resulting in more and more errors like There is insufficient system memory to run this query .

And at last the server is no more available for connections and must be rebooted.

Our server was up and running for nearly 300 days and this started unexpected&

Any solution?

Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

pvsadmAuthor Commented:
attached application eventlog of server
Luis PérezSoftware Architect in .NetCommented:
300 days running up... it's a HUGE number of days for a data server to be running up. You said: "Our server was up and running for nearly 300 days and this started unexpected". Surprised? It's a good practice to make a periodical reboot (tipically at hours of few client activity), because we all know that system's memory loads and has little leaks and does not clears itself properly. If there are no hours of few client activity, then it means that you need more than one server, for example a web-farm cluster architecture.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Luis PérezSoftware Architect in .NetCommented:
Anywhere, if this problem has appeared once, probably the cause has been the long number of days running. But if it repeats frecuently, then maybe one or more memory modules of your system are corrupt. Try replacing them (it's a low cost solution).
pvsadmAuthor Commented:
there are no inidcations that something is wrong with the hardware. but we will check.
the problem is repeating... not (yet) predictable...

(i try to attach the application eventlog... but is doesn't seem to work)
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft SQL Server

From novice to tech pro — start learning today.