Solved

Living in Fear of JRun (w/CFMX)

Posted on 2003-11-17
28
3,744 Views
Last Modified: 2013-12-24
JRun is the bain of my existance.  Since the conversion of CF 5 to CFMX (not an upgrade, complete new system) JRun has been a living hell.  

For no reason it will spiral up to 500 MB of memory usage and then die.   This happens without warning, and frequently.   It seems to happen while customers or myself experiment with new code.  This is not a standalone version of JRun, it is part of the CFMX package.

My setup is as follows:
 -Windows 2003 Server
 -CFMX 6.1 (with JDBC driver update/datadirect drivers 3.1)
 -1GB Ram
 -IIS 6.0

Question is, is there a patch hidden somewhere, or performance upgrades available anywhere to rectify this obviously fairly common problem?
0
Comment
Question by:tomkinite
  • 8
  • 3
  • 3
  • +8
28 Comments
 
LVL 6

Expert Comment

by:PE_CF_DEV
ID: 9767283
might post on macromedia's website and see if there is a hotfix for the problem.

In the mean time you might watch howmuch memory jrun.exe is using on average and se tthe JVM memory max to a little above that and the min a little below that.

Also make sure you don't have an outrageously high number in your simutanious connections.
0
 
LVL 12

Expert Comment

by:jyokum
ID: 9767752
I can't say that we've found a fix yet but we've been fighting a similar problem for about 2 months. Our environment is different (CFMX 6.1 for J2EE, Solaris, Apache) but the problem is the same... memory spikes and it all dies.

We've updated every conceivable peice of software, tweaked settings,  modified tons of code and still we have problems. Initially we were crashing multiple times per day... at the moment, we can make it about a week before it dies. Macromedia is going through all our logs every day and I think they've run out of ideas. At this point we're basically taking guesses. The graphing engine is suspect at the moment.

I'll try to put together a list of the things we've done so far, perhaps one of them will help you.
0
 
LVL 6

Expert Comment

by:PE_CF_DEV
ID: 9767770
Everynow and then our ram will spike and Jrun takes over all CPU usage on our test server, which runs dual xp athlons IIRC. We still have no solution but I think I have traced it back to something between coldfusion and our SQL server. Sometimes it will even throw and error 4096. I've search for the error but have come up with nothing...
0
 
LVL 12

Expert Comment

by:jyokum
ID: 9768122
we've found that CFMX doesn't handle loss of connectivity to a database very well. it seems the threads never get released... requests queue... memory goes bye-bye... server dies.   it's pretty reliable that if our database goes down, the CF server is soon to follow.  We primarily use Oracle db's but do have a few MS SQL & DB2 databases.
0
 
LVL 17

Expert Comment

by:anandkp
ID: 9768937
from what ive seen - JRUN does take about 47% of ur memory ... no matter what machines u have !

so even if u increase ur RAM - it wld still hover around 40 - 50% mark !
0
 
LVL 1

Expert Comment

by:techy6
ID: 9779361
Same problem, we've gone up to 3GB of RAM in our dual 2Ghz Xeon server, performance is STILL an issue!  Any ideas please let me know!
0
 
LVL 6

Expert Comment

by:PE_CF_DEV
ID: 9779606
I know one thing we did on our servers that helped with startup issues and some all around performance issues is we had our simutanious connections set to some where up in the 200-300 range which is a bit high...ok a lot high. We set it to 40 and things seem to go a lot smoother on restarts. In addition if you are using MX, which I assume we all are, if you use a lot of c++ cfx that will slow performance since JRUN and the JVM have to go through more to use the CFX, converting those to Java instead will increase performace. Same thing goes for COM objects coldfusion just runs slow when you use COM objects. I read some articles on it a while back if I can find them I'll put them up here.
0
 
LVL 1

Author Comment

by:tomkinite
ID: 9779772
To me, in a lot of ways CF is better than ASP but no one else can consider this platform with this issue.  It's not even a minor, uncommon issue.  It's very common.  The Macromedia boards are plagued with this very same issue.  Macromedia doesn't really help at all.  This is the context of most posts regarding this issue:

- User: "Memory leak, Jrun taking up 600 MB, etc"
- Macromedia: "Post your settings, configurations and process dumps"
- User: Posts all requested information.
- User (2 months later): "Hello?"

That's the general response from Macromedia.  It's very frustrating.
0
 
LVL 12

Expert Comment

by:jyokum
ID: 9780232
Which version of JVM are you using?

Here's a fix for some SQL server problems.
http://www.macromedia.com/support/coldfusion/ts/documents/cfmx61_sqlserver_cpu.htm
0
 
LVL 12

Expert Comment

by:jyokum
ID: 10049739
any progress on this?
0
 
LVL 1

Expert Comment

by:brian_summit
ID: 10097798
My main issue is with the cpu getting pegged at 90 somehting percent every so often, I have been talking with someone at Macromedia off and on for  over 3 weeks, trying to get my issue resolved with memory leaks, and cpu overutilization. (going over configs, sending in stack traces, etc) to no avail, we still have a major issue with jrun in CFMX and our current workaround is to auto restart cfmx every 4-6 hours, which is no good!
0
 
LVL 12

Expert Comment

by:jyokum
ID: 10098827
we've managed to rememdy the problem over here but I couldn't honestly tell you what "fixed" it. In the end, our sys-admins said the server was overloaded. We moved some of our sites to different servers to distribute the load a bit and the problem went away. The reason I'm not entirely sure this was the problem is that we also made several other changes at the same time.
0
Top 6 Sources for Identifying Threat Actor TTPs

Understanding your enemy is essential. These six sources will help you identify the most popular threat actor tactics, techniques, and procedures (TTPs).

 
LVL 1

Expert Comment

by:brian_summit
ID: 10099816
Is it possible to find out what other changes you made?
I have isolated one site on a new clean server, with a clean database server as well and still see the problem, guessing a magic setting change may help.....
0
 
LVL 12

Expert Comment

by:jyokum
ID: 10100117
I'll see what I can find out from everybody. we had no less than 5 people working on this so it was pretty much a big mess.
0
 
LVL 1

Expert Comment

by:techy6
ID: 10102428
FYI Jrun seems to have settled down on our servers now, having applied every updater and patch and driver under the sun.  We even got it running pretty fast on a knackered old PIII 600 server, and we've been able to scale down our live servers from 3GB of RAM to 2GB.  All our servers are running Windows 2K Advanced Server with SP4, SQL 2K with SP4, MX on Jrun with all the latest patches.  
0
 
LVL 1

Author Comment

by:tomkinite
ID: 10981193
No acceptable answer provided. Refund and close.

0
 
LVL 12

Expert Comment

by:jyokum
ID: 10987018
i'd recommend PAQ/refund
0
 
LVL 1

Expert Comment

by:brian_summit
ID: 10990833
The Jrun issue on our servers was code related and was fixed. I have confirmed with others that have not posted that this was their problem as well. Try putting the ball back in the developers court. The stack traces actually do have vauable info in them, get with macromedia support and have them point out the bad cfc or lines of code from the traces...... Good Luck.  CFMX is actually not to bad when it's running clean code.

-Brian
0
 
LVL 12

Expert Comment

by:jyokum
ID: 10991157
i agree there is no clear answer (or any "answer" for that matter) but i think it worth having the conversation archived. I don't see any harm in leaving it.
0
 

Accepted Solution

by:
ee_ai_construct earned 0 total points
ID: 11054345
PAQed, with points refunded (125)

ee_ai_construct - (re-order part number #xm34)
Community Support Moderator
0
 

Expert Comment

by:LiamFurniss
ID: 13712773
Are you instantiating any COM objects?  If so, these will cause pagefile leaks, which, in my experience, will cripple a server.

If you've got 6.1, use the ReleaseComObject() function on every COM object ref as soon as you're finished with it.

Fixed my problem, which I had for way too long, so I figured I'd share... ;-)

-Liam
0
 

Expert Comment

by:tomcropper
ID: 13740363
We are having the same problem I have a massive server, running Windows 2003, IIS6. MX with 10GB or RAM.  Everytime the jrun.exe gets over 512mb it just stops responding.  I've checked the exceptions log and see all these out of memory errors.  I have to restart the server completely, then watch the memory keep going up and up.

Any help would be great, I hate restarting this server everyday.

-Chuck
0
 

Expert Comment

by:rhantson
ID: 13903346
Been watching this post for some time...  And hopeing to find an answer...  what I have come up with (Since this is also a problem on our servers as well), is that there is no ONE thing that is causing this...  Macromedia says it's just problems with how you code your pages...  but there doesn't seem to be a way to actually pinpoint where the memory leak is actually occuring...

I see errors in the application log that says just "Occured in template...." without even an explination of WHAT the error might have been...

These appearing around the time that the memory leak decides to trash JRUN...

Incidentally, don't think you have to run your memory up to 512 or in our case as high as 1.4meg before JRUN will crash...  I've seen JRUN get trashed and the memory just hovering around 200K...  this is a little rediculous and getting to be a huge problem...
0
 

Expert Comment

by:whaleyk
ID: 14486271
turning off debugging on our production server seems to have fixed our problem!
0
 

Expert Comment

by:rhantson
ID: 14495636
We have noticed a number different things that seems to be putting this problem into the forefront...  and only one of them seems to be related to CFMX...

First off, we actually had to force a flush of all request variables at the end of each request.  Don't know why, but we had to forse it, apparently CF wasn't doing the job on its own.

Secondly, we had to really check on the DB Quereys and get them really optimized.  Loops and Joins were what was throwing fits into the CFMX system.

As well, when we went to CFMX 7 we saw a number of these problems just disappear, so the thought here is that Macromedia fixed a number of things.

Needless to say, every time we had JRUN's memory going through the roof, it was related to poor programming on our end....  and believe me, it took us quite a long time to get that through our heads because before CFMX, everything worked just fine... after, well...  you all know what happened....

Keep an EXTREMELY close eye on your application, server, and exception logs....  Undefined Variables, data locks, and other errors will just ramp the mem up in a heartbeat.  Fix them as you find them, and you'll see greater improvements.

We haven't fixed a thing by patches or upgrades...  only by fixing our code did we find our system not only stableize, but it's running better than it ever has run before!

I know everyone doesn't want to think its the way we code, but after months of fighting with the system, and changing our code, we brought everything under control.  (also opening up a gold support ticket and having Macromedia help us with the code as well helped a bit as well).

RH
0

Featured Post

Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

If you don't have the right permissions set for your WordPress location in IIS, you won't be able to perform automatic updates. Here's how to fix the problem.
Meet the world's only “Transparent Cloud™” from Superb Internet Corporation. Now, you can experience firsthand a cloud platform that consistently outperforms Amazon Web Services (AWS), IBM’s Softlayer, and Microsoft’s Azure when it comes to CPU and …
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…
This video explains how to create simple products associated to Magento configurable product and offers fast way of their generation with Store Manager for Magento tool.

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now