[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now

x
?
Solved

suggestions to find centos 5.10 memory leak on ml350 G4

Posted on 2014-08-09
16
Medium Priority
?
218 Views
Last Modified: 2015-03-17
Identical hardware ie ml350 G4's.The one running centos 5.9 doesn't miss a beat, BUT the two running centos 5.10; two different machines have memory leaks.  Same software running on all 3 machines. Machines will respond to ping, but the applications stop running & you can't log on or vnc etc onto the servers.  It doesn't seem to make a difference whether you logoff of just lock the session
0
Comment
Question by:Neale2210
  • 6
  • 4
  • 3
  • +2
16 Comments
 
LVL 38

Assisted Solution

by:Gerwin Jansen, EE MVE
Gerwin Jansen, EE MVE earned 668 total points
ID: 40251209
You could do some basic debugging to find out which process is using more memory.

A cron script that runs ps or top in batch mode every 5 minutes should give you an idea what is happening. Redirect output to a log file and analyze the log file after some time.

Example with top:
*/5 * * * * /bin/top -b -n 1 -o %MEM >> /tmp/toplog.txt

Open in new window

0
 
LVL 36

Assisted Solution

by:Seth Simmons
Seth Simmons earned 332 total points
ID: 40251406
two different machines have memory leaks

what tells you that you have memory leaks?
just having processes not respond doesn't necessarily mean something is leaking memory; could be multiple possibilities
0
 
LVL 13

Expert Comment

by:Sandy
ID: 40252804
Can you paste output of #free -m ?

and also please check which all processes are consuming most of the memory.. let us know the percentage...

TY/SA
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 
LVL 41

Expert Comment

by:noci
ID: 40259137
top, and sorting on M might help too
0
 

Author Comment

by:Neale2210
ID: 40262362
Sorry All, been busy, I double checked that the firmware was up to date on everything... raid controller was 2 updates old on one server, & memory swapped out... from the top
Gerwin... I'll try that idea
Seth... If your logs suddenly stop on the main program you're running ..either you've got a hardware fault.. or a program miss behaving.. after eliminating possible hardware issues..you then have to look at software...
Sandy...free -m wasn't that useful.. but I will post it, as you may be looking for something different
noci.... fair enough
0
 
LVL 38

Expert Comment

by:Gerwin Jansen, EE MVE
ID: 40262742
@noci - my batch mode top commmand is sorting on M(emory) :)
0
 
LVL 41

Expert Comment

by:noci
ID: 40262754
does it show growth on the usage of any process? or Cache for that matter?
0
 
LVL 38

Expert Comment

by:Gerwin Jansen, EE MVE
ID: 40262873
@noci - Growth: yes, any process: if run as root, yes,  cache: don't think so but top is just a start (other tools like vmstat will).
0
 

Author Comment

by:Neale2210
ID: 40266689
top -b -n 1 -o %MEM gives a usuage warning.... unknown argument 'o'
0
 

Author Comment

by:Neale2210
ID: 40266697
mem total:3546   used:1434    free:2111   shared:0    buffers:237   cached:764
-/+ buffers/cache:  432  3113   used/free

for 1 of the machines
0
 
LVL 36

Expert Comment

by:Seth Simmons
ID: 40266926
what is the last line of your free output?  you only pasted 2 of 3 lines
0
 

Author Comment

by:Neale2210
ID: 40267699
swap: 9339  used 0 free 9339
0
 

Author Comment

by:Neale2210
ID: 40267719
for the 2nd machine mem total:3546  used:1522   free:2024   shared:0  buffers 218  cached:902
-/+ buffers/cache: 401 3145  used/free
swap: 4094  used:0  free:4094
0
 
LVL 36

Expert Comment

by:Seth Simmons
ID: 40267883
ok...your swap partition isn't being used which is good
if there was a memory leak i would expect physical memory to be exhausted and swap space utilized but not seeing that
one place where i worked before we had an application that did file processing but if it came across a file of a certain size or data it didn't like it would leak memory until physical memoryand swap space was exhausted then the OOM killer would appear.  vendor acknowledged the issue and we put in a 2gb per-process limit on memory usage to mitigate the issue

doesn't seem to be the case here; not yet convinced of a memory leak
have you looked at syslog for anything that might help when this happens?
have you worked with the vendor at all for possible troubleshooting options for the application?
0
 

Author Comment

by:Neale2210
ID: 40281042
ok just seeing whether the firmware update to the raid controller made a difference, along with the change of memory.  The net difference in time before freezing was a gain of 3 days, probably attributable to the increase in memory.  So now I'm left with software, because I have an identical machine running centos 5.9, same program CommuniGate 6.0.5, with no freezing.

gerwin.. I still haven't had a correction to your 'top' command, to get mem stats.
0
 
LVL 38

Accepted Solution

by:
Gerwin Jansen, EE MVE earned 668 total points
ID: 40281725
I tested with another Linux distribution, sort on memory usage in CentOS(6):

top -b -n 1 -a

Can you try the above, if it works:
*/5 * * * * /bin/top -b -n 1 -a >> /tmp/toplog.txt

Open in new window

0

Featured Post

Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article was originally published on Monitis Blog, you can check it here . Today it’s fairly well known that high-performing websites and applications bring in more visitors, higher SEO, and ultimately more sales. By the same token, downtime…
This article will show you step-by-step instructions to build your own NTP CentOS server.  The network diagram shows the best practice to setup the NTP server farm for redundancy.  This article also serves as your NTP server documentation.
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial
In this video, Percona Solutions Engineer Barrett Chambers discusses some of the basic syntax differences between MySQL and MongoDB. To learn more check out our webinar on MongoDB administration for MySQL DBA: https://www.percona.com/resources/we…
Suggested Courses
Course of the Month19 days, 13 hours left to enroll

872 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question