?
Solved

Why my linux machine is showing poor performace? How to improve it ?

Posted on 2008-06-17
16
Medium Priority
?
452 Views
Last Modified: 2011-10-19
Please find my issues:

1) We deployed a simple website (no major cpu intensive computation or backend db operations) on the VM. We also have the same website deployed on a physical machine. When I hit the URL of the website deployed on the VM, I notice that the website takes way too much time (10 seconds) to load the first page (login). Note that the first page is essentially static content with no DB lookups whatsoever. I logged in to the VM and ran top at the same time as accessing the static page. I noticed that top froze ! The results didn't refresh for close to 5 seconds. The physical machine doesn't take that long to respond.

 

2) When I hit the second page, the website starts consulting the backend database and again the response time is too long (about 8 seconds). And top freezes again. The subsequent pages are okay and load up in considerably less time (< 3 seconds).

 

3) Another weird thing about this VM is that, when I try to ssh into the machine, the login prompt takes close 5 seconds to appear ! And once I enter the username, the password prompt takes another 5 seconds to appear ! However, once I have logged in, most commands that I run have reasonably good response times.

So, my guess is that the machine essentially freezes when new processes become active. In #1, "ruby" became active and in #2, "postmaster" (Postgres) became active. Once both ruby and postgres were active, the responses turned out to be much better.

Hope this helped. Please let us know what you find out. We really cannot use this VM in production especially when response times become unpredictable.

Why my linux machine is showing poor performace? How to improve it ?
0
Comment
Question by:jaisonshereen
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 7
  • 6
  • 2
  • +1
16 Comments
 
LVL 18

Expert Comment

by:larstr
ID: 21806264
What VMware product are you using?
0
 

Author Comment

by:jaisonshereen
ID: 21806379
VMware virtual centre 2.5
0
 
LVL 19

Assisted Solution

by:http:// thevpn.guru
http:// thevpn.guru earned 400 total points
ID: 21806387
Check the following commands

ps uax
top
free
0
ATEN's HDBaseT Presentation at InfoComm 2017

Hear ATEN Product Manager YT Liang review HDBaseT technology, highlighting ATEN’s latest solutions as they relate to real-world applications during her presentation at the HDBaseT booth at InfoComm 2017.

 

Author Comment

by:jaisonshereen
ID: 21806400
workstation 6.x
0
 
LVL 18

Expert Comment

by:larstr
ID: 21806437
Virtual Center 2.5 and workstation 6.x???

I'm confused.. Is this virtual machine running on ESX or in VMware Workstation?

What storage system is the host system using?

Lars
0
 

Author Comment

by:jaisonshereen
ID: 21806466
sorry for confusion this is running on ESX

please ignore workstation
iSCSi
0
 
LVL 18

Expert Comment

by:larstr
ID: 21806533
ok...
- Is this a java based web server?
- What guest OS?
- Is VMware Tools installed in the guest?
- What virtual disk controller?
- Are you using a 1000Hz kernel? SMP?
- How many vcpus did you give your guest?
- Use esxtop on the esx console to get the following values for your guest:
   %USED %RDY %CSTP
- How many ESX servers are connected to this lun?

Lars
0
 

Author Comment

by:jaisonshereen
ID: 21806667
Is this a java based web server?
Apache and Tomcat integrated is working

- What guest OS?
Redhat linux

- Is VMware Tools installed in the guest?
yes
- What virtual disk controller?

please provide me steps to check that.

- Are you using a 1000Hz kernel? SMP?
please provide me steps to check that.
- How many vcpus did you give your guest?
please provide me steps to check that.
- Use esxtop on the esx console to get the following values for your guest:
please provide me steps to check that.
   %USED %RDY %CSTP
- How many ESX servers are connected to this lun?
please provide me steps to check that.


[root@server ~]# chkconfig --list
NetworkManager  0:off   1:off   2:off   3:off   4:off   5:off   6:off
NetworkManagerDispatcher        0:off   1:off   2:off   3:off   4:off   5:off   6:off
acpid           0:off   1:off   2:off   3:on    4:on    5:on    6:off
anacron         0:off   1:off   2:on    3:on    4:on    5:on    6:off
apmd            0:off   1:off   2:on    3:on    4:on    5:on    6:off
atd             0:off   1:off   2:off   3:on    4:on    5:on    6:off
auditd          0:off   1:off   2:on    3:on    4:on    5:on    6:off
autofs          0:off   1:off   2:off   3:on    4:on    5:on    6:off
avahi-daemon    0:off   1:off   2:off   3:on    4:on    5:on    6:off
avahi-dnsconfd  0:off   1:off   2:off   3:off   4:off   5:off   6:off
bluetooth       0:off   1:off   2:on    3:on    4:on    5:on    6:off
conman          0:off   1:off   2:off   3:off   4:off   5:off   6:off
cpuspeed        0:off   1:on    2:on    3:on    4:on    5:on    6:off
crond           0:off   1:off   2:on    3:on    4:on    5:on    6:off
cups            0:off   1:off   2:on    3:on    4:on    5:on    6:off
dc_client       0:off   1:off   2:off   3:off   4:off   5:off   6:off
dc_server       0:off   1:off   2:off   3:off   4:off   5:off   6:off
dhcdbd          0:off   1:off   2:off   3:off   4:off   5:off   6:off
dovecot         0:off   1:off   2:off   3:off   4:off   5:off   6:off
dund            0:off   1:off   2:off   3:off   4:off   5:off   6:off
firstboot       0:off   1:off   2:off   3:on    4:off   5:on    6:off
gpm             0:off   1:off   2:on    3:on    4:on    5:on    6:off
haldaemon       0:off   1:off   2:off   3:on    4:on    5:on    6:off
hidd            0:off   1:off   2:on    3:on    4:on    5:on    6:off
httpd           0:off   1:off   2:off   3:off   4:off   5:off   6:off
ibmasm          0:off   1:off   2:off   3:off   4:off   5:off   6:off
ip6tables       0:off   1:off   2:on    3:on    4:on    5:on    6:off
ipmi            0:off   1:off   2:off   3:off   4:off   5:off   6:off
iptables        0:off   1:off   2:on    3:on    4:on    5:on    6:off
irda            0:off   1:off   2:off   3:off   4:off   5:off   6:off
irqbalance      0:off   1:off   2:on    3:on    4:on    5:on    6:off
kdump           0:off   1:off   2:off   3:off   4:off   5:off   6:off
kudzu           0:off   1:off   2:off   3:on    4:on    5:on    6:off
lisa            0:off   1:off   2:off   3:off   4:off   5:off   6:off
lm_sensors      0:off   1:off   2:on    3:on    4:on    5:on    6:off
lvm2-monitor    0:off   1:on    2:on    3:on    4:on    5:on    6:off
mcstrans        0:off   1:off   2:on    3:on    4:on    5:on    6:off
mdmonitor       0:off   1:off   2:on    3:on    4:on    5:on    6:off
mdmpd           0:off   1:off   2:off   3:off   4:off   5:off   6:off
messagebus      0:off   1:off   2:off   3:on    4:on    5:on    6:off
microcode_ctl   0:off   1:off   2:on    3:on    4:on    5:on    6:off
multipathd      0:off   1:off   2:off   3:off   4:off   5:off   6:off
mysqld          0:off   1:off   2:off   3:off   4:off   5:off   6:off
netconsole      0:off   1:off   2:off   3:off   4:off   5:off   6:off
netfs           0:off   1:off   2:off   3:on    4:on    5:on    6:off
netplugd        0:off   1:off   2:off   3:off   4:off   5:off   6:off
network         0:off   1:off   2:on    3:on    4:on    5:on    6:off
nfs             0:off   1:off   2:off   3:off   4:off   5:off   6:off
nfslock         0:off   1:off   2:off   3:on    4:on    5:on    6:off
nscd            0:off   1:off   2:off   3:off   4:off   5:off   6:off
ntpd            0:off   1:off   2:off   3:off   4:off   5:off   6:off
oddjobd         0:off   1:off   2:off   3:off   4:off   5:off   6:off
pand            0:off   1:off   2:off   3:off   4:off   5:off   6:off
pcscd           0:off   1:off   2:on    3:on    4:on    5:on    6:off
portmap         0:off   1:off   2:off   3:on    4:on    5:on    6:off
postfix         0:off   1:off   2:on    3:on    4:on    5:on    6:off
psacct          0:off   1:off   2:off   3:off   4:off   5:off   6:off
rdisc           0:off   1:off   2:off   3:off   4:off   5:off   6:off
readahead_early 0:off   1:off   2:on    3:on    4:on    5:on    6:off
readahead_later 0:off   1:off   2:off   3:off   4:off   5:on    6:off
restorecond     0:off   1:off   2:on    3:on    4:on    5:on    6:off
rpcgssd         0:off   1:off   2:off   3:on    4:on    5:on    6:off
rpcidmapd       0:off   1:off   2:off   3:on    4:on    5:on    6:off
rpcsvcgssd      0:off   1:off   2:off   3:off   4:off   5:off   6:off
rwhod           0:off   1:off   2:off   3:off   4:off   5:off   6:off
saslauthd       0:off   1:off   2:off   3:off   4:off   5:off   6:off
setroubleshoot  0:off   1:off   2:off   3:on    4:on    5:on    6:off
smartd          0:off   1:off   2:on    3:on    4:on    5:on    6:off
smb             0:off   1:off   2:off   3:off   4:off   5:off   6:off
spamassassin    0:off   1:off   2:off   3:off   4:off   5:off   6:off
squid           0:off   1:off   2:off   3:off   4:off   5:off   6:off
sshd            0:off   1:off   2:on    3:on    4:on    5:on    6:off
syslog          0:off   1:off   2:on    3:on    4:on    5:on    6:off
tux             0:off   1:off   2:off   3:off   4:off   5:off   6:off
vncserver       0:off   1:off   2:off   3:off   4:off   5:off   6:off
vsftpd          0:off   1:off   2:off   3:off   4:off   5:off   6:off
winbind         0:off   1:off   2:off   3:off   4:off   5:off   6:off
wpa_supplicant  0:off   1:off   2:off   3:off   4:off   5:off   6:off
xfs             0:off   1:off   2:on    3:on    4:on    5:on    6:off
xinetd          0:off   1:off   2:off   3:on    4:on    5:on    6:off
ypbind          0:off   1:off   2:off   3:off   4:off   5:off   6:off
yum-updatesd    0:off   1:off   2:off   3:on    4:on    5:on    6:off
 
xinetd based services:
        chargen-dgram:  off
        chargen-stream: off
        cvs:            off
        daytime-dgram:  off
        daytime-stream: off
        discard-dgram:  off
        discard-stream: off
        echo-dgram:     off
        echo-stream:    off
        eklogin:        off
        ekrb5-telnet:   off
        gssftp:         off
        httpportfwd:    on
        klogin:         off
        krb5-telnet:    off
        kshell:         off
        ktalk:          off
        rsync:          off
        tcpmux-server:  off
        time-dgram:     off
        time-stream:    off
[root@server ~]#

Open in new window

0
 
LVL 18

Accepted Solution

by:
larstr earned 800 total points
ID: 21807238
It would be very usefull to know what Redhat version your guest is running (cat /etc/redhat-release). Redhat AS/EL 3.0-5.0 are shipped default with 1000Hz SMP kernels which is suboptimal in a virtual environment. 5.1 and later has a built in workaround for this issue, while the versions in between might need a manual workaround.

Tomcat is indeed java based, and it would be usefull to know what java version you're using. you are also probably using some version of JBoss.

In the viclient, when looking at the VMs General settings, how many vCPUs is this VM having (pic 1)?
You can also find the scsi controller if you Edit Settings of the VM (pic 2).

Can you access the console of the ESX server? Either remotely through ssh or locally. As root you can run esxtop to get a view of the performance of the VM (pic 3)

Do you have many ESX servers? How many of them are accessing this LUN? The more servers sharing a  LUN the more latency. This latency is however minimal if you only have a few servers.

How many VMs are using this LUN? What raid setup on this LUN? Do you have any stats from the SAN side? For example Cache Hit Ratio? You might need to ask your SAN administrator since these values are not available from the ESX side.

Lars

Any active snapshots on this VM? Right click VM in viclient, and Snapshot / Snapshot Manager.

esx-vm-general-vcpu.jpg
esx-vm-scsi.jpg
esx-esxtop.jpg
0
 
LVL 3

Expert Comment

by:nmv
ID: 21813136
I find it very odd that nobody has mentioned this yet, but check your DNS settings. Long load times of webpages and especially login prompts on SSH which are slow, are mostly DNS issues.

If you have just 1 user connected to a simple server like this and hit the button, it should be instant, no matter what resources you allocate (provided that this amount is equal or more then what is required by the OS of course.)

K.
0
 

Author Comment

by:jaisonshereen
ID: 21815940
nmv:

Can u explain what u mean by this ?

"If you have just 1 user connected to a simple server like this and hit the button, it should be instant, no matter what resources you allocate (provided that this amount is equal or more then what is required by the OS of course.)"


And some advices regarding the dns issue ..in detail and how to troubleshoot ?


0
 
LVL 3

Assisted Solution

by:nmv
nmv earned 800 total points
ID: 21816818
Hi,

you should troubleshoot with tools like 'host', 'nslookup' and 'dig'. Refer to the man pages of these tools to get more info on the syntax.
More specifically, ideally you should be able to name-resolve the IP you're trying to connect to and the server you're trying to connect to should be able to reverse lookup a name to go with your IP. In your httpd.conf you probably set up a hostname for the server. Check with nslookup or host to see if this name resolves through any of your dns servers that are set up in /etc/resolv.conf like this



suppose your have a configuration like this:

# hostname --fqdn
myhost.mydomain.com
# cat /etc/resolv.conf
search mydomain.com
nameserver 1.1.1.1
nameserver 1.1.1.2

then you should be able to do this:
# host myhost.mydomain.com 1.1.1.1
[output]

Withing [output] you should see the internal IP of your machine returned, and no error messages. Check if the hostname you set up corresponds to what has been set up in your httpd.conf

Best regards,
K.

0
 
LVL 18

Expert Comment

by:larstr
ID: 21821271
..or you can put the ip and name of the hosts you're connecting through ssh from in the /etc/hosts file.

The reason ssh is slow upon connect if dns is not resolving is that it's trying to log the connecting dns-name to utmp.

0
 
LVL 18

Expert Comment

by:larstr
ID: 21853154
Did you solve your performance problems? What steps did you do?
0
 

Author Comment

by:jaisonshereen
ID: 21853946
no still i couldnt
0
 
LVL 18

Expert Comment

by:larstr
ID: 21855639
The solution to your issues is probably not so far away, but you might benefit from getting someone at your site who is a bit more familiar with Linux or VMware to help you answer the questions that has been raised in this thread.

You might also want to review an article on VMware performance and compare it to your environment:
http://vmfaq.com/entry/25/

Lars
0

Featured Post

Not sure which OpenStack Certification to get?

So you’ve realized you might want to get certified in OpenStack, but you’re not sure what the benefits might be or even which one you should take. You know there are several certification courses you can choose from, but how do you know which one is right for you?

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this article, I will show you HOW TO: Suppress Configuration Issues and Warnings Alert displayed in Summary status for ESXi 6.5 after enabling SSH or ESXi Shell.
In this article, I will show you HOW TO: Install VMware Tools for Windows on a VMware Windows virtual machine on a VMware vSphere Hypervisor 6.5 (ESXi 6.5) Host Server, using the VMware Host Client. The virtual machine has Windows Server 2016 instal…
Teach the user how to join ESXi hosts to Active Directory domains Open vSphere Client: Join ESXi host to AD domain: Verify ESXi computer account in AD: Configure permissions for domain user in ESXi: Test domain user login to ESXi host:
This video shows you how to use a vSphere client to connect to your ESX host as the root user. Demonstrates the basic connection of bypassing certification set up. Demonstrates how to access the traditional view to begin managing your virtual mac…
Suggested Courses
Course of the Month13 days, 19 hours left to enroll

801 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question