Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17


Recent Slow Website Performance in LAMP Setup When DB Job is Run in Background

Posted on 2014-09-06
Medium Priority
Last Modified: 2016-11-23
We are running a LAMP platform which has the following hardware:

-Two Dell R200's running Red Hat (one separate website on each) serving as web servers
-Cisco Smart switch
-One Dell 2950 w/32 gb RAM, 2Xquad core XEON 2.3 ghtz CPU's and 8 x 600gb SAS 10k RPB drives (two RAID groups 1-  RAID1 for Solaris OS and 2-  5+1 RAID5 w/hot spare

Things seem to be going well  until about a month ago, when the performance of the websites would take huge latency hits whenever we ran a database job in the background. Bear in mind that we have been doing this in production since early November, 2013.

Now, however, whenever we run one of these jobs (e.g., updating tables, backing up database, etc.) the response rate for the server tanks severely.

We've absolutely ruled out the issues being related to the network or the Web servers (after many hours of diagnostics).

We carefully checked out all aspects of the Dell 2950. We've run Dell diagnostics against the entire system (as well as Solaris CPU, memory test commands) and it reports that all of the CPU threads are healthy, the RAM is 100% operational in the disk drives in the raid groups are fine.

We decided that the volumes on the server are exceeding some performance threshold and we think that we're going to drop two gigabit network cards (Intel E1G42ET) into the 2950 and move everything into a Dell 3200 ISCSI San.

We could then re-create the luns across 12 drives (and probably continue to use the 2950's first raid one group for the OS).

My question is: will this alleviate these latency issues that have popped up over the last month?  What might be a better approach?
Question by:bhunger
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
LVL 11

Accepted Solution

Murfur earned 2000 total points
ID: 40312339
By your troubleshooting you seem to have decided that the problem must lie with the Dell 2950 offering the RAID but I'm not so sure.

My initial guess would be that the problem lies within MySQL

What is the slow MySQL job actually trying to do? Where is your MySQL installation - local or remote? Assumption is local given that you looked to the RAID for the solution but if not actually connecting to "localhost" then you might want to prevent MySQL from making DNS lookups by adding skip-name-resolve to your MySQL conf.

If you are running backups/dumps then you might want to consider taking them off that server as processes go, they are notoriously resource hungry. For example, you could set up MySQL replication and have the jobs run from the slave thereby leaving the sites to access the master with "normal" performance. FYI a master server will handle all inserts & updates but a slave will only accept selects and the backups running from that won't affect performance on the master.

Digging deeper, you need to look at all elements of the transaction as any one of them could be the cause of the bottleneck:
Web site -> web server -> MySQL service -> MySQL server and of course the connections in-between them all.

Activate the slow query log to see if you can spot a particular bottleneck. Look out for table locks or queries with too many joins etc.

That should get you going for a start but let me know how you get on...

Author Closing Comment

ID: 40313089
Thanks for the answer.  We'll follow your recommendations and see where it leads us.

Featured Post

U.S. Department of Agriculture and Acronis Access

With the new era of mobile computing, smartphones and tablets, wireless communications and cloud services, the USDA sought to take advantage of a mobilized workforce and the blurring lines between personal and corporate computing resources.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The article will include the best Data Recovery Tools along with their Features, Capabilities, and their Download Links. Hope you’ll enjoy it and will choose the one as required by you.
A look at what happened in the Verizon cloud breach.
This tutorial will walk an individual through the process of installing the necessary services and then configuring a Windows Server 2012 system as an iSCSI target. To install the necessary roles, go to Server Manager, and select Add Roles and Featu…
In this video, Percona Solutions Engineer Barrett Chambers discusses some of the basic syntax differences between MySQL and MongoDB. To learn more check out our webinar on MongoDB administration for MySQL DBA:…

722 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question