Solved

Heavily loaded Php Application - Expert Optimization Techniques

Posted on 2010-11-14
15
541 Views
Last Modified: 2012-05-10
Hi experts have a little trouble with optimization ;

I have developed a gold stock exchange web based application that shows live currencies, by using ; php,mysql and js and it used to work fine for a while . Now its hosted from win 2003 server with 512mb rams / 2.2 ghz cpu and there's a page stays open on server that connects to a web service , takes currency datas from there and creates data.xml once in 12 seconds which all application works over it.Also this page connects to mysql takes profit margins from there and adds it to my customer's currency value before creating data.xml.Lets call this part as update.php.

While update.php is open once on server and renews it self once in 12 seconds ,a file called as main.php , parses data.xml and saves all current currencies buy / sell information in sessions to compare with the next values in order to display up or down arrows near by updated currency values and plays a sound if my customer's currency updated.

In a file called as program.php i setted up an interval in javascript that uses jquery load to load main.php every 12 seconds in to the main div , program.php is the file which is accessed by clients and stays open all time in client's computers.

Screen Shot of application
Result seems like above everything works perfect , but nowadays the simultaneous usage of application rised and there are at least 600 simultaneous connections which makes web server fail and stuck.

What i ask for is ;
is there a better way to build this application , i mean logically ? Would caching work here since the data is renewed every 12 seconds ?

Here are my solutions ;

Hiring my vps from vps.net which is linux + apache based much more better than win + iis build , also cloud hosted.
Creating another page that stays open at server to convert program.php 's output to html and save it as output.html and serve it from program2.html in every 12 seconds , so there will be only one session created and all php processes works once.

Any other solutions ?
0
Comment
Question by:ali_kayahan
  • 7
  • 7
15 Comments
 
LVL 7

Expert Comment

by:haijerome
ID: 34134412
Hi ali_kayahan,

 Glad to help you :-)

  Even before cache your files, you have to optimize your MySQL server for better results that you are looking for. Kindly go through with the link given below

http://www.experts-exchange.com/A_1250.html

If it helps then its cool else let me know !!!


Regards,
Jerome Dennis D
0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 34136002
Grab a copy of Firefox and install Firebug + YSlow.  Those will help you with the WWW part of the application.

Since we cannot see your data base, web service or programming, we probably cannot help you with anything more than theoretical guesses.  So here are some of the things that can help improve the speed of web applications.

Be sure you have indexes on every column used my SQL statements JOIN, WHERE, ORDER BY and GROUP BY.  If you are adding a lot of rows to the tables, consider dropping the indexes and recreating them instead of having them in place during the additions.

Use EXPLAIN on every complex query to see what MySQL is doing.

Use LIMIT whenever you can.  Avoid SELECT * and choose the columns you need instead.

Add script timers around any calls to external services -- see if the services are responding quickly enough.

It sounds like "data.xml" might be a design element that could be removed - you might just put this information directly into the data base?

Whether the currency is rising or falling really only requires two rows from the data base - current and most recent.  Your SELECT statement would ORDER BY most recent and LIMIT 2.

Determine whether there are enough data pipes into the server.  The word "connections" might mean HTTP connections or MySQL connections - either could be causing you trouble.

Should you move to Linux?  I would.  But I cannot tell you that it will necessarily help your performance.  Before I did that, I would hire a DBA to examine the application in detail.  For a few hundred dollars, you can get a world-class computer science expert to share a day of experience with you.  Sounds like it might be worth it.

HTH, ~Ray
0
 
LVL 14

Author Comment

by:ali_kayahan
ID: 34137102
@haijerome: Thanks a lot for that great mysql optimization article , i already clicked yes for it :) .But in my case ; i only use one row and one column even though all optimization tips already applied on it .

@Ray_Paseur : Ray , as i wrote mysql has really few jobs to do , in this case a query gets triggered by update.php  in every 12 seconds which works on server and closed for client access  , and compiles data.xml with the datas that has taken from web service.

As far as i know , yslow can only look for image,css,dom ect.. speed which is completely client side related anyway i checked it again and looks excellent.

I think there is no need DBA while the database in question is mysql anyway would it be smarter to let mysql handle all mess instead of xml and sessions ? In this case we will get rid of all session data which really constitutes a serious part of load whilst creating a serious headache for mysql.

But we should imagine that 600 clients asks for new data in every 12 seconds while upgrade.php writing new datas in every 12 seconds , this means query rain  , in this case should we use memcached ?


0
 
LVL 108

Assisted Solution

by:Ray Paseur
Ray Paseur earned 500 total points
ID: 34137369
You might consider memcachedD - but again, without looking at your application code and data base structure, all we can offer is the most general guidance.

The code snippet will show you how to time part of a script.  You might try applying that sort of measurement to various parts of your code.  600 client requests per 12 seconds is a steady-state load is 50 hits per second.  That will never actually occur, instead you will get a surging pattern - sometimes more and sometimes less.

Not sure why you think there is no need for a DBA -- in my experience, when there are performance problems with a computer program, the answer is almost always found in the I/O subsystem.  In a web application this usually means the data base.  That's why I recommend having a good data base administrator look at the structure and the queries.
<?php // RAY_script_timer.php

error_reporting(E_ALL);





// DEMONSTRATE HOW TO TIME THE EXECUTION OF A PART OF A SCRIPT PHP 5+

// MAN PAGE http://us.php.net/manual/en/function.microtime.php







// STORE THE START TIME

$alpha_time = microtime(TRUE);







// PERFORM SOME ACTIVITY THAT YOU WANT TO TIME

$page = 'http://google.com';

$html = file_get_contents($page);







// STORE THE COMPLETION TIME

$omega_time = microtime(TRUE);









// COMPUTE THE ELAPSED TIME AT A MEANINGFUL LEVEL

$lapse_time = $omega_time - $alpha_time;

$lapse_msec = $lapse_time * 1000.0;

$lapse_echo = number_format($lapse_msec, 1);



// SHOW A DESCRIPTION OF THE WORK AND THE TIME REQUIRED

echo "SCRIPT READ $page IN $lapse_echo MILLISECONDS";

Open in new window

0
 
LVL 14

Author Comment

by:ali_kayahan
ID: 34138357
Ray ; i just thought handling 50 queries per / second for mysql would consume more time than parsing xml . I can handle all mysql operations thats why we dont need DBA :) , so i will try to rebuild application by using mysql to store datas recieved from web service and use memcached to serve it clients.
0
 
LVL 108

Accepted Solution

by:
Ray Paseur earned 500 total points
ID: 34138463
Makes sense.  I don't think XML is necessarily giving you any added value here.  If the DB server is configured correctly a lot of information will be in cache most of the time anyway.  50 queries per second means 20 milliseconds for a query response - probably that is no problem at all if your DB is indexed correctly.  But it's still all a big fat guess because you have not posted anything meaningful, like the CREATE TABLE statements or the queries

When I think of XML, I think of it as a transport mechanism, not as a data store.  And depending on the element names, it can be an inefficient transport mechanism when compared to JSON or CSV.  I use it all the time, but not for efficiency - just for ease-of-use.
0
 
LVL 14

Author Comment

by:ali_kayahan
ID: 34139244
A table build like ;

create table currency(
    cId int not null auto_increment ,
    primary key(cId),
    prevGld int,
    currentGld int,
    prevDollar int,
    currentDollar int
    );

from program.php runs on server side once in 12 seconds ;

update currency set prevGld = currentGld , prevDollar = currentDollar ,currentGld = '$currentGld' , currentDollar = '$currentDollar' where cId = '1' ;
0
Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

 
LVL 108

Expert Comment

by:Ray Paseur
ID: 34139291
Are you suggesting that there is only one row in the data base, and that this is the only change every 12 seconds?  Are there any other tables?
0
 
LVL 14

Author Comment

by:ali_kayahan
ID: 34139337
Yes only one row in database no other tables , the row gets updated by program.php in every 12 seconds , as far as we have 5 currency values that we retrieve from web service , saving their previous values for comparision for up or down arrow would be enough , wouldnt it ?
0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 34139583
Sure, unless you want to keep some kind of historical record.  You could keep another table of similar structure and just do INSERT ... DELAYED.
http://dev.mysql.com/doc/refman/5.1/en/insert-delayed.html
0
 
LVL 14

Author Comment

by:ali_kayahan
ID: 34139968
So from client side that refreshes once in 12 seconds ;

   $memcache = New Memcache();
   $memcache->connect('localhost', 11211) or die("everything will be ok , really") ;
 
  $currencies = $memcache->get( 'values:' . $updatedValues ); // i set this from update.php that works on server

0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 34140785
I don't know how to answer you since I this appears to be only a tiny fragment of the code.  It looks right, I guess.  

What is the question at this point?
0
 
LVL 14

Author Comment

by:ali_kayahan
ID: 34141019
i was just trying to make you busy by asking foolish questions to gain time for answering other questions (: ok i was kidding , i just wanted to be sure about my new build , in update.php (the file works on server side once in 12 seconds and forbidden for client access) ;

1) i get datas from webservice ,
2) insert them in to mysql ,
3) populate memcache variable 'values' with query result that holds prev and current currency values.

from client side that 600 clients connect and renews it self once in 12 seconds

1) i run the code above to get memcache variable
2) fetch datas from that variable and parse them
3) than i pray for memcache not to commit suicide

is this build true or logically is there any pitfall ?
0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 34141629
Nothing could possibly go wrong.
0
 
LVL 14

Author Comment

by:ali_kayahan
ID: 34141693
Thanks a lot for your help and patience :) , i ll re-build the project and let you know the result
0

Featured Post

What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

Join & Write a Comment

Introduction Since I wrote the original article about Handling Date and Time in PHP and MySQL (http://www.experts-exchange.com/articles/201/Handling-Date-and-Time-in-PHP-and-MySQL.html) several years ago, it seemed like now was a good time to updat…
Load balancing is the method of dividing the total amount of work performed by one computer between two or more computers. Its aim is to get more work done in the same amount of time, ensuring that all the users get served faster.
The viewer will learn how to dynamically set the form action using jQuery.
The viewer will learn how to create and use a small PHP class to apply a watermark to an image. This video shows the viewer the setup for the PHP watermark as well as important coding language. Continue to Part 2 to learn the core code used in creat…

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

21 Experts available now in Live!

Get 1:1 Help Now