Solved

Long MySQL query, browser timeouts

Posted on 2014-07-28
17
389 Views
Last Modified: 2014-08-27
I have one query that takes a very long time to process (very complicated select statement going thorugh lots of data) and my browser timeouts before it's finished. I can't do anything with the query so I need to keep the browser busy so it doesn't timeout. How do I do that?

I'm thinking I'll just let the query run in a separate thread and when it's done dump the result into a file. In the meantime, my browser will be repeatedly checking this file using ajax.

Is there a better solution? Can I just send back data to a browser so it doesn't timeout when it hangs on one query for a long time?

Thanks!
0
Comment
Question by:Carbonecz
  • 6
  • 4
  • 4
  • +2
17 Comments
 
LVL 32

Expert Comment

by:Stefan Hoffmann
Comment Utility
It depends on the use-case. Have you considered using AJAX?
0
 
LVL 21

Expert Comment

by:Kim Walker
Comment Utility
You may need to increase the mysql connect timeout in your php configuration. The default timeout is 60 seconds. You may also need to optimize your query. I ran into a similar situation recently that not only caused the connection to timeout, but also maxed out the server's RAM. I was able to improve mysql performance by creating several temporary tables.
0
 

Author Comment

by:Carbonecz
Comment Utility
The mysql connect timeout is not the issue (browser timeouts) and I can't optimize the query.
0
 
LVL 21

Expert Comment

by:Kim Walker
Comment Utility
Have you tried entering a timeout override in your code?
0
 

Author Comment

by:Carbonecz
Comment Utility
Yes, the issue is browser timing out, not the script.
0
 
LVL 21

Expert Comment

by:Kim Walker
Comment Utility
How long are we talking about? And which browser? The browser should stay open as long as it has a connection.
0
 
LVL 108

Expert Comment

by:Ray Paseur
Comment Utility
I can't optimize the query.
Why in the world not?  Is it running on your server?

Even if you can't optimize the query, you may be able to optimize the tables with indexes.  Have you used EXPLAIN SELECT to find out what the query is doing?
0
 

Author Comment

by:Carbonecz
Comment Utility
It's not mine, my job is just to solve this without touching the query.
0
6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

 
LVL 21

Expert Comment

by:Kim Walker
Comment Utility
You still didn't say how long. This sounds like server abuse.

And Ray's suggestion to index the tables is EXCELLENT if you haven't already done so.

Any negotiating regarding the restrictions to not touch the query?
0
 
LVL 108

Expert Comment

by:Ray Paseur
Comment Utility
This sounds like someone is setting you up for a failure by imposing a restriction on the one thing that would be most likely to succeed.  When queries run slowly there are only a very few things you can do to improve the response time.

1. Optimize the queries
2. Tune the MySQL configuration
3. Buy faster hardware
0
 

Accepted Solution

by:
Carbonecz earned 0 total points
Comment Utility
Well, it's not MySQL, it's Oracle. My mistake.

It takes 20 minutes or more. Firefox can't handle it and timeouts before there is a response. The query is very complicated and there is a lot of data being handled. I'm not the right person to optimize it and it's already been done by people who know more than me.

My job is to make sure the browser doesn't timeout and there is a response when the script finishes.

I was thinking I'd just trigger the script using ajax and then wait for the results to be dumped into some intermediate file which I'd check continuosly using ajax.
0
 
LVL 32

Expert Comment

by:Stefan Hoffmann
Comment Utility
Use an AJAX call to node.js..
0
 
LVL 108

Expert Comment

by:Ray Paseur
Comment Utility
I guess you could do that with AJAX, but this doesn't sound like it's really a suitable WWW application.  The general assumption of the internet community is that requests are atomic, complete and stateless and that responses are complete and usually instantaneous.  That's the way HTTP and RESTful applications like the WWW are expected to work.

How often is this query run?
0
 
LVL 11

Expert Comment

by:Murfur
Comment Utility
A 20 minute query has to make you wonder what it is doing and my first thought is inefficiency - either the database structure, poorly structured queries and/or a ridiculous number of joins. How much data is a 20 minute query expected to return to the web page (potentially loads) and is it a practical amount? The crunch though is that after a user initiates the query it simply isn't realistic to expect them to wait 20 minutes for the result.

As Ray and others have already suggested, indexes and hardware are probably your primary targets.

AJAX would be a logical approach to remove the action from the browser page but I think there may be a better approach:

Have a cron job (or similar) run the lengthy query and update a "results" table every hour or so.
Then all your page has to do is query the results table - a considerably quicker response.
0
 

Author Comment

by:Carbonecz
Comment Utility
I've requested that this question be closed as follows:

Accepted answer: 0 points for Carbonecz's comment #a40224365

for the following reason:

Well, it worked.
0
 
LVL 108

Expert Comment

by:Ray Paseur
Comment Utility
Instead of deleting the question, please post a comment showing how you solved the problem and accept your comment as the solution.  That way we have something useful for the community to learn from.  Thanks!
0
 

Author Comment

by:Carbonecz
Comment Utility
This was my solution:

"I was thinking I'd just trigger the script using ajax and then wait for the results to be dumped into some intermediate file which I'd check continuosly using ajax."
0

Featured Post

Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

Join & Write a Comment

The Confluence of Individual Knowledge and the Collective Intelligence At this writing (summer 2013) the term API (http://dictionary.reference.com/browse/API?s=t) has made its way into the popular lexicon of the English language.  A few years ago, …
I found this questions asking how to do this in many different forums, so I will describe here how to implement a solution using PHP and AJAX. The logical flow for the problem should be: Write an event handler for the first drop down box to get …
This video shows setup options and the basic steps and syntax for duplicating (cloning) a database from one instance to another. Examples are given for duplicating to the same machine and to different machines
This video shows how to configure and send email from and Oracle database using both UTL_SMTP and UTL_MAIL, as well as comparing UTL_SMTP to a manual SMTP conversation with a mail server.

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

5 Experts available now in Live!

Get 1:1 Help Now