Solved

Long MySQL query, browser timeouts

Posted on 2014-07-28
17
410 Views
Last Modified: 2014-08-27
I have one query that takes a very long time to process (very complicated select statement going thorugh lots of data) and my browser timeouts before it's finished. I can't do anything with the query so I need to keep the browser busy so it doesn't timeout. How do I do that?

I'm thinking I'll just let the query run in a separate thread and when it's done dump the result into a file. In the meantime, my browser will be repeatedly checking this file using ajax.

Is there a better solution? Can I just send back data to a browser so it doesn't timeout when it hangs on one query for a long time?

Thanks!
0
Comment
Question by:Carbonecz
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 4
  • 4
  • +2
17 Comments
 
LVL 34

Expert Comment

by:ste5an
ID: 40224031
It depends on the use-case. Have you considered using AJAX?
0
 
LVL 22

Expert Comment

by:Kim Walker
ID: 40224059
You may need to increase the mysql connect timeout in your php configuration. The default timeout is 60 seconds. You may also need to optimize your query. I ran into a similar situation recently that not only caused the connection to timeout, but also maxed out the server's RAM. I was able to improve mysql performance by creating several temporary tables.
0
 

Author Comment

by:Carbonecz
ID: 40224078
The mysql connect timeout is not the issue (browser timeouts) and I can't optimize the query.
0
Online Training Solution

Drastically shorten your training time with WalkMe's advanced online training solution that Guides your trainees to action. Forget about retraining and skyrocket knowledge retention rates.

 
LVL 22

Expert Comment

by:Kim Walker
ID: 40224124
Have you tried entering a timeout override in your code?
0
 

Author Comment

by:Carbonecz
ID: 40224183
Yes, the issue is browser timing out, not the script.
0
 
LVL 22

Expert Comment

by:Kim Walker
ID: 40224217
How long are we talking about? And which browser? The browser should stay open as long as it has a connection.
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 40224232
I can't optimize the query.
Why in the world not?  Is it running on your server?

Even if you can't optimize the query, you may be able to optimize the tables with indexes.  Have you used EXPLAIN SELECT to find out what the query is doing?
0
 

Author Comment

by:Carbonecz
ID: 40224304
It's not mine, my job is just to solve this without touching the query.
0
 
LVL 22

Expert Comment

by:Kim Walker
ID: 40224328
You still didn't say how long. This sounds like server abuse.

And Ray's suggestion to index the tables is EXCELLENT if you haven't already done so.

Any negotiating regarding the restrictions to not touch the query?
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 40224357
This sounds like someone is setting you up for a failure by imposing a restriction on the one thing that would be most likely to succeed.  When queries run slowly there are only a very few things you can do to improve the response time.

1. Optimize the queries
2. Tune the MySQL configuration
3. Buy faster hardware
0
 

Accepted Solution

by:
Carbonecz earned 0 total points
ID: 40224365
Well, it's not MySQL, it's Oracle. My mistake.

It takes 20 minutes or more. Firefox can't handle it and timeouts before there is a response. The query is very complicated and there is a lot of data being handled. I'm not the right person to optimize it and it's already been done by people who know more than me.

My job is to make sure the browser doesn't timeout and there is a response when the script finishes.

I was thinking I'd just trigger the script using ajax and then wait for the results to be dumped into some intermediate file which I'd check continuosly using ajax.
0
 
LVL 34

Expert Comment

by:ste5an
ID: 40224385
Use an AJAX call to node.js..
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 40224477
I guess you could do that with AJAX, but this doesn't sound like it's really a suitable WWW application.  The general assumption of the internet community is that requests are atomic, complete and stateless and that responses are complete and usually instantaneous.  That's the way HTTP and RESTful applications like the WWW are expected to work.

How often is this query run?
0
 
LVL 11

Expert Comment

by:Murfur
ID: 40231267
A 20 minute query has to make you wonder what it is doing and my first thought is inefficiency - either the database structure, poorly structured queries and/or a ridiculous number of joins. How much data is a 20 minute query expected to return to the web page (potentially loads) and is it a practical amount? The crunch though is that after a user initiates the query it simply isn't realistic to expect them to wait 20 minutes for the result.

As Ray and others have already suggested, indexes and hardware are probably your primary targets.

AJAX would be a logical approach to remove the action from the browser page but I think there may be a better approach:

Have a cron job (or similar) run the lengthy query and update a "results" table every hour or so.
Then all your page has to do is query the results table - a considerably quicker response.
0
 

Author Comment

by:Carbonecz
ID: 40287049
I've requested that this question be closed as follows:

Accepted answer: 0 points for Carbonecz's comment #a40224365

for the following reason:

Well, it worked.
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 40287050
Instead of deleting the question, please post a comment showing how you solved the problem and accept your comment as the solution.  That way we have something useful for the community to learn from.  Thanks!
0
 

Author Comment

by:Carbonecz
ID: 40287061
This was my solution:

"I was thinking I'd just trigger the script using ajax and then wait for the results to be dumped into some intermediate file which I'd check continuosly using ajax."
0

Featured Post

MS Dynamics Made Instantly Simpler

Make Your Microsoft Dynamics Investment Count  & Drastically Decrease Training Time by Providing Intuitive Step-By-Step WalkThru Tutorials.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Using SQL Scripts we can save all the SQL queries as files that we use very frequently on our database later point of time. This is one of the feature present under SQL Workshop in Oracle Application Express.
This article discusses four methods for overlaying images in a container on a web page
This videos aims to give the viewer a basic demonstration of how a user can query current session information by using the SYS_CONTEXT function
The viewer will learn how to create and use a small PHP class to apply a watermark to an image. This video shows the viewer the setup for the PHP watermark as well as important coding language. Continue to Part 2 to learn the core code used in creat…

739 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question