Solved

Problems with large MySQL table using ORDER BY and LIMIT

Posted on 2010-09-23
3
560 Views
Last Modified: 2013-12-16
I have a MySQL table that is pretty huge. There are 21 columns and (at this point) about 3 million rows. The interface I'm writing in C# to display data from this table will obviously need the ability to break results into pages. The way I was going to do this was using LIMIT. All results would be sorted by date. So the query for the first page would look like this:

   SELECT * FROM BigTable ORDER BY ItemDate ASC LIMIT 0,250;

This takes forever though, because before it applies the limit it goes through and orders the entire table by the date field. So I'm not really sure how to speed this up and make it work efficiently. Any recommendations?
0
Comment
Question by:elorc
  • 2
3 Comments
 
LVL 45

Accepted Solution

by:
Kdo earned 500 total points
ID: 33746540
Hi elorc,

There are a couple of ways.  :)

Is there an Identity column on the table?  That's an easy solution.

Is the ItemDate column indexed?  Assuming so, try the code below.


Kent

SELECT *

FROM

(

  SELECT ItemDate 

  FROM BigTable 

  ORDER BY ItemDate ASC 

  LIMIT 0,250

) t0

LEFT JOIN BigTable

  ON t0.ItemDate = BigTable.ItemDate;

Open in new window

0
 
LVL 1

Author Closing Comment

by:elorc
ID: 33747852
That makes a ridiculous difference. Thank you.
0
 
LVL 45

Expert Comment

by:Kdo
ID: 33747938
Hi elorc,

Just so you know how this works, MySQL recognizes that there is an index on the ItemDate column and grabs the first 250 dates from the index instead of reading the table.

It then joins only those 250 rows from the table.



Kent
0

Featured Post

Control application downtime with dependency maps

Visualize the interdependencies between application components better with Applications Manager's automated application discovery and dependency mapping feature. Resolve performance issues faster by quickly isolating problematic components.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Wouldn’t it be nice if you could test whether an element is contained in an array by using a Contains method just like the one available on List objects? Wouldn’t it be good if you could write code like this? (CODE) In .NET 3.5, this is possible…
Calculating holidays and working days is a function that is often needed yet it is not one found within the Framework. This article presents one approach to building a working-day calculator for use in .NET.
Here's a very brief overview of the methods PRTG Network Monitor (https://www.paessler.com/prtg) offers for monitoring bandwidth, to help you decide which methods you´d like to investigate in more detail.  The methods are covered in more detail in o…
Concerto provides fully managed cloud services and the expertise to provide an easy and reliable route to the cloud. Our best-in-class solutions help you address the toughest IT challenges, find new efficiencies and deliver the best application expe…

930 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now