Solved

MySQL Query Performance

Posted on 2014-03-10
1
388 Views
Last Modified: 2014-03-13
Hello,

I have a problem with one of my MySQL queries. The query takes about 2 seconds to return a result, where I would expect it to execute in about 100ms or around.

The table `routes` contains about 10 million entries.

Here is the Query, and the Explain Select output:

mysql> SELECT * FROM routes WHERE '8801856812240' RLIKE pattern AND status = 1 AND pricelist_id = '87' And `start_date` < NOW() ORDER BY `start_date` DESC,LENGTH(pattern) DESC LIMIT 1 ;
+----------+----------+------------------------------------+--------------+-----------------+--------------+--------------+------+-------------+------------+--------+---------------------+-------------+
| id       | pattern  | comment                            | connectcost  | includedseconds | cost         | pricelist_id | inc  | reseller_id | precedence | status | start_date          | import_flag |
+----------+----------+------------------------------------+--------------+-----------------+--------------+--------------+------+-------------+------------+--------+---------------------+-------------+
| 13733365 | ^88018.* | Bangladesh  Other Carriers  Mobile | 0.0000000000 |               0 | 0.0260000000 |           87 |    1 |         242 |          0 |      1 | 2014-03-05 12:45:00 |           1 |
+----------+----------+------------------------------------+--------------+-----------------+--------------+--------------+------+-------------+------------+--------+---------------------+-------------+
1 row in set (2.49 sec)

mysql> EXPLAIN SELECT * FROM routes WHERE '8801856812240' RLIKE pattern AND status = 1 AND pricelist_id = '87' And `start_date` < NOW() ORDER BY `start_date` DESC,LENGTH(pattern) DESC LIMIT 1;
+----+-------------+--------+-------------+------------------+------------------+---------+------+-------+----------------------------------------------------------------+
| id | select_type | table  | type        | possible_keys    | key              | key_len | ref  | rows  | Extra                                                          |
+----+-------------+--------+-------------+------------------+------------------+---------+------+-------+----------------------------------------------------------------+
|  1 | SIMPLE      | routes | index_merge | pricelist,status | pricelist,status | 4,1     | NULL | 38940 | Using intersect(pricelist,status); Using where; Using filesort |
+----+-------------+--------+-------------+------------------+------------------+---------+------+-------+----------------------------------------------------------------+
1 row in set (0.02 sec)

mysql> 

Open in new window


Here is the Create Table statement, and the Indexes for the Table:

CREATE TABLE `routes` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `pattern` varchar(40) NOT NULL,
  `comment` char(80) DEFAULT '',
  `connectcost` decimal(20,10) NOT NULL DEFAULT '0.0000000000',
  `includedseconds` int(4) NOT NULL,
  `cost` decimal(20,10) NOT NULL DEFAULT '0.0000000000',
  `pricelist_id` int(4) NOT NULL DEFAULT '0',
  `inc` int(4) DEFAULT NULL,
  `reseller_id` int(11) DEFAULT '0',
  `precedence` int(4) NOT NULL DEFAULT '0',
  `status` tinyint(1) NOT NULL DEFAULT '1',
  `start_date` datetime NOT NULL,
  `import_flag` tinyint(4) NOT NULL DEFAULT '1' COMMENT '0 Means delete this records 1 for active records.',
  PRIMARY KEY (`id`),
  KEY `pattern` (`pattern`),
  KEY `pricelist` (`pricelist_id`),
  KEY `reseller` (`reseller_id`),
  KEY `status` (`status`)
) ENGINE=InnoDB AUTO_INCREMENT=14203084 DEFAULT CHARSET=utf8 


mysql> show index from routes;
+--------+------------+-----------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+
| Table  | Non_unique | Key_name  | Seq_in_index | Column_name  | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+--------+------------+-----------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+
| routes |          0 | PRIMARY   |            1 | id           | A         |     9141225 |     NULL | NULL   |      | BTREE      |         |
| routes |          1 | pattern   |            1 | pattern      | A         |       79488 |     NULL | NULL   |      | BTREE      |         |
| routes |          1 | pricelist |            1 | pricelist_id | A         |        1952 |     NULL | NULL   |      | BTREE      |         |
| routes |          1 | reseller  |            1 | reseller_id  | A         |          18 |     NULL | NULL   | YES  | BTREE      |         |
| routes |          1 | status    |            1 | status       | A         |          18 |     NULL | NULL   |      | BTREE      |         |
+--------+------------+-----------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+
5 rows in set (0.14 sec)

Open in new window


And finally, here is some example data.

mysql> select * from routes limit 10;
+--------+------------+----------------------------+--------------+-----------------+--------------+--------------+------+-------------+------------+--------+---------------------+-------------+
| id     | pattern    | comment                    | connectcost  | includedseconds | cost         | pricelist_id | inc  | reseller_id | precedence | status | start_date          | import_flag |
+--------+------------+----------------------------+--------------+-----------------+--------------+--------------+------+-------------+------------+--------+---------------------+-------------+
| 440145 | ^1.*       | USA Proper                 | 0.0000000000 |               0 | 0.0071000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440146 | ^1201555.* | USA Proper                 | 0.0000000000 |               0 | 0.0071000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440147 | ^1202555.* | USA Proper                 | 0.0000000000 |               0 | 0.1716000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440148 | ^1203555.* | USA Proper                 | 0.0000000000 |               0 | 0.0071000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440149 | ^1204.*    | Canada 204-Manitoba Proper | 0.0000000000 |               0 | 0.0029000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440150 | ^1205555.* | USA Proper                 | 0.0000000000 |               0 | 0.0129000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440151 | ^1206555.* | USA Proper                 | 0.0000000000 |               0 | 0.0071000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440152 | ^1207555.* | USA Proper                 | 0.0000000000 |               0 | 0.0071000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440153 | ^1208555.* | USA Proper                 | 0.0000000000 |               0 | 0.1305000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440154 | ^1209555.* | USA Proper                 | 0.0000000000 |               0 | 0.0071000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
+--------+------------+----------------------------+--------------+-----------------+--------------+--------------+------+-------------+------------+--------+---------------------+-------------+
10 rows in set (0.00 sec)

Open in new window


The question is, what can I do, to make this query execute faster and give faster output. I'm sure the query could be tuned, and probably some indexes can be tuned as well.
0
Comment
Question by:shaf81
1 Comment
 
LVL 143

Accepted Solution

by:
Guy Hengel [angelIII / a3] earned 500 total points
ID: 39917190
>Using intersect(pricelist,status); Using where; Using filesort
shows clearly it can be optimized ...
ok, let me analyze the query
SELECT * FROM routes 
WHERE '8801856812240' RLIKE pattern 
AND status = 1 
AND pricelist_id = '87' 
And `start_date` < NOW() 
ORDER BY `start_date` DESC,LENGTH(pattern) DESC LIMIT 1; 

Open in new window


with `pricelist_id` int(4) NOT NULL DEFAULT '0', > you should not put quotes:
AND pricelist_id = '87'
hence:
AND pricelist_id = 87


next, you have indexes on single fields on your table.
given your query, you should create a index on several fields, and specifically in this orders:
pricelist_id + status  + start_date + pattern

note that:
WHERE '8801856812240' RLIKE pattern
cannot be really optimized in terms of normal indexes, the reason I put it in the index is to make the index a "covering" index

I hope this helps already a bit.
the next step would be to analyze the "pattern" part.
if all your patterns are to start with ^, you could add this condition

AND pattern LIKE '^8%'

with the 8 being the first digit from the value you search for.
0

Featured Post

Ransomware: The New Cyber Threat & How to Stop It

This infographic explains ransomware, type of malware that blocks access to your files or your systems and holds them hostage until a ransom is paid. It also examines the different types of ransomware and explains what you can do to thwart this sinister online threat.  

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

'Between' is such a common word we rarely think about it but in SQL it has a very specific definition we should be aware of. While most database vendors will have their own unique phrases to describe it (see references at end) the concept in common …
Confronted with some SQL you don't know can be a daunting task. It can be even more daunting if that SQL carries some of the old secret codes used in the Ye Olde query syntax, such as: (+)     as used in Oracle;     *=     =*    as used in Sybase …
Finds all prime numbers in a range requested and places them in a public primes() array. I've demostrated a template size of 30 (2 * 3 * 5) but larger templates can be built such 210  (2 * 3 * 5 * 7) or 2310  (2 * 3 * 5 * 7 * 11). The larger templa…

749 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question