Improve company productivity with a Business Account.Sign Up

x
?
Solved

MySQL Query Performance

Posted on 2014-03-10
1
Medium Priority
?
408 Views
Last Modified: 2014-03-13
Hello,

I have a problem with one of my MySQL queries. The query takes about 2 seconds to return a result, where I would expect it to execute in about 100ms or around.

The table `routes` contains about 10 million entries.

Here is the Query, and the Explain Select output:

mysql> SELECT * FROM routes WHERE '8801856812240' RLIKE pattern AND status = 1 AND pricelist_id = '87' And `start_date` < NOW() ORDER BY `start_date` DESC,LENGTH(pattern) DESC LIMIT 1 ;
+----------+----------+------------------------------------+--------------+-----------------+--------------+--------------+------+-------------+------------+--------+---------------------+-------------+
| id       | pattern  | comment                            | connectcost  | includedseconds | cost         | pricelist_id | inc  | reseller_id | precedence | status | start_date          | import_flag |
+----------+----------+------------------------------------+--------------+-----------------+--------------+--------------+------+-------------+------------+--------+---------------------+-------------+
| 13733365 | ^88018.* | Bangladesh  Other Carriers  Mobile | 0.0000000000 |               0 | 0.0260000000 |           87 |    1 |         242 |          0 |      1 | 2014-03-05 12:45:00 |           1 |
+----------+----------+------------------------------------+--------------+-----------------+--------------+--------------+------+-------------+------------+--------+---------------------+-------------+
1 row in set (2.49 sec)

mysql> EXPLAIN SELECT * FROM routes WHERE '8801856812240' RLIKE pattern AND status = 1 AND pricelist_id = '87' And `start_date` < NOW() ORDER BY `start_date` DESC,LENGTH(pattern) DESC LIMIT 1;
+----+-------------+--------+-------------+------------------+------------------+---------+------+-------+----------------------------------------------------------------+
| id | select_type | table  | type        | possible_keys    | key              | key_len | ref  | rows  | Extra                                                          |
+----+-------------+--------+-------------+------------------+------------------+---------+------+-------+----------------------------------------------------------------+
|  1 | SIMPLE      | routes | index_merge | pricelist,status | pricelist,status | 4,1     | NULL | 38940 | Using intersect(pricelist,status); Using where; Using filesort |
+----+-------------+--------+-------------+------------------+------------------+---------+------+-------+----------------------------------------------------------------+
1 row in set (0.02 sec)

mysql> 

Open in new window


Here is the Create Table statement, and the Indexes for the Table:

CREATE TABLE `routes` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `pattern` varchar(40) NOT NULL,
  `comment` char(80) DEFAULT '',
  `connectcost` decimal(20,10) NOT NULL DEFAULT '0.0000000000',
  `includedseconds` int(4) NOT NULL,
  `cost` decimal(20,10) NOT NULL DEFAULT '0.0000000000',
  `pricelist_id` int(4) NOT NULL DEFAULT '0',
  `inc` int(4) DEFAULT NULL,
  `reseller_id` int(11) DEFAULT '0',
  `precedence` int(4) NOT NULL DEFAULT '0',
  `status` tinyint(1) NOT NULL DEFAULT '1',
  `start_date` datetime NOT NULL,
  `import_flag` tinyint(4) NOT NULL DEFAULT '1' COMMENT '0 Means delete this records 1 for active records.',
  PRIMARY KEY (`id`),
  KEY `pattern` (`pattern`),
  KEY `pricelist` (`pricelist_id`),
  KEY `reseller` (`reseller_id`),
  KEY `status` (`status`)
) ENGINE=InnoDB AUTO_INCREMENT=14203084 DEFAULT CHARSET=utf8 


mysql> show index from routes;
+--------+------------+-----------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+
| Table  | Non_unique | Key_name  | Seq_in_index | Column_name  | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+--------+------------+-----------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+
| routes |          0 | PRIMARY   |            1 | id           | A         |     9141225 |     NULL | NULL   |      | BTREE      |         |
| routes |          1 | pattern   |            1 | pattern      | A         |       79488 |     NULL | NULL   |      | BTREE      |         |
| routes |          1 | pricelist |            1 | pricelist_id | A         |        1952 |     NULL | NULL   |      | BTREE      |         |
| routes |          1 | reseller  |            1 | reseller_id  | A         |          18 |     NULL | NULL   | YES  | BTREE      |         |
| routes |          1 | status    |            1 | status       | A         |          18 |     NULL | NULL   |      | BTREE      |         |
+--------+------------+-----------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+
5 rows in set (0.14 sec)

Open in new window


And finally, here is some example data.

mysql> select * from routes limit 10;
+--------+------------+----------------------------+--------------+-----------------+--------------+--------------+------+-------------+------------+--------+---------------------+-------------+
| id     | pattern    | comment                    | connectcost  | includedseconds | cost         | pricelist_id | inc  | reseller_id | precedence | status | start_date          | import_flag |
+--------+------------+----------------------------+--------------+-----------------+--------------+--------------+------+-------------+------------+--------+---------------------+-------------+
| 440145 | ^1.*       | USA Proper                 | 0.0000000000 |               0 | 0.0071000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440146 | ^1201555.* | USA Proper                 | 0.0000000000 |               0 | 0.0071000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440147 | ^1202555.* | USA Proper                 | 0.0000000000 |               0 | 0.1716000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440148 | ^1203555.* | USA Proper                 | 0.0000000000 |               0 | 0.0071000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440149 | ^1204.*    | Canada 204-Manitoba Proper | 0.0000000000 |               0 | 0.0029000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440150 | ^1205555.* | USA Proper                 | 0.0000000000 |               0 | 0.0129000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440151 | ^1206555.* | USA Proper                 | 0.0000000000 |               0 | 0.0071000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440152 | ^1207555.* | USA Proper                 | 0.0000000000 |               0 | 0.0071000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440153 | ^1208555.* | USA Proper                 | 0.0000000000 |               0 | 0.1305000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440154 | ^1209555.* | USA Proper                 | 0.0000000000 |               0 | 0.0071000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
+--------+------------+----------------------------+--------------+-----------------+--------------+--------------+------+-------------+------------+--------+---------------------+-------------+
10 rows in set (0.00 sec)

Open in new window


The question is, what can I do, to make this query execute faster and give faster output. I'm sure the query could be tuned, and probably some indexes can be tuned as well.
0
Comment
Question by:shaf81
1 Comment
 
LVL 143

Accepted Solution

by:
Guy Hengel [angelIII / a3] earned 2000 total points
ID: 39917190
>Using intersect(pricelist,status); Using where; Using filesort
shows clearly it can be optimized ...
ok, let me analyze the query
SELECT * FROM routes 
WHERE '8801856812240' RLIKE pattern 
AND status = 1 
AND pricelist_id = '87' 
And `start_date` < NOW() 
ORDER BY `start_date` DESC,LENGTH(pattern) DESC LIMIT 1; 

Open in new window


with `pricelist_id` int(4) NOT NULL DEFAULT '0', > you should not put quotes:
AND pricelist_id = '87'
hence:
AND pricelist_id = 87


next, you have indexes on single fields on your table.
given your query, you should create a index on several fields, and specifically in this orders:
pricelist_id + status  + start_date + pattern

note that:
WHERE '8801856812240' RLIKE pattern
cannot be really optimized in terms of normal indexes, the reason I put it in the index is to make the index a "covering" index

I hope this helps already a bit.
the next step would be to analyze the "pattern" part.
if all your patterns are to start with ^, you could add this condition

AND pattern LIKE '^8%'

with the 8 being the first digit from the value you search for.
0

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

This month, Experts Exchange sat down with resident SQL expert, Jim Horn, for an in-depth look into the makings of a successful career in SQL.
Article by: Tammy
MySQLTuner is a script written in Perl that allows you to review a MySQL installation quickly and make adjustments to increase performance and stability. The current configuration variables and status data is retrieved and presented in a brief forma…
In this video, Percona Solution Engineer Rick Golba discuss how (and why) you implement high availability in a database environment. To discuss how Percona Consulting can help with your design and architecture needs for your database and infrastr…
In this video, Percona Solutions Engineer Barrett Chambers discusses some of the basic syntax differences between MySQL and MongoDB. To learn more check out our webinar on MongoDB administration for MySQL DBA: https://www.percona.com/resources/we…

607 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question