Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people, just like you, are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
Solved

MySQL Query Performance

Posted on 2014-03-10
1
386 Views
Last Modified: 2014-03-13
Hello,

I have a problem with one of my MySQL queries. The query takes about 2 seconds to return a result, where I would expect it to execute in about 100ms or around.

The table `routes` contains about 10 million entries.

Here is the Query, and the Explain Select output:

mysql> SELECT * FROM routes WHERE '8801856812240' RLIKE pattern AND status = 1 AND pricelist_id = '87' And `start_date` < NOW() ORDER BY `start_date` DESC,LENGTH(pattern) DESC LIMIT 1 ;
+----------+----------+------------------------------------+--------------+-----------------+--------------+--------------+------+-------------+------------+--------+---------------------+-------------+
| id       | pattern  | comment                            | connectcost  | includedseconds | cost         | pricelist_id | inc  | reseller_id | precedence | status | start_date          | import_flag |
+----------+----------+------------------------------------+--------------+-----------------+--------------+--------------+------+-------------+------------+--------+---------------------+-------------+
| 13733365 | ^88018.* | Bangladesh  Other Carriers  Mobile | 0.0000000000 |               0 | 0.0260000000 |           87 |    1 |         242 |          0 |      1 | 2014-03-05 12:45:00 |           1 |
+----------+----------+------------------------------------+--------------+-----------------+--------------+--------------+------+-------------+------------+--------+---------------------+-------------+
1 row in set (2.49 sec)

mysql> EXPLAIN SELECT * FROM routes WHERE '8801856812240' RLIKE pattern AND status = 1 AND pricelist_id = '87' And `start_date` < NOW() ORDER BY `start_date` DESC,LENGTH(pattern) DESC LIMIT 1;
+----+-------------+--------+-------------+------------------+------------------+---------+------+-------+----------------------------------------------------------------+
| id | select_type | table  | type        | possible_keys    | key              | key_len | ref  | rows  | Extra                                                          |
+----+-------------+--------+-------------+------------------+------------------+---------+------+-------+----------------------------------------------------------------+
|  1 | SIMPLE      | routes | index_merge | pricelist,status | pricelist,status | 4,1     | NULL | 38940 | Using intersect(pricelist,status); Using where; Using filesort |
+----+-------------+--------+-------------+------------------+------------------+---------+------+-------+----------------------------------------------------------------+
1 row in set (0.02 sec)

mysql> 

Open in new window


Here is the Create Table statement, and the Indexes for the Table:

CREATE TABLE `routes` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `pattern` varchar(40) NOT NULL,
  `comment` char(80) DEFAULT '',
  `connectcost` decimal(20,10) NOT NULL DEFAULT '0.0000000000',
  `includedseconds` int(4) NOT NULL,
  `cost` decimal(20,10) NOT NULL DEFAULT '0.0000000000',
  `pricelist_id` int(4) NOT NULL DEFAULT '0',
  `inc` int(4) DEFAULT NULL,
  `reseller_id` int(11) DEFAULT '0',
  `precedence` int(4) NOT NULL DEFAULT '0',
  `status` tinyint(1) NOT NULL DEFAULT '1',
  `start_date` datetime NOT NULL,
  `import_flag` tinyint(4) NOT NULL DEFAULT '1' COMMENT '0 Means delete this records 1 for active records.',
  PRIMARY KEY (`id`),
  KEY `pattern` (`pattern`),
  KEY `pricelist` (`pricelist_id`),
  KEY `reseller` (`reseller_id`),
  KEY `status` (`status`)
) ENGINE=InnoDB AUTO_INCREMENT=14203084 DEFAULT CHARSET=utf8 


mysql> show index from routes;
+--------+------------+-----------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+
| Table  | Non_unique | Key_name  | Seq_in_index | Column_name  | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+--------+------------+-----------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+
| routes |          0 | PRIMARY   |            1 | id           | A         |     9141225 |     NULL | NULL   |      | BTREE      |         |
| routes |          1 | pattern   |            1 | pattern      | A         |       79488 |     NULL | NULL   |      | BTREE      |         |
| routes |          1 | pricelist |            1 | pricelist_id | A         |        1952 |     NULL | NULL   |      | BTREE      |         |
| routes |          1 | reseller  |            1 | reseller_id  | A         |          18 |     NULL | NULL   | YES  | BTREE      |         |
| routes |          1 | status    |            1 | status       | A         |          18 |     NULL | NULL   |      | BTREE      |         |
+--------+------------+-----------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+
5 rows in set (0.14 sec)

Open in new window


And finally, here is some example data.

mysql> select * from routes limit 10;
+--------+------------+----------------------------+--------------+-----------------+--------------+--------------+------+-------------+------------+--------+---------------------+-------------+
| id     | pattern    | comment                    | connectcost  | includedseconds | cost         | pricelist_id | inc  | reseller_id | precedence | status | start_date          | import_flag |
+--------+------------+----------------------------+--------------+-----------------+--------------+--------------+------+-------------+------------+--------+---------------------+-------------+
| 440145 | ^1.*       | USA Proper                 | 0.0000000000 |               0 | 0.0071000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440146 | ^1201555.* | USA Proper                 | 0.0000000000 |               0 | 0.0071000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440147 | ^1202555.* | USA Proper                 | 0.0000000000 |               0 | 0.1716000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440148 | ^1203555.* | USA Proper                 | 0.0000000000 |               0 | 0.0071000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440149 | ^1204.*    | Canada 204-Manitoba Proper | 0.0000000000 |               0 | 0.0029000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440150 | ^1205555.* | USA Proper                 | 0.0000000000 |               0 | 0.0129000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440151 | ^1206555.* | USA Proper                 | 0.0000000000 |               0 | 0.0071000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440152 | ^1207555.* | USA Proper                 | 0.0000000000 |               0 | 0.0071000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440153 | ^1208555.* | USA Proper                 | 0.0000000000 |               0 | 0.1305000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
| 440154 | ^1209555.* | USA Proper                 | 0.0000000000 |               0 | 0.0071000000 |           22 |    1 |           0 |          0 |      1 | 2013-10-31 18:05:00 |           0 |
+--------+------------+----------------------------+--------------+-----------------+--------------+--------------+------+-------------+------------+--------+---------------------+-------------+
10 rows in set (0.00 sec)

Open in new window


The question is, what can I do, to make this query execute faster and give faster output. I'm sure the query could be tuned, and probably some indexes can be tuned as well.
0
Comment
Question by:shaf81
1 Comment
 
LVL 143

Accepted Solution

by:
Guy Hengel [angelIII / a3] earned 500 total points
ID: 39917190
>Using intersect(pricelist,status); Using where; Using filesort
shows clearly it can be optimized ...
ok, let me analyze the query
SELECT * FROM routes 
WHERE '8801856812240' RLIKE pattern 
AND status = 1 
AND pricelist_id = '87' 
And `start_date` < NOW() 
ORDER BY `start_date` DESC,LENGTH(pattern) DESC LIMIT 1; 

Open in new window


with `pricelist_id` int(4) NOT NULL DEFAULT '0', > you should not put quotes:
AND pricelist_id = '87'
hence:
AND pricelist_id = 87


next, you have indexes on single fields on your table.
given your query, you should create a index on several fields, and specifically in this orders:
pricelist_id + status  + start_date + pattern

note that:
WHERE '8801856812240' RLIKE pattern
cannot be really optimized in terms of normal indexes, the reason I put it in the index is to make the index a "covering" index

I hope this helps already a bit.
the next step would be to analyze the "pattern" part.
if all your patterns are to start with ^, you could add this condition

AND pattern LIKE '^8%'

with the 8 being the first digit from the value you search for.
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Foreword In the years since this article was written, numerous hacking attacks have targeted password-protected web sites.  The storage of client passwords has become a subject of much discussion, some of it useful and some of it misguided.  Of cou…
As a database administrator, you may need to audit your table(s) to determine whether the data types are optimal for your real-world data needs.  This Article is intended to be a resource for such a task. Preface The other day, I was involved …

808 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question