[Last Call] Learn how to a build a cloud-first strategyRegister Now

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 532
  • Last Modified:

MySQL Query with Inner Join to small table suddenly takes FOREVER

Below are 2 queries.  They're exactly the same except for the inner join to the t_lob table (which has 14 rows in it and an index on account and skill).  The query with the join used to run in a matter of seconds before I upgraded to MySQL v5.6 (from 5.5).  Now it crawls.  

Why? What can I do to fix it?  

Here are the explain results:

Query with Join:
id,select_type,table,type,possible_keys,key,key_len,ref,rows,Extra
1,PRIMARY,<derived2>,ALL,NULL,NULL,NULL,NULL,6066,"Using temporary; Using filesort"
2,DERIVED,l,ref,"Account,Skill",Account,11,const,6,"Using index condition; Using where"
2,DERIVED,t,ref,"StartDate,Skill,Account",Skill,103,att.l.Skill,1011,"Using where"


Query with Join Commented out:
id,select_type,table,type,possible_keys,key,key_len,ref,rows,Extra
1,PRIMARY,<derived2>,ALL,NULL,NULL,NULL,NULL,445155,"Using temporary; Using filesort"
2,DERIVED,t,range,"StartDate,Account",StartDate,4,NULL,593540,"Using index condition; Using where"



.....So, it seems to me that the inclusion of the inner join causes fewer rows to be evaluated.  However, as I've explained it actually takes longer. Not even sure if it will complete.  I've let it go for about 30 min before killing it.  The version without the join takes 10.8 seconds

To clarify.  t_transcripts has about 10M rows in it.  t_lob has 14.


select distinct
#lob,
URL_Cleaned, count(URL_Cleaned) as URL_Count, sum(convcounter) as Conversions

from (select  
t.account,  
#l.lob,
#l.sortseq,
t.sessionid,
t.skill,  
t.startdate,  
t.chat_button_name,
t.chatreferer as URL,
substring_index(substring_index(substring_index(t.chatreferer, '://',-1),'#', 1), '?',1) as URL_Cleaned,
if(isnull(conversion), 0, 1) as convcounter

from  
t_transcripts t  

#inner join t_lob l
#on t.account = l.account
#and t.skill = l.skill  

where  
t.account = '12345678'
and t.startdate between '2014-02-12' and '2014-02-28'
)q  

group by
#lob,
url_cleaned



-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SLOW QUERY (inner join is not commented out):




select distinct
#lob,
URL_Cleaned, count(URL_Cleaned) as URL_Count, sum(convcounter) as Conversions

from (select  
t.account,  
#l.lob,
#l.sortseq,
t.sessionid,
t.skill,  
t.startdate,  
t.chat_button_name,
t.chatreferer as URL,
substring_index(substring_index(substring_index(t.chatreferer, '://',-1),'#', 1), '?',1) as URL_Cleaned,
if(isnull(conversion), 0, 1) as convcounter

from  
t_transcripts t  

inner join t_lob l
on t.account = l.account
and t.skill = l.skill  

where  
t.account = '12345678'
and t.startdate between '2014-02-12' and '2014-02-28'
)q  

group by
#lob,
url_cleaned
0
ducky801
Asked:
ducky801
  • 4
  • 3
1 Solution
 
PortletPaulCommented:
quick comment

there is absolutely no point in asking for distinct & group by...
drop use of distinct in both of your queries

select distinct (i.e. make a unique list of) X
...
group by (i.e. make a unique list of) X

i.e. you making a unique list twice
0
 
PortletPaulCommented:
do both tables have indexes on account & skill?
0
 
ducky801Author Commented:
Yea.  Muscle memory on the distinct.  It flows from the fingertips.....

Yes.  Both tables have and index on Account and Skill.
0
Get your Disaster Recovery as a Service basics

Disaster Recovery as a Service is one go-to solution that revolutionizes DR planning. Implementing DRaaS could be an efficient process, easily accessible to non-DR experts. Learn about monitoring, testing, executing failovers and failbacks to ensure a "healthy" DR environment.

 
ducky801Author Commented:
One other thing to note-

I'm also having this problem:
http://www.experts-exchange.com/OS/Microsoft_Operating_Systems/Windows/Q_28382342.html

I read somewhere today that long queries like the one i'm wresting with could be related to running out of space in your tmpdir.  Not sure if these could be related or how to go about testing that though.
0
 
PortletPaulCommented:
>> "distinct.  It flows from the fingertips"
it most definitely should not flow, it should only ever be used when it is absolutely needed

I haven't more specific ideas on the actual question I'm afraid.  Are your statistics current? Do you need to analyse those tables perhaps?
see:
https://dev.mysql.com/doc/refman/5.6/en/innodb-performance.html#innodb-persistent-stats
(assuming innob)
0
 
ducky801Author Commented:
I was able to get the query to run 180x faster by replacing the inner join with a straight_join.

Straight_join overrides the query optimizer when it makes a poor decision.

http://dev.mysql.com/doc/refman/5.0/en/join.html
0
 
ducky801Author Commented:
I found this answer on a separate message board.  Other expert comments on this post didn't help the query to run faster
0

Featured Post

Granular recovery for Microsoft Exchange

With Veeam Explorer for Microsoft Exchange you can choose the Exchange Servers and restore points you’re interested in, and Veeam Explorer will present the contents of those mailbox stores for browsing, searching and exporting.

  • 4
  • 3
Tackle projects and never again get stuck behind a technical roadblock.
Join Now