Trying to improve performance on a simple query

I have a simple SQL Query running out of a PHP app I inherited to a MySQL server.  There are 2 tables involved, and we are inserting into 1 table the values which are missing.  This query takes over 20 seconds to finish.  The table DataHistory has only 110000 rows in it, and DataNew has only 5-600 rows.  Both CodeType and Bonus are indexed.  What else can I do to improve the performance of this query? (we are occassionally experiencing timeout issues, and the low number of rows should give a much quicker response, shouldn't it?)


insert into
		DataNew (
		DataHistory.CollectorCode, 0
	left join
		DataHistory.CodeType = $DataCodeType and
		DataHistory.Bonus = 0 and
		DataNew.CollectorCode is null

Open in new window

LVL 10
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Are the join columns indexed?

I suspect selecting from only DataHistory will be fast:

      SELECT count (DISTINCT DataHistory.CollectorCode)
      from   DataHistory
      where  DataHistory.CodeType = $DataCodeType and
                   DataHistory.Bonus = 0

If so, then does adding DataNew slow it down?  If DataNew.CollectorCode isn't indexed it may be doing a full table scan of DataNew.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Create index on:

DataHistory.Bonus = 0
GeoffSuttonAuthor Commented:
I don't believe they are.  Datanew is a junk table used as a "Scratch pad" and only holds 500-600 rows.  Would it make that big a difference?
10 Tips to Protect Your Business from Ransomware

Did you know that ransomware is the most widespread, destructive malware in the world today? It accounts for 39% of all security breaches, with ransomware gangsters projected to make $11.5B in profits from online extortion by 2019.

GeoffSuttonAuthor Commented:
rpkhare -> All those indexes are already in place.
Milleniumaire-> Created new index on DataNew.CollectorCode.  Testing it now.
GeoffSuttonAuthor Commented:
I created the index on DataNew. CollectorCode and that dropped my query time from 25 or so seconds to somewhat less than 1 second.  I wouldn't have thought it necessary for such a low amount of rows.  Thank you so much for the suggestion.

It may/may not improve the performance, you would have to try it.

Alternatively, you could try re-writing the query to use a corrolated subquery rather than outer joining or to use the minus clause if this is available in MySQL.

insert into DataNew (CollectorCode,
      SELECT DISTINCT DataHistory.CollectorCode, 0
      from DataHistory
      where not exists (select null
                                             from DataNew
                                             where DataNew.CollectorCode=DataHistory.CollectorCode)
      and     DataHistory.CodeType = $DataCodeType
                and     DataHistory.Bonus = 0

insert into DataNew (CollectorCode,
               SELECT DISTINCT DataHistory.CollectorCode, 0
      from DataHistory
      where DataHistory.CodeType = $DataCodeType
                and     DataHistory.Bonus = 0
               SELECT CollectorCode,
                from DataNew      

GeoffSuttonAuthor Commented:
Indexing the other side of the join made a huge difference.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.