Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

slow delete query

Posted on 2009-03-31
11
Medium Priority
?
408 Views
Last Modified: 2012-05-06
Hey,

I'm finding this query takes about 50 minutes to run. Is there a way to optimize?

delete from A
where A.createdate IN (select B.createdate from B group by B.createdate);

I need to run this query daily. Basically, B is an daily set of rows that update Table A. So I remove any rows where the date already exists in B, then add everything in B to A. Table A has 7 million rows, but is growing at 400K rows or so a day. Table B has 2 million rows.

In both cases, "createdate" is a DATE field.
0
Comment
Question by:deckard666
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 3
  • 2
  • +2
11 Comments
 
LVL 29

Expert Comment

by:QPR
ID: 24026510
is there an index on b.createdate?
Why do you need to use group by in your sub select?
0
 
LVL 29

Expert Comment

by:QPR
ID: 24026514
I meant A.createdate.
Also is there any delete triggers on A?
0
 
LVL 5

Expert Comment

by:allmer
ID: 24026517
Try:

EXPLAIN
delete from A
where A.createdate IN (select B.createdate from B group by B.createdate);
AND or
EXPLAIN
select B.createdate from B group by B.createdate;

You may want to add indexes such that your SELECT query and or your where .. in
gets faster.
0
NEW Veeam Agent for Microsoft Windows

Backup and recover physical and cloud-based servers and workstations, as well as endpoint devices that belong to remote users. Avoid downtime and data loss quickly and easily for Windows-based physical or public cloud-based workloads!

 

Author Comment

by:deckard666
ID: 24026664
- I have indexes on both tables for createdate.
- No triggers
- Group by is to get distinct createdates (there are multiple rows with the same date).

Note, when I run this query without the select subquery, it runs really fast.

delete from A
where A.createdate = cast("2009-03-31 AS Date)
or A.createdate = cast("2009-03-30 AS Date) or ....
0
 
LVL 5

Expert Comment

by:allmer
ID: 24026698
Did you try:
delete from A
where A.createdate IN (select DISTINCT B.createdate from B);
Does that speed up your query?
0
 
LVL 5

Expert Comment

by:allmer
ID: 24026705
Did you use explain to find out whether your indexes are actually used in your query?
0
 
LVL 1

Expert Comment

by:hc2342uhxx3vw36x96hq
ID: 24026741
Try the attached code.
DELETE FROM a
      WHERE EXISTS (SELECT 'X'
                      FROM b
                     WHERE a.createdate = b.createdate AND ROWNUM = 1);

Open in new window

0
 

Author Comment

by:deckard666
ID: 24026992
ROWNUM is not a feature available in MySQL 5.0. I get an error on it.
Running the distinct statement now and i'll post the explain when its done.
0
 
LVL 14

Expert Comment

by:racek
ID: 24027069
DELETE FROM A
      WHERE EXISTS (SELECT NULL
                      FROM B
                     WHERE A.createdate = B.createdate );
0
 
LVL 5

Accepted Solution

by:
allmer earned 2000 total points
ID: 24027204
Both of the below should give you only distinct rows from B and delete them from a are there any differences in performance?

I still wonder if you checked EXPLAIN (don't worry you don't have to wait 50min for that).

Also you could try to tweak MySQL (give it more memory for certain operations)


delete from A 
where A.createdate IN (
select B.createdate from B 
UNION
select B.createdate from B LIMIT 1
);
 
delete from A 
where A.createdate IN (select DISTINCT B.createdate from B);

Open in new window

0
 

Author Closing Comment

by:deckard666
ID: 31564712
for some reason switching to distinct worked wonders
0

Featured Post

Get your Conversational Ransomware Defense e‑book

This e-book gives you an insight into the ransomware threat and reviews the fundamentals of top-notch ransomware preparedness and recovery. To help you protect yourself and your organization. The initial infection may be inevitable, so the best protection is to be fully prepared.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Does the idea of dealing with bits scare or confuse you? Does it seem like a waste of time in an age where we all have terabytes of storage? If so, you're missing out on one of the core tools in every professional programmer's toolbox. Learn how to …
In this series, we will discuss common questions received as a database Solutions Engineer at Percona. In this role, we speak with a wide array of MySQL and MongoDB users responsible for both extremely large and complex environments to smaller singl…
In this video, Percona Solution Engineer Rick Golba discuss how (and why) you implement high availability in a database environment. To discuss how Percona Consulting can help with your design and architecture needs for your database and infrastr…
In this video, Percona Solutions Engineer Barrett Chambers discusses some of the basic syntax differences between MySQL and MongoDB. To learn more check out our webinar on MongoDB administration for MySQL DBA: https://www.percona.com/resources/we…
Suggested Courses

721 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question