Solved

In MySQL I need to delete some duplicates but leave more than 1

Posted on 2014-10-20
3
132 Views
Last Modified: 2014-10-21
I have a table with a bunch of duplicate records.  This is by design.  A sample may look like:

ID     Date                  location
-----------------------------------------
1      2014-10-20       1
2      2014-10-20       1
3      2014-10-20       1
4      2014-10-20       1
5      2014-10-20       1
6      2014-10-20       1
7      2014-10-20       2
8      2014-10-20       2
9      2014-10-20       2
10    2014-10-20       2
11    2014-10-20       2
12    2014-10-20       2
13      2014-10-21       1
14      2014-10-21       1
15      2014-10-21       1
16      2014-10-21       1
17      2014-10-21       1
18      2014-10-21       1
19      2014-10-21       2
20      2014-10-21       2
21      2014-10-21       2
22      2014-10-21       2
23      2014-10-21       2
24      2014-10-21       2

I want to remove only a certain number of duplicates.  For instance, each location has 6 duplicate dates (6 records for location 1 on 2014-10-20).

I would like to remove just TWO of the duplicate records from each date-location grouping leaving me with 4 records at location 1 on 2014-10-20.  I need to do this for several hundred such groupings.

The ID is primary and unique.

Any help?
0
Comment
Question by:pmascari
  • 2
3 Comments
 
LVL 51

Expert Comment

by:Julian Hansen
ID: 40393841
Can you not  do something like this
CREATE TEMPORARY TABLE deleteids (id INT);
INSERT INTO deleteids (id) SELECT MAX(id) FROM t1 GROUP BY DATE;
DELETE FROM t1 WHERE id IN (SELECT id FROM deleteids);

Open in new window

And then just run the INSERT and DELETE query twice?
0
 
LVL 8

Author Comment

by:pmascari
ID: 40394452
Thanks for the comment.  Your suggestion could work if there were always 6 duplicates.  I've found that these groups can have up to 12 duplicates.  How can I delete a variable number of records, leaving me with groups of 4 for each date/location?
0
 
LVL 51

Accepted Solution

by:
Julian Hansen earned 500 total points
ID: 40394518
Then something like this
SET @ROW := 0, @DATE := '';
DELETE FROM t1 WHERE id IN  (
SELECT id FROM (
  SELECT id, `date`,
      @ROW := IF(@DATE = `date`, @ROW + 1, 1) AS row_number,
      @DATE := `date` AS dummy
  FROM t1
  ORDER BY `date`
) subq
WHERE subq.row_number > 4
);

Open in new window

0

Featured Post

Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

Join & Write a Comment

Fore-Foreword Today (2016) Maxmind has a new approach to the distribution of its data sets.  This article may be obsolete.  Instead of using the examples here, have a look at the MaxMind API (https://www.maxmind.com/en/geolite2-developer-package). …
Introduction In this installment of my SQL tidbits, I will be looking at parsing Extensible Markup Language (XML) directly passed as string parameters to MySQL 5.1.5 or higher. These would be instances where LOAD_FILE (http://dev.mysql.com/doc/refm…
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.
When you create an app prototype with Adobe XD, you can insert system screens -- sharing or Control Center, for example -- with just a few clicks. This video shows you how. You can take the full course on Experts Exchange at http://bit.ly/XDcourse.

706 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now