Solved

SQL to delete all but last 500 records per customer

Posted on 2013-06-19
11
383 Views
Last Modified: 2013-06-19
I have a growing table of records in a table that I would like to cull. The relevant fields are:
MessageDate
CellNo

What I want to do is delete all but the last 500 records for each CellNo. I know how to define the TOP 500 in an SQL statement but how would I define all but the TOP 500?
0
Comment
Question by:Rob4077
  • 5
  • 4
  • 2
11 Comments
 
LVL 11

Expert Comment

by:David Kroll
ID: 39259397
DELETE FROM TABLENAME
WHERE
MESSAGEDATE = ??
AND CELLNO = ??
AND ID NOT IN
(SELECT TOP 500 ID
 FROM TABLENAME
 WHERE MESSAGEDATE = ?? AND CELLNO = ??)
0
 

Author Comment

by:Rob4077
ID: 39259438
Thanks. How would I modify that to delete all but the last 500 records for EVERY CellNo regardless of date bearing in mind that not all CellNo have 500 records?
0
 
LVL 11

Expert Comment

by:David Kroll
ID: 39259449
DELETE FROM TABLENAME
WHERE
 ID NOT IN
(SELECT TOP 500 ID
 FROM TABLENAME T2
 WHERE T2.CELLNO = TABLENAME.CELLNO)

If there are not 500 records, then nothing would be deleted for a CellNo.
0
 
LVL 48

Expert Comment

by:PortletPaul
ID: 39259461
TOP n is influence by the order applied

so, in simple terms apply an order where the unwanted records occur first and use TOP n%

To achieve a hard number that is retained (500)  I suggest you need to calculate the number of rows (total) then figure out what 500 is of that number in percentage terms, and then delete the percentage that unwanted via TOP n%
something like

@totrows = select count(*) from thatTable
@nPercent = (500*100)/@totrows
@delPercent = 100 - @nPercent

then use "TOP @delPercent%" (which may need to be dynamic SQL ?, needs confirming)

I'm not sure if order by applies to delete (as I never tried it), but assuming it doesn't, to ensure you get the right records might require selecting IDs from the table-to-cull into a temp table so that order by does apply.

Then delete from the table-to-cull
joined to the temp table (which has "TOP @delPercent%" IDs in it)

then drop the temp table

Sorry if this is a bit cryptic - as I've not actually ever tried what you have asked for - but I think the above might be a workable (if not well explained) approach.

btw: I am assuming approximately 500 would be acceptable

{+edit, now I read what I just submitted maybe percent isn't needed and absolute numbers used instead - except for that, the same method would apply}
0
 
LVL 48

Expert Comment

by:PortletPaul
ID: 39259473
or, use something sensible like the one above mine.
what was I thinking?
0
Comprehensive Backup Solutions for Microsoft

Acronis protects the complete Microsoft technology stack: Windows Server, Windows PC, laptop and Surface data; Microsoft business applications; Microsoft Hyper-V; Azure VMs; Microsoft Windows Server 2016; Microsoft Exchange 2016 and SQL Server 2016.

 

Author Comment

by:Rob4077
ID: 39259513
Just to clarify.

The table contains thousands of records. One field is CellNo and another is MessageDate. Of course there are a number of other fields. Currently messages come in and are added to the table as they arrive. In some cases I now have over 2000 records with the same CellNo. In other cases I only have 2 or 3 records with the same CellNo. What I want to do is only keep the most recent 500 records for each CellNo.

I know how to do it by creating a recordset and looping through the whole table but was wondering if there was an easy way of doing it with SQL. If there's not please let me know and I will write the VBA to loop through the recordset.
0
 

Author Comment

by:Rob4077
ID: 39259523
Looking at the answers provided, I think looping through the recordset is going to be a much quicker, simpler and more elegant solution - do you agree?
0
 

Author Comment

by:Rob4077
ID: 39259582
I had something like the following in mind (untested). I'm starting to think this may be better than trying to figure out some SQL code that will do it

sSql = "Select queue.* from Queue orderby CellNo, MessageDate DESC"
set rs = currentdb.opendynaset(sSql)

LastCellNo = rs!CellNo
lRecCount = 0
Do until rs.eof
   if rs!CellNo <> LastCellNo then lRecCount = 0
   lRecCount = lRecount + 1
   if lRecCount > 500 then
      rs.delete
   else
      rs.movenext
   Endif
loop
0
 
LVL 48

Accepted Solution

by:
PortletPaul earned 500 total points
ID: 39261054
>>The table contains thousands of records.
then perhaps my rambling reply isn't so awful

A looping construct may be appropriate, although you may want to consider 'batches' rather than just a single loop that attempts to do all deletes in one operation as you also have to consider locking and transaction log impacts as well.

Take a look at this answered question for example. It's a different need (2 tables are involved) but the concept of doing the deletion in batches may be useful.

and: I would consider using a temporary table that holds the IDs of the records I want to delete which will allow you to use EXISTS in the deletion query plus it can be re-used for each batch until all is done. This table can be dropped once all the culling has been completed.
0
 

Author Comment

by:Rob4077
ID: 39261150
Thanks.. That all makes sense. Appreciate your help
0
 
LVL 48

Expert Comment

by:PortletPaul
ID: 39261169
and thank you! Cheers, Paul
0

Featured Post

3 Use Cases for Connected Systems

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, testing some more, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

When you are entering numbers in a speadsheet, and don't remember what 6×7 is, you just type “=6*7" instead. It works in every cell! This is not so in Access. To enter the elusive 42 in a text box, you have to find a calculator, and then copy the re…
Introduction The Visual Basic for Applications (VBA) language is at the heart of every application that you write. It is your key to taking Access beyond the world of wizards into a world where anything is possible. This article introduces you to…
Learn how to number pages in an Access report over each group. Activate two pass printing by referencing the pages property: Add code to the Page Footers OnFormat event to capture the pages as there occur for each group. Use the pages property to …
In Microsoft Access, learn how to “cascade” or have the displayed data of one combo control depend upon what’s entered in another. Base the dependent combo on a query for its row source: Add a reference to the first combo on the form as criteria i…

863 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

23 Experts available now in Live!

Get 1:1 Help Now