Link to home
Start Free TrialLog in
Avatar of Avinash Singh
Avinash Singh

asked on

if a row is completely same with any other row in a sheet then delete that complete row

Plz look into my actual file
if a row is completely same with any other row in  a sheet then delete that complete row
i am attaching my actual file plz look into it
i have 150000 rows , in the attachment i have puuted half data
i have to do this by vba only
Book1.xlsb
Avatar of Roy Cox
Roy Cox
Flag of United Kingdom of Great Britain and Northern Ireland image

The only way you could do this would be to set up a helper column that concatenated the cells of each row, then use Excel's Remove Duplicates feature.

This would be time consuming because you would need to set up the helper formula manually. Are you using Office 365 because if so you can use TEXTJOIN to concatenate a range of cells.

I would say thought that the amount of data in your file it would be impossible to do. The file took at least two minutes to download.
Avatar of Avinash Singh
Avinash Singh

ASKER

Roy Sir the code which i have takes 1hour 14min to process
It is taking lot of time
I have 150000 rows and it is taking 1 hour 14 min
That code is very slow
It's my request u plz make a code and i am 100% sure that max to max it will take 30 min
Excel VBA cannot cope with that amount of data, it's not just the number of rows, but the number of columns and so the number of cells to check.
You should use the new column, mentioned by Roy, to store a hash of the concatenated string values.  That way, you only need to compare fixed (and relatively short) string values when looking for duplicates.
Thnx both of u so plz guide me what i have to do now
I am using excel 2013 Roy sir
I tried add in a column and formula, the amount of data caused Excel to crash
VBA hashing code examples using .Net framework:
https://en.wikibooks.org/wiki/Visual_Basic_for_Applications/String_Hashing_in_VBA

In your case, you can use the StrConv() function to convert the concatenated cell values to a byte array.  This byte array will be accepted as a valid parameter by the .Net framework hashing functions.

Note: There are purely VBA implementations of these hashing functions if you have a problem using the .Net framework.
https://www.experts-exchange.com/viewCodeSnippet.jsp?codeSnippetId=20-42562560-1

I am runing this code in this actual file, Plz have a look
And any modification is required then plz do it to make this code fast  plz have a look
Sub DelDup()
Dim Ws As Worksheet
Dim LRow As Long
Dim LCol As Integer
Set Ws = Worksheets("Sheet1")
LRow = Ws.UsedRange.Rows.Count
LCol = Ws.Cells.Find(What:="*", After:=Ws.Range("A1"), SearchOrder:=xlByColumns, SearchDirection:=xlPrevious).Column + 1
Application.ScreenUpdating = False
With Ws.Range("A1:A" & LRow)
    .AdvancedFilter xlFilterInPlace, Unique:=True
    .SpecialCells(xlCellTypeVisible).Offset(0, LCol - 1).Value = 1
    On Error Resume Next
    Ws.ShowAllData
    Ws.Columns(LCol).SpecialCells(xlCellTypeBlanks).EntireRow.Delete
    Err.Clear
End With
Ws.Columns(LCol).Clear
Application.ScreenUpdating = True
End Sub
Sorry this code is also creating an error
I did try the following:
- Browse the workbook row by row concatenating each cells into a single string.
- Turn the string into a MD5.
- Add the MD5 string to a collection.
- Write the MD5 collection in a  temporary workbook, 1st column will be the collection's element index, 2nd column will be the MD5 string.
- Query the temporary workbook to obtain each unique MD5 string and its corresponding index.
- For each index in the query, copy the corresponding row from the workbook and paste it to a 2nd worksheet.

But I ran to memory troubles when copy/pasting (some rows have over 1500 cells >< )
Plus, the amount of data prevent from using any optimisation technic.

I'm afraid VBA isn't the right tool to achieve what you're looking for.
No problem leave this  if we cannot do this then we will try conditional copy paste
Thnx all of u for giving ur precious time and great support to this post
Thnx alot for giving a try to this problem
This question needs an answer!
Become an EE member today
7 DAY FREE TRIAL
Members can start a 7-Day Free trial then enjoy unlimited access to the platform.
View membership options
or
Learn why we charge membership fees
We get it - no one likes a content blocker. Take one extra minute and find out why we block content.