Doug
asked on
Delete Duplicates In SharePoint List
I have a list which has tens of thousands of records and I need them to be unique. I found a PowerShell script online that will work if I only have 1 unique column. However, I have to group by 2 columns and I can't figure out how to make it work.
To clarify, if I have this data in a SP list, only if both columns are duplicates should the items be removed.
Title Carrier
1 Carrier1
1 Carrier1
12 Carrier1
12 Carrier2
100 Carrier1
100 Carrier1
100 Carrier2
cls
if((Get-PSSnapin | Where {$_.Name -eq "Microsoft.SharePoint.PowerShell"}) -eq $null) {
Add-PSSnapin Microsoft.SharePoint.PowerShell;
}
$ListName = "DuplicateTest"
$web = Get-SPWeb -identity "http://MyVM/sites/TestSite"
$list = $web.Lists[$ListName]
$AllDuplicates = $list.Items.GetDataTable() | Group-Object Title | where {$_.count -gt 1}
$count = 1
$max = $AllDuplicates.Count
foreach($duplicate in $AllDuplicates)
{
$duplicate.group | Select-Object -Skip 1 | % {$list.Items.DeleteItemById($_.ID)}
Write-Progress -PercentComplete ($count / $max * 100) -Activity "$count duplicates removed" -Status "In Progress"
$count++
}
To clarify, if I have this data in a SP list, only if both columns are duplicates should the items be removed.
Title Carrier
1 Carrier1
1 Carrier1
12 Carrier1
12 Carrier2
100 Carrier1
100 Carrier1
100 Carrier2
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER