fireburn11
asked on
powershell to find duplicate records in a text file and give me a sum of each duplicate record
hi all
I have a text file that contains 14000 IP addresses. there are 5680 unique IPs in the file.
I need a powershell script to look into the text file and tell me the sum of duplicate records for each unique IP.
basically, i am thinking a script that read the first line and compare it to the rest of IPs in the file and if there is a match, sum+=1, at the end, gives me the total sum of match, then move on to the second record , compare it with all rest of the IPs...
Can someone help?
Thanks
I have a text file that contains 14000 IP addresses. there are 5680 unique IPs in the file.
I need a powershell script to look into the text file and tell me the sum of duplicate records for each unique IP.
basically, i am thinking a script that read the first line and compare it to the rest of IPs in the file and if there is a match, sum+=1, at the end, gives me the total sum of match, then move on to the second record , compare it with all rest of the IPs...
Can someone help?
Thanks
I'm thinking a simple sort-object -unique would do it for powershell. But would that give you the occurrences of each match? No. The output will be huge and ugly.
What exactly are you going for here?
Maybe we can help if you give us some backstory.
What exactly are you going for here?
Maybe we can help if you give us some backstory.
ASKER
I am going after the total occurences of each unique record.
Its really a DNS even log, we need to know in 3 minutes, how many times a IP has requested DNS to our DC.
So I have the DNS event log with all the client IPs, I need to know the total occurences of each IP.
Thanks
Its really a DNS even log, we need to know in 3 minutes, how many times a IP has requested DNS to our DC.
So I have the DNS event log with all the client IPs, I need to know the total occurences of each IP.
Thanks
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
awesome! Thanks for the excel tip! works beautifully.
I'm glad it worked for you Fireburn! If you have any other issues, please let us know how we can help. At this point, I recommend you assign points and let this be a possible solution for someone in the future.
DH
DH
You can use the powershell script described in below article.
http://techibee.com/powers hell/list- duplicate- entries-co unt-in-arr ay-using-p owershell/ 777
http://techibee.com/powers
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
1.Select the cells
2.Go to Data and click Filter
3.Click Advanced filter
4.Copy to another location
5.Choose another area to copy the unique IP address
6.Click on Unique records only
7.Click O.K