Link to home
Start Free TrialLog in
Avatar of fireburn11
fireburn11

asked on

powershell to find duplicate records in a text file and give me a sum of each duplicate record

hi all

I have a text file that contains 14000 IP addresses. there are 5680 unique IPs in the file.

I need a powershell script to look into the text file and tell me the sum of duplicate records for each unique IP.

basically, i am thinking a script that read the first line and compare it to the rest of IPs in the file and if there is a match, sum+=1, at the end, gives me the total sum of match, then move on to the second record , compare it with all rest of the IPs...

Can someone help?

Thanks
Avatar of Senthil Kumar
Senthil Kumar
Flag of India image

Simple way is to copy the text to an Excel file and follow the steps to find the unique IP address
1.Select the cells
2.Go to Data and click Filter
3.Click Advanced filter
4.Copy to another location
5.Choose another area to copy the unique IP address
6.Click on Unique records only
7.Click O.K
I'm thinking a simple sort-object -unique would do it for powershell.  But would that give you the occurrences of each match?  No.  The output will be huge and ugly.

What exactly are you going for here?

Maybe we can help if you give us some backstory.
Avatar of fireburn11
fireburn11

ASKER

I am going after the total occurences of each unique record.
Its really a DNS even log, we need to know in 3 minutes, how many times a IP has requested DNS to our DC.

So I have the DNS event log with all the client IPs, I need to know the total occurences of each IP.

Thanks
SOLUTION
Avatar of Dale Harris
Dale Harris
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
awesome! Thanks for the excel tip! works beautifully.
I'm glad it worked for you Fireburn!  If you have any other issues, please let us know how we can help.  At this point, I recommend you assign points and let this be a possible solution for someone in the future.

DH
ASKER CERTIFIED SOLUTION
Avatar of Qlemo
Qlemo
Flag of Germany image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial