Solved

# Removing Duplicate words from lines in a power shell

Posted on 2015-02-23
Medium Priority
130 Views
Hi

I am trying to find away to remove Duplicate words from a text file using Powershell

Name1 PC1234 PC1234 start Tue 2/17 6:53
Name2 PC5678 PC5678 start Tue 2/17 7:51

What I would like is

Name1 PC1234 start Tue 2/17 6:53
Name2 PC5678 start Tue 2/17 7:51

I have managed to get it working using the following but I would like to be able to open a .txt file remove the duplicate on all lines and right the content back to a new file.

$text= "Name1 PC1234 PC1234 start Tue 2/17 6:53"$text.Length
$text.remove(14)  Thank you of your help 0 Question by:Si Walton [X] ###### Welcome to Experts Exchange Add your voice to the tech community where 5M+ people just like you are talking about what matters. • Help others & share knowledge • Earn cash & points • Learn & ask questions 5 Comments LVL 53 Expert Comment ID: 40626292 Might not be a solution that your looking for, but the easiest way to accomplish this is in Excel and using the remove duplicate button. Will. 0 LVL 85 Accepted Solution oBdA earned 1000 total points ID: 40626329 So the third column is always a copy of the second? This will remove the third column: Get-Content C:\Temp\in.txt | % {$_.Split(" ")[0..1+3..($_.Length -1)] -join " "} | Set-Content C:\Temp\out.txt  0 LVL 1 Author Comment ID: 40627722 Hi oBda Almost there and I understand the$_.Split

How would I be able to get the code to ignore the 1st line in the text file.

Keep this text
Name1 PC1234 PC1234 start Tue 2/17 6:53
Name2 PC5678 PC5678 start Tue 2/17 7:51

Thank you

Si
0

LVL 71

Assisted Solution

Qlemo earned 1000 total points
ID: 40627914
$( Get-Content C:\Temp\in.txt | select -First 1 Get-Content C:\Temp\in.txt | select -Skip 1 | % {$_.Split(" ")[0..1+3..(\$_.Length -1)] -join " "}
) | Set-Content C:\Temp\out.txt

0

LVL 1

Author Closing Comment

ID: 40627936
Hi oBda and Qlemo

Thank you the solutions worked perfectly

Si
0

## Featured Post

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

A recent project that involved parsing Tableau Desktop and Server log files to extract reusable user queries for use in other systems. I chose to use PowerShell to gather the data, and SharePoint to present it...
My attempt to use PowerShell and other great resources found online to simplify the deployment of Office 365 ProPlus client components to any workstation that needs it, regardless of existing Office components that may be needing attention.
Exchange organizations may use the Journaling Agent of the Transport Service to archive messages going through Exchange. However, if the Transport Service is integrated with some email content management application (such as an antispam), the admini…
Add bar graphs to Access queries using Unicode block characters. Graphs appear on every record in the color you want. Give life to numbers. Hopes this gives you ideas on visualizing your data in new ways ~ Create a calculated field in a query: …
###### Suggested Courses
Course of the Month10 days, 19 hours left to enroll

#### 719 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.