Want to remove duplicate values from comma separated variable using batch file

deepak singla
deepak singla used Ask the Experts™
on
I have a batch file. Here I assigned some values to a variable. Want to remove duplicate values from the variable.

@echo off
set test=1,2,4,1,5,6,2,3


expected output: 1,2,3,4,5,6
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
@ECHO OFF
set test=1,2,4,1,5,6,2,3
del tmp.txt >nul

for %%a in (%test%) do echo %%a >>tmp.txt

for /f "delims=" %%F in ('sort tmp.txt') do (
  set "curr=%%F"
  setlocal enabledelayedexpansion
  if "!prev!" neq "!curr!" echo !curr!
  endlocal
  set "prev=%%F"
)

del tmp.txt >nul

Open in new window

Author

Commented:
Thanks @Pavel

Actually, I want to store the result in a variable with a comma separated instead to echo it. So that I will return the variable to the parent batch file.
eg: set result=1,2,3,4,5,6

I tried to store in a variable, but I am getting an error "Could Not Find C:\chef\tmp.txt"

Could you please help me to resolve it.
@ECHO OFF
set test=1,2,4,1,5,6,2,3
del tmp.txt >nul
del tmp_sorted.txt >nul

for %%a in (%test%) do echo %%a>>tmp.txt

for /f "delims=" %%F in ('sort tmp.txt') do (
  set "curr=%%F"
  setlocal enabledelayedexpansion
  if "!prev!" neq "!curr!" echo !curr! >>tmp_sorted.txt
  endlocal
  set "prev=%%F"
)

setlocal EnableDelayedExpansion
set "txt="
set input=tmp_sorted.txt
for /f "delims=" %%a in (%input%) do (
  set "txt=!txt!%%a,"
)
set "txt=!txt:~0,-1!"
>new.txt echo !txt!

del tmp.txt >nul
del tmp_sorted.txt >nul

Open in new window

JavaScript Best Practices

Save hours in development time and avoid common mistakes by learning the best practices to use for JavaScript.

Author

Commented:
Thanks

I also found another way to resolve it

@ECHO off
SETLOCAL EnableDelayedExpansion
SET oldString=node1node2,node3,node4,node2,node3,node5
SET newstring=

FOR %%a IN ("%oldString:,=";"%") DO (
    IF NOT !test%%~a!==TRUE (
        SET test%%~a=TRUE
        IF "!newstring!"=="" (
                  SET newstring=%%~a
        ) ELSE (
            SET newstring=!newstring!,%%~a
        )
    )
)

ECHO Old String: !oldString!
ECHO New String: !newstring!

Author

Commented:
Hi Pavel

I am getting below error from your latest batch script.

Could Not Find c:\chef\tmp.txt
Could Not Find c:\chef\tmp_sorted.txt

Open in new window

It's not a critical error... change it so:
@ECHO OFF
set test=1,2,4,1,5,6,2,3

if exist tmp.txt del tmp.txt
if exist tmp_sorted.txt del tmp_sorted.txt

for %%a in (%test%) do echo %%a>>tmp.txt

for /f "delims=" %%F in ('sort tmp.txt') do (
  set "curr=%%F"
  setlocal enabledelayedexpansion
  if "!prev!" neq "!curr!" echo !curr! >>tmp_sorted.txt
  endlocal
  set "prev=%%F"
)

setlocal EnableDelayedExpansion
set "txt="
set input=tmp_sorted.txt
for /f "delims=" %%a in (%input%) do (
  set "txt=!txt!%%a,"
)
set "txt=!txt:~0,-1!"
>new.txt echo !txt!

if exist tmp.txt del tmp.txt
if exist tmp_sorted.txt del tmp_sorted.txt

Open in new window

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial