Avatar of deepak singla
deepak singla
 asked on

Want to remove duplicate values from comma separated variable using batch file

I have a batch file. Here I assigned some values to a variable. Want to remove duplicate values from the variable.

@echo off
set test=1,2,4,1,5,6,2,3


expected output: 1,2,3,4,5,6
Windows Batch

Avatar of undefined
Last Comment
Pavel Gushchin

8/22/2022 - Mon
Pavel Gushchin

@ECHO OFF
set test=1,2,4,1,5,6,2,3
del tmp.txt >nul

for %%a in (%test%) do echo %%a >>tmp.txt

for /f "delims=" %%F in ('sort tmp.txt') do (
  set "curr=%%F"
  setlocal enabledelayedexpansion
  if "!prev!" neq "!curr!" echo !curr!
  endlocal
  set "prev=%%F"
)

del tmp.txt >nul

Open in new window

deepak singla

ASKER
Thanks @Pavel

Actually, I want to store the result in a variable with a comma separated instead to echo it. So that I will return the variable to the parent batch file.
eg: set result=1,2,3,4,5,6

I tried to store in a variable, but I am getting an error "Could Not Find C:\chef\tmp.txt"

Could you please help me to resolve it.
Pavel Gushchin

@ECHO OFF
set test=1,2,4,1,5,6,2,3
del tmp.txt >nul
del tmp_sorted.txt >nul

for %%a in (%test%) do echo %%a>>tmp.txt

for /f "delims=" %%F in ('sort tmp.txt') do (
  set "curr=%%F"
  setlocal enabledelayedexpansion
  if "!prev!" neq "!curr!" echo !curr! >>tmp_sorted.txt
  endlocal
  set "prev=%%F"
)

setlocal EnableDelayedExpansion
set "txt="
set input=tmp_sorted.txt
for /f "delims=" %%a in (%input%) do (
  set "txt=!txt!%%a,"
)
set "txt=!txt:~0,-1!"
>new.txt echo !txt!

del tmp.txt >nul
del tmp_sorted.txt >nul

Open in new window

This is the best money I have ever spent. I cannot not tell you how many times these folks have saved my bacon. I learn so much from the contributors.
rwheeler23
deepak singla

ASKER
Thanks

I also found another way to resolve it

@ECHO off
SETLOCAL EnableDelayedExpansion
SET oldString=node1node2,node3,node4,node2,node3,node5
SET newstring=

FOR %%a IN ("%oldString:,=";"%") DO (
    IF NOT !test%%~a!==TRUE (
        SET test%%~a=TRUE
        IF "!newstring!"=="" (
                  SET newstring=%%~a
        ) ELSE (
            SET newstring=!newstring!,%%~a
        )
    )
)

ECHO Old String: !oldString!
ECHO New String: !newstring!
deepak singla

ASKER
Hi Pavel

I am getting below error from your latest batch script.

Could Not Find c:\chef\tmp.txt
Could Not Find c:\chef\tmp_sorted.txt

Open in new window

ASKER CERTIFIED SOLUTION
Pavel Gushchin

Log in or sign up to see answer
Become an EE member today7-DAY FREE TRIAL
Members can start a 7-Day Free trial then enjoy unlimited access to the platform
Sign up - Free for 7 days
or
Learn why we charge membership fees
We get it - no one likes a content blocker. Take one extra minute and find out why we block content.
Not exactly the question you had in mind?
Sign up for an EE membership and get your own personalized solution. With an EE membership, you can ask unlimited troubleshooting, research, or opinion questions.
ask a question