Sort a list of files and move duplicate files to another directory and unique files to a different directory

Have a list of files in a directory and the only difference in the file name is the date/time stamp.  The first 4 values of the file name will be the same for all files except the date/time and 6th value.  Would need to check the 6 value to see if it is the same as another file because the only difference would be the date/stamp.  If the 6th value is the same then move to a dup directory else move to another directory.

Would like to use a file to store the 6th value in a file and check each file against this file.

This is what I have.  Not sure if this is a good approach.

File name  --  parm1_parm2_parm3_parm4_date/time_parm6.csv

      FILELIST=`ls -1 2> /dev/null`
  for file in $FILELIST
do
      messageid=`echo $file | cut -d "_" -f6`
      echo ${messageid} >> /app/tmp/msg_list.txt
if      [ ` grep -c -f /app/tmp/msg_list.txt $messageid ` -ne 0 ]
      then
            mv $file  /somedirectory
      else
            echo mv $file /dup_directory
fi            
done
bjeAsked:
Who is Participating?
 
simon3270Commented:
Thr text you arr searching for comes first, followed by the file(s) to search in. So,

   grep -c "$messageId" /tmp/msg/list.txt
0
 
simon3270Commented:
The "if" will always find the messageid because you've just added it to the msg_list.txt file!

To fix, move the "echo $messageid" line to just after the "mv" line (i.e. between the "mv" and the "else").
0
 
bjeAuthor Commented:
Thanks,
I am getting an error "No such file or directory".  Do I need to rename the files to check and then rename them back to move them or is there a better way to do this?

cut -d _ -f6
messageid=Value6.csv
grep -c /tmp/msg_list.txt Value6.csv
grep: Value6.csv: No such file or directory
'[' '!=' 0 ']'
./test_sort.sh: line 12: [: !=: unary operator expected
0
 
bjeAuthor Commented:
This worked well.  Thanks for the help
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.