• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 360
  • Last Modified:

Merging file by monthdate

Hi,
I want to merging all MMDD* files into one file say today_file_dumb.csv and today_file_test.csv
Files format is below.

Thanks


AAK_201104191802340972_dumb.csv
AAK_201104191802355572_dumb.csv
LLK_201104191702354722_dumb.csv

KKR_201104191802340972_test.csv
BBB_201104191802355572_test.csv
LLK_201104191802354722_test.csv
0
sam2929
Asked:
sam2929
  • 5
  • 4
1 Solution
 
MikeOM_DBACommented:
Try:
cat *_dumb.csv >today_file_dumb.csv 
cat *_test.csv >today_file_test.csv

Open in new window

0
 
sam2929Author Commented:
i guess i need to be bit clear i need to dump all latest file when i say latest i mean  0419 and if it don't find 0419 then find all 0418 file and append them

0
 
sam2929Author Commented:
i guess i need to be bit clear i need to dump all latest file when i say latest i mean  all
0419( irespect of the timestamp 0419566666 or 04197777 )and if it don't find 0419 then find all 0418 file and append them)

0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
MikeOM_DBACommented:


Today=`date +%Y%m%d`
cat *_${Today}*_dumb.csv >today_file_dumb.csv 
cat *_${Today}*_test.csv >today_file_test.csv

Yesterday=`TZ=CST+24 date '+%Y%m%d'`
cat *_${Yesterday}*_dumb.csv >today_file_dumb.csv 
cat *_${Yesterday}*_test.csv >today_file_test.csv

Open in new window

0
 
sam2929Author Commented:
Hi Mike,
This is what exactly i want and also if you can provide a ls stament which i can test before doing cat

file looks like

AAK_201104191802340972_dumb.csv
AAK_201104191802355572_dumb.csv
LLK_201104191702354722_dumb.csv

KKR_201104191802340972_test.csv
BBB_201104191802355572_test.csv
LLK_201104191802354722_test.csv




so i want

AAK_201104191802340972_dumb.csv
AAK_201104191802355572_dumb.csv
> today_file_dumb.csv

and

KKR_201104191802340972_test.csv
BBB_201104191802355572_test.csv
LLK_201104191802354722_test.csv
>today_file_test.csv
0
 
MikeOM_DBACommented:

Test by replacing cat with ls and comment out the redirection.
0
 
sam2929Author Commented:
Hi Mike,
This will not work as the function is just for today and yesterday what it should do is look for latest
ddmm_reg.csv files when i say latest as they not sending files everyday so april 25 we might get april 19 3 files so we need to get latest april 19  3files.

Thanks
0
 
sam2929Author Commented:
Hi Mike,
This script is good i need bit modification is that and that is if today is '' then we can hardcode and get
file else below,Never written script so need help
Today=`date +%Y%m%d`
cat *_${Today}*_dumb.csv >today_file_dumb.csv
cat *_${Today}*_test.csv >today_file_test.csv

0
 
MikeOM_DBACommented:
Ok, here you go:

#!/usr/bin/ksh
# Merge_Files.sh
#
echo "$0 Starting: `date`"
# Setup the directories:
#
RCVDIR='/ReceivedFiles'
MRGDIR='/MergedFiles'
BKPDIR='/BackupFiles'
#
# Merge and backup the files:
#
cd $RCVDIR
for dt in `ls -1 *.csv|cut -c5-12|sort -u`
do
  echo "Processing: $dt"
  [ `ls *_${dt}*_dumb.csv|wc -l` -gt 0 ] && cat *_${dt}*_dumb.csv >$MRGDIR/${dt}_dumbFiles.csv
  [ `ls *_${dt}*_test.csv|wc -l` -gt 0 ] && cat *_${dt}*_test.csv >$MRGDIR/${dt}_testFiles.csv
  tar cf - *_${dt}*.csv >$BKPDIR/${dt}_ReceivedFiles.tgz
  rm *_${dt}*.csv
done
echo "$0 Ended: `date`"
exit 0

Open in new window

0

Featured Post

Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

  • 5
  • 4
Tackle projects and never again get stuck behind a technical roadblock.
Join Now