rleyba828
asked on
Bash script to make file name the first column in a set of csv formatted files and then output to one large file
Hi Team,
Am just requesting a bash script where I can have the filename of the file included as the first column and have this repeated in a set of many csv files in a particular directory and then send all the output to an aggregated file, something like this:
Projecta.csv
one, two, three, four, five
six,seven,eight,nine,ten
eleven,twelve,thirteen,fou rteen,fift een
Projectb.csv
one-1, two-1, three-1, four-1, five-1
six-1,seven-1,eight-1,nine -1,ten-1
eleven-1,twelve-1,thirteen -1,fourtee n-1,fiftee n-1
....many more csv files....and output to one large file like this....
total.txt
Projecta,one, two, three, four, five
Projecta,six,seven,eight,n ine,ten
Projecta,eleven,twelve,thi rteen,four teen,fifte en
Projectb,one-1, two-1, three-1, four-1, five-1
Projectb,six-1,seven-1,eig ht-1,nine- 1,ten-1
Projectb,eleven-1,twelve-1 ,thirteen- 1,fourteen -1,fifteen -1
* I will then use another script to import that file into a mysql database.
Thanks very much for all the help.
Am just requesting a bash script where I can have the filename of the file included as the first column and have this repeated in a set of many csv files in a particular directory and then send all the output to an aggregated file, something like this:
Projecta.csv
one, two, three, four, five
six,seven,eight,nine,ten
eleven,twelve,thirteen,fou
Projectb.csv
one-1, two-1, three-1, four-1, five-1
six-1,seven-1,eight-1,nine
eleven-1,twelve-1,thirteen
....many more csv files....and output to one large file like this....
total.txt
Projecta,one, two, three, four, five
Projecta,six,seven,eight,n
Projecta,eleven,twelve,thi
Projectb,one-1, two-1, three-1, four-1, five-1
Projectb,six-1,seven-1,eig
Projectb,eleven-1,twelve-1
* I will then use another script to import that file into a mysql database.
Thanks very much for all the help.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
echo ${f%.csv},$line
suffices to remove the .csv
Inserting the " around each column in the line could be more difficult
echo \"${f%.csv}\",\"${line//,/ \",\"}\"
may almost work if the line doesn't do anything too tricky
suffices to remove the .csv
Inserting the " around each column in the line could be more difficult
echo \"${f%.csv}\",\"${line//,/
may almost work if the line doesn't do anything too tricky
ASKER
thanks guys....working exactly as I want now.
ASKER
I downloaded from our server again,and I didn't realize there are also " " in all the field names.....and can we also omit the csv extension from being picked up? the sample output is looking like this....
Projecta.csv,"one", "two", "three", "four", "five"
Projecta.csv, "six","seven","eight","nin
Projecta.csv,"eleven","twe
Projectb.csv,"one-1", "two-1", "three-1", "four-1", "five-1"
Projectb.csv,"six-1","seve
Projectb.csv,"eleven-1","t
Can we modify the script so it will be like this?
"Projecta","one", "two", "three", "four", "five"
"Projecta", "six","seven","eight","nin
"Projecta","eleven","twelv
"Projectb","one-1", "two-1", "three-1", "four-1", "five-1"
"Projectb","six-1","seven-
"Projectb","eleven-1","twe
Thanks OZO!