Link to home
Start Free TrialLog in
Avatar of rleyba828
rleyba828Flag for Australia

asked on

Bash script to make file name the first column in a set of csv formatted files and then output to one large file

Hi Team,

  Am just requesting a bash script  where I can have the filename of the file included as the first column and have this repeated in a set of many csv files in a particular directory and then send all the output to an aggregated file, something like this:

Projecta.csv
one, two, three, four, five
six,seven,eight,nine,ten
eleven,twelve,thirteen,fourteen,fifteen

Projectb.csv
one-1, two-1, three-1, four-1, five-1
six-1,seven-1,eight-1,nine-1,ten-1
eleven-1,twelve-1,thirteen-1,fourteen-1,fifteen-1

....many more csv files....and output to one large file like this....

total.txt
Projecta,one, two, three, four, five
Projecta,six,seven,eight,nine,ten
Projecta,eleven,twelve,thirteen,fourteen,fifteen
Projectb,one-1, two-1, three-1, four-1, five-1
Projectb,six-1,seven-1,eight-1,nine-1,ten-1
Projectb,eleven-1,twelve-1,thirteen-1,fourteen-1,fifteen-1

* I will then use another script to import that file into a mysql database.

Thanks very much for all the help.
SOLUTION
Avatar of ozo
ozo
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of rleyba828

ASKER

hi ozo....Excellent.....it works perfectly!   one small request...

I downloaded from our server again,and I didn't realize  there are also " " in all the field names.....and can we also omit  the csv extension from being picked up?    the sample output is looking like this....


Projecta.csv,"one", "two", "three", "four", "five"
Projecta.csv, "six","seven","eight","nine",ten"
Projecta.csv,"eleven","twelve","thirteen","fourteen","fifteen"
Projectb.csv,"one-1", "two-1", "three-1", "four-1", "five-1"
Projectb.csv,"six-1","seven-1","eight-1","nine-1","ten-1"
Projectb.csv,"eleven-1","twelve-1","thirteen-1","fourteen-1","fifteen-1"

Can we modify the script so it will be like this?

"Projecta","one", "two", "three", "four", "five"
"Projecta", "six","seven","eight","nine",ten"
"Projecta","eleven","twelve","thirteen","fourteen","fifteen"
"Projectb","one-1", "two-1", "three-1", "four-1", "five-1"
"Projectb","six-1","seven-1","eight-1","nine-1","ten-1"
"Projectb","eleven-1","twelve-1","thirteen-1","fourteen-1","fifteen-1"

Thanks OZO!
ASKER CERTIFIED SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
    echo ${f%.csv},$line
suffices to remove the .csv
Inserting the " around each column in the line could be more difficult
    echo \"${f%.csv}\",\"${line//,/\",\"}\"
may almost work  if the line doesn't do anything  too tricky
thanks guys....working exactly as I want now.