Solved

using awk to add header and separotor

Posted on 2014-11-25
23
225 Views
Last Modified: 2014-11-26
Hi,

I have following awk in linux that check all files with .out and select those files and give me out put.

awk --field-separator=";" '{ print $2,$11 }' *.csv> all.txt

what i'm looking for is:

account|name
243343|john
234343|cena
                                 

but I want to add header and separator for each field which i'm not succesfull as of yet, please suggest the same for awk or alternative.
0
Comment
Question by:hi4ppl
  • 12
  • 11
23 Comments
 
LVL 84

Expert Comment

by:ozo
ID: 40466267
account|name
243343|john
234343|cena
Is that supposed to be the input or the output?
If it is the input, what would you want the corresponding output to be?
If it is the desired output, what was the corresponding input?
0
 
LVL 1

Author Comment

by:hi4ppl
ID: 40466294
Hi,

Thanks for replay, that is out put that i'm looking for, the input file have many fields I only select those two fields,
what the input looks like is bellow:

account;name;fathername;country;city;....
243343;john;cena;USA;PA;.....
23434;|cena;john;france;AM;....

(...) represent that there more field around 25 but that is how it looks like.

regards
0
 
LVL 84

Expert Comment

by:ozo
ID: 40466300
awk -F";[|]*" -v OFS="|" '{ print $1,$2 }' <<END
account;name;fathername;country;city;....
243343;john;cena;USA;PA;.....
23434;|cena;john;france;AM;....
END
Should produce your desired output
0
 
LVL 1

Author Comment

by:hi4ppl
ID: 40466314
Hi,

thanks but in example i'm pushing files as *.cv in your example it put all records their, how can I pass the file name?

regards
0
 
LVL 84

Expert Comment

by:ozo
ID: 40466315
awk -F";[|]*" -v OFS="|" '{ print $1,$2 }' *.cv
0
 
LVL 1

Author Comment

by:hi4ppl
ID: 40466391
hi, it does not work it says syntax error:

awk : testfile.csv
awk:              ^ syntax error 

Open in new window

0
 
LVL 84

Expert Comment

by:ozo
ID: 40466396
Can you paste the command you used?
0
 
LVL 1

Author Comment

by:hi4ppl
ID: 40466410
hi here it's


awk -F";[|]*" -v OFS="|"'{ print $2,$11,$15,$46,$48,$62 }' *.csv

Open in new window

0
 
LVL 84

Expert Comment

by:ozo
ID: 40466504
There should be a space between OFS="|" and '{ print $2,$11,$15,$46,$48,$62 }'

But  print $2,$11
would skip the
account
243343
in
account;name;fathername;country;city;....
243343;john;cena;USA;PA;.....
0
 
LVL 1

Author Comment

by:hi4ppl
ID: 40466520
Hi, thanks yeah I know that that was just exmaple it works but what it does not work is it does not print out the header information, it only add |, I wanted to print the header information also.

regards
0
 
LVL 84

Expert Comment

by:ozo
ID: 40466522
Can you post an example of your desired output?
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 
LVL 1

Author Comment

by:hi4ppl
ID: 40466530
hi, this is what it does now:

243343|john
234343|cena

what I want is

account|name
243343|john
234343|cena

regards
omer
0
 
LVL 84

Expert Comment

by:ozo
ID: 40466543
If your input looks like

1;account;3;4;5;6;7;8;9;10;name;fathername;country;city;....
1;243343;3;4;5;6;7;8;9;10;john;cena;USA;PA;.....
1;234343;3;4;5;6;7;8;9;10;cena;

then your output would look like

account|name|....|||
243343|john|.....|||
234343|cena||||
0
 
LVL 1

Author Comment

by:hi4ppl
ID: 40466554
Hi, but it does not give me that result my input looks like this :

account;name;fathername;country;city;....
243343;john;cena;USA;PA;.....
23434;|cena;john;france;AM;....

but it does not give me the header  like

account|name|....|||
243343|john|.....|||
234343|cena||||
0
 
LVL 84

Expert Comment

by:ozo
ID: 40466589
Running
awk -F";[|]*" -v OFS="|" '{ print $2,$11,$15,$46,$48,$62 }' <<END
account;name;fathername;country;city;....
243343;john;cena;USA;PA;.....
23434;|cena;john;france;AM;....
END

would get
name|||||
john|||||
cena|||||

If you want

account|name|....|||
243343|john|.....|||
234343|cena||||

then you can run

awk -F";[|]*" -v OFS="|" '{ print $1,$2,$6,$7,$8,$9}' <<END
account;name;fathername;country;city;....
243343;john;cena;USA;PA;.....
234343;|cena;john;france;AM;....
END
0
 
LVL 1

Author Comment

by:hi4ppl
ID: 40466611
hi,
when i run your sample alone it works but when I run this on the file *.csv it gives me result without the header,
one how it make difference that you printed $1 and it print the column names but when I try in my file it print the column 1 not the header itself.

regards
0
 
LVL 84

Expert Comment

by:ozo
ID: 40466638
Can you post a sample of what *.csv actually looks like?
0
 
LVL 1

Author Comment

by:hi4ppl
ID: 40466701
well it's big file around 3GB but it looks like the sample I provided , at the top of the row there is all header which is around 120, then after that it's all data.

here is the sample again

account;name;fathername;country;city
243343;john;cena;USA;PA
23434;|cena;john;france;AM

Open in new window


regards
0
 
LVL 1

Author Comment

by:hi4ppl
ID: 40466713
hi sorry my bad I had one more condition here:

when I run it

awk -F";[|]*" -v OFS="|" '{ print $2,$11 }' *.csv

Open in new window


it gives me header also but when I run

awk -F";[|]*" -v OFS="|" '$11=="Active"{ print $2,$11 }' *.csv

Open in new window


a condition to output only records that $11 have Active, so is there a way to have header also and that condition will work as well?

sorry for confusion I just notice it now
0
 
LVL 84

Expert Comment

by:ozo
ID: 40466714
if
cat *.csv
prints
account;name;fathername;country;city
243343;john;cena;USA;PA
23434;|cena;john;france;AM

Open in new window

then
awk -F";[|]*" -v OFS="|" '{ print $2,$11,$15,$46,$48,$62 }' *.csv
would print
name|||||
john|||||
cena|||||

Open in new window

0
 
LVL 84

Accepted Solution

by:
ozo earned 500 total points
ID: 40466719
What does $11 look like when there is a header?  Or how can you tell which non"Active" line contains a header?
If it $11 is "name" in a header, then you might use
'$11=="Active"||$11=="Name"{ print $2,$11 }'
if the first line is always a header, then you might use
'$11=="Active"||NR==1{ print $2,$11 }'
0
 
LVL 1

Author Closing Comment

by:hi4ppl
ID: 40466779
Thanks it works :D
0
 
LVL 1

Author Comment

by:hi4ppl
ID: 40466785
Hi,

this is might be out of this question but, can I customize the header name? in other word can I give name to each variable $4 to something excluding the header they have i.e if the header in main file is AccountID can i change it to Account only?
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

This is an explanation of a simple data model to help parse a JSON feed
If you’re thinking to yourself “That description sounds a lot like two people doing the work that one could accomplish,” you’re not alone.
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
An introduction to basic programming syntax in Java by creating a simple program. Viewers can follow the tutorial as they create their first class in Java. Definitions and explanations about each element are given to help prepare viewers for future …

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now