?
Solved

How can I agregate files with shell script

Posted on 2009-04-23
10
Medium Priority
?
61 Views
Last Modified: 2016-05-20
Compare three files: File1, File2, File3 and output to result_file.txt

From File1 fields $1, $2, $4, $5, $6 and $10 (fields contain letters and numbers, field separator "|")
From File2 fields $1, $2 and $5 (fields contain letters and numbers, field separator ",")
From File3 fields $1 and $2 (fields contain letters and numbers, field separator ",")

File1 is the reference file where I need to compare File2 and File3.

I need a script that will do:

If field $2 from File2 is equal to field $10 from File1 print fields $1, $2, $5, $4, $6 and $10 from File1 followed by field $5 and $1 from File2;

If field $1 from File3 is equal to field $4 from File1 print field $2 from File3 next to it, else print that field $2 with letter G, example: $1, $2, $5, $4, $2(equal letter or G), $6, $10, $5 and $1

If field $10 from File1 starts with letters "zm" then print "zzz", "xxx" and "yyy" on the last three fields, example: $1, $2, $5, $4, $2(equal letter or G), $6, $10(zzz), $5(xxx) and $1(yyy)

Output should be (with field separator "|": |$1|$2|$5|$4|$2|$6|$10|$5|$1|

Thanks
0
Comment
Question by:Dwebb33
  • 5
  • 3
9 Comments
 
LVL 85

Expert Comment

by:ozo
ID: 24222048
what if field $2 from File2 is equal to field $10 from File1 and field $1 from File3 is equal to field $4 from File1 and  field $10 from File1 starts with letters "zm" ?
On which line of File1, File2, File3 do you want to compare fields?
Do you want to search through File2 for any line with  field $2 equal to field $10 from File1?
Do you want to search through File3 for any line with field $1 equal to field $4 from File1?
0
 
LVL 85

Expert Comment

by:ozo
ID: 24222050
what if field $2 from File2 is not equal to field $10 from File1 and field $1 from File3 is not equal to field $4 from File1 and  field $10 from File1 does not start with letters "zm" ?
0
 

Author Comment

by:Dwebb33
ID: 24223286
Hi Ozo, thanks for your quick reply.

I want to compare line by line, starting with the first.

If field $2 from from File2 doesn't match field $10 from File1 don't print. If it matches then compare field $1 from File3 with field $4 from File1, if it matches print field $2 from File3 next to field $4 from File1, if it doesn't match print field $2 from File3 with value "G"

If all above is true and field $10 starts with"zm" then print "zzz", "xxx" and "yyy" on the last three fields ($10(zzz), $5(xxx) and $1(yyy) ).
0
[Webinar] Cloud and Mobile-First Strategy

Maybe you’ve fully adopted the cloud since the beginning. Or maybe you started with on-prem resources but are pursuing a “cloud and mobile first” strategy. Getting to that end state has its challenges. Discover how to build out a 100% cloud and mobile IT strategy in this webinar.

 

Author Comment

by:Dwebb33
ID: 24223289
Hi Ozo, thanks for your quick reply.

I want to compare line by line, starting with the first.

If field $2 from from File2 doesn't match field $10 from File1 don't print. If it matches then compare field $1 from File3 with field $4 from File1, if it matches print field $2 from File3 next to field $4 from File1, if it doesn't match print field $2 from File3 with value "G"

If all above is true and field $10 starts with"zm" then print "zzz", "xxx" and "yyy" on the last three fields ($10(zzz), $5(xxx) and $1(yyy) ).
0
 
LVL 85

Expert Comment

by:ozo
ID: 24223551

paste -d, File1 File2 File3  | awk -F'[|,]' 'BEGIN{OFS=","}$12==$10{if($10~/^zm/){ $10=$10 "zzz"; $15=$15 "xxx";$11=$11 "yyy";}print $1, $2, $5, $4,$14,$16,$4,$2 (($16==$4)?$17:"G"),$6,$10,$15,$11}'
0
 

Author Comment

by:Dwebb33
ID: 24300538
Sorry Ozo, but I received an awk error message.

I uploaded a doc file for better understanding.

I'm sorry if I'm not explaining myself.
example.docx
0
 
LVL 85

Accepted Solution

by:
ozo earned 1600 total points
ID: 24300984
paste -d, File1.txt File2.txt File3.txt  | awk -F' *[|,] *' 'BEGIN{OFS="|"}$17==$10{if($10~/^zm/){ $10= "zzz"; $20= "xxx";$16= "yyy";}if( $4 == $21){ $4 = $4 "|" $22; }print $1, $2, $5, $4, $6, $10, $20,$16}'

0
 

Expert Comment

by:EE_AutoDeleter
ID: 41606091
I've requested that this question be deleted for the following reason:

No comment has been added to this question in more than 21 days, so it is now classified as abandoned.

I have recommended this question be closed as follows:

Not enough information to confirm an answer.

If you feel this question should be closed differently, post an objection and a moderator will read all objections and then close it as they feel fit. If no one objects, this question will be closed automatically the way described above.

Experts-Exchange Auto Deleter
0
 
LVL 85

Expert Comment

by:ozo
ID: 41606092
0

Featured Post

Vote for the Most Valuable Expert

It’s time to recognize experts that go above and beyond with helpful solutions and engagement on site. Choose from the top experts in the Hall of Fame or on the right rail of your favorite topic page. Look for the blue “Nominate” button on their profile to vote.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Over the years I've spent many an hour playing on hardened, DMZ'd servers, with only a sub-set of the usual GNU toy's to keep me company; frequently I've needed to save and send log or data extracts from these server back to my PC, or to others, and…
Background Still having to process all these year-end "csv" files received from all these sources (including Government entities), sometimes we have the need to examine the contents due to data error, etc... As a "Unix" shop, our only readily …
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
In a recent question (https://www.experts-exchange.com/questions/29004105/Run-AutoHotkey-script-directly-from-Notepad.html) here at Experts Exchange, a member asked how to run an AutoHotkey script (.AHK) directly from Notepad++ (aka NPP). This video…
Suggested Courses

807 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question