Link to home
Start Free TrialLog in
Avatar of Richard Kreidl
Richard KreidlFlag for United States of America

asked on

Combine output from a file with another file

I have the following Perl script that reads a pipe delimited input file called autosysjobs.txt:

commisnsDlyCompTransLoadBox|COMP
ctmhmHistMartComplete|CMHM
distdssMthCOMPDistDMCalc|DISTM
distdssSemiMthCOMPDistDMCalc|DISTS
commisnsECSXenosClearRiskFile|ES00014
fieldrptChkRpts|FMI
fieldrptDCMPGDCRptBox|GDC
commisnsDlyIncmPostCalcLoadBox|ICCA
commisnsDlyFeedForICM|ICTM
commisnsEndBatchCycleBox|NITECOMM
psledgerFinalProcessingBx|ODSFHR
psfinclPymntRecBox|PSF
psledgerPSLdBx|PSLDFHR
ctmoperRiskProdComplete|RPI
fieldrptDDMFMIODlySales|SALES
psfinclTEExEeUpdate|TEEFHR
psfinclTEExStagePmt|TEPP
ctmoperOperMartComplete|TNC
invmtsecOStgProcBx|SSEC
invstwhsDailyBox|SIVU


It runs a system command to generate the following output file:

commisnsDlyCompTransLoadBox   07/22/2011  21:47:37 07/22/2011  22:03:23 IN 16568629/0    
ctmhmHistMartComplete                07/22/2011  04:33:34 07/22/2011  04:33:45 SU 16560486/1    
distdssMthCOMPDistDMCalc           07/01/2011  06:56:02 07/01/2011  08:26:15 IN 16388061/1    
distdssSemiMthCOMPDistDMCalc    07/22/2011  07:06:56 07/22/2011  07:26:42 IN 16562006/0    
commisnsECSXenosClearRiskFile    07/22/2011  12:41:12 07/22/2011  12:42:02 IN 16564407/0    
fieldrptChkRpts                               07/23/2011  04:31:43 07/23/2011  04:32:01 SU 16570831/1    
fieldrptDCMPGDCRptBox                 07/22/2011  20:05:06 07/22/2011  22:49:40 SU 16567947/1    
commisnsDlyIncmPostCalcLoadBox 07/22/2011  20:49:13 07/22/2011  21:47:29 IN 16568223/0    
commisnsDlyFeedForICM                  07/22/2011  18:57:59 07/22/2011  18:58:06 IN 16567345/0    
commisnsEndBatchCycleBox            07/23/2011  01:00:19 07/23/2011  02:24:41 IN 16569821/0    
psledgerFinalProcessingBx               07/23/2011  01:00:18 07/23/2011  03:12:54 SU 16569816/1    
psfinclPymntRecBox                         07/23/2011  01:30:13 07/23/2011  02:49:34 SU 16569995/1    
psledgerPSLdBx                               07/22/2011  23:01:15 07/23/2011  02:54:21 SU 16569164/1    
ctmoperRiskProdComplete               07/22/2011  04:29:17 07/22/2011  04:29:21 SU 16559801/1    
fieldrptDDMFMIODlySales                 07/23/2011  02:27:24 07/23/2011  03:47:11 SU 16570283/1    
psfinclTEExEeUpdate                       07/22/2011  02:45:18 07/22/2011  02:54:01 SU 16560085/1    
psfinclTEExStagePmt                       07/22/2011  19:55:52 07/22/2011  19:58:27 SU 16567640/1    
ctmoperOperMartComplete             07/22/2011  03:57:54 07/22/2011  03:58:14 SU 16559323/1    
invmtsecOStgProcBx                        07/22/2011  22:22:44 07/22/2011  23:51:55 IN 16568913/0    
invstwhsDailyBox                             07/23/2011  02:13:28 07/23/2011  03:05:06 SU 16570231/1

What I need to accomplish is to concatanate the second field called' xmlname" from the input file to the front of the output file so it looks like this and generate a new output file:

COMP|commisnsDlyCompTransLoadBox       07/22/2011  21:47:37 07/22/2011  22:03:23 IN 16568629/0    
CMHM|ctmhmHistMartComplete                    07/22/2011  04:33:34 07/22/2011  04:33:45 SU 16560486/1    
DISTM|distdssMthCOMPDistDMCalc               07/01/2011  06:56:02 07/01/2011  08:26:15 IN 16388061/1    
DISTS|distdssSemiMthCOMPDistDMCalc        07/22/2011  07:06:56 07/22/2011  07:26:42 IN 16562006/0    
ES00014|commisnsECSXenosClearRiskFile    07/22/2011  12:41:12 07/22/2011  12:42:02 IN 16564407/0    
FMI|fieldrptChkRpts                                       07/23/2011  04:31:43 07/23/2011  04:32:01 SU 16570831/1    
GDC|fieldrptDCMPGDCRptBox                        07/22/2011  20:05:06 07/22/2011  22:49:40 SU 16567947/1    
ICCA|commisnsDlyIncmPostCalcLoadBox       07/22/2011  20:49:13 07/22/2011  21:47:29 IN 16568223/0    
ICTM|commisnsDlyFeedForICM                      07/22/2011  18:57:59 07/22/2011  18:58:06 IN 16567345/0    
NITECOMM|commisnsEndBatchCycleBox        07/23/2011  01:00:19 07/23/2011  02:24:41 IN 16569821/0    
ODSFHR|psledgerFinalProcessingBx               07/23/2011  01:00:18 07/23/2011  03:12:54 SU 16569816/1    

 and so on.... 
#!/opt/perl/bin/perl

my $InputTextFile = "/data/autosysjobs.txt";
my $OutputFile = "/data/output_jobs.txt";

open (IN, "$InputTextFile") or die "$InputTextFile $!";
open (OUT, ">>$OutputFile") or die "$OutputFile $!";

while (<IN>) {
   chomp($_);
    my ($jobname, $xmlname) = split(/\|/, $_);
    system("/opt/autosysPRD/autosys/bin/autorep -j $jobname | grep $jobname >> $OutputFile");
}
close IN;
close OUT;

Open in new window


thanks
Avatar of svgmuc
svgmuc
Flag of United States of America image
#!/opt/perl/bin/perl

my $InputTextFile = "/data/autosysjobs.txt";
my $OutputFile = "/data/output_jobs.txt";

open (IN, "$InputTextFile") or die "$InputTextFile $!";
open (OUT, ">>$OutputFile") or die "$OutputFile $!";

while (<IN>) {
    chomp($_);
    my ($jobname, $xmlname) = split(/\|/, $_);
    chomp($autorep) = `/opt/autosysPRD/autosys/bin/autorep -j $jobname | grep $jobname`;
    print OUT, "$xmlname|$autorep";
}
close IN;
close OUT;
Avatar of Richard Kreidl
Richard Kreidl
Flag of United States of America image

ASKER

I'm getting the following error:
Can't modify scalar chomp in scalar assignment at ./dailyopsUpdMSRAutoXMLFile line 12, near "`/opt/autosysPRD/autosys/bin/autorep -j $jobname | grep $jobname`;"

No comma allowed after filehandle at ./dailyopsUpdMSRAutoXMLFile line 13.
Avatar of svgmuc
svgmuc
Flag of United States of America image
Oh man, I should be back to perl for a little practice ;)

while (<IN>) {
    chomp($_);
    my ($jobname, $xmlname) = split(/\|/, $_);
    $autorep = chomp(`/opt/autosysPRD/autosys/bin/autorep -j $jobname | grep $jobname`);
    print OUT "$xmlname|$autorep\n";
}
Avatar of Richard Kreidl
Richard Kreidl
Flag of United States of America image

ASKER

Anotther error:

Can't modify quoted execution (``, qx) in chomp at ./dailyopsUpdMSRAutoXMLFile line 12, near "`/opt/autosysPRD/autosys/bin/autorep -j $jobname | grep $jobname`)"
Avatar of svgmuc
svgmuc
Flag of United States of America image
Hmm... then let's separate it

while (<IN>) {
    chomp($_);
    my ($jobname, $xmlname) = split(/\|/, $_);
    $autorep = `/opt/autosysPRD/autosys/bin/autorep -j $jobname | grep $jobname`;
    chomp ($autorep);
    print OUT "$xmlname|$autorep\n";
}
Avatar of ozo
ozo
Flag of United States of America image
or
chomp($autorep = `/opt/autosysPRD/autosys/bin/autorep -j $jobname | grep $jobname`);
Avatar of ozo
ozo
Flag of United States of America image
What does /opt/autosysPRD/autosys/bin/autorep do?
Avatar of Richard Kreidl
Richard Kreidl
Flag of United States of America image

ASKER

The last two solutions worked by OZO and svqmuc, but the output is on two lines:

COMP
|commisnsDlyCompTransLoadBox  07/22/2011  21:47:37 07/22/2011  22:03:23 IN 16568629/0    
CMHM
|ctmhmHistMartComplete        07/25/2011  02:18:22 07/25/2011  02:18:25 SU 16579973/1    
DISTM
|distdssMthCOMPDistDMCalc     07/01/2011  06:56:02 07/01/2011  08:26:15 IN 16388061/1    
DISTS
|distdssSemiMthCOMPDistDMCalc 07/22/2011  07:06:56 07/22/2011  07:26:42 IN 16562006/0    
ES00014
|commisnsECSXenosClearRiskFile 07/22/2011  12:41:12 07/22/2011  12:42:02 IN 16564407/0    
FMI
|fieldrptChkRpts              07/23/2011  04:31:43 07/23/2011  04:32:01 SU 16570831/1  


It should be on one line like this:

COMP|commisnsDlyCompTransLoadBox       07/22/2011  21:47:37 07/22/2011  22:03:23 IN 16568629/0    
CMHM|ctmhmHistMartComplete                    07/22/2011  04:33:34 07/22/2011  04:33:45 SU 16560486/1    
DISTM|distdssMthCOMPDistDMCalc               07/01/2011  06:56:02 07/01/2011  08:26:15 IN 16388061/1    
DISTS|distdssSemiMthCOMPDistDMCalc        07/22/2011  07:06:56 07/22/2011  07:26:42 IN 16562006/0    
ES00014|commisnsECSXenosClearRiskFile    07/22/2011  12:41:12 07/22/2011  12:42:02 IN 16564407/0    
FMI|fieldrptChkRpts                                       07/23/2011  04:31:43 07/23/2011  04:32:01 SU 16570831/1    
GDC|fieldrptDCMPGDCRptBox                        07/22/2011  20:05:06 07/22/2011  22:49:40 SU 16567947/1  


OZO, this ' /opt/autosysPRD/autosys/bin/autorep ' is an AutoSys command which gives me the last time a job started and completed in AutoSys which is a scheduling software.


thanks
Avatar of ozo
ozo
Flag of United States of America image
Does it the output of `/opt/autosysPRD/autosys/bin/autorep`  look like
commisnsDlyCompTransLoadBox   07/22/2011  21:47:37 07/22/2011  22:03:23 IN 16568629/0    
ctmhmHistMartComplete                07/22/2011  04:33:34 07/22/2011  04:33:45 SU 16560486/1    
distdssMthCOMPDistDMCalc           07/01/2011  06:56:02 07/01/2011  08:26:15 IN 16388061/1    
distdssSemiMthCOMPDistDMCalc    07/22/2011  07:06:56 07/22/2011  07:26:42 IN 16562006/0    
commisnsECSXenosClearRiskFile    07/22/2011  12:41:12 07/22/2011  12:42:02 IN 16564407/0    
fieldrptChkRpts                               07/23/2011  04:31:43 07/23/2011  04:32:01 SU 16570831/1    
fieldrptDCMPGDCRptBox                 07/22/2011  20:05:06 07/22/2011  22:49:40 SU 16567947/1    
commisnsDlyIncmPostCalcLoadBox 07/22/2011  20:49:13 07/22/2011  21:47:29 IN 16568223/0    
commisnsDlyFeedForICM                  07/22/2011  18:57:59 07/22/2011  18:58:06 IN 16567345/0    
commisnsEndBatchCycleBox            07/23/2011  01:00:19 07/23/2011  02:24:41 IN 16569821/0    
psledgerFinalProcessingBx               07/23/2011  01:00:18 07/23/2011  03:12:54 SU 16569816/1    
psfinclPymntRecBox                         07/23/2011  01:30:13 07/23/2011  02:49:34 SU 16569995/1    
psledgerPSLdBx                               07/22/2011  23:01:15 07/23/2011  02:54:21 SU 16569164/1    
ctmoperRiskProdComplete               07/22/2011  04:29:17 07/22/2011  04:29:21 SU 16559801/1    
fieldrptDDMFMIODlySales                 07/23/2011  02:27:24 07/23/2011  03:47:11 SU 16570283/1    
psfinclTEExEeUpdate                       07/22/2011  02:45:18 07/22/2011  02:54:01 SU 16560085/1    
psfinclTEExStagePmt                       07/22/2011  19:55:52 07/22/2011  19:58:27 SU 16567640/1    
ctmoperOperMartComplete             07/22/2011  03:57:54 07/22/2011  03:58:14 SU 16559323/1    
invmtsecOStgProcBx                        07/22/2011  22:22:44 07/22/2011  23:51:55 IN 16568913/0    
invstwhsDailyBox                             07/23/2011  02:13:28 07/23/2011  03:05:06 SU 16570231/1
Avatar of Richard Kreidl
Richard Kreidl
Flag of United States of America image

ASKER

Yes
Avatar of ozo
ozo
Flag of United States of America image
In that case, I might do it more like
#!/opt/perl/bin/perl

my %j;
open A,"/opt/autosysPRD/autosys/bin/autorep|" or die "/opt/autosysPRD/autosys/bin/autorep $!";
while( <A> ){
  my($j)=split;
  $j{$j}=$_;
}
close A;
my $InputTextFile = "/data/autosysjobs.txt";
my $OutputFile = "/data/output_jobs.txt";

open (IN, "$InputTextFile") or die "$InputTextFile $!";
open (STDOUT, ">>$OutputFile") or die "$OutputFile $!";

while (<IN>) {
    chomp;
    my ($jobname,$xmlname) = split/\W/;
    print "$xmlname|$j{$jobname}";
}
close IN;
close OUT;
Avatar of Richard Kreidl
Richard Kreidl
Flag of United States of America image

ASKER

OZO, it didn't work here's the output:

COMP|CMHM|DISTM|DISTS|ES00014|FMI|GDC|ICCA|ICTM|NITECOMM|ODSFHR|PSF|PSLDFHR|RPI|SALES|TEEFHR|TEPP|TNC|SSEC|SIVU|
Avatar of svgmuc
svgmuc
Flag of United States of America image
Chomp() the $xmlname too...

while (<IN>) {
    chomp($_);
    my ($jobname, $xmlname) = split(/\|/, $_);
    chomp($autorep = `/opt/autosysPRD/autosys/bin/autorep -j $jobname | grep $jobname`;)
    chomp ($xmlname);
    print OUT "$xmlname|$autorep\n";
}
Avatar of Richard Kreidl
Richard Kreidl
Flag of United States of America image

ASKER

I get the following error:
syntax error at ./dailyopsUpdMSRAutoXMLFile line 13, near "`/opt/autosysPRD/autosys/bin/autorep -j $jobname | grep $jobname`;"
syntax error at ./dailyopsUpdMSRAutoXMLFile line 16, near "}"
Avatar of Richard Kreidl
Richard Kreidl
Flag of United States of America image

ASKER

Ok i made an adjust I removed the "\n"

and here's the nwew output:

COMP
|commisnsDlyCompTransLoadBox  07/22/2011  21:47:37 07/22/2011  22:03:23 IN 16568629/0    CMHM
|ctmhmHistMartComplete        07/25/2011  02:18:22 07/25/2011  02:18:25 SU 16579973/1    DISTM
|distdssMthCOMPDistDMCalc     07/01/2011  06:56:02 07/01/2011  08:26:15 IN 16388061/1    DISTS
|distdssSemiMthCOMPDistDMCalc 07/22/2011  07:06:56 07/22/2011  07:26:42 IN 16562006/0    ES00014
|commisnsECSXenosClearRiskFile 07/22/2011  12:41:12 07/22/2011  12:42:02 IN 16564407/0    FMI
|fieldrptChkRpts              07/23/2011  04:31:43 07/23/2011  04:32:01 SU 16570831/1    GDC
|fieldrptDCMPGDCRptBox        07/22/2011  20:05:06 07/22/2011  22:49:40 SU 16567947/1    ICCA
|commisnsDlyIncmPostCalcLoadBox 07/22/2011  20:49:13 07/22/2011  21:47:29 IN 16568223/0    ICTM
|commisnsDlyFeedForICM        07/22/2011  18:57:59 07/22/2011  18:58:06 IN 16567345/0    NITECOMM
|commisnsEndBatchCycleBox     07/23/2011  01:00:19 07/23/2011  02:24:41 IN 16569821/0    ODSFHR
|psledgerFinalProcessingBx    07/23/2011  01:00:18 07/23/2011  03:12:54 SU 16569816/1    PSF
|psfinclPymntRecBox           07/23/2011  01:30:13 07/23/2011  02:49:34 SU 16569995/1    PSLDFHR
|psledgerPSLdBx               07/22/2011  23:01:15 07/23/2011  02:54:21 SU 16569164/1    RPI
|ctmoperRiskProdComplete      07/25/2011  02:40:22 07/25/2011  02:40:25 SU 16579585/1    SALES
|fieldrptDDMFMIODlySales      07/23/2011  02:27:24 07/23/2011  03:47:11 SU 16570283/1    TEEFHR
|psfinclTEExEeUpdate          07/25/2011  02:06:54 07/25/2011  02:09:58 SU 16580136/1    TEPP
|psfinclTEExStagePmt          07/22/2011  19:55:52 07/22/2011  19:58:27 SU 16567640/1    TNC
|ctmoperOperMartComplete      07/25/2011  02:12:42 07/25/2011  02:12:45 SU 16579511/1    SSEC
|invmtsecOStgProcBx           07/22/2011  22:22:44 07/22/2011  23:51:55 IN 16568913/0    SIVU
|invstwhsDailyBox             07/23/2011  02:13:28 07/23/2011  03:05:06 SU 16570231/1    

It not correct it should be like this:
COMP|commisnsDlyCompTransLoadBox       07/22/2011  21:47:37 07/22/2011  22:03:23 IN 16568629/0    
CMHM|ctmhmHistMartComplete                    07/22/2011  04:33:34 07/22/2011  04:33:45 SU 16560486/1    
DISTM|distdssMthCOMPDistDMCalc               07/01/2011  06:56:02 07/01/2011  08:26:15 IN 16388061/1    
DISTS|distdssSemiMthCOMPDistDMCalc        07/22/2011  07:06:56 07/22/2011  07:26:42 IN 16562006/0    
ES00014|commisnsECSXenosClearRiskFile    07/22/2011  12:41:12 07/22/2011  12:42:02 IN 16564407/0    
FMI|fieldrptChkRpts                                       07/23/2011  04:31:43 07/23/2011  04:32:01 SU 16570831/1    
GDC|fieldrptDCMPGDCRptBox                        07/22/2011  20:05:06 07/22/2011  22:49:40 SU 16567947/1  

and so on...
Avatar of Richard Kreidl
Richard Kreidl
Flag of United States of America image

ASKER

Here is the script I just ran to generate the output above:
#!/opt/perl/bin/perl

my $InputTextFile = "/data/autosysjobs.txt";
my $OutputFile = "/data/output_jobs.txt";

open (IN, "$InputTextFile") or die "$InputTextFile $!";
open (OUT, ">>$OutputFile") or die "$OutputFile $!";

while (<IN>) {
    chomp($_);
    my ($jobname, $xmlname) = split(/\|/, $_);
    $autorep = `/opt/autosysPRD/autosys/bin/autorep -j $jobname | grep $jobname`;
    chomp ($xmlname);
	 chomp ($autorep);
    print OUT "$xmlname|$autorep";
}

close IN;
close OUT;

Open in new window

Avatar of svgmuc
svgmuc
Flag of United States of America image
You need the \n if you want one entry per line.

Somehow, $xmlname carries a \n with it, that's why you get the line break in between the 2 fields.

while (<IN>) {
    chomp($_);
    my ($jobname, $xmlname) = split(/\|/, $_);
    $autorep = `/opt/autosysPRD/autosys/bin/autorep -j $jobname | grep $jobname`;
    chomp ($autorep);
    $xmlname =~ s/\n$//;
    $xmlname =~ s/\r$//;
    print OUT "$xmlname|$autorep";
}
ASKER CERTIFIED SOLUTION
Avatar of svgmuc
svgmuc
Flag of United States of America image
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of Richard Kreidl
Richard Kreidl
Flag of United States of America image

ASKER

thanks
Avatar of ozo
ozo
Flag of United States of America image
How does $xmlname carries a \n after
chomp($_);
    my ($jobname, $xmlname) = split(/\|/, $_);
What is $/ ?
Are you on a system that uses "\r" or "\r\n"?
Avatar of Richard Kreidl
Richard Kreidl
Flag of United States of America image

ASKER

OZO, I'm not sure what my system uses.

Is there a way I can find that info for you?