We help IT Professionals succeed at work.

The same perl programme runs well on v5.6.1 under solaris perl but dont give same sesult on v5.8.8 under linux 2.6.18-238.9.1.el5 perl

Seydina Fall
Seydina Fall used Ask the Experts™
on
Hi,

I met this problem : for the same program perl v5.6.1 on sun solaris I can't have the same result on perl v5.8.8 whitin v5.8.8 on linux.
Actually, when I run it on the first case I get good results. But on linux it seem like memory problems because off the width of the in file (attached).
Can somebody help me for this please (It's quite urgent) ?


Open in new window

#!/usr/bin/perl -w
use strict;

our $NOMBRE=3052;
our $FICHIER;
our $IDENT=4;
our %lignes;
our %nblignes;
our @fichier;

if($#ARGV+1 == 0) {
       die "USAGE: $0 FICHIER\n";
}
#Récuperation du nom du fichier
$FICHIER=$ARGV[0];

#Lecture du fichier
open(BDB,"<$FICHIER") or die "Ne peut Ouvrir $!";
while (read(BDB, my $struct, $NOMBRE) != 0) {
    push(@fichier,$struct);
}
close(BDB);

my $nbLues=$#fichier+1;
print STDOUT "LECTURE DE $nbLues LIGNE(S) DANS $FICHIER\n";
foreach my $ligne(@fichier) {
        if ($ligne=~/(^.{$IDENT}).*/) {
            my $ident=$1;
            if (! defined $lignes{$ident}) {
                $lignes{$ident}="";
                $nblignes{$ident}=0;
            }
            $lignes{$ident}=$lignes{$ident}.$ligne;
            $nblignes{$ident}+=1;
        }
}
foreach my $identLu(keys(%lignes)) {
        print STDOUT "OUVERTURE DE $FICHIER.$identLu\n";
        open (FICTEMPO,">>$FICHIER.$identLu");
             print FICTEMPO $lignes{$identLu};
             my $nbTotalLigne=$nblignes{$identLu};
             print STDOUT "ECRITURE DE $nbTotalLigne LIGNE(S)\n";
        close FICTEMPO;
        print "FERMETURE $FICHIER.$identLu\n";
}
T150OUT.zip
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Seydina FallIT ingineer

Author

Commented:
One précision on the input file. It is a sequential indexed file produced by COBOL on HRACCESS v5 application.
Seydina FallIT ingineer

Author

Commented:
Actually, there is no error with the new os. Just the file at the end is really too small and dont give the whole file expected.
Seydina FallIT ingineer

Author

Commented:
Can i have please a quick feedback !

I tried to replace the foreach command by this :
while (defined (my $ligne = @fichier )) {
I got the same result : after one hour nothing happen whereas it takes 10 minutes on the other platform with results expected.

Thanks for any feedback
Seydina FallIT ingineer

Author

Commented:
Hi,

Nobody has any idea !
There is another information, i think that issue may come from the COBOL mehod of sorting file. The way under unix solaris may be different to the linux way. So here is the COBOL command :
Caution : The COBOL command is encapsulated under shell command

Open in new window

SORTIN=$TMP/T120DFU."$nupro"
    export SORTIN
SORTOUT=$TMP/T150OUT."$nupro"
    export SORTOUT

echo "SIGN-EBCDIC
SORT FIELDS (
00001,00040,CH,A,
00041,00012,CH,A,
00061,00025,CH,A,
00086,00002,CH,A)
RECORD (F, 3052)
USE $SORTIN ORG (SQ) RECORD (F,3052)
GIVE $SORTOUT ORG (SQ) RECORD (F,3052)
" 1>$TMP/KJ3150."$nupro"
mfsort take $TMP/KJ3150."$nupro"
    CODE_RETOUR=$?
    unset SORTIN
    unset SORTOUT
    rm $TMP/T120DFU."$nupro"
    rm $TMP/KJ3150*."$nupro"
if
  [ $CODE_RETOUR -gt $MAX_RETOUR ]
  then
      MAX_RETOUR=$CODE_RETOUR
fi
echo "\n"
fi
if
  [ $MAX_RETOUR -le 4 ]
  then
echo "*-------------------------- STEP160A ---------------------------*"
echo "*                  Load parameters in T160DTR                   *"
echo "*---------------------------------------------------------------*"
echo "DA341"\
  >$TMP/T160DTR."$nupro"
echo "\n"
fi

Please may someone give me some help ?
Software Engineer
Distinguished Expert 2018
Commented:
Well here it goes:

The trouble seems to be a Whole bunch of remapping memory & moving it around
because of the line:
    $lignes{$ident}=$lignes{$ident}.$ligne;

[ and you are moving around a lot of memory... ]

I added a routine to generate filehandles, [ newopen() ]so it opens a file and returns a handle.
The main loop just writes out the record.

Here goes:
use strict;
use warnings;



use strict;

our $NOMBRE=3052;
our $FICHIER;
our $IDENT=4;
our %lignes;
our %nblignes;
our %files;
our @fichier;

sub newopen {
        my $path = shift;
        local  *FH;  # not my!
        open   (FH, $path)          or  return undef;
        return *FH;
}


if($#ARGV+1 == 0) {
       die "USAGE: $0 FICHIER\n";
}
#Récuperation du nom du fichier
$FICHIER=$ARGV[0];

#Lecture du fichier
open(BDB,"<$FICHIER") or die "Ne peut Ouvrir $!";
while (read(BDB, my $struct, $NOMBRE) != 0) {
       push(@fichier,$struct);
}
close(BDB);

my $nbLues=$#fichier+1;
print STDOUT "LECTURE DE $nbLues LIGNE(S) DANS $FICHIER\n";
foreach my $ligne(@fichier) {
   if ($ligne=~/(^.{$IDENT}).*/) {
      my $ident=$1;
      if (! defined $files{$ident}) {
         $nblignes{$ident}=0;
         print STDOUT "OUVERTURE DE $FICHIER.$identLu\n";
                 $files{$ident} = newopen(">$FICHIER.$ident");
      }
      print {$files{$ident}} $ligne;
      $nblignes{$ident}+=1;
   }
}
foreach my $identLu(keys(%files)) {
   my $nbTotalLigne=$nblignes{$identLu};
   print STDOUT "ECRITURE DE $nbTotalLigne LIGNE(S)\n";
   close $files{$identLu};
   print "FERMETURE $FICHIER.$identLu\n";
} 

Open in new window



And timing for this code was:
real    0m9.101s
user    0m1.738s
sys     0m1.579s
Seydina FallIT ingineer

Author

Commented:
Hi,<br /><br />Thanks for this solution.<br />I pass from 10 minutes to 10 seconds currently.<br />I just correct this line : print STDOUT "OUVERTURE DE $FICHIER.$identLu\n";<br />$ident instead of $identLu.<br />The perl program is improved but my problem is within the Cobol one which generates the input file for perl pgm. <br />I m looking on that way and.<br />Any idea is wellcome.<br /><br />Thanks again
nociSoftware Engineer
Distinguished Expert 2018

Commented:
That is hardly COBOL.

The mfsort (http://www.bostream.nu/brickhouse/mfsortexamples.html) seems more likely.

And AFAICT Linux & Unix don't differ too much in the way they work. But implementations of software might be different.
Both Unix & Linux are ASCII orientated, where your dataset seems to be EBCDIC orientated, this might explain some of your results.
[ EBCDIC on the first line of the mfsort spec ].

EBCDIC encoding is a radically different character encoding w.r.t. ASCII.
ASCII is designed to be in collating order and with all characters in one row without other signs.   https://en.wikipedia.org/wiki/ASCII
EBCDIC is based on cardreaders specs, which at first could only have 10 rows and later got more, this introduced special characters in between characters. https://en.wikipedia.org/wiki/EBCDIC

In ASCII all character codes are < value 127 [ allowing for 7 bit transmissions ]
In EBCDIC all character codes are >=128 , with all kind of specials below 128.

Might that explain your issues?