Solved

calculating the difference in value between data

Posted on 2009-07-12
22
135 Views
Last Modified: 2012-05-07
Step three in this code calculates the difference between the one value and the next.

the results shown :
232      356      434      767      786      854      avg 571.50      group 0011100210
232      124      78      333      19      68      avg 142.33      

the second line is the difference between the numbers on the first line.

The first entry on the second line reads 232. This should read 0, as we dont need the value from the start here.
Could the code be altered to reflect this ?


#!/usr/bin/perl

use strict;

use warnings;

use Data::Dumper;

use List::Util qw(sum);

 

##### Open files

open(my $IN,"invim2.vim") or die "Could not open input: $!\n";

open(my $OUT,">outvim2.vim") or die "Could not open output: $!\n";

 

my @avgs1;

my @avgs2;

my ($maxavg1, $minavg1, $maxavg2, $minavg2);

 

##### Step 1: read a line, and split it on comma

while(<$IN>){

        chomp;

        my @arr = split /,/;

        next unless @arr;

 
 

        # discard 1st and 2nd last and last item

        shift @arr; 

	pop @arr;

	pop @arr;

	# pop @arr;

 
 

        ##### Step 2: Sort data into ascending order

        @arr = sort {$a <=> $b} @arr;

        print "arr @arr\n";

 
 

        ##### Step 3: Calculate difference between neighboring pairs

        my @diff = ($arr[0], map {$arr[$_] - $arr[$_-1]} (1..$#arr));

        print "diff @diff\n";

 
 

        ##### Step 4: Calculate amount in each group

        my @groups = (0)x10;

        foreach my $a (@arr) {

                $groups[$a/100]++;

        }

        print "group @groups\n";
 
 

        ##### Step 5: Calculate average

 

        my $avg1 = sum(@arr)/@arr;

        printf "arr avg %.2f\n", $avg1;

        if(!$maxavg1 || $avg1 > $maxavg1) { $maxavg1 = $avg1; }

        if(!$minavg1 || $avg1 < $minavg1) { $minavg1 = $avg1; }

        push @avgs1, $avg1;

 

        my $avg2 = sum(@diff)/@diff;

        printf "diff avg %.2f\n", $avg2;%.2f

        if(!$maxavg2 || $avg2 > $maxavg2) { $maxavg2 = $avg2; }

        if(!$minavg2 || $avg2 < $minavg2) { $minavg2 = $avg2; }

        push @avgs2, $avg2;

 

        ##### Step 6: Print results

	

	
 
 

        print $OUT join("\t", @arr) . "\tavg " . sprintf("%.2f\t", $avg1) . "group " . join("", @groups) . "\n";

        print $OUT join("\t", @diff) . "\tavg " . sprintf("%.2f\t", $avg2) . "\n";

        print $OUT "\n";

}

 
 

	printf "input: maxavg %d minavg %d avgavg %d\n", $maxavg1, $minavg1, sum(@avgs1)/@avgs1;

	printf $OUT "For data values:\t\t avg avg %.2f\t maximum avg %.2f\t minimum avg %.2f \n", sum(@avgs1)/@avgs1, $maxavg1, $minavg1 ;

	printf "diff: maxavg %d minavg %d avgavg %d\n", $maxavg2, $minavg2, sum(@avgs2)/@avgs2;

	printf $OUT "For data difference values:\t avg avg %.2f\t maximum avg %.2f\t minimum avg %.2f \n", sum(@avgs2)/@avgs2, $maxavg2, $minavg2, ;
 

	

 

close($IN);

close($OUT);

Open in new window

0
Comment
Question by:MichaelGlancy
  • 11
  • 7
  • 4
22 Comments
 
LVL 84

Expert Comment

by:ozo
ID: 24835788
       my @diff = (0, map {$arr[$_] - $arr[$_-1]} (1..$#arr));
But do you want a 0 there, or should there be no value there?
        my @diff =  map {$arr[$_] - $arr[$_-1]} (1..$#arr);
0
 
LVL 40

Expert Comment

by:mrjoltcola
ID: 24835794
Hi Michael, looks like the program got corrupted in your cut and paste. I'll modify from my local copy from last night.
0
 
LVL 40

Expert Comment

by:mrjoltcola
ID: 24835798

#!/usr/bin/perl

use strict;

use warnings;

use Data::Dumper;

use List::Util qw(sum);
 

##### Open files

open(my $IN,"invim2.vim") or die "Could not open input: $!\n";

open(my $OUT,">outvim2.vim") or die "Could not open output: $!\n";
 

my @avgs1;

my @avgs2;

my ($maxavg1, $minavg1, $maxavg2, $minavg2);
 

##### Step 1: read a line, and split it on comma

while(<$IN>){

        chomp;

        my @arr = split /,/;

        next unless @arr;
 

        # discard 1st and last item

        shift @arr; pop @arr;
 

        ##### Step 2: Sort data into ascending order

        @arr = sort {$a <=> $b} @arr;

        print "arr @arr\n";
 

        ##### Step 3: Calculate difference between neighboring pairs

        my @diff = (0, map {$arr[$_] - $arr[$_-1]} (1..$#arr));

        print "diff @diff\n";
 

        ##### Step 4: Calculate amount in each group

        my @groups = (0)x5;

        foreach my $a (@arr) {

                $groups[$a/10]++;

        }

        print "group @groups\n";

        ##### Step 6: Calculate average
 

        my $avg1 = sum(@arr)/@arr;

        printf "arr avg %.2f\n", $avg1;

        if(!$maxavg1 || $avg1 > $maxavg1) { $maxavg1 = $avg1; }

        if(!$minavg1 || $avg1 < $minavg1) { $minavg1 = $avg1; }

        push @avgs1, $avg1;
 

        my $avg2 = sum(@diff)/@diff;

        printf "diff avg %.2f\n", $avg2;

        if(!$maxavg2 || $avg2 > $maxavg2) { $maxavg2 = $avg2; }

        if(!$minavg2 || $avg2 < $minavg2) { $minavg2 = $avg2; }

        push @avgs2, $avg2;
 

        ##### Step 5: Print results

        print $OUT join("\t", @arr) . "\tavg " . sprintf("%.2f\t", $avg1) . "group " . join("", @groups) . "\n";

        print $OUT join("\t", @diff) . "\tavg " . sprintf("%.2f\t", $avg2) . "\n";

        print $OUT "\n";

}
 

printf "input: maxavg %d minavg %d avgavg %d\n", $maxavg1, $minavg1, sum(@avgs1)/@avgs1;

printf $OUT "maxavg %d minavg %d avgavg %d\n", $maxavg1, $minavg1, sum(@avgs1)/@avgs1;

printf "diff: maxavg %d minavg %d avgavg %d\n", $maxavg2, $minavg2, sum(@avgs2)/@avgs2;

printf $OUT "maxavg %d minavg %d avgavg %d\n", $maxavg2, $minavg2, sum(@avgs2)/@avgs2;
 

close($IN);

close($OUT);

Open in new window

0
 

Author Comment

by:MichaelGlancy
ID: 24835811
hi mrjoltcola,

it want corrupted, I had altered some parts today

0
 

Author Comment

by:MichaelGlancy
ID: 24835822
at step 3,

instead of finding the first value difference (232 in the above example)

the first value difference should not be there
0
 

Author Comment

by:MichaelGlancy
ID: 24835834
maybe it has been corrupted, cant get it to work now
0
 
LVL 40

Expert Comment

by:mrjoltcola
ID: 24835838
I got syntax errors without any mods to your code.

Number found where operator expected at t.pl line 56, near "%.2"
        (Missing operator before 2?)
Bareword found where operator expected at t.pl line 56, near "2f"
        (Missing operator before f?)
syntax error at t.pl line 56, near "%.2"
syntax error at t.pl line 69, near "}"
Execution of t.pl aborted due to compilation errors.
0
 

Author Comment

by:MichaelGlancy
ID: 24835841
now Im lost again. What are mods ?
0
 
LVL 40

Accepted Solution

by:
mrjoltcola earned 500 total points
ID: 24835845
I'm a little confused because on the previous mod I made to this it was to add the 1st value in the diff array, now you want it removed.

Here you go.

#!/usr/bin/perl

use strict;

use warnings;

use Data::Dumper;

use List::Util qw(sum);
 

##### Open files

open(my $IN,"invim2.vim") or die "Could not open input: $!\n";

open(my $OUT,">outvim2.vim") or die "Could not open output: $!\n";
 

my @avgs1;

my @avgs2;

my ($maxavg1, $minavg1, $maxavg2, $minavg2);
 

##### Step 1: read a line, and split it on comma

while(<$IN>){

        chomp;

        my @arr = split /,/;

        next unless @arr;
 
 

        # discard 1st and 2nd last and last item

        shift @arr;

        pop @arr;

        pop @arr;

        # pop @arr;
 
 

        ##### Step 2: Sort data into ascending order

        @arr = sort {$a <=> $b} @arr;

        print "arr @arr\n";
 
 

        ##### Step 3: Calculate difference between neighboring pairs

        my @diff = (map {$arr[$_] - $arr[$_-1]} (1..$#arr));

        print "diff @diff\n";
 
 

        ##### Step 4: Calculate amount in each group

        my @groups = (0)x10;

        foreach my $a (@arr) {

                $groups[$a/100]++;

        }

        print "group @groups\n";
 
 

        ##### Step 5: Calculate average
 

        my $avg1 = sum(@arr)/@arr;

        printf "arr avg %.2f\n", $avg1;

        if(!$maxavg1 || $avg1 > $maxavg1) { $maxavg1 = $avg1; }

        if(!$minavg1 || $avg1 < $minavg1) { $minavg1 = $avg1; }

        push @avgs1, $avg1;
 

        my $avg2 = sum(@diff)/@diff;

        printf "diff avg %.2f\n", $avg2;

        if(!$maxavg2 || $avg2 > $maxavg2) { $maxavg2 = $avg2; }

        if(!$minavg2 || $avg2 < $minavg2) { $minavg2 = $avg2; }

        push @avgs2, $avg2;
 

        ##### Step 6: Print results
 
 
 
 

        print $OUT join("\t", @arr) . "\tavg " . sprintf("%.2f\t", $avg1) . "group " . join("", @groups) . "\n";

        print $OUT join("\t", @diff) . "\tavg " . sprintf("%.2f\t", $avg2) . "\n";

        print $OUT "\n";

}
 
 

        printf "input: maxavg %d minavg %d avgavg %d\n", $maxavg1, $minavg1, sum(@avgs1)/@avgs1;

        printf $OUT "For data values:\t\t avg avg %.2f\t maximum avg %.2f\t minimum avg %.2f \n",

                        sum(@avgs1)/@avgs1, $maxavg1, $minavg1 ;

        printf "diff: maxavg %d minavg %d avgavg %d\n", $maxavg2, $minavg2, sum(@avgs2)/@avgs2;

        printf $OUT "For data difference values:\t avg avg %.2f\t maximum avg %.2f\t minimum avg %.2f \n",

                        sum(@avgs2)/@avgs2, $maxavg2, $minavg2, ;
 
 
 

close($IN);

close($OUT);

Open in new window

0
 

Author Comment

by:MichaelGlancy
ID: 24835849
yes, you are right. Thank you. dont be confused, we did want it added, but have realised since that it affects the average in a false way, we didnt see that yesterday

thankyou
0
 

Author Closing Comment

by:MichaelGlancy
ID: 31602623
thank you :)
0
Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

 
LVL 40

Expert Comment

by:mrjoltcola
ID: 24835862
I only changed one line (besides reformatting a couple of lines that were too long into shorter lines)


Original:

my @diff = (0, map {$arr[$_] - $arr[$_-1]} (1..$#arr));

Changed:

my @diff = (map {$arr[$_] - $arr[$_-1]} (1..$#arr));


Actually also what ozo suggested above.
0
 

Author Comment

by:MichaelGlancy
ID: 24835871
if you were to right this in English, what would you be saying ?


my @diff = (0, map {$arr[$_] - $arr[$_-1]} (1..$#arr));
0
 
LVL 40

Expert Comment

by:mrjoltcola
ID: 24835882
diff list equals new list (0) plus list (for each number "i" in arr list from 1 to end, subtract value of number in "i - 1")
0
 
LVL 84

Expert Comment

by:ozo
ID: 24835893
my @diff = (map {$arr[$_] - $arr[$_-1]} (1..$#arr));
is what I said
0
 

Author Comment

by:MichaelGlancy
ID: 24835897
sorry ozo, I didnt understand what you did. I copied it into my script and couldnt get it to work for me.
0
 

Author Comment

by:MichaelGlancy
ID: 24835901
Im a complete beginner at this. I am scraping my way through with the help of you guys
0
 
LVL 40

Expert Comment

by:mrjoltcola
ID: 24835917
The difference in what ozo said:

(map {$arr[$_] - $arr[$_-1]} (1..$#arr));

and:

(0, map {$arr[$_] - $arr[$_-1]} (1..$#arr));


Is just the 1st element in the list. In Perl you can create list expressions out of other elements or lists by just listing comma delimited as:

(0, 1, 2, ...)
0
 

Author Comment

by:MichaelGlancy
ID: 24835920
should  I reassign points ?
0
 
LVL 84

Expert Comment

by:ozo
ID: 24835987
How did you copy it it into your script, and in what way did it not work?
0
 
LVL 84

Expert Comment

by:ozo
ID: 24835992
see
 List value constructors
in
perldoc perldata
0
 

Author Comment

by:MichaelGlancy
ID: 24835998
well I just copied it over, and as the program ran, the command prompt window just flashed past and there was no creation of the output file. At tha point I just went with mrjolt's code

I dont really know enough to say at this point. I really am learning this as I go along. Your code was fine, it is my inability that is the problem . .. .
0

Featured Post

What Security Threats Are You Missing?

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

Join & Write a Comment

Suggested Solutions

I've just discovered very important differences between Windows an Unix formats in Perl,at least 5.xx.. MOST IMPORTANT: Use Unix file format while saving Your script. otherwise it will have ^M s or smth likely weird in the EOL, Then DO NOT use m…
Checking the Alert Log in AWS RDS Oracle can be a pain through their user interface.  I made a script to download the Alert Log, look for errors, and email me the trace files.  In this article I'll describe what I did and share my script.
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
Sending a Secure fax is easy with eFax Corporate (http://www.enterprise.efax.com). First, Just open a new email message.  In the To field, type your recipient's fax number @efaxsend.com. You can even send a secure international fax — just include t…

706 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

23 Experts available now in Live!

Get 1:1 Help Now