Solved

Perl sort question inside of a for loop

Posted on 2016-11-21
10
43 Views
Last Modified: 2016-11-27
I'm trying to sort a list of data that I'm reading from a file and putting inside of an array. I want to sort the data by date and status.  I'm having trouble sorting because I think there's nothing to compare so it just spits it out as it's read.  I don't know if I need to pass the data somewhere first before I'm able to sort it or finish the loop then sort it. In any case, please help and let me know if you need more clarification.

Thanks!

.....

for my $i ("01" .. "30") {
        `download files`;
        open(my $fh, '< ', $download_path.'/'.$file.'-'.$i) or die("Could not open file $file-$i: $!.");

        while (my @row = <$fh>) {
                chomp @row;
                @arr = split ' ', $row[0];
        }

        my ($date, $path, $level, $num, $status) = @arr[2,4,7,8,9];
        my @pop_path = split '/', $first[4];
        my $title = $pop_path[3];
        my $day = UnixDate($date, "%Y-%m-%d %H:%M:%S");

        @sorted = (
        {
                date => $day,
                title => $title,
                path => $path,
                level => $level,
                num => $num,
                status => $status,
        }
        );

        @sorted = sort {
                $b->{date} cmp $a->{date}
        } @sorted;

        foreach my $line (@sorted){
                printf "%-22s %-15s %-4s %-2s %-5s %-30s\n" => @{$line}{qw(date title level num status path)};
        }

        close($fh);
}

.....
0
Comment
Question by:akatsuki27
  • 4
  • 4
  • 2
10 Comments
 
LVL 19

Expert Comment

by:Kim Ryan
Comment Utility
As you are just using standard date stamps, you could use a simple utility such as 'sort' on linux or cygwin to do a multi column sort. Or import your file to something like Excel and do a multi column sort there.
0
 
LVL 28

Expert Comment

by:FishMonger
Comment Utility
Your code has a number of problems, but instead of rewriting your code I'll point you to a youtube video of a YAPC talk about the use of map, grep and sort.  The sort portion of the talk gives at least one example of sorting by date fields.
0
 
LVL 19

Expert Comment

by:Kim Ryan
Comment Utility
Can you supply a sample of your input file? I think this could be solved with a sort one-liner if that's any help.
0
 

Author Comment

by:akatsuki27
Comment Utility
Here's a sample of what's being read from the input:

20161113               title1       lev  1  comp  /some/path    
20161114               title2       lev  1  comp  /some/path  
20161115               title1       lev  2  comp  /some/path    
20161116               title1       lev  3  fail      /some/path

I want to be able to sort by date and then by status (5th column) if the date is the same. Does the sort need to be outside the loop or what?  I'm not an experienced coder, I just do small scripts like these for certain tasks so I'm sure there are probably cleaner ways to do this.
0
 
LVL 19

Expert Comment

by:Kim Ryan
Comment Utility
If you are on Unix type system, you could just do this
sort -k1 -k5 input.txt > sorted.txt
Assuming you have spaces between columns. Use the -t option to specify another field delimiter.
0
6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

 

Author Comment

by:akatsuki27
Comment Utility
If it's okay, can you suggest a solution using Perl? The sorting of the data is just one step to continue manipulating the data for another purpose. I just got stuck in that part of the script so I came here for some direction using Perl only, please.  Also, I'd rather not have to make too many system calls because I'm testing with a small amount of data.  Once the script is live, the amount of data processed will be huge and the script will be too taxing.  Btw, yes this will run on a *nix system.
0
 
LVL 28

Assisted Solution

by:FishMonger
FishMonger earned 500 total points
Comment Utility
Did your watch the youtube video I pointed you to?

Instead of manually writing the sorting code our selves as shown in the video, we could use one of the soting modules on cpan.
Sort::Fields
#!/usr/bin/perl

use warnings;
use strict;
use Sort::Fields;
use Data::Dumper;

chomp(my @unsorted = <DATA>);
my @sorted = fieldsort ['1n', 4], @unsorted;

print Dumper \@sorted;

__DATA__
20161114               title2       lev  1  comp  /some/path
20161113               title1       lev  1  comp  /some/path
20161116               title1       lev  3  fail  /some/path
20161115               title1       lev  2  comp  /some/path

Open in new window


c:\test>test.pl

$VAR1 = [
          '20161113               title1       lev  1  comp  /some/path',
          '20161114               title2       lev  1  comp  /some/path',
          '20161115               title1       lev  2  comp  /some/path',
          '20161116               title1       lev  3  fail  /some/path'
        ];

EDIT:
I didn't realize how old that module was (1998) until after I posted, so it's not the best choice, but it does work.  There are more up-to-date modules available, or you can write the sort code yourself, which is pretty easy.
0
 

Accepted Solution

by:
akatsuki27 earned 0 total points
Comment Utility
Thanks, FishMonger.  Honestly, I only saw part of the video since it was late at night and it's close to an hour.  I will check it out completely later though because I'm interested in learning more Perl.

But I was able to figure it out by writing it out myself. I pushed the incoming data into an array and sorted the array outside of the for loop. I put the print statement outside the loop also. Not pretty but it's working. The change in bold below:

....
        my @temp = (
        {
                date => $day,
                title => $title,
                path => $path,
                level => $level,
                num => $num,
                status => $status,
        }
        );

       foreach my $line (@temp){
                push (@sorted, $line);
        }


        close($fh);
}

@sorted = sort {  
        $a->{lnum} cmp $b->{num} ||
        $b->{date} <=> $a->{date}
} @sorted;

foreach my $sort (@sorted){
        printf "%-22s %-15s %-4s %-2s %-5s %-30s\n" => @{$sort}{qw(date title level num status path)};
}

.....
0
 
LVL 19

Expert Comment

by:Kim Ryan
Comment Utility
Glad you worked out a Perl solution. You can also use awk, it's well suited to this type of problem?
0
 

Author Closing Comment

by:akatsuki27
Comment Utility
I wasn't given a satisfactory answer before I figured it out on my own.
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

I've just discovered very important differences between Windows an Unix formats in Perl,at least 5.xx.. MOST IMPORTANT: Use Unix file format while saving Your script. otherwise it will have ^M s or smth likely weird in the EOL, Then DO NOT use m…
I have been pestered over the years to produce and distribute regular data extracts, and often the request have explicitly requested the data be emailed as an Excel attachement; specifically Excel, as it appears: CSV files confuse (no Red or Green h…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

8 Experts available now in Live!

Get 1:1 Help Now