Solved

Sort a file with date - linux

Posted on 2016-11-03
7
28 Views
Last Modified: 2016-11-09
I have a file which contents below data (this file is date-sorted already , you can see first column)

2015-03-19 billing-gateway
2015-03-23 btnetdesign-greenfield
2015-06-01 bigbird1
2015-10-07 bigbird
2015-11-03 expedioomrm
2016-02-17 RositaSR
2016-03-23 ocsnet
2016-06-29 or-vmj
2016-06-30 perftest1_200
2016-06-30 perftest2_0
2016-06-30 perftest3_25
2016-06-30 perftest4_500
2016-06-30 perftest5_1000
2016-10-03 admin
2016-10-03 test-repo-03102016-2
2016-10-17 cmps
2016-11-02 VMM_Genesys

Need to display rows which are before 6 months date
like today we have 03/11/2016 then need to extract those rows only which are older than 2016-05-03
So in above data , from 2016-03-23 ocsnet backwards dates all rows

So logically it should calculate with current date and throw out the older than six months rows only
0
Comment
Question by:Ravi Raj
7 Comments
 
LVL 77

Expert Comment

by:arnold
ID: 41872580
Using perl, timelocal convert the date string to UNIX timestamp since epoch.
Then you can do logical test 24*3600*182
Use time to get current timestamp, ...........
1
 
LVL 26

Expert Comment

by:wilcoxon
ID: 41872667
Another alternative is simply to pattern match.  Here's how you'd do it in perl.
use strict;
use warnings;
my ($yr, $mon, $day) = (localtime)[5,4,3];
$mon -= 5;  # start with the full month to be excluded
if ($mon < 1) {
    $mon += 12;
    $yr--;
}
my $mrx = "$yr-";
if ($mon < 10) {
    $mrx = "(?:0(?:$mon-9)|1[0-2])-";
} else {
    my $tmp = $mon - 10;
    $mrx = "1[$tmp-2]-";
}
$mon--;
if ($mon < 1) {
    $mon += 12;
    $yr--;
}
my $drx;
if ($day < 10) {
    $drx = "(?:0[$day-9]|[123][0-9])";
} else {
    my @dig = split //, $day;
    my $nxt = $dig[0]+1;
    $drx = "(?:$dig[0][$dig[1]-9]|[$nxt-3][0-9])";
}
$drx = sprintf '%d-%02d-%s', $yr, $mon, $drx;
open IN, 'input_file.log' or die "could not open input: $!";
while (<IN>) {
    last if (m{^$mrx} or m{^$drx});
    print;
}

Open in new window

1
 

Accepted Solution

by:
Ravi Raj earned 0 total points
ID: 41873938
Thanks Arnold , Wilcoxon
I wrote a simple perl which works perfect for me

use Time::Piece;
$num_args = $#ARGV + 1;
if ($num_args eq 1) {
    $entryDate = $ARGV[0];
        while (defined(my $file = glob 'myfile.csv')) {
  open my $fh, "<", $file;  # lexical file handles, automatic error handling

  LINE: while (defined( my $line = <$fh> )) {

                s/, /,/;
                s/\"[^\"]+"//;
                @x = split(" ", $line);
                $date = $x[0];
$date1 = Time::Piece->strptime($date, "%Y-%m-%d");
$date2 = Time::Piece->strptime($entryDate, "%Y-%m-%d");

                if($date1 < $date2) {
print $line;
                }
}
}
}
0
DevOps Toolchain Recommendations

Read this Gartner Research Note and discover how your IT organization can automate and optimize DevOps processes using a toolchain architecture.

 
LVL 13

Expert Comment

by:F Igor
ID: 41873952
To Get the min date

MINDATE=$(date -d "-6 months" +%Y-%m-%d

Read all lines and only print lines "greater" than MINDATE:

for line in (names.txt)
do
   if ( "$line" > "$MINDATE" ) ; then
     echo $line
  fi
done

Open in new window

0
 
LVL 26

Expert Comment

by:skullnobrains
ID: 41874109
if your file is sorted, you probably should not check the dates for each line : either read the file until you find the fist date that is more recent that 6 months and exit, or read it in reverse order and start printing everything after the first date

you can code this in pure shell

see this example that reads the file in reverse order
$ tac /tmp/in | { while read date line ; do test $(date --date "$date"  +%s) -gt $(date --date '-1 month' +%s ) && break ; done ; cat ; }
2016-10-17 cmps
2016-10-03 test-repo-03102016-2
2016-10-03 admin
2016-06-30 perftest5_1000
2016-06-30 perftest4_500
2016-06-30 perftest3_25
2016-06-30 perftest2_0
2016-06-30 perftest1_200
2016-06-29 or-vmj
2016-03-23 ocsnet
2016-02-17 RositaSR
2015-11-03 expedioomrm
2015-10-07 bigbird
2015-06-01 bigbird1

Open in new window


or in actual order
$ cat /tmp/in | while read date line ; do test $(date --date "$date"  +%s) -lt $(date --date '-1 month' +%s ) || exit ; echo $date $line ; done
2015-06-01 bigbird1
2015-10-07 bigbird
2015-11-03 expedioomrm
2016-02-17 RositaSR
2016-03-23 ocsnet
2016-06-29 or-vmj
2016-06-30 perftest1_200
2016-06-30 perftest2_0
2016-06-30 perftest3_25
2016-06-30 perftest4_500
2016-06-30 perftest5_1000
2016-10-03 admin
2016-10-03 test-repo-03102016-2

Open in new window

0
 
LVL 77

Expert Comment

by:arnold
ID: 41874328
Why not award points to Wilcoxon, myself and others whose suggestions helped you resolve the issue?
1
 

Author Closing Comment

by:Ravi Raj
ID: 41880269
Wrote my own script
0

Featured Post

3 Use Cases for Connected Systems

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, testing some more, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Checking the Alert Log in AWS RDS Oracle can be a pain through their user interface.  I made a script to download the Alert Log, look for errors, and email me the trace files.  In this article I'll describe what I did and share my script.
Join Greg Farro and Ethan Banks from Packet Pushers (http://packetpushers.net/podcast/podcasts/pq-show-93-smart-network-monitoring-paessler-sponsored/) and Greg Ross from Paessler (https://www.paessler.com/prtg) for a discussion about smart network …
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
The viewer will learn how to count occurrences of each item in an array.

895 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now