Need script to do 'diff' comparison between files

Currently I am using an script that does an in-depth audit of a RHEL server platform. The script works fine and places it in the directory location without any problems. However, new scope requirements have just been added to the project and I am a bit stuck.

I need to do a 'diff' between the files created, but the problem is that the files are created on a weekly basis, thus their date stamp changes. This also means that at the start of a new month, the script will need to look back at the week before and compare against that file as well.

Example files output:
RHEL-servername.domain.com.audit.03-26-10.txt
RHEL-servername.domain.com.audit.04-01-10.txt
RHEL-servername.domain.com.audit.04-16-10.txt


When it came to the script, I was thinking of using something like this:

1) List the directory contents:

-rw-r--r-- 1 root root 0 Apr 20 12:29 RHEL-server1.domain.local.audit.03-26-10.txt
-rw-r--r-- 1 root root 0 Apr 20 12:29 RHEL-server1.domain.local.audit.04-02-10.txt
-rw-r--r-- 1 root root 0 Apr 20 12:29 RHEL-server1.domain.local.audit.04-09-10.txt

2) Extract just the file names via awk statement: ls -l | awk '{ print $9 };' | awk '{ print $1 };'                    

RHEL-server1.domain.local.audit.03-26-10.txt
RHEL-server1.domain.local.audit.04-02-10.txt
RHEL-server1.domain.local.audit.04-09-10.txt

3) Extract the date stamp per file name, excluding periods: ls -l | awk '{ print $9 };' | awk 'BEGIN { FS = "." } ; { print $5 };'

03-26-10
04-02-10
04-09-10


And this is where I get stuck. Having the script actually know which dates to use for the actual comparison (last date + previous week date) is where I am getting script block.

Anyone have an idea on how to solve this? I am using bash/shell script right now, but if someone has a Perl alternative I would likely use it as well (even though my perl experience is pretty much non-existent).
LVL 29
Michael WorshamInfrastructure / Solutions ArchitectAsked:
Who is Participating?
 
Maciej SsysadminCommented:
Ooops.. forgot to change one thing. Below correct version.
#!/bin/sh

for FILENAME in `find . -type f -name "RHEL*" | sed 's/\(RHEL.*\.audit\.\).*/\1/' | sort -u`; do
        # this is just for finding the last file
        DATE2=0
        for FILE in ${FILENAME}*; do
                DATE=`echo ${FILE} | sed 's/.*audit\.\([0-9][0-9]\)-\([0-9][0-9]\)-\([0-9][0-9]\)\.txt/20\3\1\2/'`
                test "${DATE}" -gt "${DATE2}" && DATE2=${DATE}
        done

        DATE_PREV=`date -d "${DATE2} -1 week" +%m-%d-%y`

        FILE_LAST=${FILENAME}`date -d "${DATE2}" +%m-%d-%y`.txt
        FILE_PREV=${FILENAME}${DATE_PREV}.txt

        echo diff ${FILE_LAST} ${FILE_PREV}
done

Open in new window

0
 
wilcoxonCommented:
For perl, this should work:
use strict;
use warnings;
opendir DIR, '.' or die "could not open dir: $!";
my @files = sort rev_date grep m{^RHEL-.*\.com\.audit\.\d+-\d+-\d+\.txt$}, readdir DIR;
closedir DIR;

my $newest = shift @files;
my $next_newest = shift @files;
system('diff', $next_newest, $newest);

sub rev_date {
    # multiple diff ways to compare dates
    # could use module such as Date::Manip to parse/compare
    # could do numeric compares on each component
    # since we just care about order, just do a mathematic compare
    $a =~ m{\.(\d+)-(\d+)-(\d+)\.txt$};
    my ($mma, $dda, $yra) = ($1, $2, $3);
    my $vala = $yra * 400 + $mma * 33 + $dda;
    $b =~ m{\.(\d+)-(\d+)-(\d+)\.txt$};
    my ($mmb, $ddb, $yrb) = ($1, $2, $3);
    my $valb = $yrb * 400 + $mmb * 33 + $ddb;
    return $valb <=> $vala; # get reverse order
}

Open in new window

0
 
Michael WorshamInfrastructure / Solutions ArchitectAuthor Commented:
Do note that my skills in Perl are non-existent, but this is the response I am getting...

Use of uninitialized value in system at ./rhel_audit_diff.pl line 11.
Use of uninitialized value in system at ./rhel_audit_diff.pl line 11.
diff: : No such file or directory
diff: : No such file or directory

Tried changing to the following, but I get the same result as above:

system('/usr/bin/diff', $next_newest, $newest);
system("diff", $next_newest, $newest);
system("/usr/bin/diff", $next_newest, $newest);
0
Cloud Class® Course: Microsoft Exchange Server

The MCTS: Microsoft Exchange Server 2010 certification validates your skills in supporting the maintenance and administration of the Exchange servers in an enterprise environment. Learn everything you need to know with this course.

 
Maciej SsysadminCommented:
It would be much simpler (and prettier ;)) if you use year-month-day timestamp in filenames instead of month-day-year.
#!/bin/sh

FILENAME=RHEL-servername.domain.com.audit.

# this is just for finding the last file
DATE2=0
for FILE in RHEL*; do
        DATE=`echo ${FILE} | sed 's/.*audit\.\([0-9][0-9]\)-\([0-9][0-9]\)-\([0-9][0-9]\)\.txt/20\3\1\2/'`
        test "${DATE}" -gt "${DATE2}" && DATE2=${DATE}
done

DATE_PREV=`date -d "${DATE2} -1 week" +%m-%d-%y`

FILE_LAST=${FILENAME}`date -d "${DATE2}" +%m-%d-%y`.txt
FILE_PREV=${FILENAME}${DATE_PREV}.txt

diff ${FILE_LAST} ${FILE_PREV}

Open in new window

0
 
Michael WorshamInfrastructure / Solutions ArchitectAuthor Commented:
@oklit: The filename structure is not my call. It is named as per NIST/FIPS/STIG compliance standards.
0
 
Michael WorshamInfrastructure / Solutions ArchitectAuthor Commented:
@oklit:

The FILENAME=RHEL-servername.domain.com.audit. is defined just for one server. How would I do it if I had several servers to parse/compare against?

I am attaching a sample run of the original audit script so you can see the output I am having to compare against.

RHEL-reaper.domain.local.audit.0.txt
0
 
Maciej SsysadminCommented:
So, do you have something like:
RHEL-hostname1.audit.date1.txt
RHEL-hostname1.audit.date2.txt
RHEL-hostname2.audit.date1.txt
RHEL-hostname2.audit.date2.txt
...
?
0
 
Michael WorshamInfrastructure / Solutions ArchitectAuthor Commented:
@oklit: Yes
0
 
Maciej SsysadminCommented:

#!/bin/sh

for FILENAME in `find . -type f -name "RHEL*" | sed 's/\(RHEL.*\.audit\.\).*/\1/' | sort -u`; do
        # this is just for finding the last file
        DATE2=0
        for FILE in RHEL*; do
                DATE=`echo ${FILE} | sed 's/.*audit\.\([0-9][0-9]\)-\([0-9][0-9]\)-\([0-9][0-9]\)\.txt/20\3\1\2/'`
                test "${DATE}" -gt "${DATE2}" && DATE2=${DATE}
        done

        DATE_PREV=`date -d "${DATE2} -1 week" +%m-%d-%y`

        FILE_LAST=${FILENAME}`date -d "${DATE2}" +%m-%d-%y`.txt
        FILE_PREV=${FILENAME}${DATE_PREV}.txt

        diff ${FILE_LAST} ${FILE_PREV}
done

Open in new window

0
 
Michael WorshamInfrastructure / Solutions ArchitectAuthor Commented:
@oklit: Your script works as intended. Thanks.

But since I am trying to make this a function inside of the original auditing script, it seems I will have to take a different approach as the final result needs to be automated as much as possible. I will open another question and you are welcome to try and come up with a solution there.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.