Need script to do 'diff' comparison between files

Currently I am using an script that does an in-depth audit of a RHEL server platform. The script works fine and places it in the directory location without any problems. However, new scope requirements have just been added to the project and I am a bit stuck.

I need to do a 'diff' between the files created, but the problem is that the files are created on a weekly basis, thus their date stamp changes. This also means that at the start of a new month, the script will need to look back at the week before and compare against that file as well.

Example files output:
RHEL-servername.domain.com.audit.03-26-10.txt
RHEL-servername.domain.com.audit.04-01-10.txt
RHEL-servername.domain.com.audit.04-16-10.txt


When it came to the script, I was thinking of using something like this:

1) List the directory contents:

-rw-r--r-- 1 root root 0 Apr 20 12:29 RHEL-server1.domain.local.audit.03-26-10.txt
-rw-r--r-- 1 root root 0 Apr 20 12:29 RHEL-server1.domain.local.audit.04-02-10.txt
-rw-r--r-- 1 root root 0 Apr 20 12:29 RHEL-server1.domain.local.audit.04-09-10.txt

2) Extract just the file names via awk statement: ls -l | awk '{ print $9 };' | awk '{ print $1 };'                    

RHEL-server1.domain.local.audit.03-26-10.txt
RHEL-server1.domain.local.audit.04-02-10.txt
RHEL-server1.domain.local.audit.04-09-10.txt

3) Extract the date stamp per file name, excluding periods: ls -l | awk '{ print $9 };' | awk 'BEGIN { FS = "." } ; { print $5 };'

03-26-10
04-02-10
04-09-10


And this is where I get stuck. Having the script actually know which dates to use for the actual comparison (last date + previous week date) is where I am getting script block.

Anyone have an idea on how to solve this? I am using bash/shell script right now, but if someone has a Perl alternative I would likely use it as well (even though my perl experience is pretty much non-existent).
LVL 29
Michael WorshamStaff Infrastructure ArchitectAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

wilcoxonCommented:
For perl, this should work:
use strict;
use warnings;
opendir DIR, '.' or die "could not open dir: $!";
my @files = sort rev_date grep m{^RHEL-.*\.com\.audit\.\d+-\d+-\d+\.txt$}, readdir DIR;
closedir DIR;

my $newest = shift @files;
my $next_newest = shift @files;
system('diff', $next_newest, $newest);

sub rev_date {
    # multiple diff ways to compare dates
    # could use module such as Date::Manip to parse/compare
    # could do numeric compares on each component
    # since we just care about order, just do a mathematic compare
    $a =~ m{\.(\d+)-(\d+)-(\d+)\.txt$};
    my ($mma, $dda, $yra) = ($1, $2, $3);
    my $vala = $yra * 400 + $mma * 33 + $dda;
    $b =~ m{\.(\d+)-(\d+)-(\d+)\.txt$};
    my ($mmb, $ddb, $yrb) = ($1, $2, $3);
    my $valb = $yrb * 400 + $mmb * 33 + $ddb;
    return $valb <=> $vala; # get reverse order
}

Open in new window

0
Michael WorshamStaff Infrastructure ArchitectAuthor Commented:
Do note that my skills in Perl are non-existent, but this is the response I am getting...

Use of uninitialized value in system at ./rhel_audit_diff.pl line 11.
Use of uninitialized value in system at ./rhel_audit_diff.pl line 11.
diff: : No such file or directory
diff: : No such file or directory

Tried changing to the following, but I get the same result as above:

system('/usr/bin/diff', $next_newest, $newest);
system("diff", $next_newest, $newest);
system("/usr/bin/diff", $next_newest, $newest);
0
Maciej SsysadminCommented:
It would be much simpler (and prettier ;)) if you use year-month-day timestamp in filenames instead of month-day-year.
#!/bin/sh

FILENAME=RHEL-servername.domain.com.audit.

# this is just for finding the last file
DATE2=0
for FILE in RHEL*; do
        DATE=`echo ${FILE} | sed 's/.*audit\.\([0-9][0-9]\)-\([0-9][0-9]\)-\([0-9][0-9]\)\.txt/20\3\1\2/'`
        test "${DATE}" -gt "${DATE2}" && DATE2=${DATE}
done

DATE_PREV=`date -d "${DATE2} -1 week" +%m-%d-%y`

FILE_LAST=${FILENAME}`date -d "${DATE2}" +%m-%d-%y`.txt
FILE_PREV=${FILENAME}${DATE_PREV}.txt

diff ${FILE_LAST} ${FILE_PREV}

Open in new window

0
Determine the Perfect Price for Your IT Services

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden with our free interactive tool and use it to determine the right price for your IT services. Download your free eBook now!

Michael WorshamStaff Infrastructure ArchitectAuthor Commented:
@oklit: The filename structure is not my call. It is named as per NIST/FIPS/STIG compliance standards.
0
Michael WorshamStaff Infrastructure ArchitectAuthor Commented:
@oklit:

The FILENAME=RHEL-servername.domain.com.audit. is defined just for one server. How would I do it if I had several servers to parse/compare against?

I am attaching a sample run of the original audit script so you can see the output I am having to compare against.

RHEL-reaper.domain.local.audit.0.txt
0
Maciej SsysadminCommented:
So, do you have something like:
RHEL-hostname1.audit.date1.txt
RHEL-hostname1.audit.date2.txt
RHEL-hostname2.audit.date1.txt
RHEL-hostname2.audit.date2.txt
...
?
0
Michael WorshamStaff Infrastructure ArchitectAuthor Commented:
@oklit: Yes
0
Maciej SsysadminCommented:

#!/bin/sh

for FILENAME in `find . -type f -name "RHEL*" | sed 's/\(RHEL.*\.audit\.\).*/\1/' | sort -u`; do
        # this is just for finding the last file
        DATE2=0
        for FILE in RHEL*; do
                DATE=`echo ${FILE} | sed 's/.*audit\.\([0-9][0-9]\)-\([0-9][0-9]\)-\([0-9][0-9]\)\.txt/20\3\1\2/'`
                test "${DATE}" -gt "${DATE2}" && DATE2=${DATE}
        done

        DATE_PREV=`date -d "${DATE2} -1 week" +%m-%d-%y`

        FILE_LAST=${FILENAME}`date -d "${DATE2}" +%m-%d-%y`.txt
        FILE_PREV=${FILENAME}${DATE_PREV}.txt

        diff ${FILE_LAST} ${FILE_PREV}
done

Open in new window

0
Maciej SsysadminCommented:
Ooops.. forgot to change one thing. Below correct version.
#!/bin/sh

for FILENAME in `find . -type f -name "RHEL*" | sed 's/\(RHEL.*\.audit\.\).*/\1/' | sort -u`; do
        # this is just for finding the last file
        DATE2=0
        for FILE in ${FILENAME}*; do
                DATE=`echo ${FILE} | sed 's/.*audit\.\([0-9][0-9]\)-\([0-9][0-9]\)-\([0-9][0-9]\)\.txt/20\3\1\2/'`
                test "${DATE}" -gt "${DATE2}" && DATE2=${DATE}
        done

        DATE_PREV=`date -d "${DATE2} -1 week" +%m-%d-%y`

        FILE_LAST=${FILENAME}`date -d "${DATE2}" +%m-%d-%y`.txt
        FILE_PREV=${FILENAME}${DATE_PREV}.txt

        echo diff ${FILE_LAST} ${FILE_PREV}
done

Open in new window

0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Michael WorshamStaff Infrastructure ArchitectAuthor Commented:
@oklit: Your script works as intended. Thanks.

But since I am trying to make this a function inside of the original auditing script, it seems I will have to take a different approach as the final result needs to be automated as much as possible. I will open another question and you are welcome to try and come up with a solution there.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Shell Scripting

From novice to tech pro — start learning today.