Unix ksh compare files

Posted on 2011-10-28
Last Modified: 2012-05-12
Hello Experts,

Each day an extract is created.
Want needs to happen is that the file created 'today' should be compared with the file created 'yesterday'.

Each file has these values:
Only SCRIPT (can) have spaces in it, not so the others.
So SCRIPT becomes everything between JOB (2nd column) and LOGON (4th column)
There is no header (but it could be added).

Example of file:
BEKD1859 MAEWSTLD "C:\Program Files\IBM\TWS\app_user\bin"\rmstdlist.cmd 10 app_user app_user
BEKD1859 MQSULOGFILEPURGER exit 0 app_user app_user
BEKD1859 MQSUSAVEQMGROBJECTS-TJPMQD01 exit 0 app_user app_user
BEKD1859 OTS-EXIT0-CMD exit 0 app_user pbu
BEKD1859 ROTATEDRFILES-TJPMQP01 exit 0 app_user app_user
BEKD1859 SEND-EMAIL-OBJECTS-TJPMQD01 exit 0 app_user app_user
BEKD1859 SEND-EMAIL-OBJECTS-TJPMQT01 exit 0 app_user app_user
BEKD1859 SEND-EMAIL-PURGE exit 0 app_user app_user
SANDBOX CREATEPOSTREPORTS /opt2/app_user/CreatePostReports app_user app_user
SANDBOX MAKEPLAN /opt2/app_user/MakePlan app_user app_user
SANDBOX OTS-EXIT0-CMD exit 0 app_user pbu
SANDBOX OTS-EXIT0-CMD1 exit 0 app_user pbu
SANDBOX OTS-EXIT0-CMD2 exit 0 app_user pbu
SANDBOX RMSCHDLG /tech/tws/scripts/rmschedlog root kna
SANDBOX RMSTDLST /tech/tws/scripts/rmstdlist app_user kna
SANDBOX STARTAPPSERVER /opt2/app_user/wastools/ app_user app_user
SANDBOX SWITCHPLAN /opt2/app_user/SwitchPlan app_user app_user
SANDBOX UPDATESTATS /opt2/app_user/UpdateStats app_user app_user

Open in new window

So if we take first line:
CPU = BEKD1859
SCRIPT = "C:\Program Files\IBM\TWS\app_user\bin"\rmstdlist.cmd 10
LOGON = app_user
CREATOR = app_user

- check and report on new entries (CPU JOB combination)
- check and report on removed entries (CPU JOB combination)
- check and report on changed values of existing entries (SCRIPT LOGON CREATOR)

Please be so kind to check this out.
Many thanks.
Question by:Watnog
    LVL 7

    Assisted Solution


    I am afraid I do not understand. You have to write a script. This script will "cat" or "more" the file. You can arrange the columns with the "awk" command.


    more your_file | awk '{print $1 $3}' 

    Open in new window

    The above command will read the file and pipe it to awk which in turn would display the first and third columns. You can display in any order you want: awk '{print $5 $1 $2 $3}'

    Another way would be (since each day you have a new file) to use the command "diff". You can use the command "sort" to sort out based on a chosen column.

    Read more about awk, diff and sort (by typing "man awk", "man diff", etc.).

    Good Luck!
    LVL 26

    Accepted Solution

    Per: icenick post, look / play with the diff command e.g.

    #Difference between the files.
    diff todays_file  yesterdays_file
    #Compare just the CPU and JOB entries
    cut -f1,2 -d' ' todays_file        | sort -u > todays_file.CPU_n_JOB
    cut -f1,2 -d' ' yesterdays_file | sort -u > yesterdays_file.CPU_n_JOB
    diff  todays_file.CPU_n_JOB yesterdays_file.CPU_n_JOB

    Open in new window


    Author Comment

    Yes I will. You got me a trail...

    Author Closing Comment

    Sorry for late reply, and yes this was good enough info to get in rolling. Thanks!

    Featured Post

    How your wiki can always stay up-to-date

    Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
    - Increase transparency
    - Onboard new hires faster
    - Access from mobile/offline

    Join & Write a Comment

    Let's say you need to move the data of a file system from one partition to another. This generally involves dismounting the file system, backing it up to tapes, and restoring it to a new partition. You may also copy the file system from one place to…
    FreeBSD on EC2 FreeBSD ( is a robust Unix-like operating system that has been around for many years. FreeBSD is available on Amazon EC2 through Amazon Machine Images (AMIs) provided by FreeBSD developer and security office…
    Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
    This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.

    728 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    19 Experts available now in Live!

    Get 1:1 Help Now