how to get the selective text out with linux sh script

jl66
jl66 used Ask the Experts™
on
Would like to select part of text in a text file as follows:
1) first get the whole file. The file grows with time.
2) later (say after 1 hour ) only get the part which previously was not got.
With shell commands ( and awk), is it possible?
Could any gurus shed some light on it?
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Most Valuable Expert 2013
Top Expert 2013
Commented:
You could copy the file away in the desired interval, then compare the content of the copies with "comm" to display just the part being present only in the newer file:

comm -1 -3 oldfile newfile > deltafile

Using

diff oldfile newfile

is also an option, but you'll get the result in "diff" format, of course.

wmp
An alternative would be to remember where you were, and start from that point next time.

Initialise a memory file:
  echo 1 > /tmp/file_len.log

The run this from time to time:

    startline=$(cat /tmp/file_len.log)
    if [ -z "startline" ]; then
        startline=0
    fi
    wc -l < log_file > /tmp/file_len
    sed -n ${startline},\$p log_file > tmp_log_file
    # now process tmp_log_file
jl66Consultant

Author

Commented:
Thanks a lot. Very helpful. Sorry to forget to reward the points.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial