how to get the selective text out with linux sh script

jl66 used Ask the Experts™
Would like to select part of text in a text file as follows:
1) first get the whole file. The file grows with time.
2) later (say after 1 hour ) only get the part which previously was not got.
With shell commands ( and awk), is it possible?
Could any gurus shed some light on it?
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Most Valuable Expert 2013
Top Expert 2013
You could copy the file away in the desired interval, then compare the content of the copies with "comm" to display just the part being present only in the newer file:

comm -1 -3 oldfile newfile > deltafile


diff oldfile newfile

is also an option, but you'll get the result in "diff" format, of course.

An alternative would be to remember where you were, and start from that point next time.

Initialise a memory file:
  echo 1 > /tmp/file_len.log

The run this from time to time:

    startline=$(cat /tmp/file_len.log)
    if [ -z "startline" ]; then
    wc -l < log_file > /tmp/file_len
    sed -n ${startline},\$p log_file > tmp_log_file
    # now process tmp_log_file


Thanks a lot. Very helpful. Sorry to forget to reward the points.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial