• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 146
  • Last Modified:

Write to file using tail -f through a pipe to grep

Hi -- I'm looking to write to a file after piping output from tail -f through to grep:

#write to a file for all lines with "searchtext" within in error_log:
tail -f /var/error_log | grep searchtext > output.txt

The above command fails even though the following command::

tail -f /var/error_log | grep searchtext

does produce output, as expected.  Nothing is getting through while redirecting to files after a tail -f...  The following work:

tail -f /var/error_log > output.txt                //without grep
tail /var/error_log | grep searchtext > output.txt   //without -f

So it's just the combination of grep and tail with specifically the -f (following) option that causes the problem.  It would be ideal to be able to do this with a single inline command.  Any ideas?
0
minnirok
Asked:
minnirok
2 Solutions
 
TintinCommented:
You could do

tail -f /var/error_log | grep searchtext | tee >output.txt
0
 
ozoCommented:
the problem may be that the pipe is buffering the output
0
 
edster9999Commented:
Grep is buffering the content and not delivering it until it has filled the buffer or got to end of file (which it will not do)

tail -f /var/error_log | grep  --line-buffered searchtext > output.txt

This may not work on very old copies of grep.


0
 
devil_himselfCommented:
bash is not involved in buffering the output of processes it spawns nor can be controlled from the shell.The program itself determines whether or how much buffering is performed or when the output buffer is flushed.

tail -f "file" | grep "text"

tail command buffers its output, and therefore grep only gets its input until there is a certain amount (typically 2k or 4k) or until the buffer is flushed or the tail process terminates.

tail -f "file" | grep "text" > logfile

If output is going to a 'terminal', then the buffer is flushed on every newline character, but this does not happen if it is going to a file. if the output file is a pipe their is no line buffering.

to work around it use a 'while read'  loop
tail -f file | while read line ; do echo "$line" grep 'text' >> logfile ; done & 

or you can use 'unbuffer' utility which disables the output buffering.
http://linux.die.net/man/1/unbuffer
0
 
minnirokAuthor Commented:
Apologize for not buffering my response in a timely fashion.  Great answers, guys!
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Cloud Class® Course: Amazon Web Services - Basic

Are you thinking about creating an Amazon Web Services account for your business? Not sure where to start? In this course you’ll get an overview of the history of AWS and take a tour of their user interface.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now