Solved

Linux mkfifo

Posted on 2009-04-08
8
1,092 Views
Last Modified: 2012-05-06
I need to create a buffered fifo named pipe in linux.

I mean something like mkfifo but buffered. My needing is to avoid the writing process to wait for reading process.

Can it be done in Linux?

0
Comment
Question by:phoenix128
  • 3
  • 3
  • 2
8 Comments
 
LVL 76

Expert Comment

by:arnold
Comment Utility
What is it you are trying to do?

You have to use Event handler on the writer to check whether there is data in the pipe and sleep or do something else.

I think a buffer would do exactly what you claim you do not want and could cause/lead to other problems.
http://modemhelp.net/faqs/fifo.shtml
 
0
 

Author Comment

by:phoenix128
Comment Utility
I'm writing an application that reads a log file from another software and process it in realtime.

The problem is that my process can be slow sometimes and if I use a traditional fifo I will slow down the other software too.

I was thinking to something like a buffered fifo, but maybe there is even a better solution... any idea will be appreciated.
0
 
LVL 7

Expert Comment

by:Morne Lategan
Comment Utility
how about perl's tail extension:

http://search.cpan.org/~mgrabnar/File-Tail-0.99.3/Tail.pm

or from bash:

tail -f /var/log/your.log | yourapp
0
 

Author Comment

by:phoenix128
Comment Utility
tail is what I did last time, the problem is that I am using a tmpfs and allowing the master program to write a real file will expose me to size problems.
0
Complete Microsoft Windows PC® & Mac Backup

Backup and recovery solutions to protect all your PCs & Mac– on-premises or in remote locations. Acronis backs up entire PC or Mac with patented reliable disk imaging technology and you will be able to restore workstations to a new, dissimilar hardware in minutes.

 
LVL 76

Accepted Solution

by:
arnold earned 500 total points
Comment Utility
I am not sure how your application reading a log file will slow down the other "software" What other software?

How does your application access the log file? Does it start when the system boots and attaches to the file periodically does fseek to the next line etc.?
Or does your application starts based on a cron.  Attaches to the file checks what's there and exits?

In either case you have to add logic to handle log rotation.
I.e. last time you ran, the file was 500 lines. Now your process runs and the file is 200 lines long.  This would mean that the log file rotated which means you need to make sure that you processed all the data from the old file before starting with the new file.  This is easily done through maintaining information the files that exist in the directory location where the log files are stored.

File rotation is the way to get all the information you need while limiting the amount of storage used.  The only drawback is that you have to make sure that your processing of the log file is not longer than the full rotation cycle.
I.e. you allow for five file rotation before the first file is deleted.
Your application has to be done with the processing of the first file before that file has reached its end point on the rotation schedule.


0
 
LVL 7

Expert Comment

by:Morne Lategan
Comment Utility
Hmm, here's a strange suggestion which but how you make your app exit after 100 lines, and then do this:

#!/bin/bash

while
  :
do
  tail -f /var/log/your.log | yourapp
  rm /var/log/your.log
done

0
 
LVL 7

Expert Comment

by:Morne Lategan
Comment Utility
No wait that strange suggestion will cause problems in the app that writes the log. Scratch that please.
0
 

Author Closing Comment

by:phoenix128
Comment Utility
At the end it seems the only way :)
0

Featured Post

6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

Join & Write a Comment

Juniper VPN devices are a popular alternative to using Cisco products. Last year I needed to set up an international site-to-site VPN over the Internet, but the client had high security requirements -- FIPS 140. What and Why of FIPS 140 Federa…
Setting up Secure Ubuntu server on VMware 1.      Insert the Ubuntu Server distribution CD or attach the ISO of the CD which is in the “Datastore”. Note that it is important to install the x64 edition on servers, not the X86 editions. 2.      Power on th…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

9 Experts available now in Live!

Get 1:1 Help Now