Solved

Linux mkfifo

Posted on 2009-04-08
8
1,117 Views
Last Modified: 2012-05-06
I need to create a buffered fifo named pipe in linux.

I mean something like mkfifo but buffered. My needing is to avoid the writing process to wait for reading process.

Can it be done in Linux?

0
Comment
Question by:phoenix128
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
  • 2
8 Comments
 
LVL 79

Expert Comment

by:arnold
ID: 24098418
What is it you are trying to do?

You have to use Event handler on the writer to check whether there is data in the pipe and sleep or do something else.

I think a buffer would do exactly what you claim you do not want and could cause/lead to other problems.
http://modemhelp.net/faqs/fifo.shtml
 
0
 

Author Comment

by:phoenix128
ID: 24098548
I'm writing an application that reads a log file from another software and process it in realtime.

The problem is that my process can be slow sometimes and if I use a traditional fifo I will slow down the other software too.

I was thinking to something like a buffered fifo, but maybe there is even a better solution... any idea will be appreciated.
0
 
LVL 7

Expert Comment

by:Morne Lategan
ID: 24098637
how about perl's tail extension:

http://search.cpan.org/~mgrabnar/File-Tail-0.99.3/Tail.pm

or from bash:

tail -f /var/log/your.log | yourapp
0
How To Reduce Deployment Times With Pre-Baked AMIs

Even if we can't include all the files in the base image, we can sometimes include some of the larger files that we would otherwise have to download, and we can also sometimes remove the most time-consuming steps. This can help a lot with reducing deployment times.

 

Author Comment

by:phoenix128
ID: 24098711
tail is what I did last time, the problem is that I am using a tmpfs and allowing the master program to write a real file will expose me to size problems.
0
 
LVL 79

Accepted Solution

by:
arnold earned 500 total points
ID: 24098786
I am not sure how your application reading a log file will slow down the other "software" What other software?

How does your application access the log file? Does it start when the system boots and attaches to the file periodically does fseek to the next line etc.?
Or does your application starts based on a cron.  Attaches to the file checks what's there and exits?

In either case you have to add logic to handle log rotation.
I.e. last time you ran, the file was 500 lines. Now your process runs and the file is 200 lines long.  This would mean that the log file rotated which means you need to make sure that you processed all the data from the old file before starting with the new file.  This is easily done through maintaining information the files that exist in the directory location where the log files are stored.

File rotation is the way to get all the information you need while limiting the amount of storage used.  The only drawback is that you have to make sure that your processing of the log file is not longer than the full rotation cycle.
I.e. you allow for five file rotation before the first file is deleted.
Your application has to be done with the processing of the first file before that file has reached its end point on the rotation schedule.


0
 
LVL 7

Expert Comment

by:Morne Lategan
ID: 24098799
Hmm, here's a strange suggestion which but how you make your app exit after 100 lines, and then do this:

#!/bin/bash

while
  :
do
  tail -f /var/log/your.log | yourapp
  rm /var/log/your.log
done

0
 
LVL 7

Expert Comment

by:Morne Lategan
ID: 24098870
No wait that strange suggestion will cause problems in the app that writes the log. Scratch that please.
0
 

Author Closing Comment

by:phoenix128
ID: 31568087
At the end it seems the only way :)
0

Featured Post

Do you have a plan for Continuity?

It's inevitable. People leave organizations creating a gap in your service. That's where Percona comes in.

See how Pepper.com relies on Percona to:
-Manage their database
-Guarantee data safety and protection
-Provide database expertise that is available for any situation

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This is an article about my experiences with remote access to my clients (so that I may serve them) and eventually to my home office system via Radmin Remote Control. I have been using remote access for over 10 years and have been improving my metho…
If you use NetMotion Mobility on your PC and plan to upgrade to Windows 10, it may not work unless you take these steps.
Windows 10 is mostly good. However the one thing that annoys me is how many clicks you have to do to dial a VPN connection. You have to go to settings from the start menu, (2 clicks), Network and Internet (1 click), Click VPN (another click) then fi…
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.
Suggested Courses

617 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question