• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 2076
  • Last Modified:

Pipe Synchronization

I have a pipe program which interprets every > as | in Linux.
It reads in command from user and does shell execution based on the user input.
For example if the user types in
    ls >  wc
it will work like ls | wc in command line.

However if I do this
    cat /usr/include/time.h > wc
it will not work. From what I know, the pipe is overfilled. How to make it synchronized so that it will not overflow? Or other than making it synchronized, are there any methods that will solve the overflow problem?


hongjun
0
hongjun
Asked:
hongjun
  • 4
  • 4
1 Solution
 
mtmikeCommented:
Pipe overflow should not be the problem.

Two concurrent processes that are communicating using a pipe always work in lockstep. The writing process (ls or cat in your examples) fills up the pipe until it is full and then blocks. The reading process (wc) reads out the pipe until it is empty and then blocks.

In case the pipe is full, the writing process has to wait until the reading process reads out the data and, in case the pipe is empty, the reading process has to wait until the writing process fills it up.

A pipe can only overflow when the reading process does not read out the data.

Why do you think the pipe is overfilled?
0
 
hongjunAuthor Commented:
I can only think the problem is either due to my pipe coding not being concurrent or is overfilled.
Any example coding to make the pipe concurrent?

hongjun
0
 
mtmikeCommented:
Here's a small example C program:

#include <sys/types.h>
#include <unistd.h>

int main()
{
      int fds[2];
      pid_t pid;

      if (pipe(fds)) {
            return 1;
      }

      pid = fork();
      if (pid == 0) {

            /* Child (writing) process */
            dup2(fds[1], 1);
            close(fds[0]);
            execlp("ls", "ls", 0);
      } else if (pid > 0) {

            /* Parent (reading) process */
            dup2(fds[0], 0);
            close(fds[1]);
            execlp("wc", "wc", 0);
      }
      return 1;
}

It's important to close the end of the pipe a process does not use.
0
Free Backup Tool for VMware and Hyper-V

Restore full virtual machine or individual guest files from 19 common file systems directly from the backup file. Schedule VM backups with PowerShell scripts. Set desired time, lean back and let the script to notify you via email upon completion.  

 
hongjunAuthor Commented:
i basically have similar coding.
the only problem lies when i try to pipe an output which is very large like cat /usr/include/time.h to an input.

hongjun
0
 
mtmikeCommented:
When I change
  execlp("ls", "ls", 0);

into
  execlp("cat", "cat", "/usr/include/time.h", 0);

It still works (even with a much larger file).

Can you show some of your code?
0
 
hongjunAuthor Commented:
I want to ask.
Any example of using named and unnamed pipes?
I just realised that there two types of pipes.
What are the use of them?

hongjun
0
 
mtmikeCommented:
A named pipe is similar to an unnamed pipe, except that it is created in a different way.

Named pipes can be opened by filename using mkfifo(). Two processes that want to communicate using a pipe can call mkfifo() on the same filename to open a pipe.

An unnamed pipe must be created in a single process using pipe(). The pipe file descriptors can be passed on to children by using fork().

Named pipes are seldom used and have been superseded by the more general unix domain socket mechanism.
0
 
hongjunAuthor Commented:
i solved that in another approach.
But anyway thanks for your reply and they are very relevant.

hongjun
0

Featured Post

Get free NFR key for Veeam Availability Suite 9.5

Veeam is happy to provide a free NFR license (1 year, 2 sockets) to all certified IT Pros. The license allows for the non-production use of Veeam Availability Suite v9.5 in your home lab, without any feature limitations. It works for both VMware and Hyper-V environments

  • 4
  • 4
Tackle projects and never again get stuck behind a technical roadblock.
Join Now