Why new file descriptors are created in case of a pipe/pipeline?

The child process needs to receive an EOF signal to start doing its stuff and processing its input, this is done when main process closes the write FD

If both use the same FD when child exits, main needs to receive an EOF. But the fd opened by the main process is writable, so system doesn't know that no more data will go to the pipe.

Both will not use the same FD.

PIPE IN: FD 1 (of process A)
PIPE OUT: FD 0 (of process B)

So probably you misunderstood the question.

Yeah they will not use it, because if they did the child process wouldn't know when to start/stop. I think I did not explain clear, and also very likely I didn't understand your question.

The question is, why it's not working like this:

  PROCESS A
  0 stdin              PROCESS B
  1 stdout -> pipe ->  0 stdin
  2 stderr             1 stdout
                       2 stderr

Select all Open in new window

Instead of something like this:

  PROCESS A          
  0 stdin
  1 stdout 
  2 stderr             PROCESS B
  4 new     -> pipe -> 0 stdin
                       1 stdout
                       2 stderr      

Select all Open in new window

It is done in this way to be as generic as possible. If it was done the way you are referring to, then stdout of the parent and stdin of the child would ALWAYS be connected to the pipe (as you have shown). But what about if someone had a scenario where they still needed the parent process to output something to stdout as well as being able to output something else to the pipe to go to the child. Also, the child might want to read both from stdin still AND the pipe.

So they go with creating a standalone pipe with new file descriptors and then they leave it up to you, as the programmer, to decide if you would like to connect one or both (or neither) end of the pipe to stdin/stdout via the dup2 calls that your second link talks about.

Does that help you at all?

Yes! That helps, thanks a lot!

If process "A" would want to output something to stdout as well, then they can always create a new FD (File descriptor). A FD that is similar to the default (1/stadout). However, probably/apparently this is not the best way to do it and it's better to keep stdout intact in the first place (instead of changing it first to the pipe, and then later on creating a new one). Am I right?

It's been a very long time since I have dug this deep into the subject, but my guess would be that if the above was even at all possible, it would take a reasonable amount more work to do it this way. Additionally, it is probably best to leave all the system calls that default to outputting to stdout as they are, and make the calls that are outputting to the pipe explicitly use the pipe's FD.

I guess you have to think about these pipe() calls been used for so much more than just implementing the "command line" pipe | that connects the stdout of one process to the stdin of another. They could be used, for example, between a parent "manager" process to communicate to child "worker" processes (say in a web server, etc) to send commands/receive status updates, etc, etc. And many more scenarios that I could think up.

Actually, I can expand on that example... in the case where you have the 1 parent process spawning multiple child processes, that all need a pipe between the parent and each child, which pipe would get the stdout FD? The last pipe? Then you would have had to save all the previous pipe FD's to still be able to talk to those other processes. So it would be better to leave stdout alone, and then just have say an array of structure, where each one represents a child process, and in that structure you store away the pipe FD so that when you need to talk to a particular child process, you just look up the pipe FD for that process and send it the data.

I think so too, something like that would make sense. Thanks again!

Your last post is not 100% clear to me. If one process would contain multiple pipes, then you're doing that to direct the output of the main process to the input of all these child processes, right?

But what is you would leave the standard output intact? Let's say we have:

PROCESS A
FD 1 (standard output) -> terminal
FD 4                   -> pipe

Select all Open in new window

What exactly does "standard" mean? Because if I'm using something like this:

echo 'Hi' | wc -m

Select all Open in new window

Then the situation looks like this:

PROCESS A (echo)
FD 1 (standard output) -> terminal
FD 4                   -> pipe -> input wc command

Select all Open in new window

Then why I'm not seeing "Hi" in my terminal too (besides the character count)? If this standard output stream in still intact, then I would expect to see "Hi" on my screen (in my terminal).

Your original scenario of process A stdout becomes process B's stdin is exactly what happens if your shell is connecting processes because you used the pipe symbol (e.g. tail -f /var/log/debug | cat). Relevant output from ps ax

20022 pts/5    S+     0:00 tail -f /var/log/debug
20023 pts/5    S+     0:00 cat

Select all Open in new window

Relevant output from lsof -p 20022

COMMAND   PID USER   FD      TYPE DEVICE SIZE/OFF    NODE NAME
tail    20022 dunc    0u      CHR  136,5      0t0       8 /dev/pts/5
tail    20022 dunc    1w     FIFO   0,12      0t0  343159 pipe
tail    20022 dunc    2u      CHR  136,5      0t0       8 /dev/pts/5

Select all Open in new window

Relevant output from lsof -p 20023

COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF    NODE NAME
cat     20023 dunc    0r  FIFO   0,12      0t0  343159 pipe
cat     20023 dunc    1u   CHR  136,5      0t0       8 /dev/pts/5
cat     20023 dunc    2u   CHR  136,5      0t0       8 /dev/pts/5

Select all Open in new window

Yeah I know that, but the question is why? ;)  By the way, why it's "1u" and "2u", fd 1 and fd 2 only have to write something so I would expect 1w and 2w (and 0r instead of 0u)?

But meanwhile I was reading some more about it, but actually a pipe in a terminal is not a pipe. It's:

pipe + dup2

Is that correct? They just call it a pipe and explain it like a pipe, but the pipe itself is not created on FD 1. Later on it's only copied to FD 1.

So actually this is what a pipe is:

  PROCESS A          
  0 stdin
  1 stdout 
  2 stderr             PROCESS B
  4 new     -> pipe -> 0 stdin
                       1 stdout
                       2 stderr    

P.S. 4 can also be another integer (but at least not 1).   

Select all Open in new window

But when copying fd 4 to fd 1 (dup2), you will end up with:

  PROCESS A
  0 stdin              PROCESS B
  1 stdout -> pipe ->  0 stdin
  2 stderr             1 stdout
                       2 stderr

Select all Open in new window

So the last diagram doesn't really show what a pipe is. The first diagram shows what a pipe is. Many websites are explaining a pipe with the last diagram, but actually this is incorrect. Furthermore, "|" in a terminal is not a pipe, but it's a copy of a pipe. It's pipe + dup2.

Is this correct?

1u and 2u (also 0u for tail (process A)) are a peculiarity of opening a tty: if you open it for reading and writing, all channels can read and write. That's just how it is: the program doesn't know that and won't try e.g. writing to fd 0.
Pipes are pipes: you are making it too complicated for yourself. Here's how the shell connects, say, tail and cat:
mainline: Forked child 1: Forked child 2: There is only ever 1 pipe. The ends of it get connected to the appropriate file units.

Firstable, thanks a lot! I'm really confused because everywhere I'm reading different things.

Two things are not clear to me.

1. For example, see: http://rozmichelle.com/pipes-forks-dups/#attachment_7363
You have three items (main line, child 1, child 2), but on that website there are only 2. I don't understand why you're not using the main line as child 1 so you only have two items (main/parent one and one child)?

2. And you're saying:

dup2(pipefd[1], 1)
close(pipefd[1])

But http://man7.org/linux/man-pages/man2/dup.2.html

If the file descriptor newfd was previously open, it is silently closed before being reused.

So close(pipefd[1]) is already part of the dup2 call, so why you're closing it again?

Of and about the 0u,1u,2u ... does that also mean that all these file descriptors point to the same entry/row in the "open file table"?

Of and about the 0u,1u,2u ... does that also mean that all these file descriptors point to the same entry/row in the "open file table"?

Yes indeed. They all point to the same tty.

1. For example, see: http://rozmichelle.com/pipes-forks-dups/#attachment_7363
You have three items (main line, child 1, child 2), but on that website there are only 2. I don't understand why you're not using the main line as child 1 so you only have two items (main/parent one and one child)?

I am showing a shell command that pipes 2 programs together, that's why there are 2 children. Rozmichelle's example C code is equivalent to the following script

#!/bin/sh
sort <<////
pear
peach
apple
////

Select all Open in new window

The << separator introduces a here document (you can read more about them in bash's man page). Like the web example, this script will output the 3 fruits in order - try it.
One can never use the mainline as child 1, because the mainline has to stay as the shell, but running an external command requires overwriting the current program image with that command (that's what exec does). In the web example, there is only one external command - sort.

2. And you're saying:

dup2(pipefd[1], 1)
close(pipefd[1])

But http://man7.org/linux/man-pages/man2/dup.2.html

    If the file descriptor newfd was previously open, it is silently closed before being reused.


So close(pipefd[1]) is already part of the dup2 call, so why you're closing it again?

Look again at what the man page says: the file descriptor newfd ... is silently closed
The Synopsis for dup2 is: int dup2(int oldfd, int newfd);
So the dup2 call closes fd 1, not fd pipefd[1]

@Duncan Roe: That's really helpful, thanks a lot! I don't have that much time today, so later on I'll dive deeper into everything and then later on probrobably I'll mark it as the correct answer.

About closing fd 1 instead of fd pipefd[1] you're right I see.

See: http://rozmichelle.com/pipes-forks-dups/#attachment_7365

The child calls dup2() to make its stdin be a copy of fds[0], closing file descriptor 0 first.

stdin is fd 0 (file descriptor 0). Because dup2 makes a copy of the "old" fds[0] to the "new" fd 0. This means the old one will be closed (fds[0]). Before fd 0 is a copy of fds[0], fd 0 will be closed first, but do i understand it right that this is not necessary? Anyway immediately after closing it, fd 0 is fds[0], so it's open again? Is there a reason that it will be closed first?

And Rozmichelle uses a pipe call in her example, so there is a pipe involved. Is a here document also a form of a pipe? I was trying to find a simple diagram of a here document (with file descriptors et cetera), but I couldn't find one. Because I don't have that much time right now, I just searched for "here document" in Google images, but I didn't find similar diagrams as for example on Rozmichelle's page (so I can see what exactly is happening with the file descriptors).

To read about here documents:
man bash
-i
/here documents

In Google searches, [linux] is your friend. I searched for here document [linux] and immediately found articles that you would find helpful. But no diagrams - you are delving in deeper than most folks care to. (I got the [linux] tip from a thread where we were complaining there used to be a Linux area in Google - eventually someone from Google offered [linux]). EOF is a common terminator in the examples but I prefer //// - it really stands out and anyway it was my favourite terminator for punched card input.

stdin is fd 0 (file descriptor 0). Because dup2 makes a copy of the "old" fds[0] to the "new" fd 0. This means the old one will be closed (fds[0]). Before fd 0 is a copy of fds[0], fd 0 will be closed first, but do i understand it right that this is not necessary? Anyway immediately after closing it, fd 0 is fds[0], so it's open again? Is there a reason that it will be closed first?

Just take this a bit more slowly. dup2(a1, a2); is just shorthand for close(a2); dup (a1);. dup() or open() will both choose the lowest unused fd, which is a2 that you just closed. Neither dup() nor dup2() will close a1. After either dup call you have 2 file descriptors accessing stdin: 0 and fds[0]. You don't need fds[0] so you close it.

@Duncan: Thanks a lot again! Really helpful! I was reading the following sentence the wrong way (maybe because English is not my primary language and I'm bad in it ;) ...):

The child calls dup2() to make its stdin be a copy of fds[0], closing file descriptor 0 first.

I was reading it like this (metaphor):

Copy/pasting text, writing the text first.

But I have to see it as:

Copy/pasting text, copying the text first.

And that copying is already part of copy/pasting. The same way that "closing" is already part of "dup2". I saw it as a double copy, but that's not how it's meant.

And I started to read:

https://www.tldp.org/LDP/abs/html/here-docs.html

It uses a form of I/O redirection to feed a command list to an interactive program or a command, such as ftp, cat, or the ex text editor

I know what a  I/O redirection is. You gave this example:

#!/bin/sh
sort <<////
pear
peach
apple
////

Actually immediately I don't understand it anymore, because "pear, peach and apple" are not commands ("to feed a COMMAND LIST to"). This is just a list of strings, not a list of command. So I'm stuck already after one sentence ;)?? I'm not too lazy to find out things by myself, but maybe you can say something about it, because now it looks like the first sentence already contradicts your example.

Do look up here documents in the bash man page or Google as per https:#a42694212. The redirection introduces data so it does not need to be commands. But ... it can contain references to shell variables (not shown, but it's a very powerful feature).

If the redirection introduced data, then I don't understand why Tldp.org doesn't mention this. The redirection is also part of the text there.

But none of these documents show me how it's really working. Now I know what will happen when using a here document, but I have no idea what's behind it. I can just think of some possibilities (with what I know about it till now). Let's take your example. I could see the following as a file:

pear
peach
apple

Select all Open in new window

Let's call this file: here-file

Then I can see it as:

FD TABLE SORT PROCESS:          OPEN FILE TABLE:
FD 0 (stdin), POINTS TO    ->   Data associated with here-file (pear, peach, apple)

Select all Open in new window

But I could also see it as follows:

FD TABLE HEREDOC PROCESS:       FD TABLE SORT PROCESS:
FD 1 (stdout), POINTS TO   ->   FD 0 (stdin)

Select all Open in new window

With the last one there is probably a pipe system call involved. With the first one probably not. So I still have the same question, how a here document is really working?

With a "|" character in a terminal it's also possible to bring it back to file descriptors, duplications et cetera. Then probably the same is possible for here documents? Unfortunately I find a lot on the internet when it's about pipelines, but when it's about here documents I only find what it is doing and not what's behind it.

P.S. Anyway thanks a lot again, I know I'm asking a lot and I am a bit difficult ;), but I'm just trying to find out how it's really working.

I'm just trying to find out how it's really working.

First of all, try the examples yourself. Change them a bit and see what happens.

I only find what it is doing and not what's behind it.

All right, since you asked I will show you. Some of this may be a little advanced for you at this stage, but I'll try to guide you. We will be studying the workings of the example script in https#a42694103. On my system, that script is called ee160.sh, but you can call your copy whatever you like.
We'll be using a utility called strace. You will need a command window for this. In that window, issue man strace and at least try to read some of the DESCRIPTION section.
You'll notice that strace only works on programs, not shell scripts. So we have to strace /bin/bash -c ./ee160.sh and watch how bash processes the script. bash and sort are mature programs which issue a lot of system calls to cater for unlikely error scenarios, so use strace's -e option to exclude at least some of them

strace -e '!geteuid,getegid,getuid,getgid,rt_sigprocmask,rt_sigaction,getrlimit,sysinfo,sched_getaffinity' -f /bin/bash -c ./ee160.sh 2>&1|less -N

Select all Open in new window

Expect lots of output. In the snapshots below, I've add C-style // comments to the strace output to explain what's going on

      1 execve("/bin/bash", ["/bin/bash", "-c", "./ee160.sh"], [/* 72 vars */]) = 0                  // strace starts bash with given argumenst
      2 brk(NULL)                               = 0x21ab000                                          // (malloc() initialising)
      3 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f13fff2a000    // Start of loading the dynamic libraries that bash uses.
      4 access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)              // All programs start this way: we don't care about it today

Select all Open in new window

Type in /ee160.sh to find the next mention of the the sample script. If you stay on line 1, type n to really get the next occurrence. In future, I'll just show searches in the snapshot.

    201 execve("./ee160.sh", ["./ee160.sh"], [/* 69 vars */]) = 0                             // This is Linux invoking bash to run the script.
    202 brk(NULL)                               = 0x2088000                                   // We really want the next one after this, so enter n again
n
    324 open("./ee160.sh", O_RDONLY)            = 3                                           // Bash opens its command source (fd 3)
    325 ioctl(3, TCGETS, 0x7fff18c4ed70)        = -1 ENOTTY (Inappropriate ioctl for device)  // Check whether command source is a tty - no.
    326 lseek(3, 0, SEEK_CUR)                   = 0
    327 read(3, "#!/bin/sh\nsort <<////\npear\npeach"..., 80) = 44                            // Read (to see it it's really simple(?))
    328 lseek(3, 0, SEEK_SET)                   = 0                                           // Not really simple, so rewind file and free up fd 3
    329 fcntl(255, F_GETFD)                     = -1 EBADF (Bad file descriptor)              // Is fd 255 free? - yes
    330 dup2(3, 255)                            = 255                                         // Open on fd 255
    331 close(3)                                = 0                                           // Close 3
    332 fcntl(255, F_SETFD, FD_CLOEXEC)         = 0                                           // Child processes don't get the script
    333 fcntl(255, F_GETFL)                     = 0x8000 (flags O_RDONLY|O_LARGEFILE)
    334 fstat(255, {st_mode=S_IFREG|0755, st_size=44, ...}) = 0
    335 lseek(255, 0, SEEK_CUR)                 = 0
    336 brk(0x209b000)                          = 0x209b000
    337 read(255, "#!/bin/sh\nsort <<////\npear\npeach"..., 44) = 44                          // Read new command file
    338 open("/usr/lib64/gconv/gconv-modules.cache", O_RDONLY) = -1 ENOENT (No such file or directory) // Internationalisation stuff - not relevant to this demo, so
    339 brk(0x209c000)                          = 0x209c000                                   // Skip on to the next mention of sort
/sort
    415 stat("/home/dunc/bin/sort", 0x7fff18c4ea10) = -1 ENOENT (No such file or directory)   // Bash looks along the PATH for a directory containing the sort command
    418 stat("/usr/local/bin/sort", 0x7fff18c4ea10) = -1 ENOENT (No such file or directory)   // Not here
    419 stat("/usr/bin/sort", {st_mode=S_IFREG|0755, st_size=109960, ...}) = 0                // Found it
    420 stat("/usr/bin/sort", {st_mode=S_IFREG|0755, st_size=109960, ...}) = 0                // (there were ignored system calls between this and previous call)
    421 access("/usr/bin/sort", X_OK)           = 0                                           // More of the same, read down until fork
/clone                                                                                        // fork library call becomes clone system call nowadays
    429 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7fe243abf9d0) = 2799 // Child is process 2799
    430 wait4(-1, strace: Process 2799 attached                                               // Bash waits for child to finish
    431  <unfinished ...>
    432 [pid  2799] close(255)                  = 0                                           // This is Bash code running in the child
    433 [pid  2799] brk(0x20ce000)              = 0x20ce000                                   // (malloc)
    434 [pid  2799] stat("/tmp", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=4096, ...}) = 0       // Bash is going to make a copy of the here document in /tmp
    435 [pid  2799] faccessat(AT_FDCWD, "/tmp", W_OK) = 0                                     // ..
    436 [pid  2799] statfs("/tmp", {f_type="EXT2_SUPER_MAGIC", f_bsize=4096, f_blocks=14658181, f_bfree=12918408, f_bavail=12168043, f_files=3735552, f_ffree=3594703, f_fsid={1146837871, 627725631}, f_namelen=255, f_frsize=4096, f_flags=4128}) = 0
    437 [pid  2799] open("/tmp/sh-thd-809077902", O_WRONLY|O_CREAT|O_EXCL|O_TRUNC, 0600) = 3  // Create temp file with random name
    438 [pid  2799] dup(3)                      = 4                                           // Bash holds the file open on fd 3 but operates on it
    439 [pid  2799] fcntl(4, F_GETFL)           = 0x8001 (flags O_WRONLY|O_LARGEFILE)         // via fd 4. Some kind of precaution I think
    440 [pid  2799] fstat(4, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
    441 [pid  2799] brk(0x20d0000)              = 0x20d0000
    442 [pid  2799] write(4, "pear\npeach\napple\n", 17) = 17                                 // Write the temp file via fd 4
    443 [pid  2799] close(4)                    = 0
    444 [pid  2799] open("/tmp/sh-thd-809077902", O_RDONLY) = 4                               // Re-open temp file read-only on fd 4
    445 [pid  2799] close(3)                    = 0                                           // Release fd 3
    446 [pid  2799] unlink("/tmp/sh-thd-809077902") = 0                                       // Delete the temp file. It's still accessible via fd 4
    447 [pid  2799] dup2(4, 0)                  = 0                                           // THE ACTUAL DUP2 CALL
    448 [pid  2799] close(4)                    = 0                                           // Release fd 4. fds 0, 1 & 2 are all set up now, so...
    449 [pid  2799] execve("/usr/bin/sort", ["sort"], [/* 69 vars */]) = 0                    // Overwrite bash with sort
    450 [pid  2799] brk(NULL)                   = 0x2549000                                   // The usual loader preamble. Skip on to sort's first system call
/fadvise64
    539 [pid  2799] fadvise64(0, 0, 0, POSIX_FADV_SEQUENTIAL) = 0                             // An optimisation. Tell Linux we'll be reading stdin sequentially
    540 [pid  2799] fstat(0, {st_mode=S_IFREG|0600, st_size=17, ...}) = 0
    541 [pid  2799] fstat(0, {st_mode=S_IFREG|0600, st_size=17, ...}) = 0
    542 [pid  2799] read(0, "pear\npeach\napple\n", 4096) = 17                                // Read the 3 lines from the temp file
    543 [pid  2799] read(0, "", 4096)           = 0                                           // Read more - 0 return tell us EOF
    544 [pid  2799] fstat(1, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0                      // Make sure we have a writable stdout
    545 [pid  2799] write(1, "apple\npeach\npear\n", 17apple                                  // Write the output - the "apple" at the end is actually output
    546 peach                                                                                 // (This is a line of output)
    547 pear                                                                                  // (This is a line of output)
    548 ) = 17                                                                                // 17 characters written
    549 [pid  2799] lseek(0, 0, SEEK_CUR)       = 17
    550 [pid  2799] close(0)                    = 0                                           // Close stdin &c. before exitting
    551 [pid  2799] close(1)                    = 0                                           // Everyone says that's good parctice
    552 [pid  2799] close(2)                    = 0                                           // so I suppose it must be
    553 [pid  2799] exit_group(0)               = ?                                           // The sort program exits
    554 [pid  2799] +++ exited with 0 +++                                                     // No error
    555 <... wait4 resumed> [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 2799          // Back in the parent Bash shell
    556 --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=2799, si_uid=501, si_status=0, si_utime=0, si_stime=0} ---
    557 wait4(-1, 0x7fff18c4e698, WNOHANG, NULL) = -1 ECHILD (No child processes)             // The child has finished
    558 rt_sigreturn({mask=[]})                 = 0
    559 read(255, "", 44)                       = 0                                           // Bash reads the next command from the script, gets EOF
    560 exit_group(0)                           = ?                                           // Bash exits
    561 +++ exited with 0 +++                                                                 // No error

Select all Open in new window

You're great! Although it's indeed a bit advanced for me at this stage, most of it do make sence to me because of your comments. It's good you wrote:

// Bash holds the file open on fd 3 but operates on it
// via fd 4. Some kind of precaution I think

Because when I was reading it, again I was thinking ... why they are using an extra file descriptor while it doesn't seem necessary. But then I saw your comment and I saw you thought about it, so then I can live with it ;).

But correct me if I'm wrong, but it's kind of similar what I thought already:

FD TABLE SORT PROCESS:          OPEN FILE TABLE:
FD 0 (stdin), POINTS TO    ->   Data associated with here-file (pear, peach, apple)

Select all Open in new window

First fd 4 is connected with a temporary file (pear, peach, apple). And then it's copied to fd 0 (stdin) via dup2(4, 0). Then you already have what I had and from that point it already makes sence to me.

So now I can go back to where it started ... I don't know if we can compare Rozmichelle's example with a here document. It's similar, but it's not equivalent. The here document is working with a temporary file, so:

fd 0 -> temporary file

But with Rozmichelle's example, the standard input is not coming from a file, but directly from the stdout of another process (through pipe). So if you would ask me, I would not say it's a equivalent of a here document. Maybe it is, but so far I'm not convinced yet.

Now let's go back to the main question:

Why new file descriptors are created in case of a pipe/pipeline?

Now I understand my own question also better. What I expected to see was:

pipe(fd_pipe_in, process fd_pipe_in, fd_pipe_out, process fd_pipe_out)

So then a pipe is just a connection between two fd's. This if possible for 1 involved process:

process fd_pipe_in = process fd_pipe_out

Select all Open in new window

Or two processes can be involved:

process fd_pipe_in != process fd_pipe_out

Select all Open in new window

If you would see it like that, then a pipe would work just as follows (example):

1.   parent process
2.   fork parent process:
3.   FD PROCESS A      FD PROCESS B
     fd 0              fd 0
     fd 1              fd 1
     fd 2              fd 2
4.   pipe(1,A,0,B)

And you will end up with:

FD PROCESS A               FD PROCESS B
fd 1          -> pipe ->   fd 0

In reality, the pipe system call is only about 1 process. It's this:

pipe(fd[0],fd[1])

This is only about 1 process. Many people on the internet are saying this:

a pipe is a connection between two processes

For example, see: https://www.geeksforgeeks.org/pipe-system-call/

In general, this is incorrect I think (and especially when it's about the pipe system call). It just depends how you look at it. If you just see it as the pipe system call, then it's about 1 process. Later on it can be about 2 processes, but that's only because of fork, dup2 et cetera. Fork and dup2 has nothing to do with pipe call.

If it's correct what I'm saying, then this is the answer to my question and then I understand it all. Duncan, maybe you can confirm this whether or not I'm seeing it the right way ... or otherwise which mistakes I made.

One thing

In reality, the pipe system call is only about 1 process.

The pipe() (and subsequent dup2() and close()) system calls are all about one file system object and not about one process.

3.   FD PROCESS A      FD PROCESS B
     fd 0              fd 0
     fd 1              fd 1
     fd 2              fd 2

Incomplete! Bash created the pipe before forking the children. They start thusly:
FD PROCESS A      FD PROCESS B
     fd 0              fd 0
     fd 1              fd 1
     fd 2              fd 2
     fd 3              fd 3   pipe (input end)
     fd 4              fd 4   pipe (output end)

The geeksforgeeks post looks fine to me. You need to improve your understanding. As I mentioned before, you need to appreciate the difference between a file system object and a process.

Probably you were reading my post too fast, see:

What I expected to see was

And then I was explaining some stuff. I know it's not like that in reality.

But a pipe call connects two file descriptors in a file descriptor table. This table belongs to one process, so why I can not say that the pipe call is about one process?

It's wrong to say that a pipe call is about two processes, because it only connects two file descriptors in one fd table. That fd table only belongs to one process. Later on you need to use the fork and dup2 call to connect two processes, but then there will be no extra pipe call anymore. So the pipe call doesn't connect the processes.

That's why the geeksforgeeks post is incorrect. Or what's wrong about that?

The file descriptor table "belongs" to the system. Multiple processes can access the same file system object.

It's wrong to say that a pipe call is about two processes

You are correct. Usually a pipe will have its ends open by 2 different processes but this is not essential.
After a fork() system call, the child inherits the open files of the parent (except those opened by CLO_EXEC)

What I expected to see was

Not really. The pipe() system call creates a pipe (file system object) and returns a 2-element array giving the fds of the read and write ends  respectively. Other system calls do subsequent manipulation.

This trace is an example using pipe. I excluded all system calls that aren't directly relevant to our discussion, but I had to leave close() in so you can see the pipe ends being closed. This is the script

strace -e '!'geteuid,\
stat,\
access,\
munmap,\
uname,\
arch_prctl,\
getpeername,\
getpid,\
getppid,\
gettimeofday,\
getpgrp,\
brk,\
getegid,\
getuid,\
getgid,\
rt_sigprocmask,\
rt_sigaction,\
getrlimit,\
sysinfo,\
sched_getaffinity,\
open,\
mmap,\
fstat,\
lseek,\
lstat,\
inotify_init,\
inotify_add_watch,\
mprotect -f /bin/bash -c "tail -f /var/log/debug | cat" 2>&1|less -N

Select all Open in new window

And this is the trace

execve("/bin/bash", ["/bin/bash", "-c", "tail -f /var/log/debug | cat"], [/* 72 vars */]) = 0
close(3)                                = 0
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\16\0\0\0\0\0\0"..., 832) = 832
close(3)                                = 0
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\r\0\0\0\0\0\0"..., 832) = 832
close(3)                                = 0
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\10\2\0\0\0\0\0"..., 832) = 832
close(3)                                = 0
close(3)                                = 0
read(3, "# Locale name alias data base.\n#"..., 4096) = 2997
read(3, "", 4096)                       = 0
close(3)                                = 0
close(3)                                = 0
close(3)                                = 0
close(3)                                = 0
close(3)                                = 0
close(3)                                = 0
close(3)                                = 0
close(3)                                = 0
close(3)                                = 0
close(3)                                = 0
close(3)                                = 0
close(3)                                = 0
close(3)                                = 0
close(3)                                = 0
read(3, "# GNU libc iconv configuration.\n"..., 4096) = 4096
read(3, "1002//\tJUS_I.B1.002//\nmodule\tJUS"..., 4096) = 4096
read(3, "ISO-IR-110//\t\tISO-8859-4//\nalias"..., 4096) = 4096
read(3, "\t\tISO-8859-14//\nalias\tISO_8859-1"..., 4096) = 4096
read(3, "IC-ES//\nalias\tEBCDICES//\t\tEBCDIC"..., 4096) = 4096
read(3, "DIC-CP-ES//\t\tIBM284//\nalias\tCSIB"..., 4096) = 4096
read(3, "//\nalias\tCSIBM864//\t\tIBM864//\nal"..., 4096) = 4096
read(3, "BM939//\nmodule\tIBM939//\t\tINTERNA"..., 4096) = 4096
read(3, "EUC-CN//\nalias\tCN-GB//\t\t\tEUC-CN/"..., 4096) = 4096
read(3, "T//\nmodule\tISO-2022-CN-EXT//\tINT"..., 4096) = 4096
read(3, "//\t\tISO_5428//\nalias\tISO_5428:19"..., 4096) = 4096
read(3, "CII-8\t1\n\n#\tfrom\t\t\tto\t\t\tmodule\t\tc"..., 4096) = 4096
read(3, "\tfrom\t\t\tto\t\t\tmodule\t\tcost\nalias\t"..., 4096) = 4096
read(3, "712//\t\tINTERNAL\t\tIBM12712\t\t1\nmod"..., 4096) = 2847
read(3, "", 4096)                       = 0
close(3)                                = 0
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260\5\0\0\0\0\0\0"..., 832) = 832
close(3)                                = 0
pipe([3, 4])                            = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f00f261f9d0) = 14280
strace: Process 14280 attached
[pid 14279] close(4)                    = 0
[pid 14279] close(4)                    = -1 EBADF (Bad file descriptor)
[pid 14279] clone( <unfinished ...>
[pid 14280] close(3)                    = 0
[pid 14280] dup2(4, 1)                  = 1
[pid 14280] close(4)                    = 0
[pid 14279] <... clone resumed> child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f00f261f9d0) = 14281
strace: Process 14281 attached
[pid 14279] close(3)                    = 0
[pid 14281] dup2(3, 0)                  = 0
[pid 14281] close(3 <unfinished ...>
[pid 14279] wait4(-1,  <unfinished ...>
[pid 14281] <... close resumed> )       = 0
[pid 14280] execve("/usr/bin/tail", ["tail", "-f", "/var/log/debug"], [/* 69 vars */] <unfinished ...>
[pid 14281] execve("/usr/bin/cat", ["cat"], [/* 69 vars */] <unfinished ...>
[pid 14280] <... execve resumed> )      = 0
[pid 14281] <... execve resumed> )      = 0
[pid 14280] close(3)                    = 0
[pid 14280] read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\10\2\0\0\0\0\0"..., 832) = 832
[pid 14281] close(3)                    = 0
[pid 14281] read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\10\2\0\0\0\0\0"..., 832) = 832
[pid 14280] close(3)                    = 0
[pid 14281] close(3)                    = 0
[pid 14280] read(3, "# Locale name alias data base.\n#"..., 4096) = 2997
[pid 14280] read(3, "", 4096)           = 0
[pid 14280] close(3)                    = 0
[pid 14281] read(3, "# Locale name alias data base.\n#"..., 4096) = 2997
[pid 14281] read(3, "", 4096)           = 0
[pid 14281] close(3)                    = 0
[pid 14280] close(3)                    = 0
[pid 14281] close(3)                    = 0
[pid 14280] close(3)                    = 0
[pid 14281] close(3)                    = 0
[pid 14280] close(3)                    = 0
[pid 14281] close(3)                    = 0
[pid 14280] close(3)                    = 0
[pid 14281] close(3)                    = 0
[pid 14280] close(3)                    = 0
[pid 14281] close(3)                    = 0
[pid 14280] close(3)                    = 0
[pid 14281] close(3)                    = 0
[pid 14280] close(3)                    = 0
[pid 14281] close(3)                    = 0
[pid 14280] close(3)                    = 0
[pid 14281] close(3)                    = 0
[pid 14280] close(3 <unfinished ...>
[pid 14281] close(3 <unfinished ...>
[pid 14280] <... close resumed> )       = 0
[pid 14281] <... close resumed> )       = 0
[pid 14281] close(3 <unfinished ...>
[pid 14280] close(3 <unfinished ...>
[pid 14281] <... close resumed> )       = 0
[pid 14280] <... close resumed> )       = 0
[pid 14281] close(3)                    = 0
[pid 14280] close(3)                    = 0
[pid 14281] close(3)                    = 0
[pid 14280] close(3)                    = 0
[pid 14281] fadvise64(0, 0, 0, POSIX_FADV_SEQUENTIAL) = -1 ESPIPE (Illegal seek)
[pid 14281] read(0,  <unfinished ...>
[pid 14280] read(3, "quest from 10.255.255.6:886 for "..., 2348) = 2348
[pid 14280] read(3, "", 0)              = 0
[pid 14280] write(1, "Oct  3 07:05:26 dimstar -- MARK "..., 399 <unfinished ...>
[pid 14281] <... read resumed> "Oct  3 07:05:26 dimstar -- MARK "..., 131072) = 399
[pid 14280] <... write resumed> )       = 399
[pid 14281] write(1, "Oct  3 07:05:26 dimstar -- MARK "..., 399Oct  3 07:05:26 dimstar -- MARK --
Oct  3 07:25:26 dimstar -- MARK --
Oct  3 07:45:26 dimstar -- MARK --
Oct  3 08:05:26 dimstar -- MARK --
Oct  3 08:25:26 dimstar -- MARK --
Oct  3 08:39:24 dimstar in.rlogind[13719]: connect from 10.255.255.6 (10.255.255.6)
Oct  3 09:05:26 dimstar -- MARK --
Oct  3 09:25:26 dimstar -- MARK --
Oct  3 09:45:26 dimstar -- MARK --
Oct  3 10:05:26 dimstar -- MARK --
) = 399
[pid 14281] read(0,  <unfinished ...>
[pid 14280] read(4, 0x150ba70, 31)      = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
[pid 14280] --- SIGTERM {si_signo=SIGTERM, si_code=SI_USER, si_pid=13927, si_uid=501} ---
[pid 14281] <... read resumed> "", 131072) = 0
[pid 14280] +++ killed by SIGTERM +++
[pid 14279] <... wait4 resumed> [{WIFSIGNALED(s) && WTERMSIG(s) == SIGTERM}], 0, NULL) = 14280
[pid 14279] wait4(-1,  <unfinished ...>
[pid 14281] close(0)                    = 0
[pid 14281] close(1)                    = 0
[pid 14281] close(2)                    = 0
[pid 14281] exit_group(0)               = ?
[pid 14281] +++ exited with 0 +++
<... wait4 resumed> [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 14281
close(3)                                = -1 EBADF (Bad file descriptor)
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_KILLED, si_pid=14280, si_uid=501, si_status=SIGTERM, si_utime=0, si_stime=0} ---
wait4(-1, 0x7ffc15e441d8, WNOHANG, NULL) = -1 ECHILD (No child processes)
rt_sigreturn({mask=[]})                 = 0
exit_group(0)                           = ?
+++ exited with 0 +++

Select all Open in new window

If you try this yourself, be aware that you need to enter F at the first prompt from less, to put it into follow mode. tail will never finish by itself, so you need to kill it from another xterm (you can see its PID in the trace).
Alternatively, page down until you see what PID tail has, and kill it straight away. Then you will get normal end of file from less.
No comments this time - if you look at what happens near the pipe() calls you should be fine

The file descriptor table "belongs" to the system. Multiple processes can access the same file system object.

I was and I am aware of that.

You are correct.

Exactly and that's why many websites are wrong about it, also geeksforgeeks:

a pipe is a connection between two processes

I know and understand how it's working, but the pipe call is only connecting two different file descriptors of just one process. In the end two different processes can be connected, but that's because of the fork/dup2 call later on.

And about your last sentence. I'm saying:

What I expected to see was

And you're replying with "Not really". I am the one who knows what I expect to see ;). I know it's not like that in reality, but my alternative actually does make sence. But you're thinking of how everything is working right now, but I am thinking of a new pipe function that already includes fork/dup2 etc., so you only have to call:

pipe(fd_pipe_in, process fd_pipe_in, fd_pipe_out, process fd_pipe_out)

If it would work like that, then you can say that the pipe call connects two processes. But it's not working like that, so you can only say that it is connecting two file descriptors of one specific process.

P.S. And I'm talking about the pipe call itself, because in the end of course a pipe can be in between two fd's of different processes (but that's not because of the pipe call, but more about fork/dup2).

Yeah I think I got it already for a while ;), but I'm glad you're seeing that now too.

But that's why https://www.geeksforgeeks.org/pipe-system-call/ is misleading (and many other websites). They are trying to explain a pipe call and they are saying:

a pipe is a connection between two processes

Then two connected file descriptors within one fd table would not be a pipe because it's not a connection between two processes. However, this is also a pipe. That's why this website and many other websites are incorrect about it.

A couple of days ago I didn't know yet what a pipe was, but that confused me a lot. Now I can see through this mistake.

Time for you to close the Q now.

Sorry, yeah sometimes I'll leave the question a bit longer open, because I need some time to process all the information in my head. Usually this results in some last questions about it. Anyway eventually I'll also go through all my other questions to "press the buttons". After two weeks or something I can also better determine which answer is the right one.

But this question can be closed pretty fast I think. In summary:

In general, a pipe connects two file descriptors of one specific process

So this means that https://www.geeksforgeeks.org/pipe-system-call/ is indeed wrong about pipes. They are saying:

a pipe is a connection between two processes

First you said:

The geeksforgeeks post looks fine to me.

I disagree with this and that's why I didn't close this question yet. Or meanwhile you agree with me that this is incorrect on geeksforgeeks?

Conceptually, a pipe is a connection between two processes

I agree with you - that is wrong. The statement confuses the most frequent use of a pipe with what a pipe actually is. But, if you try to re-word that statement to be pedantically correct then I suggest you are going to lose half your readers.
You asked this Q to increase your understanding and you've increased it.

Okay thanks! I prefer something is correct, rather than losing some readers ;). But that's a different discussion. My question has indeed been answered.Thanks for everything!