Solved

/proc/self directory confusion!

Posted on 2014-03-23
20
366 Views
Last Modified: 2014-03-26
Ah hello

I am reading a complex book on Linux that introduced the /proc/self directory.  Please bare in mind that I come from a heavy Windows background, so this is all really new to me.  Now, In an attempt to find more information on this directory, I read on an Oracle blog that stated the following command

ls -l /proc/self

may output the following

lrwxrwxrwx 1 root root 64 Jan 6 13:22 /proc/self -> 13833

Where process 13833 is that of ls -l; hence, the directory /proc/13883 will have existed but only for the duration of the call to ls -l was running.  I understand this, but what I don't quite get is what is happening when I type cd /proc/self: what process is this?  

1) Is it my current bash shell?  

I assumed that this was the case, so was trying to find a way to change one of the file descriptors currently open:

[Me@OLE6 fd]$ cd /proc/self/fd
[Me@OLE6 fd]$ ls -l
total 0
lr-x------. 1 Me Me 64 Mar 23 19:10 0 -> /dev/pts/0
lrwx------. 1 Me Me 64 Mar 23 19:10 1 -> /dev/pts/0
lrwx------. 1 Me Me 64 Mar 23 19:10 2 -> /dev/pts/0
lrwx------. 1 Me Me 64 Mar 23 19:24 255 -> /dev/pts/0
[Me@OLE6 fd]$


I figured if I could redirect stdin, stdout, stderr in my bash shell, to something else, then I could check back in this directory and expect to see the output from the above command change - then I would know that /proc/self is the current bash shell.  

However, I couldn't see how to do this.  I tried 2>stderr.txt in an attempt to redirect stderr but that had no effect.  I also tried opening a second terminal and commanding 1>/dev/pts/1, but that didn't work either (whereas a single string, like echo Hi>/dev/pts/1 did work as expected)

2) How can I redirect one of the three standard files to somewhere else?

3) What is FD 255 in the output I showed above?


100 points/question

TIA!
0
Comment
Question by:mrwad99
  • 8
  • 7
  • 4
  • +1
20 Comments
 
LVL 40

Expert Comment

by:omarfarid
Comment Utility
The cd command is internal command to bash (shell) and hence no new process will be created to execute.

Your question is not clear. Please elaborate.
0
 
LVL 19

Expert Comment

by:simon3270
Comment Utility
The /proc filesystem is largely a reporting device, not a control one (yes, I know you can set some values, but generally not).

So, /proc/self/fd just lists active file descriptors.

If you want to redirect one of the standard streams, use commands such as:
    exec 2>/tmp/fred
to redirect stderr to /tmp/fred.  An "ls -l " of /proc/self/fd will show the redirection.
me@localhost:~$ exec 2>/tmp/fred2
(I then typed ls -l /proc/self/fd, but it wasn't echoed)
total 0
lrwx------. 1 sar sar 64 Mar 24 09:36 0 -> /dev/pts/5
lrwx------. 1 sar sar 64 Mar 24 09:36 1 -> /dev/pts/5
l-wx------. 1 sar sar 64 Mar 24 09:36 2 -> /tmp/fred2
lr-x------. 1 sar sar 64 Mar 24 09:36 3 -> /proc/4672/fd

Open in new window

0
 
LVL 19

Expert Comment

by:simon3270
Comment Utility
3. I think the FD 255 is an artifact of the ls command.  If you specify a directory to be listed, FD 3 looks as though it is set to point to that directory (the /proc/4672/fd above), and there is no FD 255. If you don't specify a directory on the command line (i.e. just "ls -l"), you get 255 pointing to your terminal.
0
 
LVL 19

Author Comment

by:mrwad99
Comment Utility
Firstly, thanks to both for commenting.

OK I think I am learning stuff here, but it has generated more (related) questions (which is convenient since I originally stated 100 points were on offer for each of my questions, yet left the total points at 500!)

omarfarid:

For the record, as I said above ls -l /proc/self yielded 13833, which meant that in this case, /proc/self was actually /proc/13883.  If I Just sit at the command prompt and type cd /proc/self, I wanted to know what directory (that is, directory of the form, /proc/<PID>) this actually is.  I think I have got my answer to that though from what Simon was saying...

Simon:

Thanks.

Your output from the redirection of stderr got me thinking, so I had a play...

[Me@OLE6 self]$ cd /proc/self
[Me@OLE6 self]$ ls -l | grep cwd
lrwxrwxrwx. 1 Me Me 0 Mar 24 11:10 cwd -> /proc/2753
[Me@OLE6 self]$ ps -aef | grep 2753 | grep -v grep
Me      2753  2750  0 10:59 pts/0    00:00:00 bash
Me      2832  2753  0 11:12 pts/0    00:00:00 ps -aef


This definitively shows that when I cd to /proc/self, it is in fact the current bash shell process whose directory I have changed to!  So, question 1 and 3 seem to be answered :)

So I then explored what you were saying about redirecting stderr: question 2.

4)

I don't understand why 2>/tmp/fred doesn't work, why do we have to use exec?

5)

We are redirecting stderr so, as expected, if I type some unknown command and then examine /tmp/fred, I find the expected complaint within it.  But I don't understand why my commands are not being echoed as I type them.  (If I redirect stdout to this file, via exec 1>/tmp/fred, and I enter a valid command, the output from it goes to the file as expected (but an invalid command's output goes to the terminal).  So why doesn't stderr work in this way?

6)

I didn't think about it at the time, but the original Oracle blog talks about the command ls -l /proc/self, which - on my machine - yields

[Me@OLE6 self]$ ls -l /proc/self
lrwxrwxrwx. 1 root root 64 Mar 24 11:51 /proc/self -> 3192


Now, when I run ls -l /tmp, I get a load of output.  Indeed, if I cd to /proc/self, then run ls -l, I get more output.  So, we have agreed that the 3192 would correspond to a directory that once existed with the path /proc/3192, but why does ls -l not list any of those directories that would have existed: it seems the bash shell is smart enough to know that it is only a temporary directory so just lists 3192, but if I didn't know what that actually meant (i.e. process ID 3192) it would be totally confusing!  

Any input on this please?
0
 
LVL 19

Accepted Solution

by:
simon3270 earned 350 total points
Comment Utility
4) You use "exec" because that's the bash syntax.  Just
    2> /tmp/fred
  will redirect sterr for that command line - since there is no actual command, there is no error output  It's the same as if you had written
    ls 2> /tmp/fred
the standard error from the "ls" command goes to /tmp/fred. but after running that command, standard error is no longer redirected.

5) The prompt, and the command you enter, are sent to the shell's standard error, not the standard out.  If you did
    exec >/tmp/fred
then the prompt and the entered command would still appear at the console, but any standard out from the commands you ran would be sent to /tmp/fred.

6) The "ls -l" of a symbolic link displays the link itself, and the directory it links to.  if you had done "ls -lL /proc/self" (the "-L" tells ls to follow symlinks), it would have listed the contents of the directory that /proc/self points to.  Try it with normal symlinks.  You would also have go the contents of the linked directory if you had added a "/" to the end of the path, as "ls -l /proc/self/"
0
 
LVL 19

Author Comment

by:mrwad99
Comment Utility
Thanks.  Your explanation as to why I am not seeing the prompt when I type

exec 2>/tmp/fred

makes sense, but I don't understand why this command should send the prompt and the characters I type to /tmp/fred when I have only (apparently) told it to send stderr there!  I looked at http://linux.about.com/od/commands/l/blcmdln_exec.htm and it also states that this should just redirect stderr.  So what command do I need to enter so I a) see the prompt with what I type echoed, b) get all stdout on my terminal and c) get all errors to /tmp/fred?
0
 
LVL 19

Assisted Solution

by:simon3270
simon3270 earned 350 total points
Comment Utility
Sorry, I don't think you can separate them.  The shell is sending its prompt, and the echoed command line, to stderr.  When it runs a program, stdout from the program is sent to the shell's stdout, and stderr from the program to the shell's stderr.

If you want to separate them, you need to individually send stderr to /tmp/fred on the command lines that you enter.
0
 
LVL 34

Assisted Solution

by:Duncan Roe
Duncan Roe earned 150 total points
Comment Utility
All your 5 questions have been answered correctly above. Still, you may be finding it inconvenient that you no longer see prompts nor your typing. Here's something you can do about that
Suppose you have typed exec 2>/tmp/bash.out
In another window, type tail -f /tmp/bash.out
I just tried this - works fine
0
 
LVL 34

Expert Comment

by:Duncan Roe
Comment Utility
To restore normal behavior in your original window, type exec 2>/dev/tty. You need to Control-C the tail window.
Further to my previous post, all 6 questions above have been answered correctly
0
 
LVL 19

Author Comment

by:mrwad99
Comment Utility
Thanks Duncan.  I still find it ludicrous how we cannot just send stderr to a different location, but we can send just stdout.

What could possibly have been the logic behind this?
0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 
LVL 19

Expert Comment

by:simon3270
Comment Utility
I suppose that this was not intended to be used in interactive shells.  If you are running a shell script and have
    exec 2>/tmp/fred
in that script, then stderr from all subsequent commands (but not the command names themselves) will be sent to /tmp/fred.

If you are running an interactive shell, you only have 2 standard output streams to play with, and the prompt and command-line echo have to go somewhere.  Stderr was the more obvious choice, since it would otherwise get in the way of normal data.
0
 
LVL 34

Expert Comment

by:Duncan Roe
Comment Utility
You are sending stderr to a different location - the "2" in exec 2>/tmp/bash.out is stderr. Thinking about it, maybe I should have suggested you call the file /tmp/bash.err.
If you had entered 1>/tmp/bash.out you would have redirected stdout. Bash accepts simply >/tmp/bash.out for stdout, as does CMD.EXE under Windows.
(CMD.EXE also has pipes and various other flavors of input & output redirection, providing equivalent functionality to the old Bourne shell (bash is the Bourne-again shell). This from the pre-Windows days when DOS and Unix were rivals).
0
 
LVL 19

Author Comment

by:mrwad99
Comment Utility
Thanks Simon.

Duncan:

Yep, I get that; what I was asking was why we cannot send stderr alone (that is, *just* the error messages - *not* the prompt and the echoed characters I type) to another location, like I can send *just* stdout (that is, I retain my prompt and see my echoed characters) to another location.

Simon stated that this could be because this method of redirecting stderr was not meant to be used in interactive prompts; would you agree?
0
 
LVL 34

Assisted Solution

by:Duncan Roe
Duncan Roe earned 150 total points
Comment Utility
Actually no, I don't agree that it was meant to be anything in particular. (Actually I think it's rather neat that you can get to see your typing in another window from the output - I could have done with that in the past on several occasions).
Simon is absolutely right that bash only has 2 output file units and must reserve stdout for actual program output [this because doing anything else would mess up pipes (where the output from one program becomes the input to the next)].
The reason you see your typing is that bash actively echoes each character as you type it. This is an artifact of the readline library, which enables all the cool command-line editing you can do in bash. The old Bourne shell had none of that - the display was switched to local echo so you couldn't ever redirect your typing.

To separate error messages from prompts &c., bash would have to open a 3rd output channel to the display device. But it would only want to do that if stderr was a tty. Or failing that, if stdout was a tty. Maybe. It just gets too messy, don't you agree?
0
 
LVL 19

Author Comment

by:mrwad99
Comment Utility
OK, I'm sorry but I'm still not getting this, please bear with me.

The following have been stated:

Simon:

"If you are running an interactive shell, you only have 2 standard output streams to play with, and the prompt and command-line echo have to go somewhere.  Stderr was the more obvious choice, since it would otherwise get in the way of normal data."

Duncan:

"Simon is absolutely right that bash only has 2 output file units and must reserve stdout for actual program output"

But when I don't attempt to redirect stderr (leave the fds as the default) I have errors, output and the prompt/echoed characters all going to stdout!  This contradicts both of the quoted statements above!  

(Please don't get me wrong, its not that I don't believe you, but instead the fact that *I don't fully understand what is going on yet*, and until I do I can't let it go :))
0
 
LVL 19

Expert Comment

by:simon3270
Comment Utility
No, you don't have all characters going to stdout - normal data goes to stdout, prompts, command line echoes and error output from commands all got to stderr.  The confusion is that normally stdout and stderr point to the same place, so the user can't tell which stream is being used.

This is acknowledged by the "expect" program, where stdout and stderr are treated as a single stream (so "expect" is waiting for a specific string to appear in output, it doesn't matter which stream is used).  This is done precisely because "expect" was intended to automate user tasks, and if a user can't differentiate between stdout and stderr to the console, then "expect" shouldn't either.
0
 
LVL 19

Author Comment

by:mrwad99
Comment Utility
So, you are definitively saying that in a bash shell, by default all echoed characters, and the prompt, are sent to stderr, not stdout, because it would otherwise interfere with program output (the last bit I have got from your earlier comments)?  

If this is the case, can you point me at some official documentation that states this? (Again, not that I don't believe you, but when I write all this up for my personal notes, an official link would be nice)

Also, if all this is correct...

Scrolling back up shows that you said the same thing earlier in http:#a39950901

"The shell is sending its prompt, and the echoed command line, to stderr."

I hence owe you an apology: the first time I read this, I didn't realise you were stating that this is what the default behaviour is, I thought you were just re-stating what I had myself noted.  Sorry!
0
 
LVL 19

Assisted Solution

by:simon3270
simon3270 earned 350 total points
Comment Utility
The best "official" documentation is probably the source code for bash (I used bash-4.1.2-15.el6_4.src.rpm).

In parse.y, line 4889:
static void
print_prompt ()
{
  fprintf (stderr, "%s", current_decoded_prompt);
  fflush (stderr);
}

Open in new window

0
 
LVL 19

Author Closing Comment

by:mrwad99
Comment Utility
Many thanks for sticking with me on this one both; I really appreciate your help in understanding all this :)
0
 
LVL 19

Expert Comment

by:simon3270
Comment Utility
No problem - an interesting investigation for me too!

In case you ever wanted to do it, this will log stdout and stderr separately into files from an interactive shell, but still display stderr (including prompt and command line) and stdout on the console.  It assumes that your current console is /dev/pts/0 (you can check it by "ls -l /proc/self/fd").
exec 3> /dev/pts/0
exec 2> /tmp/fred2
tail -f /tmp/fred2 >&3 &
exec > $outout
tail -f $outout >&3 &

Open in new window

If you want the output to be fairly clean, make your prompt (the PS1 shell variable) as simple as possible - certainly with no colour.  Probably best to:
    PS1='$ '

(forgot to say, you'll need to kill the "tail" commands when you have finished logging!)

(Just discovered the "--pid" parameter on tail - it kills the tail if the specified PID dies.
 So, use commands such as:
    tail -f /tmp/fred2 --pid $$  >&3 &
 to get the "tail" to die when the shell exits - $$ is variable with the shell's PID)
0

Featured Post

What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

Join & Write a Comment

Currently, there is not an RPM package available under the RHEL/Fedora/CentOS distributions that gives you a quick and easy way to allow PHP to interface with Oracle. As a result, I have included a set of instructions on how to do this with minimal …
rdate is a Linux command and the network time protocol for immediate date and time setup from another machine. The clocks are synchronized by entering rdate with the -s switch (command without switch just checks the time but does not set anything). …
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now