Age of Reason

Random musing of books and stuff I am reading.

2004/12/11

File Descriptors in Bourne shell (sh,ksh,bash).

In the 1980s Steve Bourne in 11 pages specified the grammar for
a functionally complete shell (see original ATT Unix manual).
It is a tribute to his foresight, that those 11 pages still
suffice to write arbitarily complex shell scripts in 2004.

Marc Rochkind's Advanced Unix Programming book fills in the
elegant plumbing details, and how pipes in sh work.

Compare that to csh and msdos shell, where quoting/aliasing/redirection are
broken, and work only for special common cases. For details read
Tom Christiansen's csh write up
http://www.faqs.org/faqs/unix-faq/shell/csh-whynot

That is the difference between Computer Science (rather mathematical training) grammar and bad engineering hacks. What is a good grammar? It is being able to write arbitarily complex scripts, without having to wonder if it will work.

The elegance of sh has been continued in Perl, while the tradition of csh
has been carried into tcl (in tcl, the parser cannot deal with unbalanced
braces inside comments, and all data is a null terminated string).

If you find sh redirection in configure files spinning your head, read on:



Notation:
FD is file descriptor [0-9-].
&FD is reference to a file descriptor.
WORD is an unglobbed filename.

&- NULL # closed FD.
E<&- FD[E] := NULL # close fd E (default is stdin /0). E>&- FD[E] := NULL # close fd E (default is stdout/1).
<&- FD[0] := NULL # Close stdin >&- FD[1] := NULL # close stdout

FD[-] NULL # closed fd).
FD[0] stdin # default for input
FD[2] stderr

E<>F FD[E] := open(F,rw) # Redirect E to file F in read-write mode
E>&D FD[E] := FD[D] # fd D is duped and result is fd E (default is stdout).
>&D FD[1] := FD[D]
E> F FD[E] := open(F,w) # Redirect fd E to file F (default is stdout).
> F FD[1] := open(F,w) # Redirect stdout to file F.
E>>F FD[E] := open(F,a) # fd E is appeneded to F.

E<&D FD[E] := FD[D] fd D is duped and result is fd E (default is stdin). E<&- FD[E] := NULL, fd E closed. E<<<\STR \n(HERE DOCUMENT $XYZ not expanded\n)*STR E<< -STR \n(\t*HERE DOCUMENT\n)*STR .. Leading \t will be stripped from each line. Notes: > is really 1>
<>/dev/null Leaves STDOUT open
2>/dev/null Discard STDERR
2>&1 Send STDERR to STDOUT instead
2>&- Close STDERR (not recommended)
3<>/dev/tty Open fd 3 to /dev/tty in read-write mode

Only param and command subst,
Not filename or blank interpretations (no globbing).
Eg. $ >b cat a .. copies a to b.
Eg. $ echo x > *.c .. creates '*.c'

Shell ReDirection Examples: moshtag=redirection

Think of each FD as a variable pointing to a fileio_data_buffer.

N>&M FD[N] := FD[M].
N> M FD[N] := open(M,w);
2>&1 Means FD[2] := FD[1].
Now FD[2] stream is lost,
and anything written to FD[2] is sent to FD[1],
since FD[2] is pointing to the FD[1] fileio_data_buffer.

Think of redirection as connecting pipes?
2>&1 Means dup the RHS pipe and connect the LHS source to it.

Eg. $ gcc x.c 2>&1 | grep errors
Eg. $ 2>&1 gcc x.c | grep errors

Order of redirection is always left to right.
Eg. $ gcc x.c > log 2>&1
FD[1] := open(log,w) # 1. First send stdout 1> to ./log
FD[2] := FD[1]; # 2. Then send stderr 2> to &1 (ie. ./log)
# Both FD[1] = FD[2] =

Eg. $ gcc x.c 2>&1 > log
FD[2] := FD[1]; # FD[2] = FD[1] =
FD[1] := open(log,w); # FD[1] =

Eg. $ exec 5>./config.log ; echo "into config.log" 1>&5
FD[5] := open(config.log,w); # Connect pipe 5 to config.log,
FD[1] := FD[5]; # then send pipe 1 data into pipe 5.

Eg. $ cmd 3>&1 1>&2 2>&3 3>&- # Swap stdout and stderr for cmd
# fd3 = fd1 = stdout
# fd1 = fd2 = stderr
# fd2 = fd3 = stdout
# fd3 = null
1. First save stdout as &3 (&1 is duped into 3).
2. Next send stdout to stderr (&2 is duped into 1).
3. Send stderr to &3 (stdout) (&3 is duped into 2).
4. close &3 (&- is duped into 3)

Eg. stderr_output=`cmd 3>&1 1>&2 2>&3 3>&-` moshtag=stderr_output

$ 2>b ls xxx # redirect stderr,
$ cat b
ls: "xxx" not found

# close stdout .. FD[1] :=
$ exec >&-
$ cat b
cat: write error on standard output.

# close stdin .. FD[0] :=
$ exec <&- ... shell exits
Easiest way to learn is to try these in bash.
The firefox is mangling the ampersands in the posting (so watch out).

mosh

1 Comments:

  • At 4:45 PM, Anonymous Anonymous said…

    Put this in .bash_profile
    exec >&-
    PS1=login:

    For MSDOS lovers, put this
    in autoexec.bat

    PROMPT=login:

    See if your users can login in!

    - Mohsin

     

Post a Comment

<< Home