The Art of Unix Programming

haoji

it in preference to machine time.

In the early minicomputer days of Unix, this was still a fairly radical idea (machines were a
great deal slower and more expensive then). Nowadays, with every development shop and
most users (apart from the few modeling nuclear explosions or doing 3D movie animation)
awash in cheap machine cycles, it may seem too obvious to need saying.

Somehow, though, practice doesn't seem to have quite caught up with reality. If we took this
maxim really seriously throughout software development, the percentage of applications
written in higher-level languages like Perl, Tcl, Python, Java, Lisp and even shell 鈥

haoji

Parser/lexer generators are the classic examples; makefile generators and GUI interface
builders are newer ones.

(We cover these techniques in Chapter 9 (Generation).)

Rule of Representation: Use smart data so program logic
can be stupid and robust.

Even the simplest procedural logic is hard for humans to verify, but quite complex data
structures are fairly easy to model and reason about. To see this, compare the expressiveness
and explanatory power of a diagram of (say) a fifty-node pointer tree with a flowchart of a
fifty-line program. Or, compare a C initializer expressing a conversion table with an
equivalent switch statement. The difference in transparency and clarity is dramatic.

Data is more tractable than program logic. It follows that where you see a choice between
complexity in data structures and complexity in code, choose the former. More: in evolving a
design, you should actively seek ways to shift complexity from code to data.

The Unix community did not originate this insight, but a lot of Unix code displays its
influence. The C language's facility at manipulating pointers, in particular, has encouraged
the use of dynamically-modified reference structures at all levels of coding from the kernel
upward. Simple pointer chases in such structures frequently do duties that implementations
in other languages would instead have to embody in more elaborate procedures.

(We also cover these techniques in Chapter 9 (Generation).)

Rule of Separation: Separate policy from mechanism;
separate interfaces from engines.

In our discussion of what Unix gets wrong, we observed that the designers of X made a basic
decision to implement 鈥渕echanism, not policy鈥

haoji

and harder to change in response to user requirements, and it means that trying to change
policy has a strong tendency to destabilize the mechanisms.

On the other hand, by separating the two we make it possible to experiment with new policy
without breaking mechanisms. We also make it much easier to write good tests for the
mechanism (policy, because it ages so quickly, often does not justify the investment)

This design rule has wide application outside of the GUI context. In general, it implies that
we should look for ways to separate interfaces from engines.

One way to do this, for example, is to write your application as a library of C service routines
that are driven by an embedded scripting language, with the application flow of control
written in the scripting language rather than C. A classic example of this pattern is the Emacs
editor, which uses an embedded Lisp interpreter to control editing primitives written in C.
We discuss this style of design in Chapter 11 (User Interfaces).

Another way is to separate your application into cooperating front-end and back-end
processes communicating via a specialized application protocol over sockets; we discuss this
kind of design in Chapters 5 (Textuality) and 6 (Multiprogramming). The front end
implements policy, the back end mechanism. The global complexity of the pair will often be
far lower than that of a single-process monolith implementing the same functions, reducing
your vulnerability to bugs and lowering life-cycle costs.

Rule of Optimization: Prototype before polishing. Get it
working before you optimize it.

The most basic argument for prototyping first is Kernighan & Plauger's; 鈥

haoji

simplicity are everywhere. They spawn innumerable bugs and cost millions of man-hours 鈥

haoji

software design or implementation. It embraces multiple languages, open extensible systems,
and customization hooks everywhere.

Rule of Extensibility: Design for the future, because it will be
here sooner than you think.

If it is unwise to trust other people's claims for 鈥渙ne true way鈥

haoji

The Unix philosophy in one lesson

Prev Chapter 1. Philosophy

Next



The Unix philosophy in one lesson

All the philosophy really boils down to one iron law, the hallowed 鈥楰ISS principle鈥

haoji

Applying the Unix philosophy

Prev Chapter 1. Philosophy

Next



Applying the Unix philosophy

These philosophical principles aren't just vague generalities. In the Unix world they come
straight from experience and lead to specific prescriptions, some of which we've already
developed above. Here's a by no means exhaustive list:

1.
Everything that can be a source- and destination-independent filter should be one.
2.
Data streams should if at all possible be textual (so they can be viewed and filtered
with standard tools).
3.
Database layouts and application protocols should if at all possible be textual (human-
readable and human-editable).
4.
Complex front ends (user interfaces) should be cleanly separated from complex back
ends.
5.
Whenever possible, prototype in an interpretive language before coding C.
6.
Mixing languages is better than writing everything in one, if and only if using only
that one is likely to over-complicate the program.
7.
Be generous in what you accept, rigorous in what you emit.
8.
When filtering, never throw away information you don't need to.
9.
Small is beautiful. Write programs that do as little as is consistent with getting the job
done.


We'll see the Unix design rules, and the prescriptions that derive from them, applied over and
over again in the remainder of this book. Unsurprisingly, they tend to converge with the very
best practices from software engineering in other traditions. [6]

haoji

[6] One notable example is Butler Lampson's Hints for Computer System Design [Lampson],
which the author discovered late in the preparation of this book. It not only expresses a
number of Unix dicta in forms that were clearly discovered independently, but uses many of
the same tag lines to illustrate them.

Prev Up Next

The Unix philosophy in one lesson

Home Attitude matters too

haoji

Attitude matters too

Prev Chapter 1. Philosophy

Next



Attitude matters too

When you see the right thing, do it 鈥

haoji

Chapter 2. History

Prev Part I. Context

Next



Chapter 2. History

A Tale of Two Cultures

Table of Contents

Origins and history of Unix, 1969-1995
Genesis: 1969-1971
Exodus: 1971-1980
TCP/IP and the Unix Wars: 1980-1990
Blows against the empire: 1991-1995



Origins and history of the hackers, 1961-1995
At play in the groves of academe: 1961-1980
Internet fusion and the Free Software Movement: 1981-1991
Linux and the pragmatist reaction: 1991-1998



The open-source movement: 1998 and onward.
The lessons of Unix history


Those who cannot remember the past are condemned to repeat it.

--George Santayana, The Life of Reason (1905)

The past informs practice. Unix has a long and colorful history, much of which is still live as
folklore, assumptions, and (too often) battle scars in the collective memory of Unix
programmers. In this chapter we'll survey the history of Unix, with an eye to explaining why,
in 2003, today's Unix culture looks the way it does.

Prev Up Next

Attitude matters too

Home Origins and history of Unix, 1969-
1995