it in preference to machine time.

In the early minicomputer days of Unix, this was still a fairly radical idea (machines were a
great deal slower and more expensive then). Nowadays, with every development shop and
most users (apart from the few modeling nuclear explosions or doing 3D movie animation)
awash in cheap machine cycles, it may seem too obvious to need saying.

Somehow, though, practice doesn't seem to have quite caught up with reality. If we took this
maxim really seriously throughout software development, the percentage of applications
written in higher-level languages like Perl, Tcl, Python, Java, Lisp and even shell 鈥�
languages that ease the programmer's burden by doing their own memory management
[Ravenbrook] would be rising fast.

And indeed this is happening within the Unix world, though outside it most applications
shops still seem stuck with the old-school Unix strategy of coding in C (or C++). Later in
this book we'll discuss this strategy and its tradeoffs in detail.

One other obvious way to conserve programmer time is to teach machines how to do more of
the low-level work of programming. This leads to...

Rule of Generation: Avoid hand-hacking; write programs to
write programs when you can.

Human beings are notoriously bad at sweating the details. Accordingly, any kind of hand-
hacking of programs is a rich source of delays and errors. The simpler and more abstracted
your program specification can be, the more likely it is that the human designer will have
gotten it right. Generated code (at every level) is almost always cheaper and more reliable
than hand-hacked.

We all know this is true (it's why we have compilers and interpreters, after all) but we often
don't think about the implications. High-level-language code that's repetitive and mind-
numbing for humans to write is just as productive a target for a code generator as machine
code. It pays to use code generators when they can raise the level of abstraction 鈥� that is,
when the specification language is simpler than the generated code, and the code doesn't
have to be hand-hacked afterwards.

In the Unix tradition, code generators are heavily used to automate error-prone detail work.

Parser/lexer generators are the classic examples; makefile generators and GUI interface
builders are newer ones.

(We cover these techniques in Chapter 9 (Generation).)

Rule of Representation: Use smart data so program logic
can be stupid and robust.

Even the simplest procedural logic is hard for humans to verify, but quite complex data
structures are fairly easy to model and reason about. To see this, compare the expressiveness
and explanatory power of a diagram of (say) a fifty-node pointer tree with a flowchart of a
fifty-line program. Or, compare a C initializer expressing a conversion table with an
equivalent switch statement. The difference in transparency and clarity is dramatic.

Data is more tractable than program logic. It follows that where you see a choice between
complexity in data structures and complexity in code, choose the former. More: in evolving a
design, you should actively seek ways to shift complexity from code to data.

The Unix community did not originate this insight, but a lot of Unix code displays its
influence. The C language's facility at manipulating pointers, in particular, has encouraged
the use of dynamically-modified reference structures at all levels of coding from the kernel
upward. Simple pointer chases in such structures frequently do duties that implementations
in other languages would instead have to embody in more elaborate procedures.

(We also cover these techniques in Chapter 9 (Generation).)

Rule of Separation: Separate policy from mechanism;
separate interfaces from engines.

In our discussion of what Unix gets wrong, we observed that the designers of X made a basic
decision to implement 鈥渕echanism, not policy鈥� 鈥� to make X a generic graphics engine and
leave decisions about user-interface style to toolkits and other levels of the system. We
justified this by pointing out that policy and mechanism tend to mutate on different
timescales, with policy changing much faster than mechanism. Fashions in the look and feel
of GUI toolkits may come and go, but raster operations are forever.

Thus, hardwiring policy and mechanism together has two bad effects; it make policy rigid

and harder to change in response to user requirements, and it means that trying to change
policy has a strong tendency to destabilize the mechanisms.

On the other hand, by separating the two we make it possible to experiment with new policy
without breaking mechanisms. We also make it much easier to write good tests for the
mechanism (policy, because it ages so quickly, often does not justify the investment)

This design rule has wide application outside of the GUI context. In general, it implies that
we should look for ways to separate interfaces from engines.

One way to do this, for example, is to write your application as a library of C service routines
that are driven by an embedded scripting language, with the application flow of control
written in the scripting language rather than C. A classic example of this pattern is the Emacs
editor, which uses an embedded Lisp interpreter to control editing primitives written in C.
We discuss this style of design in Chapter 11 (User Interfaces).

Another way is to separate your application into cooperating front-end and back-end
processes communicating via a specialized application protocol over sockets; we discuss this
kind of design in Chapters 5 (Textuality) and 6 (Multiprogramming). The front end
implements policy, the back end mechanism. The global complexity of the pair will often be
far lower than that of a single-process monolith implementing the same functions, reducing
your vulnerability to bugs and lowering life-cycle costs.

Rule of Optimization: Prototype before polishing. Get it
working before you optimize it.

The most basic argument for prototyping first is Kernighan & Plauger's; 鈥�90% of the
functionality delivered now is better than 100% of it delivered never.鈥� Prototyping first may
help keep you from investing far too much time for marginal gains.

For slightly different reasons, Donald Knuth (author of The Art Of Computer Programming,
one of the field's few true classics) once said 鈥淧remature optimization is the root of all
evil.鈥漑5] And he was right.

Rushing to optimize before the bottlenecks are known may be the only error to have ruined
more designs than feature creep. From tortured code to incomprehensible data layouts, the
results of obsessing about speed or memory or disk usage at the expense of transparency and

simplicity are everywhere. They spawn innumerable bugs and cost millions of man-hours 鈥�
often, just to get marginal gains in the use of some resource much less expensive than
debugging time.

Disturbingly often, premature local optimization actually hinders global optimization (and
hence reduces overall performance). A prematurely optimized portion of a design frequently
interferes with changes that would have much higher payoffs across the whole design, so you
end up with both inferior performance and excessively complex code.

In the Unix world there is a long-established and very explicit tradition (exemplified by Rob
Pike's comments above and Ken Thompson's maxim about brute force) that says: Prototype,
then polish. Get it working before you optimize it. Or: Make it work first, then make it work
fast. 鈥楨xtreme programming' guru Kent Beck, operating in a different culture, has usefully
amplified this to: 鈥淢ake it run, then make it right, then make it fast.鈥�

The thrust of all these quotes is the same: get your design right with an un-optimized, slow,
memory-intensive implementation before you try to tune. Then you tune systematically,
looking for the places where you can buy big performance wins with the smallest possible
increases in local complexity.

It's worth pointing out that you don't have to optimize what you don't write. The most
powerful optimization tool in existence may be the delete key.

Finally, it is almost never worth doing optimizations that reduce resource use by merely a
constant factor; it's smarter to concentrate effort on cases where you can reduce average-case
runtime or space use from O(n2) to O(n) or O(n log n), or similarly reduce from a higher
order. Linear performance gains tend to be swamped by the exponential effect of Moore's
Law 鈥� the smartest, cheapest, and often fastest way to collect those gains is to wait a few
months for your target hardware to become more capable.

Rule of Diversity: Distrust all claims for 鈥渙ne true way鈥�.

Even the best software tools tend to be limited by the imaginations of their designers.
Nobody is smart enough to optimize for everything, nor to anticipate all the uses to which
their software might be put. Designing rigid, closed software that won't talk to the rest of the
world is an unhealthy form of arrogance.

Therefore, the Unix tradition includes a healthy mistrust of 鈥渙ne true way鈥� approaches to

software design or implementation. It embraces multiple languages, open extensible systems,
and customization hooks everywhere.

Rule of Extensibility: Design for the future, because it will be
here sooner than you think.

If it is unwise to trust other people's claims for 鈥渙ne true way鈥�, it's even more foolish to
believe them about your own designs. Never assume you have the final answer.

Therefore, leave room for your code to grow. When you write protocols or file formats,
make them sufficiently self-describing to be extensible. When you write code, organize it so
future developers will be able to plug new functions into the architecture without having to
scrap and rebuild the architecture. Make the joints flexible, and put 鈥淚f you ever need to...鈥�
comments in your code. You owe this grace to people who will use and maintain your code
after you.

You'll be there in the future too, maintaining code you may have half forgotten under the
press of more recent projects. When you design for the future, the sanity you save may be
your own.


[3] Pike's original adds 鈥�(See Brooks p. 102.)鈥� here. The reference is to an early edition of
The Mythical Man-Month [Brooks]; the quote is 鈥淪how me your flow charts and conceal
your tables and I shall continue to be mystified, show me your tables and I won't usually
need your flow charts; they'll be obvious.鈥�)

[4] Jonathan Postel was the first editor of the Internet RFC series of standards, and one of the
principal architects of the Internet. A tribute page is maintained by the Postel Center for
Experimental Networking.

[5] In full:鈥淲e should forget about small efficiencies, say about 97% of the time: premature
optimization is the root of all evil.鈥� Knuth attribites the remark to C.A.R Hoare.

Prev Up Next

What Unix gets right

Home The Unix philosophy in one lesson

The Unix philosophy in one lesson

Prev Chapter 1. Philosophy

Next



The Unix philosophy in one lesson

All the philosophy really boils down to one iron law, the hallowed 鈥楰ISS principle鈥� of
master engineers everywhere:

KEEP IT SIMPLE, STUPID!



Unix gives you an excellent base for applying the KISS principle. The remainder of this
book will help you learn how to use it.

Prev Up Next

Basics of the Unix philosophy

Home Applying the Unix philosophy

Applying the Unix philosophy

Prev Chapter 1. Philosophy

Next



Applying the Unix philosophy

These philosophical principles aren't just vague generalities. In the Unix world they come
straight from experience and lead to specific prescriptions, some of which we've already
developed above. Here's a by no means exhaustive list:

1.
Everything that can be a source- and destination-independent filter should be one.
2.
Data streams should if at all possible be textual (so they can be viewed and filtered
with standard tools).
3.
Database layouts and application protocols should if at all possible be textual (human-
readable and human-editable).
4.
Complex front ends (user interfaces) should be cleanly separated from complex back
ends.
5.
Whenever possible, prototype in an interpretive language before coding C.
6.
Mixing languages is better than writing everything in one, if and only if using only
that one is likely to over-complicate the program.
7.
Be generous in what you accept, rigorous in what you emit.
8.
When filtering, never throw away information you don't need to.
9.
Small is beautiful. Write programs that do as little as is consistent with getting the job
done.


We'll see the Unix design rules, and the prescriptions that derive from them, applied over and
over again in the remainder of this book. Unsurprisingly, they tend to converge with the very
best practices from software engineering in other traditions. [6]

[6] One notable example is Butler Lampson's Hints for Computer System Design [Lampson],
which the author discovered late in the preparation of this book. It not only expresses a
number of Unix dicta in forms that were clearly discovered independently, but uses many of
the same tag lines to illustrate them.

Prev Up Next

The Unix philosophy in one lesson

Home Attitude matters too

Attitude matters too

Prev Chapter 1. Philosophy

Next



Attitude matters too

When you see the right thing, do it 鈥� this may look like more work in the short term, but it's
the path of least effort in the long run. If you don't know what the right thing is, do the
minimum necessary to get the job done, at least until you figure out what the right thing is.

To do the Unix philosophy right, you have to be loyal to excellence. You have to believe that
software design is a craft worth all the intelligence, creativity, and passion you can muster.
Otherwise you won't look past the easy, stereotyped ways of approaching design and
implementation; you'll rush into coding when you should be thinking. Otherwise you'll
carelessly complicate when you should be relentlessly simplifying 鈥� and then you'll wonder
why your code bloats and debugging is so hard.

To do the Unix philosophy right, you have to value your own time enough never to waste it.
If someone has already solved a problem once, don't let pride or politics suck you into
solving it a second time rather than re-using. And never work harder than you have to; work
smarter instead, and save the extra effort for when you need it. Lean on your tools and
automate everything you can.

Software design and implementation should be a joyous art, a kind of high-level play. If this
attitude seems preposterous or vaguely embarrassing to you, stop and think; ask yourself
what you've forgotten. Why do you design software instead of doing something else to make
money or pass the time? You must have thought software was worthy of your passion
once....

To do the Unix philosophy right, you need to have (or recover) that attitude. You need to
care. You need to play. You need to be willing to explore.

We hope you'll bring this attitude to the rest of this book. Or, at least, that this book will help
you rediscover it.

Prev Up Next

Applying the Unix philosophy

Home Chapter 2. History

Chapter 2. History

Prev Part I. Context

Next



Chapter 2. History

A Tale of Two Cultures

Table of Contents

Origins and history of Unix, 1969-1995
Genesis: 1969-1971
Exodus: 1971-1980
TCP/IP and the Unix Wars: 1980-1990
Blows against the empire: 1991-1995



Origins and history of the hackers, 1961-1995
At play in the groves of academe: 1961-1980
Internet fusion and the Free Software Movement: 1981-1991
Linux and the pragmatist reaction: 1991-1998



The open-source movement: 1998 and onward.
The lessons of Unix history


Those who cannot remember the past are condemned to repeat it.

--George Santayana, The Life of Reason (1905)

The past informs practice. Unix has a long and colorful history, much of which is still live as
folklore, assumptions, and (too often) battle scars in the collective memory of Unix
programmers. In this chapter we'll survey the history of Unix, with an eye to explaining why,
in 2003, today's Unix culture looks the way it does.

Prev Up Next

Attitude matters too

Home Origins and history of Unix, 1969-
1995