The Art of Unix Programming

haoji

systems like VMS.

BeOS

Be, Inc. was founded in 1989 as a hardware vendor, building pioneering multiprocessing
machines around the PowerPC chip. BeOS was its attempt to add value to the hardware by
inventing a new, network-ready operating system model incorporating the lessons of both
Unix and the MacOS family, without being either. The result was a tasteful, clean, and
exciting design with excellent performance in its chosen role as a multimedia platform.

BeOS's unifying ideas were 鈥榩ervasive threading鈥

haoji

with the Unix model was in place.

Beos's intended role was as a client operating system specialized for quasi-real-time
multimedia processing. Its intended audience included technical and business end-users,
implying a moderate tolerance for interface complexity.

Entry barriers to BeOS development were low; though the operating system was proprietary,
development tools were inexpensive and full documentation was readily available. The
BeOS effort began as part of one of the efforts to unseat Intel's hardware with RISC
technology, and was continued as a software-only effort after the Internet explosion. Its
strategists were paying attention during Linux's formative period in the early 1990s, and
were fully aware of the value of a large casual-developer base. In fact they succeeded in
attracting an intensely loyal following; as of 2003 there are no fewer than five separate
projects attempting to resurrect BeOS in open source.

Unfortunately, the business strategy surrounding BeOS was not as astute as the technical
design. The BeOS software was originally bundled with dedicated hardware, and marketed
with only vague hints about intended applications. Later (199 it was ported to generic PCs
and more closely focused on multimedia applications, but never attracted a critical mass of
applications or users. BeOS finally succumbed in 2001 to a combination of anti-competitive
maneuvering by Microsoft (lawsuit in progress as of 2003) and cost pressure from variants of
Linux that had been adapted for multimedia handling.

Linux

Linux is the leader of the pack of new-school open-source Unixes that have emerged since
1990 (also including FreeBSD, NetBSD, OpenBSD, Darwin, and Cygwin), and is
representative of the design direction being taken by the group as a whole. The trends in it
can be taken as representative for this entire group.

Linux does not include any code from the original Unix source tree, but it was designed from
Unix standards to behave like a Unix. In the rest of this book, we emphasize the continuity
between Unix and Linux. That continuity is extremely strong, both in terms of technology
and key developers 鈥

haoji

was ever the case for the old-school Unixes, which have primarily aimed at the server and
technical-workstation markets. This has implications for the way Linux hackers design
software.

The most obvious change is a shift in preferred interface styles. Unix was originally designed
for use on teletypes and slow printing terminals. Through much of its lifetime it was strongly
associated with character-cell video-display terminals lacking either graphics or color
capabilities. Most Unix programmers stayed firmly wedded to the command line long after
large end-user applications had migrated to X windows-based GUIs, and the design of both
Unix operating systems and their applications have continued to reflect this fact.

Linux users and developers, on the other hand, have been adapting themselves to address the
nontechnical user's fear of CLIs. They have moved to building GUIs and GUI tools much
more intensively than was the case in old-school Unix, or even in contemporary proprietary
Unixes. To a lesser but significant extent, this is true of the other open-source Unixes as well.

The desire to reach end-users has also made Linux developers much more concerned with
smoothness of installation and software distribution issues than is typically the case under
proprietary Unix systems. One consequence is that Linux features binary-package systems
far more sophisticated than any analogues in proprietary Unixes, with interfaces designed (as
of 2003, with only mixed success) to be palatable to nontechnical end users.

The Linux community wants, more than the old-school Unixes ever did, to turn their
software into a sort of universal pipefitting for connecting together other environments.
Thus, Linux features support for reading and (often) writing the filesystem formats and
networking methods native to other operating systems. It also supports multiple-booting with
them on the same hardware, and simulating them in software inside Linux itself. The long-
term goal is subsumption; Linux emulates so it can absorb.[15]

The goal of subsuming the competition, combined with the drive to reach the end-user, has
motivated Linux developers to adopt design ideas from non-Unix operating systems to a
degree that makes traditional Unixes look rather insular. Linux applications using Windows
.INI format files for configuration is a minor example we'll cover in Chapter 10
(Configuration); various attempts to adapt CORBA for Linux desktop projects are another.
Linux 2.5's incorporation of extended file attributes, which among other things can be used
to emulate the semantics of the Macintosh resource fork, is a recent major example at time of
writing.

haoji

The remaining proprietary Unixes are designed to be big products for big IT budgets. Their
economic niche encourages designs optimized for maximum power on high-end, leading-
edge hardware. Because Linux has part of its roots among PC hobbyists, it emphasizes doing
more with less. Where proprietary Unixes are tuned for multiprocessor and server-cluster
operation at the expense of performance on low-end hardware, core Linux developers have
explicitly chosen not to accept more complexity and overhead on low-end machines for
marginal performance gains on high-end hardware.

Indeed, a substantial fraction of the Linux user community is understood to be wringing
usefulness out of hardware as technically obsolete today as Ken Thompson's PDP-7 was in
1969. As a consequence, Linux applications are under pressure to stay lean and mean that
their counterparts under proprietary Unix do not experience.

These trends have implications for the future of Unix as a whole, a topic we'll return to in
Chapter 18 (Futures).


[10] More information is available at the OpenVMS.org site.

[11] AmigaOS Portal

[12] The GEM Operating System

[13] See, for example, the OS Voice and OS/2 BBS.COM sites.

[14] Perhaps. It has been argued that the unifying metaphor of all Microsoft operating systems
is 鈥渢he customer must be locked in鈥

haoji

The elements of operating-system
style

Home What goes around, comes around

haoji

What goes around, comes around

Prev Chapter 3. Contrasts

Next



What goes around, comes around

We attempted to select for comparison time-sharing systems that either are now or have been
in the past competitive with Unix. The field of plausible candidates is not wide. Most
(Multics, TOPS-10, TOPS-20, Aegis, GECOS, RDOS, MPE and perhaps a dozen others) are
so long dead that they are fading from the collective memory of the computing field. Of
those we surveyed, VMS and OS/2 are moribund, and MacOS has been subsumed by a Unix
derivative. Only Microsoft Windows remains as a viable competitor independent of the Unix
tradition.

We pointed out Unix's strengths in Chapter 1 (Philosophy), and they are certainly part of the
explanation. But it's actually more instructive to look at the obverse of that answer and ask
which weaknesses in Unix's competitors did them in.

The most obvious shared problem is non-portability. Most of Unix's pre-1980 competitors
were tied to a single hardware platform, and died with that platform. One reason VMS
survived long enough to merit inclusion here as a case study is that it was successfully ported
off its original VAX hardware to the Alpha processor. MacOS successfully made the jump
from the Motorola 68000 to PowerPC chips in the late 1980s. Microsoft Windows escaped
this problem by being in the right place when commoditization flattened the market for
general-purpose computers into a PC monoculture.

From 1980 on, another particular weakness continually re-emerges as a theme in different
systems that Unix either steamrollered or outlasted: an inability to support networking
gracefully.

In a world of pervasive networking, even an operating system designed for single-user use
needs multi-user capability (multiple privilege groups) 鈥

haoji

Around 1980, during the early heyday of personal computers, operating-system designers
dismissed Unix and traditional timesharing as heavyweight, cumbersome, and inappropriate
for the brave new world of single-user personal machines 鈥

haoji

this is one that really foregrounds itself in any discussion of operating-system design style.

Prev Up Next

Operating-system comparisons

Home Part II. Design

haoji

Part II. Design

Prev

Next



Design

Table of Contents

4. Modularity
Encapsulation and optimal module size
Compactness and orthogonality
Compactness
Orthogonality
The DRY rule
The value of detachment



Top-down, bottom-up, and glue layers
Case study: C considered as thin glue



Library layering
Case study: GIMP plugins



Unix and object-oriented languages
Coding for modularity



5. Textuality
The Importance of Being Textual
Case study: Unix password file format
Case study: .newsrc format
Case study: The PNG graphics file format



Data file metaformats
/etc/passwd style
RFC-822 format
Fortune-cookie format
XML
Windows INI format
Unix textual file format conventions



Application protocol design
Case study: SMTP, a simple socket protocol

haoji

Case study: POP3, the Post Office Protocol
Case study: IMAP, the Internet Message Access Protocol



Application protocol metaformats
The classical Internet application metaprotocol
HTTP as a universal application protocol
BEEP
XML-RPC. SOAP, and Jabber



Binary files as caches



6. Multiprogramming
Separating complexity control from performance tuning
Handing off tasks to specialist programs
Case study: the mutt mail user agent.



Pipes, redirection, and filters
Case study: Piping to a Pager
Case study: making word lists
Case study: pic2graph
Case study: bc(1) and dc(1)



Slave processes
Case study: scp(1) and ssh



Wrappers
Case study: backup scripts



Security wrappers and Bernstein chaining
Peer-to-peer inter-process communication
Signals
System daemons and conventional signals
Case study: fetchmail's use of signals
Temp files
Shared memory via mmap
Sockets
Obsolescent Unix IPC methods



Client-Server Partitioning for Complexity Control
Case study: PostgreSQL
Case study: Freeciv



Two traps to avoid
Remote procedure calls