Free software packages power much of the Internet because they're not just free -- they also work extremely well.
Jon Udell
In August, over 1000 people gathered for the first-ever Perl Conference. Quite a turnout for a freeware product! Of course, Perl's lack of a price tag is not the main reason for its popularity. Folks like me use it because when you need to solve certain kinds of problems, there's nothing else like Perl -- at any price.
The Linux OS and the Apache Web server enjoy similar reputations, a
nd now I can understand why. Neither of these packages has failed in over a year of continuous use on the BYTE conference server. During that same period, our main Web server, running Windows NT 4.0 and Internet Information Server (IIS) 3.0, has suffered a few
software failures. Now the NT server handles about 7500 users per day while the Linux box supports only about 1000, so I can't make an apples-to-apples comparison.
Would my Linux/Apache/INND server handle 15 times more users with equal aplomb? I won't know unless I try, but clearly these freeware packages work very well. It's no accident that Apache owns a commanding share of the world's Web-server pie (
see the graph
).
Of course, the very fabric of the Internet is woven with free software. Without a trio of freeware stalwarts -- Berkeley Internet Name Daemon (BIND), sendmail, and InterNet News (INN) -- there would be no Domain Name System (DNS), Internet mail, or Usenet conferencing.
The Cathedral and the Bazaar
What's going on here? How can free software possibly compete with commercial stuff? Eric Raymond thinks he can explain the apparent paradox. In the final keynote speech at the Perl Conference, Raymond elaborated on his popular essay "The Cathedral and the Bazaar" (
http://www.ccil.org/~esr/writings/cathedral.html
). Practitioners of the cathedral style of software development, according to Raymond, work in small groups, behind closed doors, to produce infrequent major releases that are claimed to be definitive. Meanwhile, practitioners of the bazaar style -- in particular, the Linux developer community -- form a large, loose-knit collective that works in a very public way to create a stream of incremental releases that are always understood to be in
flux.
Who are the cathedral-dwellers? Raymond fingers the Free Software Foundation (FSF) -- a terrible irony, given that both the philosophy and the GNU development tools pioneered by FSF's Richard Stallman are the foundation stones upon which Linux is built. Other exemplars of the cathedral school might include Microsoft and Lotus.
What motivates Linux hackers? The driving force is prestige in the eyes of the Linux community, according to Raymond. Hackers crave recognition above all else. Linus Torvalds is not only an ace kernel hacker, but, more important, a brilliant leader who knows how to harness and satisfy that craving.
Management style alone cannot ac-count for the huge success of Linux, however. Not all freeware projects work the way Linux does; conversely, some commercial projects do. What makes Linux as solid as it has become is the wide distribution and ongoing study of its source code. "Given enough eyeballs," explains Raymond, "all bugs are shallow." That's the most powerful ar
gument in favor of free software: Anyone can inspect it, and anyone can improve it.
But I Just Want to *Use* the Thing!
Clearly, most people don't have the time, inclination, or ability to study -- much less comprehend and modify -- the internal workings of their OSes, Web servers, or programming languages. We just want to install them easily and have them work as advertised. We want documentation that's helpful. If something fails, we want to call for, and receive, timely assistance.
Freeware developers aren't required to provide this kind of service and support. Corporate users therefore regard free software with a healthy dose of suspicion. In theory, commercial developers are obliged to fix what's broken. In practice, freeware developers sometimes respond as fast or faster than their commercial counterparts. But there's no guarantee, so this is a tough argument to pitch to a CIO.
Enter commercial freeware, a seemingly paradoxical business model practiced by companies such as Cyg
nus Solutions (commercial GNU development tools
http://www.cygnus.com/
), C2Net Software (commercial Apache
http://www.c2.org/
), and Caldera (commercial Linux
http://www.caldera.com/
). Their products add packaging, customization, support, and documentation to freeware cores.
Cygnus's forte, for example, is the adaptation of the GNU C compiler to new hardware. C2Net adds Secure Sockets Layer (SSL) capability and GUI configura
tion to Apache. Caldera adds NetWare client support to Linux. These freeware commercializers walk a fine line that separates symbiosis from exploitation.
Enlightened Self-Interest
In 1987, Michael Tiemann and others founded Cygnus, seeing an opportunity to target the GNU C compiler to RISC processors. Thus was born gcc2, a gcc off-shoot that was still made freely available under the terms of the GNU General Public License (
http://www.fsf.org/copyleft/gpl.html
) but whose development was still overseen by Cygnus more than by the FSF.
Cygnus currently sees a new opportunity to adapt gcc to the needs of the embedded-systems market. "Now gcc knows a lot about RISC pipelines," says Tiemann, "but it doesn't know boo about DSPs."
So, the company is hosting another offshoot project
, egcs (
http://www.cygnus.com/egcs/
). Again, Cygnus's staff will spearhead the effort; again, the project will be an open collaboration between Cygnus and others; again, the resulting code will be assigned back to the FSF. With its multiple T1 lines, compiler test suites, test machines, source-control system, and, of course, its own team of ace compiler hackers, Cygnus hopes to focus the largest possible pool of talent on the egcs project.
The Apache project runs differently than most freeware projects. Perl, for example, continues to evolve under the watchful eye of its creator, Larry Wall. Likewise, Linux does the same under Linus Torvalds. But Apache is governed by 13 co-equal developers who share permission to commit changes to the Apache source tree.
Several of these developers also run busin
esses that depend on Apache. Bri-an Behlendorf is CTO of Organic Online (
http://www.organic.com/
), a Web consultancy that builds sites for companies such as McDonald's and Colgate. Relying on Apache wherever possible, Organic leverages not only the core group's knowledge of the product but also that of another 10 to 15 active developers and of several hundred others who have at one time or another submitted patches.
Another member of the core group is Sameer Parekh, president of C2Net Software. His Apache-based product Stronghold (which has now merged with another SSL-capable Apache variant called Sioux [see "Digital IDs," March BYTE]) incorporates Eric Young's SSleay, itself a freeware product.
Young and his partner, Tim Hudson, now work for C2Net, where they've added support for SSL version 3 to SSle
ay. C2Net could have chosen to reserve rights to the SSL-V3-enhanced SSleay, but it has given it back to Young -- and thence to the world.
This kind of enlightened self-interest can, under the right circumstances, create a successful symbiosis of freeware and commercial interests. "We will keep our value-adds separate from the core," says Parekh. In Stronghold 2.1, these will include GUI administration, embedded scripting, a search engine, and hooks into Xcert Software's (
http://www.xcert.com/
) Sentry CA, a certificate-authority package.
Caldera marketing chief Ransom Love tells a similar tale. One of the company's aims has been to attract commercial Unix applications to the Unix platform, and these include Netscape's Navigator and FastTrack server, along with WordPerfect. Caldera is now working to integ
rate Java into Linux and to use Java to create a complete GUI-based administration tool for Linux. The Java integration requires the licensing of software from Sun -- something the Linux community can't do, but a company such as Caldera can.
Caldera plans to reserve rights to its Java-integrated Linux. But it will contribute its Java-based GUI administration tool to the freeware realm. It's a smart move. Unix has always desperately needed a universal configurator that could be standard across all Unix species and that could evolve rapidly. Enlightened self-interest again. "When Ray Noorda started Caldera," says Love, "the philosophy was to grow the pie." If this project captures the interest of the freeware community, it will help grow the Linux pie.
A Dose of Reality
So what's wrong with this picture? There are, to be sure, some blurry areas. Building quality software is hard work that can command a princely wage in the commercial world. Who rewards the producers of freeware? Answers inc
lude the following:
1 Themselves. For many developers, a freeware project is an all-consuming hobby that can literally be its own reward.
2
Their peers.
As Eric Raymond points out, peer recognition is a powerful incentive in the hacker culture -- often the most powerful.
3
Their users.
Freeware developers can enjoy a kind of direct and immediate interaction with their users, a social benefit that is often unavailable in commercial settings.
4
Their employers.
Freeware projects often spring from efforts to solve business problems. Perl, for example, was Larry Wall's solution to his employer's distributed-network-management problem. To the extent that companies perceive software thus created as mission-critical, freeware projects can earn official status.
Of course, it's not a freeware project unless the employer believes that the value of keeping the software free, measured in terms of contributed development, outweighs the cost of keeping the software free, measured in terms of the competitive advantage surrendered by doing so. Behlendorf says that he occasionally has to remind Organic's board of directors that an open Apache is worth more to the company than a proprietary one would be. Clearly, this is an extremely tricky matter that will be resolved by different companies in different ways for different reasons.
Apart from all the well-known success stories, does the freeware model reveal any deep principles of software engineering that can and should be applied more broadly? It's unclear. Even the strongest freeware advocates admit that only certain kinds of projects inspire the necessary passion. There's no shortage of would-be compiler and OS-kernel hackers, but few are clamoring to write installation and configuration tools, word processors, accounting packages, or documentation.
Rob Kolstad, president of Berkeley Software Des
ign, isn't convinced that there's more to free software than meets the eye. His company's product, BSD/OS, derives from the free Berkeley Unix, but BSD/OS isn't freeware. Kolstad and his team of 19 engineers have crafted a tightly integrated system for ISPs and corporate customers. "If you buy Linux from Caldera and the rest," says Kolstad, "they're not in there fixing drivers and protocols; they just hold their cups over the dam and wait." BSDI, he adds, commits to service, support, and integration that don't depend on a volatile freeware community.
What about Eric Raymond's assertion that all bugs are shallow given enough eyeballs? Kolstad counters that it's not the number of eyeballs that matters; it's the quality of the brains behind them. That said, Kolstad doesn't deny the superior quality and utility of many freeware packages. He thinks every computer-science student should be required to build some useful package, acquire hundreds of users, support it, refine it, and then, finally, "as a parting
gift to the world, release it."
Recipe for a Good Freeware Project
Every
freeware project
is different, but successful ones are often characterized by the following three qualities:
1
A personable leader.
It's no accident that Larry Wall and Linus Torvalds are, by all accounts, immensely likeable. You can't make freeware developers do the necessary hard work; you can only inspire them to want to. That takes people skills of the same high order as the requisite technical skills.
2
A modular design.
The core of any complex software system won't necessarily succumb to the "many eyeballs" approach. But a strong plug-in interface can enable many contributors to play in the game. Linux, Perl, and Apache all exemplify this core-and-plug-in architecture. Consequently, lively development communities are able to contribute a rich and growing assortment of Linux device drivers and Perl and Apache modul
es.
3
Broad appeal.
The software should solve a general problem that lots of people care about.
Everyone owes a huge debt of gratitude to the many freeware packages without which there would never have been an Internet -- and to Richard Stallman, without whom there might never have been a freeware culture. Are there more projects like Apache, Perl, INN, and Linux ready to take wing? I hope so.
To test the waters, I've released ByteCal (see "Persistent Java," August BYTE) as freeware, along with documentation that invites users to extend it in several ways. We'll see what happens.
TOOLWATCH
DAF 2.0.....................$95 for one server
Philippe Tenenhaus
Internet: http://daf.simplenet.com/
DAF, or Dynamic Authentication Filter, is an ISAPI filter that maps thousands of Web users onto one or several NT user names. If you're running a sizable protected site using IIS, you need a tool like this. Ultimately all IIS security refers to NT security, but it would be crazy to create an NT account for each of thousands of users.
BOOKNOTE
Web Security Sourcebook.................$29.99
by Aviel D. Rubin, Daniel Geer, and Marcus J. Ranum
Wiley Computer Publishing
ISBN 0-471-18148-X
Internet: http://www.wiley.com/compbooks/
Internet: http://www.clark.net/pub/mjr/book
Practical recommendations from a trio of battle-scarred veterans.
Acme.Serve (http://www.acme.com/java/
)
A Java-based Web server; JefPoskanzer's Acme.Serve hosts our servlets, including ByteCal and Polls.
Apache (http://www.apache.org/)
A Web server; serves the Web view of BYTE's conferencing system.
EMACS (http://www.fsf.org/)
A text editor; without Richard Stallman's venerable EMACS, Unix would be useless to me.
Excite (http://www.excite.com/navigate/)
A search engine; Excite's unique query-by-example enables open-ended searches.
Hypermail (http://www.eit.com/hypermail/)
A mail-to-Web converter. Kevin Hughes contributed this invaluable tool to the Net; we use it for several internal applications.
INN (http://www.isc.org/)
An NNTP server; Rich Salz created the engine that powers both the Web and news halves of our conferencing system.
Linux (http://www.linux.org/)
An OS; actually, we use Caldera's Linux, which isn't freeware, but it would seem unfair not to mention Linux in this context.
MHonArc (http://www.oac.uci.edu/indiv/ehood/mhonarc.html)
A mail-to-Web converter. Earl Hood wrote this Perl suite to transform RFC822 messages into Web pages. It supports the Web view of our NNTP discussions.
Perl (http://www.perl.com/)
A programming language. Larry Wall's brainchild is the glue that binds together most everything on our site: search, forms processing, log analysis, and much more.
SWISH (http://www.eit.com/software/swish/)
A search engine; Another Kevin Hughes contribution, SWISH complements Excite's useful fuzziness with a more literal search capability.
Win32::Internet (http://www.perl.com/CPAN-local/)
A Perl Web-spidering module. Aldo Calpini's WinInet enabled my associate
Dave Rowell to write a really useful automated site monitor.
Win32::ODBC (http://www.perl.com/CPAN-local/)
A Perl database module. Dave Roth's ODBC module enables me to use Perl to manage and analyze a 1.6-million-user database.
illustration_link (9 Kbytes)
Apache owns a commanding share of the world's Web-server market. Source: Netcraft, http://www.netcraft.com/survey/.
photo_link (44 Kbytes)
Jon Udell is BYTE's executive editor for new media. You can reach him by sending e-mail to jon@byte.com.
The Value of Free Software
