Pro, I conclude that the answer to both questions is probably yes.
These tools define a new genus in the family of Web programming systems. They build applications that can simply use essential services, such as SQL database access and session persistence. By contrast, CGI applications in general, and Perl programs in particular, must construct these support services themselves.
With each passing month, ever-fancier Web applications expect ever-richer support systems. The problem is not that you can't build these things in Perl -- you can, and I do -- but rather that using Perl this way takes you outside its domain of competence and erodes the spectacular productivity that attracted you to the language in the first place.
Perl likely won't dominate the next era of Internet/intranet programming, but it needn't become extinct, either. It will remain vital if its developers can answer three questions: What are Perl's strengths? Which software-development problems can Perl solve best? What in Perl needs fixing?
I've been thinking hard abo
ut these questions. In January, at a Perl summit conference sponsored by O'Reilly & Associates, I was given the rare opportunity to pitch my ideas to two of the leaders of the Perl community -- Larry Wall, inventor of the language, and Tom Christiansen, a renowned Perl educator.
Why me? I'm just a halfway-decent Perl programmer, but I've focused to an unusual degree on the combination of Perl, Windows NT, and Web development. That domain introduces requirements that are different from the ones that Larry faced when he created Perl as a better way to do Unix system administration.
Today, Larry works for O'Reilly, a company known as a Unix book publisher but more recently also as a Win32 software vendor (e.g., the WebSite Web server). As O'Reilly contemplates a Perl "supported distribution" (read: product), it's imperative to understand how best to adapt the language to the needs of Win32-based developers (see the sidebar "The State of Win32 Perl").
In short, Perl's at a crossroads -- its
Web hegemony imperiled, its Unix roots increasingly irrelevant to Win32 programmers, its Win32 extensions meaningless to Unix programmers. Given all this, here are my answers to the three big questions.
1: What are Perl's strengths?
Perl has two advantages -- dynamic data structures, such as lists and associative arrays, and powerful text processing based on a high-performance regular-expression engine. There's much more, of course, including rich networking functions, persistent data (mostly on Unix), and object orientation (in all versions of Perl 5).
However, these additional features are not what makes Perl special; they're also available in other languages, notably Java. Perl's text-processing and data-manipulation features are its real claim to fame. A classic Perl application scans a quantity of structured text, absorbs pattern-matched regions into automatically allocated data structures, rearranges those objects, and emits structured text that has been transformed in som
e crucial way.
In short, Perl shines when you need to write a filter. It owes its prowess in this realm to more than raw technical underpinnings. Larry is a linguist by training, and he sought to recapture for the learner of Perl the smooth progression from beginner to expert that we all experience when we learn natural languages. He hopes you'll get decent results right away, even with minimal competence, and for many users -- including me -- that's been true.
Larry also sought to recapture in Perl some of the syntactic flexibility of natural languages. A Chomksyan linguist says that there are many ways in which surface structures (i.e., sentences) can map to deep structures (i.e., meanings). The corresponding Perl mantra is: "There's more than one way to do it." You might parse a line of text using the split operator, while I might do it quite differently with a parameterized regular-expression search; either will get the job done.
Not all Perl programmers buy into these linguistic analogi
es. Some find aspects of the language baroque or just plain weird. But we all prize the rapid development that Perl's interpretive mode makes possible. We are, as Larry says, "desperate people in a hurry." We want results now, and Perl operating in its domain of competence delivers like no other tool can.
2: Which software-development problems can Perl solve best?
In a certain sense, the Internet is just a vast collection of structured ASCII texts -- Web pages, e-mail messages, Usenet postings, and configuration and log files. That is why Perl has enabled me to build lots of useful Internet applications quickly and easily. These apps, including my log-analysis scripts, our Web/NNTP conferencing system, and the Virtual Press Room, are all examples of the kind of filtering at which Perl excels.
Data reduction and analysis.
My Web-server log-analysis scripts are data reducers that boil down tens of megabytes of daily log data into reports that show how our audience
uses our on-line content.
Data conversion.
Our conferencing system is a data converter. It relies on a text-transformation script (Earl Hood's MHonArc) that converts RFC 822 messages that are stored in an INND database into a read-only Web archive, in addition to another script that converts that simple Web archive into a fancier frame-based one that supports posting.
Document collection and management.
The Virtual Press Room manages a database of user-contributed documents. It validates input, activates uniform resource locators (URLs), automatically constructs tabbed indexes for navigation, and performs full-text indexing.
Where's the database that manages all this information? In some cases, you'll find my Perl applications talking through Open Database Connectivity (ODBC) to SQL data stores. When the site's cumulative visitor count passed the quarter-million mark, for example, my 64-MB NT box could no longer handle that data using a Perl associative arr
ay backed by a text file, so I moved it into a SQL table. And when I began to analyze survey data, Perl's procedural approach gave way to SQL's declarative one. But in many cases you'll find my apps managing data using nothing more than Perl and the file system.
This might seem perverse, but in fact it's highly productive. A great many useful apps don't require the capacity, query capability, or transactional controls of a SQL engine. Use Perl to manage the data in these cases, and you can dramatically speed up the evolution of your data model.
Have you noticed how all the rapid application development (RAD) tools bog down in the data-definition phase? You might not need to declare variables, but you do have to define SQL tables, map your code to them, and keep the code and tables in sync. You also have to use SQL tools to populate the tables with test data and then examine them while debugging.
Contrast this with Perl's fluid style, in which you can straightforwardly externalize data to tex
t files. Can real working systems possibly rely on this technique? Well, the Internet does. Human-readable protocols and data stores make it possible to extend and debug the Internet, and they account for much of its resilience.
Larry says that when the University of California's Irvine campus was built, planners just sowed grass everywhere and let the paths that emerged define where to put the sidewalks. With Perl, you can apply this method to managing data: Start growing a bunch of applications, watch how people use them, and then improve the ones that matter.
3: What in Perl needs fixing?
In the good old days, circa 1994, the Unix tools approach -- scripts connected in a pipeline -- worked pretty well on the Web. That approach didn't scale along with the Web, though. The two complaints most often heard were, "Process-creation overhead is killing us; we need to incorporate Perl into the Web server," and "Perl's too slow; we need a Perl compiler."
Both of these complaints
have now been answered. ISAPI, FastCGI, and Apache-module incarnations of Perl are now available; all locate the Perl engine inside a Web server. Malcolm Beattie's Perl compiler is also now available (
ftp://ftp.ox.ac.uk/pub/perl/
); it converts Perl source into C source that you can compile and run.
But neither of these optimizations will drastically improve Perl's prospects relative to the new generation of Web-development tools. Why not? You can't optimize a flawed algorithm. The next-century equivalent of the Unix pipeline will be a distributed-object system consisting of COM- and/or CORBA-style objects. To thrive, Perl (or any scripting language) will have to be an effective user of objects, a producer of objects, or both.
Note that by
objects
I do not mean Perl 5 objects that you can export from Perl modules, inherit from in other Perl modules, and use in Perl programs. Perl's internal object syst
em is a wonderful thing, but it's private to Perl and does not connect it to public COM or CORBA object systems.
On Win32, Perl is already an effective controller of COM objects. It has long been able to manipulate the object systems of COM-aware applications, such as Microsoft Word, and thereby script their behavior from the outside. PerlScript adds the ability to do the same thing from the inside -- that is, it can become the embedded scripting language for applications that support ActiveX Scripting.
The most interesting such application will likely be the Windows 97 shell. To unify the desktop and the Internet, Microsoft plans to turn the shell into a dynamic Hypertext Markup Language (HTML) system. In theory, that means that you could do a View Source on the shell at any time and reveal the HTML description that produced what's on-screen. That description will mix formatted text, object invocations that interpolate values into formatted text, and inclusions of active content.
Would you
rather write a Visual Basic (VB) program or a PerlScript program to generate that description? I'll take PerlScript. It's vastly more competent in this realm than VB is.
Not all uses of PerlScript are so compelling. Many of the current Active Server Pages demos use JavaScript or VBScript interchangeably to control COM objects that, for example, talk to databases. It's nifty that PerlScript can now
substitute for
JavaScript and VBScript in this context. But if your application just calls an Active Data Object (ADO) connection in a loop and prints SQL rows to an HTML table, there's no particular reason to prefer PerlScript over the alternatives. In other words, many applications don't involve the kinds of sophisticated text processing and dynamic data manipulation that are the special province of Perl.
Thanks to ActiveX Scripting, Win32 is becoming an equal-opportunity environment for embedded script engines. When PerlScript is an appropriate solution, you'll be able to deploy
it.
Object Producer
Because Perl can't produce COM or CORBA objects, it unfortunately does not enjoy equal opportunity in the realm of object services. Consider two examples from the COM world: Visual Interdev and Internet Information Server (IIS)/Transaction Server. (CORBA fans, bear with me. This point would apply equally to CORBA, but concrete examples come less easily to mind.)
Visual Interdev, Microsoft's new integrated development environment (IDE) for Web developers, advances the notion of design-time controls. These are COM objects that you use to interactively define the behavior of chunks of active content. Specifically, a design-time control packages a GUI with a code generator that writes Active Server Pages-style HTML for IIS.
What would it take to retarget Visual Interdev to, say, LiveWire? You'd need a different code generator that could write LiveWire-style HTML. Perl to the rescue? Not unless someone figures out how to package Perl's text-wrangling service
s as a COM object that Visual Interdev can use.
Now consider IIS/Transaction Server. Let's say you've got a Web application that could benefit from Perl's dynamic data structures -- if those structures could be made persistent, and if the application could scale to accommodate lots of users. IIS 3.0 has a solution for persistence. Its built-in Application object can remember things across HTTP transactions. Of course, if you have to decompose your rich Perl data into the simple scalars and lists that an Application object can hold, you've lost the benefit of Perl.
Alternatively, you can hand the Application object a reference to another COM object. If Perl could inhabit COM objects, that might be a way to retain Perl's data-wrangling services while leveraging the persistence services of the Active Server Pages environment.
Your COM-based Perl code could also enjoy the benefits of Transaction Server. That way, you could let the environment into which you plug your Perl objects solve the hard
problems of multithreading, concurrency, locking, and transaction atomicity. Re-creating these services in Perl is not the way to go! Perl, and other script languages, ought rather to configure themselves so they can plug in where needed and run efficiently in those niches.
To make this work would require what Larry likes to call "deep magic." I suspect (and he agrees) that it would also require some sort of Perl run-time system. VB 5, which can generate ActiveX controls that rely on a VB run-time system, illustrates the concept.
Will Perl go this route? I hope so. It'll be a shame if component producers can't solve Perlish problems in Perl.
TOOLWATCH
PerlScript..............free
ActiveWare
Internet:
http://www.
activeware.com/
Perl's Future
