on, capability, and testing. Don Libes wrote Expect at the National Institute of Standards and Technology (NIST) in 1987. Given the new programming paradigms offered by Expect, we can assume that it will evolve for many more years.
Tcl
Before I get into the details of Expect, I need to say a bit about Tcl, the language it is based on. Tcl entered the world of Expect in 1991 and has shaped its evolution ever since. It was released by John Ousterhout at California-Berkeley in 1989. Tcl's purpose is to be a
n embeddable interpreted language that other programs can use as an extension language. While many programs use Tcl as an embedded language, the most popular application of Tcl's extensibility has been to add new commands to its sample shell (i.e., interpreter) to enhance its scripting capability.
It is easy to add new commands to a Tcl interpreter because of the trivial interface required (similar to the main routine in a C program). Furthermore, new commands are normally added as a set of related functions that form a package. These packages usually add a major capability to Tcl, and Expect itself is a package of commands. Other packages are Tk (GUI), TclX (expanded Unix command set), dp (Internet/Unix sockets and remote procedure call [RPC]), and [incr Tcl] (object-oriented language extensions).
The developer is free to mix and match these extensions. Often, they load only on demand and provide the ideal mix of features required for a given application. With these extensions, it is easy -- comp
ared to compiled languages -- to produce powerful, object-oriented, graphical applications that interface with other programs across a network.
Expect Basics
Expect is a tty-oriented language. A tty (in Unix-speak) is an interface with the characteristics of a serial port attached to a character-only terminal. Even after 25 years -- and in the presence of GUIs -- most Unix programs are still character-oriented and have simple streaming stdin/stdout-type interfaces.
While programs that use this default I/O model are easy to interface with, programs that connect directly to the user's terminal device (tty) don't interface as well. Expect works around this problem by using a pseudoterminal (which is often known as a pty) when it communicates with other programs.
Expect uses a command called
spawn
to associate its equivalent of a file descriptor (known as a spawn ID) to a device or process. By default, Expect assumes that
spawn
's arguments represent command
s to create a new process. However, a flag tells
spawn
to treat the argument as a Tcl file descriptor (obtained via the Tcl
open
command).
This allows associating a spawn ID with a file or a serial device. The action of using
spawn
to create a new spawn ID sets the Tcl global variable
spawn_id
to the numeric value that represents that spawn ID. Expect commands that require a spawn ID use the value of the global
spawn_id
variable if one is not supplied as an argument.
This makes modest tasks trivial to write, because commands don't require an extra argument to specify a process. For complex tasks (e.g., talking to multiple programs at once), it is best to save the value of
spawn_id
to new variables so that you can use them later to specify a process.
The namesake of Expect is the
expect
command that supports pattern/action statements.
Expect
statements can range from simple one-liners to complex statements supporting mult
iple inputs. The
expect
command would not be useful without the
send
command, which provides the capability to send a message to a spawn ID.
Expect statements take the form:
expect [[-opts] pat1 body1] ... [-opts] patn [bodyn]
which simply means that an
expect
statement can support multiple match patterns with associated lines of script. The following example shows the relationship between
expect
and
send
:
expect { "How are you doing?" { send "Great!" } "What is today's date?" { send [date] } }
This script responds with "Great!" when "How are you doing?" is received and returns the current date (via the Tcl
date
command) when "What is today's date?" is received.
In the real world, things often go wrong. You want to detect when a source of input goes away for some reason, or you do not receive a required input in a certain period of time. Expect makes the special patterns
eof
and
timeout
(adjusted by the global timeout variable) available to test for these conditions. The programmer also has the special pattern
default
available, which covers both these cases.
If loss of input is not tested for, and the input goes away, Expect prints the Tcl equivalent of a stack dump. This is an unpleasant result, so it is wise to always test for
eof
at the beginning of each
expect
statement and do something intelligent when the input is lost.
The
expect
statement is most easily used when human interaction is not required. Expect supplies an
interact
command for use when the default activity is interactive, yet the script must have the capability of selectively intercepting and acting on either user or program input.
Interact
statements take the form:
interact [string1 body1] ... [stringn [bodyn]]
which is similar to the
expect
statement except that the patterns are matched by default as simple strings ra
ther than wild-card patterns. No arguments need be supplied. Another difference between the
interact
command and the
expect
command is that
interact
behaves as if it is in an infinite loop, while
expect
continues on to the next Tcl statement by default.
The simplest
interact
statement is a statement by itself. This ties the user's input to the process indicated by the
spawn_id
variable. For example,
spawn /bin/csh
interact ties the user to the shell such that it is not obvious that Expect is in the picture.
Also, there is no way to exit the process via Expect. A simple enhancement lets the user abort the process on demand:
spawn /bin/csh interact { "\033" { close; wait; exit } }
This script allows direct user interaction with the shell until you hit the Escape key. Then, the spawn ID associated with the shell is closed, causing it to exit. The
wait
command is executed to intercept the death of child and avo
id a zombie process. Finally, you use the
exit
command to exit the script.
A popular use of Expect is for testing other software. Expect can supply specific outputs, given certain inputs. The figure
"Expect Test-Bed"
shows a setup that lets a user on a Unix workstation test an embedded computer. The embedded computer has an Ethernet interface, plus a serial interface for debugging. The Expect script spawns a special test program that applies complex inputs to the embedded computer, while the serial link verifies the resulting actions. The user can select from a suite of tests written in Expect to execute.
Expect is a complicated language that will take time for you to master. Expect's 108-KB manual page organizes commands in alphabetical order, with each command description referring to other commands. Tcl, on the other hand, is a much simpler language and can be described in a more linear fashion. As a result, you will likely master Tcl before you master Expect.
Futures
Both Expect and Tcl have continued to evolve. John Ousterhout is working at Sun Microsystems, and there are now ports of Tcl/Tk to Win32 and the Mac. Don Libes works at NIST and continues to enhance Expect's portability and capabilities. There is reputed to be an effort under way (outside NIST) to port Expect to Win32. Once this port becomes available, it will open up new avenues for Windows programmers.
Expect Resources
To learn more about Expect and Tcl, you should read the books
Exploring Expect: A Tcl-Based Toolkit for Automating Interactive Programs
by Don Libes (O'Reilly & Associates, ISBN 1-56592-090-2) and
TCL and the Tk Toolkit
by John Ousterhout (Addison-Wesley, ISBN 0-201-63337-X). You can also consult the Expect home page at
http://expect.nist.gov/
and the Tcl/Tk Info page at
http://www.tcltk.com/
. You can download Expect from its home page (
http://expect.nist.gov/expect.tar.Z
). Expect requires Tcl, which you can download from
http://expect.nist.gov/tcl.tar.Z
.
Expect Offers Unix Scripting
