diff --git a/ChangeLog b/ChangeLog index 2f39685..b1f53b8 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,8 @@ +2006-06-06 Ben Elliston + + * doc/dejagnu.xml, doc/ref.xml, doc/user.xml: Edits. + * doc/dejagnu.texi: Regenerate. + 2006-06-06 Ben Elliston Import from GCC tree: diff --git a/doc/dejagnu.texi b/doc/dejagnu.texi index f802f96..8ac307a 100644 --- a/doc/dejagnu.texi +++ b/doc/dejagnu.texi @@ -459,9 +459,9 @@ dejagnu with a AMD K6 and a Mac Powerbook G3 serving as a remote target. -The tests for Windows were run under Windows NT using the -actual Cygwin version (1.3.x as of October 2001). It's target system -was a PPC embedded system running vxWorks. +The tests for Windows were run under Windows using the actual +Cygwin version (1.3.x as of October 2001). Its target system was a PPC +embedded system running vxWorks. @menu * Test your installation:: @@ -524,7 +524,7 @@ them. On Windows systems DejaGnu is part of a port of a lot of Unix tools to the Windows OS, called Cygwin. Cygwin may be downloaded and installed from a mirror of http://www.cygwin.com/. All examples were -also run on Windows NT. If nothing is said, you can assume that you +also run on Windows. If nothing is said, you can assume that you should get the same output as on a Unix system. You will need a telnet daemon if you want to use a Windows box @@ -2453,7 +2453,7 @@ host. @subsection Personal Config File The personal config file is used to customize -@code{runtest's} behaviour for each person. It's +@code{runtest's} behaviour for each person. It is typically used to set the user prefered setting for verbosity, and any experimental Tcl procedures. My personal @file{~/.dejagnurc} file looks like: diff --git a/doc/dejagnu.xml b/doc/dejagnu.xml index 6a0653a..7dd5527 100644 --- a/doc/dejagnu.xml +++ b/doc/dejagnu.xml @@ -19,60 +19,50 @@ &dj; The GNU Testing Framework - 2004 Feb 04 + January 2006 - Rob Savoye - - Free Software Foundation - + Rob + Savoye + + + Ben + Elliston -
- rob@welcomehome.org -
- + - 2004 - Rob Savoye + 2006 + Free Software Foundation, Inc. - 0.6.2 - 2002-7-16 - rob@welcomehome.org + 2002-07-16 + rob Add new tutorial as a new sect1. 0.6.1 - 2001-2-16 - rob@welcomehome.org + 2001-02-16 + rob Add info on the new dejagnu.h file. 0.6 - 2001-2-16 - rob@welcomehome.org + 2001-02-16 + rob Updated for new release. 0.5 - 2000-1-24 - rob@welcomehome.org + 2000-01-24 + rob Initial version after conversion to DocBook. @@ -82,8 +72,8 @@ Abstract - This document describes the functionality of DejaGnu, the - testing framework of the GNU project. DejaGnu is written in + This document describes the functionality of &dj;, the + testing framework of the GNU project. &dj; is written in Expect, which uses Tcl as a command language. Expect acts as a very @@ -95,7 +85,7 @@ diff or sh, with full control over its input and output. - DejaGnu itself is merely a framework for the creation of + &dj; itself is merely a framework for the creation of testsuites. Testsuites are distributed with each application. @@ -106,14 +96,14 @@ What is &dj; ? - DejaGnu is a framework for + &dj; is a framework for testing other programs. Its purpose is to provide a single front end for all tests. Think of it as a custom library of Tcl procedures crafted to support writing a test harness. A Test Harness is the testing infrastructure that is created to support a specific program or tool. Each program can have multiple testsuites, all - supported by a single test harness. DejaGnu is written in + supported by a single test harness. &dj; is written in Expect, which in turn uses Tcl -- Tool command language. There is more information on Tcl at the NIST. - Julia Menapace first coined the term ``DejaGnu'' to describe + Julia Menapace first coined the term ``&dj;'' to describe an earlier testing framework at Cygnus Support she had written for GDB. When we replaced it with the - Expect-based framework, it was like DejaGnu all over again. + Expect-based framework, it was like &dj; all over again. More importantly, it was also named after my daughter, Deja Snow Savoye (now 14 years old as of Feb 2004), who was a toddler - during DejaGnu's beginnings. + during &dj;'s beginnings. - DejaGnu offers several advantages for testing: + &dj; offers several advantages for testing: - The flexibility and consistency of the DejaGnu + The flexibility and consistency of the &dj; framework make it easy to write tests for any program, with either batch oriented, or interactive programs. - DejaGnu provides a layer of abstraction which + &dj; provides a layer of abstraction which allows you to write tests that are portable to any host or target where a program must be tested. For instance, a test for GDB can run from any supported host - system on any supported target system. DejaGnu runs tests on + system on any supported target system. &dj; runs tests on many single board computers, whose operating software ranges from a simple boot monitor to a real-time OS. All tests have the same output format. This makes it easy to integrate testing into other software - development processes. DejaGnu's output is designed to be + development processes. &dj;'s output is designed to be parsed by other filtering script and it is also human readable. Using Tcl and Expect, it's easy to create wrappers for existing testsuites. By incorporating existing tests under - DejaGnu, it's easier to have a single set of report analyse + &dj;, it's easier to have a single set of report analyse programs.. @@ -169,7 +159,7 @@ a Tcl script to run a testsuite that is not based on Expect. Expect script filenames conventionally use .exp as a - suffix; for example, the main implementation of the DejaGnu test + suffix; for example, the main implementation of the &dj; test driver is in the file runtest.exp.) @@ -201,7 +191,7 @@ Lots of little bug fixes from years of heavy use at Cygnus Solutions. - DejaGnu now uses + &dj; now uses Automake for Makefile configuration. @@ -218,7 +208,7 @@ Windows Support - To use DejaGnu on Windows, you need to first install the + To use &dj; on Windows, you need to first install the Cygwin release. This works as of the B20.1 release. Cygwin is a POSIX system for Windows. This covers both utility programs and a library @@ -238,7 +228,7 @@ Design Goals - DejaGnu grew out of the internal needs of Cygnus Solutions, + &dj; grew out of the internal needs of Cygnus Solutions, the company formerly known as Cygnus Support. Cygnus maintained and enhanced a variety of free programs in many different environments and we needed a testing tool that: @@ -269,10 +259,10 @@ environments are customized by each developer. Even when buying packaged boards from vendors there are many differences. The communication interfaces vary from a serial line to Ethernet. - DejaGnu was designed with a modular communication setup, so that + &dj; was designed with a modular communication setup, so that each kind of communication can be added as required and supported thereafter. Once a communication procedure is coded, any test can - use it. Currently DejaGnu can use rsh, + use it. Currently &dj; can use rsh, rlogin, telnet, tip, kermit and mondfe for remote communications. @@ -282,7 +272,7 @@ A POSIX conforming test framework - DejaGnu conforms to the POSIX 1003.3 standard for test + &dj; conforms to the POSIX 1003.3 standard for test frameworks. Rob Savoye was a member of that committee. The POSIX standard 1003.3 defines what a testing framework needs to @@ -307,7 +297,7 @@ repeatedly reading the standard and experimenting. One of the main things 1003.3 does specify is the set of allowed output messages and their definitions. Four messages are supported for a required feature of - POSIX conforming systems and a fifth for a conditional feature. DejaGnu + POSIX conforming systems and a fifth for a conditional feature. &dj; supports the use of all five output messages. In this sense a testsuite that uses exactly these messages can be considered POSIX conforming. These definitions specify the output of a test @@ -373,7 +363,7 @@ A test does not produce a clear result. This is usually because there was an - ERROR from DejaGnu while processing + ERROR from &dj; while processing the test, or because there were three or more WARNING messages. Any WARNING or ERROR @@ -406,7 +396,7 @@ UNSUPPORTED There is no support for the tested case. This may mean that a conditional feature of an operating system, or of a - compiler, is not implemented. DejaGnu also uses this message when + compiler, is not implemented. &dj; also uses this message when a testing environment (often a ``bare board'' target) lacks basic support for compiling or running the test case. For example, a test for the system subroutine gethostname @@ -415,9 +405,9 @@ - DejaGnu uses the same output procedures to produce these messages + &dj; uses the same output procedures to produce these messages for all testsuites and these procedures are already known to conform - to POSIX 1003.3. For a DejaGnu testsuite to conform to POSIX 1003.3, + to POSIX 1003.3. For a &dj; testsuite to conform to POSIX 1003.3, you must avoid the setupxfail} procedure as described in the PASS section above and you must be careful to return UNRESOLVED where appropriate, diff --git a/doc/ref.xml b/doc/ref.xml index fd8239e..988fe35 100644 --- a/doc/ref.xml +++ b/doc/ref.xml @@ -3,9 +3,9 @@ Reference - Obtaining DejaGnu + Obtaining &dj; - You can obtain DejaGnu from the DejaGnu web site at the + You can obtain &dj; from the &dj; web site at the Free Software Foundation, which is at www.gnu.org/software/dejagnu/ @@ -16,17 +16,17 @@ Installation - Once you have the DejaGnu source unpacked and available, you must + Once you have the &dj; source unpacked and available, you must first configure the software to specify where it is to run (and the associated defaults); then you can proceed to installing it. - Configuring DejaGnu + Configuring &dj; It is usually best to configure in a directory separate from the source tree, specifying where to find the source with the optional --srcdir option to - configure. DejaGnu uses the GNU + configure. &dj; uses the GNU autoconf to configure itself. For more info on using autoconf, read the GNU autoconf manual. To configure, execute the configure program, no other options are @@ -37,12 +37,12 @@ ../dejagnu-&version;/configure - DejaGnu doesn't care at config time if it's for testing a native + &dj; doesn't care at config time if it's for testing a native system or a cross system. That is determined at runtime by using the config files. You may also want to use the configure option - --prefix to specify where you want DejaGnu and its + --prefix to specify where you want &dj; and its supporting code installed. By default, installation is in subdirectories of /usr/local, but you can select any alternate directory altdir by including @@ -51,10 +51,10 @@ the Makefile variables prefix and execprefix}.) - Save for a small number of example tests, the DejaGnu distribution + Save for a small number of example tests, the &dj; distribution itself does not include any testsuites; these are available separately. Testsuites for the GNU development tools are included in - those releases. After configuring the top-level DejaGnu directory, unpack + those releases. After configuring the top-level &dj; directory, unpack and configure the test directories for the tools you want to test; then, in each test directory, run make check to build auxiliary programs required by some of the tests, and run the test @@ -63,9 +63,9 @@ - Installing DejaGnu + Installing &dj; - To install DejaGnu in your filesystem (either in + To install &dj; in your filesystem (either in /usr/local, or as specified by your --prefix option to configure), execute. @@ -75,7 +75,7 @@ make installdoes thes things for - DejaGnu: + &dj;: Look in the path specified for executables @@ -99,7 +99,7 @@ Copy runtest.exp into $exec_prefix/lib/dejagnu. This is the main Tcl - code implementing DejaGnu. + code implementing &dj;. @@ -108,7 +108,7 @@ Builtin Procedures - DejaGnu provides these Tcl procedures. + &dj; provides these Tcl procedures. Core Internal Procedures @@ -1012,10 +1012,10 @@ Load_lib Procedure - Loads a DejaGnu library file by searching a fixed path built - into DejaGnu. If DejaGnu has been installed, it looks in a path + Loads a &dj; library file by searching a fixed path built + into &dj;. If &dj; has been installed, it looks in a path starting with the installed library directory. If you are running - DejaGnu directly from a source directory, without first running + &dj; directly from a source directory, without first running make install, this path defaults to the current directory. In either case, it then looks in the current directory for a directory called lib. If there are @@ -1031,7 +1031,7 @@ filespec - The name of the DejaGnu library file to + The name of the &dj; library file to load. @@ -3306,7 +3306,7 @@ This invokes the compiler as set by CC to compile the file file. The default options for many cross - compilation targets are guessed by DejaGnu, and + compilation targets are guessed by &dj;, and these options can be added to by passing in more parameters as arguments to compile. Optionally, this will also use the value of the cflags field in the target @@ -3382,7 +3382,7 @@ info to execute this command on the build machine or a remote host. All config information for the remote host must be setup to have this command work. If this is a canadian cross, (where we test a - cross compiler that runs on a different host then where DejaGnu is + cross compiler that runs on a different host then where &dj; is running) then a connection is made to the remote host and the command is executed there. It returns either REMOTERROR (for an error) or the output produced when the command was executed. This is used for @@ -3447,7 +3447,7 @@ An example of this second kind of start function is gcc_start, the start function for GCC. - DejaGnu itself does not call + &dj; itself does not call ${tool}_start. The initialization module ${tool}_init.exp must call ${tool}_start for interactive tools; @@ -3478,7 +3478,7 @@ program it runs in a variable called $exec_output. Writing ${tool}_load can be the most complex - part of extending DejaGnu to a new tool or a new target, if + part of extending &dj; to a new tool or a new target, if it requires much communication coding or file downloading. Test scripts call ${tool}_load. @@ -3494,7 +3494,7 @@ ${tool}_exit Procedure - Cleans up (if necessary) before DejaGnu exits. For + Cleans up (if necessary) before &dj; exits. For interactive tools, this usually ends the interactive session. You can also use ${tool}_exit to remove any temporary files left over from the @@ -3513,7 +3513,7 @@ ${tool}_version Procedure Prints the version label and number for - ${tool}. This is called by the DejaGnu + ${tool}. This is called by the &dj; procedure that prints the final summary report. The output should consist of the full path name used for the tested tool, and its version number. @@ -3666,7 +3666,7 @@ Prune Procedure This procedure is deprecated and will be removed in - the next release of DejaGnu. If a testsuite uses this + the next release of &dj;. If a testsuite uses this procedure, a copy of the procedure should be made and placed in the lib directory of the testsuite. @@ -3674,7 +3674,7 @@ Slay Procedure This procedure is deprecated and will be removed in - the next release of DejaGnu. If a testsuite uses this + the next release of &dj;. If a testsuite uses this procedure, a copy of the procedure should be made and placed in the lib directory of the testsuite. @@ -3682,7 +3682,7 @@ Absolute Procedure This procedure is deprecated and will be removed in - the next release of DejaGnu. If a testsuite uses this + the next release of &dj;. If a testsuite uses this procedure, a copy of the procedure should be made and placed in the lib directory of the testsuite. @@ -3690,7 +3690,7 @@ Psource Procedure This procedure is deprecated and will be removed in - the next release of DejaGnu. If a testsuite uses this + the next release of &dj;. If a testsuite uses this procedure, a copy of the procedure should be made and placed in the lib directory of the testsuite. @@ -4535,7 +4535,7 @@ File Map - This is a map of the files in DejaGnu. + This is a map of the files in &dj;. runtest diff --git a/doc/user.xml b/doc/user.xml index f4fa81d..d0f0838 100644 --- a/doc/user.xml +++ b/doc/user.xml @@ -1,32 +1,31 @@ - Getting DejaGnu up and running + Getting &dj; up and running This chapter was originally written by Niklaus Giger - (ngiger@mus.ch) because he lost a week to figure out how DejaGnu works + (ngiger@mus.ch) because he lost a week to figure out how &dj; works and how to write a first test. Follow these instructions as closely a possible in order get a -good insight into how DejaGnu works, else you might run into a lot of +good insight into how &dj; works, else you might run into a lot of subtle problems. You have been warned. -It should be no big problems installing DejaGnu using your +It should be no big problems installing &dj; using your package manager or from the source code. Under a Debian/GNU/Linux systems just type (as root) apt-get dejagnu. These examples were run on a primary machine with a AMD K6 and a Mac Powerbook G3 serving as a remote target. - The tests for Windows were run under Windows NT using the -actual Cygwin version (1.3.x as of October 2001). It's target system -was a PPC embedded system running vxWorks. - + The tests for Windows were run under Windows using the actual +Cygwin version (1.3.x as of October 2001). Its target system was a PPC +embedded system running vxWorks. Test your installation -Create a new user called "dgt" (DejaGnuTest), which uses bash as +Create a new user called "dgt" (&dj;Test), which uses bash as it login shell. PS1 must be set to '\u:\w\$ ' in its ~/.bashrc. Login as this user, create an empty directory and change the working directory to it. e.g @@ -36,7 +35,7 @@ dgt:~$ mkdir ~/dejagnu.test dgt:~$ cd ~/dejagnu.test -Now you are ready to test DejaGnu's main program called +Now you are ready to test &dj;'s main program called runtest. The expecteted output is shown @@ -67,10 +66,10 @@ them. Windows -On Windows systems DejaGnu is part of a port of a lot of Unix +On Windows systems &dj; is part of a port of a lot of Unix tools to the Windows OS, called Cygwin. Cygwin may be downloaded and installed from a mirror of http://www.cygwin.com/. All examples were -also run on Windows NT. If nothing is said, you can assume that you +also run on Windows. If nothing is said, you can assume that you should get the same output as on a Unix system. You will need a telnet daemon if you want to use a Windows box @@ -84,7 +83,7 @@ http://www.fictional.net/. If you are running a Debian distribution you can find the examples under /usr/share/doc/dejagnu/examples. These examples seem to be missing in Red Hat's RPM. In this case download the sources of -DejaGnu and adjust the pathes to the DejaGnu examples +&dj; and adjust the pathes to the &dj; examples accordingly. @@ -93,7 +92,7 @@ accordingly. Create a minimal project, e.g. calc In this section you will to start a small project, -using the sample application calc, which is part of your DejaGnu +using the sample application calc, which is part of your &dj; distribution A simple project without the GNU autotools @@ -119,7 +118,7 @@ automake. There is book "GNU autoconf, automake and libtool" by Garry V. Vaughan, et al. NewRider, ISBN 1-57870-190-2 which describes this process thoroughly. -From the calc example distributed with the DejaGnu documentation +From the calc example distributed with the &dj; documentation you should copy the program file itself (calc.c) and some additional files, which you might examine a little bit close to derive their meanings. @@ -152,7 +151,7 @@ Run it to generate calc.h.in. dgt:~/dejagnu.test$ autoheader -The Makefile.am of this example was developed as port of the DejaGnu +The Makefile.am of this example was developed as port of the &dj; distribution. Adapt Makefile.am for this test. Replace the line "#noinst_PROGRAMS = calc" to @@ -271,7 +270,7 @@ calc: quit Look at the intentional bug that 2 times 4 equals 12. -The tests run by DejaGnu need a file called site.exp, +The tests run by &dj; need a file called site.exp, which is automatically generated if we call "make site.exp". This was the purpose of the "AUTOMAKE_OPTIONS = dejagnu" in Makefile.am. @@ -350,7 +349,7 @@ long. The various config files or how to avoid warnings -DejaGnu may be customized by each user. It first searches for a +&dj; may be customized by each user. It first searches for a file called ~/.dejagnurc. Create the file ~/.dejagnurc and insert the following line: @@ -527,7 +526,7 @@ Connection closed by foreign host. A test case for login via telnet In order to define a correct setup we have add a line containing "set target unix" either to ~/.dejagnurc or to ~/my_dejagnu.exp. -In ~/boards/standard.exp add the following four lines to define a few patterns for the DejaGnu telnet login procedure. +In ~/boards/standard.exp add the following four lines to define a few patterns for the &dj; telnet login procedure. Defining a remote target board @@ -540,7 +539,7 @@ set_board_info hostname "localhost" -As DejaGnu will be parsing the telnet session output for some well +As &dj; will be parsing the telnet session output for some well known pattern the output there are a lot of things that can go wrong. If you have any problems verify your setup: @@ -563,7 +562,7 @@ Create the file ~/dejagnu.test/testsuite/calc.test/remote_echo.exp and add the following few lines: -DejaGnu script for logging in into a remote target +&dj; script for logging in into a remote target puts "this is remote_echo.exp target for $target " target_info $target @@ -589,7 +588,7 @@ this is remote_echo.exp target is unix Spawn id for remote shell is exp7 -Have again a look at calc.log to get a feeling how DejaGnu and expect +Have again a look at calc.log to get a feeling how &dj; and expect parse the input. @@ -755,7 +754,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o In order to test the vxWorks as a target I changed boards/standards.exp to reflect my settings (IP, username, password). Then I reconfigured vxWorks to include a FTP and telnet server (using the same username/password combination ad in boards/standard.exp). -With this setup and some minor modification (e.g. replacing echo by printf) in my test cases I could test my vxWorks system. It sure does not seem to be a correct setup by DejaGnu standard. For instance, it still loading /usr/share/dejagnu/baseboards/unix.exp instead of vxWorks. In any case I found that (at least under Windows) I did not find out how the command line would let me override settings in my personal config files. +With this setup and some minor modification (e.g. replacing echo by printf) in my test cases I could test my vxWorks system. It sure does not seem to be a correct setup by &dj; standard. For instance, it still loading /usr/share/dejagnu/baseboards/unix.exp instead of vxWorks. In any case I found that (at least under Windows) I did not find out how the command line would let me override settings in my personal config files. @@ -788,12 +787,12 @@ powerpc-linux-gcc -g -O2 -o calc calc.o auxiliary programs or other files needed by the tests. The most common file the check builds is the site.exp. The site.exp file contains - various variables that DejaGnu used to dertermine the + various variables that &dj; used to dertermine the configuration of the program being tested. This is mostly for supporting remote testing. The check target is supported by GNU - Automake. To have DejaGnu support added to your + Automake. To have &dj; support added to your generated Makefile.in, just add the keyword dejagnu to the AUTOMAKE_OPTIONS variable in your Makefile.am file. @@ -812,7 +811,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o Runtest runtest is the executable test driver - for DejaGnu. You can specify two kinds of things on the + for &dj;. You can specify two kinds of things on the runtest command line: command line options, and Tcl variables for the test scripts. The options are listed alphabetically below. @@ -945,7 +944,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o string is a full configuration ``triple'' name as used by configure. This - is the type of machine DejaGnu and the tools to be tested are built + is the type of machine &dj; and the tools to be tested are built on. For a normal cross this is the same as the host, but for a canadian cross, they are seperate. @@ -957,7 +956,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o option to override the default string recorded by your configuration's choice of host. This choice does not change how anything is actually configured unless --build is also specified; it - affects only DejaGnu procedures that compare the + affects only &dj; procedures that compare the host string with particular values. The procedures ishost, istarget, isnative, and setupxfail} @@ -966,7 +965,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o be run on, which may not be the same as the build machine. If --build is also specified, then --host refers to the - machine that the tests wil, be run on, not the machine DejaGnu is run + machine that the tests wil, be run on, not the machine &dj; is run on. @@ -1031,7 +1030,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o runtest. This option affects only the summary and the detailed log files tool.sum and - tool.log. The DejaGnu debug + tool.log. The &dj; debug log dbg.log always appears (when requested) in the local directory. @@ -1078,7 +1077,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o running runtest. For example, use --connect to change the program used to connect to a ``bare board'' boot monitor. The choices for - type in the DejaGnu 1.4 distribution are + type in the &dj; 1.4 distribution are rlogin, telnet, rsh, tip, kermit, and mondfe. @@ -1150,7 +1149,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o (-V) - Prints out the version numbers of DejaGnu, + Prints out the version numbers of &dj;, expect and Tcl, and exits without running any tests. @@ -1164,8 +1163,8 @@ powerpc-linux-gcc -g -O2 -o calc calc.o expect as the file expect/tcl-debug.ps.. If you specify -D1, the expect shell stops - at a breakpoint as soon as DejaGnu invokes it. If you specify - -D0, DejaGnu starts as usual, but you can enter + at a breakpoint as soon as &dj; invokes it. If you specify + -D0, &dj; starts as usual, but you can enter the debugger by sending an interrupt (e.g. by typing Cc). @@ -1204,7 +1203,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o gdb.old. The default Tcl variables used for most tools are defined in - the main DejaGnu Makefile; their values are + the main &dj; Makefile; their values are captured in the site.exp file. @@ -1219,7 +1218,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o local site.exp file, created by "make site.exp". For example, if the directory gdb/testsuite - contains a collection of DejaGnu tests for GDB, you can run them like + contains a collection of &dj; tests for GDB, you can run them like this: @@ -1267,9 +1266,9 @@ powerpc-linux-gcc -g -O2 -o calc calc.o - The files DejaGnu produces. + The files &dj; produces. - DejaGnu always writes two kinds of output files: summary + &dj; always writes two kinds of output files: summary logs and detailed logs. The contents of both of these are determined by your tests. @@ -1281,7 +1280,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o Summary File - DejaGnu always produces a summary output file + &dj; always produces a summary output file tool.sum. This summary shows the names of all test files run; for each test file, one line of output from each pass command (showing status @@ -1302,7 +1301,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o For example, after runtest --tool binutils, look for a summary log in - binutils.sum. Normally, DejaGnu writes this + binutils.sum. Normally, &dj; writes this file in your current working directory; use the option to select a different directory. @@ -1336,11 +1335,11 @@ powerpc-linux-gcc -g -O2 -o calc calc.o Log File - DejaGnu also saves a detailed log file + &dj; also saves a detailed log file tool.log, showing any output generated by tests as well as the summary output. For example, after runtest --tool binutils, look for a detailed - log in binutils.log. Normally, DejaGnu + log in binutils.log. Normally, &dj; writes this file in your current working directory; use the option to select a different directory. @@ -1462,7 +1461,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o This example exhibits three properties of Expect and - DejaGnu that might be surprising at + &dj; that might be surprising at first glance: @@ -1510,19 +1509,19 @@ powerpc-linux-gcc -g -O2 -o calc calc.o - Customizing DejaGnu + Customizing &dj; The site configuration file, site.exp, captures configuration-dependent values and propagates them to the - DejaGnu test environment using Tcl variables. This ties the - DejaGnu test scripts into the configure and + &dj; test environment using Tcl variables. This ties the + &dj; test scripts into the configure and make programs. If this file is setup correctly, it is possible to execute a testsuite merely by typing runtest. - DejaGnu supports two site.exp + &dj; supports two site.exp files. The multiple instances of site.exp are - loaded in a fixed order built into DejaGnu. The first file loaded + loaded in a fixed order built into &dj;. The first file loaded is the local file site.exp, and then the optional global site.exp file as pointed to by the DEJAGNU environment @@ -1530,11 +1529,11 @@ powerpc-linux-gcc -g -O2 -o calc calc.o There is an optional master site.exp, capturing configuration values that - apply to DejaGnu across the board, in each configuration-specific - subdirectory of the DejaGnu library directory. + apply to &dj; across the board, in each configuration-specific + subdirectory of the &dj; library directory. runtest loads these values first. The master site.exp contains the default values for all - targets and hosts supported by DejaGnu. This master file is + targets and hosts supported by &dj;. This master file is identified by setting the environment variable DEJAGNU to the name of the file. This is also refered to as the ``global'' config file. @@ -1569,7 +1568,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o It is usually more convenient to keep these manual overrides in the site.exp local to each test directory, rather than in the global - site.exp in the installed DejaGnu + site.exp in the installed &dj; library. This file is mostly for supplying tool specific info that is required by the testsuite. @@ -1598,7 +1597,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o In the second section, you can override any default values - (locally to DejaGnu) for all the variables. The second section + (locally to &dj;) for all the variables. The second section can also contain your preferred defaults for all the command line options to runtest. This allows you to easily customize runtest for your preferences @@ -1654,7 +1653,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o file, namely the three config triplets, and the srcdir. It also defines several other Tcl variables that are used exclusivly by the GCC testsuite. For most test cases, the CXXFLAGS and LDFLAGS - are supplied by DejaGnu itself for cross testing, but to test a + are supplied by &dj; itself for cross testing, but to test a compiler, GCC needs to manipulate these itself. @@ -1713,7 +1712,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o In this case, we have support for several cross compilers, that all run on this host. For testing on operating systems that - don't support Expect, DejaGnu can be run on the local build + don't support Expect, &dj; can be run on the local build machine, and it can connect to the remote host and run all the tests for this cross compiler on that host. All the remote OS requires is a working telnetd. @@ -1816,7 +1815,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o Thanks to Dj Delorie for the original paper that this section is based on. - DejaGnu also supports running the tests on a remote + &dj; also supports running the tests on a remote host. To set this up, the remote host needs an ftp server, and a telnet server. Currently foreign operating systems used as remote hosts are VxWorks, VRTX, DOS/Windows 3.1, MacOS and Windows. @@ -1949,7 +1948,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o Config File Values - DejaGnu uses a named array in Tcl to hold all the info for + &dj; uses a named array in Tcl to hold all the info for each machine. In the case of a canadian cross, this means host information as well as target information. The named array is called target_info, and it has two indices. The @@ -2086,7 +2085,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o Personal Config File The personal config file is used to customize - runtest's behaviour for each person. It's + runtest's behaviour for each person. It is typically used to set the user prefered setting for verbosity, and any experimental Tcl procedures. My personal ~/.dejagnurc file looks like: @@ -2117,13 +2116,13 @@ powerpc-linux-gcc -g -O2 -o calc calc.o - Extending DejaGnu + Extending &dj; Adding A New Testsuite The testsuite for a new tool should always be located in that tools - source directory. DejaGnu require the directory be named + source directory. &dj; require the directory be named testsuite. Under this directory, the test cases go in a subdirectory whose name begins with the tool name. For example, for a tool named flubber, each subdirectory containing @@ -2138,7 +2137,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o is to read something similar. This principle applies to test cases and to testsuites. Unfortunately, well-established testsuites have a way of developing their own conventions: as test writers become more - experienced with DejaGnu and with Tcl, they accumulate more utilities, + experienced with &dj; and with Tcl, they accumulate more utilities, and take advantage of more and more features of Expect and Tcl in general. @@ -2149,8 +2148,8 @@ powerpc-linux-gcc -g -O2 -o calc calc.o There is one testsuite that is guaranteed not to grow more elaborate over time: both it and the tool it tests were created expressly - to illustrate what it takes to get started with DejaGnu. The - example/ directory of the DejaGnu distribution + to illustrate what it takes to get started with &dj;. The + example/ directory of the &dj; distribution contains both an interactive tool called calc, and a testsuite for it. Reading this testsuite, and experimenting with it, is a good way to supplement the information in this section. (Thanks to @@ -2168,7 +2167,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o Create a configure.in file in this directory, to control configuration-dependent choices for your tests. So far as - DejaGnu is concerned, the important thing is to set a value for the + &dj; is concerned, the important thing is to set a value for the variable target_abbrev; this value is the link to the init file you will write soon. (For simplicity, we assume the environment is Unix, and use unix as the @@ -2186,10 +2185,10 @@ powerpc-linux-gcc -g -O2 -o calc calc.o keyword dejagnu to the AUTOMAKE_OPTIONS variable in your Makefile.am file. This will add all the Makefile - support needed to run DejaGnu, and support the + support needed to run &dj;, and support the target. - You also need to include two targets important to DejaGnu: + You also need to include two targets important to &dj;: check, to run the tests, and site.exp, to set up the Tcl copies of configuration-dependent values. This is called the @@ -2206,7 +2205,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o Sample Makefile.in Fragment - # Look for a local version of DejaGnu, otherwise use one in the path + # Look for a local version of &dj;, otherwise use one in the path RUNTEST = `if test -f $(top_srcdir)/../dejagnu/runtest; then \ echo $(top_srcdir) ../dejagnu/runtest; \ else \ @@ -2374,13 +2373,13 @@ powerpc-linux-gcc -g -O2 -o calc calc.o Adding A New Target - DejaGnu has some additional requirements for target support, beyond - the general-purpose provisions of configure. DejaGnu must actively + &dj; has some additional requirements for target support, beyond + the general-purpose provisions of configure. &dj; must actively communicate with the target, rather than simply generating or managing code for the target architecture. Therefore, each tool requires an initialization module for each target. For new targets, you must supply - a few Tcl procedures to adapt DejaGnu to the target. This permits - DejaGnu itself to remain target independent. + a few Tcl procedures to adapt &dj; to the target. This permits + &dj; itself to remain target independent. Usually the best way to write a new initialization module is to edit an existing initialization module; some trial and error will be @@ -2393,7 +2392,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o For cross targets, most of the work is in getting the communications right. Communications code (for several situations - involving IP networks or serial lines) is available in a DejaGnu library + involving IP networks or serial lines) is available in a &dj; library file. If you suspect a communication problem, try running the connection @@ -2428,7 +2427,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o changing the linker script. Once the new baseboard file is done, add it to the boards_DATA list in the dejagnu/baseboards/Makefile.am, and regenerate the - Makefile.in using automake. Then just rebuild and install DejaGnu. You + Makefile.in using automake. Then just rebuild and install &dj;. You can test it by: There is a crude inheritance scheme going on with board files, so @@ -2881,10 +2880,10 @@ powerpc-linux-gcc -g -O2 -o calc calc.o However, some interactive programs can be tested in a simple fashion reminiscent of batch tests. For example, prior - to the creation of DejaGnu, the GDB distribution already + to the creation of &dj;, the GDB distribution already included a wide-ranging testing procedure. This procedure was very robust, and had already undergone much more debugging and - error checking than many recent DejaGnu test cases. + error checking than many recent &dj; test cases. Accordingly, the best approach was simply to encapsulate the existing GDB tests, for reporting purposes. Thereafter, new GDB tests built up a family of Tcl procedures specialized for GDB @@ -2896,14 +2895,14 @@ powerpc-linux-gcc -g -O2 -o calc calc.o Debugging A Test Case These are the kinds of debugging information available - from DejaGnu: + from &dj;: Output controlled by test scripts themselves, explicitly allowed for by the test author. This kind of debugging output appears in the detailed output recorded in the - DejaGnu log file. To do the same for new tests, use the + &dj; log file. To do the same for new tests, use the verbose procedure (which in turn uses the variable also called verbose) to control how much output to generate. This will make it easier for other @@ -2940,7 +2939,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o controls the depth of definitions expanded. Finally, if the value of - verbose is 3 or greater,DejaGnu turns on + verbose is 3 or greater,&dj; turns on the expect command log_user. This command prints all expect actions to the expect standard output, to the detailed log file, and (if is on) to @@ -3059,7 +3058,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o take an extreme case, tests that pass even when the tool will not spawn are misleading. Ideally, a test in this sort of situation should not fail either. Instead, print an error - message by calling one of the DejaGnu procedures + message by calling one of the &dj; procedures error or warning. @@ -3068,7 +3067,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o Special variables used by test cases. There are special variables used by test cases. These contain - other information from DejaGnu. Your test cases can use these variables, + other information from &dj;. Your test cases can use these variables, with conventional meanings (as well as the variables saved in site.exp. You can use the value of these variables, but they should never be changed. @@ -3129,7 +3128,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o What Is Unit Testing ? - Most regression testing as done by DejaGnu is system + Most regression testing as done by &dj; is system testing. This is the complete application is tested all at once. Unit testing is for testing single files, or small libraries. In this case, each file is linked with a test case in @@ -3148,15 +3147,15 @@ powerpc-linux-gcc -g -O2 -o calc calc.o The dejagnu.h Header File - DejaGnu uses a single header file to assist in unit + &dj; uses a single header file to assist in unit testing. As this file also produces it's one test state output, it can be run standalone, which is very useful for testing on embedded systems. This header file has a C and C++ API for the test states, with simple totals, and standardized - output. Because the output has been standardized, DejaGnu can be + output. Because the output has been standardized, &dj; can be made to work with this test case, without writing almost any Tcl. The library module, dejagnu.exp, will look for the output - messages, and then merge them into DejaGnu's. + messages, and then merge them into &dj;'s.