Running the nightly tests
Running the nightly tests is very easy. Take a look at the --help for /vdt-install-tests/bin/vdt-run-install-tests:
vail(nmueller): vdt-run-install-tests --help
If you want to test VDT 1.2.0 and 1.2.0 you run vdt-run-install-tests -v 1.2.0 -v 1.2.1 -p VDT. I you want KX509 and Condor from 1.2.0 you can run vdt-run-install-tests -v 1.2.0 -p KX509 -p Condor
Alternativly you can use the --download flag to download a spec file. Lines in the spec file have the format "version, version...: package, package..." and do the same thing as the old -v and -p tests. In addition, however, the spec file is passed through the gcc preprocesser with the hostname (all caps, - converted to _) and the day of the week (all caps) defined. This allows you do make more complex policy. The current spec file used is http://vdt.cs.wisc.edu/test-scripts/tests-to-run.
Installing a New Test Box
- Follow the instructions to get host, LDAP, and HTTP certificates for the machine
/p/vdt/workspace/grid-security/make-tarballs to include the new platform.
If needed, checkout a local copy of the
nightly-tests.what CVS module:
cvs -d /p/condor/workspaces/vdt/vdt_cvs co nightly-tests.what
- Change to your
- If you already had a
nightly-tests.what checkout, make sure it’s up to date
root/vdt-install-test/bin/vdt-run-install-tests and add the new host to the
%archs hash definition; e.g.:
'vdt-rhas3-amd64' => ["RHEL 3", "x86-64"],
root/send-mail.sh and add the new host to the list of parameters:
-a "Fedora Core 4,x86-64=1.3.11" -a "Fedora Core 4,32-on-x86-64=1.3.11"
- Commit your changes
- Follow the what instructions to set up what
How it all works
The nightly test stuff is made up of several components. The source for each one is in a different place (yea!) so editing stuff is a bit tricky. Hopefully this makes some sense.
- This is the top level script that makes everything look easy. Use it (with appropriate -p and -v options) to run VDT install tests. The master copy lives in /p/vdt/workspace/nightly-tests. If you add a new test question you need to edit this script to pre-answer it in the hash near the top.
- This is where most of the work is done. It sets a bunch of environment variables, runs pre-install scripts, installs the VDT, runs post-install scripts, tests the VDT and runs post-test scripts. It's big, nasty and takes more command line options then I care to admit. Don't attempt to run it directly. vdt-pacman-install-test also lives in /p/vdt/workspace/nightly-tests.
- Server setup and teardown scripts
- To make installation easier, vdt-pacman-install-test fetches it's pre and post scripts from the vdt website. Scripts are located at /p/vdt/public/html/test-scripts.
- The pacman package VDT-Test installs the VDT test harness (VDT-TestHarness) and the tests themselves (VDT-Certification-Tests). It lives in the regular VDT cvs tree. The test harness (vdt-run-tests) is executed by the nightly test scripts and stores it's output in the test database. It can still be used in the old standalone way.
- The test database
- This is where all the results are stored. Currently it's running on vail.cs.wisc.edu (Nate's desktop) with the data stored in /scratch/nmueller/vdt-test-db. This is less then ideal and I'm working as hard as I can to move it to the lab's database server (dumbo.cs.wisc.edu).
- The web pages
- http://www.cs.wisc.edu/vdt/test-results. These use Mason to display real-time test results. The source lives in the web tree at /p/vdt/public/html/test-results. Check out http://www.masonbook.com before you get too deep.
- The email reports
- The email reports are sent by /p/vdt/workspace/nightly-tests/vdt-test-email every night from my crontab.