Note: This web site is only kept up to date for OSG Software 1.2 (VDT 2.0.0). If you are looking for information for the most recent release, the RPM-based OSG Software 3.0, please see the OSG documentation web site

VDT Nightly Test Framwork

Running the nightly tests

Running the nightly tests is very easy. Take a look at the --help for /vdt-install-tests/bin/vdt-run-install-tests:
vail(nmueller): vdt-run-install-tests --help
Usage: vdt-run-install-tests
        --package 
        --version 
        --download 
If you want to test VDT 1.2.0 and 1.2.0 you run vdt-run-install-tests -v 1.2.0 -v 1.2.1 -p VDT. I you want KX509 and Condor from 1.2.0 you can run vdt-run-install-tests -v 1.2.0 -p KX509 -p Condor

Alternativly you can use the --download flag to download a spec file. Lines in the spec file have the format "version, version...: package, package..." and do the same thing as the old -v and -p tests. In addition, however, the spec file is passed through the gcc preprocesser with the hostname (all caps, - converted to _) and the day of the week (all caps) defined. This allows you do make more complex policy. The current spec file used is http://vdt.cs.wisc.edu/test-scripts/tests-to-run.

Installing a New Test Box

  1. Follow the instructions to get host, LDAP, and HTTP certificates for the machine
  2. Edit /p/vdt/workspace/grid-security/make-tarballs to include the new platform.
  3. If needed, checkout a local copy of the nightly-tests.what CVS module:
    cvs -d /p/condor/workspaces/vdt/vdt_cvs co nightly-tests.what
  4. Change to your nightly-tests.what directory
  5. If you already had a nightly-tests.what checkout, make sure it’s up to date
  6. Edit root/vdt-install-test/bin/vdt-run-install-tests and add the new host to the %archs hash definition; e.g.:
    'vdt-rhas3-amd64' => ["RHEL 3", "x86-64"],
  7. Edit root/send-mail.sh and add the new host to the list of parameters:
    -a "Fedora Core 4,x86-64=1.3.11" -a "Fedora Core 4,32-on-x86-64=1.3.11"
  8. Commit your changes
  9. Follow the what instructions to set up what

How it all works

The nightly test stuff is made up of several components. The source for each one is in a different place (yea!) so editing stuff is a bit tricky. Hopefully this makes some sense.
vdt-run-install-tests
This is the top level script that makes everything look easy. Use it (with appropriate -p and -v options) to run VDT install tests. The master copy lives in /p/vdt/workspace/nightly-tests. If you add a new test question you need to edit this script to pre-answer it in the hash near the top.
vdt-pacman-install-test
This is where most of the work is done. It sets a bunch of environment variables, runs pre-install scripts, installs the VDT, runs post-install scripts, tests the VDT and runs post-test scripts. It's big, nasty and takes more command line options then I care to admit. Don't attempt to run it directly. vdt-pacman-install-test also lives in /p/vdt/workspace/nightly-tests.
Server setup and teardown scripts
To make installation easier, vdt-pacman-install-test fetches it's pre and post scripts from the vdt website. Scripts are located at /p/vdt/public/html/test-scripts.
VDT-Test
The pacman package VDT-Test installs the VDT test harness (VDT-TestHarness) and the tests themselves (VDT-Certification-Tests). It lives in the regular VDT cvs tree. The test harness (vdt-run-tests) is executed by the nightly test scripts and stores it's output in the test database. It can still be used in the old standalone way.
The test database
This is where all the results are stored. Currently it's running on vail.cs.wisc.edu (Nate's desktop) with the data stored in /scratch/nmueller/vdt-test-db. This is less then ideal and I'm working as hard as I can to move it to the lab's database server (dumbo.cs.wisc.edu).
The web pages
http://www.cs.wisc.edu/vdt/test-results. These use Mason to display real-time test results. The source lives in the web tree at /p/vdt/public/html/test-results. Check out http://www.masonbook.com before you get too deep.
The email reports
The email reports are sent by /p/vdt/workspace/nightly-tests/vdt-test-email every night from my crontab.