Chapter 1. Installing a dCache System for the Open Science Grid

Ted Hesselroth

Fermi National Accelerator Laboratory

Timur Perelmutov

Fermi National Accelerator Laboratory

Neha Sharma

Fermi National Accelerator Laboratory

Michael Zalokar

Fermi National Accelerator Laboratory

E. Forrest Christian

Writer
Fermi National Accelerator Laboratory

Table of Contents

1. System Requirements
2. Designing Your dCache System
3. Getting the VDT dCache tar package
4. Configuring your dCache installation
5. Customizing Your dCache Installation Process
6. Dry Run Testing of install.sh
7. Installing dCache
8. Configuring firewalls for dCache
9. Starting dCache for the first time
10. Testing your dCache installation
11. Setting up your dCache system
12. Cleanup
13. Getting Installation Support
14. dCache Details
15. Gratia Probe
16. Samples

This document describes the process for installing dCache in an Open Science Grid site using the dCache scripts. For an overview of dCache, see dCache, the Overview” (PDF). For further details on installing dCache manually, see Installation in dCache, the Book.

Installing dCache can be non-trivial. We suggest following an installation workflow that has worked at other sites:

  1. Join the OSG community! Besides the OSG documentation wiki, mailing lists exist to tie you in with other grid storage administrators.

  2. Create your usage policies for storage. Broadly, you will need to determine what resources you are sharing, with whom, and at what level. A part of this management side of your site design is how your storage system will handle authentication and authorization, violations, precedence and support. The OSG documentation can help you with this.

  3. Determine the architecture for your system: what servers will be part of it, in what way; what transfer types will you support, etc. An important part of this technical side of your design See Designing Your dCache System.

  4. Download the VDT dCache package and configure your installation.

  5. Install dCache on your site's nodes, and clean-up after any failed installations..

  6. Test your new dCache system.

1. System Requirements

The dCache distributed components are implemented with the Cells package. Cells communicate with each other and components of dCache are implemented as cells. Hardware resources on a dCache node depend on what cells are running on that node. For a detailed description of dCache cells and the resources they require, see Descriptive Cell Listing.

In practice you should not need to know this level of detail if you follow the guidelines in this document.

dCache also uses PostgreSQL databases for SRM and pnfs.

The following are the recommended system minimums for an OSG dCache installation. All nodes should have Scientific Linux 4.2 or later.

Note

See dCache Tuning (PDF) for recommended Linux TCP/IP parameter tuning. Power PostgreSQL - PerfList and Tuning PostgreSQL have recommendations on tuning PostgreSQL. The latter also has recommendations for shared buffers for your pnfs node.

1.1. Admin Nodes

  • Dual CPU or Dual Core Intel Xeon, 2.8GHz or better

  • 4 GB RAM or more

  • Raided (mirrored) system disks, hot swappable with spare recommended

  • Raided data disks (RAID 5) with a hot spare on the Admin Nodes with the billing or srm databases running the XFS file system. These should be backed up regularly.

  • Gigabit or better network links

1.2. Pool Nodes

  • Dual CPU or Dual Core Intel Xeon, 2.8GHz or better or equivalent Opteron (e.g., quad Opteron 270)

  • 4 GB RAM or more

  • Raided (mirrored) system disks, hot swappable with spare recommended

  • Raided data disks (RAID 5) hot swappable with hot spare recommended running the XFS file system. External RAIDs are highly recommended.

  • Gigabit or better network links

1.3. pnfs Node

  • 8 GB RAM

  • Postgres databases on raided disk (RAID 5) with back-up performed regularly.

  • Disk should be used exclusively for the pnfs database.

2. Designing Your dCache System

Before configuring your site, you will need the following information about the architecture of your intended dCache system:

  • number of administration nodes and their Fully Qualified Domain Names;

  • number of door nodes required and what types you will support and their Fully Qualified Domain Names;

  • number of pool servers and their Fully Qualified Domain Names;

  • path for the mount points of the storage locations for each pool (each storage location should already exist);

  • written site policies for use posted to an externally available web page.

dCache hardware layouts provides a good guide for the layout of dCache hardware in a variety of scales. The dCache manual also has more information.

3. Getting the VDT dCache tar package

Obtain the tar package from the VDT web site. There are different distributions for Scientific Linux 3 and 4 on both 32- and 64-bit hardware:

  • http://vdt.cs.wisc.edu/software/dcache/server/<VERSION>/vdt-dcache-SL3_32-<VERSION>.tar.gz for Scientific Linux 3 on 32-bit processors

  • http://vdt.cs.wisc.edu/software/dcache/server/<VERSION>/vdt-dcache-SL3_64-<VERSION>.tar.gz for Scientific Linux 3 on 64-bit processors

  • http://vdt.cs.wisc.edu/software/dcache/server/<VERSION>/vdt-dcache-SL4_32-<VERSION>.tar.gz for Scientific Linux 4 on 32-bit processors

  • http://vdt.cs.wisc.edu/software/dcache/server/<VERSION>/vdt-dcache-SL4_64-<VERSION>.tar.gz for Scientific Linux 4 on 64-bit processors

Choose the distribution appropriate for your platform.

Note

Although untested, we believe that these should work on other for Scientific Linux 5 on 32-bit processors

  • http://vdt.cs.wisc.edu/software/dcache/server/<VERSION>/vdt-dcache-SL5_32-<VERSION>.tar.gz similar Linux distributions, especially RedHat Enterprise 3 or 4.

  • 4. Configuring your dCache installation

    Configuring your dCache installation involves getting the appropriate package for your platform and creating the configuration files for the install process. The package comes with a script that creates the configuration files for you, based on your site-specific answers to a series of questions.

    By running config-node.pl in the install directory, the system will create the necessary installation configuration files for you. You will need the following information:

    • The number of administrative nodes you need for your site's dCache installation.

    • If do not want to use the recommended distribution of services: The list of particular dCache services to run on particular admin nodes.

    • The number of door nodes that you will be using

    • Fully Qualified Domain Names (FQDN) for each admin node, door node and pool.

    • The number of pools that you will be using

    • Full paths on a particular pool server to each storage location

    Procedure 1.2. To configure your dCache installation:

    1. Login as root to the machine on which pnfs will run and obtain the vdt-dcache gzipped tarball.

      wget http://vdt.cs.wisc.edu/software/dcache/dcache/<VERSION>/vdt-dcache-<OS_VERSION>-<VERSION>.tar.gz

      The file for your platform may have a different name.

      Important

      SRM/dCache must be installed on the pnfs server machine before being installed on other machines in your installation.

    2. Unpack the package.

      tar -zxvf vdt-dcache-<VERSION>.tar.gz

    3. Change to the install directory and run config-node.pl to configure this node's installation files.

      cd ./vdt-dcache-<VERSION>/install

      ./config-node.pl

      The system will prompt you to answer several questions.

    4. How many admin nodes (non-pool and non-door nodes) do you have? <SomeInteger>

      Enter the number of administrative nodes you want for your site's dCache installation.

      If you have more than one admin node, the application will recommend a particular division of services across them. Although we have found these to be useful, the script allows you to configure the admin nodes as you wish.

    5. The recommended services for node 1 are: 
      <lmDomain poolManager adminDoor httpDomain utilityDomain gplazmaService infoProvider srm>
      Which services do you wish to run (Enter for defaults)?

      Type the services you wish to run on this node, separated by a space. Or, accept the suggested list of services.

      Note

      You can change how the services are distributed later, if needed.

    6. Enter the FQDN node name for node 1: <server.domain.gov>

      Type the Fully Qualified Domain Name (e.g., server.domain.gov) for this node.

      Repeat these steps for all admin nodes in your dCache system.

    7. How many door nodes do you have? <SomeInteger>

      Type the number of door nodes that you will have in your dCache system.

    8. Enter FQDN of door number 1: <server.domain.edu>

      Type the Fully Qualified Domain Name for this door.

      Repeat this for every door node in your system.

    9. Enter the private network that the pools are in.
      If this does not apply, just press enter to skip:

      If your pools are on a private network, type it here.

    10. Enter a pool FQDN name (Press Enter when all are done): <server.domain.com>

      Type the Fully Qualified Domain Name for the next pool in your system and press enter.

    11. Enter the first storage location (Press Enter when all are done)): </path/to/storageLocation>

      For each pool, you will be prompted to enter the paths of the associated storage locations.

      When done entering storage locations for this pool server, simply press Enter at the next storage location prompt.

    12. Repeat steps Step 10 through Step 11 for each pool. When done entering data for your pools, simply press Enter at the pool FQDN name..

    The system will create a site-info-def file. (See the sample output from config-info.pl and the sample site-info.def file.)

    Note

    If you need to, you can also customize the installation process for your installation.

    5. Customizing Your dCache Installation Process

    Some dCache options not set by config-node.pl may be set by editing site-info.def after config-node.pl has been run.

    Warning

    Changing any parameter without understanding what it does is very risky. Change only those parameters that you need to change and document the changes.

    The following parameters are the most commonly customized within the OSG:

    MAX_FTP_STREAMS

    Defines the maximum number of client streams allowed by GridFTP.

    GRIDFTP_PRIVATE_NETWORK

    Defines the network portion of the IP on the private network of the pools. For example, if the pool's IP addresses are 192.168.1.x, you would set GRIDFTP_PRIVATE_NETWORK to 192.168.1.

    USE_MULTI_MOVER_QUEUES

    Sets up the default, WAN and LAN queues on the pools; directs gridftp to use WAN; and directs dcap and gsidcap to use LAN. If desired, set to yes.

    BILLING_USE_DB

    Causes billing information to be copied to a database as well as the log file. If desired, set to yes.

    DCACHE_LOG_DIR

    Causes logs for the dCache domain output to be written to a specified log directory. The directory must exist before dCache is started. Default is /opt/d-cache/log.

    See the sample site-info.def file.

    In addition to customizing site-info.def manually, you can use etc/node_config.delta and etc/dCacheSetup.delta to change any other lines in etc/node_config or config/dCacheSetup. To do so, add the line or lines to the *.delta file, creating the file if necessary, before running install.sh.

    6. Dry Run Testing of install.sh

    If you would like to see what the installation script will do but not do the actual install, run install.sh with the --dryrun option.

    7. Installing dCache

    This section describes how to install the SRM/dCache tar package for OSG.

    Warning

    You must install the pnfs manager on your pnfs server before installing dCache on other machines. Afterwards, you can install it on the other admin nodes, then door nodes and pool servers.

    The next step is to install the dCache on every admin node, door node, and pool server in your dCache site.

    Repeat these steps on each admin node, door node and pool server in your dCache site. You will use the site-info.def file that you just created on the system running pnfs on all the dCache servers.

    Procedure 1.3. To install dCache:

    1. Login as root to the machine on which pnfs will run and obtain the vdt-dcache gzipped tarball.

      wget http://vdt.cs.wisc.edu/software/dcache/dcache/<VERSION>/vdt-dcache-<OS_VERSION>-<VERSION>.tar.gz

      Replace the filename with that appropriate for your environment.

    2. Unpack the package.

      tar -zxvf vdt-dcache-<[version]>.tar.gz

    3. Copy the site-info.def file that you created in the Configuration process to the vdt-cache-<[version]>/install directory on this machine.

    4. Change to the install directory and run install.sh to setup dCache on this machine.

      cd ./vdt-dcache-<[version]>/install

      ./install.sh

    The install.sh script installs dCache into /opt/d-cache.

    For more information about the particulars of the installation scripts that run., see The Installation Scripts.

    8. Configuring firewalls for dCache

    See Firewall Configuration in dCache, The Book.

    9. Starting dCache for the first time

    After installing dCache on all the nodes in your system, start dCache in the following sequence.

    Procedure 1.4. To start dCache after installation:

    1. On the Admin node that runs pnfs, start dCache:

      /opt/d-cache/bin/dcache-core start

    2. On each other Admin and Door node servers, start dCache:

      /opt/d-cache/bin/dcache-core start

    3. On each server running a pool, start the pool server:

      /opt/d-cache/bin/dcache-pool start

    4. Test the installation.

    10. Testing your dCache installation

    After a fresh installation, or after changing your site's configuration, test the dCache installation to ensure that files can be moved and copied back and forth.

    Attempt to move files to and from the dCache instance using client tools appropriate for your doors . Ensure that files have indeed been moved onto the appropriate hardware by checking locally.

    Important

    Do not just attempt local transfers. Attempt transfers to and from offsite clients to ensure that clients outside your internal network can access your dCache instance.

    11. Setting up your dCache system

    Post-installation configuration of the dCache servers is beyond the scope of this document. See Configuration of dCache in the dCache Manual.

    Warning

    Authorization files are not set up by the install scripts. Configure the files used by gPlazma before starting dCache. See gPlazma authorization in dCache in the dCache Manual.

    12. Cleanup

    Complete the cleanup only if you attempted to install dCache and all of the following conditions are true:

    • The installation did not finish successfully.

    • You want to do a clean up.

    • You want to run installation script again.

    12.1. Postgres

    Warning

    This will erase all postgresql rpms and delete any existing postgres databases and installations.

    Procedure 1.5. To clean up the postgres installation:

    1. Make a backup of your databases and anything else that you think you might need later.

    2. Stop the postmaster:

      /etc/init.d/postgresql stop

    3. Erase all postgresql RPMs.

    4. Delete the /var/lib/pgsql directory.

    5. As additional check, make sure following files do not exist:

      • /bin/postmaster

      • /sbin/postmaster

      • /usr/bin/postmaster

      • /usr/sbin/postmaster

      • /usr/local/bin/postmaster

    12.2. pnfs

    Warning

    This will erase the pnfs-server RPM and delete all pnfs directories created in /opt.

    Procedure 1.6. To clean up the postgres installation:

    1. Make a backup of your databases and anything else that you think you might need later.

    2. Erase the pnfs server RPM:

      rpm -e pnfs-postgresql-3.1.10-3.i386

    3. Delete the various pnfs directories

      rm -rf /opt/pnfs.3.1.10

      rm -rf /opt/pnfs

      rm -rf /opt/pnfsdb

      Note

      Depending on where your installation failed, you might not have all of these directories.

    12.3. dCache

    Procedure 1.7. To clean-up the dCache installation portion:

    1. Erase the dcache-server rpm:

      rpm -e dcache-server-1.7.0-19.noarch

      Note

      If this returns scriptlet failed, repeat it using the --noscripts option.

    2. Delete the d-cache directory:

      rm -rf /opt/d-cache

    13. Getting Installation Support

    For more detailed information, see dCache, The Book.

    Specific questions can be directed to the VDT support, .

    14. dCache Details

    This section provides details about various components of the dCache package and installation.

    14.1. The site_info.def file

    site-info.def defines which node shall run each service.

    Note

    All required services that have not been associated with a particular node will run on the admin node.

    The definition for services which may run on more than one node may be a space- separated list of fully-qualified hostnames. If a service needs further information, such as pool size and location, the may appear in colon-separated fields after the hostname.

    14.2. Directory Structure of the dCache Package

    After you untar the dCache package, you will see the following directory structure:

    • ./vdt-dcache-SL3_32-1.0.2 — the root directory

      • ./RPMS

        • dcache-server-1.7.0-19.noarch.rpm

        • postgresql-server-8.1.4-1PGDG.x86_64.rpm

        • postgresql-libs-8.1.4-1PGDG.x86_64.rpm

        • postgresql-8.1.4-1PGDG.x86_64.rpm

        • pnfs-postgresql-3.1.10-3.i386.rpm

      • /install — contains installation scripts:

        • README

        • config-node.pl

        • find_java.sh

        • install.sh

        • install_dcache.sh

        • install_pnfs.sh

        • install_postgres.sh

        • site-info.def

        • site-info.def.template — template used by config-node.pl

        • ./util

          • config_file — Yaim configuration and setup file for dCache 1.7.0-x

    14.3. The Installation Scripts

    The following describes particulars about the individual scripts that make up the dCache package.

    14.3.1. Postgres

    You must uninstall any existing Postgres installation before installing dCache. The script assumes a fresh installation of Postgres. Postgres is only installed on systems that run any of the following:

    • pnfs server node

    • SRM service

    • GridFTP service

    • billing service

    • Space manager service

    • Pin manager service

    • Replica manager service

    14.3.2. pnfs

    The dCache install scripts install pnfs server only if the following are all true:

    1. pnfs server is not already installed;

    2. this machine is defined as the pnfs server node in site-info.def (through the $DCACHE_PNFS variable); and

    3. NFS server is not running on this node.

    Note

    There can only be one pnfs server node in any single dCache installation.

    If you have previously installed pnfs, the installation script will check that the pnfs export for the doors has the value /pnfsdoors. If not, script issues a warning and the installation may fail.

    Warning

    If you have a previous installation of pnfs, it must be running before you run the install script.

    14.3.3. dCache

    The dCache install portion unpacks the dcache-server rpm (if necessary) and modifies the dCache configuration files.

    • Services will be turned on and pools created according to the information in site-info.def.

    • The pnfs node should be installed and started before any GridFTP nodes.

    • pnfsManager will run by default on the pnfs node.

    • Authorization files are not set up by the install script. Configure the files used by gPlazma before starting dCache.

    • Creates etc/node_config.delta and etc/dCacheSetup.delta. These files show exactly what lines in etc/node_config or config/dCacheSetup will be changed during the installation. These files can be used for custom settings as well. See Section 5, “Customizing Your dCache Installation Process”.

    15. Gratia Probe

    Gregory Sharp

    Original author

    Ted Hesselroth

    Modified

    This directory contains the dCache Gratia Probe. It is a python program that reads out the dCache billing database and sends any relevant information to Gratia.

    15.1. Files

    README

    The file you are now reading

    install.sh

    The installation script.

    Alarm.py

    Detects error conditions and sends a warning email

    Checkpoint.py

    Python code to record the last record sent to Gratia

    CheckpointTest.py

    set of unit tests for Checkpoint.py

    DCacheAggregator.py

    Code that reads the billing database, packages it and sends it to Gratia

    dCacheBillingAggregator.p

    The main routine

    Gratia.py

    The Gratia library

    ProbeConfig

    The Gratia configuration file. It tells the Aggregator where to find the Gratia server, security setup, etc.

    external/psycopg2-2.0.5.1.tar.gz

    psycopg python module

    external/SQLAlchemy-0.3.3.tar.gz

    SQLAlchemy python module. Also installs setuptools.

    15.2. How It Works

    The Aggregator reads the Postgres billing database produced by dCache. This version only reads the "billinginfo" and "doorinfo" tables. The first time it runs, the Aggregator scans the billinginfo table for records. It gets the corresponding "initiator" information from the doorinfo table. It packages up all the records it finds and sends them to Gratia, one at a time. After sending each record, it stores the datestamp and transaction field for the sent record in a "Checkpoint" file. The checkpoint is stored in the LogFolder directory (specified in the ProbeConfig file).

    Once all the newly found rows have been sent, it sleeps for the "UpdateFrequency" interval.

    When it wakes up again (or is restarted after a shutdown), the next search of the billinginfo table starts from the last checkpointed datestamp and transaction fields. This might resend some duplicate records to Gratia (but very few). Gratia detects duplicates and does not store them. The Aggregator assumes that Gratia will ignore duplicates and report "OK" when this happens.

    For performance reasons, sites with large dCache billing databases are advised to alter the billinginfo table by adding an index on the pair of columns (datestamp, transaction) and to alter the doorinfo table by adding an index on the (transaction) column. This should speed up the search for newly added records.

    15.3. Upgrading

    If you are upgrading, please make sure that the checkpoint file(s) are not lost or damaged. The upgraded installation will need those file(s) to know which records have been sent to Gratia already.

    15.4. Installation

    The Aggregator is written in Python. It relies on four other 3rd-party packages:

    • Python

    • Gratia

    • SQLAlchemy

    • Psycopg

    The following is a step-by-step for getting everything to work.

    1. Untar the package on the node which is running Postgres for the billing cell. The billing cell is ordinarily running in the dCache http domain and uses a Postgres installed on the same node.

    2. Ensure that you have Python 2.3.4 or newer. Make sure the python binary is in your path and the PYTHONPATH shell variable is not set. Modules psycopg2, SQLAlchemy, and setuptools will be installed or upgraded.

    3. Change to the gratia directory created from step 1 and run install.sh. You may run it as install.sh -d for the dryrun option — it will print what it would do but not execute any commands.

    4. (Optional) Edit the ProbeConfig, and modify variables to your liking. Notes:

      1. SSLRegistrationHost uses port 8880 (the production server). Set this to port 8881 (the test server) for testing purposes.

      2. The MeterName and the SiteName will have the hostname of the node you are installing on. Edit these lines if you wish to use another name, such as your SRM node.

      3. You may wish to adjust the LogLevel and DebugLevel

      4. The directory name MAGIC_VDT_LOCATION can be left as is.

      5. Values for data, log, and tmp can be set to full paths of other directories.

      6. The UpdateFrequency is in seconds. After starting the probe, you will have to wait this amount of time to start seeing entries on the Gratia server.

      7. DBHostName is localhost. Alter this if the node is not as per step 1.

      8. DCacheServerHost is the name of the host on which the dCache master server runs. This will be sent to Gratia. If you do not want to publish the server name, put a fake hostname followed by your domain name.

        Warning

        Do not use a fake domain name, or the statistics in the Gratia database will be almost useless.

      9. If you wish the logs to be separate from the installation directory, set LoggingDir to the full path of another directory.

      10. EmailToList may contain more than one email address. Separate multiple email addresses with a comma and one space. For example, EmailToList="foo@yourDomain.edu, bar@yourDomain.edu".

      11. EmailFromAddress does not need to be a valid address. It should be of the form foo@yourDomain.edu. Some mail systems will reject it, or rewrite the domain if you use something other than the local domain.

      12. The SMTP server is the mail server to which email alarms will be sent. In many cases, localhost is okay. However, test this by stopping the Aggregator. The members of the EmailToList should each receive email warning that the dCache Probe has stopped.

      13. Valid logging levels for AggrLogLevel are:

        • debug

        • info

        • warn

        • error

        Any other value will result in the default of debug.

    15.5. Starting

    1. Change directory to the directory in which the dCacheBillingAggregator is installed.

    2. Run

      nohup python dCacheBillingAggregator.py &

      This will start the program up in the background. The gratia logs and state information will appear in the LogFolder directory. The default for this is logs in the dCacheBillingAggregator directory. Messages from the Aggregator will appear in the log file dcache2gratia.log in the LogFolder directory. If you enable debugging in ProbeConfig, the nohup.out file will be filled with print messages from Gratia. Otherwise, it should be empty.

    15.6. Testing

    1. Open http://gratia-osg.fnal.gov:8880/gratia-reporting/, or other address based on the value of SSLRegistrationHost in ProbeConfig.

    2. Click Custom SQL Report in the left frame.

    3. In the text box, type

      SELECT N.Value, N.StorageUnit, N.PhaseUnit,
      N.Value/N.PhaseUnit/1024.0 as RateInMbPerSecond,
      J.VOName, J.CommonName, J.Status, J.WallDuration, J.StartTime,
      J.EndTime, J.SiteName, J.SubmitHost
      FROM JobUsageRecord J, Network N
      where J.ProbeName like 'dcache%' and J.dbid = N.dbid
      and J.CommonName like 'Ted%' and EndTime > '2007/01/24'
      

      with the applicable CommonName and EndTime.

    4. Click Execute. A table of billing records will be displayed.

    15.7. Stopping the Aggregator

    To stop the Aggregator, you can either send it kill -2 (control-C) or, to be safer, you can create the stop file specified in ProgeConfig. The Aggregator checks for the stop file after each record, so it may take a few seconds before it shuts down.

    15.8. Profiling

    If you find that the probe is not sending records quickly enough to Gratia (for example, you have a backlog of records) you can profile the performance and find the bottlenecks.

    The profiling doesn't include the sleep time, just the time it spent actually trying to discover and send records.

    If the time spent in querying the dCache database is excessive, you may benefit from adding an index on the (datestamp, transaction) fields in the billinginfo table, and on the transaction field in the doorinfo table.

    Important

    This is not intended for normal use. It is intended for debugging if you are experiencing performance problems and then only for people who understand python. If you do not understand python, only do this with the aid of a Gratia support person, or better yet, let them log in and run it.

    To start the probe with profiling enabled, run

    python dCacheBillingAggregator.py -profile

    Let it run for 5 to 10 minutes. Check the log file to ensure that it is indeed sending a significant number of records. Once it has sent at least 100 records, create the stop file (in another window) and within a few minutes it will stop and print out some statistics about where the program spent its time.

    16. Samples

    The following are a site-info.def file, vdt-installer.err file, log file and output of config-node.pl from test installations of dCache.

    16.1. Sample site-info.def file

    The following is the site-info.def file created by this user's config-node.pl.

    MY_DOMAIN="dhcp.fnal.gov"
    #
    # Location of java binary
    #
    JAVA_LOCATION="/usr/bin/java"
    #
    #
    # Components -> "hostname"
    # Please use fully qualified host names.
    # The admin host must be set.
    DCACHE_ADMIN="admin1.domain.gov"
    #
    # These default to DCACHE_ADMIN if not set
    DCACHE_PNFS="admin2.domain.gov"
    DCACHE_INFO_PROVIDER="admin5.domain.gov"
    DCACHE_LM="admin1.domain.gov"
    DCACHE_HTTP="admin5.domain.gov"
    DCACHE_POOL_MANAGER="admin1.domain.gov"
    DCACHE_UTILITY="admin1.domain.gov"
    DCACHE_DIR_DOMAIN="admin2.domain.gov"
    #
    # These default to no host if not set
    #DCACHE_STATISTICS=""
    DCACHE_ADMIN_DOOR="admin1.domain.gov"
    DCACHE_REPLICA_MANAGER="admin4.domain.gov"
    DCACHE_GPLAZMA="admin1.domain.gov"
    #
    #
    # Doors -> "hostname1 hostname2"
    # All doors default to no host
    #
    DCACHE_DOOR_SRM="admin3.domain.gov"
    DCACHE_DOOR_GSIFTP="door1.domain.gov door2.domain.gov door3.domain.gov"
    DCACHE_DOOR_GRIDFTP=$DCACHE_DOOR_GSIFTP
    DCACHE_DOOR_GSIDCAP="door1.domain.gov door2.domain.gov door3.domain.gov"
    DCACHE_DOOR_DCAP="door1.domain.gov door2.domain.gov door3.domain.gov"
    #DCACHE_XROOTD=""
    #
    #
    # Pools -> "hostname1:size1:path1 hostname1:size2:path2 hostname2:size1:path1 hostname2:size2:path2"
    #
    DCACHE_POOLS="pool1.domain.gov:all:/mnt/store1 pool1.domain.gov:all:/mnt/store2 pool1.domain.gov:all:/mnt/store3  pool2.domain.gov:all:/mnt/store1  pool3.domain.gov:all:/mnt/store1 pool3.domain.gov:all:/mnt/store2"
    #
    #
    # Customizations
    #MAX_FTP_STREAMS=
    #GRIDFTP_PRIVATE_NETWORK=
    #USE_MULTI_MOVER_QUEUES=
    #BILLING_USE_DB=
    #DCACHE_LOG_DIR=
    #
    #
    # For the first installation, the following variables must be set to 'yes'.
    # DO NOT set these values to yes on existing production services,
    # dCache internal databases will be deleted.
    #
    #RESET_DCACHE_CONFIGURATION=no
    #RESET_DCACHE_PNFS=no
    #RESET_DCACHE_RDBMS=no
    #
    #
    # VOs -> "vo1 vo2"
    #
    #VOS="atlas cms dteam ops"
    #
    #
    # Do not change this file below this line
    DCACHE_SERVER_RPM=dcache-server-1.7.0-19.noarch.rpm
    
    DCACHE_NODE_TYPES="admin:DCACHE_ADMIN,pool:DCACHE_POOLS"
    
    DCACHE_REQUIRED_SERVICES="lmDomain:DCACHE_LM,poolManager:DCACHE_POOL_MANAGER,pnfsManager:DCACHE_PNFS,dirDomain:DCACHE_DIR_DOMAIN,adminDoor:DCACHE_ADMIN_DOOR,httpDomain:DCACHE_HTTP,utilityDomain:DCACHE_UTILITY,gPlazmaService:DCACHE_GPLAZMA,infoProvider:DCACHE_INFO_PROVIDER"
    
    DCACHE_OPTIONAL_SERVICES="replicaManager:DCACHE_REPLICA_MANAGER,statistics:DCACHE_STATISTICS,GSIDCAP:DCACHE_DOOR_GSIDCAP,DCAP:DCACHE_DOOR_DCAP,GRIDFTP:DCACHE_DOOR_GRIDFTP,SRM:DCACHE_DOOR_SRM,XROOTD:DCACHE_XROOTD,"
    
    DDCACHE_TUNING_OPTIONS="MAX_FTP_STREAMS:gsiftpMaxStreamsPerClient USE_MULTI_MOVER_QUEUES:gsiftpIoQueue=WAN USE_MULTI_MOVER_QUEUES:gsidcapIoQueue=LAN USE_MULTI_MOVER_QUEUES:dcapIoQueue=LAN USE_MULTI_MOVER_QUEUES:poolIoQueue=default,WAN,LAN BILLING_USE_DB:billingToDb DCACHE_LOG_DIR:logArea"

    16.2. Sample output of config-node.pl

    The following lines show the screen output from running config-node.pl for a site with six admin nodes, three door nodes, and three pools, using the recommended default configurations for everything.

    $ ./config-node.pl
    How many admin nodes (non-pool and non-door nodes) do you have? 6
    The recommended services for node 1 are:
    lmDomain poolManager adminDoor utilityDomain gplazmaService
    Which services do you wish to run (Enter for defaults)?
    Enter the FQDN node name for node 1: admin1.domain.gov
    The recommended services for node 2 are:
    pnfsManager dirDomain
    Which services do you wish to run (Enter for defaults)?
    Enter the FQDN node name for node 2: admin2.domain.gov
    The recommended services for node 3 are:
    srm
    Which services do you wish to run (Enter for defaults)?
    Enter the FQDN node name for node 3: admin3.domain.gov
    The recommended services for node 4 are:
    replicaManager
    Which services do you wish to run (Enter for defaults)?
    Enter the FQDN node name for node 4: admin4.domain.gov
    The recommended services for node 5 are:
    httpDomain infoProvider
    Which services do you wish to run (Enter for defaults)?
    Enter the FQDN node name for node 5: admin5.domain.gov
    How many door nodes do you have? 3
    Enter FQDN of door number 1: door1.domain.gov
    Enter FQDN of door number 2: door2.domain.gov
    Enter FQDN of door number 3: door3.domain.gov
    Enter the private network that the pools are in.
    If this does not apply, just press enter to skip:
    Enter a pool FQDN name(Press Enter when all are done): pool1.domain.gov
    Enter the first storage location (Press Enter when all are done)): /mnt/store1
    Enter another storage location (Press Enter when all are done)): /mnt/store2
    Enter another storage location (Press Enter when all are done)): /mnt/store3
    Enter another storage location (Press Enter when all are done)):
    Enter another pool FQDN name(Press Enter when all are done): pool2.domain.gov
    Enter the first storage location (Press Enter when all are done)): /mnt/store1
    Enter another storage location (Press Enter when all are done)):
    Enter another pool FQDN name(Press Enter when all are done): pool3.domain.gov
    Enter the first storage location (Press Enter when all are done)): /mnt/store1
    Enter another storage location (Press Enter when all are done)): /mnt/store2
    Enter another storage location (Press Enter when all are done)):
    Enter another pool FQDN name(Press Enter when all are done):
    Created site-info.def file.
    

    16.3. Sample vdt-install.err

    The following is a sample vdt-install.err file from a test installation.

    At least one of files "/bin/postmaster,/sbin/postmaster, /usr/bin/postmaster, /usr/sbin/postmaster, /usr/local/bin/postmaster" exists Postmaster from previous install is already running..

    16.4. Sample installation log

    The following is a sample installation log from a test installation. The default location for dCache log files is /opt/d-cache/log, although you can set a different location. Other components, such as postgres and pnfs, will also have log files in their default directories. For example, pnfs has three log files in a usual installation:

    • /var/log/pmountd.log

    • /var/log/dbserver.log

    • /var/log/pnfsd.log

    As you can see in this sample dCache log, a prior installation of dCache existed on this server.

    dCache rpm already unpacked.
    Checking services running on this node ...
    postgres  6773     1  0 Nov27 pts/5    00:00:00 /usr/bin/postmaster -p 5432 -D /var/lib/pgsql/data
    PostgreSQL is already installed.
    pnfs-postgresql-3.1.10-1.i386
    PNFS Server rpm is already installed.
    start install_sedcache
    Installing dCache -
    Skipping D-Cache configuration file reset
    start yaim_config_dcache_files_key_value_dCacheSetup
    start yaim_config_file_update_prepare
    stop yaim_config_file_update_prepare
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    start yaim_config_file_update_done
    stop yaim_config_file_update_done
    stop yaim_config_dcache_files_key_value_dCacheSetup
    Modifying dCache configuration files
    start yaim_config_file_update_prepare
    stop yaim_config_file_update_prepare
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    serviceLocatorHost=cd-97177.fnal.gov
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    java=/usr/java/jdk1.5.0_07/bin/java
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    dCapPort1=${portBase}136
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    dCapPort2=${portBase}137
    start yaim_config_file_update_done
    stop yaim_config_file_update_done
    start yaim_query_conf_dcache_services_default_admin
    stop yaim_query_conf_dcache_services_default_admin
    start yaim_config_dcache_files_key_value_node_type
    start yaim_config_file_update_prepare
    stop yaim_config_file_update_prepare
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    Node type already set to admin. Will not set to pool
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    start yaim_config_file_update_done
    stop yaim_config_file_update_done
    stop yaim_config_dcache_files_key_value_node_type
    start yaim_config_file_update_prepare
    stop yaim_config_file_update_prepare
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    SERVER_ID=fnal.gov
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    ADMIN_NODE=cd-97177.fnal.gov
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    NODE_TYPE=admin
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    lmDomain=yes
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    poolManager=yes
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    pnfsManager=yes
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    dirDomain=yes
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    adminDoor=yes
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    httpDomain=yes
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    utilityDomain=yes
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    gPlazmaService=yes
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    infoProvider=yes
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    replicaManager=no
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    statistics=no
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    GSIDCAP=yes
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    DCAP=yes
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    GRIDFTP=yes
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    SRM=yes
    start yaim_config_file_set_value
    stop yaim_config_file_set_value
    XROOTD=no
    start yaim_config_file_update_done
    stop yaim_config_file_update_done
    start config_dcache_config_file_pool_path
    start yaim_config_file_update_prepare
    stop yaim_config_file_update_prepare
    processing pool=all:/mnt/win/work
    start yaim_config_file_update_done
    stop yaim_config_file_update_done
    stop config_dcache_config_file_pool_path
    start config_pnfs_install_mount
    Door is already pnfsserver.
    start config_dcap_create_doors
    Creating 2 dcap doors from /opt/d-cache/config/door1.batch
    stop config_dcap_create_doors
    Running dCache setup script
    start config_dcache_config_dcache_scripts
    
     Checking MasterSetup  ./config/dCacheSetup O.k.
    
       Sanning dCache batch files
    
        Processing adminDoor
        Processing dCache
        Processing dir
        Processing door1
        Processing door2
        Processing gPlazma
        Processing gridftpdoor
        Processing gsidcapdoor
        Processing httpd
        Processing infoProvider
        Processing lm
        Processing pnfs
        Processing pool
        Processing replica
        Processing srm
        Processing statistics
        Processing utility
        Processing xrootdDoor
    
    
     Checking Users database .... Ok
     Checking Security       .... Ok
     Checking JVM ........ Ok
     Checking Cells ...... Ok
     dCacheVersion ....... Version production-1-7-0
    
    trying to stop previous version...
    Using CATALINA_BASE:   /opt/d-cache/libexec/jakarta-tomcat-4.1.31
    Using CATALINA_HOME:   /opt/d-cache/libexec/jakarta-tomcat-4.1.31
    Using CATALINA_TMPDIR: /opt/d-cache/libexec/jakarta-tomcat-4.1.31/temp
    Using JAVA_HOME:       /usr/java/jdk1.5.0_07
    deleting previous version
    AXIS_PATH=/opt/d-cache/libexec/axis-1_3
    TOMCAT_PATH=/opt/d-cache/libexec/jakarta-tomcat-4.1.31
    installing tomcat and axis ...Done
    rm /opt/d-cache/libexec/jakarta-tomcat-4.1.31/webapps/srm/WEB-INF/lib/commons-logging*.jar
    copying jars ...Done
    modifying system CLASSPATH in /opt/d-cache/libexec/jakarta-tomcat-4.1.31/bin/catalina.sh ...Done
    modifying default tomcat SHUTDOWN phrase  and permissions of /opt/d-cache/libexec/jakarta-tomcat-4.1.31/conf/server.xml ...Done
    commenting out AJP CoyoteConnector on port 8009...Done
    Starting up tomcat ...Using CATALINA_BASE:   /opt/d-cache/libexec/jakarta-tomcat-4.1.31
    Using CATALINA_HOME:   /opt/d-cache/libexec/jakarta-tomcat-4.1.31
    Using CATALINA_TMPDIR: /opt/d-cache/libexec/jakarta-tomcat-4.1.31/temp
    Using JAVA_HOME:       /usr/java/jdk1.5.0_07
    Done
    deployng srm v2 application using axis AdminClient ...
    - Unable to find required classes (javax.activation.DataHandler and javax.mail.internet.MimeMultipart). Attachment support is disabled.
    Processing file /opt/d-cache/etc/srmv1-deploy.wsdd
    <Admin>Done processing</Admin>
    - Unable to find required classes (javax.activation.DataHandler and javax.mail.internet.MimeMultipart). Attachment support is disabled.
    Processing file /opt/d-cache/etc/srmv2.2-deploy.wsdd
    <Admin>Done processing</Admin>
    Done
    creating config files and adding configuration info into /opt/d-cache/libexec/jakarta-tomcat-4.1.31/webapps/srm/WEB-INF/web.xml ...Done
    enabling GSI HTTP in tomcat by modifying /opt/d-cache/libexec/jakarta-tomcat-4.1.31/conf/server.xml ...tmp2:/tmp/tmp2Dec060935116541932512
    Done
    turning of sendMultiRefs option in turning off sending of Multi Refs in /opt/d-cache/libexec/jakarta-tomcat-4.1.31/webapps/srm/WEB-INF/server-config.wsddDone
    commenting out HTTP CoyoteConnector on port 8080 in /opt/d-cache/libexec/jakarta-tomcat-4.1.31/conf/server.xml ...Done
    shutting Tomcat down
    Using CATALINA_BASE:   /opt/d-cache/libexec/jakarta-tomcat-4.1.31
    Using CATALINA_HOME:   /opt/d-cache/libexec/jakarta-tomcat-4.1.31
    Using CATALINA_TMPDIR: /opt/d-cache/libexec/jakarta-tomcat-4.1.31/temp
    Using JAVA_HOME:       /usr/java/jdk1.5.0_07
    installing config for startup/shutdown script
    Installation complete
    please use /opt/d-cache/bin/dcache-srm start|stop|restart to startup, shutdown or  restart srm server
    
    [INFO]  Will be mounted to cd-97177.fnal.gov:/fs by dcache-core start-up script.
    [INFO]  Link /pnfs/fnal.gov --> /pnfs/fs/usr already there.
    [INFO]  Link /pnfs/ftpBase --> /pnfs/fs already there.
    
    
    [INFO]  Checking on a possibly existing dCache/PNFS configuration ...
    [INFO]  Found an existing dCache/PNFS configuration!
    [INFO]  Not allowed to overwrite existing configuration.
    
    
    [INFO]  There already are pnfs exports '/pnfsdoors' in
            /pnfs/fs/admin/etc/exports. The GridFTP doors need access to it.
    [INFO]  You may restrict access to this export to the GridFTP doors which
            are not on the admin node. See the documentation.
    
    [INFO]  Generating ssh keys:
    Generating public/private rsa1 key pair.
    Your identification has been saved in ./server_key.
    Your public key has been saved in ./server_key.pub.
    The key fingerprint is:
    65:61:7b:b6:f6:15:a7:16:52:e3:f5:bf:7e:39:41:70 root@cd-97177
    
    
    [INFO]  Not overwriting pool at /mnt/win/work.
    stop config_dcache_config_dcache_scripts
    Finished installing dCache
    stop install_sedcache