Table of Contents
install.shThis document describes the process for installing dCache in an Open Science Grid site using the dCache scripts. For an overview of dCache, see “dCache, the Overview” (PDF). For further details on installing dCache manually, see Installation in dCache, the Book.
Installing dCache can be non-trivial. We suggest following an installation workflow that has worked at other sites:
Join the OSG community! Besides the OSG documentation wiki, mailing lists exist to tie you in with other grid storage administrators.
Create your usage policies for storage. Broadly, you will need to determine what resources you are sharing, with whom, and at what level. A part of this management side of your site design is how your storage system will handle authentication and authorization, violations, precedence and support. The OSG documentation can help you with this.
Determine the architecture for your system: what servers will be part of it, in what way; what transfer types will you support, etc. An important part of this technical side of your design See Designing Your dCache System.
Download the VDT dCache package and configure your installation.
Install dCache on your site's nodes, and clean-up after any failed installations..
Test your new dCache system.
Table of Contents
The dCache distributed components are implemented with the Cells package. Cells communicate with each other and components of dCache are implemented as cells. Hardware resources on a dCache node depend on what cells are running on that node. For a detailed description of dCache cells and the resources they require, see Descriptive Cell Listing.
In practice you should not need to know this level of detail if you follow the guidelines in this document.
dCache also uses PostgreSQL databases for SRM and pnfs.
The following are the recommended system minimums for an OSG dCache installation. All nodes should have Scientific Linux 4.2 or later.
See dCache Tuning (PDF) for recommended Linux TCP/IP parameter tuning. Power PostgreSQL - PerfList and Tuning PostgreSQL have recommendations on tuning PostgreSQL. The latter also has recommendations for shared buffers for your pnfs node.
Dual CPU or Dual Core Intel Xeon, 2.8GHz or better
4 GB RAM or more
Raided (mirrored) system disks, hot swappable with spare recommended
Raided data disks (RAID 5) with a hot spare on the Admin Nodes with the billing or srm databases running the XFS file system. These should be backed up regularly.
Gigabit or better network links
Dual CPU or Dual Core Intel Xeon, 2.8GHz or better or equivalent Opteron (e.g., quad Opteron 270)
4 GB RAM or more
Raided (mirrored) system disks, hot swappable with spare recommended
Raided data disks (RAID 5) hot swappable with hot spare recommended running the XFS file system. External RAIDs are highly recommended.
Gigabit or better network links
Before configuring your site, you will need the following information about the architecture of your intended dCache system:
number of administration nodes and their Fully Qualified Domain Names;
number of door nodes required and what types you will support and their Fully Qualified Domain Names;
number of pool servers and their Fully Qualified Domain Names;
path for the mount points of the storage locations for each pool (each storage location should already exist);
written site policies for use posted to an externally available web page.
dCache hardware layouts provides a good guide for the layout of dCache hardware in a variety of scales. The dCache manual also has more information.
Obtain the tar package from the VDT web site. There are different distributions for Scientific Linux 3 and 4 on both 32- and 64-bit hardware:
http://vdt.cs.wisc.edu/software/dcache/server/<VERSION>/vdt-dcache-SL3_32-<VERSION>.tar.gz
for Scientific Linux 3 on 32-bit processors
http://vdt.cs.wisc.edu/software/dcache/server/<VERSION>/vdt-dcache-SL3_64-<VERSION>.tar.gz
for Scientific Linux 3 on 64-bit processors
http://vdt.cs.wisc.edu/software/dcache/server/<VERSION>/vdt-dcache-SL4_32-<VERSION>.tar.gz
for Scientific Linux 4 on 32-bit processors
http://vdt.cs.wisc.edu/software/dcache/server/<VERSION>/vdt-dcache-SL4_64-<VERSION>.tar.gz
for Scientific Linux 4 on 64-bit processors
Choose the distribution appropriate for your platform.
Although untested, we believe that these should work on other for Scientific Linux 5 on 32-bit processors
http://vdt.cs.wisc.edu/software/dcache/server/<VERSION>/vdt-dcache-SL5_32-<VERSION>.tar.gz
similar Linux distributions, especially RedHat Enterprise 3 or 4.
Configuring your dCache installation involves getting the appropriate package for your platform and creating the configuration files for the install process. The package comes with a script that creates the configuration files for you, based on your site-specific answers to a series of questions.
By running config-node.pl in the install directory, the system will create the
necessary installation configuration files for you. You will need the
following information:
The number of administrative nodes you need for your site's dCache installation.
If do not want to use the recommended distribution of services: The list of particular dCache services to run on particular admin nodes.
The number of door nodes that you will be using
Fully Qualified Domain Names (FQDN) for each admin node, door node and pool.
The number of pools that you will be using
Full paths on a particular pool server to each storage location
Procedure 1.2. To configure your dCache installation:
Login as root to the machine on which pnfs will run and obtain the vdt-dcache gzipped tarball.
wget
http://vdt.cs.wisc.edu/software/dcache/dcache/<VERSION>/vdt-dcache-<OS_VERSION>-<VERSION>.tar.gz
The file for your platform may have a different name.
SRM/dCache must be installed on the pnfs server machine before being installed on other machines in your installation.
Unpack the package.
tar -zxvf
vdt-dcache-<VERSION>.tar.gz
Change to the install
directory and run config-node.pl to configure
this node's installation files.
cd
./vdt-dcache-<VERSION>/install
./config-node.pl
The system will prompt you to answer several questions.
How many admin nodes (non-pool and non-door nodes) do you have? <SomeInteger>Enter the number of administrative nodes you want for your site's dCache installation.
If you have more than one admin node, the application will recommend a particular division of services across them. Although we have found these to be useful, the script allows you to configure the admin nodes as you wish.
The recommended services for node 1 are: <lmDomain poolManager adminDoor httpDomain utilityDomain gplazmaService infoProvider srm>Which services do you wish to run (Enter for defaults)?
Type the services you wish to run on this node, separated by a space. Or, accept the suggested list of services.
You can change how the services are distributed later, if needed.
Enter the FQDN node name for node 1: <server.domain.gov>Type the Fully Qualified Domain Name (e.g., server.domain.gov) for this node.
Repeat these steps for all admin nodes in your dCache system.
How many door nodes do you have? <SomeInteger>Type the number of door nodes that you will have in your dCache system.
Enter FQDN of door number 1: <server.domain.edu>Type the Fully Qualified Domain Name for this door.
Repeat this for every door node in your system.
Enter the private network that the pools are in.If this does not apply, just press enter to skip:
If your pools are on a private network, type it here.
Enter a pool FQDN name (Press Enter when all are done): <server.domain.com>Type the Fully Qualified Domain Name for the next pool in your system and press enter.
Enter the first storage location (Press Enter when all are done)): </path/to/storageLocation>For each pool, you will be prompted to enter the paths of the associated storage locations.
When done entering storage locations for this pool server, simply press Enter at the next storage location prompt.
Repeat steps Step 10 through Step 11 for each pool. When done entering data for your pools, simply press Enter at the pool FQDN name..
The system will create a site-info-def file.
(See the sample output from
config-info.pl and the sample site-info.def file.)
If you need to, you can also customize the installation process for your installation.
Some dCache options not set by config-node.pl
may be set by editing site-info.def after
config-node.pl has been run.
Changing any parameter without understanding what it does is very risky. Change only those parameters that you need to change and document the changes.
The following parameters are the most commonly customized within the OSG:
Defines the maximum number of client streams allowed by GridFTP.
Defines the network portion of the IP on the private network
of the pools. For example, if the pool's IP addresses are
192.168.1.x, you would set
GRIDFTP_PRIVATE_NETWORK to
192.168.1.
Sets up the default, WAN and LAN queues on the pools;
directs gridftp to use WAN; and directs dcap and gsidcap to use
LAN. If desired, set to yes.
Causes billing information to be copied to a database as
well as the log file. If desired, set to
yes.
Causes logs for the dCache domain output to be written to a
specified log directory. The directory must exist before dCache is
started. Default is /opt/d-cache/log.
See the sample
site-info.def file.
In addition to customizing site-info.def
manually, you can use etc/node_config.delta and
etc/dCacheSetup.delta to change any other lines in
etc/node_config or
config/dCacheSetup. To do so, add the line or lines
to the *.delta file, creating the file if necessary,
before running install.sh.
If you would like to see what the installation script will do but
not do the actual install, run install.sh with the
--dryrun option.
This section describes how to install the SRM/dCache tar package for OSG.
You must install the pnfs manager on your pnfs server before installing dCache on other machines. Afterwards, you can install it on the other admin nodes, then door nodes and pool servers.
The next step is to install the dCache on every admin node, door node, and pool server in your dCache site.
Repeat these steps on each admin node, door node and pool server in
your dCache site. You will use the site-info.def file
that you just created on the system running pnfs on all the dCache
servers.
Procedure 1.3. To install dCache:
Login as root to the machine on which pnfs will run and obtain the vdt-dcache gzipped tarball.
wget
http://vdt.cs.wisc.edu/software/dcache/dcache/<VERSION>/vdt-dcache-<OS_VERSION>-<VERSION>.tar.gz
Replace the filename with that appropriate for your environment.
Unpack the package.
tar -zxvf
vdt-dcache-<[version]>.tar.gz
Copy the site-info.def file that you created in the
Configuration process to the vdt-cache-<[version]>/install
directory on this machine.
Change to the install
directory and run install.sh to setup dCache on
this machine.
cd
./vdt-dcache-<[version]>/install
./install.sh
The install.sh script installs dCache into /opt/d-cache.
For more information about the particulars of the installation scripts that run., see The Installation Scripts.
See Firewall Configuration in dCache, The Book.
After installing dCache on all the nodes in your system, start dCache in the following sequence.
Procedure 1.4. To start dCache after installation:
On the Admin node that runs pnfs, start dCache:
/opt/d-cache/bin/dcache-core start
On each other Admin and Door node servers, start dCache:
/opt/d-cache/bin/dcache-core start
On each server running a pool, start the pool server:
/opt/d-cache/bin/dcache-pool start
After a fresh installation, or after changing your site's configuration, test the dCache installation to ensure that files can be moved and copied back and forth.
Attempt to move files to and from the dCache instance using client tools appropriate for your doors . Ensure that files have indeed been moved onto the appropriate hardware by checking locally.
Do not just attempt local transfers. Attempt transfers to and from offsite clients to ensure that clients outside your internal network can access your dCache instance.
Post-installation configuration of the dCache servers is beyond the scope of this document. See Configuration of dCache in the dCache Manual.
Authorization files are not set up by the install scripts. Configure the files used by gPlazma before starting dCache. See gPlazma authorization in dCache in the dCache Manual.
Table of Contents
Complete the cleanup only if you attempted to install dCache and all of the following conditions are true:
The installation did not finish successfully.
You want to do a clean up.
You want to run installation script again.
This will erase all postgresql rpms and delete any existing postgres databases and installations.
Procedure 1.5. To clean up the postgres installation:
Make a backup of your databases and anything else that you think you might need later.
Stop the postmaster:
/etc/init.d/postgresql stop
Erase all postgresql RPMs.
Delete the /var/lib/pgsql
directory.
As additional check, make sure following files do not exist:
/bin/postmaster
/sbin/postmaster
/usr/bin/postmaster
/usr/sbin/postmaster
/usr/local/bin/postmaster
This will erase the pnfs-server RPM and delete all pnfs
directories created in /opt.
Procedure 1.6. To clean up the postgres installation:
Make a backup of your databases and anything else that you think you might need later.
Erase the pnfs server RPM:
rpm -e
pnfs-postgresql-3.1.10-3.i386
Delete the various pnfs directories
rm -rf /opt/pnfs.3.1.10
rm -rf /opt/pnfs
rm -rf /opt/pnfsdb
Depending on where your installation failed, you might not have all of these directories.
For more detailed information, see dCache, The Book.
Specific questions can be directed to the VDT support,
<vdt-support@cs.wisc.edu>.
Table of Contents
This section provides details about various components of the dCache package and installation.
site-info.def defines which node shall run
each service.
All required services that have not been associated with a particular node will run on the admin node.
The definition for services which may run on more than one node may be a space- separated list of fully-qualified hostnames. If a service needs further information, such as pool size and location, the may appear in colon-separated fields after the hostname.
After you untar the dCache package, you will see the following directory structure:
./vdt-dcache-SL3_32-1.0.2 — the root
directory
./RPMS
dcache-server-1.7.0-19.noarch.rpm
postgresql-server-8.1.4-1PGDG.x86_64.rpm
postgresql-libs-8.1.4-1PGDG.x86_64.rpm
postgresql-8.1.4-1PGDG.x86_64.rpm
pnfs-postgresql-3.1.10-3.i386.rpm
/install —
contains installation scripts:
README
config-node.pl
find_java.sh
install.sh
install_dcache.sh
install_pnfs.sh
install_postgres.sh
site-info.def
site-info.def.template —
template used by
config-node.pl
./util
config_file — Yaim
configuration and setup file for dCache
1.7.0-x
The following describes particulars about the individual scripts that make up the dCache package.
You must uninstall any existing Postgres installation before installing dCache. The script assumes a fresh installation of Postgres. Postgres is only installed on systems that run any of the following:
pnfs server node
SRM service
GridFTP service
billing service
Space manager service
Pin manager service
Replica manager service
The dCache install scripts install pnfs server only if the following are all true:
pnfs server is not already installed;
this machine is defined as the pnfs server node in
site-info.def (through the
$DCACHE_PNFS variable);
and
NFS server is not running on this node.
There can only be one pnfs server node in any single dCache installation.
If you have previously installed pnfs, the installation script
will check that the pnfs export for the doors has the value
/pnfsdoors. If not, script issues a warning and
the installation may fail.
If you have a previous installation of pnfs, it must be running before you run the install script.
The dCache install portion unpacks the dcache-server rpm (if necessary) and modifies the dCache configuration files.
Services will be turned on and pools created according to
the information in site-info.def.
The pnfs node should be installed and started before any GridFTP nodes.
pnfsManager will run by default on the pnfs node.
Authorization files are not set up by the install script. Configure the files used by gPlazma before starting dCache.
Creates etc/node_config.delta and
etc/dCacheSetup.delta. These files show
exactly what lines in etc/node_config or
config/dCacheSetup will be changed during
the installation. These files can be used for custom settings as
well. See Section 5, “Customizing Your dCache
Installation Process”.
Table of Contents
This directory contains the dCache Gratia Probe. It is a python program that reads out the dCache billing database and sends any relevant information to Gratia.
The file you are now reading
The installation script.
Detects error conditions and sends a warning email
Python code to record the last record sent to Gratia
set of unit tests for Checkpoint.py
Code that reads the billing database, packages it and sends it to Gratia
The main routine
The Gratia library
The Gratia configuration file. It tells the Aggregator where to find the Gratia server, security setup, etc.
psycopg python module
SQLAlchemy python module. Also installs setuptools.
The Aggregator reads the Postgres billing database produced by
dCache. This version only reads the "billinginfo" and "doorinfo" tables.
The first time it runs, the Aggregator scans the billinginfo table for
records. It gets the corresponding "initiator" information from the
doorinfo table. It packages up all the records it finds and sends them
to Gratia, one at a time. After sending each record, it stores the
datestamp and transaction field for the sent record in a "Checkpoint"
file. The checkpoint is stored in the LogFolder directory (specified in the
ProbeConfig file).
Once all the newly found rows have been sent, it sleeps for the "UpdateFrequency" interval.
When it wakes up again (or is restarted after a shutdown), the next search of the billinginfo table starts from the last checkpointed datestamp and transaction fields. This might resend some duplicate records to Gratia (but very few). Gratia detects duplicates and does not store them. The Aggregator assumes that Gratia will ignore duplicates and report "OK" when this happens.
For performance reasons, sites with large dCache billing databases
are advised to alter the billinginfo table by adding an index on the
pair of columns (datestamp, transaction) and to
alter the doorinfo table by adding an index on the
(transaction) column. This should speed up the
search for newly added records.
If you are upgrading, please make sure that the checkpoint file(s) are not lost or damaged. The upgraded installation will need those file(s) to know which records have been sent to Gratia already.
The Aggregator is written in Python. It relies on four other 3rd-party packages:
Python
Gratia
SQLAlchemy
Psycopg
The following is a step-by-step for getting everything to work.
Untar the package on the node which is running Postgres for the billing cell. The billing cell is ordinarily running in the dCache http domain and uses a Postgres installed on the same node.
Ensure that you have Python 2.3.4 or newer. Make sure the
python binary is in your path and the PYTHONPATH
shell variable is not set. Modules psycopg2, SQLAlchemy, and
setuptools will be installed or upgraded.
Change to the gratia
directory created from step 1 and run
install.sh. You may run it as
install.sh -d for the dryrun option — it will
print what it would do but not execute any commands.
(Optional) Edit the
ProbeConfig, and modify variables to your
liking. Notes:
SSLRegistrationHost uses port 8880 (the production server). Set this to port 8881 (the test server) for testing purposes.
The MeterName and the
SiteName will have the hostname of the node
you are installing on. Edit these lines if you wish to use
another name, such as your SRM node.
You may wish to adjust the LogLevel
and DebugLevel
The directory name MAGIC_VDT_LOCATION can be left as
is.
Values for data,
log, and tmp can be set
to full paths of other directories.
The UpdateFrequency is in seconds.
After starting the probe, you will have to wait this amount of
time to start seeing entries on the Gratia server.
DBHostName is localhost. Alter this
if the node is not as per step 1.
DCacheServerHost is the name of the
host on which the dCache master server runs. This will be sent
to Gratia. If you do not want to publish the server name, put
a fake hostname followed by your domain
name.
Do not use a fake domain name, or the statistics in the Gratia database will be almost useless.
If you wish the logs to be separate from the
installation directory, set LoggingDir to
the full path of another directory.
EmailToList may contain more than one
email address. Separate multiple email addresses with a comma
and one space. For example,
EmailToList="foo@yourDomain.edu,
bar@yourDomain.edu".
EmailFromAddress does not need to be
a valid address. It should be of the form
foo@yourDomain.edu.
Some mail systems will reject it, or rewrite the domain if you
use something other than the local domain.
The SMTP server is the mail server to which email alarms
will be sent. In many cases, localhost is
okay. However, test this by stopping the Aggregator. The
members of the EmailToList should each
receive email warning that the dCache Probe has
stopped.
Valid logging levels for AggrLogLevel
are:
debug
info
warn
error
Any other value will result in the default
of debug.
Change directory to the directory in which the
dCacheBillingAggregator is installed.
Run
nohup python dCacheBillingAggregator.py &This
will start the program up in the background. The gratia logs and
state information will appear in the LogFolder directory. The default for
this is logs in the dCacheBillingAggregator directory.
Messages from the Aggregator will appear in the log file
dcache2gratia.log in the LogFolder directory. If you enable
debugging in ProbeConfig, the nohup.out file
will be filled with print messages from Gratia. Otherwise, it should
be empty.
Open http://gratia-osg.fnal.gov:8880/gratia-reporting/,
or other address based on the value of
SSLRegistrationHost in
ProbeConfig.
Click Custom SQL Report in the left frame.
In the text box, type
SELECT N.Value, N.StorageUnit, N.PhaseUnit, N.Value/N.PhaseUnit/1024.0 as RateInMbPerSecond, J.VOName, J.CommonName, J.Status, J.WallDuration, J.StartTime, J.EndTime, J.SiteName, J.SubmitHost FROM JobUsageRecord J, Network N where J.ProbeName like 'dcache%' and J.dbid = N.dbid and J.CommonName like 'Ted%' and EndTime > '2007/01/24'
with the applicable CommonName and
EndTime.
Click . A table of billing records will be displayed.
To stop the Aggregator, you can either send it kill -2
(control-C) or, to be safer, you can create the stop file
specified in ProgeConfig. The Aggregator checks for
the stop file after each record, so it may take a few seconds before it
shuts down.
If you find that the probe is not sending records quickly enough to Gratia (for example, you have a backlog of records) you can profile the performance and find the bottlenecks.
The profiling doesn't include the sleep time, just the time it spent actually trying to discover and send records.
If the time spent in querying the dCache database is excessive, you may benefit from adding an index on the (datestamp, transaction) fields in the billinginfo table, and on the transaction field in the doorinfo table.
This is not intended for normal use. It is intended for debugging if you are experiencing performance problems and then only for people who understand python. If you do not understand python, only do this with the aid of a Gratia support person, or better yet, let them log in and run it.
To start the probe with profiling enabled, run
python dCacheBillingAggregator.py -profile
Let it run for 5 to 10 minutes. Check the log file to ensure that it is indeed sending a significant number of records. Once it has sent at least 100 records, create the stop file (in another window) and within a few minutes it will stop and print out some statistics about where the program spent its time.
Table of Contents
The following are a site-info.def file,
vdt-installer.err file, log file and output of
config-node.pl from test installations of
dCache.
The following is the site-info.def file
created by this user's config-node.pl.
MY_DOMAIN="dhcp.fnal.gov" # # Location of java binary # JAVA_LOCATION="/usr/bin/java" # # # Components -> "hostname" # Please use fully qualified host names. # The admin host must be set. DCACHE_ADMIN="admin1.domain.gov" # # These default to DCACHE_ADMIN if not set DCACHE_PNFS="admin2.domain.gov" DCACHE_INFO_PROVIDER="admin5.domain.gov" DCACHE_LM="admin1.domain.gov" DCACHE_HTTP="admin5.domain.gov" DCACHE_POOL_MANAGER="admin1.domain.gov" DCACHE_UTILITY="admin1.domain.gov" DCACHE_DIR_DOMAIN="admin2.domain.gov" # # These default to no host if not set #DCACHE_STATISTICS="" DCACHE_ADMIN_DOOR="admin1.domain.gov" DCACHE_REPLICA_MANAGER="admin4.domain.gov" DCACHE_GPLAZMA="admin1.domain.gov" # # # Doors -> "hostname1 hostname2" # All doors default to no host # DCACHE_DOOR_SRM="admin3.domain.gov" DCACHE_DOOR_GSIFTP="door1.domain.gov door2.domain.gov door3.domain.gov" DCACHE_DOOR_GRIDFTP=$DCACHE_DOOR_GSIFTP DCACHE_DOOR_GSIDCAP="door1.domain.gov door2.domain.gov door3.domain.gov" DCACHE_DOOR_DCAP="door1.domain.gov door2.domain.gov door3.domain.gov" #DCACHE_XROOTD="" # # # Pools -> "hostname1:size1:path1 hostname1:size2:path2 hostname2:size1:path1 hostname2:size2:path2" # DCACHE_POOLS="pool1.domain.gov:all:/mnt/store1 pool1.domain.gov:all:/mnt/store2 pool1.domain.gov:all:/mnt/store3 pool2.domain.gov:all:/mnt/store1 pool3.domain.gov:all:/mnt/store1 pool3.domain.gov:all:/mnt/store2" # # # Customizations #MAX_FTP_STREAMS= #GRIDFTP_PRIVATE_NETWORK= #USE_MULTI_MOVER_QUEUES= #BILLING_USE_DB= #DCACHE_LOG_DIR= # # # For the first installation, the following variables must be set to 'yes'. # DO NOT set these values to yes on existing production services, # dCache internal databases will be deleted. # #RESET_DCACHE_CONFIGURATION=no #RESET_DCACHE_PNFS=no #RESET_DCACHE_RDBMS=no # # # VOs -> "vo1 vo2" # #VOS="atlas cms dteam ops" # # # Do not change this file below this line DCACHE_SERVER_RPM=dcache-server-1.7.0-19.noarch.rpm DCACHE_NODE_TYPES="admin:DCACHE_ADMIN,pool:DCACHE_POOLS" DCACHE_REQUIRED_SERVICES="lmDomain:DCACHE_LM,poolManager:DCACHE_POOL_MANAGER,pnfsManager:DCACHE_PNFS,dirDomain:DCACHE_DIR_DOMAIN,adminDoor:DCACHE_ADMIN_DOOR,httpDomain:DCACHE_HTTP,utilityDomain:DCACHE_UTILITY,gPlazmaService:DCACHE_GPLAZMA,infoProvider:DCACHE_INFO_PROVIDER" DCACHE_OPTIONAL_SERVICES="replicaManager:DCACHE_REPLICA_MANAGER,statistics:DCACHE_STATISTICS,GSIDCAP:DCACHE_DOOR_GSIDCAP,DCAP:DCACHE_DOOR_DCAP,GRIDFTP:DCACHE_DOOR_GRIDFTP,SRM:DCACHE_DOOR_SRM,XROOTD:DCACHE_XROOTD," DDCACHE_TUNING_OPTIONS="MAX_FTP_STREAMS:gsiftpMaxStreamsPerClient USE_MULTI_MOVER_QUEUES:gsiftpIoQueue=WAN USE_MULTI_MOVER_QUEUES:gsidcapIoQueue=LAN USE_MULTI_MOVER_QUEUES:dcapIoQueue=LAN USE_MULTI_MOVER_QUEUES:poolIoQueue=default,WAN,LAN BILLING_USE_DB:billingToDb DCACHE_LOG_DIR:logArea"
The following lines show the screen output from running config-node.pl for a site with six admin nodes, three door nodes, and three pools, using the recommended default configurations for everything.
$./config-node.plHow many admin nodes (non-pool and non-door nodes) do you have?6The recommended services for node 1 are: lmDomain poolManager adminDoor utilityDomain gplazmaService Which services do you wish to run (Enter for defaults)? Enter the FQDN node name for node 1:admin1.domain.govThe recommended services for node 2 are: pnfsManager dirDomain Which services do you wish to run (Enter for defaults)? Enter the FQDN node name for node 2:admin2.domain.govThe recommended services for node 3 are: srm Which services do you wish to run (Enter for defaults)? Enter the FQDN node name for node 3:admin3.domain.govThe recommended services for node 4 are: replicaManager Which services do you wish to run (Enter for defaults)? Enter the FQDN node name for node 4:admin4.domain.govThe recommended services for node 5 are: httpDomain infoProvider Which services do you wish to run (Enter for defaults)? Enter the FQDN node name for node 5:admin5.domain.govHow many door nodes do you have?3Enter FQDN of door number 1:door1.domain.govEnter FQDN of door number 2:door2.domain.govEnter FQDN of door number 3:door3.domain.govEnter the private network that the pools are in. If this does not apply, just press enter to skip: Enter a pool FQDN name(Press Enter when all are done):pool1.domain.govEnter the first storage location (Press Enter when all are done)):/mnt/store1Enter another storage location (Press Enter when all are done)):/mnt/store2Enter another storage location (Press Enter when all are done)):/mnt/store3Enter another storage location (Press Enter when all are done)): Enter another pool FQDN name(Press Enter when all are done):pool2.domain.govEnter the first storage location (Press Enter when all are done)):/mnt/store1Enter another storage location (Press Enter when all are done)): Enter another pool FQDN name(Press Enter when all are done):pool3.domain.govEnter the first storage location (Press Enter when all are done)):/mnt/store1Enter another storage location (Press Enter when all are done)):/mnt/store2Enter another storage location (Press Enter when all are done)): Enter another pool FQDN name(Press Enter when all are done): Created site-info.def file.
The following is a sample vdt-install.err file from a test installation.
At least one of files
"/bin/postmaster,/sbin/postmaster, /usr/bin/postmaster,
/usr/sbin/postmaster, /usr/local/bin/postmaster" exists Postmaster from
previous install is already running..
The following is a sample installation log from a test
installation. The default location for dCache log files is /opt/d-cache/log, although you can set a
different location. Other components, such as postgres and pnfs, will
also have log files in their default directories. For example, pnfs has
three log files in a usual installation:
/var/log/pmountd.log
/var/log/dbserver.log
/var/log/pnfsd.log
As you can see in this sample dCache log, a prior installation of dCache existed on this server.
dCache rpm already unpacked.
Checking services running on this node ...
postgres 6773 1 0 Nov27 pts/5 00:00:00 /usr/bin/postmaster -p 5432 -D /var/lib/pgsql/data
PostgreSQL is already installed.
pnfs-postgresql-3.1.10-1.i386
PNFS Server rpm is already installed.
start install_sedcache
Installing dCache -
Skipping D-Cache configuration file reset
start yaim_config_dcache_files_key_value_dCacheSetup
start yaim_config_file_update_prepare
stop yaim_config_file_update_prepare
start yaim_config_file_set_value
stop yaim_config_file_set_value
start yaim_config_file_set_value
stop yaim_config_file_set_value
start yaim_config_file_set_value
stop yaim_config_file_set_value
start yaim_config_file_set_value
stop yaim_config_file_set_value
start yaim_config_file_update_done
stop yaim_config_file_update_done
stop yaim_config_dcache_files_key_value_dCacheSetup
Modifying dCache configuration files
start yaim_config_file_update_prepare
stop yaim_config_file_update_prepare
start yaim_config_file_set_value
stop yaim_config_file_set_value
serviceLocatorHost=cd-97177.fnal.gov
start yaim_config_file_set_value
stop yaim_config_file_set_value
java=/usr/java/jdk1.5.0_07/bin/java
start yaim_config_file_set_value
stop yaim_config_file_set_value
dCapPort1=${portBase}136
start yaim_config_file_set_value
stop yaim_config_file_set_value
dCapPort2=${portBase}137
start yaim_config_file_update_done
stop yaim_config_file_update_done
start yaim_query_conf_dcache_services_default_admin
stop yaim_query_conf_dcache_services_default_admin
start yaim_config_dcache_files_key_value_node_type
start yaim_config_file_update_prepare
stop yaim_config_file_update_prepare
start yaim_config_file_set_value
stop yaim_config_file_set_value
start yaim_config_file_set_value
stop yaim_config_file_set_value
Node type already set to admin. Will not set to pool
start yaim_config_file_set_value
stop yaim_config_file_set_value
start yaim_config_file_set_value
stop yaim_config_file_set_value
start yaim_config_file_set_value
stop yaim_config_file_set_value
start yaim_config_file_set_value
stop yaim_config_file_set_value
start yaim_config_file_set_value
stop yaim_config_file_set_value
start yaim_config_file_set_value
stop yaim_config_file_set_value
start yaim_config_file_set_value
stop yaim_config_file_set_value
start yaim_config_file_set_value
stop yaim_config_file_set_value
start yaim_config_file_set_value
stop yaim_config_file_set_value
start yaim_config_file_set_value
stop yaim_config_file_set_value
start yaim_config_file_set_value
stop yaim_config_file_set_value
start yaim_config_file_set_value
stop yaim_config_file_set_value
start yaim_config_file_set_value
stop yaim_config_file_set_value
start yaim_config_file_set_value
stop yaim_config_file_set_value
start yaim_config_file_set_value
stop yaim_config_file_set_value
start yaim_config_file_set_value
stop yaim_config_file_set_value
start yaim_config_file_set_value
stop yaim_config_file_set_value
start yaim_config_file_update_done
stop yaim_config_file_update_done
stop yaim_config_dcache_files_key_value_node_type
start yaim_config_file_update_prepare
stop yaim_config_file_update_prepare
start yaim_config_file_set_value
stop yaim_config_file_set_value
SERVER_ID=fnal.gov
start yaim_config_file_set_value
stop yaim_config_file_set_value
ADMIN_NODE=cd-97177.fnal.gov
start yaim_config_file_set_value
stop yaim_config_file_set_value
NODE_TYPE=admin
start yaim_config_file_set_value
stop yaim_config_file_set_value
lmDomain=yes
start yaim_config_file_set_value
stop yaim_config_file_set_value
poolManager=yes
start yaim_config_file_set_value
stop yaim_config_file_set_value
pnfsManager=yes
start yaim_config_file_set_value
stop yaim_config_file_set_value
dirDomain=yes
start yaim_config_file_set_value
stop yaim_config_file_set_value
adminDoor=yes
start yaim_config_file_set_value
stop yaim_config_file_set_value
httpDomain=yes
start yaim_config_file_set_value
stop yaim_config_file_set_value
utilityDomain=yes
start yaim_config_file_set_value
stop yaim_config_file_set_value
gPlazmaService=yes
start yaim_config_file_set_value
stop yaim_config_file_set_value
infoProvider=yes
start yaim_config_file_set_value
stop yaim_config_file_set_value
replicaManager=no
start yaim_config_file_set_value
stop yaim_config_file_set_value
statistics=no
start yaim_config_file_set_value
stop yaim_config_file_set_value
GSIDCAP=yes
start yaim_config_file_set_value
stop yaim_config_file_set_value
DCAP=yes
start yaim_config_file_set_value
stop yaim_config_file_set_value
GRIDFTP=yes
start yaim_config_file_set_value
stop yaim_config_file_set_value
SRM=yes
start yaim_config_file_set_value
stop yaim_config_file_set_value
XROOTD=no
start yaim_config_file_update_done
stop yaim_config_file_update_done
start config_dcache_config_file_pool_path
start yaim_config_file_update_prepare
stop yaim_config_file_update_prepare
processing pool=all:/mnt/win/work
start yaim_config_file_update_done
stop yaim_config_file_update_done
stop config_dcache_config_file_pool_path
start config_pnfs_install_mount
Door is already pnfsserver.
start config_dcap_create_doors
Creating 2 dcap doors from /opt/d-cache/config/door1.batch
stop config_dcap_create_doors
Running dCache setup script
start config_dcache_config_dcache_scripts
Checking MasterSetup ./config/dCacheSetup O.k.
Sanning dCache batch files
Processing adminDoor
Processing dCache
Processing dir
Processing door1
Processing door2
Processing gPlazma
Processing gridftpdoor
Processing gsidcapdoor
Processing httpd
Processing infoProvider
Processing lm
Processing pnfs
Processing pool
Processing replica
Processing srm
Processing statistics
Processing utility
Processing xrootdDoor
Checking Users database .... Ok
Checking Security .... Ok
Checking JVM ........ Ok
Checking Cells ...... Ok
dCacheVersion ....... Version production-1-7-0
trying to stop previous version...
Using CATALINA_BASE: /opt/d-cache/libexec/jakarta-tomcat-4.1.31
Using CATALINA_HOME: /opt/d-cache/libexec/jakarta-tomcat-4.1.31
Using CATALINA_TMPDIR: /opt/d-cache/libexec/jakarta-tomcat-4.1.31/temp
Using JAVA_HOME: /usr/java/jdk1.5.0_07
deleting previous version
AXIS_PATH=/opt/d-cache/libexec/axis-1_3
TOMCAT_PATH=/opt/d-cache/libexec/jakarta-tomcat-4.1.31
installing tomcat and axis ...Done
rm /opt/d-cache/libexec/jakarta-tomcat-4.1.31/webapps/srm/WEB-INF/lib/commons-logging*.jar
copying jars ...Done
modifying system CLASSPATH in /opt/d-cache/libexec/jakarta-tomcat-4.1.31/bin/catalina.sh ...Done
modifying default tomcat SHUTDOWN phrase and permissions of /opt/d-cache/libexec/jakarta-tomcat-4.1.31/conf/server.xml ...Done
commenting out AJP CoyoteConnector on port 8009...Done
Starting up tomcat ...Using CATALINA_BASE: /opt/d-cache/libexec/jakarta-tomcat-4.1.31
Using CATALINA_HOME: /opt/d-cache/libexec/jakarta-tomcat-4.1.31
Using CATALINA_TMPDIR: /opt/d-cache/libexec/jakarta-tomcat-4.1.31/temp
Using JAVA_HOME: /usr/java/jdk1.5.0_07
Done
deployng srm v2 application using axis AdminClient ...
- Unable to find required classes (javax.activation.DataHandler and javax.mail.internet.MimeMultipart). Attachment support is disabled.
Processing file /opt/d-cache/etc/srmv1-deploy.wsdd
<Admin>Done processing</Admin>
- Unable to find required classes (javax.activation.DataHandler and javax.mail.internet.MimeMultipart). Attachment support is disabled.
Processing file /opt/d-cache/etc/srmv2.2-deploy.wsdd
<Admin>Done processing</Admin>
Done
creating config files and adding configuration info into /opt/d-cache/libexec/jakarta-tomcat-4.1.31/webapps/srm/WEB-INF/web.xml ...Done
enabling GSI HTTP in tomcat by modifying /opt/d-cache/libexec/jakarta-tomcat-4.1.31/conf/server.xml ...tmp2:/tmp/tmp2Dec060935116541932512
Done
turning of sendMultiRefs option in turning off sending of Multi Refs in /opt/d-cache/libexec/jakarta-tomcat-4.1.31/webapps/srm/WEB-INF/server-config.wsddDone
commenting out HTTP CoyoteConnector on port 8080 in /opt/d-cache/libexec/jakarta-tomcat-4.1.31/conf/server.xml ...Done
shutting Tomcat down
Using CATALINA_BASE: /opt/d-cache/libexec/jakarta-tomcat-4.1.31
Using CATALINA_HOME: /opt/d-cache/libexec/jakarta-tomcat-4.1.31
Using CATALINA_TMPDIR: /opt/d-cache/libexec/jakarta-tomcat-4.1.31/temp
Using JAVA_HOME: /usr/java/jdk1.5.0_07
installing config for startup/shutdown script
Installation complete
please use /opt/d-cache/bin/dcache-srm start|stop|restart to startup, shutdown or restart srm server
[INFO] Will be mounted to cd-97177.fnal.gov:/fs by dcache-core start-up script.
[INFO] Link /pnfs/fnal.gov --> /pnfs/fs/usr already there.
[INFO] Link /pnfs/ftpBase --> /pnfs/fs already there.
[INFO] Checking on a possibly existing dCache/PNFS configuration ...
[INFO] Found an existing dCache/PNFS configuration!
[INFO] Not allowed to overwrite existing configuration.
[INFO] There already are pnfs exports '/pnfsdoors' in
/pnfs/fs/admin/etc/exports. The GridFTP doors need access to it.
[INFO] You may restrict access to this export to the GridFTP doors which
are not on the admin node. See the documentation.
[INFO] Generating ssh keys:
Generating public/private rsa1 key pair.
Your identification has been saved in ./server_key.
Your public key has been saved in ./server_key.pub.
The key fingerprint is:
65:61:7b:b6:f6:15:a7:16:52:e3:f5:bf:7e:39:41:70 root@cd-97177
[INFO] Not overwriting pool at /mnt/win/work.
stop config_dcache_config_dcache_scripts
Finished installing dCache
stop install_sedcache