2.4.3 Upgrade dcache version 1.9.5-8 (notes http//www.dcache.org/downloads/1.9/release-notes-1.9.5-8.htmll) New instalaltion procedure OSG community core and chronicle toolkit 2.3.5 Recommended for security update. Upgraded dcache-server version to 1.9.2-9 (notes http://www.dcache.org/downloads/1.9/release-notes-1.9.2-11.html) 2.3.3 Fixed srm database leak introduced in 2.3.2 - Bad release - included old trustmanager jar file, gPlazma is not working with GUMS 2.3.2 Security patch release for dcache-server. Upgraded dcache-server version to 1.9.2-9 (notes http://www.dcache.org/downloads/1.9/release-notes-1.9.2-9.html) 2.3.1 Upgraded dcache-server version to 1.9.2-5 (notes http://www.dcache.org/downloads/1.9/release-notes-1.9.2-5.html) New version of postgress (8.3.7) New version of pnfs (pnfs-postgresql-3-1-14-1f) New version of gratia probes (1.02.1-5) 2.2.8 Upgraded dcache-server version to 1.8.0-15p11 (changelog http://trac.dcache.org/trac.cgi/report/22). 2.2.7 Upgraded dcache-server version to 1.8.0-15p8. 2.2.6 Upgraded dcache-server version to 1.8.0-p4. 2.2.5 1. Option to have dCache start at boot time has been added. 2. Gratia probe rpms and installation dialog questions have been added. 3. dcache-server rpm has been upgraded to 1.8-15p3. Release notes from the dcache web site: 15 => 15p3 SRM, SrmSpaceManager and PinManager (utility Domain) only (changelog http://trac.dcache.org/trac.cgi/report/14) 2.2.2-2.2.4 Development versions. 2.2.0 1. Optional jdk install has been upgraded to 1.6_06. 2. Parsing of double backslashes in site-info.def has been fixed. 3. dcache-server rpm has been upgraded to 1.8-15p2. Release notes from the dcache web si te are below. 14 => 15 Synchronized update of SpaceManager and PoolManager required changelog http://trac.dcache.org/trac.cgi/report/12 2.1.7 1. Corrections to README file. Upgraded dcache-server version to 1.8.0-14. From the dcache.org release notes: 13 => 14 This release requires to upgrade all components at the same time (unless you run 1.8.0-13). 1. varios fixes in the new startup scripts 2. fixed socked adapter 3. new release of Chimera 4. fixed FTP protocol vilation in ftp door, which couses remote gsiftp transfers to fail. 5. added option to steer number of concurrent removes on SRM remove. 6. ls in stace manager includes more options 7. More changes. 12 => 13 This release requires to upgrade all components at the same time. 1. All changes of the ccrc08 branch have been applied to the release as well. 2. More changes. 2.1.6 Upgraded dcache-server version to 1.8.0-12p6. The dcache.org release notes only refer to the bug tracking page, http://trac.dcache.org/trac.cgi/report/9. 2.1.5 1. Fixed problem in initialization of use_srmwatch variable in config-node.pl. 2. Fixed wrong error message if no java is installed. 3. Added replica manager to default service lists. 4. Added links for resilient pools in PoolManager.conf. 5. Fixed warning messages shown when installing jdk. 6. Added code to check for whether postgres is installed but not running. 7. Fixed pnfs_running variable not initialized. 8. Added code to write resilient pools into PoolManager.conf from "rm_pools" file. 9. Fixes redirect message to screen from replica manager drop/create db. 10. Added conditional to not change PoolManager.conf if a patch upgrade is being done. 11. Added warning when PoolManager.conf is replaced. 2.1.4 1. Fixed srm db update typo. 2. Added dropdb lines for replica manager. 3. Fixed srmwatch install(4559). 4. Fix error reported by config-node.pl(4560). 5. Will not try to install jdk if same version already installed. 6. Added replica manager to default service lists. Upgraded dcache-server version to 1.8.0-12p5. From the dcache.org release notes: 12p4 => 12p5 In order to have patch 5 enabled you need to upgrade the SRM and the SpaceManager. If you need to have the "SpaceToken" passed to the MSS script you need to upgrade the gsiFtpDoors as well. 1. By default, the output of the "listInvalidSpaces" admin command is limited to 1000 "SpaceReservations". A new optional parameter allows to specify the number of "Space Reservations" to retrieve. 2. BUG FIX : DPM space token rejected as target space token in srm copy request. (Ticket #2432) 3. dCache now validates the format of the adler32 checksum, and prints a warning message, if the format is incorrect, instead of causing the transfer to fail. This is a work around for a CASTOR gridftp bug that returns the ADLER32 checksum value equal to string '(null)' 4. BUG FIX : Due to a race condition, files eventually remain on pools after a srmrm or advisorty delete has been issued. 5. For now dCache assumes that, independend of then number of '/' all paths given by remote systems using the gsiftp URL are absolute pathes. This again is a workround for CASTOR. 6. dCache is now passing SpaceTockens to the MSS (hsm) script to allow smart selection of tape sets at the back-end (ticket 2343) 7. dCache is now failing the SRM 1.1 GET and PUT requests without retries, if the protocol is not supported. 2.1.3 Added postgresql-devel rpms to postgres install. They will be needed for the Gratia probes. Upgraded dcache-server version to 1.8.0-12p4. The release notes from the dCache web page are below. Also, in the dcache.org announcement Patrick Fuhrmann writes "...an upgrade of the PoolManager is required if your current patch level is 1.8.0-12 patch 2. It might be necessary to restart the 'utilityDomain' after having the PoolManager upgraded and started." 12p2 => 12p4 Upgrade of PoolManager and ( optional ) Pools 1. added missig PoolManager fix for 12p2 2. pool will fail to start up if free space smaller than 3* 3. replica manager retry excluded files after 12 hours 4. replica manager recognize missing files 12p1 => 12p2 Upgrade of PoolManager and FTP door nodes only (Other components may stay with 1.8.0-12/p1 ). 1. PoolManager fix: wrong free space has been reported. 2. GridFtpdoor fix: performance marker generation (FTS transfers failed for long transfers). 12 => 12p1 Upgrade of SRM node only (Other components may stay with 1.8.0-12). 1. SpaceManager bug fixed : globus-url-copy and gsidcap failed if implicit space reservation had been enabled for non SRM transfers. 2.1.2 Fixed missing srmwatch rpm. Added compatibility library to postgres intall for SL4_64. Upgraded dcache server version to 1.8.0-12. The dCache changelog entry is 10 => 12 This release requires a space manager database schema migration 1. An improved SrmSpaceManager has been introduced, which requires a schema migration. 2. Space manager schema migration script added. 3. Improved addCacheLocation in PnfsManager. 4. Improved ftp door 5. Fixed file descriptor leak in ActiveAdapter 6. Inter-cell communication uses 'keep alive' sockets, which should fix domain reconnect problems. 7. make srm copy version 1 to version 2 in push mode specify TargetStorageType=PERMANENT, make srmCopy and srmPrepareToPut reject non-PERMANENT storage types, when copies are into dCache 8. Added pnfs2chimera migration scripts. 9. Fixed sticky bit leaking in pin manager. 10. Added user's DN to remoteGsiTransfers billing record 11. With this release, the (new) improved PinManager is becoming the default. 12. Fixed the 'pnfs register' command such that it does not register files that are not marked either precious or cached yet. 9 => 10 1. Fixed several space management related bugs in pools (although their likely are more bugs in this area - don't assume all problems are fixed!). 2. Restructured pool inventory generation. If pool startup fails during the inventory phase, a pool is now disabled in such a way that it cannot be enabled with 'pool enable' (as that would lead to further errors). 3. Removed the -modify flag to pools (it was not fully implemented and was dangerous to use). 4. Improved exception handling in pools. 5. Added periodic consistency checks for space accounting inconsistencies. If inconsistencies are detected, a pool is automatically marked as read-only to protect against further degradation of the pool. 6. Fixed bugs in handling of URI based HSM scripts when used with Berkeley DB based pools. 7. Fixed race condition and error handling in p2p client. 8. Fixed bug in pool repair code used during pool startup, which would previously propagate a bad file size to PNFS. 9. Added command for listing the named sticky bits in the repository file in the pool, and for creating/modifying sticky bits and their lifetimes. 10. added printouts to trace space allocations 11. added error report in case of entry.getSize() not equal to entry.getDataFile.length() on cache repository entry removeal 12. added start/stop/restart to chimera-based nfs interface 13. added namespace migration utility 14. set gap command in the pool supports unit (like 5G ) 15. fixed leading zero in adler32 checksum: 675af7 == 00675af7 16. gsidcap sends VOinfo to space manager if enabled 17. mover queues can reuse working thread ( thread pool instead of new Thread) 18. added execution bit for shell scripts in dcache-server-X.tgz 19. no hidden on-the-fly checksum computing when switched off on the target pool of a P2P-transfer 20. newer xrootd clients are working against dCache (the mover can now handle asyncronous requests) 2.1.1 Upgraded dcache server version to 1.8.0-9. The server changelog entry is below. 8 => 9 This release requires synchronous update of SRM and PooManager. Other components may run mixed. 1. Fixed dead lock in the pool 2. Fixed pool space miss calculation 3. added jar files used by replica manager 4. permission hadler replaced with a new implementation 5. intenal improvements 2.1.0 Modified PoolManager.conf to have two predefined link groups: reserved and public. Example pools have been added to each link group; modify these with the real names of pools you wish to make available for space reservations. Do not put every available pool in the link groups, or writes that do not use space reservations will have no pool to write to. The new PoolManager.conf REPLACES any existing PoolManager.conf. If you have a custom PoolManager.conf, you must merge your existing version with the new version. Implicit space reservation has been disabled by default. Files written without space reservations can still be written, but only to pools that are not in link groups. Added option to install jdk1.6. Restored option to install srmwatch. Upgraded dCache server to 1.8.0-8. Please see the release notes from dcache.org below. For upgrades, the procedure in point 4. is NOT handled by the VDT install scripts. Please reinitialize the replica database as described below. 6 => 8 1. Added some safety checks to the GFtp/2 mover to protect against potential problems in the data channel implementations. It does so by adjusting the space reservation after the transfer. 2. Older releases freed all reserved space when something goes wrong with the space manager. This is not correct when a file is left on the pool, and can lead to pool overbooking. 3. All pools participating in replication MUST be declared in PoolManager.conf in ResilientPools pool group or other group presented to Resilient Manager. 4. DB schema has changed and must be re-initialized by provided template: $ dropdb replicas $ createdb replicas $ psql -U enstore replicas < /opt/d-cache/etc/psql_install_replicas.sql 5. Pool code changed back so PnfsAddCacheLocationMessage is sent out when file added to the pool. This resolves the issue when files been in receiving.transfer state not replicated. 6. Files not found in pnfs now correctly excluded from replications. 7. One more pnfs error report signature was added to exclude files removed from pnfs. 8. Flags "all pools are resilient" ('*') are removed from templates replica.batch and dCacheSetup.template. This is not supported for sometime as it proved to prone to configuration errors, we removed it now from default configuration. 9. replica manager : auto release of excluded files after 12 hour timeout. Configurable via option ExcludeExpirationTO. 10. replica manager : All ongoing replications when pool goes down are stopped. 11. replica manager : connection to DB is split using DB connection pooling 12. replica manager : resilient manager monitoring now reports reason for the file exclusion. 1.2.5 Added code to find rpcinfo when it is not in the path. Added support for Scientific Linux 5. Upgraded dcache-server version to 1.7.0-47. Here are the release notes for 1.7.0-47, from the dcache download web page: 1.7.0-46 => 1.7.0-47 * xrootd transfers are now logged in the billing DB (or billing logfile) the same way as dcap and gridftp. * new Stager available for very high stager load, After a reasonable testing phase, the new stage will become the default * rc retry' command now accepts pattern in addition to '*' wildcard. The new syntax is 'rc retry -p OPTIONS' * When a pool had been excluded, the system has not been able to choose this pool again, even after manual 'rc retry' commands. This is fixed now. * caching of authorizations in srm and caching of gums (saml-vo-mapping) callouts in gPlazma. 1.2.4 Fixed problems with install_postgres script not able to create postgres user. The dcache-server version is upgraded to 1.7.0-46. The changes from 1.7.0-42m from the dcache download web page: 1.7.0-44 => 1.7.0-46 * The proxy component at FTP doors used for passive FTP transfers was back ported from dCache 1.8. This reduces memory consumption and fixes a bug with hanging FTP door instances in case of transfer timeouts. * Bug fix in advisory delete to prevent files remaining in the pool while being deleted from pnfs namespace. * More code to detect possible space miscalcution on pool partitions. 1.7.0-42 => 1.7.0-44 * In case a restore from tape fails, the first retry will not be done on the same host. * The disk space calcuation of the various movers (dcap,ftp...) is now monitored. * Some finalization methods have been removed to improve JVM memory management. 1.2.0 The pool size computation is now posix-compliant. The utility domain is now always deployed with the srm domain. The dcache-server version is upgraded to 1.7.0-42. The dCache changes since 1.7.0-38 are, from the dCache download web page: 1.7.0-41 => 1.7.0-42 * fixed file descriptor leak in HSM connectivity * correct reaction on fileNotExist 1.7.0-39 => 1.7.0-41 ( 1.7.0-40 internal release ) * fixed ftp doors cleanup * fixed dcap door misscounting, in case of GSS authentifivation fails * extra debug statement in poolManager 1.7.0-38 => 1.7.0-39 * Added vector IO for "root" the HEP experimental framework, leading to a potential factor of 12 improvement of performance over 1.7.0-38 for root users. 1.1.9 The dcache-server version is upgraded to 1.7.0-38. This version fixes a problem with pools using excessive memory. Fixed behaviour when options are misspelled. Fixed detection of previously-installed dcache-server rpms. Removed hard-coding of dcache-server rpm name. Will now make log directory if it does not exist. If ReplicaManager is on, will now set replicaManager=yes in dCacheSetup. Will set srmIgnoreClientProtocolOrder=true in dCacheSetup. 1.1.8 Install of pnfs now removes /pnfs/fs/admin/etc/exports/0.0.0.0..0.0.0.0. Problem detecting previously installed dcache-server has been fixed. Dialog script config-node.pl now reads previous door configuration correctly. The dcache-server version is now 1.7.0-36. From the dCache download page: 1.7.0-35 => 1.7.0-36 * Improved dns resolution code, improving the TURL's returned by SRM for multi NIC servers (off by default). * Active FTP adaptor to improve behaviour of pools on private networks (off by default). * Bugfix in the Pool 2 Pool - Client causing wrong pool size. If a p2p transfer fails, the client frees the allocated repository space properly. * The P2P-Client now honors the pool checksum policy. When 'onTransfer=on' the checksum is computed on the fly and compared compaired with the checksum in PNFS. When 'onWrite=on' the file is reread from disk for checksum calculation and check. The P2PClient checksum behaviour is now identical to (DCap-)Mover which accepts a file for WRITE from an external client. If the check fails, the p2p transfer is aborted. * Unauthenticated xrootd write access can be restricted to certain subdirectories. 1.1.7 Removed SRM Watch. Fixed mis-detection of previous dCache server RPM. 1.1.6 Values in dCacheSetup which may have been set by hand since the last upgrade are now transfered to the new dCacheSetup before values from site-info.def are applied. SRM Watch has been upgraded to version 1.0-2. 1.1.5 The package includes the latest patch of dcache-server, version 1.7.0-35. The client package should be upgraded at the same time unless srm is configured to not respect protocol orders. To do this, set (or add) the line srmIgnoreClientProtocolOrder=true in config/dCacheSetup. An option to install SRM Watch has been added to the install scripts. The option can be set through the config-node dialog or by adding the line SRMWATCH_HOST=$DCACHE_DOOR_SRM to site-info.def before running install/install.sh in the vdt-dcache directory. From user-forum at desy dot org: ...both the client and the server should be upgraded at the same time for sites running dcap. This is due to the server now honouring the protocol orders requested by the client and the old client by default sends the protocols in the order http, dcap, gsiftp. This is only believed to be a problem for sites running dcap on the server. The old behaviour of the server can be enforced, within the configuration options of dcache. ChangeLog 1.7.0-34 => 1.7.0-35 * Client now se nds by default for srm get and put operations, protocols in a different priority order. The order has changed from {http,dcap,gsiftp} to {gsiftp,dcap,http} * Adler32 script has changed to automatically determine the path to the jars * The initial state of the pin was set to PINNED, this prevented files being staged, as the PIN manger belived the files where on disk, this has been resolved. * access two different accounts with the same kerberos ticket or x509 cert now supported * Unauthenticated dCap door to be run in the read only mode * Bug fixes to the resilience manger so that 0 byte files wil not replicate in an uncontrolled way 1.1.4 The package includes the latest patch of dcache-server, version 1.7.0-34. This patch includes fixes to bugs in gridftpdoor, which were not included in patch 33. See the excerpt below from the announcement at user-forum@desy.org. 1.7.0-33 => 1.7.0-34 * Changed flag "remove-unexisting-entries-on-flush" to removeUnexistingEntriesOnFlush * Real and important bug fixes to GridFTP door. * Xrootd-redirect of the client (door->pool) now platform independent * Xrootd-mover now has a time out to avoid dangling movers in case no client connects to * Xrootd-door sends now client IP-address to the PoolManager allowing pool matching based on IP subnet. 1.1.2 Includes the newest patch for dCache, 1.7.0-31. This patch fixes srm and OPEN_WAIT problems and has faster authorization.