##############################################################################################
#                       PNFS -> CHIMERA MIGRATION                                            #
##############################################################################################
DISCLAIMER:
THE OSG-STORAGE GROUP DOES NOT ASSUME ANY RESPONSIBILITY/RISK FOR YOUR USE OF THIS DOCUMENT.
IT HAS BEEN PROVIDED FOR YOUR CONVENIENCE AND YOU AS THE USER ARE RESPONSBILE FOR ANY DATA LOSS 
THAT MAY OCCUR DURING MIGRATION. 

The purpose of this document is to provide instructions for migration of data from PNFS based
dCache setup to Chimera

It is advised that you backup the PNFS database directory

Assumptions:
 - PNFS node will become Chimera node
 - dCache and Postgres are already installed on would-be-chimera node

On all nodes, stop all currently running dCache services and Postgres
   On Pool Nodes:                   /etc/init.d/pool stop
   On Gridftp/dCap door Nodes:      /etc/init.d/dcache stop
   On SRM node:                     /etc/init.d/dcache stop
   On Admin node:                   /etc/init.d/dcache stop
   On PNFS node:                    /etc/init.d/dcache stop
                                    /etc/init.d/pnfs stop
   On Admin node:                   /etc/init.d/postgresql stop
   On SRM node:                     /etc/init.d/postgresql stop

 o On PNFS node, create the Chimera user and database and add the Chimera-specific tables and stored procedures
	> createdb -U postgres chimera
	> createuser -U postgres --no-superuser --no-createrole --createdb --pwprompt chimera
	  Enter password for new role: 
	  Enter it again: 
	> psql -U chimera chimera -f /opt/d-cache/libexec/chimera/sql/create.sql
	> createlang -U postgres plpgsql chimera
	> psql -U chimera chimera -f /opt/d-cache/libexec/chimera/sql/pgsql-procedures.sql
 o On PNFS node, modify chimera configuration file
	edit /opt/d-cache/config/chimera-config.xml and in following line
  		 
   	change user to "chimera" and password to what ever you chose
   	also change log filename to "chimera.log" if you want
 o On PNFS node, add following 2 lines to /etc/exports
  	/ localhost(rw)
  	/pnfs
 o On PNFS node, setup chimera startup script and start chimera
  	> ln -s /opt/d-cache/libexec/chimera/chimera-nfs-run.sh /etc/init.d/chimera-nfs-run.sh
	> chkconfig --add chimera-nfs-run.sh
	> chkconfig chimera-nfs-run.sh on
	> chkconfig --list chimera-nfs-run.sh
	  chimera-nfs-run.sh	0:off	1:off	2:on	3:on	4:on	5:on	6:off
	> /etc/init.d/chimera-nfs-run.sh start
 o On PNFS node, setup the directory structure
	> /opt/d-cache/libexec/chimera/chimera-cli.sh Mkdir /pnfs
	> /opt/d-cache/libexec/chimera/chimera-cli.sh Mkdir /pnfs/YOUR-DOMAIN
	> /opt/d-cache/libexec/chimera/chimera-cli.sh Mkdir /pnfs/YOUR-DOMAIN/data
	> /opt/d-cache/libexec/chimera/chimera-cli.sh Mkdir /pnfs/YOUR-DOMAIN/data/fermilab (yours may be different)
	> echo "chimera" | /opt/d-cache/libexec/chimera/chimera-cli.sh Writetag /pnfs/YOUR-DOMAIN/data sGroup
	> echo "StoreName sql" | /opt/d-cache/libexec/chimera/chimera-cli.sh Writetag /pnfs/YOUR-DOMAIN/data OSMTemplate
	> (optional) If you plan to use dCap with mounted file system instead of the URL-syntax 
	  (e.g. dccp /pnfs/desy.de/data/file1 /tmp/file1), we need to mount the root of Chimera locally 
	  (remote mounts are not allowed yet). This will allow us to establish wormhole files so dCap clients 
	   can discover the dCap doors.
		- mount localhost:/ /mnt
		- mkdir /mnt/admin/etc/config/dCache
		- touch /mnt/admin/etc/config/dCache/dcache.conf
		- touch /mnt/admin/etc/config/dCache/'.(fset)(dcache.conf)(io)(on)'
	   	- (for each dcap door) echo "FQHN-dCap-Door:Port" > /mnt/admin/etc/config/dCache/dcache.conf
		- umount /mnt
 o On PNFS node, prepare for migration
	replace the "fermilab" directory with whatever your case may be.
	> /etc/init.d/chimera-nfs-run.sh stop
	> rpm --force -Uvh VDT-DCACHE-INSTALL-DIR/RPMS/pnfs-dump-1.0.21-1.i386.rpm 
 	> Prepare the chimera DB schema to accept the shorter PNFS IDs
	  psql -U postgres -f /opt/pnfs/share/sql/prep-chimera-for-migration.sql chimera    
 	> Start PNFS and identify the PNFS ID of the base data directory
	  /etc/init.d/pnfs start
	> run "mount" to confirm pnfs is mounted
	> cat '/pnfs/YOUR-DOMAIN/data/.(id)(fermilab)'
	000100000000000000001090
	> umount /pnfs/fs
	> /etc/init.d/pnfs stop
	> Start Chimera and discover the new directory's ID
	  /etc/init.d/chimera-nfs-run.sh start
	> mount localhost:/pnfs /pnfs
	> cat '/pnfs/YOUR-DOMAIN/data/.(id)(fermilab)'
	00002290475990A14CCFAFF2EF373EB26725
	> umount /pnfs
	> /etc/init.d/chimera-nfs-run.sh stop
 o On PNFS node, do the migration
	> /etc/init.d/pnfs start
	> /opt/pnfs/tools/pnfsDump -r 000100000000000000001090 -vv -o chimera /tmp/pnfs.sql -2 -p 
	00002290475990A14CCFAFF2EF373EB26725 -o verify /tmp/verify.md5 -r -o files /tmp/files.lst -f | tee /tmp/pnfs-dump.log
	> psql -U postgres -f /tmp/pnfs.sql chimera
	 This process should be left to run to completion. Depending on the size of the namespace, this may take several hours
 o On PNFS node, test that migration was successful
	> /etc/init.d/pnfs stop
	> /etc/init.d/chimera-nfs-run.sh start
	> mount localhost:/pnfs /pnfs
	> cd /pnfs/YOUR-DOMAIN/data/fermilab
	> Run md5sum -c, but filter out the results that are OK. If below command doesn't show any output, means
	  test was successful
	   md5sum -c /tmp/verify.md5 | grep -v :\ OK$
	> cd -
	> umount /pnfs
	> /etc/init.d/chimera-nfs-run.sh stop
	> Run below 4 commands for migration check, only if your dCache version is >= 1.9.5-17
	  - /etc/init.d/pnfs start
	  - chmod 755 /opt/d-cache/libexec/migration-check.sh 
	  - /opt/d-cache/libexec/migration-check.sh -k /tmp/files.lst | tee /tmp/files-dCache-check
	  - /etc/init.d/pnfs stop
 o To let chimera know which files are stored on the pools, import the cache-location information
   into the Chimera database and add directly to the Chimera table.
   companion2chimera.sql is available in the "tools" subdirectory of the package
	> pg_dump -U  postgres -t cacheinfo companion | psql -U postgres chimera
	> psql -U postgres -f companion2chimera.sql chimera
	> echo "select companion2chimera();" | psql -U postgres chimera
	> echo "drop table cacheinfo;" |  psql -U postgres chimera
 o On all nodes except a "chimera only" node
        > edit file /opt/d-cache/etc/node_config to change value of "NAMESPACE" from "pnfs" to "chimera"
 o On PNFS node
	> chkconfig --del pnfs
 	> rm /etc/init.d/pnfs 
 o Bring up services
 	> On Chimera node
		postgres should already be running. If not, start it
		/etc/init.d/postgresql start
		/etc/init.d/chimera-nfs-run.sh start
	> On Admin Node
		/etc/init.d/postgresql start
		/etc/init.d/dcache start	
	> On Chimera Node
		/etc/init.d/dcache start	
	> On SRM Node
		/etc/init.d/postgresql start
                /etc/init.d/dcache start 
	> On Other Door (Gridftp/dcap) Nodes
		/etc/init.d/dcache start
	> On Pool Nodes
		/etc/init.d/pool start
 o Validate your SE
	Visit http://ADMIN-NODE:2288
	Run simple ping and copy commands
	Run functionality testsuite of your choice
 
References:
  - http://trac.dcache.org/projects/dcache/wiki/pnfsDump2MigratePnfs2Chimera
  - http://www.dcache.org/manuals/Book/Book.shtml

Questions:
osg-storage@opensciencegrid.org