Note: This web site is only kept up to date for OSG Software 1.2 (VDT 2.0.0). If you are looking for information for the most recent release, the RPM-based OSG Software 3.0, please see the OSG documentation web site

xrootd Notes

Meeting with Tanya L. and Brian B., 2010-10-?? (late)

Things to change for CMS/Brian when going from VDT Pacman xrootd to VDT RPM xrootd:

Is it possible to have a single configuration script for both ATLAS (which may need separate configurations for Tier 2s and Tier 3s) and CMS?

Brian submitted a patch to the xrootd developers, which they accepted in principle [and in fact, we learned later]. There was concern about an incompatible patch for GUMS integration for CMS.

Brian owns separate software components allow integration with HDFS, dCache, and CMS.

Meeting: OSG - ATLAS - xrootd, 2010-11-02

Attendees: Alain, Tim, Marco, Andy, Charles, Wei, Tanya. Notes from Tanya, then edited and formatted.

VDT Progress With RPMs

Tanya: Are these priorities coming from Atlas Tier-3? Marco: I've talked to Doug B. and it looks like the xrootd RPM will be used outside of the U.S. as well, so the xrootd RPM release has higher priority then other components.

Also, it is OK if the RPMs have to be installed as root but any authorized user should be able to change the configuration. Andy: Non-privileged authorized user should have access to configuration files and be able to start/stop services. Alain: this could be done with sudo.

The xrootd release with all the patches provided by Brian should come within next week. New xrootdfs will be included into repository and can be built simultaneously with xrootd. Tim: Please let me know when the new release is ready. Andy: you should subscribe to xrootd-l@slac.stanford.edu to get notification

Tanya: Should we change anything in VTD configuration for this release? Andy: It is backward compatible and will remain so for the next two(?) years but you can drop couple of unnecessary directives if you want.

Tanya: Should we worry about adding in configuration changes for demonstrator projects. Do Atlas Tier-3 sites need them right away? Andy: I don't think that the regular Tier-3 site will use it now and the sites that are working with demonstrator projects know how to change the configuration. You should talk to Doug and Rik to understand their requirements.

Tanya: Does this new xrootdfs have fixes that take care of file deletion by not authorized users? Wei: Yes, it is fixed but xrootdfs now requires creation of the keys for xootdfs and distribution of them to all data servers. Tanya: We need to understand this better. Could we talk about it in details?

Meeting: Tanya, Doug, Andy, ???; 2010-11-15

Configuration

Filesystem Location of xrootd Install

Doug, mostly, was arguing for installing xrootd into /opt/xrootd.

Misc (ATLAS Software Needs?)

Owner/User of xrootd Install

The conclusion here was that, if vdt-configure-xrootd allows one to change the ownership of key configuration files (if root), then it should be OK. I think.

Merge of xrootd and xrootdfs

Testing

Building

FUSE will have a configure option.

Notes on Building xrootd 3.0.0

These notes are from late December, 2010, and early January, 2011, when Tim and Mat were working on the xrootd 3.0.0 build.

Naming of Source

The source tarball is named "xrootd-v3.0.0.src.tgz" and its unpacks into a single directory named "xrootd". Most source code tarballs these days adhere to a somewhat different convention, and I recommend following it. The tarball name would become:

xrootd-3.0.1.tar.gz

And it would unpack into a single directory named:

xrootd-3.0.1

It is not that I think one naming scheme is better than the other, simply that I think it is wise to follow community practices whenever possible.

Cross-platform Building

Right now, we have packaging steps — e.g., moving files and directories around — in the build and bundle tasks. I think that we should move these steps out to a single, separate task. Doing so will make it easier to split them out into a separate, central build script someday.

On platforms that do not have libxml2 preinstalled (deb_5.0 and sles_9), we can use the libxml2-2.7.3 prereq. To get configure.classic to recognize the prereq, we need to pass the appropriate lib dir and include dir.

We can get the directory that libxml2 is in via running XML_PREFIX=`xml2-config --prefix`. For Redhat, this is /usr; for deb5/sles9, this is /prereqs/libxml2-2.7.3.

Inside, the directories are arranged the same way on both platforms: the include dir is $XML_PREFIX/include/libxml2, and the lib dir is $XML_PREFIX/lib.

We could just pass --with-xml=$XML_PREFIX to configure.classic, but that would cause it to think the include dir is $XML_PREFIX/include. So we need to override that by passing --with-xml-incdir=$XML_PREFIX/include/libxml2

What I did is run xml2-config --cflags and xml2-config --libs and use sed to strip out the extra bits:

  --with-xml-incdir=`xml2-config --cflags | sed -e 's/^-I//'`
  --with-xml-libdir=`xml2-config --libs | sed -e 's/\s*-l\S*//g' -e 's/^-L//'`

Packaging

Two issues so far:

Init script:

We include an init script for xrootdfs that depends on two helper scripts called start.sh and stop.sh. These used to come with xrootdfs, but don't seem to be included now.

I asked upstream about this, but haven't gotten a response yet.

The init script also needs to be updated to point to /opt/vdt/xrootd instead of /opt/vdt/xrootdfs.

Versioning:

3.0.0 < 20100315.1007

I added Epoch support to deal with this.

At this point, basic tests (starting Xrootd, copying a file, stopping Xrootd) pass. I haven't done any testing of Xrootdfs.