ROCKS 4.2.1 Upgrade

Below is the initial set of notes covering the upgrade from the 2007.05.16 mtg between tfoley, mhandby, and jpr.

ROCKS 4.2.1 is the latest version of this clustering solution. Documentation is available on-line.

Existing Configurations

These need to be preserved after the upgrade:

  • Reinstall R
  • 411 configuration of extra files
  • preserve ssh host id for head node

Disk partitions:

  • /var/ on the head node upped to 2 gig
  • / sized to 10gig
  • Increase home dir disk space: see ticket #1

New Features:

  • Globus 4.0.4
  • SSH to use BlaserID auth and SSL
  • Relabel nodes in rack to reflect proper numbering.
  • BIOS updated on MoBo? with a unified configuration for network booting.
  • SGE - Should we upgrade to 6.1?
    • Investigate features
    • Support for GridWay??
  • PGI compilers installed (license server needs to be set up by hpcs)

Items for future:

  • Handle account definitions fed from UABgrid Posix account definitions?

Installation summary from mhanby on 2007.06.25:

Cheaha installation is complete. I'm sure some configuration issues will crop up as people start to use it, so I anticipate resolving issues over the next couple weeks.

Some differences from our initial meeting, / is 8 GB's, not 10 and /var is 4 GB's, not 2 GB's.

Globus was installed using the Grid roll, and is 4.0.3-0, not 4.0.4 (as shown on the webapp page). Rocks 4.3 is set to release this week, after it releases, we can check to see if Globus 4.0.4 is included (along with other updated components, Grid Engine). If the updates are worth it, we can look at running the upgrade, which is supposed to go fairly smooth.

I haven't received the BIOS update from Verari. I've been in contact with a tech, he had a BIOS update in mind but wanted to clear it with his developers...

The user home directories on 4 separate nodes appears to function well (from an administrative stand point, anyway). We'll have to see how it functions as multiple users fire off jobs that access storage on several nodes.

The customized user add script that I use on all of the Linux clusters and workstations, /root/scripts/uab-linux-useradd.pl, handles all of the account creation steps, including assigning the users home directory to the next NAS device in the list (round robin approach).

PGI isn't installed yet. John-Paul, have you decided which license server Cheaha should use?

The ssh key for the head node should be the same as the previous install, although the compute nodes have all changed. I've updated the /etc/ssh/ssh_known_hosts file with all the keys for the nodes.

SSH using BlazerID hasn't been implemented. This will have to be a future part of the grid set up, I imagine.

I did preserve the users password and their shell (bash,tcsh) from the previous install.

As for R, the R RPM's for R and libRmath have been installed on the head node and all compute nodes. I've also compiled a gnu version of R 2.3.1 and 2.5.0 in /share/apps/R/R-2.3.1/gnu and /share/apps/R/R-2.5.0/gnu. For both of those, I also installed Rmpi_0.5-3 and snow_0.2-3.

I've also installed Namd, Vmd, EMAN, Intel Compilers 9.1.x, Gromacs, Amber, FFTW, Intel versions of Lam, OpenMPI and MPICH1.