Seawulf Cluster hardware/Software

The Seawulf Cluster has 235 dual processor nodes (470 individual CPU's). These are 3.4Ghz Intel Pentium IV Xeon processors from Dell. Each node has 2GB attached memory and a 40GB hard disk. The connectivity is through 1Gb copper Ethernet, and a 288-port switch from Force10. Access to this cluster is through a front end machine (headnode) and servers which will handle the firewall, also Intel dual processor nodes. Mass storage is through a 20TB Raid array, and an equal amount of back up storage. It is planned to back up the Raid array daily.

Technical support for this machine will be provided by Yan Yu, Ryan Kaufmann, and a team of undergraduates.

The machine will operate Debian GNU/Linux, with batch control by Torque and Maui. openMPI and LAM/MPI will be installed.

The front end nodes may be used for compiling. Please never execute or run a program on these nodes as they are needed by all users and cannot support running programs.

There is no budget for software, but a range of open source software will be installed.

In the table below, we list available software and its location on this machine.

  GNU Compilers /usr/bin
  Intel compilers /usr/local/pkg/intel
Parallel Computing    
  openMPI /usr/local/pkg/openmpi
  openMPI built with Intel compilers /usr/local/pkg/iopenmpi
  ddd /usr/bin
  HDF /uss/local/pkg/HDF4
  vtk /usr/lib
  opengl /usr/lib
Molecular Dynamics    
  NAMD /usr/local/pkg/NAMD_2.6
Quantum Mechanics    
  Espresso /usr/local/pkg/espresso
  NRLmol /usr/local//pkg/NRLMOL
  GAMESS /usr/local/pkg/GAMESS
Linear Algebra    
  PETSc /usr/local/pkg/
  FFTW /usr/local/pkg/
Symbolic computation    
  Octave /usr/bin


Software on Seawulf

There are many packages installed on seawulf. Most of them reside in quite
standard system locations, that is: /usr/bin/, /usr/lib/, /usr/include. How
ever, there are some packages which have to be custom compiled (either not very
common, or require specific compile options), and all of these are located in
/usr/local/pkg. All applications which are not standard will _NOT_ be added to
your PATH/LD_LIBRARY_PATH/ldconfig -- because there are sometimes multiple and
conflicting versions of the same package. So, you, as the user have to manually
add what ever packages you want to your environment.

General Case:

If you need to access a binary which has been installed to /usr/local/pkg/ you
most likely have to just add it to your PATH. For example:
$ export PATH="$PATH:/usr/local/pkg/$APPLICATION/bin"

If you need to access a library which has been installed to /usr/local/pkg/ you,
most likely have to add it to your LD_LIBRARY_PATH. For example:
$ export  LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/pkg/$LIB/lib"
You might also have to pass -I/usr/local/pkg/$LIB/include option to you compile
process or you might be able to get away with:
$ export C_INCLUDE_PATH="$C_INCLUDE_PATH:/usr/local/pkg/$LIB/include"

Setting up the enviroment for some common applications/packages:

OpenMPI (gcc compiled):
$ export PATH="$PATH:/usr/local/pkg/openmpi/bin"
$ export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/pkg/openmpi/lib"

Intel compilers:
$ source /usr/local/pkg/intel/cce/10.1.012/bin/
$ source /usr/local/pkg/intel/fce/10.1.012/bin/
$ source /usr/local/pkg/intel/cc/10.1.012/bin/
$ source /usr/local/pkg/intel/fc/10.1.012/bin/

OpenMPI (Intel compiled):
$ source /usr/local/pkg/intel/cce/10.1.012/bin/
$ source /usr/local/pkg/intel/fce/10.1.012/bin/
$ export PATH="$PATH:/usr/local/pkg/iopenmpi/bin"
$ export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/pkg/iopenmpi/lib"

LAM/MPI (Intel compiled):
$ source /usr/local/pkg/intel/cce/10.1.012/bin/
$ source /usr/local/pkg/intel/fce/10.1.012/bin/
$ export PATH="$PATH:/usr/local/pkg/ilam/bin"
$ export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/pkg/ilam/lib"

If you need to wondering how a particular application/library was compiled we
keep all working/source directories in /usr/local/src/.