Software Development

☞Note: single processor jobs on apollo/artemis should use the para queue and request one node, i.e., "-l nodes=1:ia32:ppn=2".

The following Intel Software Development Products are available on zeus, apollo and artemis:
            Fortran 7.1
                    setenv TMP /temp/`id -nu`
                    source /opt/intel/compiler70/ia64/bin/efcvars.csh (efc command on zeus)
                    source /opt/intel/compiler70/ia32/bin/ifcvars.csh  (ifc command on apollo/artemis)
            Fortran 8.1 (zeus & notus only)
                    setenv TMP /temp/`id -nu`
                    source /opt/intel_fce_80/bin/ifortvars.csh (ifort command)
            Fortran 9.1.032 (notus only)
                    setenv TMP /temp/`id -nu`
                    source /opt/intel/fce/9.1.032/bin/ifortvars.csh (ifort command)
            Fortran 9.1.045 (notus only)
                    setenv TMP /temp/`id -nu`
                    source /opt/intel/fce/9.1.045/bin/ifortvars.csh (ifort command)
            C 7.1
                    setenv TMP /temp/`id -nu`
                    source /opt/intel/compiler70/ia64/bin/eccvars.csh (ecc command on zeus)
                    source /opt/intel/compiler70/ia32/bin/iccvars.csh  (icc command on apollo/artemis)
           C 8.1 (zeus & notus only)
                    setenv TMP /temp/`id -nu`
                    source /opt/intel_cce_80/bin/iccvars.csh (icc command)
           C 9.1 (notus only)
                    setenv TMP /temp/`id -nu`
                    source /opt/intel/cce/9.1.042/bin/iccvars.csh (icc command)
           C++ 7.1
                    setenv TMP /temp/`id -nu`
                    source /opt/intel/compiler70/ia64/bin/eccvars.csh (ecpc command on zeus)
                    source /opt/intel/compiler70/ia32/bin/iccvars.csh  (icpc command on apollo/artemis)
           C++ 8.1 (zeus & notus only)
                    setenv TMP /temp/`id -nu`
                    source /opt/intel_cce_80/bin/iccvars.csh (icpc command)
           C++ 9.1 (notus only)
                    setenv TMP /temp/`id -nu`
                    source /opt/intel/cce/9.1.042/bin/iccvars.csh (icpc command)
            VTune Performance Analyzer
            Integrated Performance Primitives

    Intel® C++ Compiler 8.1 for Linux* Getting Started Guide
    Intel® C++ Compiler 8.1 for Linux* Release Notes
    Intel® C++ Compiler for Linux* User's Guide

    Intel® Fortran Compiler 8.1 for Linux* Getting Started Guide
    Intel® Fortran Compiler 8.1 for Linux* Release Notes
    Intel® Fortran Compiler For Linux* Systems User's Guide
    Intel® Fortran Programmer’s Reference
    Intel® Fortran Libraries Reference

    Parallel Programming with Intel Compilers:
        1) Use -parallel option for auto-parallelization.   
        2) Use "setenv OMP_NUM_THREADS X" in your PBS script where X is the number of processors requested.

    Links:
        Washington University Intro to OpenMP Using C Page
        Ohio Supercomputer Center Using the IA-64 Cluster Page
        NASA SGI Altix 3000 Software Page

The Portland Group PGHPF Workstation compiler is available on apollo, artemis and notus.
                    source /apps/pgi/pgivars.csh enables: pghpf, pgf90, pgf77, pgprof, pgdbg
                    Also available is the PGI User's Guide
                    And the PGI Fortran Reference

GNU Compiler Collection gcc version 3.0.4  is available on zeus, apollo, artemis and notus.

To compile, first do it interactively to determine what to eventually put in a script.
    1) qsub -q stdin -I -l nodes=apollo
    2) qsub a script such as:
        #!/bin/csh
        #PBS -q compile -l nodes=zeus
        #setenv TMP /temp/jmatrow
        source /opt/intel/compiler70/ia64/bin/efcvars.csh
        efc loop1.f
 

Libraries

Math Kernel Library

High-Performance BLAS by Kazushige Goto

Message Passing Interface

MPI is installed on zeus in /usr/bin/mpirun.

MPICH MPI is installed on apollo and artemis. Include /opt/mpich/bin in your path for mpirun, mpicc, mpif77, mpif90.
1.2.6 is installed on notus in /opt/mpich/gnu/bin. 1.2.5 is in /opt/mpich-1.2.5-icc-pbs/bin.

LAM MPI is installed on apollo and artemis. It is already in /usr/bin.
        setenv LAMHELPDIR "/etc/lam"
        setenv LAMBHOST $PBS_NODEFILE
        lamboot
        <run problem>
        lamhalt

INTEL MPI Library is installed on notus. Use this example:
        setenv  SCRDIR /temp/jmatrow/$PBS_JOBNAME.$PBS_JOBID
        # copy datafiles to $SCRDIR and cd to it
        setenv IMPI /opt/intel_mpi_10
        setenv PATH $IMPI/bin64:$PATH
        setenv LD_LIBRARY_PATH $IMPI/lib64
        setenv HOSTFILE $SCRDIR/hostfile
        setenv I_MPI_DEVICE sock
        cp /apps/LS-dyna/mpd.conf $HOME/.mpd.conf
        chown $USER $HOME/.mpd.conf
        chmod 600 $HOME/.mpd.conf
        sort $PBS_NODEFILE | uniq > $HOSTFILE
        setenv NUMNODES `wc -l < $HOSTFILE`
        setenv NUMPROCS `wc -l < $PBS_NODEFILE`
        echo "dir { local $SCRDIR transfer_files }" >pfile
        mpdboot -d -u $USER --file=$HOSTFILE --rsh=ssh -v -n $NUMNODES
        mpiexec -np $NUMPROCS -wdir $SCRDIR <MPI-enabled application goes here>
        mpdcleanup --file=$HOSTFILE --rsh=ssh
        mpdallexit
        # copy results back to home directory
        cd; rm -rf $SCRDIR

Publications

SGI has Scientific Computing Software Library Users Guide..

Intel has Math Kernel Library Reference Manual and Technical User Notes - Linux