From Documentation
Jump to: navigation, search
(Installation)
Line 64: Line 64:
 
  [isaac@orc-login2:/work/isaac/OpenFOAM/OpenFOAM-2.3.0] sqsub -r 1d -o log -e err --mpp=4G /work/isaac/OpenFOAM/OpenFOAM-2.3.0/Allwmake
 
  [isaac@orc-login2:/work/isaac/OpenFOAM/OpenFOAM-2.3.0] sqsub -r 1d -o log -e err --mpp=4G /work/isaac/OpenFOAM/OpenFOAM-2.3.0/Allwmake
  
==OpenFoam v2.0.x Installation==
+
==Submitting Parallel Jobs==
 
+
It appears that the 2.0.1 official version has a bug preventing it from compiling on our systems, but the most up-to date, git version (2.0.x) does compile. The instructions below are for our newer systems (orca, saw, hound, kraken). If compiled on one of them, it should also work on the rest without any changes.
+
 
+
===Preliminary steps===
+
 
+
Assuming we want to compile OpenFOAM on global work, the following steps should be done.
+
 
+
First, we create the environment, by executing these commands (you only have to do it once, for the compilation; for using OpenFOAM, these commands will be added in the OpenFOAM bashrc file, as described below):
+
 
+
module unload compile
+
module unload intel
+
module unload openmpi/intel
+
module load gcc/4.3.4
+
module load openmpi/gcc/1.4.2
+
 
+
Next, we create the root OpenFOAM directory, and go there:
+
 
+
mkdir /work/$USER/OpenFOAM
+
cd /work/$USER/OpenFOAM
+
 
+
Then we download, unpack, compile and install the package "flex", which is needed for OpenFOAM compilation:
+
 
+
wget http://prdownloads.sourceforge.net/flex/flex-2.5.35.tar.gz
+
tar xvzf flex-2.5.35.tar.gz
+
cd flex-2.5.35
+
./configure --prefix=/work/$USER/OpenFOAM/flex
+
make
+
make install
+
 
+
===Getting and customizing the OpenFOAM packages===
+
 
+
Getting the third party packages (currently for the 2.0.1 version):
+
 
+
cd /work/$USER/OpenFOAM
+
wget http://downloads.sourceforge.net/foam/ThirdParty-2.0.1.gtgz
+
tar xvzf ThirdParty-2.0.1.gtgz
+
mv ThirdParty-2.0.1 ThirdParty-2.0.x
+
 
+
Getting the up-to-date OpenFOAM package from the git repository:
+
 
+
git clone git://github.com/OpenFOAM/OpenFOAM-2.0.x.git
+
cd OpenFOAM-2.0.x
+
git pull
+
 
+
Now we should edit the etc/bashrc file (inside the OpenFOAM-2.0.x subdirectory), by adding these lines to the very top of the file:
+
 
+
module unload compile
+
module unload intel
+
module unload openmpi/intel
+
module load gcc/4.3.4
+
module load openmpi/gcc/1.4.2
+
export FOAM_VERBOSE=1
+
 
+
Further in the same file, change the value assigned to the foamInstall variable:
+
 
+
foamInstall=/work/$USER/$WM_PROJECT
+
 
+
Comment - if you plan to share OpenFOAM between a few group members, in the above line you have to substitute "$USER" with the actual login name of the user where OpenFOAM is being installed.
+
 
+
Finally, find the line in etc/bashrc which starts with "export WM_MPLIB=", and change it to
+
 
+
export WM_MPLIB=SYSTEMOPENMPI
+
 
+
Next, we add these lines to the bottom (but before the comment "# cleanup environment:") of the file etc/config/settings.sh :
+
 
+
FLEX=${WM_PROJECT_INST_DIR}/flex
+
_foamAddPath ${FLEX}/bin
+
_foamAddLib ${FLEX}/lib
+
export WM_CFLAGS="$WM_CFLAGS -DMPI_NO_CPPBIND"
+
 
+
===Compilation===
+
 
+
Go to the OpenFOAM root directory:
+
 
+
cd /work/$USER/OpenFOAM/OpenFOAM-2.0.x
+
 
+
and execute there
+
 
+
source etc/bashrc
+
./Allwmake
+
 
+
Attention: there is a small problem with the OpenFOAM not being able to find the flex header file, FlexLexer.h . The workaround is to let "Allwmake" run for a while (it should finish compiling in the ThirdParty-2.0.x subdirectory, and start compiling in the OpenFOAM-2.0.x subdirectory), then kill it by pressing Ctrl-C one ore more times, and execute the following command:
+
 
+
ln -s /work/$USER/OpenFOAM/flex/include/FlexLexer.h /work/$USER/OpenFOAM/OpenFOAM-2.0.x/src/OpenFOAM/lnInclude/FlexLexer.h
+
 
+
and then run Allwmake for the second time:
+
 
+
./Allwmake
+
 
+
This time it should run without generating many error messages for a long time (many hours), until OpenFOAM is completely built.
+
 
+
===Using the compiled OpenFOAM===
+
 
+
Every time you or your group member wants to use the compiled OpenFOAM, s/he has to first execute
+
 
+
source /work/$USER/OpenFOAM/OpenFOAM-2.0.x/etc/bashrc
+
 
+
where "$USER" has to correspond to the user who installed the package.
+
 
+
===Submitting Parallel Jobs For OpenFOAM 2.0.x===
+
* First Load the bashrc file of openfoam installation. 
+
* Second Run the decomposePar utility and divide the domain into the number of sub domains required.
+
* Queue you job using 
+
  
 
  sqsub -q mpi -n {No of Subdomains} -r {Run Time Required} -o {Log File} -e {error reporting file} --mpp {mem required per processor}  
 
  sqsub -q mpi -n {No of Subdomains} -r {Run Time Required} -o {Log File} -e {error reporting file} --mpp {mem required per processor}  
 
  {OpenFOAM Solver} -parallel
 
  {OpenFOAM Solver} -parallel
 
 
''Note : Parentheses are not require''
 
 
==OpenFoam v1.7 Installation==
 
 
===Module loading===
 
 
In this installation, Gcc compiler is used in conjunction with Openmpi. So, you should load appropriate modules like
 
 
module load gcc/4.3.4
 
module load openmpi/gcc/1.4.2
 
 
Please note that you may have to 'unload' the default settings like
 
 
module unload intel/11.0.083
 
module unload openmpi/intel/1.4.2
 
 
===Unpacking the source files===
 
The package requires two parts; one is a main solver part and the other is a third-party. Two gtgz files can be downloaded from the website above.
 
 
[isaac@hnd50:/work/isaac/hound/builds/OpenFOAM] ls -lrt
 
total 92196
 
-rwxr-xr-x  1 isaac isaac 38998257 Nov  3 01:27 OpenFOAM-1.7.0.gtgz
 
-rwxr-xr-x  1 isaac isaac 55390881 Nov  3 01:27 ThirdParty-1.7.0.gtgz
 
 
As shown, the mother directory is made in '/work/isaac/hound/builds/OpenFOAM' and unpacked all files there like;
 
 
[isaac@hnd50:/work/isaac/hound/builds/OpenFOAM] ls -lrt
 
drwxrwxr-x 10 isaac isaac    4096 Nov  3 02:19 ThirdParty-1.7.0
 
drwxrwxr-x 10 isaac isaac    4096 Nov  9 11:46 OpenFOAM-1.7.0
 
 
===Setting Environment Variables===
 
 
The environment variable settings are contained in files in an OpenFOAM-1.7.0/etc directory in the OpenFOAM release. Basically two environment files ('bashrc' and 'settings.sh') must be adjusted to the system.
 
 
In 'bashrc', there are some changes;
 
 
# Installation location
 
NEWHOME=/work/isaac/hound/OpenFOAM
 
..
 
foamInstall=$NEWHOME/$WM_PROJECT
 
..
 
export WM_PROJECT_USER_DIR=$NEWHOME/$WM_PROJECT/$USER-$WM_PROJECT_VERSION
 
 
...
 
# Compiler: set to Gcc, Gcc43, Gcc44, or Icc (for Intel's icc)
 
: ${WM_COMPILER:=Gcc43}; export WM_COMPILER
 
 
# WM_MPLIB = SYSTEMOPENMPI | OPENMPI | MPICH | MPICH-GM | HPMPI | GAMMA | MPI | QSMPI
 
: ${WM_MPLIB:=OPENMPI}; export WM_MPLIB
 
 
# Detect system type and set environment variables appropriately
 
# 11:05, 14 February 2011 (EST)11:05, 14 February 2011 (EST)11:05, 14 February 2011 (EST)11:05, 14 February 2011 (EST)11:05, 14 February 2011 (EST)11:05, 14 February 2011 (EST)11:05, 14 February 2011 (EST)11:05, 14 February 2011 (EST)11:05, 14 February 2011 (EST)11:05, 14 February 2011 (EST)11:05, 14 February 2011 (EST)11:05, 14 February 2011 (EST)~~
 
export WM_ARCH=`uname -s`
 
 
case $WM_ARCH in
 
Linux)
 
    WM_ARCH=linux
 
   
 
    # compiler specifics
 
    case `uname -m` in
 
      ...
 
 
        64)
 
            WM_ARCH=linux64
 
            export WM_COMPILER_LIB_ARCH=64
 
            export WM_CC='/opt/sharcnet/gcc/4.3.4/bin/gcc'
 
            export WM_CXX='/opt/sharcnet/gcc/4.3.4/bin/g++'
 
            ...
 
 
In 'settings.sh',
 
 
# Select compiler installation
 
# 11:05, 14 February 2011 (EST)11:05, 14 February 2011 (EST)11:05, 14 February 2011 (EST)11:05, 14 February 2011 (EST)11:05, 14 February 2011 (EST)[[User:Isaac|Isaac]]
 
# compilerInstall = OpenFOAM | system
 
 
 
    if [ -n "$gcc_version" ]
 
    then
 
        gccDir=/opt/sharcnet/gcc/4.3.4
 
 
# Communications library
 
# 11:05, 14 February 2011 (EST)11:05, 14 February 2011 (EST)11:05, 14 February 2011 (EST)11:05, 14 February 2011 (EST)~~
 
       
 
unset MPI_ARCH_PATH MPI_HOME
 
           
 
case "$WM_MPLIB" in
 
OPENMPI)   
 
    mpi_version=openmpi-1.4.2
 
    export MPI_ARCH_PATH=/opt/sharcnet/openmpi/current/gcc
 
       
 
    # Tell OpenMPI where to find its install directory
 
    export OPAL_PREFIX=$MPI_ARCH_PATH
 
       
 
    _foamAddPath    $MPI_ARCH_PATH/bin
 
    _foamAddLib    $MPI_ARCH_PATH/lib
 
    _foamAddMan    $MPI_ARCH_PATH/man
 
       
 
    export FOAM_MPI_LIBBIN=$FOAM_LIBBIN/$mpi_version
 
    unset mpi_version
 
    ...
 
 
===User environment setup===
 
 
The environment variable FOAM_INST_DIR can be used to find and source the appropriate resource file. Here is a bash/ksh/sh example:
 
 
    * export FOAM_INST_DIR=/data/app/OpenFOAM
 
    * foamDotFile=$FOAM_INST_DIR/OpenFOAM-1.7.1/etc/bashrc
 
    * [ -f $foamDotFile ] && . $foamDotFile
 
 
and a csh/tcsh example:
 
 
    * setenv FOAM_INST_DIR /data/app/OpenFOAM
 
    * foamDotFile=$FOAM_INST_DIR/OpenFOAM-1.7.1/etc/cshrc
 
    * if ( -f $foamDotFile ) source $foamDotFile
 
 
The value set in $FOAM_INST_DIR will be used to locate the remaining parts of the OpenFOAM installation.
 
 
===Building the Sources===
 
 
Go to the top-level source directory $WM_PROJECT_DIR and execute the top-level build script
 
 
[isaac@hnd50:/work/isaac/hound/builds/OpenFOAM/OpenFOAM-1.7.0]./Allwmake
 
  
 
==References==
 
==References==
  
 
o OpenFOAM Homepage<br>
 
o OpenFOAM Homepage<br>
http://www.openfoam.com/
+
http://www.openfoam.org/

Revision as of 23:55, 28 July 2014

OPENFOAM
Description: open source CFD software package
SHARCNET Package information: see OPENFOAM software page in web portal
Full list of SHARCNET supported software


Introduction

OpenFOAM is a free, open source CFD software package produced by a commercial company, OpenCFD Ltd (http://www.openfoam.org/). It has a large user base across most areas of engineering and science, from both commercial and academic organisations.

OpenFOAM in SHARCNET

Due to a variety of versions with third party installation for each group/user, SHARCNET does not provide the module for this software and instead we support the users to install his or her own version in the user's working directory. If you need a support for installation, please see below for instance or open a ticket for the help.

Version Selection

As of July 2014, version 2.3.0 is the latest version.

Installation

Here is a short instruction about how to install the package. Here the package is installed in /work/$USER.

0. Make directory

[isaac@orc-login2:/work/isaac] mkdir OpenFOAM

1. Download the package (source + third party) and unzip the package

[isaac@orc-login2:/work/isaac] cd OpenFOAM
[isaac@orc-login2:/work/isaac/OpenFOAM] wget http://downloads.sourceforge.net/foam/OpenFOAM-2.3.0.tgz?use_mirror=mesh
[isaac@orc-login2:/work/isaac/OpenFOAM] wget http://downloads.sourceforge.net/foam/ThirdParty-2.3.0.tgz?use_mirror=mesh
[isaac@orc-login2:/work/isaac/OpenFOAM] tar zxvf OpenFOAM-2.3.0.tgz
[isaac@orc-login2:/work/isaac/OpenFOAM] tar zxvf ThirdParty-2.3.0.tgz

2. Check the module environment and change into

[isaac@orc-login2:/work/isaac/OpenFOAM] cd OpenFOAM-2.3.0
[isaac@orc-login2:/work/isaac/OpenFOAM/OpenFOAM-2.3.0] module list
Currently Loaded Modulefiles:
 1) torque/2.5.13            3) intel/12.1.3             5) mkl/10.3.9               7) ldwrapper/1.1
 2) moab/7.1.1               4) openmpi/intel/1.6.2      6) sq-tm/2.5                8) user-environment/2.0.1
[isaac@orc-login2:/work/isaac/OpenFOAM/OpenFOAM-2.3.0] module switch intel/12.1.3 gcc/4.8.2
[isaac@orc-login2:/work/isaac/OpenFOAM/OpenFOAM-2.3.0] module switch openmpi/intel/1.6.2 openmpi/gcc/1.8.1
[isaac@orc-login2:/work/isaac/OpenFOAM/OpenFOAM-2.3.0] module list
Currently Loaded Modulefiles:
 1) torque/2.5.13            3) gcc/4.8.2                5) mkl/10.3.9               7) ldwrapper/1.1
 2) moab/7.1.1               4) openmpi/gcc/1.8.1        6) sq-tm/2.5                8) user-environment/2.0.1

3. Change basic environment script for OpenFOAM

[isaac@orc-login2:/work/isaac/OpenFOAM/OpenFOAM-2.3.0] vi etc/bashrc
 ...
45 #foamInstall=$HOME/$WM_PROJECT
46 foamInstall=/work/$USER/$WM_PROJECT
...
85 #export WM_MPLIB=OPENMPI
86 export WM_MPLIB=SYSTEMOPENMPI
...
106 #foamOldDirs="$FOAM_INST_DIR $WM_PROJECT_SITE $HOME/$WM_PROJECT/$USER"
107 foamOldDirs="$FOAM_INST_DIR $WM_PROJECT_SITE /work/$USER/$WM_PROJECT/$USER"
...
135 #export WM_PROJECT_USER_DIR=$HOME/$WM_PROJECT/$USER-$WM_PROJECT_VERSION
136 export WM_PROJECT_USER_DIR=/work/$USER/$WM_PROJECT/$USER-$WM_PROJECT_VERSION
[isaac@orc-login2:/work/isaac/OpenFOAM/OpenFOAM-2.3.0] source etc/bashrc

4. Submitting a job to compile

[isaac@orc-login2:/work/isaac/OpenFOAM/OpenFOAM-2.3.0] sqsub -r 1d -o log -e err --mpp=4G /work/isaac/OpenFOAM/OpenFOAM-2.3.0/Allwmake

Submitting Parallel Jobs

sqsub -q mpi -n {No of Subdomains} -r {Run Time Required} -o {Log File} -e {error reporting file} --mpp {mem required per processor} 
{OpenFOAM Solver} -parallel

References

o OpenFOAM Homepage
http://www.openfoam.org/