From Documentation
Jump to: navigation, search
The content on this software page is still in use for running software on dusky . Do not delete.
COMSOL
Description: Simulation package for a finite element analysis of multi-physics problems
SHARCNET Package information: see COMSOL software page in web portal
Full list of SHARCNET supported software


Introduction

Comsol is not licensed by sharcnet and therefore not free to use. See the Licensing section of the COMSOL software page for further details regarding howto obtain a license.

The current installation of COMSOL includes all available software components such as Multiphysics Documentation, MultiPhysics Models, Accoustics Module, Heat Transfer Module, RF Module, Structural Mechanics Module, ECAD Import Module and Cluster Components. In addition comsol includes Matlab LiveLink support - see the module show comsol command to determine which matlab directory is required to exist on the system in question.

Comsol be used to run batch jobs in the queue (see the Job Submission section below). It can also be used interactively to perform pre/post processing tasks using the comsol GUI (see the General Notes section below).

Version Selection

Sharcnet Hosted Licenses

Researchers who purchase a license to run on a sharcnet hosted license server must specify the port number portX and license server licenseY provided by sharcnet:

module load comsol/5.3
export LM_LICENSE_FILE=portX@licenseY.sharcnet.ca
OR
export LM_LICENSE_FILE=/opt/sharcnet/comsol/5.3/license/license.dat-groupid

Setting license.dat-groupid works in module versions 5.1 and 5.3 but is no longer supported in 5.3.1.348

CMC Hosted Licenses

Researchers who purchase a CMC hosted license should specify the license server as follows. Before running comsol for the first time CMC licensed researchers must run CADactivate as described in section "5 General Notes -> 5.6 CMC Setup" located near the bottom of this wiki document where it is not necessary to run CAD pass on sharcnet providing CMC is aware of the researchers username:

Sharcnet Legacy Systems

module load comsol/5.3
export LM_LICENSE_FILE=6601@license4.sharcnet.ca
OR
export LM_LICENSE_FILE=/opt/sharcnet/comsol/5.3/license/license.dat-cmc

Setting license.dat-cmc works in module versions 5.1 and 5.3 but is no longer supported in 5.3.1.348

Graham

Step A) Create a comsol.lic text file with following content:

[gra-login4:~/.licenses] cat comsol.lic
SERVER 199.241.162.97 ANY 6601
USE_SERVER

Step B) Load module ...

module load comsol/5.3a

Job Submission

Legacy Sharcnet

To submit jobs to the batch queue use the following command structure. Note that a minimum --mpp=5G must be specified for a job to start (required by the comsol java framework). Also note that the values for the "-n" and "-np" arguments must match in this case 16 is specified arbitrarly. As an example for version 4.4 you could do the following:

ssh goblin.sharcnet.ca
module load comsol/5.3
export LM_LICENSE_FILE=portX@licenseY.sharcnet.ca
sqsub -r 60m -q threaded -n 16 --mpp=5G  -o ofile.%J comsol batch -64 -np 16 -inputfile sample_input_file.mph

Modern Sharcnet

Comsol jobs are submitted on graham by creating a slurm script as described in https://docs.computecanada.ca/wiki/COMSOL then submitting it with:

[roberpj@gra-login1:~/example4u]  sbatch mysub.sh

where it was described above howto specify the license server to use by configuring your ~/.license/comsol.lic file.

Graham Example

Initial testing has shown using only 1 node provides the fastest solution time:

[roberpj@gra-login1:~/example4u] cat mysub.sh
#!/bin/bash
#SBATCH --time=01:00:00 
#SBATCH --account=def-roberpj      # your project
#SBATCH --job-name=myscript        # jobname
#SBATCH --cpus-per-task=1          # leave set to 1 
#SBATCH --mem=0                    # use all memory per node
#SBATCH --nodes=1                  # number of nodes (adjustable: 1,2,3,..) 
#SBATCH --ntasks-per-node=32       # cores per node (adjustable: 32,64) 
#SBATCH --output=slurm-%x.%J.out   # slurm-jobname.jobid.out
module load comsol/5.3a 
HOSTFILE=/tmp/hosts.$SLURM_JOB_ID 
srun hostname -s > $HOSTFILE 
comsol batch -nn $SLURM_JOB_NUM_NODES -nnhost 1 -np $SLURM_NTASKS -f $HOSTFILE -inputfile filename.mph -outputfile filename_solved.mph

Memory Hint

The default node memory is a rather conservative MB=ncores*256mb as mentioned on https://docs.computecanada.ca/wiki/Running_jobs#Use_sbatch_to_submit_jobs. If your job crashes due to a suspected shortage then try doubling it with MB=2*cpus-per-task*256M . This can be done in your slurm script by simply adding #SBATCH --mem=__ into the stanza section and resubmitting your job for example:

#SBATCH --mem=16000M

If the job still crashes try doubling the memory again and repeat the process until you find the minimum amount of memory required for the job to run successfully. Its possible to request all memory on a node by setting mem to zero but job start times will be severely delayed and therefore this setting is only generally used when using all (typically 32cores) on a compute node:

#SBATCH --mem=0

Example Job

Sharcnet Legacy Systems

To submit the demo cavity_radiation model problem in parallel the threaded queue should be used as follows:

mkdir ~/samples/comsol
rsync -av vdi-centos6.user.sharcnet.ca:/opt/sharcnet/comsol/4.4/models  ~/samples/comsol/comsol_4.4_models
cd ~/samples/comsol/comsol_4.4_models/models/COMSOL_Multiphysics/Heat_Transfer

sqsub -r 60m -q threaded -n 4 --mpp=5G  -o ofile.%J comsol batch -64 -np 4 -inputfile heat_convection_2d.mph
  Fails

sqsub -r 60m -q threaded -n 4 --mpp=8G  -o ofile.%J comsol batch -64 -np 4 -inputfile heat_convection_2d.mph
  Physical memory: 758 MB
  Virtual memory: 6052 MB

export _JAVA_OPTIONS="-Xms512m -Xmx1024m -Xss512k -XX:ParallelGCThreads=4"
sqsub -r 60m -q threaded -n 4 --mpp=8G  -o ofile.%J comsol batch -64 -np 4 -inputfile heat_convection_2d.mph
  Physical memory: 655 MB
  Virtual memory: 4925 MB

sqsub -r 60m -q threaded -n 4 --mpp=5G  -o ofile.%J comsol batch -64 -np 4 -inputfile heat_convection_2d.mph
  Works

General Notes

Check License Status

By setting the LM_LICENSE_FILE to your license referral file the full status of the remote comsol license server at your instition will be displayed. If an error message is returned please open a problem ticket asap and based the message verbatum since most likely jobs will not be able to check out a license and fail. The comsolstat command is not supported for CMC license holders.

1) module load comsol/5.1
2) export LM_LICENSE_FILE as described in the Version Selection section above
3) comsolstat

Interactive Gui Use

To run comsol 5.3 interactively on sharcnet one must connect with tigervnc client as described in https://www.sharcnet.ca/help/index.php/Remote_Graphical_Connections. Running comsol over ssh in version 5.3 appears no longer to be supported or recommended due to its slowness. Previously in version 5.1 or earlier one could run the gui by doing:

ssh -Y vdi-centos6.user.sharcnet.ca
module load comsol/5.3
export LM_LICENSE_FILE= ...
comsol

Before connecting with Tigervnc specify ogl (OpenGL) by editing /home/$USER/.comsol/v51/comsol.prefs and setting:

 [roberpj@vdi-centos6:~] cat /home/$USER/.comsol/v51/comsol.prefs | grep 3drend=
 graphics.rendering.3drend=ogl
 vs
 graphics.rendering.3drend=sw

Optionally if comsol can be started in the gui set OpenGL through the menu system then restart comsol:

 Options -> Preferences > Graphics and Plot Windows > Rendering > OpenGL (use when connecting by novnc)
or
 Options -> Preferences > Graphics and Plot Windows > Rendering > Software (use when connecting by ssh)

A third option to set ogl or sw is using the command line:

 comsol -3drend ogl
 [export USMESA=1]
 comsol -3drend sw

Once you are connected by tigervnc glxinfo should report the following if ogl/OpenFL rendering is working:

 [roberpj@vdi-centos6:~] glxinfo | grep rendering
 direct rendering: Yes

Sharcnet Legacy Vis Machines

1) Connect to vdi-centos6 with tigervnc
2) module load comsol/5.3.1.348
3) export LM_LICENSE_FILE=6601@license4.sharcnet.ca  (cmc license example)
4) comsol

Modern Graham Vis Machine

Running comsol on gra-vdi in the gui is appropriate when you are dealing with heavy graphics but have low compute requirements. The Graham visualization machine has 512GB memory and 32cores and is therefore double the power of vdi-centos6 or vdi-centos7. At the present time there is only one gra-vdi node however in the future this will be expanded to four nodes with transparent round robin logins.

1) connect to gra-vdi.user.sharcnet.ca with tigervnc
2) open a terminal window and run commands:
3) module load SnEnv
4) module load comsol/5.3.1.348
5) export LM_LICENSE_FILE=6601@199.241.162.97 (cmc license example)
6) comsol

Interactive Command Line Use

Development Nodes

Comsol maybe run interactively on sharcnet development nodes (as they have no memory limits). This maybe useful when actively developing a input file requiring many iterations.

ssh mosaic.sharcnet.ca
ssh mos-dev1
module load comsol/5.3
export LM_LICENSE_FILE=6601@license4.sharcnet.ca (cmc license example)
comsol batch -64 -np 8 -inputfile test.mph

Getting Help

To display comsol command line help information one must first connect into a (orca) development node ie)

ssh vdi-centos6.user.sharcnet.ca
module load comsol/5.1
comsol -help

The full output is:

[roberpj@vdi-centos6:~] comsol -help
Usage: comsol [options] [target] [target arguments]
     
COMSOL targets:
 
   comsol                       Run COMSOL Multiphysics Desktop
   comsol mphclient             Run COMSOL Multiphysics Desktop client
   comsol mphserver             Run COMSOL Multiphysics Server
   comsol batch                 Run a COMSOL job
   comsol compile               Compile a COMSOL Model java file
   comsol mphserver matlab      Run MATLAB with COMSOL Multiphysics Server
   comsol convertpre35a         Run COMSOL pre 3.5a model converter
   comsol hydra                 Run Hydra commands        
   comsol mpd                   Run MPD commands        
  
COMSOL options:
 
   -h, -help                    Show this help message
   -v, -version                 Show version information
   -np <number of processors|   Set number of processors
        auto
   -numasets <number of sets>   Set number of NUMA nodes to optimize
                                processor usage
   -numafirst <numa number>     Set first NUMA node to bind process to
   -mpmode <throughput|         Set multiprocessor mode
            turnaround|owner>
   -blas <{auto}|mkl|acml|      BLAS library to use
           path>
   -blaspath <path>             Set path to BLAS library
   -ipv6                        Activate IPv6 support
   -ckl                         Use class-kit license
   -c <path>                    Path to license file
   -prefsdir <path>             Path to preference directory
   -tmpdir <path>               Path to temporary directory
   -autosave <{on}|off>         Control saving of recovery files
   -recoverydir <path>          Path to recovery directories   
   -3drend <{ogl}|sw>           3D renderer: OpenGL or software rendering
   -comsolinifile <path>        Path to .ini-file to use when launching 
                                COMSOL
   -forcegcc                    Force load of GCC libraries
   -forcecomsolgcc              Force load of GCC libraries shipped with 
                                COMSOL
   -clustersimple               Simple startup of cluster
   -nn <no. of nodes>           Number of nodes
   -nnhost <no. of nodes>       Number of nodes on each host
   -f <path>                    Set path to hostfile
   -mpiroot <path>              Set path to MPI library
   -mpibootstrap <{ssh}|rsh|    Set bootstrap server (for Hydra)
                   fork|slurm|
                   ll|lsf|sge|>
                   jmi>
   -mpibootstrapexec <path>     Set executable used by bootstrap server
   -mpidebug <debug level>      Set the MPI debug level
   -mpienablex                  Enable Xlib forwarding
   -mpifabrics fabric1:fabric2  Select network fabrics where
                                  fabric1 is one of <shm,dapl,tcp,tmi,ofa>
                                  and fabric2 is one of <dapl,tcp,tmi,ofa>
   -mpitranslationcache <{on}|  Enable translation cache for RDMA
                          off>
   -mpd                         Use MPD instead of Hydra launcher in MPI
   -mpirsh <path to command>    Set path to RSH or SSH command (for MPD)
   -mpiarg <arg>                MPI cluster-specific command arguments
   -mpi <{auto}|mpich2|intel|   MPI library to use. path requires
          wccs2003|whpc2008|    environment variable COMSOL_MPI_PATH
          user|path>            to be set
   -mpipath <file>              Set path to MPI library
   -scalapack <{auto}|mkl|      ScaLaPACK library to use. path requires
                user|path>      environment variable COMSOL_SCALAPACK_PATH
                                to be set
   -scalapackpath <file>        Set path to ScaLaPACK library

Comsol options:

   -open <filename>             The input file name

Example:

   comsol -open <filename>

Crashes On Startup

Overquota

Comsol will crash on startup if your home account is overquota. Should this turn out to be the case, then check whether your ~/.comsol directory exceeds 1G. If it does then (assuming it doesnt contain any files you need) remove it, for example:

[roberpj@viz1-uwaterloo:~] quota
Filesystem               Limit      Used               File Count    Checked
bull:/home               1 GB       *1 GB (124%)             3489    12h ago

[roberpj@hnd19:~] du -ksh /home/$PWD/.comsol
1.14G    /home/roberpj/.comsol

[roberpj@hnd19:~] /bin/rm -rf /home/$PWD/.comsol

If the problem happens repeatedly then relocate your ~/.comsol to your work directory as follows:

/bin/rm -rf /home/$PWD/.comsol
mkdir /work/$PWD/.comsol
ln -s /work/$PWD/.comsol /home/$PWD/.comsol 

Corruption

If you submit jobs to the queue that crash giving error messages such as:

(Comsollauncher:30274): GLib-GObject-WARNING **: invalid (NULL) pointer instance
(Comsollauncher:30274): GLib-GObject-CRITICAL **: g_signal_connect_data: assertion `G_TYPE_CHECK_INSTANCE (instance)' failed
(Comsollauncher:30274): Gtk-CRITICAL **: IA__gtk_settings_get_for_screen: assertion `GDK_IS_SCREEN (screen)' failed
(Comsollauncher:30274): GLib-GObject-CRITICAL **: g_object_get: assertion `G_IS_OBJECT (object)' failed
(Comsollauncher:30274): GLib-GObject-WARNING **: value "TRUE" of type `gboolean' is invalid or out of range for property `visible' of type `gboolean'
(Comsollauncher:30274): Gtk-CRITICAL **: IA__gtk_settings_get_for_screen: assertion `GDK_IS_SCREEN (screen)' failed
(Comsollauncher:30274): GLib-GObject-CRITICAL **: g_object_get: assertion `G_IS_OBJECT (object)' failed
(Comsollauncher:30274): Gtk-WARNING **: Screen for GtkWindow not set; you must always set
a screen for a GtkWindow before using the window

The same solution used for "Overquote" should resolve this. That is, remove your ~/.comsol directory making sure first there are no valuable files contained in it. Since its highly possible going overquote in your home, in part due to the size of ~/.comsol was responsible for its corruption in the first place, creation of a link to it under work should seriously be considered.

Java

When you start the comsol gui and it crashes with heap messages involving java that are also written to the hs_err_pidxxxx.log file such as:

# 
# A fatal error has been detected by the Java Runtime Environment: 
# 
# SIGSEGV (0xb) at pc=0x000000311f270fff, pid=27587, tid=140224948848384 
# 
# JRE version: Java(TM) SE Runtime Environment (7.0_75-b13) (build 1.7.0_75-b13) 
# Java VM: Java HotSpot(TM) 64-Bit Server VM (24.75-b04 mixed mode linux-amd64 compressed oops) 
# Problematic frame: 
# C [libGL.so.1+0x70fff] _XReply+0x70fff 
# 
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again 
# 
# If you would like to submit a bug report, please visit: 
# http://bugreport.sun.com/bugreport/crash.jsp 
#
--------------- T H R E A D ---------------
Current thread (0x0000000000fe4000): VMThread [stack: 0x00007f88aa33c000,0x00007f88aa43d000] [id=27640]
siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x00007f883de4e868

Try starting comsol again, when connected over a tigervnc connection to vdi-centos6.user.sharcnet.ca like this:

[roberpj@vdi-centos6:~] module load comsol/5.1
[roberpj@vdi-centos6:~] comsol -comsolinifile /opt/sharcnet/comsol/5.1/COMSOL51/bin/glnxa64/comsol2x.ini

OR

[roberpj@vdi-centos6:~] module load comsol/5.3 
[roberpj@vdi-centos6:~] comsol -comsolinifile /opt/sharcnet/comsol/5.3/comsol53/multiphysics/bin/glnxa64/comsol2x.ini

If the heap crash still occurs try 4x, 6x, 8x or 10x and if that doesnt resolve the problem open a sharcnet ticket.

CMC Setup

June, 2015: The 15 step CADactivate procedure shown below currently does not work. Instead you must email yuhua.zhou@cmc.ca and request your sharcnet username be manually registered with the CMC authentication servers before running comsol.

Users with a CMC comsol license need to run CADactivate one time on a sharcnet machine. Doing this will register your sharcnet username with CMC thus allowing comsol to checkout your license from the CMC servers and start. The steps required are:

1) On your laptop open a browser and goto https://www.sharcnet.ca/my/systems
2) Scroll down to the Visualization Systems table
3) Click the Blue icon beside vdi-centos6. If for some reason the vdi machine is down use viz10-uwo or viz11-uwo instead.
4) Once a remote desktop session has started open a terminal and run the firefox command.
5) Goto google.ca and search for CADactivate CMC
6) Click on link with CADactivate.aspx at the end. A webpage containing "To Launch the Activation Program Click on the CADactivate logo" should appear
7) Click the CADactivate button and a popup windows with "Opening launch.jnlp" at the top should appear
8) Select the bullet "Open with IcedTea Web Start (default) and click OK. A security warning popup with buttons RUN, Sandbox and Cancel should appear
9) Click the Sandbox button, and another popup should appear with a yield logo containing Yes and No buttons. Click Yes
10) Repeat steps 9) and 10) approx three time until you get to a popup that has two buttons Francais and English
11) Click your preferred launguage and you should be taken to a "Welcome to the CMC CAD activation wizard" popup
12) Read over the View CMC ACcount terms and click button I agree to account terms then click the Next button
13) You should now be able to enter your "CMC Login Name: and Password:" and then click Next
14) Continue on to complete the registration process and then exit firefox
15) With all that out of the way, startup comsol using the standard instructions above in this document

References

o Comsol Homepage
http://www.comsol.com/

o Comsol Release History
https://www.comsol.com/release-history

o Comsol Updates
https://www.comsol.com/product-update

o Comsol MultiPhysics 5.1 Release Highlights
http://www.comsol.com/release/5.1

o Comsol Specification Chart
http://www.comsol.com/products/specifications/

o Comsol Events
https://www.comsol.com/events/

o Comsol Webinars
https://www.comsol.com/events/webinars/

o Compute Canada Comsol Wiki
https://docs.computecanada.ca/wiki/COMSOL

o How to Run Simulations in Batch Mode from the Command Line
https://www.comsol.com/blogs/how-to-run-simulations-in-batch-mode-from-the-command-line/