From Documentation
Jump to: navigation, search

Torch is a scientific computing framework with wide support for machine learning algorithms. It is easy to use and efficient, thanks to an easy and fast scripting language, LuaJIT, and an underlying C/CUDA implementation.

PyTorch on Graham

PyTorch is installed on Graham cluster. Please reference PyTorch on Graham

Running Torch7(master branch Oct14, 2016) on Mosaic and Copper

A Torch7 is installed under /opt/sharcnet/testing. To run the torch, you should set the environment paths first using the following command:

source /opt/sharcnet/testing/torch/distro/install/bin/torch-activate

Then you should be able to submit jobs:

sqsub -q gpu --mpp=10g --gpp=1 -r 1h -o test-torch.out th run.lua

You can change the mpp according to your model size. If using CPU operations, you can enable multithread by add flag "-f threaded -n <NUM_THREADS>" to sqsub command.

Torch7 (master branch Oct 14th, 2016) Installation Guide

To install Torch, you should firstly stay on login node and load it from git:

[feimao@mos-login torch]$ git clone https://github.com/torch/distro.git --recursive

You should stay in the login node and change the module and export the proper environment path:

module unload intel mkl openmpi
module load intel/15.0.3
module load hdf/serial/5.1.8.11 
module unload cuda
module load cuda/7.5.18 
module load python/intel/2.7.10
module load cmake/3.4.3
export CC=gcc
export CXX=g++
export CMAKE_INCLUDE_PATH=/opt/sharcnet/intel/15.0.3/mkl/include/:$CMAKE_INCLUDE_PATH
export CMAKE_LIBRARY_PATH=/opt/sharcnet/intel/15.0.3/mkl/lib/intel64/:/opt/sharcnet/intel/15.0.3/lib/intel64/:$CMAKE_LIBRARY_PATH
export PATH=$CMAKE_INCLUDE_PATH:$PATH
export LD_LIBRARY_PATH=/opt/sharcnet/testing/cudnn/cudnn5:$CMAKE_LIBRARY_PATH:$LD_LIBRARY_PATH

Then go inside the torch folder you downloaded from git and run:

./clean.sh && ./install.sh

After install all the components, you will be asked if to put Torch in your .bashrc setting, please input NO so that you won't break the environment path on other cluster than Mosaic.

Do you want to automatically prepend the Torch install location
to PATH and LD_LIBRARY_PATH in your /home/<your_username>/.bashrc? (yes/no)
[yes] >>> 
no

To set the environment for torch, you should run:

module unload intel mkl openmpi
module load intel/15.0.3
module load hdf/serial/5.1.8.11 
module unload cuda
module load cuda/7.5.18 
module load python/intel/2.7.10
export LD_LIBRARY_PATH=/opt/sharcnet/testing/cudnn/cudnn5:$LD_LIBRARY_PATH
source /work/your_username/torch/distro/install/bin/torch-activate
  • Running "source /work/your_username/torch/distro/install/bin/torch-activate" will break the environment setting on other cluster. Please don't put it into ~/.bashrc file.

Using luarocks to install packages

You can use luarocks to install packages you need. To check the current intstalled package:

luarocks list

To install a package, you should stay on login node to have internet access:

luarocks install <package_name>