From Documentation
Jump to: navigation, search
m (CUDA on Graham and Cedar)
Line 1: Line 1:

Latest revision as of 10:07, 6 June 2019

This page is scheduled for deletion because it is either redundant with information available on the CC wiki, or the software is no longer supported.
Description: NVIDIA Cuda Toolkit for GPGPU computation
SHARCNET Package information: see CUDA software page in web portal
Full list of SHARCNET supported software

Please also see the page dedicated to CUDA on Compute Canada documentation wiki.

CUDA on Graham and Cedar

Multiple versions of CUDA installed on Graham and Cedar. To use version 8.0.44 for example, load the module:

module load cuda/8.0.44

This will make available all the other modules compiled with GPU support.

CUDA versions on SHARCNET clusters

Please note that the default version of CUDA on the mosaic and copper clusters is at present cuda/7.5.18, with older versions also available. Should you need them, you can switch from the default to, for example, 4.1 module by executing:

module switch cuda/4.1

However, we strongly advise using the default version to ensure full compatibility with the installed driver.

Note - Please compile all your CUDA programs on the SHARCNET system you plan to use, to ensure the required libraries are found. Do not try to use a CUDA executable compiled outside of SHARCNET as it will not work. Please make sure that the "CUDA" module version you want is loaded before you compile a CUDA program by running "module list".

Important point about compile flags

To generate efficient code, the nvcc compiler should be targeted at the architecture of the system. Use the appropriate -arch flag when using nvcc compiler.

Example Programs

Note: The compiled executables will not run on the login node of a cluster as there is no GPU installed on it.

CUDA comes with a set of useful examples. Build the examples in your own user space by copying the whole gpucomputingsdk tree, which will require about 500 MB of space. This build process will create all the libraries needed by the SDK programs.

cp -R /opt/sharcnet/cuda/7.5.18/samples/ /work/$USER/gpucomputingsdk
cd /work/$USER/gpucomputingsdk

The executables will be found in "bin" directories generated by the make process.

GPU Hardware Information

To extract detailed information about gpu hardware and associated cuda software driver components installed on cluster compute nodes submit the following job:

sqsub -r 10m -q gpu --gpp 1 -o CUDA_DQ_TEST deviceQuery

It is recommended that users consult the SHARCNET GPU_Accelerated_Computing online reference even if they are familiar with CUDA, since it includes information on compiling hybrid parallel (MPI/threads/CUDA) programs and other SHARCNET-specifics not addressed in the official CUDA documentation.

Debugging CUDA Programs

The CUDA package provides a debugger called cuda-gdb, which works much like standard GNU gdb and can be used to debug CUDA kernels. Instructions on its use are available here. Our commercial parallel debugger DDT can debug CUDA programs, including mixed MPI/CUDA and threads/CUDA.

Profiling CUDA programs

CUDA comes with a profiler called nvprof. If the CUDA module is loaded, it should be in the path.

Using CUDA in Python with PyCUDA

Detailed instructions provided on separate PyCUDA page.


o NVIDIA GPU Computing Developer Home Page

o CUDA section in the SHARCNET GPU Computing Online Reference / Tutorial

o Online tutorials from NVIDIA (some free)

o Whats New in Cuda7