| This page is scheduled for deletion because it is either redundant with information available on the CC wiki, or the software is no longer supported. |
Contents
| Globus |
|---|
| Description: Compute Canada portal for intuitive, fast, secure and reliable data transfer. |
| SHARCNET Package information: see Globus software page in web portal |
| Full list of SHARCNET supported software |
Introduction
The Compute Canada Globus instance is an intuitive portal that provides fast, secure and reliable data transfers with the goal of making big data portable. Most Compute Canada storage systems are defined as endpoints for transfers and user machines can be added as endpoints. Most SHARCNET file systems can be found via the computcanada#sharcnet-dtn1 endpoint. General information regarding the Compute Canada Globus implementation can be found at the Compute Canada Globus Wiki page, and more general information about Globus can be found at the Globus site.
Usage
Login and select Endpoints
Rather than using this utility from a SHARCNET system, Globus is accessed via its Compute Canada Portal. For up-to-date details and instructions for signing into the Compute Canada Globus Portal with your Compute Canada Data Base (CCDB) credentials visit the Compute Canada Globus Wiki.
Once that you are signed into the Globus Portal you will need to select and activate the computecanada#sharcnet-dtn1 Endpoint (using your SHARCNET credentials) to access your files on SHARCNET systems (See image below).
To access the storage on the new Compute Canada clusters, graham and cedar, select endpoints computecanada#cedar-dtn and computecanada#graham-dtn.
For information about managing Globus Endpoints and transferring from a personal computer see the Personal Computers section of the Compute Canada Globus Wiki page.
Example Transfer
Once that two endpoints have been activated transferring files with the Globus Portal is simply a metter of selecting a file or folder from one end point and moving it to the selected Path at the other Endpoint. In the image below the user is sending the "batch" folder from the Path "/~/Downloads/batch_context-master" on their local computer ("jad_lem" Endpoint) to their scratch folder at Orca via the Data Transfer Node (Endpoint = "computecanada#sharcnet-dtn1" and Path ="/scratch/orca/scratch/jdesjard").
General Notes
computecanada#sharcnet-dtn1 Endpoint folder layout
By default when you activate your computecanada#sharcnet-dtn1 Enpoint the Path will point to your /home directory ("/~/" or /home/username"). This will provide access to your global access /home folder on SHARCNET systems. Similarly, in order to access your globa access /work folder you can change the computecanada#sharcnet-dtn1 Endpoint Path to "/work/username" (this also applies to the "/freezer" global access file system).
For the non-global /scratch file systems the computecanada#sharcnet-dtn1 Endpoint Path needs to specify the desired systems. Available /scratch Paths are:
- /scratch/angel/scratch/username
- /scratch/dusky/username
- /scratch/goblin/username
- /scratch/iqaluk/username
- /scratch/monk/username
- /scratch/orca/scratch/username
- /scratch/redfin/username
- /scratch/saw/scratch/username
TAR file efficiency
As with all data transfer methods it is important to recognize the difference in transfer rates between sending a few large files and sending many small files. For example, the difference between transferring 1TB in the form of a single file and transferring the same 1TB in the form of 100,000 small files can be substantial. If you intend on sending a large number of small files it is important to consider encapsulating the files using utilities such as TAR and DAR. This can substantially improve the completion time of your transfer and contribute to the overall performance of the resource.