From Documentation
Jump to: navigation, search
(I am new to parallel programming, can you help me get started with my project?)
(Getting Help)
Line 41: Line 41:
 
* [mailto:accounts@computecanada.ca accounts@computecanada.ca] -- Questions about Compute Canada accounts
 
* [mailto:accounts@computecanada.ca accounts@computecanada.ca] -- Questions about Compute Canada accounts
 
* [mailto:renewals@computecanada.ca renewals@computecanada.ca] -- Questions about Compute Canada account renewals
 
* [mailto:renewals@computecanada.ca renewals@computecanada.ca] -- Questions about Compute Canada account renewals
 
==== Fall 2018 changes to the ticket system ====
 
 
In the Fall of 2018 SHARCNET has retired its separate ticket system.  Now all tickets are handled by the Compute Canada ticket system.  The <i>help@sharcnet.ca</i>  address will still work for ticket submission, it will just redirect the email to the Compute Canada ticket system.  The new system does not make its tickets visible to all users, so you cannot search all existing tickets via a web interface like you could in the old system.  Also, in the new system there is no web form to submit the ticket, instead it must always be done by email.  On the plus side, the new system is capable of handling file attachments.
 
 
=== How do I give other users access to my files ? ===
 
 
There are two ways to provide other users with access to your files.  The first is by changing the file attributes of your directories directly with the chmod command and the second is by using file access control lists (acl). Using ACLs is more flexible as it allows you to specify individual users and groups and their respective privileges, whereas using chmod is more coarse grained and only allows you to set the permissions for your group and global access.
 
 
==== Enabling Per-user/group Access: chmod Method ====
 
 
Suppose you have a program and some files in:
 
 
<pre>/home/account/research/projectx</pre>
 
 
that you want to provide access to some users and/or groups.
 
 
The first step is to make the "top" directory you control have world execute permission. This will allow other users to be able to cd (change directory) into subdirectories under such. You only need to have world execute permission; world read permission is not needed. (Enabling world read permission will allow anyone to see all file and subdirectory names in that directory --so you may wish to keep / turn such off.)
 
 
The "top" directories you control on SHARCNET are these (where $USER is your Compute Canada userid):
 
 
* /home/$USER
 
* /scratch/$USER
 
 
(Please note that project file space is special in that the top project directory is owned by the PI of the research group. By default all group members have read access to the contents of their project directory.)
 
 
So if you want to provide access to the directory:
 
 
<pre>/home/$USER/research/projectx</pre>
 
 
you would run (once) the following command:
 
 
<pre>chmod o+x /home/$USER</pre>
 
 
or equivalently (since this is your home directory):
 
 
<pre>chmod o+x ~</pre>
 
 
If you also want to be sure others cannot see the files and subdirectories in your home directory, then add a -rw (which turns off the ability to read the directory contents or the ability to write into and delete directory contents) to the chmod command as follows:
 
 
<pre>chmod o+x-rw /home/$USER</pre>
 
 
Similarly, you can set your "top" scratch directory if that is where you want to provide access, like this:
 
 
<pre>chmod o+x /scratch/$USER</pre>
 
 
or with the added -rw as follows:
 
 
<pre>chmod o+x-rw /scratch/$USER</pre>
 
 
'''NOTE:''' If unsure and you want to err on the side of keeping things private, use "chmod o+x-rw DIRECTORY_NAME".
 
 
Now repeat this process for all directories in the path you want to provide access to except the last one. For example, to provide access to this projectx directory:
 
 
<pre>/home/$USER/research/projectx</pre>
 
 
you would need to run:
 
 
<pre>chmod o+x /home/$USER</pre>
 
<pre>chmod o+x /home/$USER/research</pre>
 
 
For the last directory, i.e., projectx, provide both read and execute permissions, and, if you want to allow others to write to that directory, also allow write permission.
 
 
To only provide read and execute permission with the last directory, run:
 
 
<pre>chmod o+rx /home/$USER/research/projectx</pre>
 
 
and to provide read, write, and execute permission, run:
 
 
<pre>chmod o+rwx /home/$USER/research/projectx</pre>
 
 
More realistically however, you would like others to be able to do either of the following:
 
 
# read everything in the projectx directory (and disallow others' ability to write/update/delete), or,
 
# read everything and be able to modify/write contents within the projectx directory.
 
 
To do the former (i.e., Item 1), run on the last directory (i.e., "projectx" in this example):
 
 
<pre>chmod -R o+rX-w /home/$USER/research/projectx</pre>
 
 
and to do the latter (i.e., Iterm 2), run on the last directory (i.e., "projectx" in this example):
 
 
<pre>chmod -R o+rwX /home/$USER/research/projectx</pre>
 
 
Now, you can tell the users you want to be able to access this directory its FULL PATH, i.e.,
 
 
<pre>/home/yourloginname/research/projectx</pre>
 
 
and those users will be able to run:
 
 
<pre>cd /home/yourlogginname/research/projectx</pre>
 
 
to have the access you've granted. (Don't tell the user a path with $USER in it --that won't work: you must use the full path. If you are unsure, "cd" to that directory and run the "pwd" command which will output the full path to the "present working directory".)
 
 
'''NOTE:''' The "other" permission settings in this section allow ANY other user actions permitted implied by the permissions you've set. If this is too open, then read the sections below that use the setfacl command.
 
 
==== Disabling Per-user/group Access: chmod Method ====
 
 
At some point, you will want to revoke permissions granted to others. If you had previously provided access to your "projectx" directory using these commands:
 
 
<pre>chmod o+x /home/$USER
 
chmod o+x /home/$USER/research
 
chmod -R o+rwX /home/$USER/research/projectx</pre>
 
 
then you would revoke access using:
 
 
<pre>chmod o-rwx /home/$USER
 
chmod o-rwx /home/$USER/research
 
chmod -R o-rwx /home/$USER/research/projectx</pre>
 
 
Know that this will revoke all "other" access. If you have other users using other directories under /home, then you will not want to run:
 
 
<pre>chmod o-x /home/$USER</pre>
 
 
as that will prevent those users from accessing those other directories.
 
 
==== Controlling Access to Files/Directories Using setfacl ====
 
 
An Access Control List (ACL) is a list of users and groups with their associated file access privileges which is associated with a file/directory. Using ACLs allow fine-grained control over which users and/or which groups of users can access files and/or directories.
 
 
'''NOTE:''' If you are granting access to multiple SHARCNET staff, you may prefer to grant access to all SHARCNET staff at one time, e.g., you may/will be/are receiving assistance from multiple staff members. If so, you may find it much easier to grant access to the '''sn_staff''' group instead of each individual SHARCNET staff. (Similarly, "cc_staff" group can be used to provide access to all Compute Canada staff.)
 
 
 
===== Enabling Per-user/group Access: setfacl Method =====
 
 
Although you can use the setfacl command to grant permissions everywhere needed, it is simpler to use the chmod command to set execute permissions on your "top" directory and all directories below the "top" one first. Suppose you want to grant access to the following "projectx" directory (to everything in and under it):
 
 
<pre>/home/$USER/research/projectx</pre>
 
 
where $USER is your userid (i.e., Compute Canada login). The "top" directory is:
 
 
<pre>/home/$USER</pre>
 
 
so you would run to give others execute permission to it:
 
 
<pre>chmod o+x /home/$USER</pre>
 
 
If you prefer giving only a specific user, called USERNAME, access then use setfacl to do this instead:
 
 
<pre>setfacl -m u:USERNAME:x /home/$USER</pre>
 
 
Notice the 'u' before "u:USERNAME". The 'u' means "user" and replace USERNAME with the user's name.
 
 
If you want to only give a specific group, e.g., sn_staff, access then use setfacl as follows:
 
 
<pre>setfacl -m g:sn_staff:x /home/$USER</pre>
 
 
Notice the 'g' before "g:sn_staff". The 'g' means "group" and replace "sn_staff" with the name of the group you want to provide access to.
 
 
Similarly, you will want to provide access to the directories under the "top" one except the last one. If you wanted to grant access to your "projectx" directory located here:
 
 
<pre>/home/$USER/research/projectx</pre>
 
 
then you will need to grant execute permission to both /home/$USER and /home/$USER/research, e.g.,
 
 
<pre>chmod o+x /home/$USER
 
chmod o+x /home/$USER/research</pre>
 
 
or use setfacl to do the same for some USERNAME:
 
 
<pre>setfacl -m u:USERNAME:x /home/$USER
 
setfacl -m u:USERNAME:x /home/$USER/research</pre>
 
 
or some group (e.g., sn_staff):
 
 
<pre>setfacl -m g:sn_staff:x /home/$USER
 
setfacl -m g:sn_staff:x /home/$USER/research</pre>
 
 
With the last directory, you will want to either grant to all content within that directory:
 
 
# read and execute permissions without the ability to modify/write,
 
# read, write, and execute permissions.
 
 
To do Item 1 (i.e., grant read and execute but no write) with setfacl for some user name to the "projectx" directory:
 
 
<pre>setfacl -R -m u:USERNAME:rwX /home/$USER/research/projectx</pre>
 
 
and for some group, e.g., sn_staff, one would write:
 
 
<pre>setfacl -R -m g:sn_staff:rwX /home/$USER/research/projectx</pre>
 
 
Now, you can tell the users you want to be able to access this directory its FULL PATH, i.e.,
 
 
<pre>/home/yourloginname/research/projectx</pre>
 
 
and those users will be able to run:
 
 
<pre>cd /home/yourlogginname/research/projectx</pre>
 
 
to have the access you've granted. (Don't tell the user a path with $USER in it -- that won't work: you must use the full path. If you are unsure, "cd" to that directory and run the "pwd" command which will output the full path to the "present working directory".)
 
 
 
===== Disabling Per-user/group Access: setfacl Method =====
 
 
At some point, you will want to revoke permissions granted to others. If you had previously provided access to your "projectx" directory using these commands for a directory:
 
 
<pre>chmod o+x /home/$USER
 
chmod o+x /home/$USER/research</pre>
 
 
then you would revoke access using:
 
 
<pre>chmod o-rwx /home/$USER
 
chmod o-rwx /home/$USER/research</pre>
 
 
Know that this will revoke all "other" users' access through these directories.
 
 
If you used setfacl, then run the same command you previously used but replace -m with -x.
 
 
For example, if you <I><b>granted</b></I> permissions using:
 
 
<pre>setfacl -m u:USERNAME:x /home/$USER
 
setfacl -m u:USERNAME:x /home/$USER/research
 
setfacl -R -m u:USERNAME:rwX /home/$USER/research/projectx</pre>
 
 
or:
 
 
<pre>setfacl -m g:sn_staff:x /home/$USER
 
setfacl -m g:sn_staff:x /home/$USER/research</pre>
 
 
then you would <I><b>revoke</b></I> these permissions using (respectively):
 
 
<pre>setfacl -x u:USERNAME /home/$USER
 
setfacl -x u:USERNAME /home/$USER/research
 
setfacl -R -x u:USERNAME /home/$USER/research/projectx</pre>
 
 
or:
 
 
<pre>setfacl -x g:sn_staff /home/$USER
 
setfacl -x g:sn_staff /home/$USER/research
 
setfacl -R -x g:sn_staff /home/$USER/research/projectx</pre>
 
 
You can verify that user's don't have access using getfacl.
 
 
==== A Brief Overview of the getfacl and setfacl Commands ====
 
 
One can see the ACL for a particular file/directory with the '''getfacl''' command, eg.
 
 
 
$ getfacl /home/sn_user
 
getfacl: Removing leading '/' from absolute path names
 
# file: home/sn_user
 
# owner: sn_user
 
# group: sn_user
 
user::rwx
 
group::r-x
 
other::--x
 
 
One uses the '''setfacl''' command to modify the ACL for a file/directory.  To add read and execute permissions for this directory for user ricky, eg.
 
 
$ setfacl -m u:ricky:rx /home/sn_user
 
 
Now there is an entry for ''user:ricky'' with ''r-x'' permissions:
 
 
$ getfacl /home/sn_user
 
getfacl: Removing leading '/' from absolute path names
 
# file: home/sn_user
 
# owner: sn_user
 
# group: sn_user
 
user::rwx
 
user:ricky:r-x
 
group::r-x
 
mask::r-x
 
other::--x
 
 
To remove an ACL entry one uses the '''setfacl''' command with the '''-x''' argument, eg.
 
 
$ setfacl -x u:ricky /home/sn_user
 
 
Now there is no longer an entry for ''ricky'':
 
 
$ getfacl /home/sn_user
 
getfacl: Removing leading '/' from absolute path names
 
# file: home/sn_user
 
# owner: sn_user
 
# group: sn_user
 
user::rwx
 
group::r-x
 
mask::r-x
 
other::--x
 
 
Note that if one wants to provide access to a nested directory then the permissions need to be changed on all the parent directories using the ''-R'' flag.  Please see the man pages for these commands ''man getfacl; man setfacl'' for further information. If you'd like help utilizing ACLs please [mailto:help@sharcnet.ca email help@sharcnet.ca].
 
  
 
=== I am new to parallel programming, where can I find quick references at SHARCNET? ===
 
=== I am new to parallel programming, where can I find quick references at SHARCNET? ===
Line 333: Line 53:
 
=== I am in a process of purchasing computer equipment for my research, would you be able to provide technical advice on that? ===
 
=== I am in a process of purchasing computer equipment for my research, would you be able to provide technical advice on that? ===
 
If you tell us what you want, we may be able to help you out.
 
If you tell us what you want, we may be able to help you out.
 
=== Does SHARCNET have a mailing list or user group? ===
 
Yes. You may subscribe to one or more mailing lists on the email list page available once you log into the web portal. To find it, please go to MyAccount - Settings - Details in the menu bar on the left and then click on Mail on the "details" page. Don't forget to save your selections.
 
 
=== How do I add/remove myself to/from a SHARCNET mailing list? ===
 
To add/remove yourself to/from a SHARCNET mailing list, do the following:
 
 
# Log in to the SHARCNET portal: https://www.sharcnet.ca/
 
# Click on the '''My Account''' menu item.
 
# Click on the '''Settings''' menu item under '''My Account'''.
 
# Click on the '''Details''' menu item under '''Settings'''.
 
# Click on the '''Mail''' link near the bottom of the page.
 
 
The page that appears has checkboxes that allow you to add/remove yourself to/from a SHARCNET mailing list. To add yourself, click the checkbox on the line of the mailing list you are interested in; to remove yourself, uncheck the checkbox instead. Finally, when done, be sure to click the '''Save''' button at the bottom of the page to record these changes.
 
  
 
=== Does SHARCNET provide any training on programming and using the systems? ===
 
=== Does SHARCNET provide any training on programming and using the systems? ===

Revision as of 13:26, 10 May 2019


Getting Help

I have encountered a problem while using a Compute Canada/SHARCNET system and need help, who should I talk to?

If you have access to the Internet, we encourage you to use the problem ticketing system (described in detail below) . This is the most efficient way of reporting a problem as it minimizes email traffic and will likely result in you receiving a faster response than through other channels.

You are also welcome to contact system administrators and/or high performance technical computing consultants at any time. You may find their contact information on the directory page.

How long should I expect to wait for support?

Unfortunately Compute Canada/SHARCNET does not have adequate funding to provide support 24 hours a day, 7 days a week. User support and system monitoring is limited to regular business hours: there is no official support on weekends or holidays, or outside 9:00 - 17:00 EST .

Please note that this includes monitoring of our systems and operations, so typically when there are problems overnight or on weekends/holidays system notices will not be posted until the next business day.

Compute Canada Problem Ticket System

What is a "problem ticket system"?

This is a system that allows anyone with a Compute Canada account to start a persistent email thread that is referred to as a "problem ticket". When a user submits a new ticket it will be brought to the attention of an appropriate and available Compute Canada/SHARCNET staff member for resolution.

You can interact with the ticket system entirely via email. There is also a web interface to see tickets you have submitted in the past.

What do I need to specify in a ticket ?

To help us address your question faster, please try to do the following when submitting a ticket:

  1. specify which of our systems is involved
  2. if the problem pertains to a job, then report the jobid associated with the job; this is an integer that is returned by the scheduler when you submit the job
  3. report the exact commands necessary to duplicate the problem, as well as any error output that helps identify the problem; if relevant, this should include how the code is compiled, how the job is submitted, and/or anything else you are doing from the command line relating to the problem
  4. if you'd like for a particular staff member to be aware of the ticket, mention them

How do I submit a ticket?

In general, you can submit a new ticket by emailing support@computecanada.ca with the email address associated with your Compute Canada account. If you are using another email address, please provide your full name, your Compute Canada default username (if available) and your university or institution.

If you like, you can also target your inquiry more specifically, by using the following addresses to submit your ticket:

I am new to parallel programming, where can I find quick references at SHARCNET?

SHARCNET has a number of training modules on parallel programming using MPI, OpenMP, pthreads and other frameworks. Each of these modules has working examples that are designed to be easy to understand while illustrating basic concepts. You may find these along with copies of slides from related presentations and links to external resources on the Main Page of this training/help site.

I am new to parallel programming, can you help me get started with my project?

Absolutely. We will be glad to help you from planning the project, architecting your application programs with appropriate algorithms and choosing efficient tools to solve associated numerical problems to debugging and analyzing your code. We will do our best to help you speed up research. If your programming project would involve a significant staff time, you should consider applying for Dedicated Programming support. (We run the competition annually; see https://www.sharcnet.ca/my/research/programming).

Can you install a package on a cluster for me?

Certainly. We suggest you make the request by sending e-mail to help@sharcnet.ca with the specific request.

I am in a process of purchasing computer equipment for my research, would you be able to provide technical advice on that?

If you tell us what you want, we may be able to help you out.

Does SHARCNET provide any training on programming and using the systems?

Yes. SHARCNET provides workshops on specific topics from time to time and offers courses at some sites. Every summer (usually late May to early June), SHARCNET holds an annual HPC Summer School with a variety of in-depth, hands-on workshops. Many materials from past workshops/presentations can be found on the SHARCNET's web portal.

SHARCNET also offers a series of online seminars (so-called "General interest webinars"), typically delivered every second Wednesday at lunch time. These are announced via the SHARCNET events mailing list and one can see the schedule at the SHARCNET event calendar. Past seminars are recorded and posted on our youtube channel. A full listing of the past webinars is available on the Online Seminars page.

Attending SHARCNET Webinars

SHARCNET makes a number of seminar events available online (New User Webinar, general interest talks, etc.) using two platforms - Vidyo and Zoom. For now only the bi-weekly General Interest webinars are delivered via Zoom, but we will be moving our weekly New User Webinars to Zoom as well. Both platforms can be used by either installing an app (available for Windows, Mac, Linux etc), or running it in a browser (no installation required). At this point we do not require creation of an account for Zoom, but we might enforce it in the future, so it is a good idea to create a free Zoom account now.

Please note that if your device has a microphone (highly recommended) and/or webcam, they will be used by Vidyo/Zoom to transmit your audio and video to all seminar participants. They are normally off by default, but you can enable them by clicking on a corresponding button at the bottom of your Vidyo/Zoom window - but only when allowed by the host. Normally we do not allow attendees to use their mikes during the webinar, but we have special time for questions and answers at the end of each webinar when the mikes can be enabled. Generally, please keep your mike muted (and webcam disabled) unless you want to ask a question.

We normally record our seminars, and make them available to all SHARCNET users. All recent and new webinars are posted on our youtube channel, http://youtube.sharcnet.ca . The links to the video recordings, slides and abstracts can be found on our online seminars page.

Please note that times for our webinars are for the Eastern Time (EST/EDT) zone.