From Documentation
Revision as of 11:44, 1 March 2016 by Edward (Talk | contribs) (Further Reading)

Jump to: navigation, search


This self-directed tutorial is mainly for people who have none or very little experience in Linux system, particularly for Windows users who want to migrate to SHARCNET Linux systems. Examples shown in the lightgrey areas were run on 'narwhal', a SHARCNET cluster. We hope that this tutorial would be a good starting point for newcomers to Linux.

What is Linux?

Linux is a free Unix-type operating system originally created by Linus Torvalds with the assistance of developers around the world in the 90's. Linux is becoming popular as a powerful, low-cost operating system for cluster servers.

Linux Features

  • multitasking: several programs running at once
  • multiusers: several users at the same machine at once
  • multiplatforms: it runs on many different CPU

Directory Structure in Linux

This is not a complete list, but the most general directories on Linux system.

Directory Meaning
/bin Directory for system command
/dev Directory with special files which enable to work with pheripheral devices
/etc System programs and data
/lib Libraries that are needed to execute the binaries in /bin/ and /sbin/
/mnt Directory for temporarily mounted file systems
/tmp Directory for temporary data sets
/usr Other system programs
/var Files which are being updated during system running
/opt Directory for large, static application software packages
/home User's home directory (quota of 10GB). User has a common home directory on SHARCNET systems
/work work directory, with a quota of 1TB on some large SHARCNET systems
/scratch scratch directory on SHARCNET systems with an expiry time of 2 months

Getting Started

Let's try a few simple commands

  • login
    • Use SSH (see our SSH page) to login onto SHARCNET linux systems
  • hostname: Print system name
[jemmyhu@saw-login2: ~]$ hostname
  • who: Print all usernames currently logged in
[jemmyhu@saw-login2: ~]$ who
inta     pts/0        Nov 29 12:10 (
jemmyhu  pts/1        Nov 29 09:33 (
jemmyhu  pts/2        Nov 29 09:34 (
tomek    pts/3        Nov 19 10:00 (
merz     pts/5        Nov 29 12:52 (nar317)
michaelg pts/4        Nov  2 12:47 (
viv      pts/6        Nov 26 19:12 (
mhashimo pts/7        Nov 26 17:21 (
msammalk pts/8        Nov 29 11:13 (
sclee    pts/9        Nov 29 14:37 (
merz     pts/10       Nov 29 14:41 (
mjhillie pts/12       Nov 27 10:56 (
zzheng   pts/14       Nov 27 11:49 (meg34.megaladon.sharcnet)
  • whoami: Print the current username
[jemmyhu@saw-login2: ~]$ whoami
  • pwd: Print working (current) directory
[jemmyhu@saw-login2: ~]$ pwd
  • date: Display the date & time
[jemmyhu@saw-login2: ~]$ date
Mon Feb  3 10:06:09 EST 2014
  • exit: or logout to leave the system

Changing Password

To change your Linux account password, use the command passwd

[jemmyhu@saw-login2: ~]$ passwd
Changing password for user jemmyhu.
Changing password for jemmyhu
(current) UNIX password:
(You will write your actual password after colon.)
New password:
(You will write your new password after colon.)
Re-type new password:
(You will repeat your new password after colon.)
Password changed

if you type your old password and a new password correctly, the new one becomes valid.

NOTE: you can change your SHARCNET password on SHARCNET webportal.

Basic Linux Commands

The basic format of a command is command [option] ..., please check the command's manpage (man command) for other options.

ls - list the contents of a directory

ls displays the names of all files and subdirectories in the working (current) directory.

[jemmyhu@saw-login2: ~]$ ls
adf2007.01                           Dakota     mpi              script         
adf2007.01.pc64_linux.hpmpi.bin.tar  debugger   Octave_matlab    test_g03
awk_example                          fftw_test  OpenMP_Summer07  test_mpiPWSCF
bin                                  HU         overLord         util
Courses                              Matlab     scratch          work

ls -l shows the detailed information of the contents (permissions, owners, size, and timestamp).

[jemmyhu@saw-login2: ~]$ ls -l
total 352192
drwxr-xr-x   8 jemmyhu jemmyhu      4096 Nov 23 09:47 adf2007.01
-rw-rw-r--   1 jemmyhu jemmyhu 360212480 Nov 12 13:10 adf2007.01.pc64_linux.hpmpi.bin.tar
drwxr-x---   2 jemmyhu jemmyhu      4096 Apr 30  2007 awk_example
drwxr-xr-x   2 jemmyhu jemmyhu      4096 Nov  9 12:14 bin
drwx------   3 jemmyhu jemmyhu      4096 Apr 12  2007 Courses
drwxr-xr-x   4 jemmyhu jemmyhu      4096 May 29  2007 Dakota
drwx------   3 jemmyhu jemmyhu      4096 Aug 22 15:39 debugger
drwxr-x---   3 jemmyhu jemmyhu      4096 May  7  2007 fftw_test
drwx------  13 jemmyhu jemmyhu      4096 Oct 31 12:08 HU
drwxrwxr-x   2 jemmyhu jemmyhu      4096 Oct 31 11:52 Matlab
drwx------   3 jemmyhu jemmyhu      4096 Oct 30 10:57 mpi
drwx------   2 jemmyhu jemmyhu      4096 Aug 24 15:31 Octave_matlab
drwx------   6 jemmyhu jemmyhu      4096 Jun 13 11:06 OpenMP_Summer07
drwx------   2 jemmyhu jemmyhu      4096 Oct 31 11:54 overLord
lrwxrwxrwx   1 jemmyhu jemmyhu        16 Aug  1 11:22 scratch -> /scratch/jemmyhu
drwx------   2 jemmyhu jemmyhu      4096 Nov 21 12:14 script
drwx------   2 jemmyhu jemmyhu      4096 Nov  6 15:29 test_g03
drwx------   7 jemmyhu jemmyhu      4096 Jun  7 21:34 test_mpiPWSCF
drwx------   2 jemmyhu jemmyhu      4096 Oct 31 11:58 util
lrwxrwxrwx   1 jemmyhu jemmyhu        13 Aug  1 11:21 work -> /work/jemmyhu

ls -a shows all subdirectories and files, even files that are hidden (these files begin with a dot.)

[jemmyhu@saw-login2: ~]$ ls -a
.                                    Courses          .octave_hist     test_g03             
..                                   Dakota           Octave_matlab    test_mpiPWSCF 
adf2007.01                           .ddt             OpenMP_Summer07  util             
adf2007.01.pc64_linux.hpmpi.bin.tar  .ddt_session     .pathscale       .viminfz.tmp 
.assistant                           debugger         .rpmmacros       work
awk_example                          fftw_test        scratch          .Xauthority
.bash_history                        HU               script    
.bash_logout                         .lsbatch         .sq            
.bash_profile                        Matlab           .ssh         
.bashrc                              mpi              test_g03
bin                                  .nodelist        test_mpiPWSCF

mkdir - create directory (i.e., a folder)

mkdir my_dir

my_dir is the name of a newly created directory, a subdirectory of the current directory.

rmdir - remove empty directory

rmdir my_dir

remove my_dir if it is empty. If my_dir is not empty, use

rm -r my_dir

cd - change directory

cd change to your home directory
cd .. change to the parent directory
cd directory change to directory
[jemmyhu@saw-login2: ~]$ cd 
[jemmyhu@saw-login2: ~]$ pwd
[jemmyhu@saw-login2: ~]$ cd ..
[jemmyhu@saw-login2:/home]$ pwd
[jemmyhu@saw-login2:/home]$ cd jemmyhu
[jemmyhu@saw-login2: ~]$ pwd

more, less - display contents of file, page by page

more file less file

User can controle the output:

  • press space the next screen is displayed
  • press enter the next row is displayed
  • press q the command is finished

cp - copy file

cp file NEWFILE Copy file to NEWFILE in the same directory.
cp /home/jemmyhu/file . Copy file from /home/jemmyhu to the current directory.
cp -r DIR NEWDIR Copy DIR and all its contents to NEWDIR.

mv - move or rename file or directory

mv File NEWNAME Rename FILE to NEWNAME in the same directory.
mv File DIR Move FILE to existing directory DIR.

rm - remove file

rm FILE Remove FILE in the current directory.
rm b*p Remove all files beginning with b and ending with p in the current directory.
rm * Remove all files in the current directory
rm -i * the system will ask for permission before removing each file.
rm -r DIR Remove DIR and its entire contents.

file - determine file type

[jemmyhu@saw-login2:/work]$ file README
README: ASCII English text

diff - display the differences between two files

diff File_1 File_2 Show differences between two versions of a file.

find - search for files that meet a desired criteria

find FILE(S) Search current dir and its subdirs for FILE(S).
find /usr -name "lpr" -print Find a file lpr in directory /usr and its subdirectories.
[jemmyhu@saw-login2: ~]$ find /usr -name "lpr" -print

ps - list processes

ps -u username List all your processes.
[jemmyhu@saw-login2: ~]$ ps -u jemmyhu
PID   TTY       TIME    CMD
11437 ?        00:00:00 sshd
11438 pts/5    00:00:00 bash
13861 pts/5    00:00:00 ps

kill - stop a process from running

kill -9 PID Stop a process with process NO. PID before it has been finished normally.

time - measure program running time

time executablecode Display the time used to run executablecode.
[jemmyhu@saw-login2: ~]$ time ./pi-serial
PI = 3.141593

real 0m0.002s
user 0m0.001s
sys 0m0.002s

top - list processes running on the system

[jemmyhu@saw-login2: ~]$ top

top - 10:09:50 up 8 days, 15:27, 16 users, load average: 0.01, 0.06, 0.36 Tasks: 472 total, 1 running, 469 sleeping, 2 stopped, 0 zombie Cpu(s): 0.2%us, 0.2%sy, 0.0%ni, 99.5%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 16463412k total, 14773748k used, 1689664k free, 6864k buffers Swap: 1999988k total, 0k used, 1999988k free, 11396408k cached

 PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                 

24801 viwanj14 20 0 119m 4720 1172 S 1.3 0.0 1:34.14 sshd 21032 jemmyhu 20 0 26180 1844 1204 R 0.7 0.0 0:00.14 top 22298 viwanj14 20 0 119m 4620 1172 S 0.7 0.0 2:44.65 sshd

1779 root      20   0     0    0    0 S  0.3  0.0  55:32.72 ptlrpcd                 
2158 nobody    20   0  332m 192m 1156 S  0.3  1.2  88:07.71 gmond                   

24806 viwanj14 20 0 38180 3424 1040 S 0.3 0.0 0:11.72 scp

   1 root      20   0 21436 1516 1196 S  0.0  0.0   0:01.51 init                    
   2 root      20   0     0    0    0 S  0.0  0.0   0:00.10 kthreadd

Enter 'q to stop 'top' command

which - locate a program file in user's path

[jemmyhu@saw-login2: ~]$ which cc
[jemmyhu@saw-login2: ~]$ which f90
[jemmyhu@saw-login2:~] which mpicc

man - help manual

[jemmyhu@saw-login2: ~]$ man sqjobs
SQJOBS(1)          User Contributed Perl Documentation             SQJOBS(1)

           sqjobs - show SQ jobs

           sqjobs [-r][-q][-z][-v][-u user][-n][--summary][jobid...]

           -a or --all show all jobs: all users and all states
           -r          show running jobs
           -q          show queued jobs
           -z          show suspended/preempted jobs
           -l          show more detail on jobs
           -L          show more detail on jobs
           -u user     show jobs for the given user
           -n, --none  show one-line summary of cluster
           -s or --summary   show a line-per-user summary of all jobs
           -h or --help        show usage
           --man               show man page
           jobid...    one or more jobids to examine

       sqjobs displays a succinct tabular listing of jobs in the SQ system.

Enter q to stop 'man' command

df - display free disk space

[jemmyhu@saw-login2: ~]$ df
Filesystem           1K-blocks      Used Available Use% Mounted on

rootfs 23441665792 6615963896 16825701896 29% /

                    23441665792 6615963896 16825701896  29% /

/dev/ram 102400 57424 44976 57% /ram /dev/cciss/c0d0p2 139053828 165624 131824668 1% /local none 8228360 168 8228192 1% /dev tmpfs 8231704 0 8231704 0% /dev/shm saw-admin:/saw_local 54558912 48159232 3583488 94% /saw_local ......

du - display file/directory disk usage (in KB)

du -sh DIRECTORY Display only the total disk usage of DIRECTORY (in MB).
[jemmyhu@saw-login2: ~]$ pwd
[jemmyhu@saw-login2: ~]$ ls
adf2007.01                           Dakota     mpi              script        
adf2007.01.pc64_linux.hpmpi.bin.tar  debugger   Octave_matlab    test_g03
awk_example                          fftw_test  OpenMP_Summer07  test_mpiPWSCF
bin                                  HU         overLord         util 
Courses                              Matlab     scratch          work

[jemmyhu@saw-login2: ~]$ du mpi
3896    mpi/mpi-2/gulper
3944    mpi/mpi-2
4160    mpi

[jemmyhu@saw-login2: ~]$ du -sh mpi
4.1M    mpi

File Permissions

  • chmod - change file/directory access permissions
chmod who=permissions Filename Gives who the specified permissions for a given Filename.
chmod who=permissions DIRECTORY Gives who the specified permissions for a given DIRECTORY.
  • Operator = is used to set up permissions
  • To add permissions, use +
  • To take away permissions, use -

who is a list of letters that specifies to whom you are going to be giving permissions. These may be specified in any order.

Letter Meaning
u user who owns the file
g group the file belongs to
o other users
a all of the above (an abbreviation for ugo)

permissions are the same letters that you see in the directory listing (ls -l).

r permission to read the file
w permission to write (or delete) the file
x permission to execute the file


[jemmyhu@saw-login2:/work]$ ls -l README
-rw-------  1 jemmyhu jemmyhu 342 Nov  7 14:17 README

The letters written from the 2nd to the 10th column are for file permissions. The first three columns are permissions for the user, the columns 5, 6, 7 for the user's group, and the last three for other users.

In the above example, user (jemmyhu) has read and write permissions on file 'README', but no permissions for the group and others.

Here are some examples to change the permissions on file 'README':

[jemmyhu@saw-login2:/work]$ ls -l README         #View the original permissions on file 'README'
-rw-------  1 jemmyhu jemmyhu 342 Nov  7 14:17 README
[jemmyhu@saw-login2:/work]$ chmod ugo=rw README  #Give read and write permissions to all

[jemmyhu@saw-login2:/work]$ ls -l README
-rw-rw-rw-  1 jemmyhu jemmyhu 342 Nov  7 14:17 README

[jemmyhu@saw-login2:/work]$ chmod ug+x README    #Add execute permission to user and group
[jemmyhu@nar316 work]$ ls -l README
-rwxrwxrw-  1 jemmyhu jemmyhu 342 Nov  7 14:17 README

[jemmyhu@saw-login2:/work]$ chmod ug-x README    #Take away execute permission from user and group
[jemmyhu@saw-login2:/work]$ ls -l README
-rw-rw-rw-  1 jemmyhu jemmyhu 342 Nov  7 14:17 README

[jemmyhu@saw-login2:/work]$ chmod a-w README     #Take away write permission for all
[jemmyhu@nar316 work]$ ls -l README
-r--r--r--  1 jemmyhu jemmyhu 342 Nov  7 14:17 README

[jemmyhu@saw-login2:/work]$ chmod u+w README     #Add write permission to user
[jemmyhu@nar316 work]$ ls -l README
-rw-r--r--  1 jemmyhu jemmyhu 342 Nov  7 14:17 README

Other Important Linux Commands

  • alias - create an alias
  • awk - find and replace text
  • cat - display a short file, or used to append files
  • echo - display message on screen
  • env - environment variables
  • grep - search file(s) for lines that match a given pattern
  • gzip/gunzip - compress or decompress named file(s)
  • make - recompile a group of programs
  • quota - display disk usage and limits
  • tar - Tape ARchiver
  • touch - create an empty file or change file timestamps
  • wc - print byte, word, and line counts

For a list of Linux commands, please see Linux Bash Commands.

What is vi?

vi (visual editor) is the default editor that comes with the UNIX/Linux operating system. Alternate editors for UNIX environments include nano (formerly named pico) and emacs (for basic emacs commands, see:

The vi editor is a full screen editor and has two modes of operation:

  1. Command mode commands which cause action to be taken on the file, and
  2. Insert mode in which entered text is inserted into the file.

In the command mode, every character typed is a command that does something to the text file being edited; a character typed in the command mode may even cause the vi editor to enter the insert mode. In the insert mode, every character typed is added to the text in the file; pressing the Esc key turns off the Insert mode. While there are a number of vi commands, just a handful of these are usually sufficient for beginning vi users.

Both UNIX/Linux and vi are case-sensitive. Be sure not to use a capital letter in place of a lowercase letter; the results will not be what you expect.

Basic vi Commands

  • Start vi
vi Filename - Open Filename in command mode

If Filename exists, the file will be displayed on the screen. If Filename does not exist, then an empty file is created for entering text.

The screen looks like:

  • Exit vi

In the following, [Enter] means press Enter key after the keystrokes.

Keystrokes Meaning
:x [Enter] exit vi, save the modified file to file named
:wq [Enter] exit vi, save the modified file to file named
:q [Enter] exit vi
:q! [Enter] exit vi, without saving the modified file
  • Insert or Add Text

The following keystroke will lead to insert mode for inserting and adding text. Esc key must be pressed to terminate the insert mode.

Keystroke Meaning
i insert text before cursor, until Esc key is pressed
a append text after cursor, until Esc key is pressed
o open a new line below current line, until Esc key is pressed
  • Cursor Movement Commands

In the command mode, move along the cursor on the screen.

Keystroke Meaning
h Cursor is moved one space to the left
l Cursor is moved one space to the right
j Cursor is moved one line down
k Cursor is moved one line up
  • Change Text
Keystroke Meaning
r Replace one character over the cursor
R Overwrite text until the keystroke of Esc
  • Delete Text
Keystroke Meaning
x delete single character under cursor
dd delete the current line
Ndd delete N lines, beginning with the current line
  • Cut and Paste Text
Keystroke Meaning
yy copy (yank, cut) the current line into the buffer
Nyy copy (yank, cut) the next N lines into the buffer
p paste the copied line(s) into the text after the current line
  • Screen Manipulation

In the table below, the symbol ^ before a letter means that the Ctrl key should be held down while the letter key is pressed.

Keystroke Meaning
^f move down (forward) one screen
^b move up (backward) one screen
^d move down (forward) one half screen
^u move up (back) one half screen
  • Search and Replace Text

A common occurrence in text editing is to replace one word or phase by another. To locate instances of particular sets of characters (or strings), use the following commands:

Keystroke(s) Meaning
/string [Enter] search forward for occurrence of 'string' in text
?string [Enter] search backward for occurrence of 'string' in text
n [Enter] move to next occurrence of search string
:%s/oldstring/newstring/cg [Enter] search 'oldstring' and replace it with 'newstring' in the entire file
  • Determine Line Number
Keystroke(s) Meaning
:.= [Enter] returns line number of current line at bottom of screen
:= [Enter] returns the total number of lines at bottom of screen
:set number [Enter] provides the line number for the entire file

Further Reading