From Documentation
Revision as of 15:28, 28 October 2015 by Merz (Talk | contribs) (File systems and permissions)

Jump to: navigation, search

Basic UNIX concepts

  • Unix is an operating system, all SHARCNET systems run some variant of Unix (eg. Linux)
  • Most Unix-based systems have a GUI interface, but the command line offers more complex and abstract interactions with far less effort
  • At login the system starts a shell process for you which acts as your command line interpreter to interface with the operating system
    • Borne Again Shell ( bash ) is the default shell at SHARCNET

Common terms

  • File
    • data stored in a standard format that behaves in a certain way depending on it’s function in the system; everything is a file in Unix
  • Program
    • a file that can be executed (run)
  • Process
    • a program that is being executed (eg. your computing job is made of one or more processes)
  • Ownership
    • files/programs/processes are owned by a user and group
  • Hierarchical Directory Structure
    • files are organized in directories (folders) that can have a parent, eg. /home/$USER/sim1
    • The base of the hierarchy is root , ie: / (forward-slash)

Managing your files and processes is crucial to effectively using the systems!

Basic bash command line behavior

Basic key sequences / commands

  • first you ssh to the system you’d like to use
    • you see the message of the day and are left at a command prompt
  • each time you type in a command you are executing one or more processes
  • you can see commands you ran in the past with history
  • you can scroll through previous commands with the ↑ and ↓ arrow keys
  • you can complete commands / arguments with the Tab ↹ key !!!
  • depending on your terminal (the software you are connecting with) you should be able to go to the start of a line with Ctrl-a or the end with Ctrl-e, and cut to the end with Ctrl-k
  • to exit, run the exit command
  • if your terminal is not responding:
    • you may be able to exit the foreground process by pressing Ctrl-c
    • you may be able to disconnect your ssh session gracefully by entering ~. (sometimes repeatedly, while mashing the Enter↲ key in between…)

Executing Commands

  • To run a command you simply type its name in and hit Enter↲
  • The command must be in your $PATH and be executable (we’ll get to that later…)

General syntax of a command:

 $ command [[-]option(s)] [option argument(s)] [command argument(s)]
  1. command: the name of the command or utility: ls, man, cat, mv
  2. options: change the behaviour of the basic command: ls -l vs. ls
    • may or may not be preceded by “-”
  3. option arguments: change the behaviour of an option: tail –c 5 file1 vs. tail –c 15 file1
  4. command arguments: what is affected by the command, usually files or the output of another command

Basic Commands

  • Getting help with commands (the most important command!):
    • man
  • Figuring out who we are and where we are:
    • whoami, hostname, date
  • Navigating directories:
    • cd, pwd
  • Manipulating files and directories:
    • cp, mv, rm, rmdir, mkdir
  • Listing files and their properties:
    • ls, file
  • Displaying the contents of files:
    • cat, tail, head, more, wc
  • Investigating running programs:
    • ps, top

File systems and permissions

Structure of the file system

  • The root of the file system hierarchy is /
    • it contains subdirectories which may contain further subdirectories
  • File systems are mounted within the hierarchy, eg.
    • one starts off in their SHARCNET /home directory after logging in: /home/$USER
      • Can also refer to this by a shortcut “~/”
      • Can always get to this directory by running cd without any arguments
  • One can refer to file / directory locations by their absolute or relative path
    • The absolute path starts with the root and ends with the file or directory in question, eg. /home/$USER/simulation1/output.txt
    • The relative path depends on which directory you are presently in within the filesystem
      • Run the pwd command to see which directory you are in
      • eg. if we are in /home/$USER the relative path to the above file is simulation1/output.txt
  • The shortcut for the current directory is “.”; for parent directory it is “..”, eg.
    • one can go up a directory with cd .. , run a file in a subdirectory by ./simulation1/program.x
  • One can also set up "links" in the filesystem with the ln -s command (symbolic links)
    • this allows you to make a directory which, upon entering, puts in you in a different directory in the file system tree
    • allows for multiple absolute paths to refer to the same file

Structure of files

  • names can be up to 255 characters, use non-standard characters and file name extensions do not matter to most command line programs
  • files starting with a “.” are hidden, one can see them by specifying: ls -a
  • files have a set of attributes associated with them, you can see a long listing that includes some of the more pertinent values by running: ls -l
    • For each file / directory it will return a record like:
 drwxr-xr-x 	1 	beaker 	honeydew 	4096 	Oct 29 2015 		test_dir
 1		2	3	4		5	6			7
  1. file type and permissions
    • File types: - (regular), d (directory), c (character), b (block) , l (link), s (socket), p (pipe)
    • Permissions: r (read), w (write), x (execute), s (setgid, setuid) and t (sticky bit)
  2. hard link count
    • Indicates the number of copies of the particular file
  3. user owner or UID of the file
  4. group owner or GID of the file
  5. size of the file in bytes
  6. date and time that the file was last modified (versus access / creation)
  7. name of the file

Users and Groups

  • Each user on the system is identified by a unique username ( stored as an environment variable: $USER ) and associated with a numeric UID
    • This is your SHARCNET username @ SHARCNET
  • Each user belongs to one or more groups. Each group has a unique group name and numeric GID associated with it
    • At SHARCNET each sponsor has their own group for them and their group members
  • Other groups exist (eg. to get access to commercial software, institutional groups, etc.)
  • These are the ownership values associated with file permissions settings
  • A file owner can possibly** change the group ownership of a file ( chgrp ) but only the superuser (root user) can change ownership ( chown )
  • One can change specific file permissions (read, write, execute, etc. permissions) with the chmod command

File permissions

  • security is based on three basic actions
    • Read, Write, Execute
  • listed in triplets: user permissions, group permissions, everyone (other)
  • meaning depends on file type
    • regular, directory and special file distinctions
 -rwx   rwx   rwx
 Owner Group Other

For example, the below grants read, write and execute for the owner, read and execute only for group members and only read for others:

 -rwxr-xr-- 	1 	beaker 	honeydew 	4096 	Oct 29 2015 		test_file

umask and chmod

  • Default file permissions are applied to a file based on a mask value: umask
  • One can set this value so that when new directories / files are created they are created with different permissions
  • Permissions are either represented as octal values or symbolic:
 rwx---r-x = 0705 or u+rwx,o+rx
 rwxr-xr-- = 0754 or u+rwx,g+rx,o+r

To change a file or directory’s permissions use chmod:

 [beaker@dtn ~]$ ls -l t1
 -rw-rw-r--. 1 beaker honeydew 0 Oct 28 09:28 t1
 [beaker@dtn ~]$ chmod u+x,g+x,o+wx t1
 [beaker@dtn ~]$ ls -l t1
 -rwxrwxrwx. 1 beaker honeydew 0 Oct 28 09:28 t1

Examples of different file / directory permissions

 -rw-rw-r-- 1 beaker honeydew 0 	2015-09-16 20:54 default_file_at_SHARCNET 
 drwxrwxr-x 2 beaker honeydew 4096 	2015-09-16 20:59 default_dir_at_SHARCNET
 -rwxrwxrwx 1 beaker honeydew 0 	2015-09-16 20:54 fully_executable
 -------rwx 1 beaker honeydew 24 	2015-09-16 20:57 public_executable
 d--------- 2 beaker honeydew 4096 	2015-09-16 20:59 root_only
 -rwx------ 1 beaker honeydew 0 	2015-09-16 20:55 user_executable
 drwx------ 2 beaker honeydew 4096 	2015-09-16 20:59 user_only

Managing files

Text editing

Command pipes and redirection

Shell variables, initialization, and scripts