Documentation - User contributions [en]

Deploying a web server

2018-05-30T14:21:58Z

M2elsakh: /* Dependencies - Launch Cloud Instance */

=Dependencies - Launch Cloud Instance=
Before deploying a web server, first a cloud instance must be lauched. The instruction to do this can be found [[Carpet|here]].
For the remainder of the instructions the ip address used is 199.241.164.95, this is for demonstration purposes and you should replace
it with your assigned floating IP address.

# login to https://cloud.sharcnet.ca/dashboard
# Use guest accounts to login , https://docs.google.com/document/d/1cQ7BSEU09hpSJ5yco33o-hS7AE2PTKdis8JMrHhtld8
# Don’t forget volume size
# Choose Debian 9.2.2 (not required, but the remaining instructions are Debian centric).
# Choose persistent (ephemeral is for shorter jobs) (4C-8GB)
# Setup keypair
# Associate floating ip

==(Optional) Apply IP Address to your name server==
If you have a registered domain name you should apply your floating IP address to it so that you can use the "let's encrypt" service to
enable the secure socket layer without client side warnings. It is often best to do this at the beginning as there is typically a delay
in updating the name service.

= Server Setup =

==Login to Server & perform preliminaries==
At this point log into the server to ensure that the service is up and running. There are a number of steps you can perform that will make
the remaining steps easier. First loging to the server. Then setup a user prompt to make navigation easier. Switch to root (or use sudo in
from of the remaining commands). Update and upgrade the system. Last install the manual pages.
$ ssh debian@199.241.164.95
$ echo 'export PS1="\[\e[33m\]\w\[\e[0m\]\n\[\e[32m\]\u@\h$ \[\e[0m\]"' >> .bash_aliases
$ sudo su root
$ apt update
$ apt upgrade
$ apt install man

==Apt error==
If you receive the "apt error" mesage put "nameserver 8.8.8.8" in /etc/resolve.conf.
$ echo 'nameserver 8.8.8.8 >>' /etc/resolve.conf

==Secure the SSH login==
The following changes will disable logging into the server by using a password on any account and prevent logging into root remotly. Root can
still be accessed by logging into a ''sudo'' enabled account and using the command ''su root''. For more information on the sshd_config file
options, go [https://man.openbsd.org/sshd_config here]. The unattended-upgrades package is used to keep they server up to date automatically.
$ sudo vim /etc/ssh/sshd_config
ChallengeResponseAuthentication no
PasswordAuthentication no
PermitRootLogin no
$ service ssh reload
$ apt install unattended-upgrades
$ dpkg-reconfigure --priority=low unattended-upgrades
$ sudo unattended-upgrade -d

=Install webserver and suppporting packages=
==MYSQL==
$ apt install mysql-server
$ mysql_secure_installation #password=password

==Apache==
Installing the Apache webserver will allow you to access a default page at your ip address. (ex http://199.241.164.95, or http://frar.ca).
Note that https will not yet work. httpd is the Apache HyperText Transfer Protocol (HTTP) server program. It is designed to be run as a standalone daemon process. When used like this it will create a pool of child processes or threads to handle requests. In general, httpd should not be invoked directly, but rather should be invoked via apachectl on Unix-based systems or as a service on Windows NT, 2000 and XP and as a console application on Windows 9x and ME.

$ apt install apache2
$ service apache2 start #add 80, 443 to default security group rules
$ a2enmod cgid.load #(optional) enable cgi scripting, a2dismod to remove

===Basic information & settings===
# Document root Directory: /var/www/html or /var/www
# Main Configuration file:
## /etc/httpd/conf/httpd.conf (RHEL/CentOS/Fedora)
## /etc/apache2/apache2.conf (Debian/Ubuntu).
# Default HTTP Port: 80 TCP
# Default HTTPS Port: 443 TCP
# Access Log files of Web Server: /var/log/apache2/access_log
# Error Log files of Web Server: /var/log/apache2/error_log
# service apache2 {start|stop|graceful-stop|restart|reload|force-reload}
# apachectl -v
## [https://httpd.apache.org/docs/2.4/programs/apachectl.html see]

===Secure Apache with SSL certificates===
Going to https is now possible but will trigger a warning.
$ a2enmod ssl
$ a2ensite default-ssl.conf
$ service apache2 restart

===Enable let's encrypt (requires a domain name)===
$ apt install git
$ sudo git clone https://github.com/letsencrypt/letsencrypt /opt/letsencrypt
$ cd /opt/letsencrypt
$ ./certbot-auto --authenticator webroot --installer apache
www.frar.ca
/var/www/html

===Use self signed certificate [https://wiki.debian.org/Self-Signed_Certificate]===
$ mkdir /etc/ssl/localcerts
$ openssl req -new -x509 -days 365 -nodes -out /etc/ssl/localcerts/apache.pem -keyout /etc/ssl/localcerts/apache.key
$ chmod 600 /etc/ssl/localcerts/apache*
$ cd /etc/apache2/sites-available/
$ cp default-ssl.conf my-ssl.conf
$ vim my-ssl.conf
NameVirtualHost *:443
<VirtualHost *:443>
SSLEngine On
SSLCertificateFile /etc/ssl/localcerts/apache.pem
SSLCertificateKeyFile /etc/ssl/localcerts/apache.key

$ a2ensite my-ssl.conf

===Add basic user-password verfication===
$ mkdir /var/www/passwd
$ htpasswd -c /var/www/passwd/passwords user
$ vim /etc/apache2/apache2.conf
<Directory /var/www/html>
Require valid-user
AuthType basic
AuthName "Restricted Files"
AuthUserFile "/var/www/passwd/passwords"
</Directory>
$ service apache2 restart

==Helper packages==
===Install PHP===
$ apt -y install php7.0 libapache2-mod-php7.0 php7.0-mysql php7.0-gd php7.0-opcache
$ echo ‘<?php phpinfo(); ?>’ > /var/www/html/test.php

===Give www-data an ssh key and ownership of home directory===
$ chmod 774 /var/www/html
$ chown www-data:www-data -R /var/www
$ mkdir /var/www/.ssh
$ chown www-data:www-data /var/www/.ssh
$ sudo -u www-data ssh-keygen -C www-data #no passphrase for now

==SSH Fuse==
Give “debian” user www-data group so that files can be uploaded to the web server
$ sudo usermod -aG www-data debian

====Copy the ssh key to graham====
(lamp) $ sudo cp .ssh/id_rsa.pub ~
(local) $ scp -3 cloud:~/id_rsa.pub graham:~ # the -3 disables host to host copying
(graham)$ cat id_rsa.pub >> authorized_keys

====Setup Graham (placeholder section)====
$ ssh graham
$ mkdir ~/project/edward/linked-cloud
$ cd linked-cloud
$ wget https://git.sharcnet.ca/edward/CSVSorterDemo/raw/master/CSVSorter.jar
$ mkdir input
$ mkdir output
$ vim submitjob.sh
#!/bin/bash
#SBATCH -t 0-00:01
#SBATCH --mem=4G
#SBATCH -A def-edward
java -Xmx10m -jar /home/edward/project/edward/linked-cloud/CSVSorter.jar \
/home/edward/project/edward/linked-cloud/input/$1 \
/home/edward/project/edward/linked-cloud/output/$1

====Link LAMP server with Graham through sshfuse====
$ ssh cloud
$ apt install sshfs
$ cd /var/www
$ sudo -u www-data mkdir linked-graham
$ sudo -u www-data sshfs edward@graham.sharcnet.ca:/home/edward/project/linked-cloud /var/www/linked-graham
- (to unmount) fusermount -u ./project

Deploying a web server

2018-05-30T14:19:13Z

M2elsakh:

=Dependencies - Launch Cloud Instance=
Before deploying a web server, first a cloud instance must be lauched. The instruction to do this can be found [[Carpet|here]].
For the remainder of the instructions the ip address used is 199.241.164.95, this is for demonstration purposes and you should replace
it with your assigned floating IP address.

# login to https://cloud.sharcnet.ca/dashboard
# Don’t forget volume size
# Choose Debian 9.2.2 (not required, but the remaining instructions are Debian centric).
# Choose persistent (ephemeral is for shorter jobs) (4C-8GB)
# Setup keypair
# Associate floating ip

==(Optional) Apply IP Address to your name server==
If you have a registered domain name you should apply your floating IP address to it so that you can use the "let's encrypt" service to
enable the secure socket layer without client side warnings. It is often best to do this at the beginning as there is typically a delay
in updating the name service.

= Server Setup =

==Login to Server & perform preliminaries==
At this point log into the server to ensure that the service is up and running. There are a number of steps you can perform that will make
the remaining steps easier. First loging to the server. Then setup a user prompt to make navigation easier. Switch to root (or use sudo in
from of the remaining commands). Update and upgrade the system. Last install the manual pages.
$ ssh debian@199.241.164.95
$ echo 'export PS1="\[\e[33m\]\w\[\e[0m\]\n\[\e[32m\]\u@\h$ \[\e[0m\]"' >> .bash_aliases
$ sudo su root
$ apt update
$ apt upgrade
$ apt install man

==Apt error==
If you receive the "apt error" mesage put "nameserver 8.8.8.8" in /etc/resolve.conf.
$ echo 'nameserver 8.8.8.8 >>' /etc/resolve.conf

==Secure the SSH login==
The following changes will disable logging into the server by using a password on any account and prevent logging into root remotly. Root can
still be accessed by logging into a ''sudo'' enabled account and using the command ''su root''. For more information on the sshd_config file
options, go [https://man.openbsd.org/sshd_config here]. The unattended-upgrades package is used to keep they server up to date automatically.
$ sudo vim /etc/ssh/sshd_config
ChallengeResponseAuthentication no
PasswordAuthentication no
PermitRootLogin no
$ service ssh reload
$ apt install unattended-upgrades
$ dpkg-reconfigure --priority=low unattended-upgrades
$ sudo unattended-upgrade -d

=Install webserver and suppporting packages=
==MYSQL==
$ apt install mysql-server
$ mysql_secure_installation #password=password

==Apache==
Installing the Apache webserver will allow you to access a default page at your ip address. (ex http://199.241.164.95, or http://frar.ca).
Note that https will not yet work. httpd is the Apache HyperText Transfer Protocol (HTTP) server program. It is designed to be run as a standalone daemon process. When used like this it will create a pool of child processes or threads to handle requests. In general, httpd should not be invoked directly, but rather should be invoked via apachectl on Unix-based systems or as a service on Windows NT, 2000 and XP and as a console application on Windows 9x and ME.

$ apt install apache2
$ service apache2 start #add 80, 443 to default security group rules
$ a2enmod cgid.load #(optional) enable cgi scripting, a2dismod to remove

===Basic information & settings===
# Document root Directory: /var/www/html or /var/www
# Main Configuration file:
## /etc/httpd/conf/httpd.conf (RHEL/CentOS/Fedora)
## /etc/apache2/apache2.conf (Debian/Ubuntu).
# Default HTTP Port: 80 TCP
# Default HTTPS Port: 443 TCP
# Access Log files of Web Server: /var/log/apache2/access_log
# Error Log files of Web Server: /var/log/apache2/error_log
# service apache2 {start|stop|graceful-stop|restart|reload|force-reload}
# apachectl -v
## [https://httpd.apache.org/docs/2.4/programs/apachectl.html see]

===Secure Apache with SSL certificates===
Going to https is now possible but will trigger a warning.
$ a2enmod ssl
$ a2ensite default-ssl.conf
$ service apache2 restart

===Enable let's encrypt (requires a domain name)===
$ apt install git
$ sudo git clone https://github.com/letsencrypt/letsencrypt /opt/letsencrypt
$ cd /opt/letsencrypt
$ ./certbot-auto --authenticator webroot --installer apache
www.frar.ca
/var/www/html

===Use self signed certificate [https://wiki.debian.org/Self-Signed_Certificate]===
$ mkdir /etc/ssl/localcerts
$ openssl req -new -x509 -days 365 -nodes -out /etc/ssl/localcerts/apache.pem -keyout /etc/ssl/localcerts/apache.key
$ chmod 600 /etc/ssl/localcerts/apache*
$ cd /etc/apache2/sites-available/
$ cp default-ssl.conf my-ssl.conf
$ vim my-ssl.conf
NameVirtualHost *:443
<VirtualHost *:443>
SSLEngine On
SSLCertificateFile /etc/ssl/localcerts/apache.pem
SSLCertificateKeyFile /etc/ssl/localcerts/apache.key

$ a2ensite my-ssl.conf

===Add basic user-password verfication===
$ mkdir /var/www/passwd
$ htpasswd -c /var/www/passwd/passwords user
$ vim /etc/apache2/apache2.conf
<Directory /var/www/html>
Require valid-user
AuthType basic
AuthName "Restricted Files"
AuthUserFile "/var/www/passwd/passwords"
</Directory>
$ service apache2 restart

==Helper packages==
===Install PHP===
$ apt -y install php7.0 libapache2-mod-php7.0 php7.0-mysql php7.0-gd php7.0-opcache
$ echo ‘<?php phpinfo(); ?>’ > /var/www/html/test.php

===Give www-data an ssh key and ownership of home directory===
$ chmod 774 /var/www/html
$ chown www-data:www-data -R /var/www
$ mkdir /var/www/.ssh
$ chown www-data:www-data /var/www/.ssh
$ sudo -u www-data ssh-keygen -C www-data #no passphrase for now

==SSH Fuse==
Give “debian” user www-data group so that files can be uploaded to the web server
$ sudo usermod -aG www-data debian

====Copy the ssh key to graham====
(lamp) $ sudo cp .ssh/id_rsa.pub ~
(local) $ scp -3 cloud:~/id_rsa.pub graham:~ # the -3 disables host to host copying
(graham)$ cat id_rsa.pub >> authorized_keys

====Setup Graham (placeholder section)====
$ ssh graham
$ mkdir ~/project/edward/linked-cloud
$ cd linked-cloud
$ wget https://git.sharcnet.ca/edward/CSVSorterDemo/raw/master/CSVSorter.jar
$ mkdir input
$ mkdir output
$ vim submitjob.sh
#!/bin/bash
#SBATCH -t 0-00:01
#SBATCH --mem=4G
#SBATCH -A def-edward
java -Xmx10m -jar /home/edward/project/edward/linked-cloud/CSVSorter.jar \
/home/edward/project/edward/linked-cloud/input/$1 \
/home/edward/project/edward/linked-cloud/output/$1

====Link LAMP server with Graham through sshfuse====
$ ssh cloud
$ apt install sshfs
$ cd /var/www
$ sudo -u www-data mkdir linked-graham
$ sudo -u www-data sshfs edward@graham.sharcnet.ca:/home/edward/project/linked-cloud /var/www/linked-graham
- (to unmount) fusermount -u ./project