Deployment
This guide explains how to deploy an instance of LibCrowds to a set of servers with NGINX and supervisor.
Three servers are required, one for LibCrowds, one for PYBOSSA and one for Explicates. The following page describes how to get all three set up.
Domains¶
Before we begin, it is important to note that both LibCrowds and PYBOSSA should be served from the same domain.
This is because LibCrowds uses SSR, so the PYBOSSA session cookie needs to be shared with the LibCrowds server (rather than the client) and this will only work properly if both applications are served from the same domain. Subdomains are fine, so you can still run your applications on different servers.
If you don't do this, the application will still run but, as it will not be possible to fully render certain pages on the server, you will see some strange behaviour with users appearing to not be signed in when the application first loads. You might also notice errors when attempting to navigate directly to restricted pages (such as the various administration dashboards).
With that in mind, let's setup our servers.
PYBOSSA¶
Rather than replicating what has already been written, for details of how to deploy PYBOSSA see the PYBOSSA documentation.
LibCrowds¶
Starting with a fresh Ubuntu 16.04 server, run through the procedures below.
Install Node.js and npm¶
LibCrowds requires Node.js >= 8.0.0 to run.
Update and upgrade the package index:
sudo apt-get update sudo apt-get upgrade
Go to your home directory and retrieve the installation script:
cd ~
curl -sL https://deb.nodesource.com/setup_8.x -o nodesource_setup.sh
Run the script:
sudo bash nodesource_setup.sh
Install Node.js and the build-essential package:
sudo apt-get install nodejs sudo apt-get install build-essential
Install git¶
LibCrowds is distributed under version control using git, install it by running:
sudo apt-get install git-core
Download and build LibCrowds¶
Run the following commands to clone the LibCrowds repository into
/var/www/libcrowds:
mkdir /var/www git clone https://github.com/LibCrowds/libcrowds /var/www/libcrowds
Enter the directory:
cd /var/www/libcrowds
Install LibCrowds' dependencies:
npm install
Copy the configuration template (see Configuring LibCrowds for details of how to edit this file):
cp local.config.js.tmpl local.config.js
Build the application:
npm run build
Tip
If the installation or build process is getting killed before finishing it is possible that your server does not have enough memory. You may want to consider upgrading the server or adding some swap space.
Setting up NGINX¶
To serve the site using NGINX, first install it:
sudo apt-get install nginx
Remove the default server configuration:
sudo rm -r /etc/nginx/sites-available/default sudo rm -r /etc/nginx/sites-enabled/default
Edit a new server configuration file:
vim /etc/nginx/sites-available/libcrowds
Copy in the following and save the file:
map $sent_http_content_type $expires { "text/html" epoch; "text/html; charset=utf-8" epoch; default off; } server { listen 80 default_server; listen [::]:80 default_server; client_body_buffer_size 10K; client_header_buffer_size 1k; client_max_body_size 10m; large_client_header_buffers 2 1k; gzip on; gzip_comp_level 2; gzip_types text/plain application/xml text/css application/javascript; gzip_min_length 1000; gzip_proxied expired no-cache no-store private auth; location / { expires $expires; proxy_redirect off; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_read_timeout 1m; proxy_connect_timeout 1m; proxy_pass http://127.0.0.1:8080; } }
Enable the LibCrowds configuration:
ln -s /etc/nginx/sites-available/libcrowds /etc/nginx/sites-enabled/libcrowds
Restart the server:
sudo service nginx restart
Setting Up Supervisor¶
Supervisor is used to run LibCrowds in the background and restart it when the server boots up.
sudo apt-get install supervisor
Edit a new Supervisor program configuration:
vim /etc/supervisor/conf.d/libcrowds.conf
Copy in the following and save the file:
[program:libcrowds] directory=/var/www/libcrowds command=npm run start autostart=true autorestart=true log_stdout=true log_stderr=true logfile=/var/log/libcrowds.log logfile_maxbytes=10MB logfile_backups=2
Restart supervisor:
sudo service supervisor stop sudo service supervisor start
Your LibCrowds server should now be running at http://your.domain.com. However, we still need to configure the Annotations server.
Explicates¶
Starting with a fresh Ubuntu 16.04 server, run through the procedures below.
Install Python¶
Explicates requires Python >= 2.7 to run.
Update and upgrade the package index:
sudo apt-get update sudo apt-get upgrade
Install Python:
sudo apt install python
Install some additional dependencies:
sudo apt install python-virtualenv python-dev python-setuptools python-pip
Install git¶
Explicates is distributed under version control using git, install it by running:
sudo apt-get install git-core
Download and build Explicates¶
Run the following commands to clone the Explicates repository into
/var/www/explicates:
mkdir /var/www git clone https://github.com/alexandermendes/explicates /var/www/explicates
Enter the directory:
cd /var/www/explicates
Create and enter a virtual environment:
virtualenv env
source env/bin/activate
Install Explicates:
pip install -r requirements.txt
Copy the configuration template (see Configuring Explicates for details of how to edit this file):
cp settings.py.tmpl settings.py
Important
You will need to edit the SERVER_NAME setting now to point to the location
of your server (e.g. annotations.example.com). It is required to build
IDs outside of the request context and this is first done when the database
is created below.
Copy the alembic configuration template:
cp alembic.ini.tmpl alembic.ini
Seting up the database¶
Explicates is backed by a PostgreSQL database.
Install dependencies:
sudo apt-get install postgresql postgresql-server-dev-all python-psycopg2 libpq-dev
Add a database user:
sudo -u postgres createuser -d -P explicates
Enter the password tester when prompted.
Info
If you choose a different password you should also edit the database
path in settings.py and alembic.ini.
Create the database:
sudo -u postgres createdb explicates -O explicates
Populate the database:
python /var/www/explicates/bin/db_create.py
To check that everything has gone OK so far, you should now be able to
type python run.py and see some messages that let you know the develpoment
server is running (then press CTRL+C to quit).
Setting up NGINX¶
To setup NGINX, first install it:
sudo apt-get install nginx
Remove the default server configuration:
sudo rm -r /etc/nginx/sites-available/default
Edit a new server configuration file:
vim /etc/nginx/sites-available/explicates
Copy in the following and save the file:
server { listen 80 default_server; listen [::]:80 default_server; # Change this to match your domain server_name annotations.example.com; client_body_buffer_size 10K; client_header_buffer_size 1k; client_max_body_size 10m; large_client_header_buffers 2 1k; gzip on; gzip_comp_level 2; gzip_types text/plain application/json application/ld+json; gzip_min_length 1000; gzip_proxied expired no-cache no-store private auth; location / { expires -1; include uwsgi_params; uwsgi_pass unix:/tmp/explicates.sock; } }
Enable the Explicates configuration:
ln -s /etc/nginx/sites-available/explicates /etc/nginx/sites-enabled/explicates
Restart the NGINX server:
sudo service nginx restart
Setting up uWSGI¶
To setup uWSGI, first install it:
pip install -U uwsgi
Edit a new server configuration file:
vim /var/www/explicates/explicates.ini
Copy in the following and save the file:
[uwsgi] socket = /tmp/explicates.sock chmod-socket = 666 chdir = /var/www/explicates pythonpath = .. virtualenv = /var/www/explicates/env module = run:app cpu-affinity = 1 processes = 2 threads = 2 buffer-size = 65535
Setting Up Supervisor¶
Supervisor is used to run Explicates in the background and restart it when the server boots up.
sudo apt-get install supervisor
Edit a new Supervisor program configuration:
vim /etc/supervisor/conf.d/explicates.conf
Copy in the following and save the file:
[program:explicates] command=/var/www/explicates/env/bin/uwsgi /var/www/explicates/explicates.ini directory=/var/www/explicates autostart=true autorestart=true log_stdout=true log_stderr=true logfile=/var/log/explicates.log logfile_maxbytes=10MB logfile_backups=2
Restart supervisor:
sudo service supervisor stop sudo service supervisor start
Your Annotation server should now be running at http://your.domain.com.
Security¶
There are lots of potential steps that you could take to secure your servers. Running through all of the options here is outside the scope of this guide, but here are some quick wins for the LibCrowds and Explicates servers (details might need to be modified a bit for the PYBOSSA server).
Firewall¶
UFW, or Uncomplicated Firewall, is an easy way to manage a frontend firewall.
Install it:
sudo apt-get install ufw
Deny incoming and allow outgoing by default:
sudo ufw default deny incoming sudo ufw default allow outgoing
Allow connections:
sudo ufw allow ssh
sudo ufw allow 22/tcp
Check the rules:
ufw status
You should see the following output:
To Action From -- ------ ---- 22 ALLOW Anywhere 80 ALLOW Anywhere 443 ALLOW Anywhere 22 (v6) ALLOW Anywhere (v6) 80 (v6) ALLOW Anywhere (v6) 443 (v6) ALLOW Anywhere (v6)
If the rules do indeed appear as above, enable the firewall:
sudo ufw enable
Secure Sockets Layer (SSL)¶
Let's Encrypt provide free SSL certificates. Use the Certbot client to set them up for your operating system.
After following the Certbot guide, above, you can setup a scheduled task to automatically renew the certificates.
Edit a new daily task:
vim /etc/cron.daily/certrenew
Copy in the following and save the file:
certbot renew
Make the file executable:
chmod +x /etc/cron.daily/certrenew
Your certificate should now be checked for renewal daily.
The next step is to install the required LibCrowds Plugins.