Django Channels with Daphne, Gunicorn and Nginx on AWS, Google Cloud, Azure, DigitalOcean: All In One Guide

Django, Django-Channels, PostgreSQL, Gunicorn, Daphne, Nginx, Websockets, Ubuntu Server, AWS, Google Cloud, Azure, Digital Ocean: All In One Guide

Introduction

Django is an amazing and powerful web framework. It provides a simplified development server for testing our code on local host but for the production level there must be a more secure, powerful and fast server. This is a kind of ALL IN ONE guide. We will demonstrate Ubuntu user setup, installation of required packages such as Python, PostgreSQL and their dependencies. We will setup a PostgreSQL database for this project. We will configure the Gunicorn application server to serve simple HTTP/HTTPS requests and will configure Daphne application server to serve web socket WS/WSS requests. We will then set up Nginx to reverse proxy to Gunicorn, giving us access to its security and performance features to serve our application.

Note: In this guide we will create a Ubuntu User saurabh and our django project name will be djangoproject.

Prerequisites and Goals

In order to follow this guide, you all need a fresh Ubuntu Server instance with root user and basic knowledge of Ubuntu. Then we will create a non-root user saurabh with sudo privileges. We will be installing Django within a virtual environment. Once we have our database and application up and running, we will install and configure the Gunicorn and Daphne application server. This will serve as an interface to our application, translating client requests in HTTP or WS to our application. We will then set up Nginx in front of Gunicorn and Daphne to take advantage of its high performance connection handling mechanisms with more security features.

Let’s Get Started !

Installation of Required Packages on our Ubuntu Server

Login to Ubuntu Server via SSH

To access your remote server, Open terminal or cmd on your local system and simply paste it and hit enter.

ssh root@<server IP address>

Now enter your server’s root password and that’s it!

Ubuntu User Creation

Once you are logged in as root, we’re going to add the new user account that we will use to log in from now on. In order to add a user saurabh simply type:

# adduser saurabh

Now let’s grant root privileges to newly created user saurabh by typing:

# usermod -aG sudo saurabh

Installation of Server Dependencies

In this section we will be installing some required packages such as Python, Postgresql, Nginx from the Ubuntu Repositories. Firstly we need to update the local apt package index and then download and install these dependencies.

sudo apt-get update

and then if you are using Python 2, enter:

sudo apt-get install python-pip python-dev libpq-dev postgresql postgresql-contrib nginx

And for Python 3, enter:

sudo apt-get install python3-pip python3-dev libpq-dev postgresql postgresql-contrib nginx

Setup of PostgreSQL Database and User

Let’s create a Postgresql database and a database user for our Django web application. Login to Postgres console by typing:

sudo -u postgres psql

Now let’s create a database first, with name djangoprojectdb:

postgres=# CREATE DATABASE djangoprojectdb;

Create a database user (djangoprojectuser) with password mypassword123 by typing:

postgres=# CREATE USER djangoprojectuser WITH PASSWORD 'mypassword123';

By default Django uses UTF-8 encoding, hence we are setting default encoding to UTF-8 and timezone to UTC and default transaction isolation scheme to “read committed”.

postgres=# ALTER ROLE djangoprojectuser SET client_encoding TO 'utf-8';

postgres=# ALTER ROLE djangoprojectuser SET default_transaction_isolation TO 'read committed';

postgres=# ALTER ROLE djangoprojectuser SET timezone TO 'UTC';

And lastly grant permissions to our newly created database user djangoprojectuser on database djangoprojectdb

postgres=# GRANT ALL PRIVILEGES ON DATABASE djangoprojectdb TO djangoprojectuser;

Now exit out from the Postgresql prompt bt typing:

postgres=# \q

Python Virtual Environment Setup

Let’s create our python virtual environment for our project. First we need to upgrade pip and install virtualenv package. If you are using Python 2, use:

sudo -H pip install --upgrade pip

sudo -H pip install virtualenv

and for Python 3, use:

sudo -H pip3 install --upgrade pip

sudo -H pip3 install virtualenv

Now we can setup our project files. First we need to create a folder for our project with name djangoproject and navigate to this folder:

mkdir ~/djangoproject

cd ~/djangoproject

In this project directory, create a Python Virtual Environment with name env by typing:

virtualenv env

Now we need to activate our virtual environment env, so that we can install our project dependencies.

source env/bin/activate

Your prompt should change to indicate that you are now operating within a Python virtual environment. It will look something like this:

(env)user@host:~/djangoproject$

As we are using PostgreSQL Database and django, install dependencies:

(env) $ pip install django gunicorn psycopg2-binary

Bring your Django Project to the Picture

Now we need to create our Django project or we can simply pull a project from our existing git repository. Let’s create a fresh project djangoproject within our djangoproject directory.

(env) $ django-admin startproject djangoproject

Now add your server IP or domain name to ALLOWED_HOSTS in settings.py file of our djangoproject. We can edit any file on the terminal by using any text editor like nano, vim etc. Navigate to project base directory and to edit settings.py file simply enter:

(env) $ sudo nano settings.py

After adding our domain or IP to ALLOWED_HOSTS we need to setup our django databse, we will simply add this config to settings.py:

DATABASES = {    
   'default': {        
       'ENGINE': 'django.db.backends.postgresql_psycopg2',        
       'NAME': 'djangoprojectdb',        
       'USER': 'djangoprojectuser',        
       'PASSWORD': 'mypassword123',        
       'HOST': 'localhost',        
       'PORT': '',    
    }
}

Now we need to configure our static and media files so that nginx can also serve them, our configuration will look similar to this:

STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, "static")
MEDIA_URL='/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, "media")

To save changes press CTRL+X and enter y and press ENTER.

Now, we can migrate the initial database schema to our PostgreSQL database and collect project static files using the management script:

python manage.py makemigrations
python manage.py migrate
python manage.py collectstatic

UFW firewall is protecting your server. In order to test the development server, we’ll have to allow access to the port we’ll be using.

Create an exception for port 8000 by typing:

(env) $ sudo ufw allow 8000

Finally, we can test our project by starting up the development server with this command:

(env) $ python manage.py runserver 0.0.0.0:8000

Now in your browser visit your server’s domain name or IP address followed by :8000

http://server_domain_or_IP:8000

You should see your project’s index page.

Create a Gunicorn systemd Service

Firstly we want to test Gunicorn to make sure that it can serve the application. We can do this by entering our project directory and using gunicorn to load the project’s WSGI module:

(env) $ gunicorn --bind 0.0.0.0:8000 djangoproject.wsgi

This will start Gunicorn on the same interface that the Django development server was running on. You can go back and test the app again. Now press CTRL+C in the terminal to stop gunicorn and let’s exit from virtual environment too by typing:

(env) $ deactivate

We have tested that Gunicorn can interact with our Django application, but we should implement a more robust way of starting and stopping the application server. To accomplish this, we’ll make a systemd service file.

Let’s create and open a systemd service file for Gunicorn in nano text editor:

sudo nano /etc/systemd/system/gunicorn.service

And our gunicorn.service file will look similar to this:

Note: In our gunicorn.service file we set the path for djangoproject.sock file in saurabh directory (outside of our base directory /home/saurabh/djangoproject.sock) instead of our project base directory (/home/saurabh/djangoproject/djangoproject.sock) because if we put this file in project base directory this file will be removed in every git pull operation and we have to create it every time or we have to include it in our git project repository. That’s why we put this file outside our project’s base directory.

Save and close it now. We can now start the Gunicorn service we created and enable it so that it starts at boot:

sudo systemctl start gunicorn
sudo systemctl enable gunicorn

Check the status of the process to find out whether it was able to start:

sudo systemctl status gunicorn

Next, check for the existence of the djangoproject.sock file within your home directory:

ls /home/saurabh

In output you will see djangoproject.sock file.

OUTPUT:
djangoproject djangoproject.sock

If you don’t find djangoproject.sock file or there is an error in response of systemctl status command, this indicates that gunicorn didn’t start correctly. To check the gunicorn logs simply type:

sudo journalctl -u gunicorn

Whenever you make any changes to /etc/systemd/system/gunicorn.service file, don’t forget to reload the daemon and restart the gunicorn by typing:

sudo systemctl daemon-reload
sudo systemctl restart gunicorn

Nginx Configuration

Now after the successful set up of Gunicorn, we need to configure Nginx to pass traffic to the process. Let’s create a new server block in Nginx’s sites-available folder by typing:

sudo nano /etc/nginx/sites-available/djangoproject

In this configuration file we need to specify some important things such as port numbers for websocket: 8001 and HTTP: 80 requests, server name or IP address, locations of our project’s static and media files and required configurations for websocket connections so that Nginx can redirect websocket requests WS or WSS to Daphne and simple HTTP or HTTPS requests to Gunicorn.

And final Nginx configurations will look similar to this:

upstream channels-backend {
 server localhost:8001;
}
server {
    listen 80;
    server_name server_domain_or_IP;
    location = /favicon.ico { access_log off; log_not_found off; }
    location /static/ {
        root /home/saurabh/djangoproject;
    }
    location /media/ {
        root /home/saurabh/djangoproject;
    }
location / {
        include proxy_params;
        proxy_pass http://unix:/home/saurabh/djangoproject.sock;
    }
location /ws/ {
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_redirect off;
        proxy_pass http://127.0.0.1:8001;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Host $server_name;
    }
}

Note: On Digital Ocean server we don’t need to specify full path to our project’s static and media directories because it automatically appends static and media to the static and media locations respectively. That’s why we wrote /home/saurabh/djangoproject instead of /home/saurabh/djangoproject/media OR static in our nginx config file.

Save and close the config file enable the file by linking it to the sites-enabled directory:

sudo ln -s /etc/nginx/sites-available/djangoproject /etc/nginx/sites-enabled

Above command will create a symlink in sites-enable directory for Nginx config file in sites-available directory. Now test your Nginx configuration for syntax errors by typing:

sudo nginx -t

If everything is fine then restart Nginx by typing:

sudo systemctl restart nginx

Now update these firewall rules and allow full access to Nginx:

sudo ufw delete allow 8000
sudo ufw allow 'Nginx Full'

Redis Setup for Django Channels

Redis is used to store messages temporarily for django channels. Install Redis by typing:

sudo apt install redis-server

Now edit redis.conf by typing:

sudo nano /etc/redis/redis.conf

Inside the file, find the supervised directive. This directive allows you to declare an init system to manage Redis as a service, providing you with more control over its operation. The supervised directive is set to no by default. Since you are running Ubuntu, which uses the systemd init system, change this to systemd:

# If you run Redis from upstart or systemd, Redis can interact with your
# supervision tree. Options:
#   supervised no      - no supervision interaction
#   supervised upstart - signal upstart by putting Redis into SIGSTOP mode
#   supervised systemd - signal systemd by writing READY=1 to $NOTIFY_SOCKET
#   supervised auto    - detect upstart or systemd method based on
#                        UPSTART_JOB or NOTIFY_SOCKET environment variables
# Note: these supervision methods only signal "process is ready."
#       They do not enable continuous liveness pings back to your supervisor.
supervised systemd

Now restart the redis service by:

sudo systemctl restart redis.service

Now check the status of redis by:

sudo systemctl status redis

and the output must be like this:

Now install net-tools and confirm that Redis is running at 127.0.0.1. and port should be 6379 by default and restart the Redis using the following commands:

sudo apt install net-tools
sudo netstat -lnp | grep redis
sudo systemctl restart redis.service

Now navigate to your project directory (where settings.py file resides) and create or update the existing asgi.py file according to this sample file:

Daphne Setup for Django Channels

Gunicorn is what we use to run the WSGI application — which is our django app. To run the ASGI application we need something else, an additional tool. Daphne was built for Django channels. Let’s install Daphne by:

sudo apt install daphne

Now we need to update the daphne.service, open it with nano editor by:

sudo nano /etc/systemd/system/daphne.service

Here we will be using port 8001 for our websocket connections and this file will be look similar to this:

Now let’s reload daemon and restart daphne.service by:

systemctl daemon-reload
systemctl start daphne.service
systemctl status daphne.service

You should see something like this:

Now close it using CTRL+C and we need to start daphne using a systemd service when the server boots, just like we start gunicorn and then gunicorn starts the django application. For this purpose let’s create a daphneboot.sh file by typing:

sudo nano /root/daphneboot.sh

And paste the content:

Make this script executable by the following command:

chmod u+x /root/daphneboot.sh

Now we need to tell systemd to run the our daphneboot.sh script when the server boots. For this purpose we have to create a systemd service and to create this service, simply paste:

sudo nano /etc/systemd/system/daphneservice.service

And paste the content:

Now reload the daemon; start and enable our newly created daphneservice.service and allow access to the port 8001 by typing:

systemctl daemon-reload
sudo systemctl start daphneservice
sudo systemctl enable daphneservice
ufw allow 8001

And restart the server by:

sudo shutdown -r now

Now log on to the server again and check the status of our daphneservice.service and daphne.service by:

systemctl status daphneservice.service
systemctl status daphne.service

If everything is fine then you can see websocket working properly on the server. Further you can add domain name to your ubuntu server and can install SSL certificates so that your server can serve HTTPS and WSS requests as well. You can install SSL certificates python-certbot package (To install python-certbot for Nginx simply enter: sudo apt install certbot python3-certbot-nginx). And then run this command to generate SSL certificates:

sudo certbot --nginx -d <your-domain-name.com> -d www.<domain-name.com>

Then you can also set a timer to auto renew your SSL certificates by typing:

sudo systemctl status certbot.timer
sudo certbot renew --dry-run

Nginx Server Error Logs

You can check your Nginx server’s error logs by typing:

sudo tail -F /var/log/nginx/error.log

Conclusion

Hence, in this guide we setup our Django server in this way that it can serve both HTTP and WS requests. Nginx will redirect all HTTP requests to the Gunicorn and all WS requests to Daphne. And then respective server will serve these client requests accordingly. Still, you have a query regarding Django Hosting Tutorial, please leave comments below.

Guys, please Follow me for more such articles! It motivates me to write more amazing stuff related to Python Django Stack.

Thank you :)