Setting up a Django Website in the Princeton CS Environment

Overview

This guide is not about how to use the Django framework. Rather, it describes the Princeton CS environment and how one might set up a Django site in "project space" at https://mywebsite.cs.princeton.edu that has reasonable portability to other hosting services when/if the need should arise.

These instructions roughly parallel the initial steps from these Django website documents: Quick install guide, Writing your first Django app, part 1, and, because this is a deployment, Deployment checklist.

Initial Setup

Request Resources and Set up the Python/WSGI Environment

Follow the steps in the Requesting Resources / Initial Set-up section of the CS Guide document Python-based Websites Using WSGI. Now, when visiting https://mywebsite.cs.princeton.edu, you should see a "Hello, World" page. Since we will be editing the site, put the site into maintenance mode:

  1. Run: touch /n/fs/myproject/mywebsitefiles/www-approot/tmp/maint.txt
  2. Run: touch /n/fs/myproject/mywebsitefiles/www-approot/tmp/restart.txt
  3. Reload the site a few times until it shows the maintenance page. This can take up to 10 seconds.

The remainder of this document uses the same naming conventions and project layout as Python-based Websites Using WSGI.

Activate your Python Virtual Environment

The shell commands in this document assume that you are in your Python virtual environment:

source /n/fs/myproject/mywebsitefiles/venv/bin/activate

Install Django and Create your Django Project

Install Django, create mysite project, and make directories for secrets and logs:

# in your venv:    
pip install Django
cd /n/fs/myproject/mywebsitefiles
django-admin startproject mysite
mkdir -p /n/fs/myproject/mywebsitefiles/log
chmod go= /n/fs/myproject/mywebsitefiles/log
mkdir -p /n/fs/myproject/mywebsitefiles/secrets
chmod go= /n/fs/myproject/mywebsitefiles/secrets
touch /n/fs/myproject/mywebsitefiles/secrets/credentials.env
chmod go= /n/fs/myproject/mywebsitefiles/secrets/credentials.env

Your /n/fs/myproject/mywebsitefiles should look roughly like this:

/n/fs/myproject
+-- python-3.12.0/
|   +-- ...
+-- mywebsitefiles/
    +-- log/
    +-- mysite/
    |   +-- manage.py
    |   +-- mysite/
    |       +-- __init__.py
    |       +-- asgi.py
    |       +-- settings.py
    |       +-- urls.py
    |       +-- wsgi.py
    +-- secrets/
    |   +-- credentials.env
    +-- venv/
    |   +-- ...
    +-- www-docroot/
    +-- www-approot/
        +-- config.toml
        +-- passenger_wsgi.py
        +-- tmp/
            +-- maint.txt
            +-- restart.txt

"Productionize"

The initial settings.py file is meant for local development. We need to make some changes to it so that the site suitable for production. In addition, it is good practice to keep hard-wired values out of the settings.py file.

Parameterize ALLOWED_HOSTS

In your settings.py file, Just above the from pathlib import Path line, add:

import os

Replace the ALLOWED_HOSTS line with:

ALLOWED_HOSTS = [os.environ['MYSITEURL']]

We will add MYSITEURL to the environment when we edit the config.toml file, below.

Generate and Protect the SECRET_KEY

Install the python-dotenv package and create a credentials.env file defining DJANGO_SECRET_KEY:

# in your venv:    
pip install python-dotenv
cd /n/fs/myproject/mywebsitefiles/secrets
python -c 'from django.core.management import utils; print(f"DJANGO_SECRET_KEY={utils.get_random_secret_key()}")' >>credentials.env

In your settings.py file, Just below the from pathlib import Path line, add:

from dotenv import dotenv_values

In your settings.py file, Replace the SECRET_KEY line with:

mysecrets = dotenv_values(os.environ['MYSITESECRETS'])
SECRET_KEY = mysecrets['DJANGO_SECRET_KEY']

We will add MYSITESECRETS to the environment when we edit the config.toml file, below.

Configure Logging

In your settings.py file, append this to the end of the file:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'default': {
            'format': '%(asctime)s %(levelname)s %(name)s '
                      '%(threadName)s : %(message)s',
        },
    },
    'handlers': {
        'file': {
            'level': 'DEBUG',
            'class': 'logging.handlers.RotatingFileHandler',
            'filename': os.environ['LOG_FILENAME'],
            'maxBytes': 40000,
            'backupCount': 2,
        },
    },
    'loggers': {
        'django': {
            'handlers': ['file'],
            'level': 'DEBUG',
            'propagate': True,
        },
    },
}

We will add LOG_FILENAME to the environment when we edit the config.toml file, below.

Edit the uwsgi-shim config.toml file

The startproject also creates mysite/mysite/wsgi.py with an application callable. Update the [wsgi] section of your /n/fs/myproject/mywebsitefiles/www-approot/config.toml to chdir to the top-level Django directory (a requirement of Django) and use the app setting to point to the application callable relative to the Django directory:

chdir = "/n/fs/myproject/mywebsitefiles/mysite"
app = "mysite.wsgi.application"

In your config.toml file, add these lines to the [environment] section:

MYSITEURL = "mywebsite.cs.princeton.edu"
LOG_FILENAME = "/n/fs/myproject/mywebsitefiles/log/logfile"
DJANGO_SETTINGS_MODULE = "mysite.settings"

In your config.toml file, add this line to the [secret_files] section:

MYSITESECRETS = "/n/fs/myproject/mywebsitefiles/secrets/credentials.env"

Test the Site and disable DEBUG

As initially created, the settings.py file enables DEBUG mode. Before we disable it, let's test the site:

Take your site out of Maintenance Mode and restart the server:

  1. Run: rm /n/fs/myproject/mywebsitefiles/www-approot/tmp/maint.txt
  2. Run: touch /n/fs/myproject/mywebsitefiles/www-approot/tmp/restart.txt
  3. Reload the site a few times until it shows the Django page. This can take up to 10 seconds.

You should see the Django page, "The install worked successfully! Congratulations!"

Warning: you are running your site in DEBUG mode on the open Internet. Let's take care of that now.

  1. Run: touch /n/fs/myproject/mywebsitefiles/www-approot/tmp/maint.txt
  2. Run: touch /n/fs/myproject/mywebsitefiles/www-approot/tmp/restart.txt
  3. Reload the site a few times until it shows the Maintenance page. This can take up to 10 seconds.
  4. Edit settings.py and change the DEBUG setting to: DEBUG = False.

At this point, you could re-enable your site. However, since you have not created any Django views, re-loading the site would only show the generic-looking "Not Found" page.

Start coding!

While the above steps set up a bare-bones Django application, this was directly in a production environment. Typically, one would develop code in a local environment (e.g., a laptop) and check it into a repository on a service like GitLab or GitHub. When ready for deployment, the repository would be checked out to a directory in /n/fs/myproject/mywebsitefiles with suitable adjustments to the [wsgi] section of your config.toml file.

Database

Assuming your Django project needs a database (most will), the databases officially supported by Django that are also available in the CS Infrastructure are SQLite (lightweight, disk-based, suitable for development) and MariaDB (suitable for production).

For production websites strictly using the CS Infrastructure, we recommend MariaDB over SQLite.1 The remainder of this document assumes you are using MariaDB and specific configuration suggestions are given in Database Configuration, below.

Here are more references on using MariaDB / MySQL with Django:

Requesting a MariaDB Database

To request a MariaDB database, use the "Request Forms" block on the CS Guide and click on "Project Database". The key fields are:

Use: Collaborative

Database Name: mydatabasename

Brief description of use: Django database for mywebsite.cs.princeton.edu

Change DB Password and Character Set

When the database is set up, you will get an email from CS Staff with instructions on changing the database password. While you are changing the password, also change the character set and collation as follows:

mysql> ALTER DATABASE `mydatabasename` CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

Managing Database Credentials

For this example, we define database-related variables in your secrets/credentials.env file:

DB_NAME=mydatabasename
DB_USER=mynetid
DB_PASS=mychangeddatabasepassword
DB_HOST=publicdb.cs.princeton.edu
DB_PORT=3306

Database Configuration

With the database variables in our secrets/credentials.env file, we need to extract them for the DATABASE dictionary in the Django settings.py file.

In your settings.py file, replace the DATABASES definition with:

 DATABASES = {
     'default': {
         'ENGINE': 'django.db.backends.mysql',
         'NAME': mysecrets['DB_NAME'],
         'USER': mysecrets['DB_USER'],
         'PASSWORD': mysecrets['DB_PASS'],
         'HOST': mysecrets['DB_HOST'],
         'PORT': mysecrets['DB_PORT'],
         'CONN_MAX_AGE': 600,
         'CONN_HEALTH_CHECKS': True,
         'OPTIONS': {
             # escalate several InnoDB-specific warnings to errors
             'init_command': 'SET innodb_strict_mode=1',
             # Tell MySQLdb to connect with 'utf8mb4' character set
             'charset': 'utf8mb4',
         },
         # Tell Django to build the test database with the 'utf8mb4' character set
         'TEST': {
             'CHARSET': 'utf8mb4',
             'COLLATION': 'utf8mb4_unicode_ci',
         },
     },
 }

Install the Database Driver

 # in your venv:    
 pip install mysqlclient

Static Files

If you have already visited the /admin pages, you'll notice that the layout looks lousy. This is because, in production, one must explicitly manage static files. Because the CS webserver will first search for files from the DocRoot and only when they are not found connect to your Django application, one can put static files in the www-docroot directory. This is described in the Django documentation at Serving the site and your static files from the same server.

A simpler approach that does not require you to remember to open the world-read permissions on the files in the DocRoot is to use WhiteNoise.

Install with:

# in your venv:    
pip install whitenoise

From Using WhiteNoise with Django:

Add this to the bottom of your settings.py file

STATIC_ROOT = BASE_DIR / 'staticfiles'

And edit your settings.py file and add WhiteNoise to the MIDDLEWARE list2, above all other middleware apart from Django’s SecurityMiddleware:

MIDDLEWARE = [
    # ...
    'django.middleware.security.SecurityMiddleware',
    'whitenoise.middleware.WhiteNoiseMiddleware',
    # ...
]

Use collectstatic to copy static files from various Django apps to the STATIC_ROOT directory:

# in your venv:
cd /n/fs/myproject/mywebsitefiles/mysite
export MYSITESECRETS="/n/fs/myproject/mywebsitefiles/secrets/credentials.env"
export MYSITEURL="mywebsite.cs.princeton.edu"
export LOG_FILENAME="/n/fs/myproject/mywebsitefiles/log/logfile"
python manage.py collectstatic

Take your site out of Maintenance Mode and restart the server:

  1. Run: rm /n/fs/myproject/mywebsitefiles/www-approot/tmp/maint.txt
  2. Run: touch /n/fs/myproject/mywebsitefiles/www-approot/tmp/restart.txt
  3. Visit the admin page at: https://mywebsite.cs.princeton.edu/admin If you get the maintenance page, reload the site a few times until it shows the Django admin login.

Next Steps

Obviously, your next steps depend on the requirements of your site. Typically, you will want to run your first database migration and then create an admin user. These are described below.

Your First Migration

If you are starting a brand-new project, consider the Django recommendation Using a custom user model when starting a project before running your first migration. In the simplest case, this entails creating a Django app called user:

# in your venv:
cd /n/fs/myproject/mywebsitefiles/mysite
python manage.py startapp user

Edit user/models.py to have the following contents:

  from django.contrib.auth.models import AbstractUser
  
  class User(AbstractUser):
      pass

Edit user/admin.py to have the following contents:

  from django.contrib import admin
  from django.contrib.auth.admin import UserAdmin
  from .models import User
  
  admin.site.register(User, UserAdmin)

Now make two edits to your settings.py file. First, in the INSTALLED_APPS list, add 'user.app.UserConfig', after the django.* entries:

  INSTALLED_APPS = [
      ...
      'django.contrib.staticfiles',
      'user.apps.UserConfig',
  ]

Second, at the end of the file add this line:

  AUTH_USER_MODEL = 'user.User'

Now, perform the migration:

# in your venv:
cd /n/fs/myproject/mywebsitefiles/mysite
export MYSITESECRETS="/n/fs/myproject/mywebsitefiles/secrets/credentials.env"
export MYSITEURL="mywebsite.cs.princeton.edu"
export LOG_FILENAME="/n/fs/myproject/mywebsitefiles/log/logfile"
python manage.py makemigrations
python manage.py migrate

Creating an admin user

With the first migration complete, you can follow the steps on the Django page: Creating an admin user.

...and continue with that tutorial to learn more about Django.

  1. While way beyond the scope of this document, it is also possible to use a database from an outside service (e.g., PostgreSQL in AWS RDS).↩︎

  2. For local development, also see: Using WhiteNoise in development.↩︎

/node/4117 built from django.md on 2024-01-23 12:26:36

Tags: