Schema migration with Neon Postgres and Django
Set up Neon Postgres and run migrations for your Django project
Django is a high-level Python framework to make database-driven web applications. It provides an ORM (Object-Relational Mapping) layer that abstracts database operations, making it easy to interact with databases using Python code. Django also includes a powerful migration system that allows you to define and manage database schema changes over time.
This guide demonstrates how to use Django with a Neon Postgres database. We'll create a simple Django application and walk through the process of setting up the database, defining models, and generating and running migrations to manage schema changes.
Prerequisites
To follow along with this guide, you will need:
- A Neon account. If you do not have one, sign up at Neon. Your Neon project comes with a ready-to-use Postgres database named
neondb. We'll use this database in the following examples. - Python installed on your local machine. We recommend using a newer version of Python, 3.8 or higher.
Setting up your Neon database
Initialize a new project
- Log in to the Neon Console and navigate to the Projects section.
- Select a project or click the
New Projectbutton to create a new one.
Retrieve your Neon database connection string
On your Neon project dashboard, navigate to the Connection Details section to find your database connection string. It should look similar to this:
postgresql://alex:AbC123dEf@ep-cool-darkness-123456.us-east-2.aws.neon.tech/dbname?sslmode=requireKeep your connection string handy for later use.
Setting up the Django project
Set up the Python environment
To manage our Django project dependencies, we create a new Python virtual environment. Run the following commands in your terminal to set it up.
python -m venv myenvActivate the virtual environment by running the following command:
# On macOS and Linux
source myenv/bin/activate
# On Windows
myenv\Scripts\activateWith the virtual environment activated, we can create a new directory for our Django project and install the required packages:
mkdir guide-neon-django && cd guide-neon-django
pip install Django "psycopg2-binary"
pip install python-dotenv dj-database-url
pip freeze > requirements.txtWe installed Django and the psycopg2-binary package to connect to the Neon Postgres database. We also added the python-dotenv to read environment variables easily, and the dj-database-url package to parse the Neon connection string into Django settings. We also saved the installed packages to a requirements.txt file so the project can be easily recreated in another environment.
Create a new Django project
Run the following command to create a new Django project in the current directory:
django-admin startproject guide_neon_django .This command creates a new Django project named guide_neon_django in the current directory.
Set up the Database configuration
Create a .env file in the project root directory and add the DATABASE_URL environment variable to it. Use the connection string that you obtained from the Neon Console earlier.
# .env
DATABASE_URL=NEON_POSTGRES_CONNECTION_STRINGFor Django to read the environment variables from the .env file, open the settings.py file located in the guide_neon_django directory and add the following code, updating the DATABASES setting:
# settings.py
import os
import dotenv
import dj_database_url
dotenv.load_dotenv("../.env")
DATABASES = {
"default": dj_database_url.parse(
url=os.getenv("DATABASE_URL", ""),
conn_max_age=600, conn_health_checks=True
)
}Create a new Django app
Inside your project directory, run the following command to create a new Django app:
python manage.py startapp catalogThis command creates a new app named catalog inside the Django project.
Defining data models and running migrations
Specify the data model
Now, open the models.py file in your catalog app directory and define the database models for your application:
# catalog/models.py
from django.db import models
class Author(models.Model):
name = models.CharField(max_length=100)
bio = models.TextField(blank=True)
created_at = models.DateTimeField(auto_now_add=True)
def __str__(self):
return self.name
class Book(models.Model):
title = models.CharField(max_length=200)
author = models.ForeignKey(Author, on_delete=models.CASCADE)
created_at = models.DateTimeField(auto_now_add=True)
def __str__(self):
return self.titleThis code defines two models: Author and Book. The Author model represents an author with fields for name, bio, and a created_at timestamp. The Book model represents a book with fields for title, author (as a foreign key to the Author model), and a created_at timestamp. Django automatically creates an id field for each model as the primary key.
Generate migration files
We first add the new application catalog to the list of installed apps for the Django project. Open the settings.py file in the guide_neon_django directory and add the catalog app to the INSTALLED_APPS setting:
# settings.py
INSTALLED_APPS = [
"django.contrib.admin",
"django.contrib.auth",
"django.contrib.contenttypes",
"django.contrib.sessions",
"django.contrib.messages",
"django.contrib.staticfiles",
"catalog"
]To generate migration files based on the defined models, run the following command:
python manage.py makemigrationsThis command detects the new Author and Book models that were added and generates migration files in the catalog/migrations directory.
Apply the migration
Now, to apply the migration and create the corresponding tables in the Neon Postgres database, run the following command:
python manage.py migrateThis command executes the migration files and creates the necessary tables in the database. Note that Django creates multiple other tables, such as django_migrations and auth_user for its internal usage.
Seed the database
To populate the database with some initial data, we can create a custom management command for our app. Create a new file named populate.py in the catalog/management/commands directory.
mkdir -p catalog/management/commands
touch catalog/management/commands/populate.pyNow, add the following code to the populate.py file to create some authors and books:
from django.core.management.base import BaseCommand
from catalog.models import Author, Book
class Command(BaseCommand):
help = 'Seeds the database with sample authors and books'
def handle(self, *args, **options):
# Create authors
authors = [
Author(
name="J.R.R. Tolkien",
bio="The creator of Middle-earth and author of The Lord of the Rings."
),
Author(
name="George R.R. Martin",
bio="The author of the epic fantasy series A Song of Ice and Fire."
),
Author(
name="J.K. Rowling",
bio="The creator of the Harry Potter series."
),
]
Author.objects.bulk_create(authors)
# Create books
books = [
Book(title="The Fellowship of the Ring", author=authors[0]),
Book(title="The Two Towers", author=authors[0]),
Book(title="The Return of the King", author=authors[0]),
Book(title="A Game of Thrones", author=authors[1]),
Book(title="A Clash of Kings", author=authors[1]),
Book(title="Harry Potter and the Philosopher's Stone", author=authors[2]),
Book(title="Harry Potter and the Chamber of Secrets", author=authors[2]),
]
Book.objects.bulk_create(books)
self.stdout.write(self.style.SUCCESS('Successfully seeded the database.'))Now, run the custom management command in your terminal and seed the database:
python manage.py populateImplement the application
Create views to display data
We can now create views to display the authors and books in our application. Create a file views.py in the catalog app directory and add the following code:
# catalog/views.py
from django.http import JsonResponse
from django.core import serializers
from .models import Author, Book
def list_authors(request):
authors = Author.objects.all()
data = [serializers.serialize('json', authors)]
return JsonResponse(data, safe=False)
def list_books_by_author(request, author_id):
books = Book.objects.filter(author_id=author_id)
data = [serializers.serialize('json', books)]
return JsonResponse(data, safe=False)We defined two views: list_authors to list all authors and list_books_by_author to list books by a specific author. The views return JSON responses with the serialized data.
Define URLs for the views
Next, create a file urls.py in the catalog app directory and add the following code:
# catalog/urls.py
from django.urls import path
from . import views
urlpatterns = [
path('authors/', views.list_authors, name='list_authors'),
path('books/<int:author_id>/', views.list_books_by_author, name='list_books_by_author'),
]The URLs are mapped to the views defined previously using the Django URL dispatcher.
Include the app URLs in the project
Finally, include the catalog app URLs in the project's main urls.py file, by updating the urlpatterns list:
# guide_neon_django/urls.py
from django.contrib import admin
from django.urls import path, include
urlpatterns = [
path('admin/', admin.site.urls),
path('catalog/', include('catalog.urls')),
]Run the Django development server
To start the Django development server and test the application, run the following command:
python manage.py runserverNavigate to the url http://localhost:8000/catalog/authors/ in your browser to view the list of authors. You can also view the books by a specific author by visiting http://localhost:8000/catalog/books/<author_id>/.
Applying schema changes
We will demonstrate how to handle schema changes by adding a new field country to the Author model, to store the author's country of origin.
Update the data model
Open the models.py file in your catalog app directory and add a new field to the Author model:
# catalog/models.py
class Author(models.Model):
name = models.CharField(max_length=100)
bio = models.TextField(blank=True)
country = models.CharField(max_length=100, blank=True)
created_at = models.DateTimeField(auto_now_add=True)
def __str__(self):
return self.nameGenerate and run the migration
To generate a new migration file for the schema change, run the following command:
python manage.py makemigrationsThis command detects the updated Author models and generates a new migration file to add the new field to the corresponding table in the database. Now, to apply the migration, run the following command:
python manage.py migrateTest the schema change
Restart the Django development server.
python manage.py runserverNavigate to the url http://localhost:8000/catalog/authors to view the list of authors. You should see the new country field included and set to empty for each author entry, reflecting the schema change.
Conclusion
In this guide, we demonstrated how to set up a Django project with Neon Postgres, define database models, and generate migrations and run them. Django's ORM and migration system make it easy to interact with the database and manage schema evolution over time.
Source code
You can find the source code for the application described in this guide on GitHub.
Resources
For more information on the tools and concepts used in this guide, refer to the following resources:
Need help?
Join our Discord Server to ask questions or see what others are doing with Neon. Users on paid plans can open a support ticket from the console. For more detail, see Getting Support.