Local development

General Workflow

Start stack with or without GitHub OAuth (see below)
Visit http://localhost/ to see the frontend
Make modifications
Wait for automatic reload

NOTE: You might get 502 when visiting http://localhost/ at first: this is normal. Wait a few seconds and try again.

It’s recommended that you initialize your corgi environment before starting by running:

source corgi

in the root of the repository. This will initialize your environment with helpful aliases.

With your environment initialized:

You can run corgi without prefixing it with ./
npm will be prefixed with your frontend directory (you can run npm from anywhere)
Your PS1 should be updated to include (corgi)

GitHub OAuth Disabled (Leashed Mode)

To start corgi without GitHub OAuth, in the project root directory, run:

./corgi start-leashed

or

./corgi start

then you can initialize the database with sample data by running

./corgi create-jobs

In leashed mode, corgi only uses data that is stored locally.

Currently, that data is stored in backend/app/tests/unit/data.

NOTE: By default, you will only be able to use tiny-book as the repository and book-slug1 as the book when queuing jobs. This data can be updated: see Run backend unit tests for more information.

GitHub OAuth Enabled

Prerequisites

You will need access to a GitHub OAuth app. You can create your own if you do not have one already.
Create an env file, backend/app/app/core/.env, with two entries:
1. GITHUB_OAUTH_ID (your oauth app client id)
2. GITHUB_OAUTH_SECRET (your oauth app client secret)

Build/start the stack with the Corgi script:

./corgi start-dev

View the API Docs here:

http://localhost/docs (Swagger)
http://localhost/redoc (ReDoc)

To check the logs run:

./corgi logs backend/frontend/etc.

Functions Patched in Leashed Mode

Leashed mode attempts to patch any app.github.api function functions starting with get_. It looks for the associated mock functions in backend/app/tests/unit/init_test_data.py.

Hot Reloading

Using the development stack, the Svelte frontend is rebuilt inside the container as you make changes: no restarts required. The page should reload automatically as well. The same is true for the backend: as you make modifications, the backend server should reload with your latest changes.

Run backend unit tests

To run unit tests:

cd backend/app
poetry run pytest tests/unit

The unit tests use vcr to store response from GitHub. To update the test data:

cd backend/app
poetry run pytest tests/unit --init-test-data --github-token "<token>"

Run integration and UI tests

To run the tests execute:

./corgi ui-tests <BASE_URL> <TOKEN>

# Run UI tests on staging
# IMPORTANT: no trailing slash
./corgi ui-tests https://corgi-staging.ce.openstax.org <github-token>

# Run UI tests on localhost (see note)
./corgi ui-tests http://host.docker.internal fake-token

NOTE: host.docker.internal refers to the host machine’s localhost instead of the container’s (see documentation for more info). The ui tests will still fail if CORGI is running in leashed mode because of how the the github api calls are patched. If you want the ui tests to work locally, you will need to configure the OAuth credentials accordingly and run CORGI in dev mode.

How to develop UI tests

Playwright UI tests are stored in backend/app/tests/ui.

Clear the database

Start the stack as described above

Run the reset-db command that is contained in the manage.py file.

docker-compose exec backend python manage.py reset-db

Migrations

Automatic migration files can be generated for the database. After a change is made you’ll want to create a new revision.

Make sure the stack is running in development mode.

Enter the backend Docker container:

docker-compose exec backend bash

Create a revision:

docker-compose exec backend alembic revision --autogenerate -m "added new column X"

A new revision will be added in the migration folder.

Open the file and ensure the changes are what you expect.

Do the migration:

docker-compose exec alembic upgrade head

Load testing for the backend (Under Construction)

UNDER CONSTRUCTION

Load testing with Locust.io is in the directory ./backend/app/tests/performance/

Please look at the README in this directory on how to run load tests locally and for production systems.