Development

Once you have installed the backend core, you can start working on it. This document will guide you through our development process.

When installing the package, you’ll want to change install to develop. This will make sure that updates to any Python files are immediately available to the project’s code. Otherwise it would be necessary to issue a install command on each and every file change you make. That wouldn’t be much fun!

Migrate Mongodb

You can migrate data with /manage.py mongo migrate. This will check which scripts in the migrations/ folder are already applied and which are new. The new ones will be executed. Generally migrations are idempotent (can be executed several times without breaking things).

Using Git

The codebase is versioned using the Git version control software. Development follows a lenient branching structure, that you can easily follow.

Primary Branches

• master is the default branch in Git and this is where all work comes together, new features always land in this branch.

• stable/1.0 releases are based on stable branches, which themselves are branched off the master branch. each major/minor version combination gets its own branch and will be maintained individually.

Feature development

To start on a new feature or a similar task, a branch is created off the current master branch. Its name should follow a “category/description” convention, where you are free to include a ticket number as well. Examples: feature/1288-feature-x, task/rename-thing.

To have your new code approved, push it to our GitHub repository and create a pull request into master (the default). A fellow colleague will promptly review your changes and either approve them or request alterations before being merged into the master branch.

When creating your pull request, provide any additional useful information not already provided by the commit message and please tag the PR with appropriate labels from the sidebar.

Bug fixes

Before fixing a bug, ensure which versions are affected by it. If the bug is only present in the master branch, fixing the issue can be done the same as above.

Otherwise, create a bug fix from the original stable/x.x branch where it first occured or whichever release branch is still being maintained. The pull request should be created targetting the relevant stable release branch.

After review and a successful merge, the bug fix should also be made available to the latest development version in master. This ususally means cherry-picking the fixing commit from the release branch onto the master branch (via a pull request branch).

Releases

Releasing a new version involves updating the version numbers mentioned in the code files as well as any possibly affected supporting files (such as this documentation). Then, those changes are submitted as a pull request onto the desired stable/x.x branch, to complete the release. Tagging this commit can be done via the GitHub “Releases” interface after merging.

You can do this automatically for xmm-core by running invoke release x.x, or invoke release x.x --nopush, if you do want to generate local changes but not push to Github yet.

Code quality

Code quality is automatically verified using continuous integration. We use a sub- and superset of common pep8 rules, as well as some other conventions. The current exact configuration for each code analysis tool can be seen in the setup.cfg file.

To run the code checks locally on your machine, make sure to install the additional build requirements found in the requirements/build.txt file. After installing those, you can run flake8:

flake8 path/to/file.py

This will let you know of any code quality issues in your changes. We recommend setting this up as a git pre-commit hook, which will automatically run every time you commit your changes. To do this, create an executable file called pre-commit in the directory .git/hooks:

#!/bin/bash

flake8 $(git status -s | cut -c 4- | sed -s 's/.*-> //')

Using git status as input will make sure you only get issues reported on files you actually worked on!

Testing

Tests are written using py.test. Test files are located in the tests directory at the root of the project. Each file containing tests should start with the prefix test_ to make sure it will be discovered and run.

Running tests

Running the test suite is straightforward, simply execute this command(in the root of the xmm-core project directory):

$ py.test

This will run all tests and report back to you with a success or failure message. Individual tests can also be run by appending their file name and identifier. Check the help available for further details.

If you want to see coverage data, you can run the tests with --cov=xmm, our continuous integration environment will automatically do this for you.

Writing tests

A little bit more tricky, but just as easy once you get the hang of it: writing more tests.

There are two main types of tests, both of which we have separated in their respective directory under the primary tests folder. Unit and integration tests.

Unittests are simple and small tests that will test one single unit in the code. This can be a single class or even a single function. Writing methods or functions with a lot of side effects will make writing unittests a lot harder, so try to keep this in mind.

Integration tests on the other hand, can be considered “blackbox” tests. They will typically run a very toplevel function and check if the final output matches our expectations. In our example, we fire virtual HTTP requests to the Flask backend and check if the server responded with the proper answer.

Your first test

Create a file inside of the integration test folder, it should have a name that looks like test_part_of_app.py, where “part_of_app” is whatever part you want to test, easy!

Within this file, try to follow our general style guide and don’t forget the encoding declaration at the top of the file.

You can create one or more functions, each representing a single test case and every function should also be prefixed with test_ so py.test knows which functions to run. You may also use a class with many tests, should you wish to have shared functionality between the test cases.

Each test should have one or more asserts to make sure, everything is as expected. That’s it! There are no special testing methods to remember, py.test will automatically figure out what to do with each assert statement.

Using fixtures

Todo

You probably want to use test fixtures.

Documentation

Todo

This section needs work.