An Opinionated Tour of Testing Tools: How to Test and Deploy a Python Web App

• 

  Bear [Mike Taylor]Senior Staff Platform Engineer
  

On February 17th, CircleCI held their regular Office Hours Meetup at Heavybit’s San Francisco Clubhouse. This post was written by the evening’s main speaker, Mike ‘Bear’ Taylor.

To get started, let me introduce who I am. My name is Bear and my job at CircleCI is the operations side of the Site Reliability Engineering team. I’ve been using Python to develop applications and ops tools for many decades. When I joined CircleCI, I naturally started to use CircleCI in my Python projects.

One of my goals is to make CircleCI a best-in-class tool for Python Continuous Integration. To do that I decided to create a simple web application and then apply the current best practices for testing and deploys to that project.

This is the result of that research.

This post will not be a deep dive into any one area, nor will it be a battle-of-the-frameworks. What it will be is an opinionated list of what tools and practices I consider useful for developing and deploying a modern Python web application. At the end my goal is to give you some ideas and thoughts about Python testing, even if you don’t agree with them.

This is the path that any app travels while being tested.

The flow is from the Developer, thru components (aka UI), then on to Integration testing and finally into more formal acceptance and system testing. The area of responsibility is divided between the Developer and your Continuous Integration solution.

This slide also points out that testing must be very fast in order for it to be useful. The slower tests are reserved for the parts of the path that do not see a lot of change.

Let’s take a look at Developer Testing, which is the most interesting part. To be honest this part is the least interesting part for me as an Ops person, except when it fails and the CI starts failing and every developer starts posting their favorite version of “works for me.”

If even a few of the items I show you today are followed, then we can start to bring together the dev and ops teams.

The first step towards a sparkly devops princess future is the Makefile.

The makefile is where we document what steps are taken to setup an environment, what steps are needed for each test type and also what steps are needed for deploys. Here is where we discover what is required to get a clean test environment, and what is required to run lint and the different test types. This is also where we document the steps required to setup the development environment.

All which are essential for current developers, and to enable new developers ,to come up to speed.

The info target is useful to help document the environment currently being run which is useful when you are looking back in history to find out what has changed.

Both Flask and Django make use of a management script to run the app. The role of manage.py is to put into a single location what checks are needed before starting the app and what options are available to someone wanting to run the app.

We are going to make use of this to define some custom commands that will be used to start the unit, web and integration tests.

We need to ensure that the developers and qa/ci environments are identical. To do that we use PyEnv to define the virtualenv and to also ensure that it is active for the project.

This removes the one single largest cause of CI/CD pipeline failure: a developer’s laptop having multiple changes to the system’s Python.

Pytest is another key point along our testing path because it allows us to create tests without a lot of boilerplate, to use assertions as they are normally written, and to be able to run xUnit, doctests and other test formats, in a standard manner.

Pytest also gives the ability to mark (using attributes) different tests as web, integration, or unit tests ,which will come in handy later in the pipeline.

Pre-commit hooks are essential to enforcing that our tests, lint’ing and a variety of other validation steps, are performed before the code makes it to the git repo. This will allow everyone in the CI pipeline to feel confident that the code being tested is not going to fail from someone typo’ing a json blob or something equally face-palm-y.

The Python module git-pre-commit-hook is an amazing tool for this and comes out of the box with quite a few plugins to make life more sane.

A majority of the tools covered so far can be customized for the specific environment. The file setup.cfg is used by pytest, flake8, tox and a number of other tools to read in any overrides, so it is worthwhile to keep it current.

Here we are letting flake8 know which warnings and errors it can skip reporting about. We also are defining what markers are present in our tests so pytest can know how to parse the command line we give it later.

In order to properly test the web UI, you need to be able to run the app and have it respond to requests from a web server. In the past this required different staging or testing servers that would constantly be out of date; require that the magic phrase be uttered to the sysadmin just to get that server updated.

Fortunately, now we can take advantage of Docker (and other VM environments) to create, deploy, and run all of the required items on the developer’s laptop.

Docker-compose hides a *lot* of the complexity involved in coordinating multiple vm’s – the ports involved, getting host names to match across vms, setting up network tunnels for port forwarding, etc.

Here we see three docker VM’s being defined: uwsgi, web and chromedriver.

• uwsgi is the VM that runs the app using uwsgi
• web is the VM that will run nginx which then will proxy-pass all requests to the uwsgi VM
• chromedriver is the VM that creates a headless browser environment that can be manipulated by selenium’s web driver

The Build

Lets us store the docker-compose files in the same directory as our application so we can take advantage of the docker tools to inject our app into the VM’s file space. The Links, expose and ports items are all the magic items to let docker know what the networking relationship is between the VM’s – it then goes off and ensures each vm has the proper port configurations.

We are going to use our Makefile to document what the specific commands we need to manage our Docker environment. Docker-build outlines the three steps to creating; storing and cleanup of the vm’s Docker-start shows the command to get the Docker environment up and running webtest runs a bash script that determines what the Docker IP address is.

It’s worth noting this step varies between linux and OS X. Then the script waits for the exposed ports to become available and exits. We then use our management script to run the web tests and afterwards tell Docker to stop.

The best part about going thru the process to deploy our app locally using nginx and uwsgi is that it’s almost the exact same process involved in deploying our app in a production environment.

The nginx config shown here would only need to be tweaked a little bit – such as changing the upstream server configs and also the server_name and listen items. This is all information that the ops team would have to discover the hard way, so having this already documented gives them a heads-up.

Once our application has passed the unit, integration and web UI tests, it is ready for further testing downstream within the CI pipeline. Because we have now proved that the application is functional, we can bundle the app into a named tarball and use that for any further deploys. This removes the need for git credentials to be required.

The tarball and the uwsgi line paired with the uwsgi-app.py are examples of what could be used in acceptance and system testing of our app as a part of other pieces of the system being tested. Our app could be an external requirement for another part of the CI/CD pipeline so this would document the steps required to get our deploy running in that test environment.

Now deploys are not as simple as tossing a few files at a server and crossing your fingers. Fortunately there are already good walk-thrus on how to deploy to Google AppEngine and Google Cloud Compute.

These are tools that would normally be present in any production app’s environment but are hard to describe or demonstrate for a talk.

Tox would be used as part of the Acceptance phase of any sane CI/CD pipeline.

Coverage.py is an amazing tool that looks into your code and shines a light into those dark recesses of old and unused code – all of which are breeding grounds for future bugs.

Mock you will see in this test_owm.py integration test so we can check our use of the external API without having to hammer that API.

Locust.io is a python based load testing tool.

The URLs shown here are very good guides to python testing that go much deeper into the details and whys and how-tos, much more than I ever could.

• https://pages.18f.gov/testing-cookbook/python
• http://docs.python-guide.org/en/latest/writing/tests
• https://pythonhosted.org/testing/

I hope you learned something new about how to test and deploy Python web apps. If you have any questions, feel free to reach out at sayhi@circleci.com.