Continuous Deployment of a Python Flask Application with Docker and Semaphore

Source – semaphoreci.com

Introduction

In this tutorial, we’ll go through the continuous integration and deployment of a dockerized Python Flask application with Semaphore. We’ll deploy the application to Heroku.

Continuous integration and deployment help developers to:

• Focus on developing features rather than spending time on manual deployment,
• Be certain that their application will work as expected,
• Update existing applications or rollback features by versioning applications using Git, and
• Eliminate the it works on my machine issue by providing a standardized testing environment.

Docker is an application containerization tool that allows developers to deploy their applications in a uniform environment. Here are some of the benefits of using Docker:

• Collaborating developers get to run their applications in identical environments that are configured in the same way,
• There is no interference between the OS environment and the application environment,
• Application portability is increased, and
• Application overhead is reduced by providing only the required environment features, and not the entire OS, which is the case with virtual environments.

Docker works by utilizing a Dockerfile to create images. Those images are used to spin up containers that host and run the application. The application can then be exposed by using an IP address, so it can be accessed from outside the container.

Prerequisites

Before you begin this tutorial, ensure the following is installed to your system:

• Python 2.7 or 3.x,
• Docker, and
• A git repository to store your project and track changes.

Setting Up a Flask Application

To start with, we’re going to create a simple Flask todo list application, which will allow users to create todos with names and descriptions. The application will then be dockerized and deployed via Semaphore to a host of our choice. It will have the following directory structure:

app/
├── templates/
│   └── index.html
├── tests/
│   └── test_endpoints.py
├── app.py
├── Dockerfile
├── docker-compose.yml
├── requirements.txt

The app.py file will be the main backend functionality, responsible for the routing and view rendering of HTML templates in the templates folder.

First, we’ll set up a Python environment following this guide, and create a virtual environment, activate it and install the necessary requirements. A virtual environment in Python applications allows them to have their own runtime environment without interfering with the system packages.

$   virtualenv flask-env
$   . flask-env/bin/activate
$   pip install -r requirements.txt

Creating Application Tests

Test Driven Development (TDD) makes developers consider the structure and the functionality of their application in different situations. Also, writing tests reduces the amount of time a developer needs to spend manually testing their application by enabling them to automate the process.

We’ll set up the test environment using the setUp(self) and tearDown(self)methods. They allow our tests to run independently without being affected by other tests. In this scenario, every time a test runs, we create a Flask test application. We also clean the database after every test in the tearDown(self)method. This ensures that the data stored by the previous test does not affect the next test.

# tests/test_endpoints.py

from app import app, db
from flask import url_for
import unittest


class FlaskTodosTest(unittest.TestCase):

    def setUp(self):
        """Set up test application client"""
        self.app = app.test_client() 
        self.app.testing = True

    def tearDown(self):
        """Clear DB after running tests"""
        db.todos.remove({})

In this section, we’ll write tests for our endpoints and HTTP methods. We’ll first try to assert that when a user accesses the default homepage(/) they get an OK status(200), and that they are redirected with a 302 status after creating a todo.

# tests/test_endpoints.py

class FlaskTodosTest(unittest.TestCase):
    # ..... setup section.....

    def test_home_status_code(self):
         """Assert that user successfully lands on homepage"""
        result = self.app.get('/')
        self.assertEqual(result.status_code, 200)

    def test_todo_creation(self):
        """Assert that user is redirected with status 302 after creating a todo item"""
        response = self.app.post('/new',
                                 data=dict(name="First todo",
                                           description="Test todo")
                                 )
        self.assertEqual(response.status_code, 302)


if __name__ == '__main__':
    unittest.main()

The tests can be run using nosetests -v.

Creating the Actual Application

The application uses MongoDB hosted on mlab, which can be changed in the configuration. It provides two routes. The first one, default/index route(/), displays the available todos by rendering a HTML template file. The second route, (/new), accepts only POST requests, and is responsible for saving todo items in the database, and then redirecting the user back to the page with all todos.

# app.py

import os
from flask import Flask, redirect, url_for, request, render_template
from pymongo import MongoClient

app = Flask(__name__)

# Set up database connection.
client = MongoClient(
    "mongodb://username:password@database_url:port_number/db_name")
db = client['db_name']


@app.route('/')
def todo():
    _items = db.todos.find()
    items = [item for item in _items]
    # Render default page template
    return render_template('index.html', items=items)


@app.route('/new', methods=['POST'])
def new():

    item_doc = {
        'name': request.form['name'],
        'description': request.form['description']
    }
    # Save items to database
    db.todos.insert_one(item_doc)

    return redirect(url_for('todo'))


if __name__ == "__main__":
    app.run(host='0.0.0.0', debug=True)

We can then run the application with python app.py, and access it in our browser localhost http://127.0.0.1:5000. If no other port ID is provided, Flask uses port 5000 as the default port. To run the application on a different port, set the port number as follows:

app.run(host='0.0.0.0', port=port_number, debug=True)

Dockerizing the Application

Docker is used to create the application image from the provided Dockerfile configuration. If this is your first time working with Docker, you can follow this step-by-step tutorial to learn more about installing Docker and setting up the environment.

Dockerfile

FROM python:2.7         
ADD . /todo
WORKDIR /todo
EXPOSE 5000
RUN pip install -r requirements.txt
ENTRYPOINT ["python", "app.py"]

The Dockerfile dictates the environmental requirements and application structure.

The application will run in a Python 2.7 environment. A folder named todo is created and set as our work directory.

Since the Flask application is running on port 5000, this port will be exposed for mapping to the external environment. Application requirements are installed within the container. The application will be run using python app.pycommand, as specified by the ENTRYPOINT directive.

All of the above happens within the Docker container environment, without interference with the OS environment.

Docker Compose is a tool for defining and running multi-container Docker applications. The docker-compose file is used to configure application services by specifying the directory with the Dockerfile, container name, port mapping, and many others. Those services can then be started with a single command.

web:
  build: .
  container_name: flock
  ports:
    - "5000:5000"
  volumes:
    - .:/todo

The build command directs Compose to build the application image using the Dockerfile in the current folder, and map the application port 5000 in the container to port 5000 of the OS. We then build and run our application in a Docker container.

$    docker-compose build
$    docker-compose up

Docker downloads the necessary dependencies, builds up the image, and starts the application in a container accessible at http://127.0.0.1:5000.

Continuous Integration and Deployment (CI/CD)

With CI/CD, developers set up a pipeline for testing and deployment. This allows them to concentrate on developing the features, since the application is automatically built, tested, and deployed from a CI server whenever some changes are made.

To create a new project, log into your Semaphore account, click on Create new, and choose Project on the drop down list.

On the next page, choose whether your project repository is hosted on GitHub or Bitbucket.

Select the project repository by searching for it in the provided filter.

Next, select which branch to load:

After you’ve selected the project owner, Semaphore will analyze the repository and detect the platform:

Analyzing repository

Detecting Platform

Semaphore automatically detects Docker projects and recommends using the Docker platform for the application. You then need to provide project settings in order to define the commands that should be run. Semaphore automatically runs the commands to build an image, and runs the tests before deployment. This ensures that an application version is deployed only if it passes all the tests.

After the build and the tests have completed, the application can be deployed to the chosen platform.

Click on Set Up Deployment and choose the deployment platform.

A complete deployment to Heroku looks as follows:

You can choose to have automatic deployment on subsequent changes. Every time any changes are pushed to GitHub, a build is triggere, and automatic deployment occurs. However, for the first deployment we will need to do it manually.

The application is finally launched on Heroku.

Conclusion

The advantages of continuous integration range from reducing the amount of work done by developers to automatic updates and reduced errors in the application pipeline. Docker enhances this by allowing the provision of uniform environments for running applications.

In this tutorial, we explored how you can create a Flask application and run it using Docker. We also learned how to use Semaphore to create a pipeline that automates running the tests and the necessary build commands, as well as deploying the application to Heroku.