This post is part of my series on setting up a self-hosted continuous integration server. Part 0 has a table of contents.

Using Docker containers for Jenkins builds is attractive because it means that any code run as part of your build stays within the container and does not affect the host system. In addition, since the containers and built and torn down every build, it's easy to reproduce the environment used to get results as well.

Prerequisites: Basic Installation of Jenkins and Docker

To start off, I'll presume that you have a minimal installation of Jenkins and Docker. The process isn't too difficult and it's well-documented on other sites, so I'll defer to them. In particular, I recommend this very detailed tutorial on how to install Jenkins on Ubuntu 16.04 and this tutorial on how to secure your Jenkins installation with SSL, courtesy of LetsEncrypt.

The Docker installation process is similarly well-documented. For that, I recommend this tutorial for installation on Ubuntu 16.04 --- make sure that you complete at least steps 1 and 2, but the entire article is a good read if you've never used Docker before.

By the time you move on to the next step, you should have a Jenkins instance ready on your server and should be able to run docker commands without sudo on a normal user account on the server.

Giving the jenkins user Docker permissions

When Jenkins is installed, a service account named jenkins is automatically set up as well. To use Docker in Jenkins, we have to give this jenkins service account permissions for Docker-related operations.

To add the jenkins account to the Docker group:

sudo gpasswd -a jenkins docker  

Now, I'd recommend restarting docker and your server to ensure changes go into effect:

sudo service docker restart  
sudo shutdown -r now

Installing Docker-related Jenkins plugins

Jenkins has a few plugins that make building and running commands within Docker containers a breeze. I'd recommend the following plugins, which you can install by accessing your Jenkins instance, clicking Manage Jenkins on the left hand side, then Manage Plugins:

Docker Commons Plugin
Docker Pipeline
Docker plugin
Docker Slaves Plugin
docker-build-step

With that, you have everything you need in order to start running your Jenkins builds in Docker containers! Next, we'll look through how we can use Jenkins to build a container from a Dockerfile and automatically run some commands in it.

Using Jenkins to test Github code in Docker containers

To begin, create a new repo on Github (either public or private) and clone it to your disk. We'll be using this repo to test out Jenkins.

Specifying the build environment with the Dockerfile

We'll start by setting up a basic Dockerfile that specifies the container that our build will run in. This demonstration Dockerfile below creates a new Ubuntu 16.04 container, installs a few packages from apt-get, and then sets up a Python development environment (with an optional user-defined Python version) with conda:

Copy this file, name it Dockerfile.simple and commit it to your repo.

Configuring the build with the Jenkinsfile

The Jenkinsfile is a file that sits in the root directory of your repository and essentially tells Jenkins what to run (think of it as a travis.yml, if you're familiar with what that is). Jenkinsfiles are written in the Groovy language, which is syntactically like Java but dynamic like Python. You don't have to know Groovy to set up a Jenkinsfile --- I'll walk through it bit by bit, and it isn't hard to pick up via example.

Here's the Jenkinsfile we'll be using, rife with comments:

Similarly, copy this Jenkinsfile to your repo.

Connecting the Github repository to Jenkins

To begin, I'd recommend downloading and setting up the Blue Ocean plugin, which makes creating Jenkins pipelines from Github repositories much easier. To do so, just install the Blue Ocean plugin from the Manage Plugins page in Jenkins.

After installing Blue Ocean, launch it (by either hitting the Blue Ocean buttons on the left hand side of Jenkins or navigating to http://jenkins_server_url/blue/). If it's your first pipeline, there will be a prominently displayed dialogue with Create a New Pipeline. Else, create a new pipeline by pressing the New Pipeline button in the top right. You should be greeted with a screen like the below:

jenkins_blue_ocean_create_pipeline_step_1

Click the Github button. If it's your first time creating a pipeline, you'll be prompted to set up a Github personal access key so Jenkins can access the repository and update the commit status. After you add the personal access key, you should see a screen with all the Github organizations you have access to:

jenkins_blue_ocean_create_pipeline_step_2

Next, it'll prompt you to choose whether to create a new Pipeline from a single repository or create Pipelines for any repo in the organization with a Jenkinsfile. Choose the first option (New Pipeline from single repository).

Choosing it should open a list of the various repositories, public or private, in your user account. Choose the repository that has the Dockerfile.simple and Jenkinsfile from above.

With that, you should be able to hit "Create Pipeline" and Jenkins will go ahead and do so. If you navigate back to the Blue Ocean home, you should see an entry for the repository and that Jenkins began a build (it always does so when first adding a Pipeline).

If things went well, clicking on the build should show something like this:

jenkins_blue_ocean_create_pipeline_step_3

If you see that, congratulations --- you just launched your first containerized Jenkins build. If you don't see that, investigate the cause of the issue and see if you can figure out why it isn't working.

Auto-building from the Github repository

At the current moment, if you were to make a change to your repository and push a new commit, Jenkins would not build it. In order to have Jenkins automatically build our new code after each push, we need to connect it to Github.

To do so, navigate to the settings page of your Github repository. On the left, select the Integrations and Services menu item. Then, use the dropdown menu to add the Jenkins (Github Plugin)

add_jenkins_github_plugin

Lastly, to finish configuring the service, add the Github URL webhook. This should be https://{jenkins-server-url}/github-webhook. You should be able to open that link in your browser and see a java.lang.Exception: Method POST required come up.

configure_jenkins_github_plugin

Hit save, and you're done! Now, future pushes to the repository will automatically trigger Jenkins builds.

What's Next?

In this tutorial, you learned about how to use a basic Dockerfile to configure your build environment and configure your builds with a basic Jenkinsfile. This tutorial barely scratches the surface of what's going on, so here are some additional resources:

Part 2 (coming soon) will cover matrix builds, or how to run tests with multiple different configurations (e.g. running tests with three different versions of Python).