Setting up CI/CD for Docker and Kubernetes Using Drone

I have been a Travis CI user. However, Travis has gotten less reliable for me lately. On top of that, I have qualms about how the acquisition of Travis by Idera, and the subsequent layoffs, were handled. Travis is also a square peg in the octagonal hole of my Kubernetes environment. It is a hosted, external service. Everything else I use to develop my applications is hosted inside of my cluster. My Docker registry and my gitops operator run in my cluster. My databases are in my cluster. My storage provider and object store run in cluster. My apps run in cluster. Why would I run my CI/CD service outside of the cluster?

I came across Drone in my research about alternatives. Drone is a fully container-native, container-loving CI solution. It's Docker all the way down. Since my application is already "Dockerified", my hosting environment is all Docker all day, and my deployments are already in the form of a Docker push, why not do CI/CD in Docker as well?

Drone is basically a small framework for running CI jobs made of docker containers. You build your pipeline as a series of steps, each of which is a docker base image, some configuration, and your test commands. Drone also has all the standard integrations you would expect for a CI service-- it talks to GitHub, GitLab, Bitbucket, and more.

To use Drone, you'll have to embrace the Docker way. Let go of your test scripts that are building everything from a ubuntu image or a language version manager. Let go of the "special case magic" way DBs and other supporting services are handled in other CI platforms. In exchange, you'll find that Drone will let you use any language, any tools, and any languages, so long as they have a Docker image.

Drone Setup on Kubernetes

I followed this official (but experimental) guide for setting up Drone on Kubernetes.

Here's the configuration I used for Kubernetes:

1 ---
2 apiVersion: v1
3 kind: Namespace
4 metadata:
5 name: drone
6 ---
7 kind: Deployment
8 apiVersion: extensions/v1beta1
9 metadata:
10 labels:
11 app: drone
12 name: drone
13 namespace: drone
14 spec:
15 replicas: 1
16 template:
17 metadata:
18 labels:
19 app: drone
20 name: drone
21 namespace: drone
22 spec:
23 containers:
24 - name: drone
25 image: drone/drone:1.0.0
26 env:
27 - name: DRONE_KUBERNETES_ENABLED
28 value: "true"
29 - name: DRONE_KUBERNETES_NAMESPACE
30 value: "drone"
31 - name: DRONE_GITHUB_SERVER
32 value: "https://github.com"
33 - name: DRONE_GITHUB_CLIENT_ID
34 value: "REDACTED"
35 - name: DRONE_GITHUB_CLIENT_SECRET
36 value: "REDACTED"
37 - name: DRONE_RPC_SECRET
38 value: "REDACTED"
39 - name: DRONE_SERVER_HOST
40 value: "REDACTED.example.com"
41 - name: DRONE_SERVER_PROTO
42 value: "https"
43 - name: DRONE_USER_FILTER
44 value: "prehnRA"
45 - name: DRONE_USER_CREATE
46 value: username:prehnRA,admin:true
47 - name: DRONE_DATABASE_DRIVER
48 value: postgres
49 - name: DRONE_DATABASE_DATASOURCE
50 valueFrom:
51 secretKeyRef:
52 name: drone-postgres-url
53 key: url
54 ports:
55 - containerPort: 80
56 - containerPort: 443
57 ---
58 apiVersion: v1
59 kind: Service
60 metadata:
61 labels:
62 app: drone
63 name: drone
64 namespace: drone
65 spec:
66 ports:
67 - name: http
68 port: 80
69 protocol: TCP
70 targetPort: 80
71 selector:
72 app: drone
73 type: LoadBalancer
74 ---
75 apiVersion: extensions/v1beta1
76 kind: Ingress
77 metadata:
78 annotations:
79 ingress.kubernetes.io/ssl-redirect: "true"
80 nginx.ingress.kubernetes.io/ssl-redirect: "true"
81 kubernetes.io/tls-acme: "true" # enable certificates
82 certmanager.k8s.io/cluster-issuer: letsencrypt
83 kubernetes.io/ingress.class: "nginx"
84 labels:
85 app: drone
86 name: drone
87 namespace: drone
88 spec:
89 rules:
90 - host: REDACTED.example.com
91 http:
92 paths:
93 - backend:
94 serviceName: drone
95 servicePort: 80
96 path: /
97 tls: # specify domains to fetch certificates for
98 - hosts:
99 - REDACTED.example.com
100 secretName: drone-tls
101 ---
102 apiVersion: kubedb.com/v1alpha1
103 kind: Postgres
104 metadata:
105 name: drone-postgres
106 namespace: drone
107 spec:
108 version: "10.2-v1"
109 storageType: Durable
110 storage:
111 storageClassName: "rook-block"
112 accessModes:
113 - ReadWriteOnce
114 resources:
115 requests:
116 storage: 256Mi
117 terminationPolicy: DoNotTerminate

Note that I use kubedb in my cluster, so I am able to request a new Postgres database through Kubernetes configuration YAML. If you don't use kubedb or similar, you'll have to provide a db to Drone differently. By default, Drone uses a sqlite3 database, but this isn't much good in Kubernetes by default, because if you Drone pods get restarted, you will lose your configuration and job history.

I give the Postgres url to Drone via a Kubernetes secret called drone-postgres-url in the url key. This secret must be in the drone namespace. Here's how you can do that:

1 echo -n 'YOUR_POSTGRES_URL' > ./url
2 kubectl create secret generic drone-postgres-url -n drone --from-file=./url

Testing and Deploying An App

The application I wanted to test and deploy through Drone is an Elixir app that is released by semantic-release as a Docker image. I've written previously on the Revelry blog about how to do that.

You configure Drone using a .drone.yml file. The main YAML object defined in that file is a "pipeline", which defines a series of steps and services which Drone will use to run your tests and deploy your app.

A basic pipeline for my app might look like this:

1 ---
2 kind: pipeline
3 name: default
4
5 steps:
6 - name: backend
7 image: elixir:1.8.0-alpine
8 commands:
9 - mix local.hex --force && mix local.rebar --force
10 - export MIX_ENV=test
11 - mix do deps.get, deps.compile, compile, phx.digest, ecto.create, ecto.migrate, test

There's a problem: my tests won't pass without a working database, and we don't have one yet. In Drone, the way to get a supporting container for something like a database or a cache is via a service. Services are also just a Docker container which runs with a certain configuration. Drone will run them before your steps. Other parts of your pipeline can communicate with services over a network (services are given a hostname that matches their service name) or via a shared volume.

Here's what the same pipeline looks like with a mariadb service:

1 ---
2 kind: pipeline
3 name: default
4
5 steps:
6 - name: backend
7 image: elixir:1.8.0-alpine
8 commands:
9 - mix local.hex --force && mix local.rebar --force
10 - export MIX_ENV=test
11 - mix do deps.get, deps.compile, compile, phx.digest, ecto.create, ecto.migrate, test
12
13 services:
14 - name: cms-database
15 image: mariadb
16 ports:
17 - 3306
18 environment:
19 MYSQL_DATABASE: "cms_test"
20 MYSQL_USER: "REDACTED"
21 MYSQL_PASSWORD: "REDACTED"
22 MYSQL_RANDOM_ROOT_PASSWORD: "yes"
23

I also configured my test suite to use a mariadb database at the host cms-database, port 3306, with the given username and password.

My tests pass!

Next, I need to add my deployment step. As I mentioned before, this app deploys as a docker image, via semantic-release and semantic-release-docker. In order to do that, I need to use a "docker in docker" image-- which is just what it says, a Docker image containing the Docker daemon and Docker CLI.

It's actually best to use two of these. Drone (and Docker) prefer if any long running services, such as the Docker daemon, run in their own containers. In my experience, trying to run the Docker daemon in the background of a container that is also doing other commands overcomplicates things. Running the Docker daemon in an isolated container works better, because Docker has provided out of the box initialization scripts for that scenario.

Here's the pipeline with both "dind" parts added:

1 ---
2 kind: pipeline
3 name: default
4
5 steps:
6 - name: backend
7 image: elixir:1.8.0-alpine
8 commands:
9 - mix local.hex --force && mix local.rebar --force
10 - export MIX_ENV=test
11 - mix do deps.get, deps.compile, compile, phx.digest, ecto.create, ecto.migrate, test
12 - name: deployment
13 image: docker:dind
14 volumes:
15 - name: dockersock
16 path: /var/run
17 environment:
18 GH_TOKEN:
19 from_secret: gh-token
20 DOCKER_USERNAME: ci
21 DOCKER_PASSWORD:
22 from_secret: docker-password
23 commands:
24 - apk add nodejs nodejs-npm openssl git
25 - npm install -g npm
26 - (cd assets; npm install; npm run deploy)
27 - npm install
28 - docker build --network=host . -t REDACTED.example.com/my_repo/my_app
29 - npx semantic-release
30
31 services:
32 - name: cms-database
33 image: mariadb
34 ports:
35 - 3306
36 environment:
37 MYSQL_DATABASE: "cms_test"
38 MYSQL_USER: "REDACTED"
39 MYSQL_PASSWORD: "REDACTED"
40 MYSQL_RANDOM_ROOT_PASSWORD: "yes"
41 - name: docker
42 image: docker:dind
43 privileged: true
44 network: host
45 hostNetwork: true
46 mtu: 1200
47 volumes:
48 - name: dockersock
49 path: /var/run
50
51 volumes:
52 - name: dockersock
53 temp: {}

That's a lot of new pieces. What's going on? Well, we've added a docker:dind step to the pipeline. This will run our Docker commands, for building and pushing the Docker image. We also need npm and NodeJS, because semantic-release is a Node package. We install npm and update it to the latest version for good measure. Then (cd assets; npm install; npm run deploy) builds my assets for production. npm install installs semantic-release and the various plugins I use (see my article from the Revelry blog). I build the Docker image. Then, I run semantic-release. Since I'm using semantic-release-docker, it will login to my Docker repo and push the image.

In order to actually build the image, we need a running Docker daemon. That's where the second dind comes in. This one is a service. It runs in privileged mode (required for the dind daemon), which means my Drone project must be flagged as "trusted." We use a shared volume to allow the Docker CLI running in my deployment step to communicate with the Docker daemon running as a service.

I need to provide some credentials for GitHub and for my Docker registry. I do that by exposing them as environment variables. Since I don't want to check in a sensitive credential to git, I use Drone secrets for passwords and tokens. You can set the value of the secret in the project settings UI.

When you put it all together, Drone will do the following sequence:

Other Drone Features

I had a lightbulb moment while I was working with Drone. I had been wondering things like "I wonder which DBs Drone supports" or "I wonder which languages Drone supports." Then I realized: Drone supports everything that has a Docker image. To me, this makes it a more flexible and powerful tool than a CI architected like Travis or Codeship. In those CIs, you either have to wait for official support for your language (or service), or you have to hack together a working test script from a base intended for a different language or DB. The way Drone does this means that Drone immediately has a "feature list" longer than I can write here.

Beyond this, there are other Drone features worth mentioning:

The only real issue that I hit along the way is that I could not get Drone to accept an encrypted secret from the .drone.yml file. Supposedly, Drone supports encrypting secrets via the CLI and including the encrypted version in your .drone.yml. I could not get that to work. I had to use the database secrets method instead.

The Future of Drone

After experimenting with Drone, I think we're going to see it get a lot of traction. It's such a powerful tool, and it so well leverages the Docker ecosystem, that I can't help but think that it is the future of CI.','Setting up CI/CD for Docker and Kubernetes Using Drone | Robert Prehn','Drone is a CI/CD framework that is all in on Docker. In this article, I will describe how I switched my application off of Travis CI, and onto a container-native CI using Drone, Docker, and Kubernetes.