Walkero

What I love to do when I develop is to optimise as much as possible my time by finding tools that fit my needs and automating processes. The reason is that I like to be involved in many different projects running in parallel, and most of the time these are based on completely different technologies, from PHP and JS to C, Python and Docker containers. I like to keep my hands "dirty" by working on various projects all the time. One thing that I do not change though is the tool I use for automated pipelines and I would like to talk about it in this post.

For the past 3 years, I am using Drone CI as my CI/CD product of choice. Professionally I have used other solutions like Jenkins and Bamboo and I have a good knowledge of alternative solutions like GitHub actions. One of the main reasons I am using this solution is the perfect integration with Docker. You see, Drone CI is an open-source (Community Edition) product that mainly utilises Docker containers for everything. Even Drone itself and its runners are Docker containers. This means I can run it anywhere Docker works, in a matter of minutes.

There is a plethora of available plugins for any job I might need to do. So, if I want to use Ansible scripts or deploy my code to AWS, there are plugins for that. Do I want to keep my team informed on Slack or Discord? This is also already available. There is a whole Marketplace where I can go and find the plugins I need, with information on how I can use them. And guess what, these are Docker images as well. And if something does not cover my needs, I can find its source and extend it or even build my images that I can use like every plugin.

For setting up the Drone CI I use a docker-compose.yml file and I define it as can be seen below:

version: "3"

services:
  drone-server:
    image: drone/drone:${DRONE_TAG}
    restart: unless-stopped
    container_name: "${PROJECT_NAME}_drone_server"
    environment:
      DRONE_GITHUB_CLIENT_ID: ${DRONE_GITHUB_CLIENT_ID}
      DRONE_GITHUB_CLIENT_SECRET: ${DRONE_GITHUB_CLIENT_SECRET}
      DRONE_RPC_SECRET: ${DRONE_RPC_SECRET}
      DRONE_SERVER_HOST: ${PROJECT_BASE_URL}
      DRONE_SERVER_PROTO: https
      DRONE_TLS_AUTOCERT: "true"
      DRONE_USER_FILTER: walkero-gr
      DRONE_USER_CREATE: username:walkero-gr,admin:true
      DRONE_LOGS_TRACE: "false"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /var/projects/drone-github/data:/data
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.${PROJECT_NAME}_droneserver.rule=Host(`${PROJECT_BASE_URL}`)"
      - "traefik.http.routers.${PROJECT_NAME}_droneserver.entrypoints=websecure"
      - "traefik.http.routers.${PROJECT_NAME}_droneserver.tls.certresolver=le"
      - "traefik.http.routers.${PROJECT_NAME}_droneserver.tls=true"
      - "traefik.http.middlewares.https-redirect.redirectscheme.scheme=https"
      - "traefik.http.routers.${PROJECT_NAME}_droneserver-web.middlewares=https-redirect"
      - "traefik.http.routers.${PROJECT_NAME}_droneserver-web.rule=Host(`${PROJECT_BASE_URL}`)"
      - "traefik.http.routers.${PROJECT_NAME}_droneserver-web.entrypoints=web"

As you can see from the labels block, I use traefik as a reverse proxy server, to manage the domains of the Docker containers and the usage of Let's Encrypt SSL for them. This way I can use Drone with HTTPS, which is necessary to connect it with GitHub and provide access to my projects.

The server itself is not that useful if you do not have runners, who are responsible for running the jobs in the pipelines. As a general example, I define a runner in the same docker-compose.yml file as below.

  drone-runner:
    container_name: drone-runner
    image: drone/drone-runner-docker:${DRONE_RUNNER_TAG}
    command: agent
    restart: always
    depends_on:
      - drone-server
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      DRONE_RPC_HOST: ${PROJECT_BASE_URL}
      DRONE_RPC_PROTO: https
      DRONE_RPC_SECRET: ${DRONE_RPC_SECRET}
      DRONE_RUNNER_CAPACITY: 2
      DRONE_RUNNER_MAX_PROCS: 2
      DRONE_RUNNER_NAME: "github-agent-init"
      DRONE_LOGS_TRACE: "false"

The awesome thing with the runners is that you can have them installed anywhere and connect them to the main server. This way I have a Raspberry Pi in my home network that I use with the Drone CI for building Arm-based Docker images. I plan to write more about runners in a future post.

One more reason why I prefer to use a solution like Drone CI is that I use different Git providers. Right now I have some projects on my GitHub account, as well as on Bitbucket. But I also keep my Gitea server where I have a few experimentations and Amiga projects based mainly on C/C++. By using Drone CI I keep myself free of any pipelines those platforms provide, using the same environment for all of them. This way I can reuse parts of the pipeline code on multiple projects by just copy-pasting.

To set up a pipeline on any project, you only need to connect it with Drone and include a YAML file at its repo, that contains info on the steps and the jobs that need to be executed based on events. By default, this file is named .drone.yml and should be at the root of the repo, but it can be changed easily inside the Drone CI project settings. I will get into more detail on this kind of files in future posts because they are extremely convenient for setting up my pipelines.

But there are some things I would like to see coming in the future by the Drone development team. One of them is a way to have multiple YAML files instead of a single one that sooner or later will be difficult to maintain. This way different pipelines could be in different YAML files, responsible for one or more events, i.e. a pipeline for the pull requests, or a pipeline for deployment to production. Now, Drone provides templates, to make pipelines' maintenance easier for big companies. These can be manipulated with Jsonnet or Starlark scripts, which to be honest I haven't gotten deeper into them. But I believe a YAML solution would be easier for everyone, like the ones we see on GitHub actions.

As I said earlier, I use GitHub, BitBucket and Gitea for my projects. The Drone CI right now is not able to support all of them in one installation, but I need to have three different installations, one for each. That makes the maintenance a little bit more difficult, although since they run on Docker is a matter of a docker pull and update. But still, it would be more convenient if there was a central installation for all of them.

Last but not least, although Drone supports a lot of GIT platforms, none of them uses Subversion. Some people prefer to use SVN for their projects and I'd love to see that supported as well in the future.

Of course, no solution is perfect and no platform suits everyone. I tried above to let you know why I prefer to use the Drone CI for my projects and what benefits I am having by using it. Maybe it fits you as well. If you want to learn more, please watch this space because I plan to add more info in the future.