Docker… my precious

Continuous Integration: From Jenkins to GitLab

Jenkins job:

Let’s have a look on our Jenkins Job:

MONGOPORT=23500
BASEDIR=`pwd`

# Update tools
(cd repo && composer update)

# Run code style checker
./repo/vendor/bin/phpcs -p --standard=repo/vendor/sabre/dav/tests/phpcs/ruleset.xml --report-checkstyle=checkstyle.xml repo/lib/

# Cleanup
rm -rf mongodb
rm -f mongo.pid
rm -f mongo.log
mkdir -p mongodb


# Start temporary mongo server
mongod --dbpath mongodb \
       --port $MONGOPORT \
       --pidfilepath $BASEDIR/mongo.pid \
       --logpath mongo.log \
        --fork

sleep 2

# Configure
cat <<EOF > repo/config.json
{
  "webserver": {
    "baseUri": "/",
    "allowOrigin": "*"
  },
  "database": {
    "esn": {
      "connectionString" : "mongodb://localhost:$MONGOPORT/",
      "db": "esn",
      "connectionOptions": {
        "w": 1,
        "fsync": true,
        "connectTimeoutMS": 10000
      }
    },
    "sabre": {
      "connectionString" : "mongodb://localhost:$MONGOPORT/",
      "db": "sabredav",
      "connectionOptions": {
        "w": 1,
        "fsync": true,
        "connectTimeoutMS": 10000
      }
    }
  },
  "esn": {
    "apiRoot": "http://localhost:8080/api"
  }
}
EOF

# Run unit tests
(cd repo/tests && ../vendor/bin/phpunit \
    --coverage-clover=$BASEDIR/clover.xml \
    --log-junit=$BASEDIR/junit.xml \
    .)

# Clean up
kill `cat mongo.pid`

I know, it is horrible to read this configuration, but you know we have to configure everything from A to Z in Jenkins. I can confirm that this job is one of the simplest job we have, cause it depends on only one external service “MongoDB”. We passed almost half of this job configuring this external service, starting it, cleaning it and killing it. Whereas, our main job is only about 10 lines. Furthermore, we suppose that on the Jenkins machine, we already have installed PHP, all PHP plugins and composer. So if we change the machine we have to reconfigure the new machine before starting using it. Docker… help please.

GitLab job:

Before starting, it worth mentioning that a good documentation about this part is presented here. If you got it right, all GitLab’s CI configuration is to be done in a file called .gitlab-ci.yml. I will start presenting the final result before discussing the details:

image: linagora/php-deps-composer:5.6.30

services:
  - mongo:3.2

stages:
  - build
  - deploy_dev

build:
  stage: build
  script:
    - composer up
    - cp config.tests.json config.json
    - ./vendor/bin/phpcs -p --standard=vendor/sabre/dav/tests/phpcs/ruleset.xml --report-checkstyle=checkstyle.xml lib/
    - cd tests
    - ../vendor/bin/phpunit --coverage-clover=${CI_PROJECT_DIR}/clover.xml --log-junit=${CI_PROJECT_DIR}/junit.xml .

deploy_dev:
  stage: deploy_dev
  only:
    - master
  script:
    - cd /srv/sabre.dev
    - git fetch --all
    - git checkout ${CI_COMMIT_SHA}
    - composer up

Here is the migration procedure:

• We start defining the image of which the GitLab’s Docker executor will run to perform the CI tasks. This is done by using the image keyword (line 1). This is a custom image we build to provide all the dependencies we need for our CI tasks. Here is the corresponding Dockerfile

FROM php:5.6.30

MAINTAINER Linagora Folks <lgs-openpaas-dev@linagora.com>

RUN apt-get update && \
    apt-get -y install unzip git php5-curl php5-dev php-amqplib && \
    docker-php-ext-install bcmath && \
    pecl install mongo && \
    docker-php-ext-enable mongo && \
    curl https://getcomposer.org/installer | php && \
    mv composer.phar /usr/local/bin/composer.phar && \
    ln -s /usr/local/bin/composer.phar /usr/local/bin/composer

• As I mentioned before, our CI requires an external MongoDB service. But this time, Docker is here to do the magic. It helps us configuring, starting and killing the service correctly. All what we have to do is to declare mongo as a service (line 4), et Voila!.
• Now we have setup our environment, we can leverage script tag to test our code (lines 13–17) and deploy it (lines 24–27). It is worth noting that config.test.json contains all the configuration we have had in Jenkins (Lines 28–56 from Jenkins configuration)

Running the GitLab job locally:

We can easily test our GitLab builds locally using GitLab Runner. Here is the procedure:

• Install it locally, either using a package repository or directly from here. If you do not want to install the GitLab Runner locally, you can always leverage Docker to do so. Have a look here.
• Run the build: gitlab-runner exec docker {my-job}. Whereas, my-job is the name of the job defined in .gitlab-ci.yml. In our case, it is called build.

Wrap up

As you can see, our CI job becomes easier to read thanks to GitLab and Docker. Along the same lines, we do not need anymore to configure our machine to run tests. Docker gets our back. In my own opinion, the must important advantage using Docker to run tests is to guarantee that our tests are always being run in the same conditions each time. These tests, are totally isolated (and also independent) from the machine on which they run.