Skip to main content
Gå til innhold

Pipelines

Pipelines in Gitlab are defined in the .gitlab-ci.yml file that lives in the main folder of your repo.

.gitlab-ci.yml is a YAML file that provides instructions to GitLab CI/CD, in this file you define the structure and order of jobs that the runner should execute. Conditionals enable the runner to make decisions and jobs themselves execute a set of specified commands. If any job fails (exit status != 0), then the whole pipeline will fail and the following jobs, such as deployment, will simply not execute.

For Platon PaaS this configuration file is crucial and used to build your project on GitLab and then deploy the application to the cluster.

By default, there are three shared GitLab CI/CD runners configured and these are capable of docker-in-docker abbreviated as dind. If you have a large and frequent workload, or high security requirements consider setting up a dedicated runner for just the specific project.

Good to know

Make sure to read Getting started first to get an idea of how to best structure your project.

You may also check out the README.md of the hello-world example.

Official quick start guide: https://docs.gitlab.com/ee/ci/quick_start/index.html
YAML keyword reference: https://docs.gitlab.com/ee/ci/yaml/
Collection of examples: https://docs.gitlab.com/ee/ci/examples/index.html

.gitlab-ci.yml walkthrough​

A fully functional .gitlab-ci.yml configuration can be found in one of the example projects here: .gitlab-ci.yml

The .gitlab-ci.yml configuration file is enough to deploy to PaaS 2.0 on its own, but an advanced walkthrough of the contents can be found below:

include​

The first part of this file uses the "include"-keyword, which imports YAML from an external YAML file into your local CI/CD configuration. An include is used to increase readability and reduce duplication of the same configuration. Make sure to have a look at the configuration file gitlab-ci-helpers.yml in asm/gitlab-ci-helpers to understand what functionality is provided.

# This is a global keyword and needs to be located at the top-level scope
include:
  - project: 'asm/gitlab-ci-helpers'
    file: '/gitlab-ci-helpers.yml'

It is recommended that you use gitlab-ci-helpers.yml as your base configuration as this configuration will be maintained by Platon.

variables​

Secondly, we have some crucial variables that need to be defined. The variables-keyword is used to define environment variables for running CI-jobs.

# This is a global keyword and needs to be located at the top-level scope
variables:
  KUBE_PROD_DOMAIN: hello.paas2.uninett.no
  KUBE_TEST_ID: hello
  HTTP_PORT: '80' # The port number of the web server.
  REPLICAS: '1'

These variables create the Ingress configuration that maps a unique full qualified domain name to your application, which will make it possible to access it from the internet.

  • KUBE_PROD_DOMAIN: This variable will be the final FQDN for the production version of your application.
  • KUBE_TEST_ID: This variable is used to generate test instances of your application. By default, only staging is configured, and in this case the FQDN of staging would be: $KUBE_TEST_ID-staging.paas2.uninett.no. Please note that testing instances of your application are not available to the public.
  • HTTP_PORT: This variable defines what port web requests should be routed to by the load balancer running outside of the Kubernetes cluster. TLS/HTTPS is terminated by the Ingress-controller/load balancer, so you only need to accept HTTP requests on this port.
  • REPLICAS: This variable is used in the deployment.yaml file to define the amount of replication you want for your application. Typically, this is set to 1 for testing, but there is an override in place which sets replicas to 2 for the production deployment. See stage production for more details.

stages​

Thirdly, we have the stages-keyword which defines the order of which GitLab CI/CD should execute the jobs when it runs a pipeline from your commit. It is important to note that all jobs in a single stage can be run in parallel, but that jobs in a stage must be completed before jobs in the next stage are started.

# This is a global keyword and needs to be located at the top-level scope
stages:
  - build
  - test
  - review
  - staging
  - production

In this case we will first run the build stage, which will build the Docker container. After this is finished, we run the test stage which runs some important testing of the containers before continuing. When this is done there are only deployment steps left: review, staging and production.

At every step in our example, we import a template from gitlab-ci-helpers, which implies that you need to understand the contents of gitlab-ci-helpers in order to know what is running in each step. In this case, it is very simplified to make it easy for anyone who only wants to deploy their application and does not worry about the nitty-gritty details. Templates are managed by Platon which means they will be updated over time without the need for any changes in the individual projects.

build​

build:
  extends: .docker-build
  stage: build
  only:
    - branches

In the first stage: build, we run the build job which extends the .docker-build template from the gitlab-ci-helpers YAML file imported at the very top. This job runs on every commit to any branch.

test​

imagescan:
  extends: .imagescan
  stage: test
  only:
    - branches

The imagescan job runs during the test stage to make a static scan of the newly built Docker image for vulnerabilities. This job extends the .imagescan template from the gitlab-ci-helpers YAML file. Like the build job it runs on every commit to any branch.

staging​

# This is a job and needs to be placed at the top-level scope
staging:
  extends: .staging
  stage: staging
  script:
    - deploy deployment.yaml

The staging job runs during the staging stage and deploys the testing version of the application. This job extends the .staging template from the gitlab-ci-helpers file.

Implicit here is that the staging job only runs on the master branch, but this can only be seen in the .stating template. It will be deployed continuously if all stages before it has passed successfully.

One important thing to note here is that the script-keyword configures what commands the job should run. This is the first time we explicitly define what command is being run, and in this case, it is being used to execute the deploy-script from asm/kubernetes-deploy using the contents that have been populated into the deployment.yaml file in the project.

production​

# This is a job and needs to be placed at the top-level scope
production:
  extends: .production
  stage: production
  variables:
    REPLICAS: "2"
  script:
    - deploy deployment.yaml

The production job runs during the production stage and deploys the application to the production environment. This job extends the .production template from the gitlab-ci-helpers file.

Implicit here is that the production job only runs on the master branch, but this can only be seen in the .production template. It is also configured with the manual option, meaning that there is a need for manual intervention on the pipeline from the end-user to trigger the actual deployment. Hence there is no continuous deployment to production by default.

There is a hardcoded REPLICAS: "2" variable set to the production job which overrides the default replicas specification and ensures there are two copies of the web-application once it is deployed to production. This ensures high availability (HA) for the production-level application.

review​

# This is a job and needs to be placed at the top-level scope
review:
  extends: .review
  stage: review
  script:
    - deploy deployment.yaml

The review job runs during the review stage and makes sure a review copy of the application will be deployed when working in a branch separate from master. This job extends the .review template from the gitlab-ci-helpers file.

Implicit here is that the review job will run on any branch, and it will do continuous deployment to the given review environment if all stages before it has passed successfully.

This is essentially what lets you test changes without affecting staging or production. The FQDN of the staging environment will be created by a combination of $KUBE_TEST_ID set as an environment variable at the very top of the file and the branch name as given by the environment from the variable $CI_ENVIRONMENT_SLUG.

stop_review​

# This is a job and needs to be placed at the top-level scope
stop_review:
  extends: .stop_review
  stage: review

The stop_review job runs upon manual intervention during the review stage and makes sure that the created review copy of the application will be deleted when there is no longer any use for it. In order to manually trigger the job click the stop-button inside the review stage from pipeline of the commit that triggered it, which will delete the review copy. The job extends the .stop_review template from the gitlab-ci-helpers file.

When a branch is deleted, for example upon a successful merge request, the review copy environment will be deleted from Platon PaaS.

Additional configuration​

Manual destroy action for staging and production​

Normally there would be no reason to stop the staging and the production environment for your application, but sometimes an application gets scheduled for removal. For example, when it is no longer necessary, superseded or you have discovered a critical security vulnerability.

A manual intervention to destroy the staging and production environments can be added through importing the relevant templates from gitlab-ci-helpers. This is not enabled by default to prevent the accidental deletion of the production application.

# This is a job and needs to be placed at the top-level scope
stop_staging:
  extends: .stop_staging
  stage: staging
stop_production:
  extends: .stop_production
  stage: production

Once the pipeline from your last commit has finished running a stop-button should be visible in the staging and production stage of your pipeline. This works in the exact same way as the stop_review job does for review copies and clicking the stop button will destroy the deployment in that environment.