Skip to main content
Gå til innhold

Fetching Vault secrets from a GitLab pipeline

Things to consider​

Before you start, you must consider:

  • Who will access the secret? Who else, besides the GitLab pipeline, should be able to read/write it?
    This will influence where in Vault the secret should be located. If the secret is located outside the /secret/gitlab tree, your GitLab pipeline will need a special role with an attached policy giving read access to the secret.
  • Do you need to update the secret while your application is running? If yes, then maybe the secret should be read from Vault directly by the application, not the pipeline? Or maybe it's ok to redeploy the application when the secret changes?

Reading a secret from your pipeline​

In the pipeline job that needs the secret, you need to add two sections. The first one is id_tokens, and it should look like this:

  id_tokens:
    VAULT_ID_TOKEN:
      aud: "https://vault.sikt.no:8200"

Then you need to tell the pipeline where in Vault it should look for the secret, by adding a secrets section:

  secrets:
    MY_SECRET:
      token: $VAULT_ID_TOKEN
      vault: "gitlab/mygroup/myproject/SecretKey@secret"

That section will read the key SecretKey from the secret named myproject, located at secret/gitlab/mygroup, and put the value in a file on the GitLab runner. The variable $MY_SECRET will contain the filename.
From the Vault UI, this secret would be available at https://vault.sikt.no:8200/ui/vault/secrets/secret/kv/list/gitlab/mygroup/myproject.

If you don't want GitLab to put the value in a file, you must add file: false to the section.

Example: secret in a file​

Case: SSH key
Let's say our GitLab project is myApp, in the GitLab group myGroup. (Project URL: https://gitlab.sikt.no/myGroup/myApp)
This project needs to run a playbook on a remote server using Ansible.
Ansible will be using SSH to connect to the server as the user deploy, so we need to provide an SSH key for authentication.
We need to fetch the key from Vault, which is located at gitlab/myGroup/myApp. The SSH key is stored in the deploy_ssh_key key of that secret. (Vault URL: https://vault.sikt.no:8200/ui/vault/secrets/secret/kv/list/gitlab/myGroup/myApp)
Here is our job:

run_ansible:
  id_tokens:
    VAULT_ID_TOKEN:
      aud: "https://vault.sikt.no:8200"
  secrets:
    ANSIBLE_PRIVATE_KEY_FILE:
      token: $VAULT_ID_TOKEN
      vault: "gitlab/myGroup/myApp/deploy_ssh_key@secret"
  script:
    - ansible-playbook playbook.yml -i server.example.com, -u deploy

Example: secret in a variable​

Case: Push a docker image to Artifactory
In the same app as the previous example, we build a docker container (myApp), but we need a username and password to publish the docker image to Artifactory. These credentials are stored in Vault, at the same location as the previous example. The key containing the username is named artifactory_username, and the password is stored in artifactory_password. The job could be like this:

build:
  id_tokens:
    VAULT_ID_TOKEN:
      aud: "https://vault.sikt.no:8200"
  secrets:
    ARTIFACTORY_USER:
      token: $VAULT_ID_TOKEN
      vault: "gitlab/myGroup/myApp/artifactory_username@secret"
      file: false
    ARTIFACTORY_PASSWORD:
      token: $VAULT_ID_TOKEN
      vault: "gitlab/myGroup/myApp/artifactory_password@secret"
  variables:
    ARTIFACTORY_HOST: "artifactory.sikt.no"
    VERSION_TAG: "1.2.3"
  script:
    - docker build --label version="${VERSION_TAG}" --label com.jfrog.artifactory.retention.maxDays="180"
    - docker login --username $ARTIFACTORY_USER --password-stdin $ARTIFACTORY_HOST < $ARTIFACTORY_PASSWORD
    - docker push --all-tags $ARTIFACTORY_HOST/docker-public-local/myApp
    - docker logout $ARTIFACTORY_HOST

Secret outside the gitlab-tree in Vault​

If your secret is located somewhere else than the gitlab tree in Vault, you need Platon to create a special role for your pipeline.
Let's say the pipeline needs to read secrets in productareas/myPO/myApp/databases, or maybe service/myService/databases. You would then have to send an email to Platon (hjelp@sikt.no), with subject "Platon: ny Vault rolle for lesing av secret fra GitLab". Remember to supply the following information:

  • the Vault path your project needs access to
  • what permissions (read-only or read-write)
  • what group(s) or project(s) in GitLab need this access
  • a suitable name for the role, not conflicting with existing roles

Please bear in mind to keep the number of roles as few as security allows for your case. For example, maybe several projects (or an entire GitLab group?) can have access to the same subtree in Vault?

You then need to add a special variable to your project. Go to Settings -> CI/CD and expand Variables. Add a variable called VAULT_AUTH_ROLE, containing the name of the role. If all projects in a GitLab group should use the same role, you can make this variable a group-level variable instead of a project-level variable, so you won't have to add it to every project in your group.