Config Management Camp 2024 Ghent

Fun with downstream pipelines and artifacts in GitLab CI
2024-02-06, 16:45–17:35, B.3.036

This talk will present some insights into how to combine pipelines from different projects in GitLab CI.
The goals are

  • to allow single pipelines to run individually and
  • to pass parameters between different pipelines

However, it's not as simple as that, as our second requirement, passing parameters, can easily break the first.

At our organization we have defined many pipelines to solve specific problems, like:

  • providing infrastructure using the IaC tool of our choice (primarily terraform)
  • installing and configuring software using Ansible
  • running tests on software
  • cleaning up test environment

We require that each of these pipelines can be triggered individually but also want to be able to combine them into some super-pipeline, without decreasing maintainability.
In GitLab CI the mechanism for such a combo construct are downstream-pipelines. Sharing data and parameters between these, however, requires well-defined relations between projects - which can potentially break the ability to run the pipelines independently.

I will present my approach to solve this conundrum and describe the boundary conditions for the building blocks implied by the given requirements.

Jan works is Senior orcharhino QA Engineer at ATIX - the Linux & Open Source Company.
He is specialized in infrastructure automation and has several years of experience setting up orcharhino in customer environments to automate their datacenters.

Jan is currently living in Munich - a city most famous for its beer.
Although he is originally from there and was raised with the Munich Helles, he got to know a lot of other beer styles while travelling around the world.
So he started brewing his own beer - of course supperted by a lot of automation.

This speaker also appears in: