Unified CI/CD platform
Improved collaboration across projects and teams
Interoperability across multiple cloud providers
“ GitLab takes the culture of the community and brings it to where you can actually codify how humans can interact together well. That’s difficult to capture, and I think GitLab does a really excellent job of not forcing people but really encouraging a collaborative beneficial work environment. ”
The Cloud Native Computing Foundation (CNCF) is the largest open source sub-foundation operated under the Linux Foundation. The mission of the CNCF is to create and drive adoption of a new computing paradigm, cloud native computing. As champions of the cloud native movement, the CNCF is dedicated to fostering the growth and evolution of cloud native systems through the stewardship of open source projects including ensuring the technology is available, accessible, integrated, and reliable. The CNCF currently hosts a dozen open source projects such as Kubernetes, Prometheus, and CoreDNS.
The CNCF Continuous Integration (CI) Working Group, lead by Camille Fournier, was established to increase collaboration between all of the CNCF’s projects and to demonstrate best practices for integrating, testing, and deploying projects within the CNCF ecosystem. Each CNCF project already has their own CI system that runs on every commit. However, the foundation was interested in improving inter-project compatibility via a separate CI system.
To improve cross-project and cross-cloud integration the CNCF funded the Cross-Cloud CI Project within the CI Working Group. Led by Denver Williams and Chris McClimans, the Cross-Cloud CI Project was chartered to continuously test the interoperability of each CNCF project, for every commit, across multiple cloud providers and to publish the results on a public dashboard.
The CNCF has experienced large adoption of many of their projects, however, quite a few of them did not have a continuous integration (CI) solution. In addition to needing a continuous integration tool per project, they also needed to find a way for all of the projects to interact. “We needed to be able to do cross-project interaction between pipelines,” said Williams. “The CoreDNS pipeline had to interact with the Kubernetes pipeline.”
Since each project has their own governance, style of interacting, and way of creating releases, the CNCF could not impose a particular CI and enforce each project to be involved. They needed a unified CI solution to integrate all of the projects, even if the project creators weren’t going to be actively involved in configuring the CI.
Initially, they started looking for a way that the different groups could easily add a CI tool to their project that would allow them to integrate their solution with the rest of the team and the rest of the CNCF. However, as they realized the complexity of trying to manage all of the different groups, they needed to find a way to centralize it. To pull off the level of integration and interoperability they were aiming for, McClimans and Williams knew they needed a solution that had an internal Docker registry to store containers, an artifact store, git integration for direct ties with the CI/CD system, and be able to trigger CI pipelines across projects.
“The initial desire was to have a per repo environment configuration that didn't require setting up Jenkins, but could handle the complexity of managing the cross-project configuration. It was a lot simpler to setup and use GitLab,” said McClimans. “We needed something that was easy for people to get started and so they could collaborate together in a meaningful way.”
Starting with a simple yaml configuration file, McClimans and Williams were able to get continuous integration tests up and running on the various CNCF projects. Then, using GitLab’s multi-project pipeline graphs, they were able to integrate the projects together with pipeline triggers allowing them to visualize the upstream and downstream dependencies and relationships between the connected projects in a single view.
“Being able to store artifacts per commit and extend that to store our containers on commit gave us a collection of our binaries and our Docker containers at the end of a pipeline,” said McClimans. “Having that, we could then pass along and trigger another project. This was crucial for us being able to do cross-project configuration and pass that along to our cross-cloud CI for all of the projects. We were not able to do that well without the multi-project pipelines.”
Showing the initial pipelines of the projects combining and passing their tests was a good first step toward showing how the various CNCF projects and teams could work together. “GitLab provided us with a foundation that was really straight forward,” said McClimans. “That’s hard to get—[usually] it’s going to be super custom and more complex to use.”
Next, the group needed to demonstrate deployment best practices per project without imposing custom configuration. Using environment specific variables, Williams and McClimans could build in a standard way and reuse upstream tests without having to create a custom build solution. “When someone makes a commit on a CoreDNS repo and they want to do a deployment, we need to be able to pin a specific environment with exactly which type of cluster CoreDNS is going to be deployed to,” said Williams. “The environment specific variables allows us to go out and do that.”
Choosing GitLab for their CI/CD solution enabled the CNCF to develop an integrated pipeline without a lot of customization and without forcing developers working on specific projects to change their workflow. A developer working on a specific project can now easily provision the right environment to test changes made to their project as well as the impact of that change on other CNCF projects without changing their workflow.
“Empowering your community to focus on their own project but also provide them the ability to combine it with other projects is a difficult problem. Having a group like the CNCF CI Working Group to provide that guidance, and have that conversation so each team is aware of their place and context within the larger group is something that we're still working out as we talk to the different groups within the CNCF,” said McClimans. “Having GitLab available to us so quickly after we had those conversations allowed us to put something together that's meaningful and shows results. It’s something that I've not seen happen as quickly in my life as a DevOps and CI guy,” said McClimans.
The Cross-Cloud CI project is the first of its kind. The project asked new questions and required solving specific problems that didn’t have known answers. For McClimans and Williams, having an open and flexible platform with an active community of contributors was critical to their success.
“It was also important that it was open source,” said McClimans. “There are features that we need that are not there yet, that are not fully on the radar for GitLab, but we were able to modify the source and build our own EE version that had the features that we needed. That really contributed to the success of our project. It's not often that a company offers their enterprise source code for use.”
“See if you can prototype using the tools that are out there before building everything on your own. Just come at it from that perspective you might find something like GitLab versus taking another tool and saying, ‘It works OK, let me build everything else that's missing, versus finding out that GitLab has all of these other features that you're able to configure and put together and get as far along as the project is able to go.” — Taylor Carpenter