CERN advances understanding of the universe with the help of GitLab

Established in 1954, CERN first activated the Large Hadron Collider (LHC) in 2008. Called “one of the greatest engineering milestones of mankind,” the collider, a 17-mile, underground, vacuum-sealed loop with a number of accelerating structures, smashes particle beams together to create showers of new particles that replicate conditions in the universe just moments after its conception. The collider is designed to explore concepts like the Big Bang, dark matter, and the Higgs boson.

For years, CERN had used multiple DevOps tools, including GitHub, to build the software needed by the CERN scientific community. But their teams needed a more comprehensive solution: a move to a full end-to-end DevSecOps platform that adds security into their entire software development lifecycle and would increase their efficiency and delivery speed. They adopted GitLab’s platform with GitLab Starter in 2015. With the addition of the platform, the facility’s teams began updating, iterating, and rebuilding all of their software. And every new project from then on was built and deployed with GitLab. To further increase their efficiency and visibility, they upgraded to GitLab Premium in 2020, and then upgraded again, this time to GitLab Ultimate in 2023, to take advantage of its advanced security framework.

“All of our software—the software running our complex, running the collider, creating collisions— is all built on and hosted on GitLab, says Michi Hostettler, Large Hadron Collider engineer-in-charge at CERN. “It’s important software for us. Because of the platform’s integration, automation, issue tracking, security scanning, documentation—everything is developed using the platform. ”

The collider is an integral component of the facility’s infrastructure and scientific mission, so using GitLab-built software to keep it running optimally and to analyze the exabyte of raw data it produces each year is fundamental to what the world-class scientists there do. Understanding that data is fundamental to expanding humans’ knowledge of the world.

“GitLab is considered an important piece of the ecosystem at CERN,” says Ismael Posada Trobo, version control systems tech lead and engineering manager at CERN. “With software built in and hosted on GitLab, we’re simulating particle collisions, and recreating collisions with data from the LHC. It enables scientists to research and to imagine. From a developer’s point of view, GitLab plays a valuable role in getting their work done.”

GitLab’s platform isn’t just being used by DevSecOps teams at CERN to build software. According to Trobo, 30% of their use of GitLab is to connect researchers from universities, laboratories, and research institutes around the world so they can communicate about and collaborate on scientific projects.

For instance, researchers working on the ATLAS experiment, one of the largest collaborative efforts ever attempted in science, are using the platform for their Athena software. The project—designed to study the fundamental constituents of matter, using the full discovery potential of CERN’s collider— involves approximately 6,000 members and 3,000 scientific authors from 182 institutions and 42 countries. This extensive group of researchers distributed across the globe needed a way to visualize each other’s work, offer insights, and work together on problems. GitLab’ DevSecOps platform gave them the ability to do that.

“ATLAS Athena is not just large in terms of lines of code, data being analyzed or the number of scientists involved,” says Zach Marshall, computing coordinator with the ATLAS experiment and senior scientist at the Lawrence Berkeley National Laboratory. “It is, by far, the biggest and most powerful project we have—in every regard. In the end, we needed a way to efficiently and collaboratively work on this massive and dynamic project. That’s why everything we’re doing with it is concentrated in GitLab. It’s important for our scientists. Whether they’re developing software or writing papers, we can easily share knowledge, iterate, and collaborate in real-time.

“This makes the research both quicker and more effective,” he adds. “A huge amount of our collaborative work goes through GitLab. They can work in GitLab, visualizing the project and progress. They can comment and offer feedback. It accelerates our work.”

DevSecOps teams at CERN have been trimming their toolchain, which has included tools like Jenkins, Bamboo, and GitHub. The five-piece toolchain has been completely replaced with GitLab’s single platform, says Trobo.

Eliminating their toolchain saves context switching, along with having to update, manage and pay for multiple tools. And using an end-to-end platform improves the facility’s application and software supply chain security, as well. And that means it also increases security for scientific research, the scientists, and CERN’s valued reputation.

"Like any other organization, university, or company, CERN is under constant attack. Balanced with the academic nature of the organization, we adapt our security posture accordingly to maintain our surety,” explains Trobo. “GitLab helps us set up security policies and compliance frameworks for all the developers and the entire community. That’s significant for us.”

A significant security benefit that CERN gained with GitLab—along with automated scanning, secret detection, and static application security testing—was the ability to visualize the security posture of its applications. With built-in dashboards, teams can automatically see a collection of metrics, ratings, and charts for any vulnerabilities detected by the platform’s security scanners. All of the information is aggregated in one place.

“I get scan results for everything from vulnerabilities, to critical policy findings, and required approval resolution for specific scan policies,” says Trobo. “It’s really useful because the dashboards give us an overview where we can look at all the security findings in one place. We’re now well aware of what is happening in all of our projects. The reports are almost created automagically. With one click, you can make your project more robust.”

A few years ago, CERN’s DevSecOps team members were spending a lot of time building their own runners, which operate builds in pipelines. They only had about 40 runners on up to 80 concurrent jobs throughout their ecosystem. Today, they have three times as many because they rely on GitLab runners, enabling them to automate their software delivery process so they can deliver value faster and quality code more often.

CERN began using more runners in the summer of 2023 when GitLab delivered runner updates as part of the GitLab 16 release. New instances of the lightweight, scalable agents that run CI/CD jobs can be spun up in GitLab, instead of developers having to build their own private runners. The platform also enables teams to re-use runner configurations to register multiple runners with the same capabilities, and to take advantage of Kubernetes, which automates container management and includes commands for deploying applications.

“Now the whole CERN community can use GitLab’s runners, instead of taking the time to build our own,” says Trobo. “Our use of runners is exploding now. This automation is saving developers time, which they now can use to focus on more important work. That saves us money and enables us to scale our software development more efficiently.”

Alejandro Iribarren, a member of the AlmaLinux Board of Directors and an engineering tech lead at CERN, notes that some of their automated workflows are a good example of that.

“These workflows allow us to build new cloud images, instantiate multiple virtual and physical machines, test them with various configurations and, if all the tests pass, automatically promote new images to production— all in one click,” he adds. “Thanks to these CI pipelines, we can now update our cloud images much more frequently, while having much higher confidence in the end result. The biggest value of GitLab CI is that it allows us to reduce the number of things we have to worry about.”

DevSecOps teams at CERN haven’t begun testing artificial intelligence (AI) capabilities in the platform but Trobo says they’re eager to begin doing so. “I hear developers talking about it. They’re asking me when they can use the AI features in GitLab,” he adds. “We know having AI will help our community build better code.”

With GitLab Duo tools like Test Generation, Code Explanation, and Vulnerability Resolution, Trobo says he sees his teams being able to gain efficiency, save money, and be able to spend less time on repetitive tasks and more time on creating innovative software. “In GitLab Duo, you can edit your code on the fly, which helps you gain a lot of speed,” he notes. “If you’re developing and you have something that can help you all along the software development lifecycle, it’s going to add so much efficiency. It’s going to make developers’ jobs easier.

“There is a clear desire to use artificial intelligence here,” says Trobo. “CERN is one of the most advanced scientific organizations, known for using cutting-edge technologies. We need to be at the forefront of using AI.”

CERN team members see GitLab as a key point of development and research, enabling efficient software development, data analysis, and global scientific collaboration. Because of the breadth of their usage and benefits, Trobo sees them only expanding their use of the platform. “When anyone is developing, reviewing, managing projects, tracking issues, doing security scanning, or deploying, they’re doing it in GitLab,” he adds. “No matter what they’re doing, they are working in GitLab. I don’t see that changing.”

“We know having AI will help our community build better code,” says Ismael Posada Trobo, tech lead and engineering manager at CERN. “If you’re developing and you have something that can help you all along the software development lifecycle, it’s going to add so much efficiency. It’s going to make developers’ jobs easier.