- Science & Research
Optimized for Kubernetes
Increased operational efficiency
Clarity and collaboration for stakeholders
Improved CI workflow
Replacing five tools with one
The Square Kilometre Array (SKA) is a project to build large telescopes in Australia and South Africa. It is an expansive collaboration involving about 20 countries in order to deliver this scientific undertaking. There are approximately 1,000 people partnering internationally to create a panoramic view of the sky with SKA.
SKAO, the organization leading that project, is based in Manchester, England with 100 direct employees. However, it is in the planning stages to become the SKA Observatory, an intergovernmental organization established by treaty, to undertake the construction and operation of the telescope within the next few years. In the meantime, there are telescopes around the globe that act as demonstrators for SKA.
“The SKA will conduct transformational science and help to address fundamental gaps in our understanding of the universe, including the formation and evolution of galaxies, fundamental physics in extreme environments, and the origins of life,” said Marco Bartolini, Lead Software Architect.
“Nowadays a telescope, as with everything else in the world, is 50% software. It is really a software machine. So there are some big software development efforts going on to build the Square Kilometre Array,” Bartolini explained. SKAO has the authority over the design and technical choices, however, the planning and execution of the projects are carried out with collaboration from around the world. The software development efforts leading to the SKA construction are also globally distributed.
SKAO requires a set of tools that embraces open source collaboration in order to standardize practices across a variety of cultures, geographic locations, and timezones. The software team was looking for consistency in the development processes.
SKAO had worked with a number of different tools and platforms, including CircleCI, Jenkins, and Travis. The added plug-ins and integrations required significant management time and effort that was taken away from the scientific project. The team required a platform that provides transparency, shared ownership, a single source of truth, and operational efficiency in order to control the proliferation of DVCS systems and CI server technologies.
“The large success is having been able to onboard code and software projects from many different organizations and with very different tools and technology into one single platform, easily. It was not a pain, and now we got it all under control. So that's brilliant.”Marco BartoliniLead Software Architect
The software prototypes that were in place had been developed by various people around the world including different tools, techniques, and platforms. SKAO intended to adopt a single coherent toolset. They initially selected GitHub, but soon realized that everyone was using different servers for continuous integration, which didn’t solve the need for a single platform providing unified standards.
SKAO then tested GitLab and found that it offers multiple solutions in one tool. “Now we have everything in GitLab. All the CIs are running in GitLab. Every day, I can see in one single view what’s happening in terms of coding activity, in terms of testing. It’s really a big improvement for us,” Bartolini added.
The overhead maintenance, management, and resources have improved with GitLab. User management is in one place and everyone has the same credentials on the same platform. GitLab runners are on field programmable gate arrays (FPGAs) and on cluster computers. The GitLab server is becoming a hub, not only of code but even of hardware resources hooking into that. It’s really becoming a central point of connection for many different activities, ranging from code reviews to hardware integration, and testing to code development. This is great work from GitLab in our opinion,” Bartolini said.
SKAO deploys into a virtual Kubernetes environment for testing and integration. “The tight integration between Kubernetes and GitLab is really helpful and makes it straightforward to deploy our stack on the cloud and run the related testing activity, also promoting the software to a staging environment where UAT takes place,” Bartolini said. The team can integrate into any cloud platform through Kubernetes. They monitor all deployments through GitLab onto various cloud providers.
The biggest simplification that GitLab has enabled is the ability to aggregate test results seamlessly from all code bases. Prior to GitLab, test results were spread throughout the globe, creating unnecessary steps. SKAO has replaced at least five different tools with GitLab as their single application.
The templates have also provided a simplified workflow in creating a new repository. With GitLab, they’re creating a model of common code ownership. “Having access to part of the code you’re not directly developing is also super easy now, because everything is there. It’s very clear, who is doing what, how to get access to different parts. So these help cross-collaboration,” Bartolini said.
As the SKA project evolves, the software development community is making strides towards creating an atmosphere of transparency, inclusion, and shared ownership. “Adopting the GitLab platform ties us in with a community of like-minded engineers and developers, based on principles that we acknowledge as being fundamental in the coming era,” Bartolini said. “Having clear visibility on the Gitlab roadmap and the possibility to influence it, enabled by an open and transparent process, is of great value.”
Developers are able to give feedback to the evolution of GitLab. Bartolini and his team have added input to the GitLab roadmap and received feedback in return. The process of collaboration between users is a great example for the SKA community. “It’s very similar to the model we want to push for our own organization. It’s a double effect for us now. Not only is it good because our developers get to see what you’re doing and interact with that, but they also learn from it,” Bartolini said.
“GitLab, the Social Coding Platform, provides complex and critical infrastructure for our organization as a managed service with the follow the sun support service and availability 24 hours a day. With the deep and well thought out cloud native integration for modern DevSecOps, it aligns and supports our aspirations for delivering software in the modern scientific era,” said Piers Harding, Software Quality Engineer.
“We are working with a variety of teams with different backgrounds from all over the world and GitLab enables us to have common infrastructure and tools to uphold DevSecOps principles with ease. Now, I can manage the CI/CD pipelines, make adjustments, observe and act accordingly within the same screen and see the software evolve from a merge request to deployment to Kubernetes securely,” said Uğur Yılmaz, DevOps Engineer.