ExtraHop fully embraces CI/CD process transformation with GitLab

ExtraHop applies real-time analytics and advanced machine learning to every business transaction to deliver unprecedented visibility, definitive insights, and immediate answers that enable security and IT teams to act quickly and with confidence. The world’s leading organizations, including Microsoft, Paypal, Sutter Health, and many others, trust ExtraHop to support core digital business initiatives like security, IT modernization, and application service delivery.

Like many engineering organizations, the ExtraHop development team was deeply divided on how to best execute changes to their code. While the organization traditionally used Gerrit, a new generation of engineers experimented with Bitbucket. What was to be a six-month trial turned into a four-year standoff, with the company finding itself sandwiched between two factions of engineers rallying behind the tool that allowed for their preferred method of commits: bulk versus single.

“Gerrit would always have, essentially, just one single commit at a time that was being amended on each change,” said Bri Hatch, Director of IT at ExtraHop. “That worked great for dinosaurs like myself who like a pristine linear commit history where each commit pushed is fully functional. An equally valid option championed by those who cut their teeth on the GitHub model is to iterate and push multiple commits together, dead-ends and all, via a single merge request.”

ExtraHop needed to strike a balance between maintaining the happiness of veteran and newer engineers alike.

Hatch continued, “Engineers are an opinionated bunch, and the holy war centered around amended commits (Gerrit) versus merge requests (Bitbucket). But unlike vi versus emacs – where every engineer can use the tools they like and not interfere with each other’s workflows – here we needed a single production system for code review that supported meaningful cross-team collaboration.”

The division regarding commit models was also impacting the engineering team’s ability to fully embrace the CI/CD model. While continuous delivery was possible with their multi-tool approach, continuous integration – and the ability to identify issues in the code before it ever got to test – was still largely aspirational.

“Achieving a true CI/CD model is kinda like Nirvana: great if you can get there, but pretty hard to do,” said Hatch. “We had a lot of the processes in place, but we lacked critical alignment and the right technology to support the model.”

With no end in sight to the multi-year Gerrit/Bitbucket standoff, engineering management was open to other options. Their first experience with GitLab came during the annual ExtraHop hackathon: “One of the engineers spun up their own GitLab instance on their workstation,” said Kevin Tatum, IT Systems Engineer at ExtraHop. “They migrated some repositories over and worked in parallel between the systems, doing all the code review in GitLab on the sly.”

When the team looked at the results, they saw a promising compromise. GitLab offered a model that aligned to the two different workflows that had long divided the development team: single and bulk commits.

GitLab Auto DevOps also delivered the technology component required for true CI/CD, accelerating product delivery with an end-to-end pipeline out of the box.

“With GitLab, we finally had a single tool that not only aligned to divergent engineering workflows, but also allowed for meaningful continuous integration,” said Hatch.

While ExtraHop had been following a CI/CD model for a few years, the use of disparate developer tools left gaps in the model. Moreover, while the tools they were using supported continuous delivery, they didn’t function well for the critical continuous integration piece.

With GitLab, ExtraHop is well on the way to fully embracing the CI/CD model from both a process and tooling perspective. When the engineering team first began working with GitLab, they found that GitLab CI/CD runners were powerful yet easy to manage.

“One of our IT engineers had some ‘spare time’ during a meeting and implemented CI against the unit tests they’d been running manually in the Gerrit world. When training the rest of the team they waited for the other shoe to drop, but no - it really was as straightforward as it looked.”

Now the ExtraHop engineering team gets web and email notifications on build breaks and cannot commit code that will not work in production.

“Our initial GitLab scope did not include adding or migrating to their CI/CD up front,” said Hatch. “Given the ease of use, however, we’ve bumped the priority of moving to GitLab’s CI/CD and it’s something we’re actively working on. When we saw what was possible for our team, it was a no-brainer.”

Moving to GitLab has also allowed ExtraHop to meet the code review preferences of its entire development team with ease. With GitLab’s integration capabilities, single and bulk commits are available to the entire development team. The versatility appeases the varied workflow needs of ExtraHop engineers who may prefer Gerrit over Bitbucket and vice versa.

The intuitive user interface (UI) was also a significant bonus for the ExtraHop team.

“When looking at UIs, we wanted something that was extremely intuitive to use, as well as fresh, dynamic, and evolving. GitLab was great. I find GitLab, the UI, infinitely more accessible than the Bitbucket UI,” said Hatch.

ExtraHop is still in the process of transitioning to GitLab, with 80% of its repositories now migrated over to the application. They are currently migrating their legacy build environments and are interested in GitLab’s Kubernetes capabilities.

“GitLab was the right choice for us. Every tool is different; every tool has ups and downs,” he said. “But for us, GitLab worked on so many levels.”