Blog AI/ML ML experiment: Generate tests for code changes
Published on April 27, 2023
3 min read

ML experiment: Generate tests for code changes

Learn how GitLab is experimenting with ML-powered test suggestions in this latest installment of our ongoing 'AI/ML in DevSecOps' series.


This blog post is part of an ongoing series about GitLab's journey to build and integrate AI/ML into our DevSecOps platform. The series starts here: What the ML is up with DevSecOps and AI?. Throughout the series, we'll feature blogs from our product, engineering, and UX teams to showcase how we're infusing AI/ML into GitLab.

Proposing changes and new features via merge requests is great, but what about the tests? Sometimes, tests can be the hardest part of any code change you make. Maybe you're not sure how to start writing the tests? Maybe the test doesn't cover all the scenarios that need to be tested? Maybe you just want to get a second opinion on the tests that were written? We believe that we can use generative AI and large language models (LLMs) to help provide relevant test coverage for the proposed changes, so reviewers and authors can have confidence in the quality of code changes being submitted.

In a rapid prototype, Phil Hughes, Staff Frontend Engineer for our Code Review Group, used AI to generate suggested test coverage for changes directly in the merge request. He added a new option on merge request files to provide suggested tests in a sidebar:

Merge request test generation AI

Iterating on AI/ML features

While just an experiment today, we are iterating on how to effectively bring features like this to our customers. We're beginning by generating these test suggestions, and seeking ways to incorporate them into the review flow. We're exploring ideas like:

  • Automatic detection of missing tests, with suggestions to add coverage
  • Automated review of the proposed tests in the merge request, for appropriateness and completeness

This experiment is just the start of the ways we're infusing GitLab with AI/ML capabilities to help GitLab users become more efficient and effective at their jobs. We are looking across the software development lifecycle for painful and time-consuming tasks that are ideal for AI-assisted features. We'll continue to share these demos throughout this blog series.

Interested in using these AI-generated features? Join our waitlist and share your ideas.

Continue reading our "AI/ML in DevSecOps" series.

Disclaimer: This blog contains information related to upcoming products, features, and functionality. It is important to note that the information in this blog post is for informational purposes only. Please do not rely on this information for purchasing or planning purposes. As with all projects, the items mentioned in this blog and linked pages are subject to change or delay. The development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab.

We want to hear from you

Enjoyed reading this blog post or have questions or feedback? Share your thoughts by creating a new topic in the GitLab community forum. Share your feedback

Ready to get started?

See what your team could do with a unified DevSecOps Platform.

Get free trial

New to GitLab and not sure where to start?

Get started guide

Learn about what GitLab can do for your team

Talk to an expert