Category Direction - Code Quality

The following page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features or functionality remain at the sole discretion of GitLab Inc.

Code Quality

Code Quality


Stage	Secure
Maturity	Minimal
Content Last Reviewed	`2023-05-26`

Introduction and how you can help

This direction page describes GitLab's plans for the Code Quality category, which helps you keep your code maintainable, idiomatic, and correct.

This page is maintained by the Product Manager for Static Analysis, Connor Gilbert.

Everyone can contribute to where GitLab Code Quality goes next, and we'd love to hear from you. The best ways to participate in the conversation are to:

Comment or contribute to existing Code Quality issues in the public gitlab-org/gitlab issue tracker.
If you don't see an issue that matches, file a new issue.
- Post a comment that says @gitlab-bot label ~"group::static analysis" ~"Category:Code Quality" so your issue lands in our triage workflow.
If you're a GitLab customer, discuss your needs with your account team.

Overview

GitLab Code Quality helps you eliminate bugs, style violations, and mistakes before they're merged. These features can:

Make code authors more productive by catching errors earlier.
Help code reviewers focus on logic, design, and other higher-order concerns, not style nitpicks or problems that an automated tool could surface more easily.
Reduce the risk of bugs when software lands in production.

Code Quality shows quality findings in:

A merge request widget, in all tiers.
A pipeline report, in GitLab Premium and Ultimate.
The merge request changes (diff) view, in GitLab Ultimate.
A project quality summary view, in GitLab Ultimate.

You can provide findings from any tool by saving a JSON report as a CI/CD artifact. This approach is great if you already use linters or other tools in your development process, or if you have custom organization-specific rules.

GitLab also provides a scanning system based on CodeClimate open-source scanning tool and its analyzers. However, we are currently planning to remove this dependency.

To learn more, check the Code Quality documentation.

Outside of the Code Quality Detection category, GitLab also offers other features that help you build quality software:

Strategy and Themes

Ambitious vision: Code Quality is the single best way in GitLab to tell a code author and reviewer about a quality or linting problem in the diff they've submitted.

We believe we can accomplish this vision by shaping Code Quality into a flexible, open, community-oriented feature area. Adopting a more "pluggable" model should help us:

Meet teams where they are instead of requiring a one-size-fits-all approach. This includes respecting team preferences about the tools they wish to use, including tools that may be organization-specific or closed-source.
Encourage individual users and teams to experiment and take the initiative to move their teams forward. We want users to be able to build their workflows around GitLab Code Quality freely without relying on first getting patches included in GitLab or an upstream project like CodeClimate.
Maximize compatibility with different project organization structures and requirements, including when projects need particular tool versions or must be executed in certain ways.
Enable reporting on a wider variety of languages—and even non-code content like technical documentation—than a single GitLab team could support.

This approach is similar to the way that other products take standard coverage file formats. Within GitLab, similar examples include accepting the JUnit XML format for test reporting, and accepting CycloneDX reports for Software Bill of Materials information.

This approach does mean de-emphasizing the "automatic" scanning that has been a key part of how GitLab Code Quality has historically been presented and framed. We're still exploring ways that we could maintain this kind of automatic experience, but have become convinced that the current CodeClimate-based scanning system is not the way forward.

1 year plan

Our one-year focus includes two top priorities:

Usability: Improving how quality findings are shown and explained in the merge request and elsewhere.
Pluggable Scanning: Refocusing the feature category on an approach that meets teams where they are, supports the tools they already use, and encourages a broad set of GitLab- and community-developed integrations.

We intend to, at some point in 2023, announce the deprecation and upcoming End of Support for CodeClimate-based scanning. We currently plan to decide this timeline once we've evaluated how other tools can best be integrated directly with Code Quality; having that knowledge and documentation prepared will allow us to specify actionable steps to migrate away from CodeClimate-based scanning.

What is next for us

In the next 3 months, we are planning to:

Using the results of our dogfooding in gitlab-org and in security analyzers to identify iterative steps toward our Bring-Your-Own-Tool direction. This will clarify how users can avoid CodeClimate and its Docker-in-Docker requirements.
Expanding the amount of information shown in the UI for each finding. We expect this will make findings easier to understand and fix.
Aligning with a new design for findings in the merge request that will include both SAST and Code Quality findings.

What we are currently working on

We are currently working on:

Dogfooding Code Quality in gitlab-org by configuring various code scanning jobs to output Code Quality reports.
Using the results of dogfooding in gitlab-org and in security analyzers to identify iterative steps toward our Bring-Your-Own-Tool direction. This will clarify how users can avoid CodeClimate and its Docker-in-Docker requirements.
Updating the way quality findings are displayed in the merge request as part of adding SAST findings in the diff view.

What we recently completed

Our recent work includes:

Development of a "drawer" UI concept to show more detail about quality findings. This feature is currently deactivated due to problems with the amount of data being saved and proecssed in these more detailed reports.
Analysis of how existing quality tools can be integrated with the existing report format, to inform our future direction in this area.
Showing more findings in the merge request changes tab (15.8).
Supporting multiple reports per pipeline, making it possible to assess quality using more than one tool (15.7).

Check older release posts for our previous work in this area.

What is Not Planned Right Now

Proprietary quality analysis: We currently plan to focus on integrating with existing quality tools instead of creating our own quality analyzer.
- We believe that improving the experience for users of all quality tools—including open-source linters, third-party products, and custom tools—delivers value sooner, and doesn't detract from our ability to bring a proprietary tool to market later.
Mixing quality and security: We aren't planning to blur the lines between security and quality findings. We believe it's valuable to keep quality and security findings distinct, since these tools and processes are maintained by different personas and are subject to different organizational requirements.
- For example, while quality tools and their rules are typically managed by software engineering departments, security tools (and rules about how to handle security findings) typically involve security professionals and are monitored for adherence to compliance controls.
- However, users should see a consistent user experience for quality and security findings, and we shouldn't have to inefficiently build each workflow feature twice either.
Modifications to CodeClimate: We don't believe we have an effective path forward to modify CodeClimate or its plugins. This conclusion is based on the collective weight of a mix of factors:
- We don't have a clear technical path toward removing the Docker-in-Docker requirement, despite more than one investigation.
- The AGPL license used for most CodeClimate core components and plugins makes contributions and operations more difficult for GitLab and for customers.
- Many existing plugins do not have active communities and do not receive regular updates.
- The workflow for typical development tool usage includes development teams updating and maintaining their project configurations to use tools in many contexts, rather than just in CI. (See UX research.)
- It is simpler and more efficient to run tools like eslint directly, rather than mediating their configuration and operation through multiple layers of indirection.
- Mediating access through CodeClimate doesn't provide unique capabilities: the underlying tools for each official CodeClimate plugin are each available to use directly.

Dogfooding

GitLab teams collaborate more efficiently by using Code Quality to surface issues during code review. Team members have integrated:

The vale linter used for product documentation.
The markdownlint linter, used in the GitLab internal handbook (accessible to team members only).
Go linting, in security analyzers.
CodeClimate-based analysis, in gitlab-org/gitlab.

Team members have also proposed integrations to enforce design system migration.