The Secure engineering sub-department is responsible for the Secure Stage of the product.
To provide content and tools to support the best possible assessment at the earliest possible moment.
Following our single application paradigm, we integrate and build scanning tools to supply security and compliance assessment data to the main GitLab application where we develop our vulnerability management system and other features. While it might be technically feasible, we do not aim at building standalone products that provide this data independently from the GitLab application.
For more details about the vision for this area of the product, see the Secure stage page.
To support the success of GitLab by developing highly usable, hiqh quality tools for customers to build more secure software.
The Secure team works on GitLab's Secure stage.
| Person | Role |
|---|---|
| Wayne Haber | Director of Engineering for Growth, Sec, and Data Science |
| Thomas Woodham | Senior Engineering Manager, Secure |
The following people are permanent members of the Secure Sub-Department:
The following members of other functional teams are our stable counterparts:
| Person | Role |
|---|---|
| Connor Gilbert | Senior Product Manager, Secure:Static Analysis |
| Russell Dickenson | Senior Technical Writer, Secure |
| Nikhil George | Senior Security Engineer, Application Security, Secure (Static Analysis, Dynamic Analysis, Composition Analysis, Vulnerability Research), Growth (Acquisition, Activation). |
| Will Meek | Senior Software Engineer in Test, Secure:Composition Analysis |
The Secure Team (previously known as the Security Products Team) is responsible for the security checks features in the GitLab platform, and maps to the secure transversal stage. You can learn more about our approach on the Secure Vision page.
The features provided by the Secure Team are mostly present at the pipeline level, and mostly available as Docker images. This particularity shapes our processes and QA, which differs a bit from the other backend teams.
We still refer to "Security Products" as the tools developed by the Secure Team. Hence the home of our projects in GitLab: https://gitlab.com/gitlab-org/security-products/.
We strive to maintain a consistent User Experience across our Security Products but we do not enforce consistency at the implementation level. Each group faces its own challenges and is in the best position to make the technical choices it deems are the most suitable to achieve its goals. While UX inconsistencies are considered as bugs, we rely on individual teams to make smart decisions about when consistency is important and when divergence makes more sense — either because the divergence itself creates a better experience or because of velocity considerations.
SAST (Static Application Security Testing) refers to static code analysis. GitLab leverages the power of various opensource tools to provide a wide range of checks for many languages and support. These tools are wrapped inside docker images which ensure we get a standard output from there. An orchestrator, developed by GitLab, is in charge of running these images, and gathering all the data needed to generate the final report.
DAST (Dynamic Application Security Testing) is used to hit a live application. Because some vulnerabilities can only be detected once all the code is actually running, this method complements the static code analysis. DAST is relying on OWASP Zed Attack Proxy Project, modified by GitLab to enable authentication.
Dependency Scanning is used to detect vulnerabilities introduced by external dependencies in the application. Because a large portion of the code shipped to production is actually coming from third-party libraries, it's important to monitor them as well. Dependency Scanning is relying mostly on the Gemnasium engine.
Coverage-guided fuzzing and API fuzzing are used to automatically input data into applications or web apis that has the potential to cause crashes or bugs. Coverage-guided fuzzing relies on open-sourced language-specific fuzzers. API Fuzzing is based on a proprietary GitLab engine.
License Compliance helps with the licenses introduced by third-party libraries in the application. Licence management relies on the LicenseFinder gem.
The Vulnerability Research team's purpose is to perform research and develop proofs of concepts that increase the capabilities and effectiveness of the Secure stage.
If you are submitting an issue about a Secure Stage feature, use ~devops::secure and one of the following group labels to get the issue in front of the most appropriate team members.
| Label | Use |
|---|---|
~devops::secure |
All issues related to the Secure Stage |
~group::static analysis |
Secret Detection |
~category::fuzz testing |
Fuzz Testing |
~Category:Software Composition Analysis |
Software Composition Analysis |
~group::vulnerability research |
Security Benchmarking |
Additional labels should be added according to the Workflow Labels Documentation.
Because we have a wide range of domains to cover, it requires a lot of different expertises and skills:
| Technology skills | Areas of interest |
|---|---|
| Ruby on Rails | Backend development |
| Go | SAST, Dependency Scanning, DAST |
| Python | DAST |
| SQL (PostgreSQL) | Dependency Scanning / all |
| Docker | Container Scanning / all |
| C# | API Security |
Our team also must have a good sense of security, with at least basic skills in application security.
We provide tools for many different languages (ex: sast, dependency scanning, license compliance). It means our team is able to understand the basics of each of these languages, including their package managers. We maintain tests projects to ensure our features are working release after release for each of them.
See Versioning and release process.
See QA Process for more info.
500 errors on gitlab.com are reported to Sentry. Below are some quick links to pull up Sentry errors related to Secure.
Our team occasionally schedules synchronous brainstorming sessions as a method of deep-diving on a specific topic. This approach can be useful in breaking down complexity and deriving actionable steps for problems that lack definition.
These are purposefully freeform to allow for creative problem solving. When possible, time should be reserved for a list of actions to be taken from the open discussion.
Brainstorming Sessions Doc (Internal): https://docs.google.com/document/d/179JL5RzbgSIz2XZewbYn79cuX7_vUtte_TcoLwUUC5o/edit#
Examples of previous brainstorming topics:
As the product evolves, it is important to maintain accurate and up to date documentation for our users. If it is not documented, customers may not know a feature exists.
To update the documentation, the following process should be followed:
~Documentation label, outline in the description of the issue what documentation is needed, and assign a Backend Engineer and Technical Writer(TW) to the issue (find the appropriate TW by searching the product categories).~Px label.Since we are a remote company, having daily standup meetings would not make any sense, since we're not all in the same timezone. That's why we have async daily standups, where everyone can give some insights into what they did yesterday, what they plan to do today, etc. For that, we rely on the geekbot slack plugin to automate the process.
description in backquote + [link to issue](#)" format when mentioning issues in your standup report.What did you do since yesterday? to denote the current state:
for successfully accomplished tasks (:ci_passing: emoji)
for tasks that were due on some period of time but were not accomplished (:ci_failing: emoji)
for tasks currently in progress (:ci_running: emoji)
for paused or postponed tasks (:ci_pending: emoji):ci_... icon you find applicableExample:
What did you do since yesterday?
Accomplished the Spotbugs java analyzer compareKey is not unique https://gitlab.com/gitlab-org/gitlab-ee/issues/10860 Still working on Allow guests to create an issue from a vulnerability https://gitlab.com/gitlab-org/gitlab-ee/issues/7813 Catch-up on all emails and threads after the vacationSlack Channels:
As our teams focus on different areas, we have Geekbot configured to broadcast to separate channels in addition to our common one at #s_secure-standup.
Our important meetings are recorded and published on YouTube, in the GitLab Secure Stage playlist. They give a good overview of the decision process, which is often a discussion with all the stakeholders. As we are a remote company, these video meetings help to synchronize and take decisions faster than commenting on issues. We prefer asynchronous work, but for large features and when the timing is tight, we can detail a lot of specifications. This will make the asynchronous work easier, since we have evaluated all edge cases.
We welcome team members to join meetings that are on our shared calendar. The Secure Calendar is available to all logged in GitLab team members.
GitLab is an extremely active organization which generates a lot of news and activity each week. Everyone in Secure are encouraged to keep themselves informed as to what is happening in the larger organzation. Everyone is also encouraged to contribute to these channels and communication paradigms when you have information to share.
In addition to this, each group in Secure conducts a weekly synchronous meeting. These meetings are publicized on the Secure Calendar mentioned above. As always at GitLab, we strive to make meeting attendance optional.
In addition to keeping yourself informed, team members are encouraged to keep others informed as well. Secure groups have adopted a practice of including the following topics as standing agenda items in their weekly meetings, with example topics for each bullet point.
Engineering Managers are responsible for populating this section of weekly group meetings, though everyone can contribute. In addition to helping the group keep itself informed about what's happening each week, the SEM for Secure will collect this information weekly and broadcast a curated list to the section.
New hires should go through these steps and read the corresponding documentation when onboarding in the Secure Team. Every new hire will have an assigned onboarding issue that will guide them through the whole process.
Secure largely follows our Product Development Flow
See Issue Refinement to learn how we evaluate complexity, level of effort, our implementation plan and assign issue weights.
From time to time, the demand for frontend engineering tasks does not match the capacity available in Secure groups. If a group could use frontend support, they are encouraged to add the
~secure::frontend backlog label to issues which are ready to be picked up. Additionally,
Engineering Managers are encouraged to look at this queue of work during milestone planning if there is frontend capacity available.
The Secure Team follows the coding standards and style guidelines outlined in the company-wide Contributor and Development Docs, however, please consult the following guidelines which are specific to the Secure Team:
Some components of the architecture that support Secure features are shared between multiple groups like the common Go library, the Security Report Schemas, the rails parsers, etc.
Modifying these shared pieces might impact other groups so we should rely as much as possible on approval rules to ensure such changes are reviewed by the relevant teams before being merged.
Impactless two-way door changes could skip the approval process, please use sound judgement and common sense in such situations.
The author of changes should announce broadly the changes made on these components to raise awareness (weekly meeting agenda, slack channel).
We occasionally need to build out new analyzer projects to support new frameworks and tools. In doing so we should follow our engineering Open Source guidelines, including licensing and code standards.
In addition, to write a custom analyzer that will integrate into the GitLab application a minimal featureset is required:
The Dockerfile should use an unprivileged user with the name gitlab. The reason this is necessary is to provide compatibility with Red Hat OpenShift instances, which don't allow containers to run as an admin (root) user. There are certain limitations to keep in mind when running a container as an unprivileged user, such as the fact that any files that need to be written on the Docker filesystem will require the appropriate permissions for the gitlab user. Please see the following merge request for more details: Use gitlab user instead of root in Docker image.
Please see our security-report-schemas for a full list of required fields.
The security-report-schema repository contains JSON schemas that list the required fields for each report type:
As our product evolves, the engineering teams are researching ways to achieve new functionality and improve our architecture.
The outcome of this research can be found in our Technical Documentation section.
We keep a list of data sources in our internal wiki. This includes advisory databases, package license information, and related data.
The Secure sub-department conducts retrospectives at the group level that follow our engineering workflow. Each group's DRI is responsible to prepare and schedule the retrospective sync sessions and the async retrospective issues can be found in the corresponding project.
After all groups have completed their retrospective, we conduct a Section Retrospective.
NB: we use to have a sub-department wide retrospective whose issues are still accessible in the deprecated project.
The Secure group reviews analytics to help understand customers and their usage of the tools. This data helps drive product and technical decisions. The following links show usage of Secure functionality.
We also track our backlog of issues, including past due security and infradev issues, and total open SUS-impacting issues and bugs.
MR Type labels help us report what we're working on to industry analysts in a way that's consistent across the engineering department. The dashboard below shows the trend of MR Types over time and a list of merged MRs.