The Secure engineering sub-department is responsible for the Secure Stage of the product.
To provide content and tools to support the best possible assessment at the earliest possible moment.
Following our single application paradigm, we integrate and build scanning tools to supply security and compliance assessment data to the main GitLab application where we develop our vulnerability management system and other features. While it might be technically feasible, we do not aim at building standalone products that provide this data independently from the GitLab application.
For more details about the vision for this area of the product, see the Secure stage page.
We will give back to the Open Source community We will provide Responsible Disclosure We will share our knowledge and findings at public speaking events
The Secure team works on GitLab's Secure stage.
The following people are permanent members of the Secure Sub-Department:
| Person | Role |
|---|
| Person | Role |
|---|---|
| Olivier Gonzalez | Backend Engineering Manager, Secure:Composition Analysis |
| Fabien Catteau | Staff Backend Engineer, Secure:Composition Analysis |
| Tetiana Chupryna | Senior Backend Engineer, Secure:Composition Analysis |
| Adam Cohen | Senior Backend Engineer, Secure:Composition Analysis |
| Igor Frenkel | Senior Backend Engineer, Secure:Composition Analysis |
| Person | Role |
|---|---|
| Thomas Woodham | Backend Engineering Manager, Secure:Static Analysis |
| Lucas Charles | Staff Backend Engineer, Secure:Static Analysis |
| Ross Fuhrman | Senior Backend Engineer, Secure:Static Analysis |
| Daniel Searles | Senior Backend Engineer, Secure:Static Analysis |
| Zach Rice | Backend Engineer, Secure:Static Analysis |
| Saikat Sarkar | Senior Backend Engineer, Secure:Static Analysis |
| Person | Role |
|---|---|
| Seth Berger | Backend Engineering Manager, Secure:Dynamic Analysis |
| Cam Swords | Senior Backend Engineer, Secure:Dynamic Analysis |
| Craig Smith | Senior Backend Engineer, Secure:Dynamic Analysis |
| Philip Cunningham | Senior Backend Engineer, Secure:Dynamic Analysis |
| Mike Eddington | Staff Backend Engineer, Secure:Dynamic Analysis |
| Herber Madrigal | Senior Backend Engineer, Secure:Dynamic Analysis |
| Aditya Tiwari | Senior Backend Engineer, Secure:Dynamic Analysis |
| Person | Role |
|---|---|
| Neil McCorrison | Frontend Engineering Manager, Secure |
| Fernando Arias | Senior Frontend Engineer, Secure |
| Mark Florian | Senior Frontend Engineer, Secure |
| Paul Gascou-Vaillancourt | Frontend Engineer, Secure |
| Dheeraj Joshi | Senior Frontend Engineer, Secure |
| Jannik Lehmann | Frontend Engineer, Secure |
| Person | Role |
|---|---|
| Mark Art | Manager, Vulnerability Research Engineering |
| Julian Thome | Senior Vulnerability Research Engineer, Vulnerability Research |
| Isaac Dawson | Staff Vulnerability Research Engineer, Vulnerability Research |
| Person | Role |
|---|---|
| Thiago Figueiró | Backend Engineering Manager, Secure:Threat Insights, Protect:Container Security |
| Jonathan Schafer | Senior Backend Engineer, Secure:Threat Insights |
| Michał Zając | Senior Backend Engineer, Secure:Threat Insights |
| Mehmet Emin Inac | Staff Backend Engineer, Secure:Threat Insights |
| Subashis Chakraborty | Senior Backend Engineer, Secure:Threat Insights |
| Person | Role |
|---|---|
| Lindsay Kerr | Frontend Engineering Manager, Secure:Threat Insights, Protect:Container Security |
| Dave Pisek | Senior Frontend Engineer, Secure:Threat Insights |
| Daniel Tian | Senior Frontend Engineer, Secure:Threat Insights |
| Savas Vedova | Senior Frontend Engineer, Secure:Threat Insights |
The following members of other functional teams are our stable counterparts:
| Person | Role |
|---|---|
| Nicole Schwartz | Senior Product Manager, Secure:Composition Analysis |
| Matt Wilson | Senior Product Manager, Secure |
| Derek Ferguson | Senior Product Manager, Secure:Dynamic Analysis |
| Taylor McCaslin | Principal Product Manager, Secure:Static Analysis |
| Andy Volpe | Senior Product Designer, Secure:Threat Insights |
| Rebecca 'Becka' Lippert | Product Designer, Secure:Static Analysis |
| Camellia X. YANG | Senior Product Designer, Secure:Fuzz Testing |
| Michael Fangman | Product Designer, Secure:Dynamic Analysis |
| Russell Dickenson | Senior Technical Writer, Secure |
| Will Meek | Senior Software Engineer in Test, Secure:Composition Analysis |
| Nikhil George | Senior Security Engineer, Application Security, Secure (Static Analysis, Dynamic Analysis, Composition Analysis, Fuzz Testing, Threat Insights, Vulnerability Research), Protect (Container Security) |
| Justin Mandell | Product Design Manager, Configure, Monitor, Secure & Protect |
The Secure Team (previously known as the Security Products Team) is responsible for the security checks features in the GitLab platform, and maps to the secure transversal stage. You can learn more about our approach on the Secure Vision page.
The features provided by the Secure Team are mostly present at the pipeline level, and mostly available as Docker images. This particularity shapes our processes and QA, which differs a bit from the other backend teams.
We still refer to "Security Products" as the tools developed by the Secure Team. Hence the home of our projects in GitLab: https://gitlab.com/gitlab-org/security-products/.
We strive to maintain a consistent User Experience across our Security Products but we do not enforce consistency at the implementation level. Each group faces its own challenges and is in the best position to make the technical choices it deems are the most suitable to achieve its goals. While UX inconsistencies are considered as bugs, we rely on individual teams to make smart decisions about when consistency is important and when divergence makes more sense — either because the divergence itself creates a better experience or because of velocity considerations.
SAST (Static Application Security Testing) refers to static code analysis. GitLab leverages the power of various opensource tools to provide a wide range of checks for many languages and support. These tools are wrapped inside docker images which ensure we get a standard output from there. An orchestrator, developed by GitLab, is in charge of running these images, and gathering all the data needed to generate the final report.
DAST (Dynamic Application Security Testing) is used to hit a live application. Because some vulnerabilities can only be detected once all the code is actually running, this method complements the static code analysis. DAST is relying on OWASP Zed Attack Proxy Project, modified by GitLab to enable authentication.
Dependency Scanning is used to detect vulnerabilities introduced by external dependencies in the application. Because a large portion of the code shipped to production is actually coming from third-party libraries, it's important to monitor them as well. Dependency Scanning is relying mostly on the Gemnasium engine.
Coverage-guided fuzzing and API fuzzing are used to automatically input data into applications or web apis that has the potential to cause crashes or bugs. Coverage-guided fuzzing relies on open-sourced language-specific fuzzers. API Fuzzing is based on a proprietary GitLab engine.
License Compliance helps with the licenses introduced by third-party libraries in the application. Licence management relies on the LicenseFinder gem.
The Vulnerability Research team's purpose is to perform research and develop proofs of concepts that increase the capabilities and effectiveness of the Secure stage.
If you are submitting an issue about a Secure Stage feature, use ~devops::secure and one of the following group labels to get the issue in front of the most appropriate team members.
| Label | Use |
|---|---|
~devops::secure |
All issues related to the Secure Stage |
~group::static analysis |
Secret Detection |
~category::fuzz testing |
Fuzz Testing |
~group::vulnerability research |
Security Benchmarking |
~group::threat insights |
Vulnerability Management |
Additional labels should be added according to the Workflow Labels Documentation.
It is important to delineate who the EM and PM DRIs are for every functionality, especially where this may not be obvious. It is documented on a dedicated delineation page.
Because we have a wide range of domains to cover, it requires a lot of different expertises and skills:
| Technology skills | Areas of interest |
|---|---|
| Ruby on Rails | Backend development |
| Go | SAST, Dependency Scanning |
| Python | DAST |
| SQL (PostgreSQL) | Dependency Scanning / all |
| Docker | Container Scanning / all |
Our team also must have a good sense of security, with at least basic skills in application security.
We provide tools for many different languages (ex: sast, dependency scanning, license compliance). It means our team is able to understand the basics of each of these languages, including their package managers. We maintain tests projects to ensure our features are working release after release for each of them.
See QA Process for more info.
500 errors on gitlab.com are reported to Sentry. Below are some quick links to pull up Sentry errors related to Secure.
Our team occasionally schedules synchronous brainstorming sessions as a method of deep-diving on a specific topic. This approach can be useful in breaking down complexity and deriving actionable steps for problems that lack definition.
These are purposefully freeform to allow for creative problem solving. When possible, time should be reserved for a list of actions to be taken from the open discussion.
Brainstorming Sessions Doc (Internal): https://docs.google.com/document/d/179JL5RzbgSIz2XZewbYn79cuX7_vUtte_TcoLwUUC5o/edit#
Examples of previous brainstorming topics:
In order to help the Sec section come to resolution on defining approaches to section-wide issues, we have formed an Architectural Council. This group comprises tech leads (and SMEs to provide context) in order to keep the team small and nimble. The group is a non-authoritative, supportive group whose members provide representative insights from their respective teams.
Common scenarios/architectural concepts will likely be resolved/discovered. These should be captured generically in an architectural guidelines page of the handbook and, if possible, codified into our processes/templates.
The table below captures characteristics (requirements?) of work that is in-scope, opt-in, or out-of-scope. All requirements must be met for each category.
| Reason/Condition | In-Scope | Opt-in | Out-of-Scope |
|---|---|---|---|
| Does not involve architectural decisions | x | ||
| Is after-the-fact | x | ||
| Is not already covered by architecture guidelines/handbook | x | x | |
| Has broad impact within #secure1 | x | ||
| Is a new unit of work | x | x | |
| Is strictly #secure or #protect | x | x | |
| Could not come to an agreement (escalation) | ? | ||
| Involves architecture decisions | x | x |
GitLab’s Stance for Architectural issues: https://about.gitlab.com/handbook/engineering/architecture/
As the product evolves, it is important to maintain accurate and up to date documentation for our users. If it is not documented, customers may not know a feature exists.
To update the documentation, the following process should be followed:
~Documentation label, outline in the description of the issue what documentation is needed, and assign a Backend Engineer and Technical Writer(TW) to the issue (find the appropriate TW by searching the product categories).~Px label.Since we are a remote company, having daily standup meetings would not make any sense, since we're not all in the same timezone. That's why we have async daily standups, where everyone can give some insights into what they did yesterday, what they plan to do today, etc. For that, we rely on the geekbot slack plugin to automate the process.
description in backquote + [link to issue](#)" format when mentioning issues in your standup report.What did you do since yesterday? to denote the current state:
for successfully accomplished tasks (:ci_passing: emoji)
for tasks that were due on some period of time but were not accomplished (:ci_failing: emoji)
for tasks currently in progress (:ci_running: emoji)
for paused or postponed tasks (:ci_pending: emoji):ci_... icon you find applicableExample:
What did you do since yesterday?
Accomplished the Spotbugs java analyzer compareKey is not unique https://gitlab.com/gitlab-org/gitlab-ee/issues/10860 Still working on Allow guests to create an issue from a vulnerability https://gitlab.com/gitlab-org/gitlab-ee/issues/7813 Catch-up on all emails and threads after the vacationSlack Channels:
As our teams focus on different areas, we have Geekbot configured to broadcast to separate channels in addition to our common one at #s_secure-standup.
Our important meetings are recorded and published on YouTube, in the GitLab Secure Stage playlist. They give a good overview of the decision process, which is often a discussion with all the stakeholders. As we are a remote company, these video meetings help to synchronize and take decisions faster than commenting on issues. We prefer asynchronous work, but for large features and when the timing is tight, we can detail a lot of specifications. This will make the asynchronous work easier, since we have evaluated all edge cases.
We welcome team members to join meetings that are on our shared calendar. The Secure Calendar is available to all logged in GitLab team members.
New hires should go through these steps and read the corresponding documentation when onboarding in the Secure Team. Every new hire will have an assigned onboarding issue that will guide them through the whole process.
Secure largely follows our Product Development Flow
See Issue Refinement to learn how we evaluate complexity, level of effort, our implementation plan and assign issue weights.
The Secure Team follows the coding standards and style guidelines outlined in the company-wide Contributor and Development Docs, however, please consult the following guidelines which are specific to the Secure Team:
Some components of the architecture that support Secure features are shared between multiple groups like the common Go library, the Security Report Schemas, the rails parsers, etc.
Modifying these shared pieces might impact other groups so we should rely as much as possible on approval rules to ensure such changes are reviewed by the relevant teams before being merged.
Impactless two-way door changes could skip the approval process, please use sound judgement and common sense in such situations.
The author of changes should announce broadly the changes made on these components to raise awareness (weekly meeting agenda, slack channel).
We occasionally need to build out new analyzer projects to support new frameworks and tools. In doing so we should follow our engineering Open Source guidelines, including licensing and code standards.
In addition, to write a custom analyzer that will integrate into the GitLab application a minimal featureset is required:
The Dockerfile should use an unprivileged user with the name gitlab. The reason this is necessary is to provide compatibility with Red Hat OpenShift instances, which don't allow containers to run as an admin (root) user. There are certain limitations to keep in mind when running a container as an unprivileged user, such as the fact that any files that need to be written on the Docker filesystem will require the appropriate permissions for the gitlab user. Please see the following merge request for more details: Use gitlab user instead of root in Docker image.
Please see our security-report-schemas for a full list of required fields.
The security-report-schema repository contains JSON schemas that list the required fields for each report type:
As our product evolves, the engineering teams are researching ways to achieve new functionality and improve our architecture.
The outcome of this research can be found in our Technical Documentation section.
The Secure sub-department conducts retrospectives at the group level that follow our engineering workflow. Each group's DRI is responsible to prepare and schedule the retrospective sync sessions and the async retrospective issues can be found in the corresponding project.
NB: we use to have a sub-department wide retrospective whose issues are still accessible in the deprecated project.
The Secure group reviews analytics to help understand customers and their usage of the tools. This data helps drive product and technical decisions. The following links show usage of Secure functionality.
Meaning that the proposed work requires knowledge or impacts multiple groups ↩
