2021 has turned out to be another … interesting year, especially for those of us in the security industry. Like so many software companies in the business, much of our recent focus has shifted to collective, cross-organizational research efforts to identify, mitigate and help resolve the threat posed by the Log4j vulnerability (See our response, as well as our post where we detail how to use GitLab to detect Log4j vulnerabilities).
Thankfully though, 2021 was also focused on growing the Security department and adding additional teams and roles, bolstering enterprise SaaS security, reducing our threat landscape with improvements to supply chain security and APT threat protection, and fulfilling our mission of working to enable GitLab to succeed in the most secure way possible (see our vision and mission statements). We achieved impressive results through expansion of our security third-party certification and self-attestation portfolio, contribution of GitLab and customer impacting product security features and improved security across all teams and domains in our security program. Our security teams also focused on improving processes and programs that enable customers on their trust journey, educate and engage team members to contribute toward improving our security posture, and encourage and enable collaboration from our community to strengthen GitLab. These efforts have been successful due to the contributions of our talented and dedicated Security team members, as well as the groups and individuals we partner with each day; including our wider community. THANK YOU for making GitLab stronger!
Improving assurance for the GitLab community
Our Security Assurance sub-department spent the last year working across our organization to pursue and complete certifications, test and strengthen governance, assess and manage risk, and provide overall support and enablement to GitLab teams and our customers through a number of initiatives.
Certification portfolio expansion
Our Security Assurance team built on a successful 2020 by focusing on our ambitious pursuit of compliance certifications with the issuance of GitLab’s first SOC 2 Type 2/SOC 3 reports for the Security Trust Service Criteria (TSC) dated December 2020. Then, to support customers who need reports by the end of the calendar year, we adjusted our 2021 SOC reporting period to end on October 31st. For our most recent SOC reports we also added the Confidentiality TSC to further highlight the maturity of our operating environment.
In addition, we delivered our very first ISO/IEC 27001:2013 certification in 2021. Certification against this highly-regarded baseline security standard recognizes our proven commitment to the highest level of information security management.
Lastly, in alignment with our continued commitment to transparency we publish all of our security certifications and attestation as part of GitLab’s Customer Assurance Package (learn more below).
True, continuous control monitoring
Our Security Compliance team upgraded our GitLab Control Framework (GCF) in 2021 by adopting the Secure Control Framework (SCF) and moving into a new GRC tool: ZenGRC. This upgraded control framework has increased testing efficiency and allowed GitLab to achieve our external compliance and regulatory obligations with minimized impact to our teams. This, along with our system/profile-based approach to testing, enabled us to achieve successful external audits and the implementation of strong IT general controls (ITGCs) for SOX with a small team of highly-skilled compliance engineers.
We believe our approach to control monitoring has a natural bias towards automation which allows our program to scale, along with GitLab. We’ve continued automating our compliance and regulatory workflows and, where possible, testing evidence as we work towards true continuous control monitoring with proactive alerting of control risks.
Next generation customer assurance services
Our Field Security team deployed GitLab’s Trust Center and next generation Customer Assurance Package to further support our customers on their GitLab trust journey. As part of this effort we expanded our Customer Assurance Package to include the Standard Information Gathering (SIG) Lite pre-completed questionnaire, completed an ISO 20243 Self-Assessment for both our SaaS and Self Managed service offerings, and became a CSA STAR Trusted Cloud Service Provider. To support this program internally we dogfooded GitLab’s Service Desk module to deliver a more efficient way of monitoring, completing and responding to customer assurance requests.
For this group, 2022 will bring a heavy focus on tooling and automation in support of continued control monitoring, certification expansion and regulated market specialization.
Note: Shout out to @mmaneval20, @jburrows001, @tdilbeck and @julia.lake who provided content for this section!
Shoring up our defenses
Our team of “breakers, builders, and defenders” in our Security Operations sub-department were quite busy this year identifying, preventing, detecting and responding to risks and security events targeting GitLab, our users and the business.
Identify, analyze and minimize the threat
To enhance visibility and increase protection of our ever-growing laptop fleet, our Security Incident Response Team(SIRT) completed early testing of multiple endpoint detection and response platforms this year. After our IT Ops team successfully deployed our solution, our SIRT team took over support for the tool and owns the endpoint incident response lifecycle. Alerts from the platform have helped to identify possible issues and allow us to respond quickly to keep GitLab secure. Future plans currently include proactive threat hunting and creating advanced detection mechanisms based on available data points.
Security automation to address that ever-increasing threat landscape
To ensure our team’s ongoing incident response efforts are effective against the expanded attack surface and threat landscape that comes with our continued growth and expansion, we’re onboarding incident response automation. This solution has enabled us to automate the handling of reported phishing emails, user attestation on GCP documents access, and the assignment of appropriate response priority level via an incident severity calculator. These enhancements allow our engineers to focus on incident response and devising solutions to more complex issues and incidents.
Strengthening GitLab’s security in the shadows
As for our Red Team, they continued toeing the line of that ever-present balancing act between their stealth, exploratory testing and their commitment to GitLab’s value of transparency; all while helping GitLab implement effective cyber defenses. They held an external-facing AMA this year in which they answered many questions from our community and shared tips on how developers can secure themselves against RCE drive-by attacks; including details on a real-life disclosure on the GitLab GDK and shared our expertise surrounding offensive and defensive perspectives of attacks hiding malicious code in #OSS contributions at BlackHat Europe with "Picking Lockfiles: Attacking & Defending Your Supply Chain". And, much more … which we can’t talk about 😉 😎 .
Note: Shout out to @hasharma, @mjozenazemian, @smanzuik, @vmairet and @blutz1 who provided content for this section!
Strengthening and securing GitLab the product
Our Security Engineering sub-department endeavors to ensure all aspects of GitLab that are exposed to customers or that host customer data are held to the highest security standards, and to be proactive and responsive to ensure the security of anything GitLab offers. Throughout the year, this group collaborates with teams across the organization, and beyond with the GitLab community, to support our business and their bid to ensure that all GitLab products securely manage customer data.
Enhance the product with new tooling: Spamcheck and Package Hunter
Last year we blogged about how we work to detect and mitigate spam on GitLab.com. This year our Security Automation team worked closely with the GitLab Trust and Safety team to introduce Spamcheck, our new anti-spam engine that has been enabled for all projects on GitLab.com and we're targeting inclusion of Spamcheck in the 14.6 release for our GitLab self-managed customers. By allowing us to better detect and prevent spam, we believe Spamcheck has significantly improved GitLab’s resilience to it. We recently blogged about the technical decisions behind Spamcheck, as well as some of the early performance data points. You can also check out the code behind Spamcheck.
In July 2021, the GitLab Security Research team released Package Hunter, a tool that helps identify malicious dependencies via runtime monitoring. Powered by Falco, Package Hunter installs a program’s dependencies in a sandbox environment and analyzes system calls for malicious code and other unexpected behavior. Testing of NodeJS and Ruby Gems is currently supported. The project is open source and we are continually working to improve upon it. Community contributions and feedback are very much welcome!
Risk reduction and vulnerability management
Scaling our Application Security efforts has been a big priority for our teams. Again, the key to doing so successfully is thru automation, particularly when it comes to keeping track of a growing list of codebases that are constantly changing, adding new components, and relying on different dependencies. For this reason we’re very excited about the progress that has been made on the GitLab Inventory Builder, a very handy tool capable of generating and maintaining a complete list of projects and their dependencies hosted on GitLab.com or self-hosted instances. This is also our first iteration of using policy-as-code to monitor and control various aspects of our projects. Not only can we track where security scans are not well configured, but we believe we can also spot project configuration issues precisely. With the automatic creation of violation issues in GitLab, we can organize, track, and scale the work of our Security Engineers more efficiently. Take a look at this live action demo and view the example code supporting it for more information!
During 2021 we bootstrapped our Infrastructure Security team and program. This new team works as a stable counterpart to the Infrastructure team and collaborates across Infrastructure and Security to help identify and mitigate security issues, vulnerabilities, and misconfigurations by applying their in-depth knowledge of operating systems, infrastructure, and cloud providers. With this new team and program we’ve bolstered our security observability, added an operating system instrumentation platform, enhanced monitoring, and created an analytics framework for our hosts; all of which help give us insight into all aspects of our production systems. We’ve also deployed an intuitive security graph tool across our cloud platforms that inventories all of our assets and shows the connections between them, but also enables querying based on various metadata. We believe these efforts have already resulted in significant security risk reduction, enhanced vulnerability management, increased observability, and granular monitoring capabilities.
To help team members understand the security implications of the systems and features they design and work on, this year our team formalized and integrated a threat modeling process here at GitLab. Building upon the evidence driven threat modeling approach that we started working towards adopting last year, we’ve iterated on the threat modeling processes and tooling in order to increase adoption, usage, and understanding across GitLab teams. We’ve also added issue templates to our internal threat modeling repository and improved upon our threat modeling runbook. We talk about some of the basics of threat modeling and how we’ve developed a framework that will work for GitLab in our blog.
Strengthening our product through global expertise and contributions
This past year we received 752 reports from 404 talented bug bounty reporters from all across the globe who helped us to strengthen our product through the identification of security vulnerabilities.
In February, we moved to a managed bug bounty program with HackerOne. This enables us to scale our report triage process, filter out the noise, and ultimately present the most important reports to our development teams faster. In November, we kicked off, “Our 3rd annual bug bounty contest: the swagtastic sequel to the sequel“, announced a near double in [bounty rewards and detailed our move to standardize bounty payments by using CVSS along with a nifty CVSS calculator. This program, and the amazing bug bounty hunters who contribute to it, continue to raise GitLab’s security bar and reduce risk for our customers. You can read more about what happened in our bug bounty program this past year in “2021: Smashing bugs and dropping names”.
Note: Shout out to @ankelly, @jritchey, @plafoucriere, @heather and @laurence.bierner who provided content for this section!
Everyone can contribute…to Security
When we say that Security is a team effort, we mean it. These three sub departments, and the 12 teams that sit within them work collaboratively (and sometimes tirelessly) with dozens of teams across GitLab, and community members, to keep GitLab secure and protect our company, the community and our customers. Thank you to everyone who contributes here and best wishes for a safe, healthy and happy 2022! 🥂
