Log Aggregation Working Group

1. You are here:
2. Meet the GitLab team
3. Organizational Structure
4. Working Groups
5. Log Aggregation Working Group

On this page

• Attributes
• Business Goal
• Exit Criteria
• Roles and Responsibilities

Attributes

Property	Value
Date Created	March 6, 2019
Date Ended	TBD
Slack	#wg_log-aggregation (only accessible from within the company)
Google Doc	Log Aggregation Working Group (only accessible from within the company)
Issue Label	WG-LogAgg (gitlab-com/-org)

Business Goal

Increase the quality, value, and accessibility of our GitLab.com logs.

Intent is to:

1. Analyze and document the locations, retention, and variety of production logs as they exist today, including analysis of the data classification and access controls
2. Develop troubleshooting and investigation guides to make best use of GitLab.com logs
3. Perform gap analysis of log quality and completeness, and where improvements can be made to streamline investigations
4. Work with Infra and Development teams to integrate the proposed improvements into GitLab.com

Exit Criteria

1. Completed audit of existing log sources, storage locations, and means of accessing the aggregated logs along with a quality and accuracy evaluation of these logs
2. Creation and delivery of a standardized logging lifecycle providing:
   1. Contribution: The how, when, and why Developers should add logged events to their code
   2. Production: Ensuring each service as part of the GitLab package is writing logs in a consistent and configurable fashion
   3. Aggregation: The consistent, reliable, and configurable collection and shipping of logs to defined location(s)
   4. Accessibility: Tooling and functionality to provide timely access and manipulation of the log data
   5. Retention: Define, provide means to enforce, and verify the necessary retention policies on log data
3. Publish documentation and training material on the GitLab logging lifecycle for use by the GitLab.com service and self-managed GitLab

Roles and Responsibilities

Working Group Role	Person	Title
Facilitator	Paul Harrison	Senior Security Engineer
Member	Stan Hu	Engineering Fellow
Security Lead	Antony Saba	Senior Threat Intelligence Engineer
Infrastructure Lead	Andrew Newdigate	Staff Engineer, Infrastructure
Member	Amar Amarsanaa	Site Reliability Engineer
Member	Alex Groleau	Security Software Engineer, Automation
Member	Shawn Sichak	Security Engineer, Operations