Category Vision - Logging

1. You are here:
2. Category Vision - Logging

• Logging
• Target Audience and Experience
• What's Next & Why
• Maturity Plan
• Competitive Landscape
• Analyst Landscape
• Top Customer Success/Sales Issue(s)
• Top Customer Issue(s)
• Top Internal Customer Issue(s)
• Top Vision Item(s)

Logging

A fundamental requirement for running applications is to have a centralized location to manage and review the logs. While manually reviewing logs could work with just a single node app server, once the deployment scales beyond one you need solution which can aggregate and centralize them for review.

Much like we do with Prometheus, GitLab can integrate with existing open source logging solutions to provide an integrated experience.

We currently allow users to access access pod logs, but we should also support aggregated logging as well.

Target Audience and Experience

Being able to capture and review logs are an important tool for all users across the DevOps spectrum. From pure developers who may need to troubleshoot their application when it is running in a staging or review environment, as well as pure operators who are responsible for keeping production services online.

The target workflow includes a few important use cases:

1. Aggregating logs from multiple hosts/containers
2. Filtering by host, container, service, timespan, regex, and other criteria. These filtering options should align with the filter options and tags/labels of our other monitoring tools, like metrics.
3. Log alerts should also be able to be created, triggering alerts under specific user defined scenarios.

What's Next & Why

The next step is to determine which solution we should base our aggregated logging solution on. One of the leading candidates is Loki, which we will be evaluating next.

This allows multiple time series to be represented in the same chart, that require different queries to generate. For example showing both the p95 and p99 buckets in the same chart.

Maturity Plan

• Base Epic

Competitive Landscape

Loggly and Elastic are the top two competitors in this space.

Analyst Landscape

There does not seem to be a Forrester or Gartner analysis for this product category.

We will be reaching out and setting up meetings with analysts to get their insight soon.

Top Customer Success/Sales Issue(s)

Aggregated logs

Top Customer Issue(s)

Aggregated logs

Top Internal Customer Issue(s)

Top Vision Item(s)

Aggregated logs

A fundamental requirement for running applications is to have a centralized location to manage and review the logs. While manually reviewing logs could work with just a single node app server, once the deployment scales beyond one you need solution which can aggregate and centralize them for review.

Much like we do with Prometheus, GitLab can integrate with existing open source logging solutions to provide an integrated experience.

We can start with a Kubernetes cluster first, and then allow manual configuration of an existing ElasticSearch cluster later.

Proposed MVC:

1. Button to deploy ElasticSearch to a cluster
2. Button to deploy fluentd to cluster, configured to ship logs from nodes to ElasticSearch
3. Build a log view with a simple plain text search filter