The following page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features or functionality remain at the sole discretion of GitLab Inc.
This is the product direction for Monitor:Observability. If you'd like to discuss this direction directly with the product managers for Monitor:Observability, feel free to reach out to Sebastien Pahl (Principal PM of Monitor:Observability) (GitLab, Email).
We aim to democratize access to open-source observability tools and APIs in order to create a world where developers and operators can easily, and seemlessly collaborate to ensure production applications are availble and responsive to their users.
On Dec 14, 2021, GitLab announced the acquisition of Opstrace. Opstrace is an observability distribution that will become integrated and usable out-of-the-box for all GitLab users. Using Opstrace, users gain the benefit of having a full observability platform, starting with Metrics, powered by Prometheus.
On March 29, 2021 we pubished a blog post outlining our vision for a new UI to be fully integrated with the devops platform.
Over the coming months, we will focus on integrating Opstrace into GitLab. Observability will be available, by default, for both GitLab SaaS and Self-Managed users, starting at the free tier. Using GitLab’s observability capability, you will be using a completely open-sourced platform. You do not have to worry about vendor lock-in from instrumentation to alerting.
Organizations often have the choice of using an observability vendor or building their own observability platform. With the former, teams outsource the problems of operating a system that needs to be scaled and consistently updated but are locked-in to proprietary software and APIs. With the latter, teams have to manage the complexity of an observability platform, figure out how to make all the open-source components work together and keep them running up to date. We want that choice to be easy to make.
We are different because we will focus on making the toil of operating open-source observability tools easy. Instead of staffing a team to keep toolings up to date, you can easily and confidently upgrade with the observability distribution. Furthermore, this observability tool is integrated with the rest of GitLab so you have a single tool to build, test, collaborate, deploy, and monitor your applications.
Over time, we’ll add additional delightful experiences such as:
We'll pursue our vision based on the following principles:
We want developers to have immediate access to a robust observability platform as a first principle. That means we will ensure it is available by default, starts with convention over configuration, and is easy to utilize.
We believe that observability data is most valuable when it is easy to glean insights across data types of metrics, logs, traces and errors. As a result we're working to store all types of observability data in a unified datastore - Clickhouse.
A common interface, and most importantly a common collaboration space is key to bringing observability to where developers work. We'll create an integrated UI that allows for collaboration on observability data directly within GitLab.
The Monitor:Observability will work to help all GitLab customers Monitor their apps by providing a simple on-by-default observability stack. We'll focus first on more users, and more contributors' adoption in our dual flywheel strategy. We'll know if we are trending in the right direction by looking at the Monitor:Observability Monthly Active Users adoption, which will grow if we are providing value to our users.
While SaaS is the first goal to be able to have a good testing ground and enable quick on-boarding, we want to quickly focus on self-managed enterpise users.
The Observability group will be focused on bringing observability, including Metrics, Tracing, Error tracking and Logging to market within the GitLab platform.
Now that Opstrace is part of GitLab, the first thing we’ll do is integrate Opstrace into GitLab. For additional details on the integration progress, follow this GitLab epic.
We are currently focused on integrating the first pieces with the Gitlab platform.
Our goal is to make Open-source Observability as easy as SaaS offerings. Having a fully usable observability Saas that one can also deploy in a dedicated manner.
Build an alternative Open Source UI for observability with different paradigms and goals than the current single offering (grafana). More here.
Open Source observability has many popular APIs such as Prometheus, Opentelemetry, … But metrics, logs, traces and other observability events all go into separate databases. Treating them as separate systems prevents building new higher level applications. We are working on getting all the data in a single backend based on clickhouse to enable richer user experience in the future.
While it’s true that we built a lot of kubernetes features, all that is needed is compatibility with APIs such as prometheus, not running kubernetes oneself.
We do use kubernetes to deploy opstrace itself but that is not something the user has to manage and it’s separate from their infrastructure.
As mentioned above, all we require is a cloud account. We deploy and configure everything with our terraform installer.
Kubernetes is used to run the system, but that is transparent to the user, operators are deployed on the kubernetes cluster to maintain the rest of the stack.
Terraform will be used to rollout new upgrades.
For omnibus users, since this kind of stack is bigger than what a single machine/VM can provide, this will also be available through the same manner. They would deploy it into one of their cloud accounts.
At first, there will be a separate UI. This is because we are using Grafana as a base and we need to use its technologies (react…) to build our UI. To bring data into the GitLab UI we will use iframes. The simplest thing to start with first would be to Iframe our entire UI (this keeps the feeling for the user of not leaving GitLab) and later allow framing and more directly embedding single dashboards & more.
There are two types of datastores.
Clickhouse for storing the observability data.
Postgres is used for our settings and users. This will disappear over time as we could use the existing GitLab one. No timelines yet as we first focus on other parts.
We are going to move away from supporting Cortex. We will be finding another Prometheus API and we are going to use Clickhouse as our metrics storage backend.
We are mapping to the GitLab ones.
We are going to sunset it. What we are building is part of the GitLab devops platform. Timeline and work that needs to be done are TBD.