The following page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features or functionality remain at the sole discretion of GitLab Inc.
Hello GitLab Community,
With 2022 behind us, I wanted to provide an overview of the plans for the Package stage in 2023.
There are a few direction items that I'm excited about.
The first is adding a pipeline trigger for package registry events and container registry events. This is will help improve interoperability between the Package and Verify stages. It will also create the scaffolding for expanding GitLab Workflows in general. It also helps to make GitLab more secure. How? Right now users don't have an automated way of kicking off pipelines when their package has been updated. This prevents them from doing things like vulnerability scans and compliance checks when a new version has been pushed to their registry. This functionality will also unblock (from the Package Registry side) the Secure team from adding real-time or continuous scanning, a highly requested feature.
The second is building an importer tool to help customers import packages from JFrog's Artifactory to GitLab. Why is this important? Right now if customers want to migrate from Artifactory to GitLab they have to script everything themselves. The import tool will leverage
.yml to map and migrate all of their packages.
It's also worth mentioning that in FY24 we will only focus on our GA (generally available) package manager formats. Why? Because npm, Maven, PyPI, NuGet, and generic packages make up 80% of the package registry user base. In 2023, I want those 5 formats to be the best in class. To do so, we need to work through several issues for each format that will significantly improve the user experience and usability of the Package Registry.
For the Container Registry, I'm most excited about starting to rollout the next-generation container registry for self-managed customers. The next-generation container registry supports online garbage collection which can help our larger customers save hundreds of thousands of dollars in storage costs. It also includes significant performance improvements and lots of new, planned functionality that self-managed customers will need to upgrade for. This is expected to be a challenging project due to the complexity of self-managed installs and the number of unknowns. We plan to offer a beta program to help identify bugs/issues prior to making it generally available to customers.
One other improvement that's worth highlighting. Right now users can publish Helm charts and other types of OCI formatted files. The problem is that it's not possible to distinguish between container images and helm charts (or other files) The issue [GitLab-#38047](https://gitlab.com/gitlab-org/gitlab/-/issues/38047 will make it possible to distinguish between Helm charts and container images. But more importantly, it allows us to take a meaningful step forward on a unified artifact repository which has potential for the stage on the 5-year time horizon.
What's not being prioritized?
Please consider watching this 8-minute overview of GitLab Package's 2023 plans.
For the Container Registry, we are currently working using notifications for data instrumentation.
For the Package Registry, we are working on two key features. The first is an import tool that will help you to import packages from Artifactory to GitLab. The second is introducing GitLab Workflows for Package. Each feature will start with npm and then be rolled out to the other generally available formats. (Maven/PyPI/NuGet/Generic/Terraform)
We don't have any issues scheduled for the Dependency Proxy.
The goal of the Package Group is to build a product, that within three years, is our customer's single source of truth for storing and distributing images and packages.
Yes. As the PM for the Package stage, I hear regularly from customers and prospects that would like to migrate off of Jfrog's Artifactory. Their reasons for wanting to consolidate on GitLab are:
Typically the needs of these customers can be predictably segmented by the size of their organization. For the sake of simplicity, let's classify their needs as
Typically they’d like to know if we support format
x and if not when will we support it. The formats that we don’t support that we hear most often are:
(All of the above will be useful for ~Dogfooding as well)
If we support their requested format, these customers are often able to consolidate.
They are typically blocked by issues and bugs that are fairly straightforward to address. They are most likely to engage in issues or on Twitter. They may use a single project as their universal registry. They are concerned about inconsistent token support, storage costs, and management.
We often hear from large, enterprise organizations that they'd like to consolidate on GitLab and move away from their existing vendor. But, our advice to these organizations is that they wait until the GitLab Package product matures. When comparing GitLab to Artifactory or Sonatype, there are several key missing features that must be considered.
If you'd like to learn more, the below information contains a summary, competitive info, and other helpful content for each product category associated with the Package stage.
The GitLab Container Registry is a secure and private registry for Docker images. Built on open source software and completely integrated within GitLab. Use GitLab CI/CD to create and publish branch/release specific images. Use the GitLab API to manage the registry across groups and projects. Use the user interface to discover and manage your team's images. GitLab will provide a Lovable container registry experience by being the single location for the entire DevOps Lifecycle, not just a portion of it. We will provide many of the features expected of a container registry, but without the weight and complexity of a single-point solution.
Open source container registries such as Docker Hub and Red Hat's Quay offer users a single location to build, analyze, and distribute their container images. Docker Hub recently introduced rate limits for pulls from Docker Hub.
The primary reason people don’t use DockerHub is that they need a private registry and one that lives alongside their source code and pipelines. They like to be able to use pre-defined environment variables for cataloging and discovering images. Often DockerHub is used as a base image for a test, but if you are building an app, you will likely customize an image to fit your application and save it GitLab's private registry alongside your source code.
Artifactory integrates with several different CI servers through dedicated plug-ins, including Jenkins and Azure DevOps, but does not yet support GitLab. However, you can still connect to your Artifactory repository from GitLab CI. Here is an example of how to deploy Maven projects to Artifactory with GitLab CI/CD.
GitHub offers a container registry that supports Docker image formats. There are several nice features that they've included. One nice feature is that you can publish images to your namespace or your user account. We would like to create that same functionality via gitlab-#241027. Also, their user interface includes helpful metadata, such as how often it's downloaded and a readme. One concern worth raising is that we don't see a way to programmatically delete images. Given the cost of storing images, this could be a concern for organizations that heavily use GitHub's registry. Another limitation is that they only support authentication using your Personal Access Token. This is not ideal for organizations that would like to avoid using individual-level credentials. With the GitLab Container Registry, you may use a PAT, Deploy, or Job token to authenticate to the registry. According to the Jet Brain's 5th Annual Developer Ecosystem survey, GitLab is second to GitHub for artifact/package management.
Amazon offers a fully-featured registry and plans to add support for highly available, publicly hosted images.
Google Cloud offers a container registry that allows you to integrate with any CI/CD platform. The registry is free, although they do charge for storage and network egress. Google's registry includes container scanning and high availability.
JetBrains offers a container registry that allows you to add a project repository and publish images and tags using the Docker client or your JetBrains project. Although they do not currently have any documentation for administrative features, such as cleanup policies or garbage collection.
Digital Ocean offers a container registry that allows you store and configure private Docker images. In addition, they support global load balancing and caching in multiple regions. One potential drawback is that each Digital Ocean account is limited to 1 registry, whereas with GitLab each Project can have its own registry.
Our goal is for you to rely on GitLab as a universal package manager, so that you can reduce costs and drive operational efficiencies. The backbone of this category is your ability to easily publish and install packages, no matter where they are hosted.
You can view the list of supported and planned formats in our documentation here.
The below table lists our supported and most frequently requested package manager formats. Artifactory and Nexus both support a longer list of formats, but we have not heard many requests from our customers for these formats. If you'd like to suggest we consider a new format, please open an issue here.
|GitLab||Artifactory||Nexus||GitHub||Azure Artifacts||AWS CodeArtifact||Google Artifact Registry|
☑️ indicates support is through community plugin or beta feature
Interested in contributing a new format? Please check out our suggested contributions.
Artifactory and Nexus are the two leading universal package manager applications on the market. They both offer products that support the most common formats and additional security and compliance features. A critical gap between those two products and GitLab's Package offering is the ability to easily connect to and group external, remote registries. To date, GitLab has been focused on delivering Project and Group-level private package registries for the most commonly used formats. We plan on bridging this gap by expanding the Dependency Proxy to support remote and virtual registries.
Azure and AWS both offer support for hosted and remote registries for a limited amount of formats. Google has a product called Artifact Registry that is in Alpha and supports Java and Node. All of the cloud providers charge for Cloud storage and network egress.
GitHub offers a package management solution as well. They offer project-level package registries for a variety of formats. However, looking at GitHub's roadmap, they've deprioritized many features
GitHub charges for storage and network transfers. GitHub does a nice job with search and reporting usage data on how many times a given package has been downloaded. They do not have anything on their roadmap about supporting remote and virtual registries, which would allow them to group registries behind a single URL and allow them to act as a universal package manager, like Artifactory or Nexus or GitLab.
JetBrains offers a Package Registry with support for npm and more planned formats. They have an ambitious and exciting roadmap for 2021, including adding support for Maven, Python and PHP. It's interesting to see that they'd like to support signing of packages and virtual registries, two features we are interested in adding at Gitlab.
Many projects depend on a growing number of packages that must be fetched from external sources with each build. This slows down build times and introduces availability issues into the supply chain. For organizations, this presents a critical problem. By providing a mechanism for storing and accessing external packages, we enable faster and more reliable builds.
Our vision for the Dependency Proxy is to provide a product that will provide fast, reliable access to all of your dependencies, whether they are hosted on GitLab or any other vendor. In addition, the Dependency Proxy will work hand-in-hand with the planned Dependency Firewall, which will help to prevent any unknown or unverified providers from introducing potential security vulnerabilities.
Currently the Dependency Proxy allows you to proxy and cache images from DockerHub. This can help you to speed up your pipelines and reduce your external dependencies. However this is only the first step. In the coming milestones, we will expand the Dependency Proxy from a single, hardcoded endpoint, to the place where you can setup and manage all of your registries (both packages and images) in one place.
There are a few important terms that are worth sharing:
The below diagram demonstrates how you can use the Dependency Proxy to create a virtual registry which will look for and fetch dependencies from your hosted and remote registries. This will allow you to download all of your dependencies with a single URL, instead of having to remember which packages are hosted where.
Note: The above diagram shows all of your dependencies being resolved through the Dependency Proxy. Usage of this feature is not required. You can easily use your hosted and remote registries without grouping them in a virtual registry.
Artifactory is the leader in this category. They offer 'remote repositories' which serve as a caching repository for various package manager integrations. Utilizing the command line, API or a user interface, a user may create policies and control caching and proxying behavior. A Docker image or package may be requested from a remote repository on demand and if no content is available it will be fetched and cached according to the user's policies. In addition, they offer support for many of major packaging formats in use today. For storage optimization, they offer check-sum based storage, deduplication, copying, moving and deletion of files.
The below tables outline our current capabilities compared to JFrog's Artifactory and Sonatype's Nexus.
|Virtual registries||Coming soon||✔️||✔️|
*The Dependency Proxy currently supports one hardcoded remote registry, which allows you to proxy and cache container images hosted on DockerHub.
|Virtual registries||Coming soon||✔️||✔️|
**By default, when an npm or PyPI package is not found in the GitLab registry, the request will be forwarded to the public registry, either npmjs.com or PyPI.org respectively. Check out this speed-run to see how it works.
Many projects depend on packages that may come from unknown or unverified providers, introducing potential security vulnerabilities. GitLab already provides dependency scanning across a variety of languages to alert users of any known security vulnerabilities, but we currently do not allow organizations to prevent those vulnerabilities from being downloaded to begin with.
The goal of this category will be to leverage the dependency proxy, which proxies and caches dependencies, to give more control and visibility to security and compliance teams. We will do this by allowing users to create and maintain an approved/banned list of dependencies, providing more insight into the usage and impact of external dependencies and by ensuring the GitLab Security Dashboard is the single source of truth for all security related issues.
By preventing the introduction of security vulnerabilities further upstream, organizations can let their development teams work faster and more efficiently.
JFrog utilizes a combination of their Bintray and XRay products, to proxy, cache and screen dependencies. They also provide dependency graphs across multiple languages and centralized dashboards for the review and remediation of vulnerabilities. It is a mature product, that is generally well received by users. JFrog recently acquired Vdoo to and plans to update XRay to to include Vdoo’s extensive data and improved scanning across multiple dimensions, including configuration and applicability scanning.
GitHub's new package registry does a really nice job of creating visibility into the dependency graph for a given package, but they do not give users the ability to control which packages are used in a given group/project.
Users or organizations that deploy complex pieces of software towards Kubernetes managed environments depend on a standardized way to automate provisioning those external environments. Helm is the package manager for Kubernetes and helps users define, manage, install, upgrade, and rollback even the most complex Kubernetes application. Helm uses a package format called Charts to describe a set of Kubernetes resources.
Helm charts are easy to create, version, share and publish right within GitLab.
An important distinction between competitive products is that Helm 3 now officially supports using an OCI container registry as a Helm repository.