- You are here:
- GitLab Direction
- Product Vision - Package
- Category Vision - Dependency Proxy
Many projects depend on a growing number of packages that must be fetched from external sources with each build. This slows down build times and introduces availability issues into the supply chain. In addition, many of these external sources come from unknown and unverified providers, introducing potential security vulnerabilities.
For organizations, this presents a critical problem. By providing a mechanism for storing and accessing external packages, we enable faster and more reliable builds. By empowering SecOps to set security policies and remediate known vulnerabilities, we will limit exposure to security issues. By providing a transparent, performant supply chain, we will improve collaboration and drive conversational development for our users.
Interested in joining the conversation for this category? Please join us in our
public epic where
we discuss this topic and can answer any questions you may have. Your contributions
are more than welcome.
Target audience and experience
This feature impacts five types of users:
- Software Developer: As a developer, you don’t want to spend time worrying about external packages. You want fast build times and a seamless deployment process. The dependency proxy will allow you to store packages for faster and more reliable access. We will start with Docker, our most popular package management tool and expand to include other tools such as NPM and Maven. In the future, we will provide more transparency to help troubleshoot build issues. You will be able to view build logs, track diffs and easily search through the entire proxy.
- Security Analyst: As a security analyst, you need visibility into which external dependencies are being introduced into the supply chain and by whom. The dependency proxy will provide that visibility, but also will grant you the ability to set policies and create lists of approved and banned packages. We know you work on more than one project, so we will give you the tools and ability to discover and navigate all of the proxy repositories at an organizational level. We’ll start with the command line, add support for APIs, and evaluate adding a centralized dashboard.
- QA Engineer: As a QA engineer, you need the ability to work across multiple projects and teams seamlessly. The goal of the dependency proxy will be to ensure that each codebase you test not only runs on your machine (or environment) but that you can start testing quickly and get back to automation.
- Systems Administrator: As a systems administrator, you need to know that your entire DevOps pipeline runs smoothly and efficiently. For those of you at organizations that limit internet access, the dependency proxy will allow you to control the chaos and manage a single, optimized storage system for all your external packages. We will provide deduplication, garbage collection and the ability to set your own policies to help optimize storage and save money.
- Compliance Teams: At GitLab, we strive to create a Platform where everyone can contribute. We know at large organizations that compliance teams have a vested interest in controlling and discovering which packages are used throughout the organization. As we continue to develop features for the dependency proxy, we will provide visibility into the supply chain for compliance teams. This may be reports and alerts or a centralized dashboard depending on future user research.
- Provide a single method of reaching upstream package management utilities, in the event they are not otherwise reachable. This is commonly due to network restrictions.
- Let proxy packages act as a cache for increased pipeline build speeds.
- Verify package integrity from one single place. See what has been changed and test them for security vulnerabilities (part of black duck model).
- Filter the available upstream packages to include only approved, whitelisted packages.
- Track which dependencies are utilized by which projects when pulled through the proxy. (Perhaps when authenticated with a
- Audit logs in order to find out exactly what happened and with what code.
- Operate when fully cut off from the internet with local dependencies.
- Enforce policies at the proxy layer (e.g. scan packages for licenses and only allow packages with compatible licenses).
What's next & why
We have launched the MVC of the dependency proxy with limited availability. In order to make the feature more widely available, we are focusing on authentication, performance and visibility.
gitlab-ee#11582, will focus on adding authentication to enable use of the dependency proxy with private projects. gitlab-ee#11548 will add support for Unicorn, allowing instances using Unicorn web servers to leverage the dependency proxy. gitlab-ee#11639 aims to increase awareness and drive adoption of the feature by improving navigation.
This category is currently at the "Minimal" maturity level, and
our next maturity target is Viable (see our definitions of maturity levels).
Key deliverables to achieve this are:
JFrog is the leader in this category. They offer 'remote repositories' which serve as a caching repository for various package manager integrations. Utilizing the command line, API or a user interface, a user may create policies and control caching and proxying behavior. A Docker image may be requested from a remote repository on demand and if no content is available it will be fetched and cached according to the user's policies. In addition, they offer support for many of major packaging formats in use today. For storage optimization, they offer check-sum based storage, deduplication, copying, moving and deletion of files.
However, since they have focused on solving all possible usecases, there is room for simplification and design improvements. We believe this will allow GitLab to provide a more accessible and easier-to-navigate solution. In addition, we provide added value by combining this with our own CI/CD services, improving speed and having everything on-premise.
Top Customer Success/Sales issue(s)
The top customer success and sales issues involve enabling the dependency proxy for the Container Registry. The MVC was the first step towards achieving that goal, but in order to achieve that, we must make the feature more widely available. gitlab-org#1372 tracks all of the features required make the dependency proxy available to all users.
Top user issue(s)
To improve capabilities for our existing users, we want to deliver gitlab-ee#9164 (npm) and gitlab-ee#9095 (Maven), which will add support for the dependency proxy to the npm and Maven repository.
Top internal customer issue(s)
Our top internal customer is the Distribution team, which would like to avoid relying on external sources for downloading dependencies. gitlab-ee#11548 will improve performance and scalability and is a key first step in allowing the Distribution team to leverage the dependency proxy.
Top Vision Item(s)
Our top vision item is gitlab-ee#11548 which will allow us to extend support for the dependency proxy to GitLab.com users. gitlab-ee#11582 will add authentication and allow for use of the dependency proxy with private projects.