Gitlab hero border pattern left svg Gitlab hero border pattern right svg

Category Direction - Importers

Last Reviewed: 2021-03-12

Introduction and how you can help

Thanks for visiting the direction page for Importers in GitLab. This page belongs to the Import group of the Manage stage and is maintained by Haris Delalić who can be contacted directly via email. This vision is a work in progress and everyone can contribute. If you'd like to provide feedback or contribute to this vision, please feel free to comment directly on issues and epics at GitLab.com.

Mission

The mission of the Importers category is to provide a great experience importing from other applications in our customer's DevOps toolchains, thereby removing friction for migrating to GitLab. Our goal is to build the Importers that our customers find valuable, reliable and easy to use in order to create a more positive first impression when migrating to GitLab. This also includes GitLab-to-GitLab migrations, particularly self-managed GitLab to GitLab.com.

Problems to solve

A typical organization looking to adopt GitLab already has many other tools. Artifacts such as code, build pipelines, issues and epics may already exist and are being used daily. Seamless transition of work in progress is critically important and a great experience during this migration creates a positive first impression of GitLab. Solving these transitions, even for the complex cases, is crucial for GitLab’s ability to expand in the market.

At this time, GitLab is targeting the following high-level areas for import: GitLab self-managed to GitLab.com, planning tools, CI/CD tools, and source control tools.

The Manage:Import group that is responsible for this direction page is currently focused on GitLab self-managed to GitLab.com import/export and our source code importers such as GitHub and Bitbucket. At this point in time the Jira Importer is owned by the Plan:Project Management group and the Jenkins Importer is owned by the Verify:Continuous Integration group.

In addition to the migration from other tools, creating easy and reliable GitLab-to-GitLab migrations allows our customers to choose how they access GitLab. While Projects and Groups can be exported and imported one at a time, there is no ability in the UI to perform bulk migrations which makes GitLab-to-GitLab migrations slow and not scalable.

Areas of focus:

Importer coverage

Offering importers for the main competitors is crucial to enabling sales when competing against an incumbant solution. Our data shows that the most used importers for 3rd party solutions are GitHub, Bitbucket Cloud and Git importers. In addition to the GitLab importer, these importers are our main focus. Additionally, we will continue to evaluate building importers for solutions that we currently don't support, such as Trello and Azure DevOps.

Reliability

A large portion of our current importer issues is about the reliability of the solution. Our imports don't always succeed and when they fail, there is little guidance for the user on steps they can take to remedy the failure. We will need to make our Importers more reliable for our customers to have confidence in the migration process.

Ease of use

The current importer flow is hard to discover and not always there where the users are looking for it. The user experience is not always user friendly and making improvements in this area will make our importers more lovable.

Overview of existing Importers

This table provides a quick overview of what GitLab Importers exist today and which most important objects they each support. This list is not exhaustive and the detailed information can be found on the Importers documentation page.

Import source Repos MRs Issues Epics Milestones Wiki Designs API *
GitLab Group Migration
GitLab Group Export
GitLab Project Export
GitLab GitLab .com
GitHub
Bitbucket Cloud
Bitbucket Server
Gitea
Git (Repo by URL)
Manifest file
CSV
Jira
FogBugz
Phabricator

* This column indicates whether this importer is accessible via API, in addition to the UI.

Maturity Plan

Importers is a non-marketable category, and is therefore not assigned a maturity level. However, we use GitLab's Maturity framework to visualize the current state of the high priority importers and discuss their future roadmap. Generally speaking:

GitLab Importer

The GitLab Importer functionality currently consists of two separate importers, the Single Project Importer and the Single Group Importer. Together, they allow for Group and Project migrations between two instances of GitLab.

The GitLab Importer is currently a Viable feature in GitLab. Both the Single Group and the Single Project importers are available through the API and the UI and most of the objects in each are being exported and imported. We recognize that larger customers have needs that exceed the capabilities of the current importer. Therefore, we are focused on developing a fast and scalable replacement to GitLab Importer (see GitLab Group Migration below).

GitLab Group Migration

GitLab Group Migration is the next step in our quest to make GitLab-to-GitLab migrations seemless and easy to use and this feature can be tracked in this epic. On our path to reach the Complete maturity level, we are creating a user-friendly migration tool that will be able to migrate entire groups with all of the belonging subgroups and projects in a single click. This maturity level will also include the ability to migrate users, which has been identified as the highest priority gap in the current functionality.

Once the GitLab Group Migration reaches feature parity with the existing GitLab Importers, these file-based solutions will be deprecated, which will free up the Import Team to accelerate future feature development of all other importers.

GitHub Importer

The GitHub Importer is currently a Viable feature in GitLab. It is accessible through both the API and the UI and uses a direct link to the GitHub server to import the highest priority objects, such as merge requests, issues, milestones and wiki pages, in addition to the repository.

Haris Delalić walks through the current experience of importing from GitHub into GitLab:

We are currently in the planning stages to reach the Complete maturity level for GitHub Importer. This maturity level will allow the user to easily import a large number of projects and include most of the available objects, such as groups, snippets and members.

Bitbucket Cloud Importer

The Bitbucket Cloud Importer is currently a Viable feature in GitLab. It is accessible through the UI and uses a direct link to the Bitbucket Cloud repository to import the highest priority objects, such as merge requests, issues, and milestones, in addition to the repository.

There is currently no ongoing work to achieve the Complete maturity level for Bitbucket Cloud Importer. This maturity level would include most of the available objects and ability to invoke the import via the API.

Git Importer

The Git Importer is currently a Complete feature in GitLab. This is a basic importer that migrates any Git repository into GitLab. Most of our competitors (i.e. GitHub, Microsoft Azure, Atlassian Bitbucket) only offer this type of importer.

There is currently no ongoing work to achieve the Lovable maturity level for Git Importer. This maturity level would include an improved user experience.

Consuming Import capabilities

While the long-term goal for the Import group is to provide all the GitLab importing capabilities needed by our customers in our application, we recognize that GitLab's native capabilities may not support specific migration scenarios. Often, we're not aware of these requirements until a large customer provides us with specific migration requirements.

An example of such requirement * For example, a large customer importing data from their Bitbucket Server into GitLab was not able to use the direct import feature. Due to their firewall restrictions, they had to stage an intermediary Bitbucket server to which they copied their project. However, the intermediary server did not contain the user emails and the import into GitLab was not able to map the users, as the mapping was done using user emails. In order to solve this problem for the customer, we [implemented](https://gitlab.com/gitlab-org/gitlab/-/issues/218609) a secondary way of mapping users - by username.

To support such requirements, GitLab's import capabilities are addressed in 3 ways:

Interface Description User preference Time to implement Cost to implement
UI Selecting Import project on the New project page and Import group on the New group page allows you to create a new project or group by importing from existing data using an intuitive user interface. High High High
Import API The import/export API allows you to script project imports from GitHub, Bitbucket Server, trigger the GitLab Project Export/Import, or the GitLab Group Export/Import. Medium Medium Medium
Professional Services Our Professional Services team builds customizable tools that leverage the Import API to help customers automate scaled migrations. These tools allow Professional Services to quickly react to customer needs. An example of those tools is Congregate (GitLab team members only), which can be used to coordinate and customize large migration projects. Low Low Medium

Note

When import capabilities are already available in the API, Professional Services is typically the fastest way of responding to urgent customer requests. In the interest of a quick response, Professional Services may even add a capability to the API and then use it in Congregate, which builds out the capabilities of our Importers.

When capabilities are not available (via API or UI) and Professional Services is blocked, the Import group will prioritize improvements to fill the gap.

Over time, we see Professional Services focusing entirely on white-glove migration work for customers rather than scripting functional gaps. To support, the Import group will move API-only features into the UI and prioritize API improvements whenever needed. This epic tracks the work that will allow us to port the GitLab migration business logic from Congregate.

What's next & why

  1. Following the delivery of GitLab Group Migration MVC in 13.8, the Import group is focused on moving toward parity with Group Export/Import for Group Migrations by completing Epic migration. This feature aims to create feature parity with the existing Group Export/Import and make the Group Migration a viable feature.
  2. In parallel to this, we are working on UX improvements, such as adding pagination to the Group list, and the ability for the user to select the page size.
  3. As time allows, we'll also work to improve our GitHub Importer by iterating toward a scaled experience that would make imports of large data sets from GitHub quick and easy, starting with automatically refreshing the import status for GitHub repositories. We practice Ruthless prioritization in the Import Team. This means that the GitHub Importer enhancements are prioritized behind the GitLab Group Migration feature.

What is not planned right now

Group Import continuously evaluates and updates the Importers' direction and roadmap. As part of that effort, new Importers such as Trello, CircleCI, Subversion and Azure DevOps (TFS) are being discussed. While these discussions may ultimately lead to the implementation of a new feature or a new Importer, none of them are being planned at this time.

Note

If you'd like to contribute feedback on areas you'd like to see prioritized, please add them as comments in the corresponding epic for this category.

Git is a trademark of Software Freedom Conservancy and our use of 'GitLab' is under license