Gitlab hero border pattern left svg Gitlab hero border pattern right svg

Category Direction - Importers

Last Reviewed: 2021-07-31

Introduction and how you can help

Thanks for visiting the direction page for Importers in GitLab. This page belongs to the Import group of the Manage stage and is maintained by Haris Delalić who can be contacted directly via email. This vision is a work in progress and everyone can contribute. If you'd like to provide feedback or contribute to this vision, please feel free to comment directly on issues and epics at GitLab.com.

Mission

The mission of the Importers category is to provide a great experience migrating from other applications in our customer's DevOps toolchains. This also includes GitLab-to-GitLab migrations, particularly self-managed GitLab to GitLab.com.

Our goal is to build importers that our customers find valuable, reliable and easy to use, thereby removing friction for migrating to GitLab and creating a more positive first impression when adopting GitLab.

Problems to solve

A typical organization looking to adopt GitLab already has many other tools. Artifacts such as code, build pipelines, issues and epics may already exist and are being used daily. Seamless transition of work in progress is critically important and a great experience during this migration creates a positive first impression of GitLab. Solving these transitions, even for the complex cases, is crucial for GitLab’s ability to expand in the market.

At this time, GitLab is targeting the following high-level areas for import: GitLab self-managed to GitLab.com, planning tools, CI/CD tools, and source control tools.

The Manage:Import group that is responsible for this direction page is currently focused on GitLab self-managed to GitLab.com import/export and our source code importers such as GitHub and Bitbucket. At this point in time the Jira Importer is owned by the Plan:Project Management group and the Jenkins Importer is owned by the Verify:Continuous Integration group.

In addition to the migration from other tools, creating easy and reliable GitLab-to-GitLab migrations allows our customers to choose how they access GitLab. While Projects and Groups can be exported and imported one at a time, there is no ability in the UI to perform bulk migrations which makes GitLab-to-GitLab migrations slow and not scalable.

Areas of focus:

Reliability

A large portion of our current importer issues are related to the reliability of the solution. Our imports don't always succeed and when they fail, there is little guidance for the user on steps they can take to remedy the failure. We will need to make our importers more reliable for our customers to have confidence in the migration process.

Importer coverage

Offering importers for the main competitors is crucial to enabling sales when competing against an incumbant solution. Our data shows that the most used importers for 3rd party solutions are GitHub, Bitbucket Cloud and Git importers. In addition to the GitLab importer, these importers are our main focus. Additionally, we will continue to evaluate building importers for solutions that we currently don't support.

Ease of use

Customers looking to import their data sometimes struggle to find the place in GitLab where they can initiate imports. Once found, the user interactions are not always intuitive and the flow is not fully user-friendly. Improving the user experience in this area will make our importers more lovable.

Overview of existing Importers

This table provides a quick overview of what GitLab Importers exist today and which most important objects they each support. This list is not exhaustive and the detailed information can be found on the Importers documentation page.

Import source Repos MRs Issues Epics Milestones Wiki Designs API *
GitLab GitLab Migration
GitLab Group Export
GitLab Project Export
GitLab GitLab .com
GitHub
Bitbucket Cloud
Bitbucket Server
Gitea
Git (Repo by URL)
Manifest file
CSV
Jira
FogBugz
Phabricator

* This column indicates whether this importer is accessible via API, in addition to the UI.

Maturity Plan

Importers is a non-marketable category, and is therefore not assigned a maturity level. However, we use GitLab's Maturity framework to visualize the current state of the high priority importers and discuss their future roadmap. Generally speaking:

GitLab Importer

The GitLab Importer functionality currently consists of two separate importers, the Single Project Importer and the Single Group Importer. Together, they allow for Group and Project migrations between two instances of GitLab.

The GitLab Importer is currently a Viable feature in GitLab. Both the Single Group and the Single Project importers are available through the API and the UI and most of the objects in each are being exported and imported. We recognize that larger customers have needs that exceed the capabilities of the current importer. Therefore, we are focused on developing a fast and scalable replacement to GitLab Importer (see GitLab Migration below).

GitLab Migration

GitLab Migration (fka. GitLab Group Migration) is the next step in our quest to make GitLab-to-GitLab migrations seemless and easy to use and this feature can be tracked in this epic. On our path to reach the Complete maturity level, we are creating a user-friendly migration tool that will be able to migrate entire groups with all of the belonging subgroups and projects in a single click. This maturity level will also include the ability to migrate users, which has been identified as the highest priority gap in the current functionality.

Once GitLab Migration reaches feature parity with the existing GitLab Importers, these file-based solutions will be deprecated, which will free up the Import Team to accelerate future feature development of all other importers.

GitHub Importer

The GitHub Importer is currently a Viable feature in GitLab. It is accessible through both the API and the UI and uses a direct link to the GitHub server to import the highest priority objects, such as merge requests, issues, milestones and wiki pages, in addition to the repository.

Haris Delalić walks through the current experience of importing from GitHub into GitLab:

We are currently in the planning stages to reach the Complete maturity level for GitHub Importer. This maturity level will allow the user to easily import a large number of projects and include most of the available objects, such as groups, snippets and members.

Bitbucket Cloud Importer

The Bitbucket Cloud Importer is currently a Viable feature in GitLab. It is accessible through the UI and uses a direct link to the Bitbucket Cloud repository to import the highest priority objects, such as merge requests, issues, and milestones, in addition to the repository.

There is currently no ongoing work to achieve the Complete maturity level for Bitbucket Cloud Importer. This maturity level would include most of the available objects and ability to invoke the import via the API.

Git Importer

The Git Importer is currently a Complete feature in GitLab. This is a basic importer that migrates any Git repository into GitLab. Most of our competitors (i.e. GitHub, Microsoft Azure, Atlassian Bitbucket) only offer this type of importer.

There is currently no ongoing work to achieve the Lovable maturity level for Git Importer. This maturity level would include an improved user experience.

Consuming Import capabilities

While the long-term goal for the Import group is to provide all the GitLab importing capabilities needed by our customers in our application, we recognize that GitLab's native capabilities may not support specific migration scenarios. Often, we're not aware of these requirements until a large customer provides us with specific migration requirements.

To support such requirements, GitLab's import capabilities are addressed in 3 ways:

UI

Selecting Import project on the New project page and Import group on the New group page allows you to create a new project or group by importing from existing data using an intuitive user interface.

Import API

The import/export API allows you to script project imports from GitHub, Bitbucket Server, trigger the GitLab Project Export/Import, or the GitLab Group Export/Import.

Professional Services

Our Professional Services team builds customizable tools that leverage the Import API to help customers automate scaled migrations. These tools allow Professional Services to quickly react to customer needs. An example of those tools is Congregate (GitLab team members only), which can be used to coordinate and customize large migration projects.

Note

When import capabilities are already available in the API, Professional Services is typically the fastest way of responding to urgent customer requests. In the interest of a quick response, Professional Services may even add a capability to the API and then use it in Congregate, which builds out the capabilities of our Importers.

When capabilities are not available (via API or UI) and Professional Services is blocked, the Import group will prioritize improvements to fill the gap.

Over time, we see Professional Services focusing entirely on white-glove migration work for customers rather than scripting functional gaps. To support, the Import group will move API-only features into the UI and prioritize API improvements whenever needed. This epic tracks the work that will allow us to port the GitLab migration business logic from Congregate.

What's next & why

  1. Following the addition of Epics to Groups in GitLab Migration, the Import team is focused on completing the parity with Group Export/Import for GitLab Migration by implementing a GitLab Migration API. Once completed, the GitLab Migration will have feature parity with the existing Group Export/Import and become a viable feature.
  2. After the above is delivered, we will work to improve the reliability of our GitHub Importer in order to enable more customers to self-service their migrations from GitHub into GitLab. We practice Ruthless prioritization in the Import Team. This means that the GitHub Importer enhancements are prioritized behind the Parity with Group Export/Import feature.

What is not planned right now

Group Import continuously evaluates and updates the Importers' direction and roadmap. As part of that effort, new Importers such as Trello, CircleCI, Subversion and Azure DevOps (TFS) are being discussed. While these discussions may ultimately lead to the implementation of a new feature or a new Importer, none of them are being planned at this time.

Given our focus on GitLab Migration and completing the GitHub Importer, we are not prioritizing any work that is not in scope of these two efforts. This includes all other importers, as well as issues for GitHub Importer which are not related to our focus area.

Group Import is not focused on the ability to regularly back up and restore your GitLab data, for example nightly backups of all your data. For more information on this use-case, please see the Backup and Restore category direction page.

Note

If you'd like to contribute feedback on areas you'd like to see prioritized, please add them as comments in the corresponding epic for this category.

Git is a trademark of Software Freedom Conservancy and our use of 'GitLab' is under license