Gitlab hero border pattern left svg Gitlab hero border pattern right svg

Importing Projects for Customers

  1. You are here:
  2. Support Team Handbook
  3. Support Workflows
  4. Importing Projects for Customers

Overview

This workflow is meant to provide guidance on when GitLab Team members might offer to import projects on behalf of customers or prospects as a courtesy, and the process for doing the imports.

Due to the shifting nature of what issues might be relevant, the specifics of this workflow may change.

If helpful, there is a video recording from 2019-05-22(internal) where the first half is a walkthrough of the process, and the second half talks through troubleshooting issues.

Security reviewed

In 2020-03, the process passed a security review in addition to the process review from 2019-02.

Criteria

The Support Team can offer to import a couple of projects as a best effort courtesy, but anything complex with many users or projects is out of scope. These users should be referred to Professional Services.

Additionally:

  1. The import method is a GitLab project export file.
  2. The requestor should be an existing customer or a prospect.
  3. The import was attempted and failed one or more times.
  4. The target location is a group (not a personal namespace, can be transferred if necessary).
  5. Target group is not empty.
  6. No import error is found (see below).

Identifying import errors

If the user is getting a version import/export error, ensure that the export originated from a compatible version of GitLab.

The following are often reported by users as errors, but are known:

See errors workflow for more details on searching Kibana and Sentry.

If there is an error, search for an existing issue. Errors where the metadata is throwing an error and no issue exists, consider creating one from Sentry.

If no error is found and the import is partial, most likely it is a timeout issue.

Pre-Approved Cases

Case 1: Large imports

Due to GitLab issue 24806, users with large projects (~1GB+) will often hit the sidekiq timeout. Imports stop when it hits an error or a timeout, but will always create the barebones resulting in partial imports. We have feature requests to force import and resume import.

Options to reduce project size:

Note, these are noted in the macro GitLab.com::Import::Verify import criteria.

Case 2: User mapping of items

An admin is required to corrently map users in GitLab as per our documentation. Without an admin account, repo commits will have correct user attribution, but issues, MRs, etc. will not. There is a feature proposal to not require an admin user being discussed.

Other cases

When in doubt, file an issue in dotcom-escalations and ask for a manager's input. If a manager approves, proceed with the import.

Process

Verify Permissions

Before offering to do the import, verify the following:

  1. User is a group owner.
  2. User has rights to create new projects in the space.

If not verified, let the user know we might be able to offer an import, but a group owner needs to contact us or they need to adjust their settings (depending on which of the above is not verified) using GitLab.com::Import::Verify import criteria macro.

Offer import and gather information

Once verified:

  1. Offer for the GitLab team to do an import on their behalf as a courtesy using the GitLab.com::Import::Offer import macro.
  2. Ask the user to (in macro):
    1. confirm the group where the project should be imported.
    2. confirm the full path where the project should be imported.
    3. confirm no project currently exists in the namespace with the project's name.
    4. provide a copy of their project's export.
    5. Depending on the case:
      • Correctly mapped users: that all active users have accounts on GitLab.com with matching company email address as their primary email account. All email addresses in the project export should have the company domain. A copy of the project export should be provided as soon as possible so that Support can pull the list of users for verification ahead of time.
      • No user mapping: provide the username for items to be attributed to.
    6. Provide date/time with timezone when they will send us the export file for importing (at least 1 business day in advance).
      • Note: If doing an admin import, additional lead time is required for the access request and doing the user list comparison.
  3. If scheduling ahead of time, create the infra issue with all available information (see [Request import section][#request-import) for details).
  4. For correctly mapped users only: Fill out an access request template on the access-requests tracker for admin level access to perform the import (see Access request information section for details).

Access request information (mapping users only)

Use the Single User Access Request template and provide the following information (replace group with org or group name):

Admin user access for customer import for infra's use with username: `group-import` 
Import request: INFRA-ISSUE-LINK

Note: Part of [the Support import process](https://about.gitlab.com/handbook/support/workflows/importing_projects.html) for customer.

Account Creation Information:

GitLab PRD | Role: `admin` | Rationale: `customer import` ; Please confirm, and enable 2FA for this user.

Verifying list of users (mapping users only)

For correctly mapped users only, the customer should send you a copy of the project export ahead of time so that there is ample time to do the next section and for the customer to verify the list.

  1. In a command line window, navigate to the folder where the project export resides.
  2. Use the export file user checker to pull a report of email addresses in GitLab.com (or not).
    • Reminder: The API and importer only checks primary email address, so users will match only on the one email address. Users can temporarily switch primary/secondary and switch them back if necessary.
  3. Also check that domain of email addresses belong to the company in the resulting list.
  4. Send the resulting list, listing both the users in GitLab.com and users not in GitLab.com to the customer to confirm. ZD macro: GitLab.com::Import::Verify user list
  5. If the customer responds that emails not in GitLab.com are employees no longer with the company, we can proceed. For these users, items will be mapped to the admin account then the ghost user once the admin account is deleted.

Request import

  1. Fill in the import template on the infra issue tracker with all available information.
    • If scheduling ahead of time, add date/time with timezone (recommend 1 hour later than expected time of receiving the import) and post in Slack #production (no ping). Add export link later. Example message: "Hi team, got an import request scheduled for 2019-00-00 at 00:00 UTC. Appreciate if you could assign this to whomever is on-call at the time. LINK"
    • If not scheduling, use today's date and "at earliest convenience". Ping the on-call with the request. Example: "Hi @user, got an import request. Could you kick this off when you have a few minutes? LINK"
  2. Add infra issue link as internal note to the ZD ticket.

After receiving export

  1. When receiving the project export: download, and ensure it unpacks. If not, ask the user for another copy.
    1. Create a folder to unpack into: mkdir project_export
    2. Unpack the project into a folder: `tar -zxvf filename.tar.gz -C project_export
  2. For correctly mapped users only:
    1. In a command line window, navigate to the folder where the project export resides.
    2. Use the export file user checker again to verify that all the email addresses matches the previously confirmed list.
    3. If the list does not match, send the list to the customer noting differences and ask them to re-confirm. If the customer does not respond, you may need to reschedule the import.
  3. Assuming success in unpacking (and email list is confirmed), respond to the user that we will get the import started and will send an update when the import is complete.
  4. If the customer shared a one-time download link, upload the export to the appropriate team gDrive folder and use the link to this version for the import issue. Share the file with the assigned infra SRE.
    • Note: As per the template, once infra has a copy "in hand", delete the file from gDrive.
  5. Update infra issue with all relevant information and add a comment to let the infra team know it's ready.
  6. Delete any local copies of the export.
  7. Imports may take a long time, so pass it on to the next region if necessary.

After import is complete

  1. Infra should comment on the issue when complete. Close infra issue.
  2. Ensure the export file is deleted, or remind the customer to delete theirs in the response.
  3. Let the user know the import is done and they should double check its completeness. ZD macro: GitLab.com::Import::Complete customer to verify
  4. For correctly mapped users only: After customer confirms everything looks okay, submit account removal request to delete admin account.
Edit this page — open Web IDE — please contribute. Creative Commons License