Gitlab hero border pattern left svg Gitlab hero border pattern right svg

Importing Projects for Customers

  1. You are here:
  2. Handbook
  3. Support Team Handbook
  4. Support Workflows
  5. Importing Projects for Customers

Overview

This workflow is meant to provide guidance on when GitLab Team members might offer to import projects on behalf of customers or prospects as a courtesy, and the process for doing the imports. Due to the shifting nature of what issues might be relevant, the specifics of this workflow may change. Overall though, the import process should follow the flow outlined below.

graph TD; M[Customer does test imports] -->A[Determine Eligibility] -->B[Determine Import Type]; B-->C[Users Mapped]; B-->D[Users Not Mapped]; C-->F[Create Admin Account Access Request]; F-->E[Verify User List]; E-->H[Create Import Issue]; H-->K[Import Completes]; K-->G[Create Remove Access Request]; D-->I[Verify Permissions]; I-->J[Create Import Issue]; J-->L[Import Completes];

Stage 1: Determine Eligibility

When a request to import a project on behalf of a customer or prospect is received, we first need to determine if the request fits the criteria.

If the customer requires that only a couple projects or less be imported and those projects have a reasonable number of users within them, we can do it. If the request is complex or there are many projects that need importing, the requester should be referred to Professional Services instead.

Once you've determined that GitLab Support is able to process the import, proceed with verifying the Baseline Eligibility of all of the projects to be imported or determine that the requestor is approved because they meet the criteria of a Pre-Approved Case.

Baseline Eligibility

  1. The import method is a GitLab project export file.
  2. The requestor is an existing customer or prospect.
  3. The import was attempted and failed one or more times already.
  4. The target location is a group on GitLab.com, not a personal namespace. If this isn't the case, we can advise the user on how to create a group if needed.
  5. The target group is not empty.
  6. No import error is found (see Identify Import Errors).

Pre-Approved Cases

In addition to the above criteria, we can automatically offer to import a project for a customer if their case falls under the criteria of any of the following sections.

Large Imports

Due to 24806, users with large projects (typically ~1GB+ or 1000+ MRs) will often hit the sidekiq timeout. Imports stop when they hit a timeout, but an incomplete project will always be created, resulting in a partial import.

A project's size can be reduced by:

  1. Cutting down the repository size (if that's what's large) via a cleanup first using git filter-branch and running git gc and housekeeping afterward.
  2. Cutting down what's exported by editing the import_export.yml file on the self-managed instance. For more context see this comment.

These are also noted in the GitLab.com::Import::Determine_Eligibility.json macro.

NOTE: Once 196875 is implemented, imports should be able to be restarted, which should get around this case.

User Mapping

A GitLab.com admin is required to correctly map users in GitLab, per our documentation. Without an admin account, repository commits will have correct user attribution but issues and merge requests will not. We can import a project for a customer if they require this.

NOTE: Using an admin account will not be required once 223137 is implemented.

Other Cases

If you're unsure of whether or not we should perform an import for a specific requestor, get input via #support_escalations Slack channel or an internal issue. If a manager approves, proceed with the import.

Identify Import Errors

We can only offer an import if no validation (not timeout) error is found on a previously attempted import. Most timeout related imports end up with a partial import with very few or zero issues or merge requests. Where there is a relatively smaller difference (10% or less), then there are most likely errors with those specific issues or merge requests.

Anytime there is an error, ensure that the export originated from a compatible version of GitLab.

NOTE: See the Diagnose Errors on GitLab.com workflow for more details on searching Kibana and Sentry.

Here are some tips for searching for import errors in Kibana:

If there is an error, search for an existing issue. Errors where the metadata is throwing an error and no issue exists, consider creating one from Sentry.

If no error is found and the import is partial, most likely it is a timeout issue.

NOTE: In all cases, the import process is the same except that infra has a longer timeout period. If any validation errors are not worked out ahead of time, these will crop up during the infra import as well.

Known Import Issues

Use the GitLab.com::Import::Determine_Eligibility.json Zendesk macro to make the requestor aware of all of these requirements and get additional details from them. If the requestor's case is approved based on their followup reply, move on to Stage 2: Offering Import & Preparation.

Stage 2: Offering Import & Preparation

Follow the steps for either Users Mapped or Users Not Mapped depending on the type of import we're performing. Once the steps in either section are complete, move on to Stage 3: Import.

Timing and scheduling ahead

When customers request a specific time period for the imports to be done, they should always do a test import for each project and make note of how long it takes. It can be approximate, but should give everyone a clear idea of whether it's reasonable to be done within the given time period. Remember that both Support and Infra require additional time in addition to the actual import time, and too short of a time interval might be interrupted by an incident.

Users Mapped

If the customer requires that all issues, merge requests, and commits be mapped to the appropriate users in their organization, use the GitLab.com::Import::Offer Import (Users Mapped) Zendesk macro and then follow the next sections in sequence.

1. Create Admin Account Access Request

  1. Open an issue using the Single Person Access Request template and fill it out in its entirety (replace any instances of group with the group name).
  2. In a comment on the issue, tag your manager requesting their approval.
  3. If the account is not provisioned within a few days before the scheduled date, tag @it-ops-team in the #it_help Slack channel with a link to the request to ensure quick provisioning as soon as you receive a manager's approval.

In the access request, enter the following in the Person Details section.

- Admin user access for customer import for Infra's use with username: `group-import`
- Set email to: `group-import@gitlab.com`
- Ticket: <TICKET LINK>
- Import request: <INFRA ISSUE LINK>

**Note:** This is part of the [project import process](/handbook/support/workflows/importing_projects.html) for customers.

Then, enter the following for the Access Request section:

GitLab PRD | Role: `admin` | Please confirm, and enable 2FA for this user. | Rationale: `customer import`

2. Verify User List

The customer should send you a copy of the project export ahead of their chosen import time (if scheduled) so that there is ample time to do the next section and for the customer to verify the list and correct any errors.

  1. In your terminal use DCEF to pull a report that contains a list of user primary email addresses that are unique to the export file and ones that are unique to just the requestor's GitLab.com group.
  2. Examine the User emails unique to export file: section of the results and ensure that all email addresses listed are on the requestor's company domain, meaning no users have an email address on a generic domain such as Gmail.com.

If issues within the list are found:

  1. Reply to the requestor with the GitLab.com::Import::Verify User List (Problem Found) Zendesk macro, which will ask them to resolve the issues we found and send us a new project export once that's done. Once they have, repeat the 2. Verify User List section.

If no issues within the list are found:

  1. Send the resulting list of users not in GitLab.com to the customer to confirm using the GitLab.com::Import::Verify User List (Looks Good) Zendesk macro and await their reply. Once they reply and confirm that all email addresses that were found only in the export file are from employees no longer with the company, this section is complete.

NOTE: For these users, items will be mapped to the admin account, then the ghost user once the admin account is deleted.

Users Not Mapped

If the customer does not mention that they need project contributions (issues, MRs, etc.) mapped to their users, then we'll be importing their project(s) as a user of their choosing. Use the GitLab.com::Import::Offer Import (Users Not Mapped) Zendesk macro and then perform the steps in the following section:

1. Verify User Permissions

The customer should have replied stating which GitLab.com account they'd like the project imported as. The user they choose will have all contributions mapped to it. We need to verify that the following are true of this user.

  1. They have Owner permissions in the namespace that the project will be imported into.
  2. That user is Allowed to Create Projects in the namespace that the project will be imported into.

If the user is not an Owner, use the GitLab.com::Import::Require Group Permission macro as a response.

Stage 3: Import

This stage is applicable to both Users Mapped and Users Not Mapped imports. For Users Mapped, it may be started once the requestor has confirmed that the User List has been verified. For Users Not Mapped, it may be started once User Permissions have been verified.

Proceed in the following order:

  1. Check Project Export
  2. Either Import Scheduled or Import ASAP depending on when the user has requested that the import be performed.
  3. Stage 4: Cleanup once previous step and the import is finished.

Check Project Export

  1. Unpack the project export file into a folder: tar -zxvf filename.tar.gz -C project_export.
  2. [USERS MAPPED ONLY!] Perform one last Verify User List check to ensure no changes have been made.
  3. If the customer shared a one-time download link, create a folder in the ticket attachment folder and upload the export. Make sure to share the folder with the appropriate infra SRE(s), and use the link to this version for the import issue.
  4. Delete the export from GDrive once Infrastructure has confirmed that they have a copy (as per the infra import issue).
  5. Delete any local copies of the export.

Import Scheduled

Note that the expectation from the Reliablity team is 5 business days of lead time to accomodate proper scheduling of the work.

  1. Open an issue in the Infrastructure tracker using the Project Import template with all available information. This issue template will auto-assign the Reliability team managers and label the issue for triage.
  2. Add date/time with timezone (~1 hour later than expected time of receiving the import is recommended).
    • If there is no scheduled time, use 5 business days from date of receipt with "anytime" for time.
  3. Add the infrastructure issue link as an internal note to the Zendesk ticket.
  4. Once we receive a link to the export, update the issue with the link to the project.

Stage 4: Cleanup

After the import has completed successfully, perform the following steps.

  1. Close the infrastructure issue once an SRE has commented on it that the import has completed.
  2. Ensure the export file is deleted, or remind the customer to delete theirs in your next response.
  3. Send the macro GitLab.com::Import::Complete Customer to Verify to let the customer know that the import has completed and that they should double check that everything is in order.
  4. [USERS MAPPED ONLY!] After the customer confirms that everything looks okay, open an issue using the Access Change Request template to request that the admin account used for the import be deleted.

Zendesk Macros & Resources

Macros

Resources

If it's helpful, there is a video recording from 2019-05-22 (GitLab internal) where the first half is a walkthrough of the process, and the second half talks through troubleshooting issues.

Time And Date can be used to convert timezones to UTC, useful for when imports are scheduled for a future time.

Git is a trademark of Software Freedom Conservancy and our use of 'GitLab' is under license
View page source — Edit in Web IDE — please contribute. Creative Commons License