Ultimate guide to migrating from AWS CodeCommit to GitLab

On July 25, 2024, AWS made a significant announcement regarding its CodeCommit service. As detailed in their official blog post, AWS has decided to close new customer access to CodeCommit. While existing customers can continue using the service, AWS will not introduce new features, focusing only on security, availability, and performance improvements.

This announcement has prompted development teams to consider migrating their repositories to alternative Git providers. In light of these changes, we've prepared this comprehensive guide to assist teams in migrating to GitLab and integrating with other AWS services.

Note: For more details on AWS's official migration recommendations, please refer to their blog post.

About this guide

This guide provides comprehensive information for development teams using GitLab who are considering integration with AWS services or planning to migrate from AWS-hosted Git repositories to GitLab.com. The guide is structured into three main sections:

• Parallel migration to GitLab: Explains how to gradually migrate from existing AWS-hosted repositories to GitLab.com while minimizing risks.

• Integration with AWS CodeBuild: Provides steps to integrate GitLab repositories with AWS CodeBuild, setting up a powerful continuous integration (CI) environment.

• Integration with AWS CodePipeline: Details how to connect GitLab repositories with AWS CodePipeline to build efficient continuous delivery (CD) pipelines.

• Downstream integrations for CodePipeline and CodeStar Connections: Explains how to leverage GitLab-AWS connections for widespread service access, unlocking a cascade of integration possibilities across the AWS ecosystem.

Through this guide, you'll learn how to combine the powerful features of GitLab and AWS to create an efficient and flexible development workflow.

Section 1: Parallel migration to GitLab

For those considering migrating Git repositories hosted on AWS to GitLab.com, this section, which is a phased approach, introduces methods to achieve migration while minimizing risks. By leveraging GitLab's mirroring capabilities, you can maintain existing development flows while testing the new environment.

Why is parallel migration important?

Large-scale system migrations always involve risks, particularly potential impacts on ongoing development work, existing integrations, and automated processes. Adopting a parallel migration approach offers the following benefits:

1. Risk minimization: Test the new environment while keeping existing systems operational.
2. Seamless transition: Development teams can gradually acclimate to the new system.
3. Integration testing: Thoroughly test all integrations and automation in the new environment.
4. Future-proofing: Enable teams to gradually migrate to GitLab CI/CD in parallel to existing CI.

Parallel migration is not required if it is already known that you want to cut over directly to GitLab.

Steps for migrating to GitLab.com

Step 1: Get set up on GitLab.com

• Check if your company already has a group in use on GitLab.com and whether they have single sign-on (SSO) set up – if they do, then you will want to use both.

• If your company does not have a presence on GitLab.com, visit GitLab.com and create a new account or log in to an existing one.

• Create a new company namespace (a group at the root level of gitlab.com).

• Pick a name that reflects your entire company (and is not already taken).

Step 2: Import repository

For parallel migration: Use GitLab's pull mirroring feature to automatically sync changes from AWS-hosted repositories to GitLab.com.

1. Navigate to the target group GitLab.com.
2. In the upper right, click "New project."
3. On the "Create new project" page, click "Import project."
4. On the "Import project" page, click "Repository by URL."
5. Enter the URL of your AWS-hosted repository in the "Git repository URL" field.
6. Underneath the Git repository URL field, check "Mirror repository."
7. Set up authentication: in the AWS CodeCommit console, select the clone URL for the repository you will migrate. If you plan on importing CodeCommit repositories into GitLab, you can use the HTTPS CodeCommit URL to clone the repository via GitLab Repository Mirroring. You will need to also provide your Git credentials from AWS for your identity and access management (IAM) user within GitLab. You can create Git credentials for AWS CodeCommit by following this AWS guide.

This setup will automatically pull changes from the AWS-hosted repository to GitLab.com every five minutes by default.

For more information, read our repository mirroring documentation.

Step 3: Test and validate integrations

1. CI/CD pipelines: Set up the .gitlab-ci.yml file in GitLab CI to replicate existing pipelines. You can read more about planning a migration from other CI tools into GitLab CI/CD.
2. Issue tracking: Import project issues and test workflows.
3. Code review: Set up the merge request process and test review workflows.

Step 4: Gradual migration

1. Start with small or non-critical projects to familiarize yourself with working on GitLab.com.
2. Provide training for team members and allow time to adapt to new workflows.
3. Gradually migrate more projects while ensuring integrations and workflows are problem-free.

For more information, see Automating Migrations from CodeCommit to GitLab.

Step 5: Complete migration

Once all tests and validations are complete and the team is comfortable with the new environment, plan for full migration. For each project:

1. Set a migration date and notify all stakeholders.
2. Perform final data synchronization.
3. Remove mirroring settings from the GitLab project.
4. Set AWS-hosted repositories to read-only and transition all development work to GitLab.com.

Step 6: Assess adoption of new capabilities

GitLab collaboration and workflow automation for developers is far richer than CodeCommit. It merits some time to learn what these capabilities are. The merge request process is especially rich compared to CodeCommit.

After repositories are stable on GitLab, it is very easy to experiment with GitLab CI/CD in parallel to an existing solution. Teams can take time to perfect their GitLab CI/CD automation while production workflows remain unaffected.

GitLab artifact management is also very capable with the Releases feature and many package registries.

Section 1: Summary

By adopting a parallel migration approach to GitLab, you can achieve a smooth transition while minimizing risks. This process allows teams to gradually adapt to the new environment and ensure all integrations and automations function correctly. Cutover migrations only omit a single setting checkbox if it is known that a parallel migration is not necessary.

Section 2: Integrating GitLab with AWS CodeBuild

For those wanting to build and test code from GitLab repositories using AWS CodeBuild, this comprehensive guide will help you set up an efficient CI pipeline.

Prerequisites

• GitLab.com account
• AWS account
• AWS CLI (configured)

Step 1: Create GitLab connection in AWS CodeStar Connections

1. Log in to the AWS Management Console and navigate to the CodeBuild service.
2. Select "Settings" > "Connections" from the left navigation panel.
3. Click the "Create connection" button.
4. Choose "GitLab" as the provider.
5. Enter a connection name and click "Connect to GitLab."
6. You'll be redirected to the GitLab authentication page.
7. Approve the necessary permissions.
8. Once successful, the connection status will change to "Available."

Step 2: Create AWS CodeBuild project

1. Click "Create build project" on the CodeBuild dashboard.
2. Enter a project name and description.
3. For source settings, select "GitLab" as the provider.
4. Choose the connection you just created and specify the GitLab repository and branch.

Note: From Step 3 forward, please configure the settings according to your specific environment and needs.

Summary of Section 2

This section explained in detail how to integrate GitLab repositories with AWS CodeBuild. This setup enables a continuous integration pipeline where code changes in GitLab are automatically built and tested using AWS CodeBuild.

Section 3: Integrating GitLab with AWS CodePipeline

For those looking to implement continuous delivery from GitLab repositories using AWS CodePipeline, this detailed guide will be helpful. The integration has become even easier now that GitLab is available as an AWS CodeStar Connections provider.

Prerequisites

• GitLab.com account
• AWS account
• AWS CLI (configured)

Step 1: Create GitLab connection in AWS CodeStar Connections

1. Log in to the AWS Management Console and navigate to the CodePipeline service.
2. Select "Settings" > "Connections" from the left navigation panel.
3. Click the "Create connection" button.
4. Choose "GitLab" as the provider.
5. Enter a connection name and click "Connect to GitLab."
6. You'll be redirected to the GitLab authentication page.
7. Approve the necessary permissions.
8. Once successful, the connection status will change to "Available."

Step 2: Create AWS CodePipeline

1. Click "Create pipeline" on the CodePipeline dashboard.
2. Enter a pipeline name and click "Next."
3. Select "GitLab" as the source provider.
4. Choose the connection you just created and specify the GitLab repository and branch.
5. Select the Trigger type: You can trigger CodePipeline pipeline execution based on either pull or push events against specific branches and file types within your repository.

Note: From Step 3 forward, please configure the settings according to your specific environment and needs.

Summary of Section 3

This section detailed how to integrate GitLab repositories with AWS CodePipeline. This setup enables a continuous delivery pipeline where code changes in GitLab are automatically deployed to your AWS environment.

Section 4: Migrating to GitLab

Integrating GitLab with AWS unlocks powerful capabilities for streamlining your development and deployment workflows and helps to solve your source code management woes. This integration can be achieved in several ways, each offering unique benefits:

• Using AWS CodeStar Connections to link GitLab with AWS services enables a more cohesive workflow by allowing external Git repositories, like GitLab, to connect with various AWS services. This setup supports automated builds, deployments, and other essential actions directly from your GitLab repository, making your development process more integrated and streamlined.

• Connecting GitLab with AWS CodePipeline via AWS CodeStar Connections takes automation to the next level by allowing you to create a full CI/CD pipeline. This approach integrates GitLab with AWS CodePipeline, enabling you to automate the entire process – from source control and builds to testing and deployment – using AWS services like CodeBuild and CodeDeploy. This ensures a robust, scalable, and efficient delivery process.

1. Connecting GitLab with AWS services using AWS CodeStar Connections

AWS CodeStar Connections is a service that allows you to connect external Git repositories (such as GitHub or Bitbucket) to AWS services. You can also connect GitLab to AWS services via CodeStar Connections. When using GitLab, you may need to set up a custom connection as an HTTP Git server. The following AWS services can be connected to GitLab using this method:

• AWS Service Catalog

AWS Service Catalog helps organizations standardize and manage AWS resources. Integrating it with GitLab improves transparency in resource management and simplifies change tracking. Specifically, you can automate catalog updates based on GitLab commits, enhancing operational efficiency.

• AWS CodeBuild

AWS CodeBuild is a managed build service that compiles source code, runs tests, and produces deployable software packages. Integrating GitLab with CodeBuild allows automated build processes to start whenever code changes are pushed to GitLab. This ensures consistency in builds and facilitates easier collaboration and version control.

• AWS Glue Notebook Jobs

AWS Glue Notebook Jobs is a service that allows you to interactively develop and run data preparation and ETL (Extract, Transform, Load) tasks. Integrating GitLab with Glue Notebook Jobs enables version control for notebooks and ETL scripts, promotes collaboration among team members, and improves the quality management of data processing pipelines.

• AWS Proton

AWS Proton is a service that automates the development and deployment of microservices and serverless applications. By integrating GitLab with AWS Proton, you can manage infrastructure as code, automate deployments, and ensure consistent environment management, leading to more efficient development processes.

As AWS CodeStar Connections supports more services, connecting GitLab with additional AWS services will become easier. It's advisable to regularly check for new services that support CodeStar Connections.

2. Connecting CodePipeline with GitLab via AWS CodeStar Connections (including CodeDeploy)

AWS CodePipeline is a continuous delivery service that automates the release process for software. To connect GitLab with CodePipeline, you need to use AWS CodeStar Connections. This setup allows you to designate a GitLab repository as the source and automate the entire CI/CD pipeline. The primary actions supported by CodePipeline include:

• Source control: AWS CodeCommit, GitHub, Bitbucket, GitLab
• Build and test: AWS CodeBuild, Jenkins
• Deploy: AWS CodeDeploy, Elastic Beanstalk, ECS, S3
• Approval: Manual approval
• Infrastructure management: AWS CloudFormation
• Serverless: AWS Lambda
• Testing: AWS Device Farm
• Custom Actions: AWS Step Functions

By integrating GitLab with CodePipeline, you can automatically trigger the pipeline whenever code changes are pushed to GitLab, allowing a consistent process from build to deployment. Additionally, combining this with GitLab's version control capabilities makes it easier to track deployment history and states, leading to more flexible and reliable software delivery.

What you've learned

This guide has provided comprehensive information on migrating to and integrating GitLab with AWS. Through the four main topics, we've covered:

• Parallel migration to GitLab: How to gradually migrate from existing AWS-hosted repositories to GitLab.com while minimizing risks.
• Integration with AWS CodeBuild: Steps to set up a powerful CI environment integrated with GitLab repositories.
• Integration with AWS CodePipeline: How to build efficient continuous delivery pipelines using GitLab repositories.
• Downstream integrations for CodePipeline and CodeStar Connections: Leveraging GitLab-AWS connections for widespread service access, unlocking a cascade of integration possibilities across the AWS ecosystem.

As every organization's code hosting and integration implementation strategy is unique, this tutorial may be used as a starting point for your own GitLab + AWS integration and implementation strategy.

Additional resources

For more detailed information and advanced configurations, refer to the following resources:

• GitLab documentation
• AWS CodeBuild User Guide
• AWS CodePipeline User Guide
• GitLab CI/CD documentation
• Integrate with AWS

If you have questions or need support, please contact GitLab Support or AWS Support. We hope this comprehensive guide helps you in your AWS-GitLab integration journey.