Automating container image migration from Amazon ECR to GitLab

"We need to migrate hundreds of container images from Amazon Elastic Container Registry (ECR) to GitLab. Can you help?" This question kept coming up in conversations with platform engineers. They were modernizing their DevSecOps toolchain with GitLab but got stuck when faced with moving their container images. While each image transfer is simple, the sheer volume made it daunting.

One platform engineer perfectly said, "I know exactly what needs to be done – pull, retag, push. But I have 200 microservices, each with multiple tags. I can't justify spending weeks on this migration when I have critical infrastructure work."

The challenge

That conversation sparked an idea. What if we could automate the entire process? When platform teams move their CI/CD to GitLab, migrating container images shouldn't be the bottleneck. The manual process is straightforward but repetitive – pull each image, retag it, and push it to GitLab's Container Registry. Multiply this by dozens of repositories and multiple tags per image, and you're looking at days or weeks of tedious work.

The solution

We set out to create a GitLab pipeline that would automatically do all this heavy lifting. The goal was simple: Give platform engineers a tool they could set up in minutes and let run overnight, waking up to find all their images migrated successfully.

Setting up access

First things first – security. We wanted to ensure teams could run this migration with minimal AWS permissions. Here's the read-only identity and access management (IAM) policy you'll need:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ecr:GetAuthorizationToken",
                "ecr:BatchCheckLayerAvailability",
                "ecr:GetDownloadUrlForLayer",
                "ecr:DescribeRepositories",
                "ecr:ListImages",
                "ecr:DescribeImages",
                "ecr:BatchGetImage"
            ],
            "Resource": "*"
        }
    ]
}

GitLab configuration

With security handled, the next step is setting up GitLab. We kept this minimal - you'll need to configure these variables in your CI/CD settings:

AWS_ACCOUNT_ID: Your AWS account number
AWS_DEFAULT_REGION: Your ECR region
AWS_ACCESS_KEY_ID: [Masked]
AWS_SECRET_ACCESS_KEY: [Masked]
BULK_MIGRATE: true

The migration pipeline

Now for the interesting part. We built the pipeline using Docker-in-Docker to handle all the image operations reliably:

image: docker:20.10
services:
  - docker:20.10-dind

before_script:
  - apk add --no-cache aws-cli jq
  - aws sts get-caller-identity
  - aws ecr get-login-password | docker login --username AWS --password-stdin
  - docker login -u ${CI_REGISTRY_USER} -p ${CI_REGISTRY_PASSWORD} ${CI_REGISTRY}

The pipeline works in three phases, each building on the last:

1. Discovery

First, it finds all your repositories:

REPOS=$(aws ecr describe-repositories --query 'repositories[*].repositoryName' --output text)

2. Tag enumeration

Then, for each repository, it gets all the tags:

TAGS=$(aws ecr describe-images --repository-name $repo --query 'imageDetails[*].imageTags[]' --output text)

3. Transfer

Finally, it handles the actual migration:

docker pull ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com/${repo}:${tag}
docker tag ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com/${repo}:${tag} ${CI_REGISTRY_IMAGE}/${repo}:${tag}
docker push ${CI_REGISTRY_IMAGE}/${repo}:${tag}

What you get

Remember that platform engineer who didn't want to spend weeks on migration? Here's what this solution delivers:

• automated discovery and migration of all repositories and tags
• consistent image naming between ECR and GitLab
• error handling for failed transfers
• clear logging for tracking progress

Instead of writing scripts and babysitting the migration, the platform engineer could focus on more valuable work.

Usage

Getting started is straightforward:

1. Copy the .gitlab-ci.yml to your repository.
2. Configure the AWS and GitLab variables.
3. Set BULK_MIGRATE to "true" to start the migration.

Best practices

Through helping teams with their migrations, we've learned a few things:

• Run during off-peak hours to minimize the impact on your team.
• Keep an eye on the pipeline logs - they'll tell you if anything needs attention.
• Don't decommission ECR until you've verified all images transferred successfully.
• For very large migrations, consider adding rate limiting to avoid overwhelming your network

We've open-sourced this pipeline in our public GitLab repository because we believe platform engineers should spend time building valuable infrastructure, not copying container images. Feel free to adapt it for your needs or ask questions about implementation.

Get started with this and other package components with our CI/CD Catalog documentation.