Blog News GitLab.com CI artifacts to use Google Cloud CDN
Published on: October 25, 2022
4 min read

GitLab.com CI artifacts to use Google Cloud CDN

GitLab CI users might benefit from faster downloads from edge caches closest to the user's location.

ESA_case_study_image.jpg

Over the next month and going forward, requests for GitLab CI artifacts downloads may be redirected to Google Cloud CDN instead of Google Cloud Storage. We anticipate that GitLab CI users may benefit from faster downloads from edge caches closest to your location.

Disclaimer: This blog contains information related to upcoming products, features, and functionality. It is important to note that the information in this blog post is for informational purposes only. Please do not rely on this information for purchasing or planning purposes. As with all projects, the items mentioned in this blog and linked pages are subject to change or delay. The development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab.

How will this work?

Currently when a CI runner or other client downloads a CI artifact, GitLab.com responds with a 302 redirect to a time-limited, pre-signed URL with a domain of storage.googleapis.com.

After this change, the domain will change to cdn.artifacts.gitlab-static.net.

The exception is for requests originating from within the Google Cloud Platform. These will continue to be redirected to Cloud Storage.

When will this change occur?

We expect to start the transition around the end of October 2022. This will be a gradual transition using a percentage-based rollout, so we anticipate that you will see an increasing number of your requests redirected to Google Cloud CDN instead of Google Cloud Storage until all of the requests are served by the former.

You can follow along with the progress of this initiative and raise any questions in this issue. We will post more detailed timelines in that issue as we refine the rollout plan.

How does this change impact you?

Since GitLab CI runners and certain clients automatically handle URL redirections already, we expect that downloads for CI artifacts should continue to work without any action.

We encourage upgrading to the latest version of the GitLab Runner in order to take advantage of the CDN. This feature was introduced in GitLab Runner v13.1.0. If a runner cannot download from the CDN host, it will retry without the CDN and download the artifact directly through GitLab.com.

However, if you have a firewall that only allows storage.googleapis.com, you will need to add cdn.artifacts.gitlab-static.net (34.110.204.38) to the allow list.

What do these warning messages mean?

With this change, users may see warning messages in the CI job logs:

read: connection reset by peer

ERROR: Downloading artifacts from coordinator... error couldn't execute GET against https://gitlab.com/api/v4/jobs/<job id>/artifacts?direct_download=true: Get "https://cdn.artifacts.gitlab-static.net/...
read tcp 172.17.0.2:59332->34.110.204.38:443: read: connection reset by peer  id=1234 token=<some token>
WARNING: Retrying...                                error=invalid argument
Downloading artifacts from coordinator... ok        id=1234 responseStatus=200 OK token=<some token>

This error suggests the runner was not able to access the CDN. Check your network firewalls and allow access to the IP 34.110.204.38.

Note that there are two Downloading artifacts from coordinator messages. The second attempt succeeded because the runner retried without the CDN.

x509: certificate signed by unknown authority

ERROR: Downloading artifacts from coordinator... error couldn't execute GET against https://gitlab.com/api/v4/jobs/<job id>/artifacts?direct_download=true: Get "https://storage.googleapis.com/gitlab-gprd-artifacts/...: x509: certificate signed by unknown authority  id=1234 token=<some token>

If you see this error with a Windows runner, upgrade to v15.5.0 since it is compiled with Go 1.18, which supports using the system certificate pool.

Otherwise, this error suggests the runner is configured with custom SSL certificates. You may need to update your certificates or include the certificates directly in the bundle.

Authentication required

Some clients may report a 401 error with Authentication required after requesting to download a job artifact:

<?xml version='1.0' encoding='UTF-8'?><Error><Code>AuthenticationRequired</Code><Message>Authentication required.</Message></Error>

This error message suggests the HTTP client is following the 302 redirect and sending the Authorization header with the redirected URL. This is a known issue with Java HTTP clients.

Update your client to drop the Authorization header the redirect. Google Cloud Storage ignores this header if it were set, but Cloud CDN rejects requests that have the Authorization header set.

We want to hear from you

Enjoyed reading this blog post or have questions or feedback? Share your thoughts by creating a new topic in the GitLab community forum. Share your feedback

Ready to get started?

See what your team could do with a unified DevSecOps Platform.

Get free trial

Find out which plan works best for your team

Learn about pricing

Learn about what GitLab can do for your team

Talk to an expert