We’re sharing details on a vulnerability that caused CI jobs containing masked CI variables to be revealed. We’re communicating here to ensure affected users are aware and take action as well as to uphold our transparency value.

Am I affected?

• If you have masked variables on Gitlab.com, you could be affected. These variables could be at either the project or group level.
• If you ran a CI pipeline between February 11th 13:00 through Feb 16 01:16am UTC on GitLab.com shared runners that output masked variables,you are affected.
• If you are a self-managed customer who has deployed runner version 13.9.0-rc1, pipelines that are run by that runner which output masked variables are affected.

How could my masked variables be printed in the build logs?

Any mechanism that show the variable would have been done in an unmasked state while using runner version 13.9.0-rc1. Example commands include:

• echo $variable_name
• echo $variable_name > variable.txt; cat variable.txt
• some-command $masked_variable -> If this command outputs the parameters passed, it would have been unmasked.

If I am affected, what should I do?

All users who may be affected by this should review their jobs for any printed variables and rotate any secrets contained in masked variables immediately.

GitLab.com users

GitLab.com projects using shared runners between February 11th 13:00 until Feb 16 01:16am UTC and printed masked variables are affected by this regression and should rotate any secrets contained in masked variables immediately.

If you are using your own runner on gitlab.com, you should validate that you are not using runner version 13.9.0-rc1

Please note:

• Project visibility settings were not affected. If your project or project’s pipelines section was private, the masked variables would only be visible to members of your project with access to CI pipelines.
• If your jobs ran on a self-managed (not shared) runner using any version aside from 13.9.0-rc1, you should not be affected by this regression.
• If your jobs did not print any masked variables during this time, you should not be affected by this regression.
• In any of these cases, you may choose to use this opportunity to rotate your secrets out of caution, or as part of your regular security hygiene.

Self-managed users

If you are using 13.9.0-rc1, please upgrade to v13.9.0-rc2 or downgrade to 13.8.0 as soon as possible. You only need to upgrade to v13.9.0-rc2 if you are using 13.9.0rc1 and want to continue using the latest release candidate at your own risk.

Please note: The latest stable version v13.8.0 of GitLab Runner does not have this vulnerability so you do not need to upgrade to 13.9.0-rc2 if you are using 13.8.0.

For additional information and guidance on how to secure your GitLab instance, you can review the blog post "GitLab instance: security best practices".

Some background on this vulnerability

What does rc1 mean?

As part of our release process, runner release candidates (RC) are constantly deployed to and monitored within GitLab.com infrastructure. We roll out code in a scaled process, starting with our internal private runner managers, then move to the remainder of the shared runner fleet – allowing for a day in between deployments. Self-managed users can elect to use the latest RCs if they want to get code changes as fast as possible or test the latest code for regressions in their environments.

Details about our deployment timeline

On February 11th 13:00 until Feb 16 01:16am UTC, GitLab deployed version 13.9.0-rc1 of GitLab Runner on GitLab.com’s Shared Runners fleet.

Self-managed customers who are testing latest release candidates may have deployed 13.9.0-rc1 during that timeframe and should update immediately to either runner version 13.8.0 (latest stable) or 13.9.0-rc2 (latest release candidate). We have removed runner version 13.9.0-rc1 from distribution.

Actions GitLab has taken

We learned of the regression on Feb 15, and deployed a new version of runners v13.9.0-rc2 across the Shared Runners fleet to mitigate the issue.

We have contacted GitLab.com project owners who may be affected by this regression directly via email between Feb 16-18 with information on the regression and the instructions to rotate tokens. We are also sharing that information here. The emails were titled:

• Security Alert: Your public GitLab.com project needs your attention
• Security Alert: Your private GitLab.com project needs your attention
• Security Alert: Masked group variable vulnerability in Runner version 13.9.0-rc1

For self-managed users, we sent out a security alert email to all members of our security alerts mailing list. This email was titled "Security Alert: Runner version 13.9.0-rc1 has a masked variable vulnerability". The same information is shared in this blog post. You can subscribe to our security alerts to receive similar alerts, as well as notices of when regular and critical security releases are published.

What to do if you have questions

For GitLab.com customers

For GitLab.com customers with questions on how to determine if you are affected, please open a GitLab.com support ticket so a support engineer can help you.

For self-managed customers

If you are a self-managed customer that used runner version 13.9.0-rc1 and have questions on how to determine the scope of impact, you can open a self-managed support ticket and a support engineer can help you investigate in your specific environment.