Network Security Management Procedure
This is a Controlled Document
In line with GitLab’s regulatory obligations, changes to controlled documents must be approved or merged by a code owner. All contributions are welcome and encouraged.Purpose
GitLab architects a defense-in-depth methodology that enforces the concept of “least functionality” through restricting network access to systems, applications and services and ensures sufficient security and privacy controls are executed to protect the confidentiality, integrity, availability and safety of the organization’s network infrastructure, as well as to provide situational awareness of activity on GitLab’s networks.
Scope
GitLab’s network architecture is available to both internal and external users and hosts our DNS with Cloudflare incluing gitlab.com and gitlab.net.
Roles and Responsibilities
| Role | Responsibility |
|---|---|
| Infrastructure Team | Responsible for configuration and management |
| Infrastructure Management (Code Owners) | Responsible for approving significant changes and exceptions to this procedure |
Procedure
Cloudflare
Cloudflare provides a web application firewall (WAF), domain name system (DNS), and content delivery network (CDN) for the following zones:
- gitlab.com
- staging.gitlab.com
- gitlab.net
On-Call Reference
False Positive Triage Process
Change Workflow
When to use a Page Rule vs WAF Rules vs cf_allowlists internal-only
Whatever it is, create an issue in the Firewall tracker first and link it to the relevant issues. This firewall tracker is used to keep track of existing rules. This applies to all rules, regardless of how they are managed.
Next decide whether:
- Is it a redirect or changing a caching policy? Use page rules. Afterwards add an entry in the
page_rules.jsonin thecloudflare_importand MR it as described here internal-only - Is it a bulk allow of IP addresses for internal customers? Use cf_allowlists. internal-only
- Is it anything else? Use WAF Rules added via the firwall tracker and web UI.
Quick reference: WAF Rules
Temporary rules are subject to automatic expiration! See managing traffic for details.
To make it easier to know where to put the rule priority-wise, categorize the type of rule and pick the priority range from below
- 00000-14999: vulnerability hot-patch (block for everyone)
- 15000-29999: offender blocks (bots, attackers, etc.)
- 30000-44999: general WAF exceptions (bypass for everyone, except offenders)
- 45000-49999: internal and customer allow lists (managed via cf_allowlists. Not to be used manually)
- 50000-64999: WAF exceptions or blocks for non-allowlisted users
Then add the firewall tracker issue ID to the range. For example an attack, that is tracked in issue 1234 would get assigned priority 15000+1234 = 16234.
How we use Page Rules and WAF Rules to Counter Abuse and Attacks
Updating the WAF and Page Rules in Cloudflare
Adding Page Rules Using Terraform
The page rules are managed via Terraform. While changes can be made via the Cloudflare Web UI, that is not the preferred method to manage rules.
Where to make changes
The three zones that use Cloudflare each have a dedicated
cloudflare-pagerules.tf file in its Terraform environment.
How to make changes
The Cloudflare provider for Terraform will not adhere to the priority value
set in a page rule’s resource. All but the lowest priority rule will need a
depends_on section to point to the rule just below it in priority. And the
rule above it will need to be updated to depend on the new rule.
This forces Terraform to apply the rules in a specific order, preserving their priority.
Adding WAF Rules to the cf_allowlists
With any modification to the WAF rules in Cloudflare, the first step is creating an issue in the Firewall Issue Tracker. Refer to the managing traffic document to see how to create the proper issue type with proper labels and description.
cf_allowlists internal-only is a Terraform module that we’ve written to write WAF rules allowing customers'
or GitLab service IPs to bypass Cloudflare and any block that it may cause. The
allowlist is handled in the allowlist.json of the linked module. To add an IP
to it, simply update the file with the required information. A sample entry is
provided in the README of the module. Once the change is merged to master, you
will need to run terraform on the gstg and gprd environments to apply the
rules. If you are running it locally, you may need to run tf init -upgrade to
ensure you fetch the latest module with your updates.
Adding WAF Rules via the Web UI
Any modification to the WAF rules in Cloudflare requires an issue in the Firewall Issue Tracker. Refer to the managing traffic document to see how to create the proper issues type with proper labels and description.
Making manual changes via the Cloudflare UI can be read about here. A good practice is to create a new rule, but save it as a draft. This will allow the rule to be turned on and off as part of a production change process.
Verifying WAF and Page Rules
How cf_audit works
The cf_audit project
is designed to help us keep a “known good” dump of our Cloudflare configuration.
This is only an audit tool and not used to update any configuration on its own.
The script itself gets all of the configuration data for our Cloudflare zones
and account from the API and outputs the data into the reports/ directory of
the project.
There is a CI job that runs periodically to gather said data and commit it to
the cloudflare_import branch. The cloudflare_import branch is considered
the source of truth for the configuration. It then compares this information to
the known_good branch to determine what (if anything) has changed. The
known_good branch is considered to be the expected configuration. If a
configuration has changed, the job will be marked as failed, prompting manual
review of the changes.
If you’d like to watch a more detailed video about its inner workings, you can view this demonstration video which goes into much more detail.
How Page Rules work
General Information
Domain Name System (DNS)
- For the zones listed above, Cloudflare is the DNS resolver.
- DNS in Terraform is used to manage Cloudflare DNS entries.
Exceptions
Exceptions to this procedure will be tracked as per the Information Security Policy Exception Management Process.
References
- Parent Policy: Information Security Policy
- Production Architecture
- Encryption Policy
3fb9ff9f)
