Customer was opted into cloud licensing because of a bug and their instance activated cloud licensing, an entity change happened and the new subscription was not on cloud licensing
l = License.find 123
and then l.destroy
. You need to provide the license number for the first command. You can find this number in the URL of the cloud license in the customersdot license menu (example: https://customers.gitlab.com/admin/license/123123123)l = License.find 123
and then l.destroy
. You need to provide the license number for the first command. You can find this number in the URL of the cloud license in the customersdot license menu (example: https://customers.gitlab.com/admin/license/123123123)Zuora Subscriptions
tab in customersdot (URL example: https://customers.gitlab.com/admin/customer/123123/zuora_subscriptions)Cloud Licensing
checkbox. Click the Update
button.Cloud Activations
tab. Ensure that the code was generated for the correct subscription.Cloud Licensing
checkbox from step 2 should now be checked.Cloud licensing requires a connection to customers.gitlab.com
over port 443 (HTTPS), and this connection must remain available throughout the duration of using a cloud license. The GitLab server will typically check in once per day, as well as once during activation, and any time a manual sync is performed. While general networking is typically outside the scope of what we can support, there are a number of things we can easily test for to help users diagnose any potential network or HTTPS issues blocking the connections.
The connection will rely on the hostname customers.gitlab.com
, meaning a DNS lookup will be performed. The IP addresses of customers.gitlab.com may change periodically, so it's important to verify what they are right now:
For example, using dig or host
commands:
$ dig customers.gitlab.com
$ host customers.gitlab.com
Keep in mind, IPv6 may occasionally be a factor in network troubleshooting. If a server is configured to prefer IPv6 over v4, then be sure that their server is able to resolve IPv6 (AAAA) records:
$ dig AAAA customers.gitlab.com
$ host -t AAAA customers.gitlab.com
In most cases, a ping
will suffice to verify that a connection is available over an IP address, be it IPv4 or v6. After obtaining the current IP addresses:
$ ping 104.18.21.224
$ ping -6 2606:4700::6812:14e0 #ping6 may be available too, but is an alias for ping -6
Pinging the hostname (customers.gitlab.com
) is fine too, but directly at an IP ensures both IPs are available. A simple one liner can be used to ping both IPs:
dig +short customers.gitlab.com | xargs -I% ping -c1 -W5 %
Note that ping
is not technically a TCP or UDP connection, but rather an ICMP, which may occasionally be blocked for various networking reasons. A slightly more advance technique can be used to test TCP connections, using either nc
(preferred) or telnet
:
$ nc -zv customers.gitlab.com 443
Connection to customers.gitlab.com 443 port [tcp/https] succeeded!
$ echo "." | telnet customers.gitlab.com 444
Trying 104.18.20.224...
Connected to customers.gitlab.com.
Escape character is '^]'.
Connection closed by foreign host.
The simplest method to check this is by using curl
. Specifically, sending a HEAD request to https://customers.gitlab.com
, which will both verify TCP connectivity and ensure a secure TLS connection is available:
$ curl -I https://customers.gitlab.com/
A successful reply will generally look like a 302
(Found), meaning the connection was succesful, and customers.gitlab.com replied with a redirection. More concisely:
$ curl -IL -q https://customers.gitlab.com/ | grep -i location
location: https://customers.gitlab.com/customers/sign_in
Implies success.
In most circumstances, if system curl
is working, then there won't be a need to check further TLS settings. However, because GitLab does offer several options for configuring various TLS and SSL certificate settings, including custom certificate authorities, it may be necessary to perform some more advanced SSL/TLS troubleshooting.
For example, if the customer server is making use of custom certificate authorities (CA), such as when SSL packet inspection is employed, they will need to add that root CA certificate to /etc/gitlab/trusted-certs
on the server, then reconfigure GitLab.
openssl
to check and verify SSL connectivity:$ echo | /opt/gitlab/embedded/bin/openssl s_client -connect customers.gitlab.com:443
It may be useful to run a SSL Server Test and then provide the page to customers. It's beyond the scope of our support to assist with most of the output on this page, but providing it may prove useful to customer network admins for additional troubleshooting on their end.
A user's browser's DevTools can be particularly useful in diagnosing cloud license connectivity failures, especially since the GitLab internal API (graphql) response can be viewed.
ctrl+shift+i
) and navigate to the Network tab/admin/subscription
graphql
resources and view the Response pane. There may be errors recorded here, feel free to share.
graphql
resources may be present, and not all will be related to the cloud licensing activation process.Since there will be a lot of information presented in the DevTools, feel free to suggest that the customer generate a network HAR file and attach it to the ticket for closer inspection by us.
System tools and binaries such as ping
and curl
are helpful, but keep in mind that GitLab is a highly complex application that bundles many of its own tools and binaries. GitLab the application may handle outbound connections slightly differently than system-supplied binaries will. The best way to verify this is to check that the connection is possible directly from rails console. Login to the server and enter rails console with sudo gitlab-rails console
and run:
URI_PATH = '/api/v1/seat_links'
base_uri = EE::SUBSCRIPTIONS_URL
head_resp = Gitlab::HTTP.head(base_uri)
pp head_resp
That command will return output very similar to what a typical curl -I
would return, but it relies on internal ruby classes to route the request in the same way GitLab would, therefore exposing any potential issues that are happening specifically within the GitLab application.
At present, cloud licensing does not officially support network proxies, deep packet inspection, etc. But if the customer is aware of custom network proxy configurations on their end, they may be able to configure GitLab to ignore them via the no_proxy
environment variable. More information is available in our documentation on Setting custom environment variables
Cloud Activations
tab.