Diagnose Errors on GitLab.com

On this page


This guide provides resources for the diagnosing of 50x errors on GitLab.com. This is used when a user contacts support stating they're receiving an error on GitLab.com.


For reports of slowness, take a look at GitLab Grafana Monitor, especially:

Check on the #alerts, #production, and #incident-management Slack channels to ensure this isn't an outage or infrastructure issue.

  1. Obtain the full URL the user was visiting when the error occurred.

  2. Search Kibana
  3. Login to https://log.gitlab.net/app/kibana
  4. Select the correct time filter (top right) - e.g 30 minutes, 24 hours
  5. Use the search field to narrow down the results. For example you can search the gitlab-ee project for any mention of error using the query "gitlab-ee" AND "error"
  6. It's recommended to apply a Negative Filter to the logs below, these generate a large amount of noise and may not be relevant to your search.
    1. gitlab_error.log gitlab_access.log
  7. See the Kibana guide for more information.

  8. Search Sentry
    1. Login to https://sentry.gitlap.com/gitlab/gitlabcom/
      1. Use the search field to specify a relevant value. For example the following search would look for any errors with the URL shown that are unresolved. is:unresolved url:https://gitlab.com/gitlab-org/gitlab-ce/issues
      2. You can also search by username example user.username:MyUserName (this is case-sensitive)
      3. You can also select filtering values by clicking the "filter" button .
    2. See the Sentry guide for more information.
  9. Escalation
    1. Gather as much information. Make an internal note on the ticket including links to the logs found in Kibana or Sentry.
    2. Search the GitLab-CE project for any related issue.
    3. Make a comment in the #Support slack channel with a link to the error or short description of the error.
    4. Confirm if the issue is known or unknown. Issue is known - Issue is unknown
  10. Response
    1. Issue is known
    2. If the issue is known it should have a corresponding GitLab project issue.
    3. Respond to the user with information about the cause of the issue, linking to the related GitLab project issues.

    4. Issue is unknown
    5. Issues found in Sentry
      1. Convert the issue to a GitLab project issue by using the "Create GitLab Issue" button on the issue page.
      2. Comment on the issue providing a link to the ZenDesk ticket.
      3. Respond to the user with a link to the GitLab project issue.
    6. Issues found in Kibana
      1. Get a "short url" to the Kibana logs.
      2. Create and label a new GitLab-CE issue.
      3. Comment on the issue providing a link to the ZenDesk ticket and the Kibana logs.
      4. Respond to the user with a link to the GitLab project issue.