Gitlab hero border pattern left svg Gitlab hero border pattern right svg

Meeting Service Level Objectives (Ticket SLAs)

  1. You are here:
  2. Handbook
  3. Support Team Handbook
  4. Support Workflows
  5. Meeting Service Level Objectives (Ticket SLAs)

On this page

Introduction

The Support team use one of two systems to ensure tickets get replies within the Service Level Objective we have with our customers:

  1. AMER and EMEA: Support Response Crew
  2. APAC: First Response Time Hawk and SLA Hawk

Support Response Crew

What is the Support Response Crew?

The Support Response Crew helps us to:

How does it work?

Crew Responsibilities

  1. On your crew day you should aim to be available for most of your normal working hours
  2. Join the #support_response-crew Slack channel
  3. Optionally join the permanent Zoom room linked at the top of the channel
    1. This is not a requirement
    2. It's fine to join with camera off
    3. Chat by text in Slack and join the Zoom when it's useful to coordinate / pair on tickets

The Response Crew in Action

  1. Send first replies to All-Regions and Preferred Region tickets and take ownership of those tickets
  2. Triage tickets (process 'needs-org' view)

The response crew does not work with Licensing and Subscription tickets. Support Engineers focused 100% on those tickets are not part of the response crew rotation.

Success Criteria

A Support Response Crew is succesful when:

  1. All new All Regions and Preferred Region tickets receive a reply within SLA.
  2. 100% of All Regions and Preferred Region tickets have an assignee.

1. Sending first replies

Work with your crew to send first replies as tickets arrive.

When new Self-managed or SaaS tickets with an SLA are received, there will be a notification in the #support_response-crew Slack channel. We have a contractual obligation to reply within the SLA so it's important we reply to all these tickets within the SLA.

  1. Look for new ticket notifications in the #support_response-crew Slack channel or check your Zendesk '(Your region) Needs Org/Triage/FRT' view for tickets that need a response.
  2. If your crew works in Slack, add the :eyes: emoji reaction to show you're looking at a ticket. (You can remove this if you've looked and don't know how to reply, or start a thread and ask your crew to collaborate with you.)
  3. Some new tickets will have received an automatic reply in Zendesk and already be Pending. If your crew works in Slack, add the :white_check_mark: emoji reaction so the rest of your crew knows that it's been taken care of. (You might like to read the customer's request and send an additional human reply if that will be helpful to the customer.)
  4. Send a reply to the customer (see FAQ below for tips on how to send a quick initial reply and then work on the ticket later)
    1. If the ticket priority does not match our Definitions of Support Impact let the customer know and agree on the new priority. Use the macro General::Changed priority.
    2. Assign the ticket to yourself when sending the first reply for tickets within your preferred region. You can always reassign later in the ticket's life. If the ticket has a preferred region outside of your region or you're taking PTO following your crew day, CC yourself on tickets you send during your crew shift. Doing so will keep you updated on the ticket progress while giving others the option to assign the ticket to themselves.
  5. If your crew works in Slack, add the :white_check_mark: emoji reaction to the bot message when you've replied to the ticket so the rest of your crew knows that it's been taken care of.
  6. If the ticket has the potential to become an emergency ping the engineer on-call for a heads-up.
FAQ about sending first replies
  1. If I assign tickets I send first replies on, won't I end up with too many assigned tickets?
    Since each person does the crew for one day a week, you'll pick up most of your tickets to work on during your crew day. If you find you have too many you can find another owner and reassign. Talk to your team and offer to swap tickets or find new owners. Assign to your manager if you have too many tickets and can't find someone to take ownership from you.
  2. I've got a nice new ticket I want to reply to, but it'll take a few hours to investigate. What should I do?
    You can send a quick reply and follow up with the customer later. Ask any clarifying questions and let them know when you plan to work on it and when they can expect a reply. You should then assign to yourself and put on-hold. You can use the 'Due Date' and 'Reminder' apps in Zendesk to remind you when to follow up. If it's not an urgent ticket it might be that you'll work on it the next day. The customer has an opportunity to reply to you if this isn't soon enough for them. The 4-minute SLA Clock video gives general guidance on this workflow.
  3. Can I take unassigned tickets when I'm not in the crew?
    Absolutely! All team members should check for unassigned tickets when not on a crew day. To find tickets that need a reply when you're not on the crew, use the '(Your region)+All Regions Needs Assignee' view and start with any tickets that need a next reply but have not yet been assigned. Please assign them to yourself before replying to prevent others spending time researching the same ticket.

2. Triaging tickets - process 'needs-org' view

The crew is responsible for Triaging tickets to make sure they have:

Chat with your crew and work on these tickets when there are no imminent breaches and you have some spare capacity.

Collaborate with your crew!

You are responsible for collaborating with your crew. This is a key part of your crew day. Here's how you can do that:

  1. Let your crew know in the response crew Slack channel when you start your day (a wave emoji is good if you don't want to write a message).
  2. Let your crew know when you need to step away for food or another call and how long you think you'll be away for.
  3. Use emoji responses to bot alerts to show you are viewing a ticket and when you've replied (see 'Preventing Breaches' above).
  4. Start a thread from a bot alert if you have questions about a ticket or would like to collaborate on a reply.
  5. Join the Zoom call if you'd like to collaborate in real time - ask your crew in Slack if someone will join with you.
  6. Reply to your crew when they ask for help and be prepared to join the Zoom if someone wants to collaborate on a ticket.

These responsibilities are not optional! You are expected to communicate and collaborate on your crew day.

Response Crew FAQ

There are too many tickets and we can't keep up!

If your crew is under pressure:

  1. First replies can usually be done quickly. Follow the advice in the FAQ about sending first replies section "I've got a nice new ticket I want to reply to, but it'll take a few hours to investigate. What should I do?".
  2. Ask in #support_team-chat if others can help out during busy periods.
  3. Ping the manager on-call and let them know the crew is under pressure.

What about tickets that have already breached?

Please reply to these as soon as you're able. We should aim to have no 'New' 'red' breached tickets in the queue, so please don't ignore these.

How many hours do I have to do?

Be available in the channel during your normal office hours - you don't have to be there for all 8 hours - make sure to be available for 'most of your day' and communicate with the rest of the crew in Slack or the Zoom room for some of the time.

It's OK if you need to go to another call for an hour or head out for your lunch break etc. By being a crew of several people there will usually be someone available. Keep the crew updated with your availability.

If you're in the East of your region consider being available early in your day to handover from the West of the previous region. If you're in the West of your region, consider being available towards the end of your day to handover to the next region.

What happens if lots of people are out on a day?

If you notice that several people are out, please contact a manager and ask them to request volunteers to help out.

I'm in EMEA; how do I manage the schedule?

See 'EMEA Crew Schedule' below.

Regional Crew schedules

Each region will organize the crew schedule to fit the needs of their region.

EMEA Crew Schedule

How do I know what day I am in the crew?

Request in #spt_managers for you to be added to the "EMEA Support Response Crew" Google Calendar. When you are added, you will be able to add that calendar into your personal GCal view (see below).

I'm not available for some or all of my day - what do I do?

Please follow these steps in order:

  1. If you're out for the whole week, please edit the entry in the response crew calendar and delete yourself from the event for your day. (This shows that you're not available and not available to swap that week) That's it, nothing else to do.
  2. If you're unavailable for part of your day, that's OK. Help out with the crew when you can, but do keep in touch with your crew in Slack and let them know when you'll be unavailable.
  3. If you're on-call for emergencies, that's OK too - help with the crew as normal, and it's OK to stop crew work if an emergency comes in.
  4. If you're unavailable for more than half your day on a given week, but are available on other days, please swap with someone else.
    1. Chat with the team / people you know in Slack and find someone who is willing to swap
    2. Edit the 'EMEA Support Response Crew' calendar attendee lists to reflect the swap - delete yourself from the day you can't do and add yourself as an attendee for the day you can do. The person you swap with should do the same.
      1. Don't 'decline' the event - you need to remove yourself and save the calendar event.
      2. Only change the calendar for the days you're swapping - there's no need to change multiple weeks or change the crew you are in.

The EMEA scheduling system

The schedule is managed and updated by EMEA Support Managers.

We use a Google Calendar to schedule events, a spreadsheet to track and plan and Zapier to glue those together:

  1. Google Calendar EMEA Support Response Crew – you can view in several ways:
    1. Calendar (view in browser): EMEA Support Response Crew
    2. Go to your Google Calendar > Settings > Add calendar > Subscribe to calendar > enter "EMEA Support Response Crew"
    3. Calendar (ics format, to subscribe From URL): EMEA Support Response Crew
  2. Spreadsheet: EMEA Support Response Crew Tracker
  3. A zap to update the Sheet when the calendar changes

The calendar and sheet are owned by techsupport@gitlab.com google account. Login details for this account and Zapier are in 1Password team vaults.

How it works:

  1. The team have been added to the calendar with edit permissions
  2. The spreadsheet has a tab called Crew Rotations' with team details pulled from support-team.yml
    1. A column on this tab shows Roation 1 with 5 crews
    2. Each crew has at least one person with .com admin access to help with SAAS tickets
  3. Calendar entries have been manually created for the next 5 weeks
    1. Each crew does one day per week
    2. Each week the day your crew does changes (rotating through the week, e.g. Week 1 = Monday, Week 2 = Tuesday)
  4. At the end of 5 weeks we can create a new set of crews if desired (so that people get to work with new people)

For people onboarding: You'll be added to the calendar during your third or fourth week at GitLab. While you're still onboarding don't worry about crew work - but please feel free to join in to help learning as soon as you feel ready.

Weekly review by managers:

  1. Each Friday, a support manager will check the calendar for next week and make sure it is up-to-date to reflect people's out of office (have they swapped / removed themselves from their day).
  2. If we notice a day where there are not enough people we will ask for volunteers to cover the light days.
  3. Since the spreadsheet tracks days you've completed with the crew, volunteering can free you up in the future while we ask others to volunteer to ensure fair distribution of contributions.

Daily reminders to Slack

The Support Response Crew Bot sends reminders to the crew in Slack each morning.

AMER Crew Schedule

How do I know what day I'm in the crew?

Schedule is coordinated via Pager Duty

I'm not available for some or all of my day - what do I do?

Please follow these steps in order:

  1. If you're unavailable for part of your day, that's OK. Help out with the crew when you can, but do keep in touch with your crew in Slack and let them know when you'll be unavailable.
  2. If you're on-call for emergencies, that's OK too - help with the crew as normal, and it's OK to stop crew work if an emergency comes in.
  3. If you're unavailable for more than half your day on a given week, but are available on other days, please swap with someone else.

Daily reminders to Slack

The Support Operations Bot sends reminders to the crew in Slack each morning.

Response Crew Dashboards

You can see how we're doing on this Zendesk Explore Dashboard.

Look at the '.com and Self Managed NRT SLA achieved charts' for this week and last week. We're aiming for 85% achievement. If we can get to 95% that would be awesome! The 'this week' chart updates hourly.

First Response Time Hawk (If relevant in your region)

What is a First Response Time Hawk?

This is a rotating role, where someone is "on point" weekly (Monday to Friday), to make sure:

  1. new Support tickets get a first response as soon as possible and do not breach
  2. all new tickets are appropriately categorized and prioritized as they arrive

First Response Time Hawks directly drive achievement in our Service Level Agreement KPI.

FRT Hawk in Action

  1. Every Week on Fridays, Managers will add a First Response Time Hawk for each region to the Support Week in Review to announce who will be responsible that week.
  2. If you are FRT Hawk, you are responsible for:
    1. Helping facilitate first replies within SLA times for new tickets from 9am - 5pm in your time zone
      • Note: if you are a US Citizen and have access to the Federal instance, this should also be a part of your rotation.
    2. Triaging tickets to make sure they have:
      • The correct ticket form (such as GitLab.com, if they're asking for GitLab.com support). Take special notice of tickets with form Other Request. This is usually because the ticket was created by email. Note that a form change results in an auto-reply for tickets that don't have an org associated yet. Hence, it is recommended to first associate the user with an appropriate org and then change the form to the most relevant form type and fill in additional metadata where possible.
      • An associated organization
      • The correct priority
      • The correct Problem Type selected
  3. Keep an eye on the FRTH View in ZenDesk
  4. Ask your Support team mates for help if volume is high, or you are stumped.
  5. You should expect to see new and different things that you are not an expert in. Where possible, take the time to dive deep and try and understand the problem.

Things for an FRT Hawk to be mindful of

  1. Given that there's a FRT Hawk per region, your shift will overlap with another hawk.
    • This means you have a teammate within a team; yay! Be sure to communicate with your fellow hawk so you can help each other out in case one of you is stuck on a ticket, and also to make sure you're not both quietly working on the same ticket
  2. Your focus should be facilitating replies for tickets in the First Response Hawk queue and associating needs-org tickets with the appropriate organization.
  3. You might not cover all the tickets you wanted to cover on your shift, and that's okay
    • Just do your best
    • Take a break if you need one - making sure your team knows you're stepping away by posting in #support_team_chat
    • Ask the rest of the team for help when needed in #support_self-managed
  4. You may see tickets requesting information that is not in the realm of support. For example, you may get a ticket from a job applicant to GitLab requesting information on their application. In these cases, do your best to direct them to an appropriate contact and close the support ticket. In the case of hiring questions, you can ping the #talent acquisition channel in Slack to inform them that the candidate reached out, and update the candidate via the ticket.

FRT Dashboards

You can track your progress and see global results on this Zendesk Explore Dashboard

SLA Hawk (If relevant in your region)

What is the SLA Hawk role?

The SLA Hawk is a rotating role that you will carry out for one week at a time (Monday to Friday). You can view the rotations in PagerDuty

The SLA Hawk is responsible for ensuring that action is taken on the 'most breached' and 'next to breach' tickets in order to meet our SLAs and so that difficult tickets are not ignored. NOTE: It is the responsibility of all Support team members to ensure customer tickets are responded to within our SLA times. The SLA Hawk is not expected to reply to tickets themselves - your job is to find team members to help get a reply out (see below for more info)

The SLA Hawk should also perform a queue tidy up:

SLA Hawks drive achievement via our KPIs for Service Level Agreement KPI and Customer Satisfaction by making sure our customers are updated in a timely manner.

SLA Hawk in Action

  1. Each day, sort the Zendesk view for SM with SLA by 'Next SLA Breach' ascending, starting with most breached ticket first.
  2. Read the ticket yourself and if you feel you know what the next action is, do that (e.g. ask the customer for more information, solve the ticket if the issue is resolved, send your own reply if you have a great idea). There's no need to spend too long at this stage - 10 minutes at most. If the next action requires more work move on to the next step.
  3. If you feel the priority does not match our Definitions of Support Impact reach out to the customer and agree on the new priority. Use the macro General::Changed priority.
  4. If a ticket has an assignee:
    1. If the assignee is in your Region, link the ticket in #support-self-managed and ping them asking if they are able to take a look. We've adopted a convention of prefixing your message in Slack with [SLAH] so that folks know you're currently in that role.
    2. If the assignee is from another Region and it's during their “office hours” ask them if they could take a look.
    3. If the assignee is offline and the ticket/customer needs an update, follow the has no assignee process below.
    4. If assignee can’t currently work the ticket ask them to follow the Process in action for Working on Tickets.
  5. If a ticket has no assignee, or assignee is offline:
    1. Check to see if an engineer from your region has previously responded on the ticket, if so, ask them would they have the bandwidth to take the ticket.
    2. If the ticket is about a problem with a specific feature or specialized area of GitLab, reach out to any subject matter experts or stable counterparts in your region to see if they can assist in getting a reply out.
    3. If you can't locate a specific teammate to get a reply, let your regional team know (e.g. @support-team-<region> ) about the ticket. Once you have one or two volunteers, let the team know the ticket is taken care of via the thread.
    4. If you’re not getting any traction, and you feel you need help, ping your manager to let them know that noone is available to help.
    5. Once someone has taken ownership of the ticket, ask them if they're happy to assign it to themselves.
    6. If there are no imminent breaches (e.g. less than 2 to 4 hours) in the 'SM with SLA' queue - work on Needs Org & Triage and My Assigned Tickets queues.
    graph TD A[Sort 'Premium and Ultimate view by
    'Next SLA Breach'] --> B(Open the
    top ticket) B --> C{Can you reply
    to the ticket?} C -->|Yes| D[Send
    reply] --> H C -->|No| E[Ask for help in Slack
    and link the ticket] E -->|Help received| F[Send
    reply] --> H E -->|No help received| G[Mention your manager in Slack
    on the ticket thread] --> H[Go to the next ticket
    - if no imminent breaches -
    go to 'Free' view] --> B

Team Responsibilities for the SLA Hawk

FAQ about SLA Hawk role

  1. Do I need to do the SLA Hawk role all day every day? The role is not to work the tickets yourself but rather find an engineer to respond to the ticket. You probably only need to check for breached and breaching tickets 3 or 4 times per day, which fits in with our asynchronous working environment while ensuring that synchronous customer interaction is still maintained. This should make the role light-weight. Between ticket checkins, you can continue with your planned day.
  2. I feel uncomfortable asking someone else to reply to a ticket We recommend prefixing your message in Slack with [SLAH], this lets people know that you've been given the role to find people to help. We assume positive intent and the team knows that this is the job of the SLA Hawk.
  3. What do I do if the ticket assignee is on PTO? It can happen that you come across the ticket that is about to breach and which has the assignee that is on PTO (e.g. you get a message from PTO by Roots after you ping them in the thread). In that case, consider letting the customer know that the person who worked with them so far is on PTO (make the judgment if this is appropriate in your situation) and asking them whether they would like for their ticket to get the new assignee or they would like to wait for the original assignee to come back from PTO to continue working with them. If this is not appropriate, continue with the SLA Hawk workflow without relying on the current assignee (e.g. ask the help from the team or send a next response about their issue if you are able to). Either way, remind the person about support guidelines for taking time off.
  4. It's my first time as SLA Hawk, any suggestions?
    • This role feels harder than it really is when you're anticipating it - once you actually do it, you'll be surprised! You'll learn so much and gain confidence in handling any type of ticket.
    • The hawk is the manager of the queues, not the responder to the tickets - if you suspect that getting involved and sending a reply will take more than 15m, try to delegate getting a reply out to someone else (assignee, cc:. recent replier, or subject matter expert).
    • Your best is good enough! Realistically, you're unlikely to stop every ticket from breaching - if you’ve stopped even a handful then you’ve done a great job, that’s 2 or 3 fewer breaches that day.
    • Clear down the easy replies first - That's great, I'll mark the ticket solved. and so on. It improves the signal:noise ratio and someone has to write that reply.
    • Look after yourself. Set a timer to remember to take breaks and eat. The hawk roles are easy to get into a groove and forget to get up all day. Go outside, take a breather, then come back. See the first point in the FAQ as well.
    • You're not alone. You have a full team with you! Re-read the SLA Hawk in action section above - particularly points 4 & 5, and related…
    • Don't be tempted to take ownership of tickets. You will most likely NOT get to them later!
    • If there's a wave of soon to breach tickets rolling down the queue, ping the team so they know.
    • Remember your manager is there to support you - reach out to them (or other support managers).
    • Communicate/share feedback on your experience with the workflow in an issue or with support managers.

SLA Hawk Dashboards

You can see how we're doing on this Zendesk Explore Dashboard.

Look a the 'Self Managed NRT SLA achieved charts' for this week and last week. We're aiming for 85% achievement. If we can get to 95% that would be awesome! The 'this week' chart updates hourly.

Git is a trademark of Software Freedom Conservancy and our use of 'GitLab' is under license
View page source — Edit in Web IDE — please contribute. Creative Commons License