Gitlab hero border pattern left svg Gitlab hero border pattern right svg

Data Team - How We Work

  1. You are here:
  2. Business Technology
  3. Data Team
  4. Data Team - How We Work

Working With The Data Team

We're happy to help you achieve your goals with Data. Most of our work is driven through our Data Fusion Teams, but we do reserve some capacity to work on requests not linked to these initiatives. Here's the process to follow to create a new Data issue:

  1. Open a New Issue in the Data Team Analytics Project
  2. Choose a Template from the table below that best matches the work you'd like us to perform
  3. Complete the templated Description as completely as possible.
  4. Leave the Assignees blank. The Data Team will process your request as a part of our Daily Triage.

New Issue

Request Type Template To Choose
New Dashboard 'Visualization or Dashboard - New Request'
Add Data Source 'New Data Source'
Data Export 'Data Export Request'
New KPI 'KPI Template'
Data Quality Issue Coming soon

How we Work

Deciding What And How To Build

Not all data solutions require the same level of quality, scalability, and performance so we have defined a Data Development framework to help match required outcomes with level of investment. The Data Team works with all teams to build solutions appropriate to the need, but focuses on Trusted Data using Trusted Data Development.

Design Spike

Experimentation is a great approach to performing Explorational Data Development. Oftentimes, there can be multiple solutions to solve a business problem and we need a process to efficiently evaluate the different approaches and ultimately select a solution to promote to a Trusted Data solution. We use Design Spikes to facilitate experimenting. Design Spikes are particularly useful when the proposed solutions result in breaking changes or significant changes to the overall design, structure, and computing of the data tech stack.

The below steps should be followed when performing a Design Spike:

  1. Calculate Value: Establish the Value the data solution can provide GitLab. Value can be measured in a variety of ways, ranging from efficiency, to increased Sales, to reduced compute.
  2. Define Requirements: Create a Requirements document for the Design Spike to define the business and technical requirements the data solution must meet to be successful. Indicate whether each requirement is a Must Have or Nice to Have.
  3. Experimentation Design: This is the most complex part of the Design Spike. We need to decide how to test the data solutions versus defined requirements. Oftentimes, successfully testing data solutions requires simulating production workloads. We need to define the minimal viable sample size of data to include in the design to achieve experimentation results that would be representative of the whole data set in scope for the solution.
  4. Perform the experiment: This involves putting "fingers on the keyboard" and standing up a data solution in the Explorational data development environment. It also involves collecting the results of the experiment pursuant to what has been defined in the Define Requirements phase.
  5. Assessment: After the experimentation is complete, we should perform an analysis of how well each data solution performed against the requirements from the Define Requirements phase and see how well each data solution performed against the Must Have or Nice to Have requirements. We should put this analysis into a document along with a recommended solution based on the results of the Design Spike. We can then submit this to the respective DRIs and Stakeholders on the project for review, feedback, and final decisions on how we will proceed with the use case.

Documentation

The Data Team, like the rest of GitLab, works hard to document as much as possible. We believe this framework for types of documentation from Divio is quite valuable. For the most part, what's captured in the handbook are tutorials, how-to guides, and explanations, while reference documentation lives within in the primary analytics project. We have aspirations to tag our documentation with the appropriate function as well as clearly articulate the assumed audiences for each piece of documentation.

Prioritization

As a central shared service with finite time and capacity and with a responsibility to operate and develop the company's central Enterprise Data Warehouse, the Data Team must focus its time and energy on initiatives that will yield the greatest positive impact to the overall global organization towards improving customer results.

The Data Team uses a Value Calculator to quantify the business value of new initiatives (issue, epic, OKR, strategic project) to enable prioritization and ranking of the Data Team development queue. The Value Calculator provides a uniform and transparent mechanism for ranking and enables all work to be evaluated on equal terms. The value calculator approach is similar to the RICE Scoring Model for Product Managers and the Demand Metric Prioritization Model for Marketing.

Standing Priorities

Every day in Data brings a new challenge or opportunity. However, The Data Team strives to spend the majority of its time developing and operating the Enterprise Data Warehouse and related systems, keeping fresh data flowing through the system, regularly expanding the breadth of data available for analysis, and delivering high-impact strategic projects. Our standing priorities are listed below.

Rank Priority Description Target Allocation
1 Production Operations Activities required to maintain efficient and reliable data services, including triage, bug fixes, and patching to meet established Service Level Objectives. 10-20%, though will fluctuate as driven by incident frequency and complexity
2 Data Team OKRs The Data Team identifies 3-4 strategic-level OKRs per quarter, primarily focused on core infrastructure and data development that will be beneficial to the entire company. 60-75%, though this will fluctuate as driven by larger Functional Team OKRs
3 Other Other work, including Functional Team OKRs, as prioritized and ranked using the Value Calculator 15-25%

We use scoped labels in GitLab to track our issues across these priorities.

Resolving Prioritization and Blocking Issues

In rare situations established SLOs do not meet turnaround needs and in these cases the Data Team provides an expedite response capability. The Data Team will provide an date estimate if expedited request cannot be handled per the expedite response SLO.

Data Team Value Calculator

The calculator below is based on the following Value Calculator spreadsheet. Please select the values below to define the value of new work.

Quarterly and Milestone Planning

Our planning process is called the Planning Drumbeat and it encompasses Quarterly Planning and Milestone Planning. The Planning Drumbeat is one of the most important activities the Data Team performs because it helps us align our work with the broader company, while remaining agile enough to manage shifting business priorities.

Weekly Roll Up Reporting

Business Technology developed Rolly Bot to automate the creation and dissemination of weekly status updates. By using their tool, the Data Team is able to send out regular updates about their OKRs via email and Slack with minimal time commitments from team members. One roll up is generated for the Data Fusion team and a second is created for the Data Platform team.

Process:

  1. Create Epics for tracking KRs related to our Quarterly Objectives
  2. Add the appropriate Label to each epic
    • Data Platform Team: Data-Platform-weekly-rollup
    • Data Fusion Team: Data-Fusion-weekly-rollup
    • Data Science Team: Data-Science-weekly-rollup
    • Data Collaboration Team: Data-Collaboration-weekly-rollup
  3. Add the following text in each Epic description, filling in the details for each KR: ```
    1. Stakeholders: [Business stakeholder(s) tags so they receive the weekly roll up email]
    2. DRI: @iweeks [team member(s) tags responsible for completing the KR]
    3. Next Release Date: [Date of Release]
    4. Status: [Red, Yellow, Green]
    5. Stage: [Triage, Validation, Scheduling, Scheduled, Development, Review, Completed, Blocked]
    6. Achieved: [Bulleted list of what has been completed]
    7. Features Included in Next Release: [Bulleted list of what is scheduled to be completed in the next release]
    8. Blockers: [Details of any blockers (technical, business, other)]
    9. Overall Estimated Size: [T-Shirt Size Estimate]
    10. Business Impact Statement: [Short summary of business impact] ```
  4. During milestone planning, the DRI will be responsible for updating the above details for the KR in the Epic. The best practice is to write the update for the entire milestone.
  5. At the end of the quarter, the roll up labels will be removed from the KR Epics and added to the new quarter's KR Epics.

Note: The release date will often be the next milestone based on our milestone planning process, but it could be sooner or later than that, depending on the specific KR.

Current limits:

  1. Only Epics can be added to the weekly roll up (i.e. not Issues)
  2. Only 20 Epics can be added to the weekly roll up

Issue Types

There are three general types of issues:

Not all issues will fall into one of these buckets but 85% should.

Discovery issues

Some issues may need a discovery period to understand requirements, gather feedback, or explore the work that needs to be done. Discovery issues are usually 2 points.

Introducing a new data source

Introducing a new data source requires a heavy lift of understanding that new data source, mapping field names to logic, documenting those, and understanding what issues are being delivered. Usually introducing a new data source is coupled with replicating an existing dashboard from the other data source. This helps verify that numbers are accurate and the original data source and the data team's analysis are using the same definitions.

Work

This umbrella term helps capture:

It is the responsibility of the assignee to be clear on what the scope of their issue is. A well-defined issue has a clearly outlined problem statement. Complex or new issues may also include an outline (not all encompassing list) of what steps need to be taken. If an issue is not well-scoped as its assigned, it is the responsibility of the assignee to understand how to scope that issue properly and approach the appropriate team members for guidance early in the milestone.

Incidents

Incidents are times when a problem is discovered and some immediate action is required to fix the issue. When this happens, we make an Incident Issue in the Data Team Project. The process for working through incidents is as follows:

Data Team Incidents can be reviewed in Incident Overview page within the main project.

Workflow Summary

Stage (Label) Responsible Description Completion Criteria
workflow::1 - triage Data New issue, being assessed Item has enough information to enter problem validation.
workflow::2 - validation Data, Business DRI Clarifying issue scope and proposing solution Solution defined with sign off from business owners on proposed solution that is valuable, usable, viable and feasible
workflow::3 - scheduling Data Waiting for scheduling Item has a numerical milestone label
workflow::4 - scheduled Data Waiting for development Data team has started development
workflow::5 - development Data Solution is actively being developed Initial engineering work is complete and review process has started
workflow::6 - review Data Waiting for or in Review MR(s) are merged. Issues had all conversations wrapped up.
workflow::X - blocked Data, Business DRI Issue needs intervention that assignee can't perform Work is no longer blocked

Generally issues should move through this process linearly. Some templated issues will skip from triage to scheduling or scheduled.

Issue Pointing

Issue pointing captures the complexity of an issue, not the time it takes to complete an issue. That is why pointing is independent of who the issue assignee is.

Weight Description
Null Meta and Discussions that don't result in an MR
0 Should not be used.
1 The simplest possible change including documentation changes. We are confident there will be no side effects.
2 A simple change (minimal code changes), where we understand all of the requirements.
3 A typical change, with understood requirements but some complicating factors
5 A more complex change. Requirements are probably understood or there might be dependencies outside the data-team.
8 A complex change, that will involve much of the codebase or will require lots of input from others to determine the requirements.
13 It's unlikely we would commit to this in a milestone, and the preference would be to further clarify requirements and/or break into smaller Issues.

Issue Labeling

Think of each of these groups of labels as ways of bucketing the work done.

All issues should get the following classes of labels assigned to them:

Optional labels that are useful to communicate state or other priority

Merge Request Workflow

Ideally, your workflow should be as follows:

  1. Confirm you have access to the analytics project. If not, request Developer access so you can create branches, merge requests, and issues.
  2. Create an issue, open an existing issue, or assign yourself to an existing issue. The issue is assigned to the person(s) who will be doing the work.
  3. Add appropriate labels to the issue (see above)
  4. Open an MR from the issue using the "Create merge request" button. This automatically creates a unique branch based on the issue name. This marks the issue for closure once the MR is merged.
  5. Push your work to the branch

  6. Update the MR with an appropriate template. Our current templates are:

    • dbt Model Changes - used for any change involving dbt. Analysts will most often use this one
    • add_manifest_tables - for adding tables to pgp extract
    • periscope - for getting a Periscope dashboard reviewed
    • python_changes - for general changes to Python code
    • All Other Changes - for work that doesn't generally fall into these categories

  7. Run any relevant jobs to the work being proposed

    • e.g. if you're working on dbt changes, run the job most appropriate for your changes. See the CI jobs page for an explanation of what each job does.
  8. Document in the MR description what the purpose of the MR is, any additional changes that need to happen for the MR to be valid, and if it's a complicated MR, how you verified that the change works. See this MR for an example of good documentation. The goal is to make it easier for reviewers to understand what the MR is doing so it's as easy as possible to review.
  9. Request a review by assigning the MR to a peer using the Merge Request Reviewer feature.
    • Requesting a review in this manner indicates to the person that you would like their code review and approval if everything is good. This does not mean they will merge the MR if they approve it.
    • The peer reviewer should use the native approve button in the MR after they have completed their review and approve of the changes in the MR.
    • After approval, the reviewer can unassign themselves from the Reviewer list. The reviewer is not responsible for the final tasks. The author is responsible for finalizing the checklist, closing threads, removing Draft, and getting it in a merge-ready state.
  10. If the MR is approved, remove the Draft: label, mark the branch for deletion, mark squash commits, and assign to the project's maintainer. Ensure that the attached issue is appropriately labeled and pointed.
    • Generally, assigning the MR to a maintainer indicates you would like for them to merge it if there are no issues. If the maintainer needs to approve the merge request before merge as part of a CODEOWNER group, then they will do a review before merging. Otherwise, they will simply merge. If you would like the maintainer's review regardless, simply leave a comment to that effect.
    • Note that assigning someone an MR means action is required from them.
    • If Draft: is still in the title of the MR, then the Maintainer will assign the MR back to the author to confirm that the MR is ready for merge.

Other tips:

  1. The Merge Request Workflow provides clear expectations; however, there is some wiggle room and freedom around certain steps as follows.

    • For simple changes, it is the MR author who should be responsible for closing the threads. If there is a complex change and the concern has been addressed, either the author or reviewer could resolve the threads if the reviewer approves.
  2. Reviewers should have 48 hours to complete a review, so plan ahead with the end of the milestone.
  3. When possible, questions/problems should be discussed with your reviewer before submitting the MR for review. Particularly for large changes, review time is the least efficient time to have to make meaningful changes to code, because you’ve already done most of the work!
  4. Consider bringing the latest commits from the primary branch so the MR is caught up. You can do this quickly by typing /rebase into a comment and GitLab will make this happen automatically, barring any merge conflicts.
KPI Development Workflow
  1. Once a KPI or other Performance Indicate is defined and assigned a prioritization, the metric will need to be added to Sisense by the data team.
  2. Before syncing with the data team to add a KPI to Sisense, the metric must be:
    • Clearly defined in the relevant section in the handbook and added to the GitLab KPIs with all of its parts.
    • Reviewed with the Financial Business Partner for the group.
    • Approved and reviewed by the executive of the group.
  3. Once the KPI is ready to be added into Sisense, create an issue on the GitLab Data Team Issue Tracker using the KPI Template or PI Request Template.
    • The Data team will verify the data sources and help to find a way to automate (if necessary).
    • Once the import is complete, the data team will present the information to the owner of the KPI for approval who will document in the relevant section of the handbook.
Data Quality Issue Workflow

Ideally, your workflow should be as follows:

  1. Create an issue, open an existing issue, or assign yourself to an existing issue. The issue is assigned to the person(s) who will be doing the work.
  2. Add appropriate labels to the issue (see above)
    • The Data Team will Triage the issue and assign it a priority.

SLO for Issues and Merge Requests

Removal and deletion process

After some time, environments will have software/code/components that are not needed any more. It feels risky to delete software and code, even when its not being used, seems not being used or asked not to being used (i.e. users access).

There are multiple reasons to perform deletions. I.e:

  1. We see software (or parts of software) that is not used anymore
  2. There is a request to remove user access
  3. There is a request to delete a user account
  4. There is a request to indefinitely stop a data pipeline

To address observations and requests, and ensure that deletion will take place in a controlled manner, open an issue with the Cleanup Old Tech template.

State down in the issue the scope of deletion

Write down what will be deleted and where possible link to existings issues.

Calculate a Risk score

The Risk score is build upon 2 variables.

  1. Probability that it will break something
  2. Impact if the deletion is executed by mistake or is execute wrongly

Each variables will be scored 1 to 3.

Probability Score
Low 1
Medium 2
High 3
Impact Score
Negligible 1
Lenient 2
Severe 3

Probability * Impact = Risk Score

Risk Negligible Lenient Severe
Probability
Low 1 2 3
Medium 2 4 6
High 3 6 9
Risk Score Outcome
1 - 2 Create a MR and have it reviewed by 2 code owners
3 - 4 Create a MR, tag @gitlab-data/engineers with a deadline to object and have it reviewed by 2 code owners
6 - 9 Create a MR, to be discussed in the DE-Team meeting and have it reviewed by 2 code owners

Request to Expedite Responses

Requests to expedite responses, triage issues, or review MRs is rare. Given the Data Team's shared-service model, expediting an item is asking to de-prioritize other work. To request an expedited response:

  1. Confirm there is a valid reason for moving your request ahead of others.
  2. Post a note to #data, along with a link to your Issue and a reason why you need an expedited response. Please do not DM an individual on the Data Team directly.
  3. A member of the Data Team will respond within 1 business day.

Customer Satisfaction Survey (CSAT)

We regularly measure how ow satisfied our internal customers are with the services and products we provide.

YouTube

We encourage everyone to record videos and post to GitLab Unfiltered. The handbook page on YouTube does an excellent job of telling why we should be doing this. If you're uploading a video for the data team, be sure to do the following extra steps:

Data Hiring and Interview Process

  1. Review the People Interviewing Guide to ensure you are up-to-date on the current process and policies.
  2. Contact your People BP (Business Partner) and alert them of the opening. The People BP is responsible for managing the job opening, requisition, and interview processes.
  3. Ask your People BP to provide you a Requisition # for tracking purposes. If you have multiple openings to fill, communicate with your People BP using this Req# because juggling multiple openings gets confusing fast.
  4. Develop an Interview Plan, which will cover Responsibilities, Tips, Reminders, and custom Interview questions for each Interview Job Role type. For examples, see the Data Scientist Interview Plan and Analytics Engineer Interview Plan.
  5. Each Data Job Role and Job Grade has a customized Homework Assessment. Review and update the Homework Assessment as needed. If a Homework Assessment is not available for the Job Role, create one and save it in the Homework Assessment Google Drive
  6. Send a Homework Assessment to the People BP for inclusion in the Hiring process. The Hiring Process is included in each Data Job Family to help set candidate and interviewer expectations. An example is the Data Engineer Hiring Process.
  7. Create a new slack channel, e.g. bt-data-data-science-interview to help coordinate with your interviewers. Share the Interview Plan with your interviewers through Slack.
Open in Web IDE View source