There are many ways to develop engineering metrics. Ideally you need a representation of different aspects of the operational data such as throughput, quality and cycle time to name a few.
The goal of gathering this data is to define what a healthy and successful team looks like and to foster a healthy dialogue amongst the engineering team and with stakeholders. These metrics are helpful for conversations around planning, addressing technical debt, capacity to work on bugs, identifying bottlenecks and improving engineering efficiency. Metrics are a good way to also see how change affects the team’s execution.
Engineering metrics are project or team based and are not intended to track an individual's capacity or performance. It is important to note that if metrics are used punitively, these goals are hampered and the team psychological safety could be at risk.
As a manager you should ensure you are on top of your team's metrics, driving the right behavior, and understanding any fluctuations. Expect to be able to discuss your team's metrics with your manager on a weekly basis.
To access our current dashboard, please visit the GitLab Insight Quality Page. We are currently working to implement these graphs into the GitLab product.
We collect all related Sisense dashboards in its own
Engineering: Development Metrics topic in Periscope to have them all in one place.
regression:x.ylabel. It is possible that some regressions are not captured due to it not having a regression label.
We are currently working on including all of engineering's projects. The following projects included can be seen in this list.
We also capture data from our
https://dev.gitlab.org/ instance which currently includes the projects below.
The data from dev instance is only reflected in a few group level charts, see list of known issues below.
https://dev.gitlab.org/, only the 3 group-level charts below are reflected with data from this source.
If you or your team needs to include a new project in the metrics or add a new team, please create an issue in the GitLab Insights issue tracker.
We have a list of projects that are still not included in this issue.
Considering how one of our development department KPIs is to have a goal of 10 MRs per Development Engineer per Month, we have a team metrics dashboard that can easily be used to see MR statistics per team.
team_idfilter (under the headline) for which team you want to see the statistics.
Considering how one of our development department KPIs is to have a goal of 10 MRs per Development Engineer per Month, we have a section metrics dashboard that can easily be used to see MR statistics per section.
section_idfilter (under the headline) for which section you want to see the statistics.
Note: These dashboards will be available internally to everyone at GitLab (as per default Sisense permissions). Considering that the majority of engineering MR activities are public, we do not anticipate this being an issue. In addition, anyone that can view the dashboard will also be able to toggle the filter to see individual team member's metrics. We purposefully do not chart all team members in a specific chart against each other because throughput is just one data point and is not a holistic evaluation of a team member's productivity/performance.
Note: MRs that are merged in dev.gitlab.org are not available in Sisense: gitlab-data/analytics#2789
Currently we update the team member records in Sisense with a manually-executed script. This script maps members from team.yml to teams in Sisense by the
departments list for each member. The script expects exact string matches, so if a team member is not appearing in Sisense please check their section or team is not missing or misspelled in team.yml. If you make an update to team.yml, please let a director know to rerun the script.
We currently map team members between team.yml and Bamboo through their full name. But they can be different due to various reasons so we have as a pragmatic solution for now a Google sheet to map GitLab ID's to Fullnames as in Bamboo so that we can track when people were members of a specific team. Please add missing team members to the sheet with the first and last name from fields in bamboo and then let a director know to rerun the script.
We created many SQL snippets to centralize counting and the actual structure of data for best reuse.
For example paste
[development_team_mrsperengineer_by_members('manage_be')] into a chart and it will give you all MR/engineer metrics for the team
Gives you all metrics directly for a chart around MR/engineer metrics for one specific team.
Gives you all metrics directly for a chart around MR/engineer metrics for one specific section.
Gives you all data for a MR total graph based on team members.
Gives you all data for a MR total graph based on section members.
Gives you all relevant queries for MR metrics based on team members. (dev_product_mrs_merged, dev_team_members, dev_counts, dev_monthly_analysis)
Gives you all relevant queries for MR metrics based on section members. (dev_product_mrs_merged, dev_team_members, dev_counts, dev_monthly_analysis)
Returns all Mean time to Merge metrics for a team.
Returns all Development Team Members by team name.
Returns all Development Team Members by section.
Currently half-automated select statement that defines all team members based on data from team.yml and GitLab API to gather GitLab user ID's.
The script is located in the manager-tools repository. The plan is to automate it ASAP through automatic importing, currently half manual as it is easier to iterate.
There are two different indicators in Sisense for Merge Request data;
is_included_in_engineering_metricsis a deprecated legacy indicator that was intended to align reporting project population in the Quality Dashboard.
is_part_of_productis the indicator that identifies projects which are included in the GitLab release. This metric is used to identify the projects for the Average MRs/Development Engineers/month.
is_part_of_product to identify MRs to include for team dashboards to ensure alignment with