MLOps Incubation Engineering

MLOps Single-Engineer Group

DRI: @eduardobonet

MLOps is a Single-Engineer Group within our Incubation Engineering Department. This group works on early feature exploration and validation related to the MLOps group within the ModelOps stage.

Vision & Mission

Mission: Make GitLab a tool Data Scientists and Machine Learning Engineers love to use.

Vision: Identify opportunities in our portfolio to explore ways where GitLab can provide a better user experience for Data Science and Machine Learning across the entire Machine Learning life cycle (model creation, testing, deployment, monitoring, and iteration).

Updates

May 1st 2023: Model 15.11 Overview

Current Project

Model Registry & Model Experiments

Mission: Make it dead simple for Data Scientists to manage their model lifecycle within GitLab, from testing different candidates to, packaging a new model version.

More on ML Experiment Tracking

More on ML Model Registry

Usage

Jobs to Be Done

MLops is a large and young field. We used Jobs to be Done to make it explicit what are the problems users expect MLOps to solve:

MLOps Jobs To Be Done

Backlog

We are keeping a backlog for potential exploration areas. Anyone is welcomed to pitch in new ideas using the Backlog Epic

Reach Impact Confidence Effort Colab MLOps Branding* RICE+
JupyterLab-GitLab Plugin 2 2 2 2 1 3 24
Enable GitLab Runners for ML Use cases 3 3 3 2 2 1.5 13.5
Analysis Repository, GitLab Pages for Data Science 1 2 1 2 1 3 6
Improve pipeline usage for ML Use cases 3 1 3 3 3 1 2
DVC Integration/Data Registry - - - - - - -

Machine Learning Experiment Tracking
This is a project led by the Incubation Engineer - MLOps DRI: @eduardobonet Epic All Merge Requests All Issues Usage Mission Make it dead simple for Data Scientists to track their model experiments within GitLab, while making it easy to access the experiments results across the product (on Model Registry, on MRs, on Issues, etc) Guiding Principles GitLab enables Data Scientists to use Experiment Tracking without requiring any support from Platform Engineers Data Scientists can start using GitLab Experiment Tracking without any changes to their codebase Experiment Tracking is not just a vertical feature, it integrates across the platform to enhance experience in different steps of model development Approach: GitLab as a backend for MLFlow MlFlow is the most popular open source option for Experiment Tracking, providing a powerful client that allows Data Scientists to log a wide range of models.
ML Model Registry
This is a project led by the Incubation Engineer - MLOps DRI: @eduardobonet Epic All Merge Requests All Issues Mission Make it dead simple for Data Scientists to track their model lifecycle with GitLab, tracking different candidates, promoting them into model versions so that they can be served and deployed. Guiding Principles GitLab enables Data Scientists to use Model Registry without requiring any support from Platform Engineers Model Registry is not just a vertical feature, it integrates across the platform to enhance experience in different steps of model development Approach: Model registry as Package registry Model registry is implemented using the GitLab package registry.
MLOps Incubation Engineering Updates & Showcases
MlOps Incubation Engineering Showcases Model Experiment Tracking - 15.10 (March 2023) Previous showcases https://youtu.be/BRsU4TGawg8 Month Page 2023/01 https://youtu.be/qC8yssVEh8A MLOps Incubation Engineering Updates Date Page Recording 2023/02/06 Update https://youtu.be/dz7soyNKGPo Previous Updates Prior Updates
MLOps Jobs to Be Done
What are the problems users want MLOps to solve?
Last modified February 22, 2024: Remove sisense refs for dev misc pages (417d9afe)