Group Direction - AI Framework

The following page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features or functionality remain at the sole discretion of GitLab Inc.

Overview

The AI Framework group is focused on accelerating the adoption and innovation of advanced AI/ML technologies while ensuring quality. We develop tools for engineers and product teams to enhance their product offerings, enabling them to focus on customer needs and create solutions best suited to their use cases.

AI is evolving rapidly, and the AI Framework group is dedicated to leading the way with innovative approaches to improve efficiencies across the entire DevSecOps cycle. Our Abstraction Layer and AI Gateway are designed to provide a plug-and-play infrastructure, bringing AI as a service to the GitLab platform.

Goals

We aim to:

Increase the velocity of AI-powered feature development.
Enhance the quality of AI-powered features in the platform.
Drive the adoption of GitLab as a comprehensive platform.

Themes

AI Gateway as a Tool to Increase Privacy and Performance

The AI Gateway is a standalone service that provides access to AI features to all GitLab users, whether using self-managed, dedicated, or GitLab.com instances.

Key Features:

Global Reach: Handles traffic from a globally reachable route (cloud.gitlab.com/ai/*), directing it to AI providers and models.
Future Flexibility: Potential for regional or customer-specific gateways if needed.
Scalability: Easier to scale compared to GitLab Rails, as it is a stateless service.
Direct Connections: Future plans include direct client connections to improve request latency, with optional backward compatibility.
Stable API: Single-purpose, version-agnostic endpoints designed for future scalability, composability, and security.
Protocol: Uses a JSON-based API for communication, with REST+JSON chosen for its compatibility with existing and future features.

Abstraction Layer for Enhanced AI Integration

The Abstraction Layer standardizes and simplifies the integration of AI/ML technologies within GitLab, providing an extendable GraphQL API called aiAction.

Key Features:

Standardized API: Provides a consistent interface for different AI actions, such as code suggestions and chat features.
Modular Design: Supports the addition of new AI actions by extending the aiAction input with key/value pairs.
Secure and Scalable: Ensures secure access to AI features while being scalable to accommodate new functionalities and models.

Local Developer Experience

To enhance the local developer experience, GitLab provides comprehensive setup instructions and tools to streamline AI feature development.

Improving Developer Experience:

Single-Command Setup: Developing a setupai command for simplified setup.
Centralized API Key Management: Using 1Password CLI to manage critical API keys securely.
Enhanced Documentation: Providing step-by-step guides and best practices for new developers.
Incremental Improvements: Identifying and implementing small, impactful changes to enhance productivity.

1-Year Plan

Improve Local AI Quality Development: Focus on enhancing tools and processes to support high-quality AI development locally.
Support AI Feature Teams: Provide robust support to various teams working on AI features to ensure seamless integration and innovation.
AI Gateway Development: Continue to develop and enhance the AI Gateway to support scalable and secure AI functionalities.
Regional Deployments and Model Migrations: Implement regional deployments and facilitate model migrations to improve performance and compliance.
Duo Workflow: Develop and refine Duo Workflow to streamline workflow execution and enhance developer experience.
Collaboration with Feature Teams: Work with teams like Cloud Connector to support AI on self-managed instances and ensure robust integrations.
Streamlining AI Interactions and Holistic UX Patterns: Focus on creating cohesive and user-friendly AI interactions and UX patterns across GitLab.

Team HQ

AI Framework Team HQ

Last Reviewed: 2024-10-05
Last Updated: 2024-10-05