Custom rules in GitLab Duo Agentic Chat for greater developer efficiency

Transform GitLab Duo from a generic AI assistant into your team's personalized coding expert with custom rules. Stop constantly correcting AI suggestions that use wrong Java versions, incorrect Python binaries, or violate your style guides. This deep-dive shows you how to create intelligent custom rules that automatically enforce your development standards.

We'll cover:

• Version control: Lock AI to Java 8, handle Python3 environments, and generate multi-platform C++ code
• Style enforcement: Prevent C goto anti-patterns, enforce VueJS design patterns, and ensure Ansible linter compliance
• DevSecOps automation: Bootstrap projects with proper CI/CD security scanning and documentation standards

Each example includes working GitLab projects to fork, complete configurations, and before/after demonstrations. Learn how banking systems stay Java 8 compliant, IoT collectors work cross-platform, and VueJS components follow GitLab's production standards.

Table of Contents

• First steps with custom rules for Duo Agentic Chat
  • Requirements
  • Quick start: 5-minute success
  • Guidelines for custom rule development
  • Ask GitLab Duo Chat about existing development style guides
• More custom rules use cases
  • Use cases: Version and platform support
    • Java version requirements
    • C++ Multi-platform support (Windows, Linux, macOS)
  • Use case: Development environments
    • Python 3 development environment
    • Ansible linter compliance
  • Use case: Design patterns
    • Avoid anti-patterns with C and goto statements
    • Frontend style guides for VueJS 3
  • Use case: DevSecOps workflows
    • Issue and MR templates
    • Build tools
    • CI/CD configuration preferences
    • Security scanning preferences
    • Tests and linters
    • Documentation generation
    • Refactoring and code change requirements
    • Onboarding, requirements, licenses
    • Git flows
• Distribution and testing of custom rules
  • Custom Rules resources
• Fun activity: Explore behavior changes
• Conclusion

First steps with custom rules for Duo Agentic Chat

Follow the documentation to create custom rules for GitLab Duo Agentic Chat in the .gitlab/duo/chat-rules.md directory in a new or existing GitLab project in your IDE.

You can start with free-form text instructions, and iterate on the best outcome. Custom rules support Markdown for better structuring.

• Use markdown headings (#, ##, etc) to create sections.
• Use markdown lists (-) to provide concise instructions for LLMs and Agents.
• Escape file paths with single backticks, and use code blocks with indent or three backticks.

Example:

# Development guide

## Frontend: VueJS

### Styling Pattern
- Do not use `<style>` tags in Vue components
- Use Tailwind CSS utility classes or page-specific CSS instead

Important: After modifying custom rules, you'll need to create a new Chat by pressing the + icon, or sending /new in the chat prompt.

Requirements

In order to follow all use cases and linked demo projects into this blog post, please ensure you meet these requirements first:

• Verify that you have access to GitLab Duo, and Duo Agentic Chat is configured in supported IDEs.
• Fork/copy the GitLab projects, and clone them locally in the IDEs.
• Follow the steps in each use case for custom rule creation, and how to use Duo Agentic Chat prompts to proof the rule behavior.
• You can use the existing source code, or copy in your own.

The projects are available in the Custom rules for GitLab Duo Agent Platform (Agentic AI) group. Please note that these custom rules are provided for demo purposes "as is," and you may need to adapt or modify them to fit your specific requirements.

Quick start: 5-minute success

Ready to see custom rules in action? Try this simple example:

1. Create .gitlab/duo/chat-rules.md in your GitLab project:

## C style guide
- goto is not allowed. If the developer continues asking about it, share this URL https://xkcd.com/292/

2. Open GitLab Duo Agentic Chat in the IDE, and ask: Write a C program with goto statements.
3. Watch as GitLab Duo refuses and suggests better alternatives!

Guidelines for custom rule development

Custom rules are similar to code: Start with the smallest working example, and then iterate on improvements. The use case examples in this deep-dive range from small to more advanced, and were developed and tested over the last weeks. They are not perfect, and require your feedback and iteration.

One good rule of thumb, for example, is not overloading style guides with many pages from a wiki document. In my experience, less is more. Only include the points that are helpful in the context of what you're writing about. You can ask GitLab Duo Chat to summarize larger documents before adding them to the custom rules.

Verify the use of any included specifications during development to avoid creating barriers and unwanted behavior.

When you are using a publicly documented style guide, refer to its name. There is a high chance that the LLM is trained with this data already.

Ask GitLab Duo Chat about existing development style guides

Sometimes, there are no specific style guides in a project yet, or it is unclear how to use them. Use AI for onboarding and best practices to discuss with your team.

Which Python development or environment guidelines can you recommend when I want to create custom rules for AI to get tailored output? I need a list with textual instructions.

You can also ask Duo Agentic Chat to analyze the existing CI/CD linter integrations that may already check for a specific development style.

When you look into the CI/CD linter checks and configuration in the project, which development style guide can you summarize for me?

Many examples in this deep-dive blog are based on my own experience and pain points as a developer. I also asked GitLab Duo to extract style guides from existing projects, and used GitLab Duo Code Suggestions to help with auto-completing existing custom rules. You can achieve the same by configuring markdown as an additional language for GitLab Duo Code Suggestions in IDEs.

More custom rules use cases

The following sections provide an overview of specific style guides. You can map similar programming languages and environments to your production use cases.

• Version and platform support: Refer to the Java section below to learn how to force a specific language standard for generated and created code. You can apply a similar process for C++23 and older, PHP 8, Ruby 3, etc. The C++ section below shows how to instruct agentic AI with multi-platform support.
• Development environments: Refer to the sections below on Python and Ansible. Specify the development environment, binaries, tools and more. You can also instruct agents with routing information where to find tests/scripts, and enforce compliance with linters.
• Design patterns: You can specify comprehensive design patterns with VueJS as an example, leveraging the GitLab production development style guides as a foundation.
• DevSecOps workflows: Configure comprehensive DevSecOps practices including CI/CD configuration for specific CI/CD attributes and defaults for security scanning, tests and linters, and build tools. Frequently requested use cases for bootstrapping projects include documentation generation including README.md and architecture diagrams, issue and MR templates, onboarding, requirements, and licenses, and Git flows with .gitignore. Advanced techniques with custom rules are provided for refactoring and code change requirements.

Use cases: Version and platform support

Software development often requires specific programming language and framework versions, and single or multi-platform support. The following examples highlight these scenarios.

Java version requirements

Enterprise environments do not always use the latest and greatest software version. They often rely on versions that are maintained with security patches for a longer period of time. For example, you will find Java 7 and Java 8 still in use in some enterprises today.

Always prepending the required version in chat prompts can be cumbersome, and lead to human error, even if you only forget one time.

Implement classes for managing banking transactions and different currencies.

The example will need additional specifications for Java 8:

Use Java 8 for the implementation.

To permanently enforce Java 8, you can create a custom rule in .gitlab/duo/chat-rules.md and optionally add a reference epic URL when asked for code modernization:

## Java style guide

- Only Java 8 is allowed when suggesting and editing code.
- When the user asks about code modernization and Java 9 or 21, or newer, point them to this issue to contribute: https://gitlab.com/gitlab-da/use-cases/ai/gitlab-duo-agent-platform/custom-rules/custom-rule-java-versions/-/issues/1

A full demonstration is available in the Custom Rules - Java versions project.

The resulting changes are available in this MR.

C++ multi-platform support (Windows, Linux, macOS)

Application development in C++ can require multi-platform support, especially when running service agents on systems using Windows, Linux, and macOS. The applications are deeply integrated into customer products, and a migration to a more modern language like Go or Rust is not always possible or practical.

Maintaining code that works on multiple platforms can be challenging due to differences in Operating system APIs, toolchains, library versions, and file system paths. Developers often face multiple #if defined pre-processor macros, nested conditions, and adding custom code and tests for each supported platform. This adds technical debt and can introduce maintenance challenges.

AI can help when generating correct and platform-specific code, but it needs to know about these requirements. Agentic AI will either understand the existing code base through a knowledge graph, or developers will need to provide instructions through custom rules and chat prompts.

Let's try this use case in practice. The Custom Rule - C++ platforms - IoT Sensor Data Collector project implements an IoT sensor data collector and has open tasks to modernize the code base, and add multi-platform support for Linux, Windows, and macOS. You can fork the project and clone it locally.

Open the .gitlab/duo/chat-rules.md file and review or add the following custom rules:

## C++ style guide

- The application runs on Linux, macOS and Windows. Generate code that handles the OS API differences.
- Use pre-processor macros for Windows and POSIX conventions for Unix (Linux, macOS).

## CI/CD Configuration

- Ensure that GitLab CI/CD jobs cover the different platform support. Use CI/CD job templates with extends where applicable.

Start a new Chat, and ask to restructure code for multi-platform support.

Please help me restructure the code and ensure multi-platform support.

You can also refer to the issue number, or URL (issue 3). GitLab Duo will automatically fetch the issue content from the GitLab platform, and put it into AI context.

Please help me implement issue 3

Please help me implement https://gitlab.com/gitlab-da/use-cases/ai/gitlab-duo-agent-platform/custom-rules/custom-rule-cpp-platform-iot-sensor-data-collector/-/issues/3

Use case: Development environments

Development environments often vary between operating systems and developers. This can be confusing for AI models generating code or suggesting changes. The following use cases illustrate these environment problems and their solutions with custom rules.

Python 3 development environment

A Python development environment usually comes with the python executable and pip package manager. However, on systems like MacOS or Ubuntu, you need to use python3 and pip3 to get access to more recent Python 3 versions. This can create confusion running Python scripts, creating virtual environments, and installing package dependencies.

For this custom rules use case, I installed Python using Homebrew which results in a binary executable called python3 and a package manager pip3.

As an example, set up a Python virtual environment, install dependencies using pip, and run the application:

python -m venv myenv
source myenv/bin/activate

pip install -r requirements.txt

python script.py

This doesn't work as expected because we need to use specific binaries with version 3:

python3 -m venv myenv
source myenv/bin/activate

pip3 install -r requirements.txt

python3 script.py

We can test this problem with Agentic Chat for both, suggested code blocks, and the approval request for commands to execute. The Custom Rule - Python3 Env Shop app project implements a web shop application in Python, and provides the default Python executable paths in its README.md file which typically gets added into Agentic Chat context.

In order to overcome the problem, review .gitlab/duo/chat-rules.md which contains the following custom rules to enforce the Python executable names.

## Python style guide

- For Python binaries, always use python3 and pip3 when suggesting or running shell commands.
- Detect the Python environment automatically when possible.

You can also instruct agents with pre-defined routes to gather additional information through tool calling and/or MCP, when they do not attempt this automatically already.

Open a new Agentic Chat, and ask How to run this application? to see custom rules in action, using python3 and pip3 as desired.

The full source code is available in the Custom Rule - Python3 Env Shop app project.

Ansible linter compliance

Modern Ansible for Infrastructure-as-Code tasks enforces a strict style guide, which can be verified using ansible-lint: It detects when Boolean values (true/false) are required instead of strings (yes/no), builtin module actions requiring the FQCN (Fully Qualified Collection Name) as parameter names, and trailing whitespaces that need trimming. CLI and IDE integrations, such as the VS Code Ansible extension by Red Hat help visualize these errors to developers. LLMs and chat agents might not always generate correct Ansible code and need manual work to fix it.

Let's look at the problem with a concrete use case. The following example implements a basic Ansible playbook in the Custom Rule - Ansible Environment project to set up a GitLab server on Ubuntu. You'll notice the Boolean values are incorrectly typed as strings (yes/no), and additional problem reports for builtin module actions and whitespace trimming.

Let's see how we can create custom rules to help Duo Agents fix the Ansible linter errors, and prevent them from happening in the future.

Fork and clone the Custom Rule - Ansible Environment project and open .gitlab/duo/chat-rules.md in the IDE inspect the custom rules:

## Ansible styleguide

- Boolean values in Ansible should be typed as "true" or "false" and never as string.
- Ansible module builtin actions must use the FQCN (Fully Qualified Collection Name).
- Always trim whitespaces in Ansible YAML.

Open a new GitLab Duo Agentic Chat prompt, and ask Duo Agent for the same Ansible playbook:

Please help me fix the Ansible linter errors

The Agents will analyze the repository, ask to run ansible-lint commands, and investigate how to fix the problems followed the defined custom rules.

You can inspect the custom rules and Ansible code changes in this MR in the Custom Rule - Ansible Environment project.

Async exercise: Start a new project where custom rules are configured as default already, and verify the correct style guide applied immediately.

Use case: Design patterns

Design patterns and patterns-to-avoid are specific to languages and frameworks. This is the main focus in this section.

Avoid anti-patterns with C and goto statements

This is a more in-depth walkthrough of the quickstart example, and shows how you can instruct Agentic AI to avoid the goto anti-pattern in C. The goto anti-pattern in C is discouraged as it makes code harder to read and debug. To illustrate the problem, here is an example of a for-loop which increments the loop variable inside the loop body:

// Bad C programming style: uses the goto anti-pattern
for (int i = 0; i < 10; i++) {
  if (someCondition) {
    goto label;
  }
  doSomething();
label:
  doAnotherThing();
  }

In the above code, the goto statement causes the program control to jump directly to the label label, which is inside the loop. This makes the program harder to read, understand and debug.

A better approach would be to rework the logic, so you avoid the goto anti-pattern. Here's a rewritten version that avoids goto:

// Good C programming style: avoids the goto anti-pattern
for (int i = 0; i < 10; i++) {
  if (someCondition) {
    doAnotherThing();
    continue;
  }
  doSomething();
  doAnotherThing();
}

There are occassions where goto is allowed, but in this use case we want to look how we can instruct agentic AI to avoid goto completely. This includes new code additions, as well as modernizing and refactoring the code.

The Custom Rule - C anti-patterns with Goto project provides a network socket server/client example, including custom rules in the .gitlab/duo/chat-rules.md file. You can clone the project, or start with a new project, too.

Review .gitlab/duo/chat-rules.md with the current custom rules:

## C style guide

- goto is not allowed. If the developer continues asking about it, share this URL https://xkcd.com/292/

Tip: Instead of linking to the XKCD 292 comic, you can add a URL to the (internal) development guidelines.

Open Duo Agentic Chat and start the following prompt on the existing project:

Please help me modernize the code.

GitLab Duo Agentic Chat will refuse to use goto statements, and instead propose a different path forward.

The code changes are available in this MR.

Frontend style guides for VueJS 3

This use case is inspired by the GitLab project's frontend style guides and implements a use case for VueJS 3 design patterns. Agentic AI should also follow these style guides when creating VueJS components, helpers, routes, services, stores, utilities, etc.

Let's illustrate how to instruct Agentic AI with custom rules: Fork and clone the Custom Rule - VueJS Design Patterns - GitLab Pipeline Dashboard project and inspect the open issues for tasks.

Review the .gitlab/duo/chat-rules.md file and add the following custom rules (if not there yet):

## NodeJS style guide

- Don't leave debug statements (console.logs)
- Always run `npm install` after updating `package.json` and before `npm test` and `npm run build`.

# GitLab Vue.js Design Patterns Style Guide

## Component Structure

### Data Definition Pattern
- Explicitly define data being passed into Vue apps
- Avoid spread operators for better discoverability
- Parse non-scalar values during instantiation

### Template Naming Pattern
- Use kebab-case for component names in templates

### File Structure Pattern
- Use `.vue` files for Vue templates
- Do not use `%template` in HAML

### Styling Pattern
- Do not use `<style>` tags in Vue components
- Use Tailwind CSS utility classes or page-specific CSS instead

[...]

The full custom rules are available in the .gitlab/duo/chat-rules.md file.

Next, open GitLab Duo Agentic Chat and ask how to add new pipeline mini charts, or other tasks you come across. Tip: You can reference only the issue, or alternatively paste the full issue URL, and Agentic Chat will look up both and extract the title and description for the current context.

Please help me implement issue 6

The resulting code changes are available in this MR.

Note: The VueJS style guide was extracted from the gitlab-org/gitlab project asking GitLab Duo Agentic Chat with the following prompt sequence:

What is the development style guide for VueJS?

Can you print the styleguide as Markdown formatted list with headings.

Create a file in the repo, and only print the style guide rules there, no codeblocks.

Use case: DevSecOps workflows

DevSecOps workflows range from best practices for bootstrapping a project with issue/MR templates, .gitignore, GitLab CI/CD configuration, README.md documentation, licenses and much more. The following section explores a variety of use cases. You can use them as inspiration for your own custom rules.

Common DevSecOps automation with custom rules:

• Project bootstrap: Auto-create README, .gitignore, CI/CD config
• Security defaults: Enforce SAST, dependency scanning, secrets detection
• Documentation: Generate issue/MR templates, architecture diagrams

A combined use case example is available in the Custom Rule - DevSecOps workflows - Git README build tools issue MR templates project. You can inspect the custom rules, and fork/clone it locally to ask Agentic Chat with a new prompt such as: I need to bootstrap this project. Please help me with that.

The next sections provide detailed prompts, and most of them are shown in the recording, too.

Issue and MR templates

You can instruct agentic AI to create issue/MR templates when they are missing, and offer to add project-specific information or labels. Agents will automatically query the GitLab API in the background, and put the current project structure into a template.

## Issue and MR templates

- If no issue templates for `Default` and `Feature Proposal` exist in .gitlab/issue_templates, create them using the following raw template sources:

        Default: https://gitlab.com/gitlab-org/gitlab-vscode-extension/-/raw/main/.gitlab/issue_templates/Default.md
        Feature Proposal: https://gitlab.com/gitlab-org/gitlab-vscode-extension/-/raw/main/.gitlab/issue_templates/Feature%20Proposal.md

- If no default MR template `Default` exists in `.gitlab/merge_request_templates`, create them using the following raw template sources:

        Default: https://gitlab.com/gitlab-org/gitlab-vscode-extension/-/raw/main/.gitlab/merge_request_templates/Default.md

- Update the project URLs, and available labels in the fetched templates accordingly, or remove anything unknown and let the user know about TODOs.
- Create a test issue/MR, when bootstrapping a new project.

Build tools

There is a variety of build tools, package managers, compilers, container builders available per programming language. When asking about updates and dependencies, agentic AI can use these default tools without asking for input.

## Build tools

- Always use a virtual env with Python, and set it up before executing any Python commands
- For C/C++: Prefer CMake, and gcc on Linux, clang on macOS, MSVC on Windows.
- For Python: Always use pip
- For Java: Always use Gradle
- For Node.js, suggest to use npm/yarn.
- For Rust: Always use cargo
- For Go: Always use go.mod
- For Ruby: Always use Bundler
- For PHP: Always use Composer
- For .NET: Always use .NET CLI
- For Scala: Always use SBT
- For Elixir: Always use Mix
- For Haskell: Always use Cabal
- For Swift: Always use Swift Package Manager
- For Kotlin: Always use Gradle or Maven.
- For TypeScript: Always use npm or yarn.
- Always suggest using a package manager or build tool based on the main programming language of the project.
- When asking for dependencies, assume that the user wants to update all current dependencies to the latest version available.

- Always suggest to create a Dockerfile if there isn't one. Always use a minimal image and use a tag for security scanning.
- Always add a `Dockerfile` with the base image and the entrypoint if one does not exist.
- Always include a `.dockerignore` and use it in the Docker build process.

CI/CD configuration preferences

Use rules to prefer specific container images, variable and job name patterns, etc.

## CI/CD Configuration

- If no GitLab CI/CD configuration exists in `.gitlab-ci.yml`, ask the user for approval to create.
- Create a GitLab CI/CD configuration automatically when a new project gets bootstrapped

- Always use alpine as container image to build the application.
- Add caching for detected programming languages and frameworks.

Security scanning preferences

Security scanners can also be enforced in GitLab CI/CD configuration. The following example instructs agents to always include Advanced SAST, dependency scanning, and secret detection templates. It has been successfully tested across other use cases in this tutorial.

## Security scanning

- Always use Advanced SAST.
- Always include SAST, Dependency Scanning, Secrets Detection templates, similar to the following format:

    include:
        - template: Jobs/SAST.gitlab-ci.yml
        - template: Jobs/Secret-Detection.gitlab-ci.yml
        - template: Jobs/Dependency-Scanning.gitlab-ci.yml

    variables:
        GITLAB_ADVANCED_SAST_ENABLED: 'true'

Tests and linters

You can also directly instruct Agentic Chat where to find the tests, and how to run them. The same idea applies to calling linter commands. Thanks Jessie Young for sharing this neat tip!

## Tests and linting details

- Tests in this project are located in __ directory and are run using the ___ command
- Linting is done with the ___ command

Documentation generation

Best of documentation custom rules.

- If a README is missing, ask the user if they want to create one. If the user agrees, create a basic `README.md` for them.
- When the user asks for an architecture proposal, always respond with generating an architecture diagram in Mermaid, and ask the user if they want you to add it to the README.md or another documentation file.
- For documentation in Markdown, always use GitLab flavored Markdown.
- Always add correct code block syntax highlighting support.

Refactoring and code change requirements

When a project should gradually be modernized with newly generated code, and not result in large refactors, the following rules can be helpful.

## Keep the changes minimal

- The project uses <this standard and version>. For newly generated code, use this standard.
- Do not attempt to refactor code already created in a project to this standard, but for new code, always ensure this standard is used.
- If unsure whether a file requires modification or refactoring, document this as a todo task.
- Never fix detected problems, whitespaces, code formatting, unless the user instructs you specifically in a comment.
- The code must be retained in its original format and only changes specific to solving the user request are allowed.

## Summaries

- List all items to address at the bottom in a summary section with TODO: followed by a textual description.
- Identify 3 critical items, and ask the user if you should create GitLab issues from those items.

Onboarding, requirements, licenses

Always include a specific documentation link, and guidelines to follow in chat responses.

## Link to guidelines
- Always refer to our developer guidelines when answering questions. You can find those guidelines here: https://docs.gitlab.com/development/

## Context and planning
- Always start with finding existing issues with the desired topic. Only then propose new implementation work.

## License
- Always add the MIT license into `LICENSE` and use `GitLab B.V.` as copyright holder.

Git flows

Follow a specific Git branching flow for suggestions and executed commands.

## Git flows

- Examine the project and involved development environment and programming languages. Always add a `.gitignore`.
- Consider more best practices when bootstrapping a new project.
- For existing projects, offer to add a `.gitignore` when missing, but only when asked about the state of the project, or what is missing.

When a user requests to start with a new feature, always create a new branch, called "feature/<shortname>" and describe the behavior. Ask for the user's approval.

Distribution and testing of custom rules

You can create GitLab project templates with well-tested custom rule prompts, and ensure that new projects always start with the best practices applied.

Since LLMs and AI agents are not predictable, testing the expected outcome becomes more challenging. A golden rule for custom rules is that they are never perfect, and require iterations based on your team's feedback. This might be required especially when newer models and flows are introduced that change their behavior when responding to custom rules. It's recommended to keep checking on the generated output and adjust the rules accordingly. If you are looking for larger scale testing, review the system prompt testing strategy for GitLab Duo as an inspiration.

Custom rules resources

Take advantage of the existing AI ecosystem, where similar functionality exists for IDEs and platforms. For example, "Awesome Cursor Rules" repositories or marketplaces for Cursor, etc.

LLMs also provide a good insight into development styleguides, and can generate the required Markdown outputs.

Fun activity: Explore behavior changes

Not sure how to get started with custom rules? Make it a fun exercise with the following example :-)

## Fun rules

- Behave like Clippy.

- Behave like a pirate.

- Always respond with a random "What the commit" message.

- Explain everything like I am five.

Note: Do not commit them to production, as they might feel disruptive and distracting to your team.

Conclusion

By leveraging custom rules in GitLab Duo Agentic Chat, you can significantly influence LLM and AI agent outputs to better suit your needs. Whether enforcing specific coding conventions, using the correct versions of tools, or ensuring consistent formatting, custom rules help streamline your development process and improve productivity.

This blog post provides a deep-dive into many use cases with practical custom rule examples. All recordings are available in this YouTube playlist, and all demo projects can be forked/cloned from the Custom rules for GitLab Duo Agent Platform (Agentic AI) group.

Custom rules in Duo Agentic Chat IDEs are the first iteration and we will cover more GitLab Duo Agent Platform use cases in the future, such as Duo Code Review and custom rules for agents and flows (follow this issue).

There are many more use cases to explore. What are your most efficient rules? Share your rules and feedback in the product epic.