Secure AI agent deployment to GKE

Building AI agents is

exciting, but deploying them securely to production shouldn't be

complicated. In this tutorial, you will learn how GitLab's native Google Cloud integration makes it straightforward to deploy AI agents to Google Kubernetes Engine (GKE) — with built-in scanning and zero service account keys.

Why choose GKE to deploy your AI agents?

GKE provides enterprise-grade orchestration that connects seamlessly with GitLab CI/CD pipelines through OIDC authentication. Your development team can deploy AI agents while maintaining complete visibility, compliance, and control over your cloud infrastructure. This guide uses Google's Agent Development Kit (ADK) to build the app, so you can expect increased seamlessness as this is deployed using GitLab.

Three key advantages to this approach:

Full infrastructure control - Your data, your rules, your environment. You maintain complete control over where your AI agents run and how they're configured.

Native GitLab integration - No complex workarounds. Your existing pipelines work right out of the box thanks to GitLab's native integration with Google Cloud.

Production-grade scaling - GKE automatically handles the heavy lifting of scaling and internal orchestration as your AI workloads grow.

The key point is that GitLab with GKE provides the enterprise reliability your AI deployments demand without sacrificing the developer experience your teams expect.

Prerequisites

Before you start, make sure you have these APIs enabled:

• GKE API

• Artifact Registry API

• Vertex AI API

Also make sure you have:

• GitLab project created

• GKE cluster provisioned

• Artifact Registry repository created

The deployment process

1. Set up IAM and permissions on GitLab

Navigate to your GitLab integrations to configure Google Cloud authentication (IAM).

Go to Settings > Integrations and configure the Google Cloud integration. If you're using a group-level integration, notice that default settings are already inherited by projects. This means you configure once at the group level, and all projects benefit and inherit this setting.

To set this up from scratch, provide:

• Project ID

• Project Number

• Workload Identity Pool ID

• Provider ID

Once configured, GitLab provides a script to run in Google Cloud Console, via Cloud Shell. The outcome of running this script is a Workload Identity Federation pool with the necessary service principal to enable the proper access.

2. Configure Artifact Registry integration

Still in GitLab's integration settings, configure Artifact Management:

1. Click Artifact Management.

2. Select Google Artifact Registry.

3. Provide:

   • Project ID
   • Repository Name (created beforehand)
   • Repository Location

GitLab provides another script to run in Google Cloud Console.

Important: Before proceeding, add these extra roles to the Workload Identity Federation pool:

• Service Account User

• Kubernetes Developer

• Kubernetes Cluster Viewer

These permissions allow GitLab to deploy to GKE in subsequent steps.

3. Create the CI/CD pipeline

Now for the key part — creating the CI/CD pipeline for deployment.

Head to Build > Pipeline Editor and define your pipeline with four stages:

• Build - Docker creates the container image.

• Test - GitLab Auto DevOps provides built-in security scans to ensure there are no vulnerabilities.

• Upload - Uses GitLab's built-in CI/CD component to push to Google Artifact Registry.

• Deploy - Uses Kubernetes configuration to deploy to GKE.

Here's the complete .gitlab-ci.yml:

default:
  tags: [ saas-linux-2xlarge-amd64 ]

stages:
  - build
  - test
  - upload
  - deploy

variables:
  GITLAB_IMAGE: $CI_REGISTRY_IMAGE/main:$CI_COMMIT_SHORT_SHA
  AR_IMAGE: $GOOGLE_ARTIFACT_REGISTRY_REPOSITORY_LOCATION-docker.pkg.dev/$GOOGLE_ARTIFACT_REGISTRY_PROJECT_ID/$GOOGLE_ARTIFACT_REGISTRY_REPOSITORY_NAME/main:$CI_COMMIT_SHORT_SHA
  GCP_PROJECT_ID: "your-project-id"
  GKE_CLUSTER: "your-cluster"
  GKE_REGION: "us-central1"
  KSA_NAME: "ai-agent-ksa"

build:
  image: docker:24.0.5
  stage: build
  services:
    - docker:24.0.5-dind
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - docker build -t $GITLAB_IMAGE .
    - docker push $GITLAB_IMAGE

include:
  - template: Jobs/Dependency-Scanning.gitlab-ci.yml
  - template: Jobs/Container-Scanning.gitlab-ci.yml
  - template: Jobs/Secret-Detection.gitlab-ci.yml
  - component: gitlab.com/google-gitlab-components/artifact-registry/upload-artifact-registry@main
    inputs:
      stage: upload
      source: $GITLAB_IMAGE
      target: $AR_IMAGE

deploy:
  stage: deploy
  image: google/cloud-sdk:slim
  identity: google_cloud
  before_script:
    - apt-get update && apt-get install -y kubectl google-cloud-sdk-gke-gcloud-auth-plugin
    - gcloud container clusters get-credentials $GKE_CLUSTER --region $GKE_REGION --project $GCP_PROJECT_ID
  script:
    - |
      kubectl apply -f - <<EOF
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: ai-agent
        namespace: default
      spec:
        replicas: 2
        selector:
          matchLabels:
            app: ai-agent
        template:
          metadata:
            labels:
              app: ai-agent
          spec:
            serviceAccountName: $KSA_NAME
            containers:
            - name: ai-agent
              image: $AR_IMAGE
              ports:
              - containerPort: 8080
              resources:
                requests: {cpu: 500m, memory: 1Gi}
                limits: {cpu: 2000m, memory: 4Gi}
              livenessProbe:
                httpGet: {path: /health, port: 8080}
                initialDelaySeconds: 60
              readinessProbe:
                httpGet: {path: /health, port: 8080}
                initialDelaySeconds: 30
      ---
      apiVersion: v1
      kind: Service
      metadata:
        name: ai-agent-service
        namespace: default
      spec:
        type: LoadBalancer
        ports:
        - port: 80
          targetPort: 8080
        selector:
          app: ai-agent
      ---
      apiVersion: autoscaling/v2
      kind: HorizontalPodAutoscaler
      metadata:
        name: ai-agent-hpa
        namespace: default
      spec:
        scaleTargetRef:
          apiVersion: apps/v1
          kind: Deployment
          name: ai-agent
        minReplicas: 2
        maxReplicas: 10
        metrics:
        - type: Resource
          resource:
            name: cpu
            target: {type: Utilization, averageUtilization: 70}
      EOF
      
      kubectl rollout status deployment/ai-agent -n default --timeout=5m
      EXTERNAL_IP=$(kubectl get service ai-agent-service -n default -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
      echo "Deployed at: http://$EXTERNAL_IP"
  only:
    - main

The critical configuration for GKE

What makes this work — and why we need this extra configuration for GKE— is that we must have a Kubernetes Service Account in the cluster that can work with Vertex AI. We need that service account to be permitted to access the AI capabilities of Google Cloud.

Without this, we can deploy the application, but the AI agent won't work. We need to create a Kubernetes Service Account that can access Vertex AI.

Run this one-time setup:

#!/bin/bash


PROJECT_ID="your-project-id"


GSA_NAME="ai-agent-vertex"


GSA_EMAIL="${GSA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com"


KSA_NAME="ai-agent-ksa"


CLUSTER_NAME="your-cluster"


REGION="us-central1"



# Create GCP Service Account


gcloud iam service-accounts create $GSA_NAME \
    --display-name="AI Agent Vertex AI" \
    --project=$PROJECT_ID

# Grant Vertex AI permissions


gcloud projects add-iam-policy-binding $PROJECT_ID \
    --member="serviceAccount:${GSA_EMAIL}" \
    --role="roles/aiplatform.user"

# Get cluster credentials


gcloud container clusters get-credentials $CLUSTER_NAME \
    --region $REGION --project $PROJECT_ID

# Create Kubernetes Service Account


kubectl create serviceaccount $KSA_NAME -n default



# Link accounts


kubectl annotate serviceaccount $KSA_NAME -n default \
    iam.gke.io/gcp-service-account=${GSA_EMAIL}

gcloud iam service-accounts add-iam-policy-binding ${GSA_EMAIL} \
    --role=roles/iam.workloadIdentityUser \
    --member="serviceAccount:${PROJECT_ID}.svc.id.goog[default/${KSA_NAME}]" \
    --project=$PROJECT_ID

4. Deploy to GKE

Once you're done, push this change to the pipeline and you're good to go.

You can see the pipeline has just deployed. Go to CI/CD > Pipelines and you'll see the four stages:

• Build

• Test (with all defined security scans)

• Upload to Artifact Registry (successful)

• Deploy to Kubernetes in GKE (success)

Summary

With GitLab and Google Cloud together, you're able to deploy your AI agent to GKE with ease and security. We didn't have to go through a lot of steps — we were able to do that thanks to GitLab's native integration with Google Cloud.

Watch this demo:

Use this tutorial's complete code example to get started now. Not a GitLab customer yet? Explore the DevSecOps platform with a free trial. Startups hosted on Google Cloud have a special perk to try and use GitLab.