Gitlab hero border pattern left svg Gitlab hero border pattern right svg

Design :: [Elasticsearch integration for one namespace]

On this page

Implementation Considerations

Infra epic

infra epic

Elastic Cloud

we'll be using a cluster in GCP so we shouldn't be hit by: https://gitlab.com/gitlab-org/gitlab-ee/issues/4334

staging cluster

staging

production cluster

production

Operational Considerations

HA

do we need HA? yes:

how many zones should we use? at least two

Which zones the cluster should be in? as close to gitlab infra as possible (ideally in the same zone). At the moment of writing, majority of prod and stg infra is in us-east. However, elastic.co doesn't allow you to create clusters in that zone.

Monitoring

See monitoring Elastic Cloud in runbooks repo

Alerting

See alerting Elastic Cloud in runbooks repo

Resizing cluster

See runbooks repo

Maintenance

See runbooks repo

Initial indexing overloading Gitaly

This shouldn't occur since the number of simultaneous sidekiq jobs is limited. Gitaly fleet should already be sized to handle connections from the existing sidekiq nodes.

Initial indexing overloading sidekiq

During tests on staging, there were 2 machines, 8 sidekiq workers in total. They were able to process incoming jobs in real-time (queue = 0). The bottleneck was with network connection between indexer and gitaly