Software development often happens on a single machine, with a single application version and almost no load.
This configuration is very different from what happens on GitLab.com and our customers' installations.
The problem "at scale" comes from a different order of magnitude than the development and testing environments. Things like the number of servers, the number of incoming requests, the number of rows on a table or the number of application versions will make the difference between something that works on your computer and something that works in production.
An extra challenge, almost unique to GitLab, is that we deploy from the main branch multiple times each day, but we have a monthly release cycle and zero downtime updates is a requirement for both releases.
Overlooking the compatibility with multiple versions of the application running at the same time can induce a production incident.
|Expand and Contract pattern||Link|
|Zero Downtime Updates||Link|
|Sidekiq Compatibility across Updates||Link|
|Avoiding downtime in migrations||Link|
|Uploads development documentation||Link|