Load testing is a common and typically important stage in the CI process, ensuring changes to the app can scale before they are introduced into production. Be confident in the performance of your changes by ensuring that they are validated against real-world load scenarios.
Advanced deployment strategies like Canary and Blue/Green can mitigate the risk of deploying code that is not performant or does not scale. However, that is not a valid alternative for a few reasons:
Canary and Blue/Green do minimize risk, but you are still essentially testing in production. Best practices would dictate that you use CI test what you can, to reduce the frequency and occurrence of rollbacks and errors in production.
Interested in joining the conversation for this category? Please join us in our public epic where we discuss this topic and can answer any questions you may have. Your contributions are more than welcome.
This page is maintained by the Product Manager for Testing, James Heimbuck (E-mail)
Up next is the Integrated Load Testing MVC (gitlab-org#952). This epic brings performance testing as a category to viable maturity.
This category is currently at the "Minimal" maturity level, and our next maturity target is Viable (see our definitions of maturity levels). Key deliverables to achieve this are:
Azure DevOps offers in-product load testing: https://docs.microsoft.com/en-us/azure/devops/test/load-test/get-started-simple-cloud-load-test?view=azure-devops. This consists of different types of tests including:
For URL type tests, the output contains information about the average response time, user load, requests per second, failed requests and errors (if any).
While, just as one could orchestrate any number of performance testing tools with GitLab CI today, Travis or CircleCI could be used to orchestrate performance testing tools, it does not have any built-in capabilities around this.
The top Vision item is gitlab-ee#10681 which will add Integrated Load Testing as an Auto DevOps step. This will mean that what we have defined as viable for this category will now be added to every Auto DevOps pipeline.