Our 2022 Global DevSecOps Survey is out now! Learn the latest in DevOps insights from over 5,000 DevOps professionals.
The software testing life cycle can feel like the DevOps punching bag, and for good reason: For the last three years, our annual survey participants have unanimously named/blamed test as the number one reason for release delays. In our latest survey, participants had some very pointed commentary about the software test life cycle:
"Testing can be both slow in writing and running."
"Testing is not yet fully automated in the deployment cycle; hoping to improve that with our move from BitBucket + Jenkins/drone to GitLab."
"Testing delays everything."
"Some software delivery teams have delegated their testing to QA personnel instead of writing comprehensive end-to-end testing suites. These teams suffer from very long (several days) bottlenecks in shipping to production."
But for all the complaints, our 2021 Global DevSecOps Survey did find some signs that the software test life cycle, like many other components of DevOps, is beginning to mature. For starters, almost 25% of survey respondents said they’ve achieved full test automation, more than double the number reported last year. And 28% said their teams are at least halfway to full test automation.
Changing roles
In our 2020 survey we found DevOps roles are changing, and this year that pattern seems to be continuing, even in testing. Roughly 34% of participants said devs are testing their own code (up from 31% last year) and 32% said code is tested as it’s written, a significant bump from 25% last year.
At the same time though, when we asked devs what they need to be doing more of the vast majority of responses mentioned testing, whether it was pen, smoke, A/B, manual or simply test automation. For all the forward momentum, 25% of teams are either just beginning to consider test automation or have none at all.
An improving picture, but testing is simply irritating to some of our respondents:
"Automated testing is ignored ‘due to time constraints.’"
"Testing? That's an interesting idea."
"We intended to do test-driven development (TDD) but it usually ends up being after the fact."
"I try to write my code with TDD when it's possible; it's complicated when writing React components, or when changing a function that is not tested with many side effects and many inputs and the tech lead forbids (me) to refactor it at the moment .... ='(."
A potential game-changer
Although it sounds like Space Odyssey meets DevOps, there is another reason for optimism around software testing: Artificial intelligence/machine learning is on the rise now and what could be more perfect than bots running endless tests? Bots can be deployed in the thousands and they don’t take vacations, or even lunch breaks.
The appeal of endless testing was clear in our survey responses this year. Just over 41% of respondents told us bots were testing their code and/or AI/ML was reviewing code before human intervention. That’s up dramatically from just 16% last year. All told, 25% of respondents use bots to test their code, 16% use AI/ML to review code before a human sees it, and 34% are exploring the idea of AI/Ml but haven’t done anything about it yet. Exactly one-quarter of respondents aren’t using AI/ML in test.
Software testing is just one small part of what DevSecOps Survey covers. Our 2022 Global DevSecOps Survey has the latest insights from over 5,000 DevOps professionals.
