AI Won’t Fix Your Bad Practices

Despite my general disregard for AI hype, there are very legitimate concerns around the use of AI toolchain. For example DORA is one of the most well respected voices in software best practice, and their data is quite concerning:

“…our data shows AI adoption not only fails to fix instability, it is currently associated with increasing instability.”

DORA: https://dora.dev/research/2025/dora-report/

So, we know that software teams that use AI are actually making their code base worse.

Does this mean that AI is bad?

No, certainly not.

What it means is that our lazy and sloppy practices get magnified with AI because AI allows us to be lazy but still ship with high velocity. Thus, we can make more mistakes quickly. Not a recipe for quality software.

Shockingly, most software teams do not have a fully automated test stack:

“Most organizations plateaued at 25% automation of their testing.”

Forrester: https://www.forrester.com/blogs/the-autonomous-testing-platform-wave-q4-2025-is-out/

While some use cases are difficult or even impossible to automate, the bar is not 25%. In my experience, only about 5-10% of all use cases are difficult to automate and very few are impossible to automate.

Yes, there is a huge amount of effort in creating, managing and debugging these test cases, but it’s clearly worth it.

In the past, there was always a cost to doing automated testing. The engineer that spent two weeks building test cases was not building new customer facing features for those two weeks. This opportunity cost often caused software teams to stint on automated testing. It’s also common for developers to be praised more for customer facing work than backend work. Thus, your best developers tend to move away from things like testing.

However, in an AI optimized software team, that is no longer true. An LLM doesn’t care if it’s writing test code or not and the LLM can write them very quickly meaning that the opportunity cost is very low.

However, teams still aren’t writing tests.

This is just bad practice.

As usual, the things that have always been true are still true today.

As the state of the art advances, velocity continues to increase. This happened with the PC and then again with Cloud and then again with containerization, etc.. AI is yet another of these fundamental shifts that speeds us up.

However, none of these innovations came with quality for free. In each case, we had to optimize our processes and be methodical in our approach.

AI is no different.

Every time you check in a new feature, it must have a full E2E test. Every time. All code must have unit testing. Security scans are not optional. AI can configure and maintain all of these things for you, but you have to ask.

My personal favorite is API contracts. It is possible today to use things like Open API Specification (formerly Swagger) to document and test API contracts. In my experience, most teams don’t use tools like this which is a shame. I always require that my AI toolchain document APIs using Open API Specification and then I have an explicit test to ensure that the API conforms to the contract at each commit.

While this approach isn’t perfect, it reduces the amount of regressions I have and reduces the amount of API drift that gets created by my AI coding agents.