If you think smoke testing is cheap, try doing it across concurrent releases and multiple envs, then see how fast the "just check it manually" step gets dropped when nobody owns it. Manual QA finds weird edge cases, but if that's all you have, regression will eat you alive.
replySmoke tests and unit tests do different jobs, and calling unit tests low value is backwards unless your code almost never changes. If you skip automated tests because they are "expensive", you are betting release quality on hope and a lot of repeat work.
> Unit tests are very expensive and return little value
replyWhy are unit tests very expensive? This goes against everything I know.
Unit tests very roughly double the amount of effort required to make any meaningful change to your codebase. They are also require maintenance same as ordinary code - but the customer does not care in the slightest whether or not they pass. On the other side, they can only really tell you about low level bugs that you already expected, they cannot surface system level bugs - the actual hard bugs that cause problems for you and your customers.
replyThen there is the danger of thinking that green=all good, an example of 'automation bias' where we learn to trust the automation even as things go wrong.
As makers, it is also tempting to believe that [all] problems can be solved by making something (i.e. code), but actually many problems are not of that nature, and cannot be solved in that way.
Thank you, that makes sense. What I meant was that today all unit tests are basically written by an AI so the "cost" is almost zero. Am I wrong?
replyAutomated tests are no longer expensive to write and nowadays less expensive to execute.
reply