upvote
> at the end of the day, developers will write whatever works for them in the moment, and not care if it fits your theoretical box.

That's description versus prescription: we should still be able to describe those "whatever works" tests, in a way that others will understand.

I've worked on teams which, if shown the exact same test, would describe it with completely different terms (e.g. "functional unit test" versus "behaviour-driven integration test", etc.); and who, if asked to write a particular type of test (e.g. "check the DB code with a few integration tests"), would interpret that in completely different ways (say, one team might interpret that as "test how it actually works against a real RDBMS", whilst the other might interpret that as "test that the Query class instantiates Result instances correctly", or whatever).

It would be nice to have terminology which has a particular meaning, and for that to be stable over time. I've actually written about this before:

http://www.chriswarbo.net/blog/2017-11-10-unit_testing_termi...

http://www.chriswarbo.net/blog/2020-07-07-more_testing_termi...

reply
I read your paper. It's quite interesting! "a large number of tests the developers actually write are using test-automation frameworks and infrastructure in ways the framework authors may not have intended".

I do think it'll be really interesting to use current gen LLMs to look through pre-AI githubs and do a richer profile of the kinds of tests that were hand crafted for high quality projects.

My experience with agents writing tests fully autonomously is that they are mostly ... a waste of their time and mine: they don't really catch the sort of regressions that modern agents are likely to cause.

I've heard much of the commercial value prop of SQLite is their test / regression suite.

reply
That is not limited to tests. People either do refactoring or they don’t. I saw some people come around on a trunk based development project but that was still incremental.
reply