> For example, I worked with a client that had a test suite of about 7000 or so strings that should return a 500 error

> We "failed" and were not in compliance as you could make a request containing one of those strings--ignoring that neither Apache, SQL, or Windows were in use.

this causes me pain

What is their desired behaviour if not a 404? A 500? a FIN? a RST?

Why in the world should those be 500 even? Those all are "40x client fuckup".

I guess someone was told, when compiling those strings, that they should observe this known-good implementation (that actually crashed upon receiving such things) and record whatever it returns, and then mandate it of everyone else from now on.