I should probably put together some actual stats at some point, but from what I see in the live logs you'd be shocked - some of them will happily sit there for over 5 minutes being drip-fed packets!

Looking at you, Meta.

I do still get some requests going up to 15 minutes which is quite ridiculous, though those don't seem to be super common (maybe they're learning).

I think the shortest timeouts I've seen are between 2-5 seconds, but they're surprisingly infrequent and actually that's still a massive inconvenience for them. I did at one point just drop any request I couldn't keep open for more than 5 seconds but it actually caused more server load. I'm essentially inverting the server load by making slow requests more efficient, which is counter to everything I ever learnt about performance for web serving :D

I think the average I see is probably somewhere between 15-30 seconds. I'm guessing many crawlers don't want to go too low because they'll be trying to fetch large files like images, or perhaps they know they'll be slowing servers down and want to ensure they get the response. Most likely, the majority of crawler authors aren't expecting some arsehole to exploit this and they've just left the setting as the default.