I touched on it in my way-too-long post elsewhere on here, but I think this is exactly it: there's a (fuzzy at some boundary, sure, but useful) distinction to be drawn on something like where the game happens. Does "the game" (the software) supply most or all of "the game"? Or is "the game" (the software) a toy in service of a game that the player brings and gives shape?

Both types of software plausibly "are video games" but can take extremely different forms, and their appeal may diverge wildly—someone who likes one to an extreme, may have zero interest in the other. Others may like both sorts of play, but not regard them as interchangeable (i.e. if what you're wanting at the moment is an e-sport, a visual novel may not be any amount of a satisfactory substitute, even if you like visual novels).

We tend to draw a "toy/game" distinction (with games perhaps being a subset of "toys", but still its own sub-category, anyway) with physical objects to divide those with built-in goals from those without, and that seems to serve us well, but we've not translated that to the digital realm very well (and maybe we shouldn't, I dunno)