The difference between ownership of a physical object and ownership of an intellectual one is a matter of conventional. It's easier to define ownership of an object that is excludable, but that's human convenience, not a physical law.
Is that why it did fall apart?
Fighting physically for ownership predates fighting judicially for ownership.
To the extent that you can "own" another animal: the ownership of a female by a male is definitely a thing in the animal kingdom.
And before the first law was ever written, human slavery (estimated to be at least 4000 BC, with mentions in the first law ever written) did exist too.
Ownership predates the law that later on codified the concept of ownership.
Article I, Section 8, Clause 8 of the US Constitution
Which empowered Congress to "promote the progress of science and useful arts, by securing for limited times to authors and inventors the exclusive right to their respective writings and discoveries."
Scientists and the artists and their "exclusive rights" have built quite a lot over the centuries.