Is that why it did fall apart?
Fighting physically for ownership predates fighting judicially for ownership.
To the extent that you can "own" another animal: the ownership of a female by a male is definitely a thing in the animal kingdom.
And before the first law was ever written, human slavery (estimated to be at least 4000 BC, with mentions in the first law ever written) did exist too.
Ownership predates the law that later on codified the concept of ownership.