He wrote his note in his copy of Arithmetica around 1637.
He most likely wrote his proof for the case of n=4 in the 1640s.
He sent letters to other mathematicians in 1640, 1657 where he talks about the case of n=3 but writes in such a way that it seems like he does not have the answer.
Why would he write n=4 after? If he had a generalized proof?
Why would he tease other mathematicians with a special case in 1657 if he already had a generalized proof?
My personal guess is he figured out later that his proof was broken or limited and never got around to fixing it.
I think you have an odd definition of "responsible". Many (most?) theorems start out as conjectures, and I would strongly disagree that just because someone first formulated a problem that they're "responsible" for the eventual solution.
That's not true. Even early on, most mathematicians doubted he had a proof. But I'd check out the history of the problem as described on Wikipedia. There was considerable research into the problem before Fermat, and general research into Diophantine equations had gone on for over a thousand years before Fermat. I have no doubt there would have been significant interest into solving the problem even if Fermat had never written his "I have a proof but it's too big to fit in the margin" note.
But yes had him not pointed it out maybe it would have been relegated to a mathematical curiosity or something
I think there's a deeper (but simpler) reason for it, besides the way Wiles proved it