Sure, placing launch infrastructure in French Guiana is probably less risky than setting up shop in New Caledonia. Clearly, the risk profile is not the same for all of Overseas France.
My point is that if the decision is between mainland and Overseas France, the latter introduces elevated risk in the long term in general. French Guiana could be fine today, but it is not impossible that the next few years could see the rise of a populist pro-independence movement that would eventually impact launch infrastructure (similar to how it did for the USSR).
Reality is fundamentally chaotic after all.