The complexity of society has got to be somewhat proportional to its population. If we halve the population, we won't just half the GWP. It will be lower still due to less specialisation.

Also, unless you throw the elderly to the wolves, having very low birth rates leads to a huge drag from having too many retirees to support with your shrinking working age population.