I feel like government borrowing sometimes and government borrowing more and more every year and never paying it down until the end of time or more likely bankruptcy are two different things

If your business can't self-fund the investment, borrowing is justified. But if you're earning revenue that allows you to self-fund, why borrow? You're just incurring extra costs.