Typically you are on the hook for paying the actual taxes and penalties, but if you acted in good faith on the negligent advice of a tax professional many jurisdictions allow you to recover the costs of penalties and interest accrued as a result of the negligent advice.