2 attorney answers
You may be able to get the IRS to "abate" or waive the penalty under the circumstances. You should apply for that abatement, just by sending a letter along with the form that the IRS sent you. The interest is appropriate because you didn't have to pay the money earlier, and they will generally not waive the interest. You can try to sue the employer in small claims court, but it may not be worth it, if you get the penalty waived.
Your former employer is not responsible for correctly calculating your taxes. Unfortunately, I don't think you're going to win that particular issue.