The age of consent in North Carolina is 16. He cannot have you arrested for Statutory rape. It requires that the crime have occurred when the victim was either 13, 14, or 15 years old. Your son's father has no case based on the above information.
I will agree with Ms. Goodwin and will add that he cannot sue you for statutory rape. It is a criminal statute not a civil claim...I am assuming he was being literal when he said sue.
Also, just to clear another point you made, statutory rape is a strict liability crime. So when it actually occurs, not knowing that the person was underage is not a defense. So if he was, say, 15, when you two were sexually active, even if you thought he was 18, you could be charged with statutory rape.