Talk with your attorney. If that attorney who recommended suing him, ask him why? with what goal? There may be some panorama of Goals, which includes a fallback position of him consenting to full custody, consenting to change of name, consenting to adoption (were you to remarry), etc. Don't just blow off the idea; it may not be for "drama."
NOTE: This answer is made available by the lawyer for educational purposes only. By using or participating in this site you understand that there is no attorney client privilege between you and the attorney responding. This site should not be used as a substitute for competent legal advice from a licensed professional attorney that practices in the subject practice discipline and with whom you have an attorney client relationship along with all the privileges that relationship provides. The law changes frequently and varies from jurisdiction to jurisdiction. The information and materials provided are general in nature, and may not apply to a specific factual or legal circumstance described in the question.