His parental rights cannot be terminated absent very strong evidence of abandonment, abuse, or neglect. "liberal visitation as the parties agree" implies that visitation depends upon agreement about when, where, how long, etc.. Hopefully, you have an attorney helping you with the modification. If not, you should as Florida custody law has changed since your divorce.
This is for informational purposes only and is not legal advice upon which anyone should rely. Nor does it create any attorney client relationship.
Stanton L. Cobb
Board Certified Marital & Family Law Specialist
Fellow - American Academy of Matrimonial Lawyers
P.O. Box 149223
Orlando, FL 32814-9223