Some states, like Maryland, have expressly answered this question in the State law or Code. In Maryland, when a child is sixteen (16) he can petition the Court on his own behalf to change a court order for custody or visitation. As a practical matter, when a child is younger than 16, if he has the appropriate maturity and a good reason for not wanting to visit with his father (like the ones you've described above), then he will also not be required to spend time with his father. With this being said, it is important to make sure that you, as the custodial parent, are not prohibiting the older child from seeing his father: likewise, you should not encourage this in anyway.
It is always best to have your child work with a child therapist on the issues and feelings associated with the child's desire not to see his father. This will help both parents understand the situation and work towards repairing the damage to the parent-child relationship in an appropriate setting and timeline. Notably, this will also help you reduce the chance that father files a contempt action against you and it will help your defense if he does.
In Georgia, courts have said that a child cannot be held in contempt for refusing to visit with the other parent. Once the child is 14, he has the right to decide who he wants to live with, and the court will honor the child's request unless it is not in the best interest of the child.
That being said, as long as the father has visitation rights, he can file a motion for contempt if you are trying to talk your son out of visiting with the father (this would also be parental alienation and our Georgia courts take that very seriously). Your best option would be to file an action for modification of visitation rights based on the child’s desire to have visitation with his father only at his discretion. Once the order is changed, you will not have to worry about being held in contempt.
Georgia has the over-riding standard of "the best interests of the child". Since your son in 16, he has the right to decide who he lives with, but it is not applicable the same way to visitation. Judges do not like to remove either parent from a child's life.
However, if your ex's behavior continues, it may require court intervention. You can file for a modification, but I would recommend that you try other avenues beforehand, if only to have more persuasive evidence for the Judge.
Start by writing a letter (keep a copy) to your ex explaining the situation, and encouraging counseling for him and his spouse. Second, encourage your son to go to some counseling with someone experienced in these areas. if it works out that you can encourage joint counseling, that may be best for all involved.
If it doesn't work out, you have excellent witnesses for your modification action - notably, if the Dad refuses, that is excellent evidence as well.
I wish you the best of luck - feel free to contact me firstname.lastname@example.org if you have any further questions or comments.
Sign up to receive a 3-part series of useful information and advice about child custody law.