You would need to file a Petition for Modification of Custody, and request that the Court change the current visitation schedule so as to limit the Non-Custodial Parent's visitation time with the child. You will need to show the Court several compelling reasons for restricting his access to the child, and that such a restriction is in the child's best interests. It sounds like you have a few facts above, that would help your argument. You need to consult a local Atlanta attorney to discuss this matter further, and provide more facts. Many offer free consultations.
Sign up to receive a 5-part series of useful information and advice about child custody law.