It may be possible. The question for the court is how your wife's behavior affects the "best interests of the child/ren." If this individual was seen by the children, used drugs in the presence of the children or was high around the children, or even, whether the children saw something inappropriate (that is, sexual) between the man and your wife, then these are all things the court needs to consider. You may be able to get the court to order the appointment of a guardian ad litem to investigate, talk to you, the wife, the kids, other witnesses, and make a recommendation to the court. You definitely want to talk to a Georgia licensed attorney about this and you should not consider representing yourself. Don't hire the biggest "pit bull" in the bar. Hire someone who is assertive and cares about protecting children. A former prosecutor in juvenile dependency will be very fine tuned to the effect this situation has had on your kids. Find one in private practice and take good care of your kids!
They are counting on you to do the right thing!
My response does not constitute legal advice but is provided for the purposes of advertising and education. Please contact a Georgia attorney if you wish to obtain confidential legal counsel.