You should seek legal advice from an experienced family law attorney in your area.
In determining custody, courts universally look at what is in the child's best interest. Drug use, let alone trafficking, is a huge negative. Having a felony record does not look good for the father at all. A good attorney will make the father prove his legitimate income can support his lifestyle. If successful, it will be clear that the father is engaged in illegal matters. This is how the feds took down Al Capone.
Parents have rights, not just fathers and not just mothers. Courts are pragmatic and look to the child's best interest. Best of luck to you.
In Georgia, courts are concerned with the CHILD'S rights (not those of the mother or the father). As such, visitation will only be limited, or denied, if you're able to successfully prove that the child is better off spending little or no time with her father. Yes, prior felony convictions and substance abuse are bad. However, a child does not get denied a parent because that parent's history is bad - Is he CURRENTLY trafficking or abusing?
You should do whatever you think is in the best interest of your child. If that is attempting to limit visitation, then you should hire an attorney to discuss your concerns and strategy.
The above answer is a general explanation of legal rights and procedures. It does not constitute legal advice. Nor does it establish an attorney-client relationship between the individual posting the question and the attorney providing the answer.