What you have described provides a basis for reinstatement of supervised visitation. As you may not know, it is the judge - not the guardian ad litem - that makes the decision on custody and visitation. The GAL's job is to investigate and report to the judge. The GAL can make a recommendation, but only the court can enter an order. The court can - and often does - reject the GAL's recommendation. That being said, the case law in Connecticut makes extremely difficult to remove a GAL. (I can tell you that it is very common in custody disputes that one or the other parent wants the GAL removed.) So, what you should focus on is bringing a motion to modify the unsupervised visits, and get a good lawyer to help you do it.
Sign up to receive a 5-part series of useful information and advice about child custody law.