I think I have been slandered what action can I take

I was dating a girl,we got pregnant,9 months go by,and my sweet little daughter was born,#2 for me...# 4 for my girlfriend.Her ex-husband and his new girlfriend,are the source of this problem. This is embarrassing for me to say ,but I when my wife left me I turned to the bottle pretty heavy,for about 3 months,I ended up with a charge of communicating with a minor,for a poor choice I made.
Anyway when my girlfriend was in the hospital giving birth to my daughter,her 3 kids were at the ex husbands house. His girlfriend held up a picture of me and told those 3 kids that I raped a 10 year old!Ok that would be bad enough but now the ex husband forbids me to be around his 3 kids so that means between my girlfriends work schedule and not being able to be around the kids I cant see my baby ever!