I am assuming you were never married to your daughter's father. If there has never been a court order allocating parental rights and responsibilities, he was wrong to keep your daughter. Either he or you must go to court to establish his parental rights. Paying child support by itself is not enough. Being declared the father is not enough. I would contact the police and explain that you are the mother (produce a birth certificate) and that the father has never been grated parenting time. Ask an officer to accompany you when you pick up your child for your safety. If the cops will not help, it's time to get an attorney who can file a motion to allocate parental responsibility solely to yourself and an emergency order giving your daughter back to you. Once you have your daughter back in custody, dismiss the action and let dad file it on his own dime if he wants parenting time. In the meantime you obviously shouldn't be relying on him as a babysitter if he's so unstable so as to not give your daughter back to you.
If you were not married to the father when the child was born, or shortly thereafter, then by law you would be the sole custodian of the child. If he refuses to give her back, you should be able to call the police and see if they will assist you in doing such.
If you are married to the father, then he has equal rights and you need to go to court to get a parenting plan outlining your and his rights and when each of you would have your daughter.
Either way, to prevent this in the future, I would suggest you hire an attorney to get a court order so you do not put your child in the middle of a tug of war when her father wishes to start something like this.
Child support nothing to do with your situation at this time. The fact that you recieve child support does not mean you have custody. Now that the child is with him, he can go the the support agency and have the order turned around and have you pay him child support..
Sign up to receive a 3-part series of useful information and advice about child custody law.