If you were not married to you ex at the time that your child was born and if you have never been to court to establish custody orders then you ex does not have any right to parenting time. He will have to go through the court in order to set up a visitation schedule.
In the mean time, you need to get an Order of Protection against your ex IMMEDIATELY. Make sure that both you and your child are named as protected parties. If your ex does in fact file a Petition with the court to establish legal decision making and parenting time, I suggest that you hire an experienced family law attorney.
Sign up to receive a 5-part series of useful information and advice about child custody law.