You may not need to do anything. If you are unmarried, you have sole legal custody (major decision-making) unless and until he goes to court to seek shared legal. You have sole physical custody. Unless and until he goes to court to establish himself as the father, you can deny him visitation, and you can oppose it if he does seek visitation. To have him totally out of your life, however, you would be sacrificing child support. To get child support, you will have to go to court and establish him as the father and get an order of payment. Now, child support, visitation and custody are all separate things, so just because he pays support does not mean he gets custody or visitation. But he'll have the right to seek them. If he follows what he has been doing, it seems like he won't want any involvement. Bu think long and hard about your needs and your daughter's needs before agreeing to give up child support.
To questioners from West Virginia & New York: Although I am licensed to practice in your state (in WV, on inactive status as of 9/13), I practice on a day-to-day basis in Massachusetts. I answer questions in your state in areas of the law in which I practice, and in which I feel comfortable trying to offer you assistance based on my knowledge of specific statutes in your state and/or general principles applicable in all states. It is always best, however, to work with attorneys and court personnel in your own area to deal with specific problems and factual situations.
While it is sad and neglectful, it is unlikely that the court will terminate his parental rights based on those facts alone. You should go to family court for an order of support and an order of custody. If he files for visitation in response, address your concerns about his lack of parenting skills to the court.
The content of this post is not legal advice and should not be relied upon as such. I am not your attorney and no attorney-client relationship has been formed based upon this exchange.