You and your child sound like good candidates for relocation. The lack of visiation and support by father would make it appear that he won't be inconvenienced by the move, and you are able to create a better environment for your child.
This may be a case that requires an attorney. Although the courts make it easy to bring a petition for relocation, you want to make sure that you have all the details necessary in order to prove your case just in case father opposes your proposed move.
In January 2011 the rules regarding Relcoation changed in custody siutations. Before you decide to relcoate out-of-state you need to talk to an attorney to understadn the rules; howver, it sounds as if you meet many of the factors but you need to understand what you are facing if you would decide to move. I would consult an attorney in your area before you do anything.
Sign up to receive a 3-part series of useful information and advice about child custody law.