You need to file an Original Petition to Declare Marriage Void in the county in which you reside as soon as possible. Contact an attorney to help you with the paperwork or a reference librarian at your local law library.
This is general legal advice. An attorney-client relationship has not been formed. Please contact an attorney for specific advice on your case.
I'm sorry you're in this situation--that's got to be a shock. I know this seems like a nitpicky little thing, but actually you're not going to qualify for either an annulment or a divorce. Both of those would presume that you currently are married to him, and legally, you're just not. If you're concerned about him trying to come back later and claim you're married, first, know that that won't fly legally as long as you leave him before his divorce becomes final. If you're still concerned, you can file with the family court for a declaratory judgment (where the court will hear evidence about the existence of his prior marriage and say, "No, your marriage is not valid"). I'd have to look it up to be sure, but I think it would be called something pretty close to a "Declaration of Void Marriage". Good luck getting this straightened out.
I agree with the earlier posting to file an Original Petition to Declare Marriage Void. You do need to either serve your "husband" with a citation and copy of the petition or get him to sign a waiver of service AFTER the petition is filed. If he signs the waiver, you may go to court to get the order/judgment signed. If he is personally served with citation, then you have to wait unitl the "Monday next after the expiration of 21 days from service" to get the order/judgment signe.
I am not intending this to be legal advice, because I don't know the particulars of your situation. Call me if you would like to discuss this or other isues.
A weekly guide with tips and legal advice for each stage of the process.