It is important to have open and honest communication with your attorney about the legal representation and the fee. Did you sign a written fee agreement? Review the agreement carefully. Typically, a lawyer can withdraw from representation, or ask the Court to allow him or her to withdraw, if there is a conflict between the lawyer and the client. It is important to talk these things through and try to find an amicable resolution. You also have the right to consult with independent counsel about the situation.
This answer does not include any legal advice. You should personally contact an attorney. This answer does not create an attorney-client relationship.