You can fire your attorney at any time. You will likely owe for any work performed prior to disengagement and any subsequent work required to disengage. For example, it might cost the attorney several thousand dollars to disengage if the attorney is attorney of record in some court case and the counterparty does not want to dismiss and the judge does not want to allow the attorney to withdraw. (Judges are reluctant to let an attorney, particularly a defense attorney, out of a case and leave some client pro se with all the inevitable hassle that presents for the court.)
I am not your lawyer and you are not my client. Free advice here is without recourse and any reliance thereupon is at your sole risk. This is done without compensation as a free public service. I am licensed in IL, MO, TX and I am a Reg. Pat. Atty. so advice in any other jurisdiction is strictly general advice and should be confirmed with an attorney licensed in that jurisdiction.
Disclaimer: The materials provided below are informational and should not be relied upon as legal advice.
Generally, there are no "grace periods" or "cooling off periods" for attorney-client retainer agreements. You should read your agreement and see if one was provided in your matter. If the attorney has not yet performed the services under the agreement, you may be entitled to a refund (or partial refund) of your retainer deposit.
Just advise your attorney to cease work. You are entitled to a refund of the unearned portion of any retainer fee deposited.
The above is general legal and business analysis. It is not "legal advice" but analysis, and different lawyers may analyse this matter differently, especially if there are additional facts not reflected in the question. I am not your attorney until retained by a written retainer agreement signed by both of us. I am only licensed in California. See also avvo.com terms and conditions item 9, incorporated as if it was reprinted here.