It depends on why your lawyer does not want to give you the documents. You may have a right to copies or there may be a court order preventing disclosure on third party privacy issues. You may be creating a conflict with your lawyer that will force the lawyer to withdraw. Before you do anything hasty, you should sit down with another lawyer and go over the details of what is going on. Since you say "accuser" I assume this is a criminal matter and I have changed the practice area.
This is not legal advice. You should always discuss the specifics of your issue in person with an attorney. Be aware that there are time limits on all claims that depend on the kind of claim, so do not delay in seeking an attorney.
You can't prevent him from filing a motion to withdraw. The judge may not grant it if you are on your fourth attorney. The only entity that can hold the attorney accountable is the stat bar. I would send him a certified letter demanding a complete copy of your file.
Macy Jaggers's answer to a legal question on Avvo does not establish an attorney-client relationship. Ms. Jaggers offers everyone a free consultation to discuss their case. Feel free to call her office at 214-365-9800 to make an appointment (phones are answered 24 hours) or visit her website at www.macyjaggers.com for more information about her services and recent victories.