Unfortunately, there is probably not much you can do about your previous attorney, as he is entitled to ask for the retainer prior to starting to work on your case. You should call your local bar association or court clerk to find out if you can get low cost or free legal help in your custody matter. You may also find an attorney who will charge you a smaller fee. Good luck.
Sign up to receive a 10-part series of useful information and legal advice about the divorce process.