You can file suit but there are various practicalities on these types of contracts. Many courts do not like to honor them. You should contact an attorney that specializes in this field to understand the likelihood on winning your case. Considering there probably is someone that they can recover then you probably can find a contingent attorney although they'll want a huge amount because of the risk.
This is just my opinion and you assume the risk because I am not promising it is completely accurate.
Of course you can sue them, but your chances of success are unknown. You won't have an easy time finding a lawyer to litigate a case when your maxmimum upside is $20,000. A lawsuit can easily cost that much a matter of a few months, and it often takes more than a year, sometimes 2, to get a case ready for trial, and mayb also involve several thousand dollars in our-of-pocket court costs, deposition costs, expert witness fees, etc.
Most claims settle before trial, so it's possible you could find a lawyer to try to negotiate a settlement, depending on how strong your claim is, but obvioulsy your claim has problems or the agency wouldn't have refused to release your money. No one can tell from this post, since you're asking about what your contract rights are, and no one has had the chance to review your contract to see what its exact language says.
Consult a business litigator to get your conract reviewed and discuss the details of your situation.
PLEASE READ THIS BEFORE YOU COMMENT, EMAIL ME OR PHONE ME. I'm only licensed in CA. This answer doesn't make me your lawyer, and neither do follow-up comments and/or emails and/or phone calls, and you shouldn't expect me to respond to your further questions if you haven't hired me. We need an actual agreement confirmed in writing before any attorney-client relationship is formed. This answer doesn't constitute legal advice, and shouldn't be relied on, since each state has different laws, each situation is fact specific, and it is impossible to evaluate a legal problem without a comprehensive consultation and review of all the facts and documents at issue.
Yes, you can file a lawsuit against the agency, but your chances of success at trial and/or for settlement is difficult to predict without having read the contract you signed. Moreover, you will probably not have much luck finding an experienced contingency fee lawyer of the total recovery is only $20,000.