It sounds like you are talking about a "mortgage contingency" clause in the contract. That means that the contract is contingent on you obtaining a mortgage. In other words, if you make all reasonable efforts to obtain a mortgage, but can't, then you can cancel the contract. It does not mean you can cancel the contract at any time prior to getting approval for a mortgage. The seller will likely try to retain the down payment. They could sue you to enforce the contract, but that is unlikely as it would likely be too time consuming and expensive for them.

Once the seller accepted your offer you had a contract. When you notified them you changed your mind you breached the contract. The seller could sue you for that breach. Before you do anything further discuss this with a local attorney.

There is no incentive for the seller to release you from any liability you may have for your purported breach of the purchase contract. It seems the seller is reserving her rights to sue you if deemed necessary.

The Seller and their attorney are positioning themselves to keep your downpayment. You must hire an attorney )if you do not have one), today.

It would be impossible for anybody but the seller to tell you what they are either planning group to or whether they will sue or hold you to the contract. As I have not seen the contract itself, I strongly advise you to speak with an experienced real property attorney in your area to review the applicable terms.
Best of luck.
The foregoing discussion does not establish an attorney-client relationship, is qualified by the limited facts presented above, and should not be relied upon as legal advice. To obtain definitive legal advice upon which one can rely necessitates retaining an attorney who is qualified in this particular area of the law. Any comments are merely suggestions for you to think about in discussing your situation with your local attorney.