Best suggestion is to get a legal consultation immediately.
As a general rule, you don't want to hold one part of a court order "hostage" because another court order is not being complied with. A proper review of the original orders plus what has transpired since is necessary in order to make sure you get completely off the house and loan and have no lasting repercussions.
Your response should request the relief that you believe is most fair and equitable. Can't say what exactly to tell you as the judgment would be important to see first before you respond. Please seek the help of an attorney.
If I understand your post correctly, you were ordered as a result of the final divorce settlement to sign over the property to him but you refused. Now, he has brought your failure to comply with the order to the court's attention for assistance. When you write that you are not sure what to specifically say, in response to his motion, please understand that your post is far too complicated and fact specific for an answer in this forum. You should have this matter handled by a family law attorney. You may be in contempt, you may have a basis for a revised order, you may need to be seeking your own relief. In the end, you should attempt to justify your refusal to sign and show all changes in circumstances since the original settlement date to support your position. Lastly, expect that any plan for the property that removes the adverse credit impact (such as sale or refinance) will likely be supported by the judge if your refusal to sign stems from his adverse impact on your credit.
As stated above, this is an incomplete answer but an attempt to provide a cursory overview of a subject that attorneys 'practice' and improve upon during the entirety of their legal careers. This response is not meant as legal advice or as a legal opinion. Such advice would be impossible or impractical without additional information and more facts giving rise to the question. A consultation with an attorney is necessary.
A weekly guide with tips and legal advice for each stage of the process.