The answer to any question of the form "Can I sue X for Y?" is always "Yes." In other words, in this country, anyone who can afford a filing fee can file a lawsuit against anyone else for anything they choose, with no exceptions of which I am aware.
The real question, which I am sure you intended to ask, if whether your ex-GF could possibly prevail if he were to sue you for $40,000 in damages or, in the alternative, for title to the home. That is a far more difficult question to answer. However, based on the information presented, this appears to be a matter of contract law, although your ex-BF might also have what is known as an "equitable claim," such as unjust enrichment.
If you watch "Judge Judy" or any of the "real case" television programs such as that, you would note that almost every episode features at least one case in which former lovers are fighting over whether money given from one litigant to another, or spent on his/her behalf. Courts resolve such disputes by examining what agreements were actually reached between the parties and, if there was no express agreement, by the conduct of the parties. Because of your relationship, you always have an argument that all of the funds he spent on the home and your debt was in the nature of a gift, while he will claim it was a loan. You have several options as to how to proceed, and given the amount of money involved, I would strongly suggest that you consult legal counsel in exploring these options, although if you are not all that worried about his threats, you can also simply wait for him to file suit.
THIS RESPONSE DOES NOT ESTABLISH AN ATTORNEY-CLIENT RELATIONSHIP AND DOES NOT CONSTITUTE LEGAL ADVICE. IF YOU WISH TO OBTAIN LEGAL ADVICE UPON WHICH YOU CAN RELY, PLEASE CONTACT MY OFFICE OR THAT OF ANOTHER ATTORNEY FAMILIAR WITH THE SUBJECT MATTER OF YOUR INQUIRY.
For purposes of this answer, I am assuming that the home is titled to you under a joint tenancy arrangement. In Maryland, a joint or co-tenant is entitled to contribution for capital improvements made toward the home. Generally, these claims are made in connection with a request for the court to sell the property (this is allowed by a statute that permits the Court to conduct a sale in lieu of partition). The level of proof on this type of issue can be daunting for any practitioner. First, there must be some evidence that the capital improvement was discussed and agreed upon (either expressly or implicitly). You do not, however, receive a dollar-for-dollar credit on capital improvements under a contribution claim. The issue becomes the extent to which the improvement has actually enhanced the value of the property. I say that this becomes challenging insofar as with the fluctuations and volatile nature of the real estate market, it would be difficult to determine how the improvements have "appreciated" the value of the home while depressed market conditions have diminished equity for most homeowners. This issue is further exacerbated by the "commingling" you mentioned which I believe could be fatal to any contribution claim. If you were commingling your income with his income, then it becomes impossible to determine whose income was actually advanced for the purpose of paying for the improvements. To the extent that you paid for other joint expenses that enabled him to have funds with which make the improvements, it will be incredibly difficult for him to proceed forward with any viable contribution claim.