Anyone can sue anyone for anything so "can they" is never the right question, and every litigant has a case, but the better question is whether or not it's a meritorious case that they could win.
This question depends on what your contract was, and yes, an oral agreement can be an enforceable agreement. Breach of a contract is proved by the 1) the contract 2) the breach 3) no legal excuse for the breach and 4) damages. Here you've gpt an oral contract that was amended apparently at least a couple of times. Each amendment, like the contract itself, has to be proved. Proof is through documentary evidence such as emails and texts, and through testimony of witnesses including you and him. Unless and until he does sue, don't worry about it. If he does, time to see a lawyer.
Avvo doesn't pay us for these responses, and I'm not your lawyer just because I answer this question or respond to any follow-up comments. If you want to hire me, please contact me. Otherwise, please don't expect a further response. We need an actual written agreement to form an attorney-client relationship. I'm only licensed in CA and you shouldn't rely on this answer, since each state has different laws, each situation is fact specific, and it's impossible to evaluate a legal problem without a comprehensive consultation and review of all the facts and documents at issue.
If you are the title holder, and you made all the payments, then you are the owner. You say you agreed to buy him out, was that a written agreement?
Then he "gifted" it? I am not sure what that means, but if he gifted the balance to you, then you can argue that he gave it to you.
If he can show that he made payments, then he may be entitled to an offset for some of the payments.
Threatening to sue and suing are two different things. If he sues, then you should immediately contact an attorney for a consultation.
You are more than "on the verge of breaking up" if he is threatening to sue you.
It is unlikely he will sue given the facts you have stated. However, there is nothing you can do about it right now. Wait to see if he sues and you have your defense lined up.
Good luck to you.
This answer should not be construed to create any attorney-client relationship. Such a relationship can be formed only through the mutual execution of an attorney-client agreement. The answer given is based on the extremely limited facts provided and the proper course of action might change significantly with the introduction of other facts. All who read this answer should not rely on the answer to govern their conduct. Please seek the advice of competent counsel after disclosing all facts to that attorney. This answer is intended for California residents only. The answering party is only licensed to practice in the State of California.