This is tough. As far as I know, you cannot really take your name offf a car you own. I'm saying the car has to be registered and owned and the title must be in someone's same. You are going to go broke letting someone else drive your car. If he parks it somewhere and leaves it or he gets too many tickets, then you will have to pay to get it out of the impound...What if he slams into someone drunk or on drugs or just for being a bad driver AND YOU knowing he is a bad driver could get sued for negligent entrustment. Get your car back. Ask firmly, then ask him very firmly, then if he won't bring it back he has essentially absconded with your car and you call the police, but after you send emails and text message and written certified mail to prove you asked for the car back. No one will just take your word for it that you asked him. He will deny it and say "she told me I could use it."
You will not be able to remove your name from the registration if you are the owner of the car unless you and your ex go to the tag agency and you transfer ownership of the car to him. This would only be possible if you own the car outright (no bank loan) and if he would agree to do it. It sounds, however, like he is perfectly content just driving it with absolutely no worries.
I recommend that you demand that he return the car that you own (make sure to document your requests as the attorney above stated). If he refuses to return the car, you will have no other choice but to go to the authorities.
Best of luck.