If he becomes beligerent, then make sure you are safe and keep the authorities posted. If you need to resort to a restraining order once physical violence is threatened, contact a lawyer.
But, all this starts with you asking him to leave. Have you?
As you were represented by counsel you acknowledge you talked it all through with the lawyer, too,
Good luck to you.
NOTE: This answer is made available by the lawyer for educational purposes only. By using or participating in this site you understand that there is no attorney client privilege between you and the attorney responding. This site should not be used as a substitute for competent legal advice from a licensed professional attorney that practices in the subject practice discipline and with whom you have an atttorney client relationship along with all the privileges that relationship provides. The law changes frequently and varies from jurisdiction to jurisdiction. The information and materials provided are general in nature, and may not apply to a specific factual or legal circumstance described in the question.