Unfortunately, you are going to have to take legal action in order to remove him without putting yourself at risk. In Virginia, the eviction process requires the court's assistance - if you use self-help, you open yourself up to being sued.
First, make sure to give him formal written notice that he is being evicted. You must give him at least 5 days notice and a demand to pay, or 30 days notice. Should he not leave after that time, you will need to get the court to issue a Summons for Unlawful Detainer. After the hearing on that summons, you will then need to file a Writ of Possession with your local sheriff's office. Within 30 days of that date, the sheriff will come and remove him. I would strongly suggest retaining a competent attorney to handle all of these steps for you.
Any other mechanism for removing him could be considered self-help, which is unlawful in Virginia. If you do something like call the police to just kick him out, or throw all of his property out and change the locks while he's out, he may be able to sue you for damages.
This response is not offered as legal advice, but is only a general informational response for public interest. No one reading this is authorized to claim that an attorney client relationship exists with this writer or the writer's law firm.