If he has not paid the rent, then he is in violation of the terms of the lease and you can give him a three day notice. If he does not pay, then you can move to evict him. That is at least a quicker process.
If he was stealing from you, you might want to report it to the police. His being arrested would be an even quicker way of getting him out.