In Colorado you can establish a landlord tenant relationship without a written lease. You will need to post a three day notice and file with the county court. A lawyer can easily assist you with this. Once the eviction is ordered the sheriff will have to come out to the property to assist with the physical removal of the personal property of the former tenant. Remember no good deed goes unpunished.
You have a verbal agreement that the person can stay with you and pay a certain monthly fee. This is enough to create a month to month tenancy. If you no longer want them to stay after the end of the month, you will need to give them notice that you are terminating the month to month tenancy. If they actually pay you as agreed, they are entitled to stay until the end of the first month of tenancy. If they don't pay, you can post a 3 day notice and start eviction right away.
It's not clear what you mean by "some stuff was stolen". If you have evidence that your new roommate stole things from you, you can contact the police. If they are arrested, that may result in a no-contact order which will keep them away from the property. However, you will need some evidence to support such allegations.
Obviously, in the future you need to think through offers you make to other people. You need to decide if you really want to open your house to them and you need to consider obtaining a written lease and a deposit before you let people start moving their things in. Hopefully this less won't end up being too expensive.
You can reach Harkess & Salter LLC at (303) 531-5380 or info@Harkess-Salter.com. Stephen Harkess is an attorney licensed in the state and federal courts of Colorado. This answer is for general information only and does not create an attorney client relationship between Stephen Harkess or Harkess & Salter LLC and any person. You should schedule a consultation with an attorney to discuss the specifics of your legal issues.