Can my tenants break their lease if I can't get rid of the mice?

The reality is that in the cold, mice are a practical reality of living in town. However, the degree of infestation can certainly impact habitability. The good news is that you're remediating the problem. The bad news is that if I were representing the tenants, I would seek a rate concession until the problem is resolved. This next exterminator report will be important because it will either confirm or deny the veracity of the renters' claims. They might have a basis to break the lease because of infestation issues. As for the deposit, this goes to damages beyond the usual wear and tear. If they have damaged the property, you will have the right to itemize and deduct. if not, the deposit belongs to them.