You may want to put your health ahead of other considerations. If you move out and stop paying rent the landlord may or may not sue you; and if he sues you he may or may not win. But--according to your statement--if you stay in the apartment you will continue to get sick and miss work.
Have you tried simply asking the landlord to cancel the lease effective at some near future date? While I understand that you may be peculiarly sensitive to mice, one mouse is not going to constitute an adequate infestation of vermin to allow you to cancel the lease without running the risk of being sued. However, the rental climate is pretty good for landlords right now and you could be seen by yours as a problem case. Landlords often walk away from problem cases that require too much time, effort or money to resolve. Good luck.
Please be sure to mark the best answer to your question. My answers are general and do not form an attorney-client relationship. I'm happy to talk to prospective clients in my areas of concentration and geographical location.