I can't say it is impossible that he could get partial custody and visitation but the facts look like they are solidly in your favor. Your husband has a fundamental misunderstanding of how child custody and support work. He can be ordered to pay support without getting visitation. You would be able to best protect your rights by hiring a local attorney.
Ryan Ballard is licensed to practice law in Idaho. He can be reached by phone at 208-538-9727 or by email at email@example.com. His website is BallardLawIdaho.com. This answer is for informational purposes only. It is not intended as specific legal advice regarding your question. The answer could be different if all of the facts were known. This answer does not establish an attorney-client relationship.
If you can prove what you have said, it is not likely he would get custody. However, men like that frequently get visitation. If the abuse is toward you alone, then that will not affect whether he gets visitation with the children. Definitely his cheating on you will not affect his visitation. That's between you and him. The things that would prevent visitation would be things like physical (or serious emotional) abuse of the children, having overnight guests of the opposite sex while the children are present, introducing the children to things that are inappropriate (such as pornographic magazines laying around the house, doing drugs in their presence, etc.). Judges are very sensitive to allowing BOTH parents to have a relationship with the children. And it is not up to the children whether they see the non-custodial parent -- it is up to the judge. He will have to support the children whether he is given visitation by the judge or not; but likewise, you have to let him see them whether he pays child support or not -- unless a judge tells you otherwise. Child support and visitation are mutually exclusive. If he doesn't support them, he can be held in contempt. But if you don't let him see them, you can be held in contempt (if a court has ordered visitation for him).
Your best option is to retain a family law attorney immediately to protect your rights and those of your children. If you cannot afford an attorney (and with six kids, I can understand that may be a possibility), then you should call your local legal aid office immediately. They will generally represent you if there has been abuse (you must be able to prove it) and if your income is low enough to qualify. And if the abuse has been toward the children, legal aid will almost always step in and help the custodial parent.
Get on this right away. You want a temporary custody and child support order in place as soon as possible. That temporary order will also either deal with visitation or will deny him visitation, depending on the facts of your case. You have some good judges in your jurisdiction who are all about protecting children. Be sure you remain above reproach. NO boyfriends. NO wild partying. NO drinking or carousing. You want to take the high road so the judge has no reason to deny what you're asking. And do not discuss the "mistress" with the children. No judge likes a parent -- regardless of whether you feel justified -- who brings the children into the parents' drama. Lean on your friends for support in regard to this cheating jerk; but do not lean on your children for that support. Not saying you have been -- just saying this is good advice when dealing with custody/visitation issues.
No attorney-client relationship is established with this answer. It is not to be considered legal advice, but is merely given to point you in the right direction and give you a general answer as to the law regarding the question you have asked.