Why do men have to pay child support even if the father stays with the family

Child support, in theory, is supposed to be gender neutral. When a child support action is filed (such as in a divorce or after establishing paternity if unmarried) one of the parents becomes the obligor and the other parent becomes the obligee. The obligor pays child support and is usually the parent who does not have primary custody. A man who lives with his children and supports them by being around and providing for their needs on a daily basis might be able to get a deviation from the support guidelines on the grounds that he has extraordinary parenting time with the children. It's really up to the judge.

You might want to talk to a lawyer and get some advice regarding your situation. Do so quickly, if you're entitled to a modification, it will only cover future payments, never retroactively.