You indicate you have a "schedule" with the children's father. If the schedule is by way of a Court order, and Father does not agree to your request to modify the Court order, then you must go back to Court and ask the Court to modify the current order.
If you violate the Court order, you are in contempt of Court.
If you have a case involving these children, do the right thing, go to court now, and ask for permission to move.
It would be extremely inconvenient, and possibly a criminal matter, if you move, the ex then runs to court, claims you violated the court order, or worse, kidnaped the children.
Do it once, do it right, go to court if you cannot get a written agreement with the children's father.
Please note that this answer does not constitute legal advice, and should not be relied on, as each situation is fact specific, and it is not possible to evaluate a legal problem without a comprehensive consultation and review of all the facts and court pleadings filed in the case. This answer does not create an attorney-client relationship