Stick to the orders, call the cops if it is really getting to you... does he have grounds to claim that the Court will take your children away? As for attorney's fees, if he presses in bad faith and to harass you, the Court might award attorney's fees to you.
This answer is intended for informational and educational purposes only and should not be considered legal advice nor forming the attorney client relationship. This attorney is licensed in Texas.
How is he getting the children when it is not his time? Is he picking them up from school? If so, school needs to be notified of the custody schedule. Keep a record of when he removes them when it is not his time, and also make certain the court's visitation advocate keeps a record.
Sign up to receive a 3-part series of useful information and advice about child custody law.