If, when you say: "sign over my rights" you mean that you want to abandon your relationship with your children and thereby not have to pay child support. The answer is NO!
Rather than giving up on your children - who are losing out on having a father in their lives through no fault of their own - you could fight for your right to have a relationship with them. If you have joint custody as you say, then the mother is in violation of the court order. You need to file a show cause order. She would then need to demonstrate why she should not be held in contempt of court for violating the parenting provisions of the order. Withholding parenting time would also be a consideration in changing custody or reducing her parenting time (and increasing your parenting time) with the children. Stop feeling sorry for yourself and take some action!
IMPORTANT LEGAL NOTICE: Mr. Piper's response set forth above is not legal advice and it does not create an attorney-client relationship. Mr. Piper's responses to all questions posted on AVVO are intended to provide general information based upon the his understanding of the facts stated in the question, and are for the general educational purposes of the public, not any specific individual. Also, a particular case may involve additional facts and circumstances which might invalidate some or all of the concepts provided in this answer and therefore you should not rely upon this answer in any individual situation. In order to offer legal advice about this or any similar situation, a qualified attorney would likely need to consider many factors not stated in the question and would need to question the potential client in order to clarify the specific facts operable in that case. If you would like to obtain specific legal advice about this issue, it is recommended that you contact an attorney in your state. Mr. Piper is licensed to practice law in the State of Ohio, and may be contacted directly via email at: firstname.lastname@example.org.
Your situation is unfortunate, especially for the children. The short answer to your question is no. Your rights are set forth in the parenting order issued from your local court. It is up to you to exercise those rights or not to exercise them. If your ex is not following the terms of the order, then you need to seek legal counsel and have your rights enforced.
If you do not enforce your rights your children will not have their father in their lives. Please think long and hard about this matter and seek a consultation before deciding to simply not have a relationship with your children.