I doubt CPS is willing participant in any harassment against you, since they have a duty to investigate claims child abuse, neglect, etc. Unfortunately, the ex-wife is abusing the system to have them repeatedly investigate her claims. I doubt that you can convince CPS that they cannot accept any more complaints from her, but you might hope that they become more stringent in screening her complaints against you.
You might consider sending her a Certified Letter putting her on notice that every time that she files a complaint with CPS that is later determined to be unfounded (or whatever the official disposition of the complaint is), and as a consequence you suffer any lost wages or other expenses in cooperating with the CPS investigation, that you shall hold her personally liable for the total amount of your expenses and/or lost wages that she has caused you because of her wrongful acts. And, if she fails to reimburse you after you send her a bill, put her on notice that you intend to sue her in civil court for your damages, and that if she loses in court, she may also be liable to you for your attorney's fees in bringing the lawsuit against her.
Keep it simple and to the point, do not make any threats of violence or criminal charges.
Consult with an attorney.
I am an attorney who is only licensed in the State of Florida. My answer is general legal advice based upon what I perceive your question to be, and should not be relied upon because every person's facts and circumstances are unique, and because specific laws vary from state to state. To completely evaluate a legal issue requires reviewing and evaluating all relevant facts, applicable laws and other information. My answer does not create an attorney-client relationship, and offered for informational purposes only.