Any proof that you have that he is committing crimes including assault, harassment, criminal mischief, etc., will help establish that he may not be a fit person to have custody of the children. Are you actually set for a hearing on the matter? If so, you will want to have witnesses come to testify to what they have seen him do, heard him do, his reputation, their personal opinion, etc. If you have text messages, phone records demonstrating that he calls you repeatedly or at all hours, copies of police reports, etc., these may help. (A lot of this stuff would be admissible but not in the form I am suggesting. If he does not have a lawyer, then it will be less of a problem. However, some stuff may be kept out because it is not in the admissible form - one reason that eye and ear witnesses are excellent sources of information.)
Although I have answered the question to try to help you, you should consult with a lawyer in your area in person on the matter. In addition, my answer does not establish an attorney-client relationship between us.
Sign up to receive a 5-part series of useful information and advice about child custody law.