That really is a question that is best put before a judge to not be found in contempt or at the wrong end of a custody hearing. Is the brother cleaning up? Has he relapsed? Did he relapse while the children were in his presence? What did their father do. When did all of the negative aspects of his life take place?
My name is Stephen R. Cohen and have practiced since 1974. I practice in Los Angeles and Orange County, CA. These answers do not create an attorney client relationship. My answers may offend I believe in telling the truth, I use common sense as well as the law. Other state's laws may differ.. There are a lot of really good attorneys on this site, I will do limited appearances which are preparation of court documents it is , less expensive. However generally I believe an attorney is better than none, but many will offer a free consultation and a face to face meeting generally will be better, I like my clients to write a short one page history of the fact and questions they have prior to meeting with them, so nothing is forgotten.
I totally can see why you'd be concerned under these set of limited facts. If you truly believe that there is a threat to the well being of your children, you can file an RFO (Request for Order) to modify the existing child custody orders. You will have to show a significant change of circumstances since the last court order. You will need evidence to back up your claims. Your ex-husband will be afforded an opportunity to present his evidence as well. I truly wish the best for your children and you.
This AVVO Answer does not form an attorney-client relationship, nor is it intended to be anything other than the educated opinion of the author. It should not be relied upon as legal advice. Each case is different and not enough facts exist to formulate a proper and thorough response. To the extent additional or different set of facts exist or the hypothetical or scenario presented herein is different, the opinion or commentary of the Attorney might possibly change. Attorney is licensed to practice law only in the State of California and opinions are based solely on California law unless stated otherwise. For answers to specific legal questions, always consult with a licensed Attorney.