Since you have sole physical custody and your ex is facing serious prison time, it should be a simple matter to have a court order sole legal custody as well. Working with DCF and the GAL, you should be able to significantly restrict visitation, potentially even being able to avoid bringing the child(ren) to prison to see your ex.
You should file for sole physical and sole legal custody in the family court as a criminal court does not have jurisdiction to hear your custody matter. The custody needs to be filed in family court and handled there.
As for terminating parental rights, even with your ex's situation, it will probably be a steep uphill battle without DCF assistance.
Brian S. Karpe, Esq. (860) 242-2221 Note: This response DOES NOT constitute legal advice and therefore no specific action should be taken in reliance thereon. No attorney-client relationship is created through this response. You should speak to an licensed attorney in your state who is competent to answer your question before taking any action with regard to this question.
Has DCF made any indication that they may proceed with a case in the juvenile courts? That is one venue to remove parental rights. The other would be the Probate Court. It seems like there are a lot of details and you may not be able to relay the entire story of your situation in one short blurb. I am going to recommend that you find an attorney who will evaluate your case and give you an idea of what you can expect to happen. Good Luck.
Providing answers to questions on AVVO does not create an attorney/client relationship. It is not possible to provide legal advice in the context of question and answers on the internet. The "answers" posted are solely for the purpose of providing general information about legal matters. Consult an attorney licensed to practice in your state or jurisdiction before taking any action. Be advised that what you post on AVVO is not confidential and in a civil law suit may be discoverable.