First, you say "we have joint" then you say "he has sole". Not sure what this means. So the answer is a bit general.
If you have filed for sole custody and social services are involved, then you may want to check with them on your options. You should also bring to your court hearing as much data/proof of the dv situation he went to jail for. And you should get some legal advice. You can always seek the help of the family law court self help clinics.