It is possible, if you immigrated as a derivative and he was not entitled to the visa as the primary visa beneficiary.
The above is intended only as general information, and does not constitute legal advice. You must speak with an attorney to discuss your individual case.
Taking away someone's U.S. citizenship is not so easy. It will depend on many factors including your complicity or lack there of for one. I strongly urge that you seek immigration counsel with experience in complex immigration litigation to protect your rights under the law. Good luck!