Exemption eliminates an officer or proprietor from the need to have w/c coverage. In the construction setting every employee must be covered. I think you need to review the issue with an attorney, but as a sole proprietor who hires employees and works on construction sites, he likely has to have coverage for all the employees including you. Your husband may want to consider the business structure being utilized and whether it best suits his current needs. You mention whether you have to be an officer, which would likely be a corporation or partnership and yet you say your husband is a sole proprietor which is a different structure. This is why i suggest you see an attorney to review the entire process prior to any events occurring triggering the need for coverage.
I am not licensed in your state, but in NY a sole proprietor is not required to have workers' compensation insurance if he or she has no employees. Once there is an employee, such insurance must be obtained. Also, a family member who is not a shareholder, but does work for the company is considered an employee and triggers the need for workers' compensation insurance. Failure to provide this can subject the sole proprietor or in a corporation, the president, secretary and treasurer to personal liability should an employee make a claim for an on the job injury and even without an injury, there may be substantial penalties assessed for not having this insurance. You should consult with a licensed attorney in your state who is eperienced in handling workers' compensation claims and is familiar with coverage requirements.
This answer does not constitute legal advice and does not create any attorney client relationship. You should consult with an attorney licensed in your state who is experienced in the area of law involved in your question.
You are working "under the table" but since you are doing construction you are running the risk of getting injured. You and your husband should consider WC coverage even if it not required. I have seen this situation too often. People who cut corners like you are doing often get "burned."
THESE COMMENTS ARE NOT LEGAL ADVICE. They are provided for informational purposes only. Actual legal advice can only be provided after consultation by an attorney licensed in your jurisdiction. Answering this question does not create an attorney-client relationship or otherwise require further consultation.