What he has done is clearly unlawful. He is obligated to withhold taxes for employees, etc. The question is whether you paid taxes on the income. To be completely lawful, he should have withheld social security and income taxes and you should have paid taxes on your gross income. If either of those did not happen, and it appears he did nothing, the law has been violated. Although I would encourage you to get your own taxes taken care of if you have not done so so that you are compliant, if you elect not to do that you could elect to report him to the IRS and possibly the Department of Labor. One of the losses to you and others here is you are not gaining Social Security credit which you will regret come retirement time.
The employer would also have been required to have you fill out Form I-9 from the Immigration and Naturalization Service. This form, required of all employers, determines whether the employee is in the US lawfully. Employers must keep these forms on file for all employees.