This person sounds like they have a problem, other than being classless. I am sorry that you're going through this. If you work for the landlord, he is your employer and does have a responsibility to have you in a safe work environment. California, however, is an at will employment state and, if you complain, he could just fire you and that wouldn't be real helpful in your situation.
My suggestion would be: describe to the landlord that this person is harassing you and it's illegal; indicate that, if they continue, you will call the police; make a report; and file for a civil harassment order. Or, the landlord could just not renew their lease and get another tenant who isn't abusive.