Can he do that? And what's my rights?

Can he? Yes. Is it legal for him to do so? Maybe. Might his conduct violate your right to the peaceful use and enjoyment of your apartment? Maybe. It is not black or white, and may depend to some extent on the totality of all circumstances. If you want to explore whether a claim can be mounted, you will need to talk to an attorney.