As a general rule, the owner of the dominant tenement cannot interfere with your use of the right of way. That does not mean that he cannot also utilize the are - just that his use cannot unreasonably interfere with your use.
You should speak to a local attorney about your rights and what steps you can take to limit the other party's activities.
I am not a CA attorney, laws vary from state to state, therefore you should always consult a local attorney.
If this answer was helpful, please mark it as helpful or as a best answer. This answer is for general education purposes only. It neither creates an attorney-client relationship nor provides legal guidance or advice. The answer is based on the limited information provided and the answer might be different had additional information been provided. You should consult an attorney.