Yes, if you're an "at wil" employee with no written contract and no labor union that says otherwise, your employer can fire you without notice for any reason, They can also do less than fire you, like reduce your hours, and/or your benefits and/or your salary and change your title and/or your job duties, etc. etc.
Your corresponding right is to quit whenever you want to for any reason.
PLEASE READ THIS BEFORE YOU COMMENT, EMAIL ME OR PHONE ME. I'm only licensed in CA. This answer doesn't make me your lawyer, and neither do follow-up comments and/or emails and/or phone calls, and you shouldn't expect me to respond to your further questions if you haven't hired me. We need an actual agreement confirmed in writing before any attorney-client relationship is formed. This answer doesn't constitute legal advice, and shouldn't be relied on, since each state has different laws, each situation is fact specific, and it is impossible to evaluate a legal problem without a comprehensive consultation and review of all the facts and documents at issue.
Unless you have a contract or some sort of agreement that he won't do that, then yes. On the good side, you are free to find another job at any time.
My suggestion is that you tell your employer that it is a struggle to work only two hours, and ask him nicely if you and he can agree on a minimum shift length. You might even ask if there is something that you can do on your side to make this a reasonable good thing for both of you.
I'm not your attorney; my answer to your question includes assumptions. If you want me to be your attorney, I'm easy to find.