You are correct that covenants not to compete are highly disfavored in California. However, you should take care that you are not using any proprietary information obtained from your prior employer. I recommend that you speak with an attorney and have the attorney review the agreement that your former employer claims that you signed.
There is a big difference between what you can legally do, and what you might get sued for. In California, non-compete provisions are unenforceable. If your former employer is also a California "resident" then it is likely you would prevail if they sued you for breach of that provision.
That said, there is no guarantee you will not be sued. You have to litigate the matter to win it. Many companies will use predatory litigation to kill off baby competitors before they get big. You need to be on the watch of that.
Another way a former employer can attempt to crush the life out of a new business is to use the claim that you are using confidential information taken from your former employer to carry on your business. Therefore, be squeaky clean in that regard. Do not use customer lists, unique methods, or even forms that look like forms used at the prior company.
Good luck to you.
This answer should not be construed to create any attorney-client relationship. Such a relationship can be formed only through the mutual execution of an attorney-client agreement. The answer given is based on the extremely limited facts provided and the proper course of action might change significantly with the introduction of other facts. All who read this answer should not rely on the answer to govern their conduct. Please seek the advice of competent counsel after disclosing all facts to that attorney. This answer is intended for California residents only. The answering party is only licensed to practice in the State of California.
Noncompete clauses in contracts are unenforceable in CA. However, you should get copies of all agreements you signed to determine if there were other clauses or contracts you could be sued for if you violate them. Example: Nondisclosure of trade secrets, customer lists, etc. Then consult with an employment law attorney.