Generally, accrued but unused vacation time cannot be forfeited. If your employer put a cap on accrual in 2012, that cap cannot be used to forfeit your previously accrued vacation.
It is difficult to answer the rest of your question without reviewing your paycheck stubs and having more information about how much vacation time you took.
In California, accrued vacation pay is equivalent to earned wages and cannot be waived. If you accrued vacation pay per the policy then-in-effect, your employer must, by law, pay you for that time. The employer has the legal right to change the policy to allow employees to accrue a maximum amount, but it cannot take away what you already accrued. The clearest thing for the company to have done would have been to pay you your accrued vacation at the time it changed its policy. Evidently it did not, adding to the current confusion.
I have a few concerns based on your short summary. First, did you keep you pay stubs? These stubs usually show accrued vacation balances and may reveal additional, helpful information. Do you have copies of the policies in effect throughout your employment? Second you wrote "the handbook when the vacation time was earned had no limit and was also use it or lose it." This statement is internally contradictory. If there was no limit on vacation accrual, there cannot also be a "use it or lose it" policy at the same time.
Note that it is the employer's obligation, not the employee's, to maintain accurate records of time worked and all earning, including vacation. If the employer fails to do so, the employee is legally entitled to reconstruct his or her hours, etc. to the best of the employee's ability; these reconstructed records are presumed correct, subject to the employer's proof they are not.
Your situation sounds too fact-specific for a general response to be of any help. Most likely, you will need to retain an attorney to figure out what your rights are. For this type of assistance, you can expect to pay hourly. Plaintiffs employment attorneys in California charge anywhere from $250 to $750 an hour depending on many factors including experience, area of law, geographic location, work load, interest in the case, difficulty of the case and more. You should expect to need at least three hours for this kind of consultation.
To find a plaintiffs employment attorney in California, please go to the web site of the California Employment Lawyers Association (CELA). CELA is the largest and most influential bar association in the state for attorneys who represent working people. The web site is www.cela.org. Click on "Find a CELA Member" and you can search by location and practice area. Many CELA attorneys represent clients throughout the state.
I hope you can resolve your situation and wish you the best.
@MikaSpencer * * * twitter.com/MikaSpencer * * * PLEASE READ: All legal actions have time limits, called statutes of limitation. If you miss the deadline for filing your claim, you will lose the opportunity to pursue your case. Please consult with an experienced employment attorney as soon as possible to better preserve your rights. * * * Marilynn Mika Spencer provides information on Avvo as a service to the public, primarily when general information may be of assistance. Avvo is not an appropriate forum for an in-depth response or a detailed analysis. These comments are for information only and should not be considered legal advice. Legal advice must pertain to specific, detailed facts. * * * No attorney-client relationship is created based on this information exchange. * * * Marilynn Mika Spencer is licensed to practice law before all state and federal courts in California and can appear before administrative agencies throughout the country. She is eligible to represent clients in other states on a pro hac vice basis.