Was it your car or someone you know? If so, it shouldn't take police long to track it to you. You should talk to a criminal defense attorney to assess your options. If you carried insurance, you should notify your insurance company. If you didnt carry insurance, you could be liable to pay for the damage to the fence.
By leaving the scene of an automobile accident with property damage, you are in violation of California Vehicle Code 20002. Law enforcement will be reported of the your abandon vehicle and damages to the fence, they will proceed to investigate, and past the file to the District Attorney's Office to file charges for hit and run pursuant to CVC 20002.
Since you are looking at criminal charges, the best thing to do is not to say anything or volunteer any information to assist the prosecution. Exercise your 5th Amendment Right to remain silent and immediately hire a criminal lawyer. The criminal lawyer can contact the owner of the fence and quickly work out payment of damages (by paying for it or using your automobile insurance to cover it). Then by making attempt to report the accident and providing insurance, you will have a good chance of avoiding criminal charges. I have done this for several clients in Southern California and have avoided criminal charges for them. Contact a local criminal lawyer quickly and you will be fine.
Your best option is to contact a criminal defense attorney as soon as possible. You need to provide him/her with all the information necessary. You can use Avvo's "Find a Lawyer" tool to find an experienced attorney that is suited for your needs.