If there was an agreement signed, it is possible you may be responsible to pay that back, but it is unlikely they will do much to seek the recovery beyond sending you a letter or two. I'd get counsel involved as soon as possible. There is no basis to appeal a termination unless the employer has put an appeal system in place or there is a union contract involved. You can, however, challenge the termination decision as discriminatory or retaliatory if there is a basis for believing that occurred.
Some companies have a vacation policy under which employees accrue vacation days throughout the year. The employer may allow an employee to use vacation time before it is accrued (go into the negative), and then recoup those used by not accrued days if the employee is terminated. Such a policy is not unlawful.
My answers to questions posted on AVVO are intended to provide general information only, and are not intended to be legal advice. Employment law issues typically require a careful case-by-case analysis. Consequently, if you feel that you need legal advice, I would encourage you to consult in person with an employment attorney in your area.