Yes. If you have received benefits and are later determined to not be entitled to those benefits, you will be required to repay those benefits. You should consult with an attorney that handles unemployment because the hearing stage is very important and it becomes increasingly difficult to overcome the decision of the appeals referee at each level of appeal.
These comments are not intended to be legal advice. They are provided for informational purposes only. Actual legal advice can only be provided after consultation by an attorney licensed in your jurisdiction. Answering this question does not create an attorney-client relationship.