It depends on the facts of your case. I would advise consulting with an experienced immigration attorney, who would review your previous I-601 filing and determine whether the best case strategy is appealing the decision or re-filing a new I-601.
It will depend on the quality of the materials you originally submitted, and the strength of the potential appellate arguments.
Because the appeal process itself is so long, it is often a better idea just to refile, but this time with a better prepared waiver packet.
But you should really consult with an immigration attorney and have her review all of your materials, and the agency decision, before decideing which strategy fits your situation best.
The reason that many people state just to re-file the I-601 is because appeals can take a very long time. But, each case is different and you can't go by a generalized opinion.
I am not sure if you used an attorney for the initial filing, but I would strongly recommend that you consult one to handle your case. This attorney will review all your documentation and immigration history and will be able to determine what your best course of action.