You may defend on the grounds that the document was not signed nor agreed upon by you AND you may defend if sued on the grounds you never knew of the document while you were employed.
Many other facts will play on the outcome of those defenses.
If you signed the agreement, then it may be enforceable against you. However, other factors would have to be considered.
The information provided above is for general purposes only. It does not constitute legal advice or create an attorney-client relationship. Seek competent legal representation, because the facts of each case are different.
I do not believe you have any concerns with complying with a confidentiality order. The concern should be if this is a non-compete agreement, which stops you from working in or near your former employer for a stated period of time. Such agreements (non-competes) must be reasonable to be enforced.
The fact you did not sign the agreement may not make a different as in NJ, you are an "at will" employee." You work so long as the employer agrees to employ you unless you are government or union employee with certain union or civil service protections. The only prohibition against termination is if it is done in violation of the NJ or Federal statutes prohibiting discrimination for age, sex, sexual orientation, religion or nationality.
Oral agreements can be enforced. The problem is proving that it was agreed to and that it was done knowingly.
You should contact an employment law attorney. I know at least two here in NJ in Livingston. You can reach me at 973-994-9080.
The foregoing is not intended to be legal advice upon which you may rely as I have not been retained for this purpose.