If you did not cause the damage you are not responsible , if the landord's agent (the handyman) did the damage. The problem you may have is proving it. Talk with your LL First. He may be willing to reach an agreement with you. If not you will have to sue. NJ provides for double damages if you are successful.
I agree with Mr. Schutz. You do not have to pay for damage you did not cause. If you take the route of going to court, you will have the classic "he-said / she-said" situation with two different versions of the facts. In NJ, the landlord has the burden of proof of establishing that you caused the damage.
If this answer was helpful, please mark it as helpful or as a best answer. This answer is for general education purposes only. It neither creates an attorney-client relationship nor provides legal guidance or advice. The answer is based on the limited information provided and the answer might be different had additional information been provided. You should consult an attorney.