It's a genuinely tough situation.
Technically, under the law (the Uniform Child Custody Jurisdiction Enforcement Act), NJ has jurisdiction since the child was born here. He could, legally, leave with the child and file an action here, including an emergency request for custody.
But real-world.... we're talking about a 4 month old baby. With "infants" (less than 2 years old or so), NJ still recognizes a "tender years" doctrine and unless mom is a crack addict, abusive with a DYFS history, etc., a judge will be VERY reluctant to grant residential custody to someone other than the mom. I don't know FL law on this, but it's probably similar.
If he just left (and assuming he doesn't have proofs of abuse, etc, as outlined above), I'd wager FL would enter an emergency order and the authorities would return the child. He probably isn't violating a criminal law, but it's going to make things difficult, to say the least.
Sorry I can't offer more solid advice. It's a tough situation. I'd suggest contacting FL counsel and seeing what they say, as well.
The above is said without seeing your case file and without my understanding the entirety of the facts of your case. Depending on those facts, the above information be may incomplete or may be completely inaccurate. The above is intended as general information only based on what you described and not as legal advice. I advise you to consult with counsel who may be able to provide better information commensurate with a better understanding of your situation.
He needs to consult with an attorney in Florida regarding jurisdiction and relocation/removal.
DISCLAIMER This answer is provided for educational purposes only. By using or participating in this site you agree and understand that there is no attorney client privilege between you and the attorney responding. This site cannot be used as a substitute for competent legal advice from a licensed attorney that practices law in the State where this offense is charged; and, who has experience in the area of law you are asking questions about and with whom you would have an attorney client relationship. The law changes frequently and varies from jurisdiction to jurisdiction. The information and materials provided are general in nature, and may not apply to a specific factual or legal circumstance described in the question, or in the State where this charge is filed