You could actually do it either way. If you set up in NY you will need a physical address.
This e-mail may contain confidential or privileged information. If you are not the intended recipient, please notify the sender immediately by return e-mail and delete this e-mail and all copies and attachments. If you are not the intended recipient, or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. IRS Circular 230 Notice: Unless specifically stated otherwise, any tax advice contained in this communication (including any attachments) is not intended or written to be used, and cannot be used, for the purpose of (i) avoiding penalties under the Internal Revenue Code or (ii) promoting, marketing or recommending to another party any transaction or matter addressed herein. Unless specifically stated otherwise, this communication shall not be deemed to be legal or tax advice, and no attorney-client relationship shall be deemed to have been created.
From the facts your provided, since you are performing the services in New Jersey, and are a New Jersey resident, most likely you should set up the LLC in New Jersey. Depending on how you are rendering the services, you may have to be authorized to do business in New York and other states, but that may also not be a requirement. Frankly, this is not something you should do yourself. You should hire a good tax and business attorney to help form the LLC, decide where should be formed and to help establish your business correctly. This will save you money and help you grow better in the long run.
I hope this helps!
If you do not like this answer or disagree, please look at one of the other answers provided. It is not necessary for you to try prove this answer is "wrong" or something with which you do not agree. This is a free service for you based on limited facts. Nevertheless, many times you need to consult an attorney with the details to get actual advice specific to your concerns. Do not put too many details in your questions or comments because this makes the information public and could hurt you. Government Regulations contained in IRS Circular 230 regulate written communications about Federal tax matters, including e-mail, between us and our clients. This is another attempt by the government to limit your rights and to extend the control of government over individuals and businesses. Nevertheless, such communications are either opinions or other written communications. This is not an opinion. It is other written communication and was not written to be relied upon, by itself, to avoid any tax penalties. In order to receive assurances of protection from tax penalties from a written communication, you should get an opinion letter. If you would like to discuss an opinion letter relating to any matter, please contact me and I will explain what is involved and what it will cost.
I agree with my colleagues. It seems best that you would register your company in New Jersey. Unless there are compelling tax reasons or other logistical reasons to be in New York, it appears that registering in New Jersey would be a sound option. If there are other facts to consider, it would be worth it to sit down with an attorney and talk through all the facts of your situation to find the best possible solution for your business.