Each state's law is different and I don't know what Virginia provides. Generally speaking I believe most states impose a duty of loyalty on the operators of businesses to not start competing companies while working for a company. This is why people will resign from a company prior to actually competing with that company by starting a new company. Under this scenario, you could possibly resign and then start the competing business.
In some states, this duty may extend to the owners of small companies as well. You should check Virginia law. Your best bet is to hire a VA business or employment attorney.
Other potential solutions: Not sure what your operating agreement for your LLC provides, is it possible for you to call an owners meeting and resolve to just close down the company? Pay off all creditors, properly split the remaining assets and dissolve the entity with the state? Then there is no potential conflict with the company as it is no longer in business. Your operating agreement should provide guidance. If no operating agreement, then need to look at state law. If this is manager managed and you are the sole manager - you may have the ability to decide more than you think.
If you are in a deadlocked situation where nothing can happen because your "partner" has decision making power under your company organizational documents and applicable state law, check the state laws regarding deadlock and see what your remedies are - they are usually spelled out in the statutes. May require going to court.
Also, for this size business what are the odds that if you pay off creditors and pay each of you 1/2 of the remaining assets (assuming that's the proper distribution - can't tell based on facts) and tell your "partner" that you quit, that your "partner" will actually sue you in the new venture? Of course, if you make it big, the odds go up and that's why its better to properly dissolve the entity and at the least resign all positions.
If you can't resolve it by quitting the business yourself and having him agree to wind it up, then you are probably best served to hire an attorney to represent you - especially since he already has an attorney representing him.
The first and most important thing for you to do is to refer to the terms of your Operating Agreement. If it was drafted properly, the Operating Agreement for your LLC should set forth any rights (or limitations) you may have with respect to the issues you are having with your partner.
In addition to the Operating Agreement, you should also refer to the terms of any other agreements or contracts between you and your partner as such contracts may set forth guidelines for what you can and cannot do. (One such agreement that you and your partner may have is a Buy-Sell Agreement.)
Assuming you don't have an Operating Agreement (or assuming that your Operating Agreement does not address the issues you are having), then you have two real options to consider:
1) You can retain a lawyer to negotiate with your partner's attorney in furtherance of achieving an amicable, realistic resolution.
2) Under the Virginia Limited Liability Act, you can file an action for dissolution in Court and request that a judge dissolve your business. The dissolution process will be overseen by the Court and any outstanding liabilities or profits will be divided accordingly.
Based on the facts that you have provided, I would recommend that you obtain an attorney to explore a settlement (Option # 1). If a settlement cannot be achieved, then you should proceed to Option # 2 and seek a judicial resolution.
I wouldn't recommend unilaterally stopping operations of your current business in order to start a new, similar business. To do so, without your partner's agreement, may result in legal liability. You (as is true with your partner) have a duty to act loyally and in good faith with respect to the operation of your business. To form a new, similar business without properly dissolving and winding down your current business may very well constitute a breach of your duty (for which your partner may seek damages).
I hope this helps and I wish you the best of luck!