The state "license" is obtained when you incorporate with the Secretary of State. You will also need a business license from either the county or city where the business is located, depending on whether you are within the city limits or in unincorporated county.
You should be concerned, but there is no way to tell you whether you are in fact infringing on their trademark, or if they even have a protect-able mark. Such things are very fact-specific. Retain a local intellectual property attorney to discuss and gather all of the facts to decide next steps.
Unless you qualify for an exception, the business will need to register as a foreign entity. However, why be registered in Delaware in the first place if you have no connection there? Why not simply register in Georgia and not have to worry about it?
No, not properly. If the company has any outstanding debts/monetary obligations, it is not proper to dissolve the company. Doing so can lead to personal liability for the individual dissolving the company. However, you should discuss this with the attorney assisting you with the tax issue. For example, if you already have personal liability for the tax obligation, it may not ultimately make a difference.
Since you are just using a trade name, you should use your name as the plaintiff. A DBA is not a separate entity and therefore cannot be a party. However, you can include it: "John Doe doing business as ABC Company."
Names used in commerce automatically get trademark protection (assuming they otherwise qualify as a proper trademark). You do not need to register the name to protect it, although federal registration may make sense at some point. However, you should make sure the name will be a proper trademark before investing resources into it. Retain a local business and intellectual property attorney to help you.
Your situation is not uncommon, but can be very complex if your partner is not willing to voluntarily transfer her interest to you. If she is willing, she would transfer her stock/membership interest to you via a purchase agreement. Then, you would attempt to amend whatever contracts she is a party to. However, keep in mind that the other party to those agreements may not be willing to allow her to be removed.
If she is not willing to give up her interests, the situation becomes too...