The only way for someone to have your power of attorney is if you gave it to them, in a written document, which requires witnesses and a notary public. Furthermore, power of attorney can be revoked by the grantor at will, by another written document, called a Revocation of Power of Attorney.
If you have any concerns that someone is utilizing a forged power of attorney, you should immediately consult a local attorney to see whether or not your state required that powers of attorney be recorded. If so, there will be a straightforward way to tell, and if so, you can immediately record a revocation. However, if someone is utilizing a forged document, that is fraud, and also a violation of the criminal law if they have used it to do your business or appropriate any of your assets.
Repeat: consult an attorney immediately if there is any possibility that there is someone out there claiming to have a power of attorney for you that you did not give.
If you are legally competent -- which is to say, an adult (age 18 or older) who has not been declared incompetent by a court with jurisdiction over such matters -- then the only person who can create a "power of attorney" to act on your behalf is YOU.
If you mean "power of attorney" in the traditional, legal sense -- a document by which you authorize someone to act on your behalf, as if they were you, either for some specific and limited purpose or perhaps more broadly and generally, depending on the way it's drafted -- then you would have had to have signed the power of attorney document.
If you mean "attorney" in the sense of someone who's licensed to practice law and who has undertaken to represent you as a client (or claims that you're his client), that sort of situation could have come about without you necessarily signing anything. For example, if you consulted with me about an automobile accident, and I were a careless lawyer who was easily confused, and I thought you'd hired me and misremembered whether we'd both signed an engagement letter, then I might file some pleading in court in which I claimed to be your attorney. But I'd be mistaken about my authority, and there would be procedures available for you to correct the any false impression that I had the power to act for you.