You should be able to just resign as agent. A letter should do it. If the principal (person making the power) did not pick a successor agent in the POA, the court may have to get involved to appoint a guardian or conservator to act on the person's behalf.
Disclaimer: I am a VA attorney and only licensed in that state. This answer is general in nature and should not be relied on as legal advice. Nothing in this answer creates an attorney-client relationship.
The problem with your inquiry is I do not know from your description whether the person who gave you the Power of Attrorney is competent. If he or she is not competent, a resignation by you may be ineffective, as the resignation is given someone who lack capacity to do anything about it notice that you no longer wish to act.
You should consult with an experienced elder law attorney to discuss what can/should be done.
MARY TOM, ESQ.
HUNZIKER, JONES & SWEENEY, P.A.
Wayne Plaza II
155 Route 46 West
Wayne, NJ 07470
Phone No. (973) 256-0456
Fax No. (973) 256-4784
This message does not constitute legal advice and is for informational purposes only. This message does not establish an attorney-client relationship, which can only be established once a retainer agreement has been fully executed between you and this firm. For legal advice, please contact and retain an attorney of your own choosing.