Meet with an elder care lawyer to discuss potential to bring a petition before the court having jurisdiction over the guardianship. If there is a more appropriate guardian the a removal petition could be instituted to replace e current guardian.
My answer is not intended to be giving legal advice and this topic can be a complex area where the advice of a licensed attorney in your State should be obtained. Please click "helpful" or "best answer" if my answer added any value or add a "comment" if you have more info for me to help you get a better answer.
I agree that it may be necessary to substitute guardians under these facts. You can certainly contact me further for further assistance.