Your status as an agent per the POA does not let you do as you wish with the assets, even if you think it's for his own good. He is necessarily always the beneficiary of your actions as an agent, and you cannot intervene unless he wishes you to, and for that he does not need you. Also, keep in mind he can revoke your authority at any time.
If you found this "helpful" or "best answer," please click it with my appreciation. My response is for educational purposes and does not constitute legal advice nor creates an attorney client relationship which requires all the details and a personal conference.