You can ask him to demonstrate how you money was depleted. It could have been lost in a market downtown, or it could have been improperly invested and churned. You have the right to arbitrate to demonstrate that he handled your money improperly.
Generally, ordinary market losses are the basis for a claim against a broker or a broker/dealer. If those losses were occasioned as the result of bad advice from the broker, however, in some (emphasize some) circumstances it may be actionable. Were you getting monthly statements from the broker? What did they show you? Did you ever inquire about what he was doing? And if he was losing your money, why did you follow him to AIG after he left Prudential? Do you suspect that he stole or converted your money?
These are among the questions you'll have to answer at an arbitration hearing and to whomever you hire as counsel to examine the matter further. First things first, though -- check out the broker's record on the FINRA website:
If I can be of further assistance, let me know.
The foregoing is not legal advice nor is it in any manner whatsoever meant to create or impute an attorney/client relationship.