It is not possible fir to say for sure whether or not you have a claim. It is possible that you have claims for intentional interference with contractual relationships and other business torts, but it is far from clear. It could be that fair competition is a defense to some or all of your claims, but again, it is not possible to tell based on this outline of the facts.
I strongly suggest you start contacting business lawsuit and litigation lawyers about your potential claims here. I have litigated these types of cases, and can tell you that (1) you need an exceptionally strong relationship with your lawyer to make this type of case; and (2) this type of fight can be quite bitter.
If there was a noncompete clause in the contract your ex colleague had with you or your brokerage, you may seek to enforce it. However, I suspect that if such a clause existed in the contract you would have raised it in your question. Additionally, you may have causes of action for misappropriation of trade secret, unfair competition, intentional interference with contract or intentional interference with economic advantage. Give me a call and I would be happy to discuss with you at no charge. My direct line is 949-954-0346. I will be available after 10 AM today.