Probably not. Judges and lawyers participate on committees and in various clubs and organizations. The fact that a another lawyers from the same firm as the defendant;s lawyers sits on the same committee with a judge is not likely a conflict of interest.
Grounds for recusal are usually apparent bias, such as being related to a party or attorney, or having made public statements about one of the parties or attorneys. It's hard to recuse a judge generally because they work to avoid conflicts of interest.
This answer does not create an attorney-client relationship. It is advisable to consult with an attorney with full disclosure of relevant facts for a comprehensive leagl opinion.