Probably not. If you are concerned about it, raise the issue to him in writing and request a written response. That may be enough for him to withdraw from the case.Ask a similar question
Conflicts of interest are regulated by the Rules of Professional Conduct. Rule 1.8 is the general rule. It does not appear from your description that would be a conflict, since you said that what you spoke to the lawyer about was a matter unrelated to the suit that you have now brought.
You can read the RPC provisions at: http://www.courts.wa.gov/court_rules/?fa=court_rules.list&group=ga&set=rpcAsk a similar question