Some states still recognize an action called "alienation of affection". Briefly, the "injured" spouse can sue the person with whom the other spouse has an affair. The states that allow the action likely has elements that must be met for a successful action.
It is dependent where the parties are divorced. Colorado is a no fault state, so you cannot be sued for alienation of affection; however, if the parties are divorced in another state, regardless of where they were married, the third- party may be liable.