Prosecutors will VERY rarely prosecute the crime of adultery although it is still technically a crime. Prosecutors have great discretion in what cases they prosecute and the prevailing attitude is that adultery is not worth prosecuting. Most DA's offices believe that adultery should be addressed by the parties.
You're correct that it is a crime but, if you are unable to get a divorce, I'd recommend simply trying to move on with your life.
If you had him properly served, you can simply serve him by mail at the most recent address with an RJI and Request for Preliminary Conference. File both docs with the court and you'll get a conference with the judge.