The terms of your settlement agreement would govern this matter. Unless specifically specified otherwise, her alimony should terminate upon her cohabitation with her boyfriend pursuant to Pa.C.S. 23 § 3706.
Accordingly, you would have to present this matter to the court to have your payments formally ended. If you had an attorney handle your divorce, you should review this issue with them. Otherwise, you should contact a local attorney to assist you with ending this obligation. Hiring a lawyer will pale in comparison to having to pay alimony for another four years.
A link to the section of the divorce code dealing with alimony is here for your review: http://www.legis.state.pa.us/WU01/LI/LI/CT/HTM/23/00.037..HTM
Sign up to receive a 10-part series of useful information and legal advice about the divorce process.