The property was purchased by you subject to it being in an association. Your neighbors bought with that same expectation. To have certain units in and certain units out would defeat the obligations and benefits expected by all. So, while there are ways to eliminate an association, it usually involves getting unanimous approval. As a practical matter, it is unlikely. You would need to review the Declaration in relation to the Condominium Property Act to determine the exact requirements and consider your rights.
Concerning the regular and special assessments, your attorney and real estate agent at closing should have reviewed the association's financial circumstances with you, including whether appropriate reserves were on hand for expected expenditures. It appears from your question that the condominium board is finding that the monthly fees collected over the years are now inadequate to meet the obligations of the association to the members to see to the upkeep of the property.
If you are not satisfied, you should attend the board meetings and participate. You can also seek election to the board, but once there you may feel compelled in exercising your duty on behalf of the association to vote very similarly to the current board members.
You property value is best preserved by the association being managed effectively, which may mean higher assessments in the future to establish the stability and reserves that should have been there in the first place.