You need to it down with a real estate order in your area. The HOA might have a provision that covers it, or they may be arguing the common law theory of "boundary by acquiescence", and "adverse possession."
The information provided in this answer does not create an attorney-client relationship. If you are interested in his legal services, feel free to call Chris at (303) 409-7635 at his law office in the Denver Tech Center. All initial consultations are free of charge.