You have two options. You could send neighbor a letter telling them that you are aware of the fence crossing the property line, that you consent to the fence being there for now, but that you do not intend the consent to create an easement of any sort, and that you reserve to revoke the consent at any time. If you do this, you'll have to deal with the weed problem between the fences, but you won't have to worry about adverse possession because your neighbor's possession would not meet the "hostile" requirement to gain title by adverse possession. If you want the fence off your property now, you should consult an attorney to take legal action to eject the trespassing neighbor from your property. If you make it known that you object to the presence of the fence, but then do not follow through with ejecting (i.e. forcing neighbor to remove fence) neighbor, then over time neighbor might gain title to that portion of the property by virtue of adverse possession. If you want the fence off your property right away, consult an attorney. Good luck!