The first thing you need to do is find out if they have any other legal basis to locate the driveway where it is, such as an easement. Given that the builder put it in, it is more likely they may have an easement. You likely obtained title insurance when you purchased the property. Your title insurance policy should tell you if there is an easement. On the other hand, if there was one, I would think your neighbor would give that as the reason for keeping it there.
The doctrine of adverse possession in Washington gives someone title to part of someone else's property if certain criteria are met and continue to exist for a period of 10 years (or 7 years if the adverse possessor is also paying the tax on that property, which does not sound like the case). Either way, if the adverse possession has not lasted more than 10 years, they have not gained any rights to the property, and you can require them to remove the driveway and stop using your property. The 10 years would include use by anyone who owned their property before them.
You cannot stop the clock on the 10 years -- and their future adverse possession claim -- by publishing a notice or now telling them it is OK. In other words, you cannot foist permission on them after they have started adversely possessing the property. If you tell them to remove the driveway and get off of your property and they don't, then you have to take some type of legal action, either filing a suit or getting them to relinquish any claim (typically by quitclaiming any interest in your property). You should not try to do any of this yourself. You will need an experienced real estate lawyer.
PLEASE BE ADVISED: This answer and any information contained herein is not intended to be treated, and should not be construed, as legal advice. Rather, this answer is offered solely for general information purposes. This answer does not create an attorney-client relationship, nor does it create any kind of legal relationship, duty, or privilege. This attorney is licensed only in Washington.
Since you gave us so much information I pulled up your property on the online county records and took a look at the county aerial photo and the plat map. The small portion of your property which may have a portion of your neighbors driveway on it really shouldn't be a problem...at worst, only that portion which goes only to his home could ever possibly be adversely possessed since you use the rest of the driveway - thus, he does not have exclusive use and could not gain ownership of any portion of your driveway. If you are concerned about whatever small piece of his driveway is not on his property or the road right-of-way, but is on your property, you could give him an easement over that area - which by itself would not stop any future adverse possession, but would make it clear what was going on. If you really wanted to totally clean it up you could do a maintanence agreement with him which you both would sign...which would pertty much guarantee that his use was permissive. But, all in all, unless there is some major issue between you two this really should never become an issue - he would need to spend thousands of dollars on a quiet title claim to gain legal title to that small area. He would have a very weak case in this situation.