The timing of the wall is critical since the period for adverse possession is TEN years. It also depends on whether the neighbors have been using their side of the wall "adversely" to your rights. The best thing you could do would be to write a letter to each neighbor and grant them an "easement" temporary in nature, that allows them to continue to use their side of the wall and the small portion of land on their side which is your legal property with your permission. This will end their potential claim of "adverse" possession and avoid their subsequently obtaining legal title to your property through a subsequent legal action for "adverse" possession.
Attorney is Licensed in Arizona, California, and Colorado only. The opinions and comments offered are in the nature of general business advice relating to generic questions that might be raised. The use of this site is not intended to form an attorney client relationship of any kind. The reader is advised that every situation is different and you should always consult in person with a licensed attorney for the particular jurisdiction in question when your legal rights may be effected.
Your situation does not fit within the parameters of adverse possession as your father gave them the property in writing. If there is a block wall separating your from your neighbors, I do not see how you can 'reclaim' the additional property outside the wall. You might want to discuss this with a real estate attorney for more clarification.
Please note that I am answering this question as a service through Avvo but not as your attorney and no attorney-client relationship is established by this posting. An attorney-client relationship can only be established through signing a Fee Agreement and paying the necessary advanced fees.