Boundary disputes are complex and difficult. From what you have posted, you will not be able to get a satisfactory answer to your problem in an online forum. You should consult an experienced real estate lawyer in your area about the situation. You are probably going to have to litigate to get a satisfactory solution.
Disclaimer: This answer is provided for informational purposes only, does not constitute legal advice, and does not create an attorney-client relationship. Actual legal advice can only be provided after completing a comprehensive consultation in which all of the relevant facts are discussed and reviewed.
You will need local real estate counsel to help you sort this out, but here are some basic factors. First, what does your deed say. If the deed actually conveys everything then it is likely yours. Traditionally under the merger rule the deed controls after the sale is closed, so if the deed conveyed all to you you should be on reasonably solid ground. I assume the sale agreement specified what you were buying as well.
Beyond that there is a doctrine known as "estoppel" in which if one person misleads another and the other relies on the misleading information, the first person cannot assert his claim. Since Jerry has continually led you to believe that all was yours and you invested time and money into erecting another structure, he may be estopped from making his claim now.
You need a lawyer and need one quickly so that no harm is done that cannot be undone. Without knowing more, one cannot be certain of the outcome. However, this is not really a boundary dispute alone--there are much bigger issues at play here.