First - you need to seek the advice of a land use attorney in your area licensed to practice law in your state who can advocate for you at the local level. This type of situation involves local zoning law (and politics), possibly state law, and at least an understanding of FAA noise regulations as to how they may or may not apply.
There have been cases around the country where local governments, which own and operate public airports have been successfully sued under the theory of inverse condemnation where low flying aircraft over private property cause noise, vibration, and other typical nuisance-type problems. Inverse condemnation occurs when there is a 'taking' of private property in violation of the 5th Am. of the U.S. Constitution - where the government has 'taken' private property without just compensation. The process of Eminent Domain is the legal process where the government lawfully takes private property for a public purpose and then compensates the private property owner. This can be a very complex area of law.
However, your situation sounds different because the airport is a private airport. There may be zoning laws that apply anyway and these zoning laws may or may not be the cause of a regulatory taking. However, because of the way you have explained your facts, it sounds as if you are in a situation where your property rights might be affected by another private land owner's acts. In addition, it's possible you may need to seek legal action against both the local government and the private owner of the airstrip - but you need an attorney to examine your situation to know for sure.
Here is a helpful website that focuses on typical issues between neighboring property owners and airports (public and private). This website focuses on Federal law and some state law. This site appears to have accurate legal citations, but we have not checked to be certain. The site can at least inform you of generalized issues you may be dealing with:
It's imposible to give you specific advice here because of the complex nature of your case and the need to know more details. Consult a land use attorney in your area. Good luck!