The Building Inspector determines the interpretation of the zoning ordinance. The language could mean the height of the bottom of the sign, the middle of the sign, or the top of the sign. Very likely, the inspector is using the most restrictive interpretation, the top of the sign. If you don't like his interpretation, you can appeal to the ZBA for a area variance or an interpretation. The "hardship" would mean some kind of "practical difficulty" you would have with complying with the Building Inspector's interpretation, such as (in your instance), unless the sign was higher, it would not be as visible in a highway corridor with higher pre-existing signs, or visual clutter, or relatively high speed limits, or the particular location you need to place the sign in and it's setback from the road, etc. A photo rendering of your proposed sign placement and height and showing it would be consistent with the neighborhood and not a visual blight if a variance were granted would usually be helpful to your case.
In other words, the "hardship" is SOME GOOD, LOGICAL REASON, why you you cannot comply with the ordinance, other than you don't like the ordinance, don't feel like complying, or don't think the gubmint should interfere with your use of your land as you please.
This answer is provided under the Avvo.com “Terms and Conditions of Use” (“ToU”), particularly ¶9 which states that any information provided is not intended as legal advice or to create an attorney-client relationship between you and me or any other attorney. Such information is intended for general informational purposes and should be used only as a starting point for addressing your legal issues. In particular, my answers and those of other attorneys on Avvo.com are not a substitute for an in-person or telephone consultation with an attorney licensed to practice in your jurisdiction about your specific legal issue. I can only answer your question based on the brief statement of facts you present in your question which may omit other relevant facts which would be disclosed in an in-person interview. While I attempt to provide reasonable care in answering your questions in an online forum which encourages attorneys to provide brief "off the cuff" responses within several hours of the posting of a question, I cannot perform any further detailed legal research concerning possibly relevant statutory or case law which might apply to your situation that I would normally do in the course of my representation of clients where an attorney-client relationship exists (and you are paying for my services!). You should therefore not rely solely upon legal information you obtain from this website or other resources which may be linked to an answer for informational purposes. You understand that questions and answers or other postings to the Site are not confidential and are not subject to attorney-client privilege. The full Avvo ToU are set forth at http://www.avvo.com/support/terms . In addition, while similar legal principles often apply in many states, I am only licensed to practice in the State of New York and Federal Courts. Any general information I provide about non-New York laws should be checked with an attorney licensed to practice in your State. Lastly, New York State Court rules (22 NYCRR Part 1200, Rule 7.1, available online at http://jackle.bo/_prof ) also require me to inform you that my answers and attorney profile posted on the Avvo.com site may be considered "attorney advertising" and that "prior results do not guarantee a similar outcome".
The building inspector generally decides the interpretation of the zoning ordinance.
I see you are from East Troy. It may be worth a few minutes of your time to call a local attorney to see if they have some advice. Oftentimes, attorneys and zoning administrators know each other and it may be helpful for you to find a lawyer who can assist you and deal with the zoning issue. The lawyer will know how to present your claim in a light most favorable to you which is more likely to result in the variance.
If you believe this is the best answer, please indicate. If you agree with my answer, please click on agree. No attorney client relationship has been established by my answering your question.