If a hat has orange and blue colors with the word FLORIDA on it, am I infringing on their trademark?

The name of the state, Florida, is free for anyone to use. But the school you're referring to probaby has a trademark for it's name, probably including the class for clothing and hats, and perhaps it's mark includes its school colors, which can spill over into their "trade dress" rights. That's why you want to use those particular colors, right? To make customers think your hats are the school's products?

Without looking at the school's mark rights and your proposed hat, it's hard to say, but it sure sounds like you're trying to confuse the school's customers with your hats, which sounds like trade dress infringement. You're best off consulting an IP lawyer to fully disclose the details of your situation and get some specific advice.

Disclaimer: Please note that this answer does not constitute legal advice, and should not be relied on, since each state has different laws, each situation is fact specific, and it is impossible to evaluate a legal problem without a comprehensive consultation and review of all the facts and documents at issue. This answer does not create an attorney-client relationship.

I agree with my colleague. You certainly would be risking trade dress litigation and in the case of Florida, there are a number of trademarks related to Florida and clothing (owned by a not-for profit established to own and license such rights).

It is typically easier to work with an existing licensee rather than trying to get a new license on your won for your hats. Most universities have a trademark licensing program or an official licensing representative who can supply you with a list of the licensee who produces clothing/hats and/or provide you with an application for a new license.

Using your example of Florida: Florida has a Trademark Licensing Program and the contact person is:
Ms. Debbie Gay, Licensing Manager
Licensing Department, Athletic Association
P.O. Box 14485
Gainesville, FL 32604-2485
Phone:(352) 375-4683
Fax:(352) 374-7065

Alternatively, the "official" licensing representative for Florida is:
The Collegiate Licensing Company
290 Interstate North
Suite 200
Atlanta, Georgia 30339
Phone: (770) 956-0520
Fax: (770) 955-4491

This answer is for informational purposes only and is not intended to provide legal advice or create an attorney-client relationship. Please consult with a local IP attorney who is experienced with contract issues for legal advice that is tailored to your particular circumstances and facts. Good luck to you.