As my mother used to say, you should always wear clean underwear in case you are in an accident and have to go to the hospital. Also, some states require you to wear some sort of footwear while driving, presumably to have a better grip on the brake pedal. And, seriously, you don't need your genitals to be more easily pierced by glass shards or metal, or smacked by the airbag if you crash.
If a policeman or prudish truck driver can see that you are naked from where they are situated, you could easily be arrested for public exposure. If you are giving or receiving oral sex while driving, it would likely be considered reckless, even if you're wearing a condom. While prohibiting oral sex is generally no longer considered constitutional, the statute still is on the books to be used in conjunction with other charges (such as with an underage person). There is probably no constitutional right to sex of any type while driving. But I digress... You could also burn your buttocks if the seat is too hot when you get in. Your seat belt might suddenly retract across your privates while you are driving if not latched properly, causing you to lose control (of the car). So, for a variety of reasons, both legal and practical, I strongly urge you to keep your pants on.