In the end it is your skin...I don't push the ATGATT agenda on anyone. But before you do it, consider that hitting the pavement at 20mph is going to result in NASTY scrapes...at a minimum. Put it this way: would you want to take your chosen shorts/t-shirt/boat shoes, get up a good hard run, and then dive onto the asphalt? If the answer is no, you should probably pick better clothing.
I tend to cheat on ATGATT with the short rides, but I never ride in shorts and a t-shirt. I have had too many wipe-outs even at slow speed on gravel to want the kind of road rash that would result...and I already do know the road rash that results, having been there. Picking stones and gravel out of your arm is no fun. So basically, for me, cheating means jeans and lighter boots still covering the ankle, and a decent quality jacket (like a carhartt). Never helmet-less.