Boils down to personal preference, riding style, degree of laziness.
I beat up the bike with and without, currently they are on.
Taking hits without bars on the fuel pump side has caused leaks at the pump mounting point. I'll also found the tanks to be more durable than you would think.
Do the Geek's crash bar mounting mod
if you are sick of struggling with taking them on and off.