I have ridden and drove most of the passes in that area. While skill will play a big part, your comfort level should be the biggest indicator of whether or not to proceed. The highest pucker factor I had was on the Yankee Boy Basin road headed towards Ouray when my rear brakes failed. Riding down those narrow roads with only the front brake was not fun.
But the views were amazing