Blaming whoever sweeped the road is ridiculous IMO, hell, if you cant sue the state for dirt, sue them because the roads blend in with the dirt...
You're riding a road at 9-11,000ft that gets snow, rockslides, god knows what else in all months of the year. They probably never actually sweep the road since it is closed through all of winter, and opens again when they get it plowed and the snow stops falling in heaps (usually late May). Probably just not enough traffic to wash the passing lane as clean as the other lanes. Probably not even sand from winter but blew in the day before... things get windy up there.
Expect the unexpected, especially on a mountain pass. I'm not trying to be a dick here, it could happen to anyone and its not like either of you were doing something unsafe in and of itself, but there is really nobody else to blame but the guy who hit you...