Where I live the paved roads can have sand and gravel across the intersections all year long. There's a tight banked turn very near here with gravel along the tire tracks and rain will always add sand/gravel stripes across the lanes. Riders here have to get used to it.
It kinda sucks that you live where gravel on pavement is so rare that you want to avoid it. Going out and practicing on iffy surfaces can make all riding more enjoyable.
"The belief that there is only one truth, and that oneself is in possession of it, is the root of all evil in the world."-- Max Born, Nobel Physicist