I seriously doubt non riders give it any thought at all.
Jealousy about cutting line? I doubt it. Other than perhaps some momentary indignation, thinking "what a dick" when they see it done illegally, same as we all do when we see someone do something illegal, or the few dudly-do-rights playing traffic cop, the vast majority of folks probably couldn't care less one way or the other.
There are just too few of us to force the issue, our best chance would be to play the "green" card, that bikes create less pollution, use less resources, and don't create traffic issues.
Green sells, and par for the course, doesn't need any real substance, or have any real benefit other than fulfilling our desires.
"Take care, sir," cried Sancho. "Those over there are not giants but windmills".