It really depends on the kid. My oldest was terrified at 11 and didn't ride much with me growing up, he was happier on his own dirtbike.
My youngest boy loved riding at 8, he'll do anything to this day..jump off bridges, bungee jump and stuff, great passenger on a bike.
When my daughter came along I tried her on the back seat at 4, but she was too scattered and couldn't pay attention, stuck her in the sidecar with something to do and she was good at that..she got good at riding on the back around 11 or so.
Be sure the safest rule is that we should not dare to live in any scene in which we dare not die. ~Lewis Carroll~