You might also have a misbalance in your front wheel. BestRest has a product (i think they're called DynaBeads) that go inside your tire and go where they are needed when rolling. The misbalance will be more distinct with the luggage on the back.
But also per a previous inmates comments, check your head bearing. Without it having it properly checked you would probably feel a clunk over big bumps if it was loose. Quite common from the factory.
I've had the bike comfortably at 140kmh with full luggage and passenger... and even at 160kmh on gravel with full luggage (no passenger).