One way to look at it is...
With the 650, you can more easily ride the bike to the trail. With a new 250/450 dirt bike, you'll be hauling it in a pickup or on a trailer or? Add to the cost the transport.
Personally I chose to outfit my 650 to be truly trail ready and put the extra money into gas and forest passes.
"Yes. It looks insubordinate, but it isn't really." - T.E. Lawrence