I've had a number of collectable cars, including 4 first series Mustangs.
As long as you understand it's a toy, a hobby, etc. rather than an everyday driver you'll be fine. It will take some effort to keep it on the road, but if you're OK with that, it's a lot of fun to have something different. Buying a newer Mustang is not the equivalent.
Now, specifically to the Mustang: before you even look at the exterior/interior, get down on your back and do a complete lap around the underside of the car. Bring a small screwdriver and poke, everywhere, looking for rust, extra heavy undercoating covering rust, and Bondo, you guessed it, filling in rust. Also check the front and rear fender wells and shock towers. Lift the carpet and look for rust. If you haven't got the message yet, these cars rust. When they were first producing them, there was a buying frenzy to the point that the factories were shipping them out, as fast as possible, without even spraying the inside door metal. Yes, bare metal was being sold! They sold almost 1 1/2 million of them in 3 years, a record not yet equaled. This is what made Lee Iacocca.
There are sheet metal boxes under the car that add strength, plus the rockers help support the car. The car is unibody, so no separate frame, the floor pan and related sheet metal parts are the frame, so be very, very careful. This rust can be repaired, but it's pretty expensive work.
There are many vendors selling repro parts, probably almost enough to assemble a new car. So getting parts, patch panels, floors, etc. is not a problem except for cost.
There are several mods to increase the handling ability to the level of the Shelby's, including lowering the suspension points, better shocks and springs, poly bushings, better brakes, etc., but it's still going to handle, and stop, like a '60's car. Just be sure you know what you're getting into.
The early fastbacks are arguably the best looking model, and currently the most expensive. Good luck with your toy!
Oh I used to be disgusted, but now I try to be amused-E.Costello
'01 Kawasaki W650, '05 Yamaha XT225