The sport package on the Z is quite a bit more legit than sport packages on some other cars (the Corolla 'S' model comes to mind)
Most significant upgrades are the wheels/tires, which on a car known for its handling make a significant improvement. The 18" base Einkei's are light enough but narrow compared to the 19" Rays in the sport package, which are also light for OEM wheels. With the wider wheels in the Sport package your tire width goes up 20mm front, 30mm rear, a significant difference. Brakes also improve, the BBK on the sport package is a big upgrade. Finally, aerodynamic improvement comes from a front chin spoiler and a rear hatch spoiler. The rear hatch spoiler is the main reason I opted against the sport package on my car, but I've been working on some bits and pieces upgrades to my base. One thing I can't upgrade to is the SRM, if you're going manual you should test drive the sport with the SRM feature before making up your mind. I didn't, and that one feature alone may have changed my mind. Oh yeah, LSD too.
In short, if you're even the least inclined to get the sport, the Z will not disappoint. You get a lot for the $3k price of admission.
Edit: Triple's beat me too it. 