You know, I ask myself that same question every day
I think a lot of it has to do with the fact that my car very rarely sees heavy load on the clutch or trans - I'm usually driving around in town, and in 1st/2nd gear, the only gears where I can really stretch my legs, the tires just spin. The real load isn't put on unless I'm on the highway in 3rd+, and I don't do that too often.
That being said, my car is a ticking timebomb. The clutch will give out eventually. If it lasts another 5k miles, I'll be happy. The tranny is slowing dying on me. I have zero issues with it yet. but it's an inevitable situation. So yes, you're right, I would have more fun in the Z. I've pretty much determined that I'm going to try to find a used 370 w/ sports/touring package if possible, and then pick up a cheaper subie impreza for the winter months.