I was able to get the handle part separated from the assembly. I drilled the heads of some of the rivets. Then i had a local machine shop drill and tap end end so I could re-assemble.
Check out the link to some pictures below.
Nissan 370Z Forum - HockeyAnimal1's Album: Door Handle Project
I thought it was worth the trouble to make sure the paint was even.