Taking trunk panels off in not a big deal, I did it several times on multiple MYs. You just have to keep an eye on those small white clips on side panels as they tend to fall off and hide in the most difficult nook possible.
But I never had rattles after putting them back.