I wanted to update everyone on my experiences since I began this post.
After trying every possible type of drive and format option, I had Tesla come and replace my USB module because they couldn't figure it out either over the phone. The new module was a different (and newer, obviously) chipset, so I had high hopes with that swap, but... the problem remained. No matter what kind of device or formatting method, the car would re-scan every time.
So I went back to basics- I tried a small USB drive with just a couple of thousand songs. That worked fine, no problem. Then I tried one with more... and more. But even with just a 32 gig drive, it started to sometimes have to re-scan when I entered the car. It seems obvious to me that the issue is simply: the technology isn't built to handle more than a few thousand files. Once you go beyond that, it can't keep the indexing stored on its tiny flash memory (or wherever it's storing this info.) I wish it wrote some kind of index file to the drive itself that it could reference every time you turn the car on, but it doesn't.
So my solution, after much digging, was to purchase this
FIIO M6 mp3 player, which allows up to a 2TB microSD card (they currently only make 1TB) and runs on a modified Andriod OS. So you connect to the car via Bluetooth and play the music that way. It's a little quirky (being from China, it's hard to get good support documentation) but it works well. One really annoying negative is that there's no way I have found to import a playlist- you can find hack-ish tutorials on how to do it, but it hasn't worked for me. Which means I have to make the playlists manually on the device itself, and that's a pain when you have 40,000 files... but it works well otherwise and is praised as having incredibly good high-resolution bluetooth audio. At $150, that was the best method I could come up with to play my music.
Of course, one really annoying thing is that Tesla won't allow album covers to transport over bluetooth, so all my hard work tagging these obscure albums goes out the window, though the M6 itself does show them on screen- so I mount it on the phone charge port (USB-C) and let it play that way. It does have a battery that lasts several hours (I haven't quite tested that) but if you leave it plugged in, then obviously you're not even tapping into that. Now I can shuffle through 40K (and growing) songs and have my own custom radio station with no ads or crappy algorithm playlists from Spotify.
In case that helps anyone!