I can't think of an application that will write its data to a random file system that may or may not be there
No application ever needs to write data to a random filesystem. There is one singularly constant location true to every situation with no permissions issues or any of the other hiccups associated with a non-portable install. The directory where the .exe is. It's always there.
All user settings should be under that for a portable install. Like Arduino and it's portable folder.
Be aware I started up fritzing for a basic schematic and the parts I wanted to use weren't in it. I started to create them and realised they weren't saved with the application which means that when I pulled my key and stuck it in one of my other machines all my custom parts were left behind. And good quality USB3 keys are fast.