I've been trying to make an avatar prefab that has buttons which fallback users can press, but since they don't have contacts, and cannot tell the wearer of the avatar that the button has been pressed, there is desync.
Of course, with remote sync, you'd need a Local Only contact (for everyone, not just wearer) to avoid network spam, and probably a rate limit on the parameter driver component that would send the parameter.
I hide nearly all VeryPoor avatars, so making a prefab that's reliant on showing them feels like a stab in the gut. xD
Here's a video example of the current limitations: