1. Open VRChat
  2. Enter an instance with a lot (like 40+) unoptimized avatars, visible and fully enabled
  3. Observe that your FPS is probably something like 10 FPS
  4. Set the 'maximum shown avatars' value to 0, and turn the feature on.
  5. Observe that the room is now full of diamonds instead of avatars, as expected.
  6. Despite this, observe that your FPS is STILL something like 10 FPS!
  7. Leave the instance.
  8. Re-enter the instance that you were just in.
  9. Observe that NOW your frame rate is something like 30 FPS.
The one other thing I've tried that can somewhat trigger this change is to change your shield level; it seems like diamondizing avatars doesn't actually make them stop eating up performance until something forces them to reload completely.