When selecting an avatar from a pedestal then swapping back to the avatar you were in previously without locking in the balls the represent your trackers and controllers do not go away until you complete a full calibration sequence.

Not sure if this applies to other situations of avatar swapping, more testing needed.