Use the following world to test. I advise using three clients, ideally with three remote players, and use the two non-instance owners to do the test.
To reproduce:
  1. Find the two buttons in the world that spawn pickups: a sword or a gun. You can use these to spawn up to about 100 pickups.
  2. As a remote player, not the instance owner, spawn a pickup. Pick it up and hold it.
  3. Have another non-owner take the pickup out of your hand.
Expected Behaviour:
The underlying VRCObjectSync should sync about as well when commanded by the new owner.
Actual Behaviour:
The ObjectSync is heavily lagged. The effect is that the pickup looks like it's flying behind the player like a drone, rather than being held in their hand.
Additional Notes:
This appears to not occur if you drop the pickup, and then your friend picks it up from the floor.