Large number of MeshRenderer child objects under synced UdonBehaviour causes lag for non-owners
available in future release
aurycat
If there are a large number of GameObjects with MeshRenderer components in the child hierarchy of a synced (Continuous or Manual) UdonBehaviour, even if the child GameObjects are all disabled/inactive, all players who DON'T own the synced UdonBehaviour will experience poor FPS. The FPS will decrease the more MeshRenderers there are.
Link to reproducer scene unitypackage, helper script to generate gameobjects, and demo video: https://drive.proton.me/urls/JZW3SPS5MR#APbuVgwi0Wy9
Steps to reproduce:
- Create a new empty world project
- Make an empty GameObject called "Example"
- Add a blank UdonBehaviour to Example, and set it to Continuous or Manual sync
- Import the GenerateCubes.cs helper script from the link above to the project and run Tools > Generate Cubes to generate 60,000 default cube child objects. (The script just creates 60,000 cubes without Box Colliders, all set to inactive. You can do it manually too.)
- Build & Test the project with two VRChat instances. Desktop is fine, enable Unlimited fps.
- Observe that whoever the instance master is will get great FPS (around 500 fps on my machine), and whoever is not the master will get much worse FPS (40-70 fps for me). You can have the master rejoin the instance to swap masters, and the new master will instantly have much much better FPS.
Background:
This problem was originally encountered in development of xCirrex's WIP project, which has a randomized hexagon map containing about 60,000 mesh renderers (not all active at once) for all the possible variants of all the tiles. xCirrex found that framerates were horrible (~30 fps for people who would normally get 90+) for everyone except the instance master. Of course, it's uncommon for worlds to have 10s of thousands of Mesh Renderers in a scene under a UdonBehaviour, but it still could be a source of performance loss / a point of optimization, to fix this for worlds that have fewer, but still many, Mesh Renderers.
Extra info:
- The issue happens on live build 1407 and the current open beta 1433. I was using SDK 3.5.1 .
- The problem is not limited to just the instance master, but whoever is the owner of the object.
- The 60,000 number is arbitrary, but it's around how many MeshRenderer objects xCirrex has in their project map.
- Setting the cube GameObjects to inactive is not necessary for the bug, but it just removes the rendering performance cost from the equation. In xCirrex's world, the MeshRenderers are spread out over many hundreds of meters, and the main camera is set to a far clipping plane of 100m, so the rendering cost isn't a major factor. Deactivating the cubes also shows that they don't need to be active for the issue to occur, which is interesting.
- I've only tested with a Graph UdonBehaviour, but it probably happens with a U# behaviour too.
- I also noticed a related issue where in ClientSim, and possibly also in the actual game, if an interaction is added to the UdonBehaviour on Example, looking at the collider to click on it will result in massive lag. In ClientSim it takes me down to only a few FPS! In the actual game it was much less significant, but still appeared to happen. For ClientSim, looking at the profiler, it's clear that all the lag is coming from the UdonBehaviour searching for child objects with colliders or mesh renderers to show the highlight on. This issue is probably related to the main issue somehow -- it seems like something is searching the hierarchy for mesh renderers every frame, but only for non-owners.
- In xCirrex's original project, there was a very strange thing, that turning ON a significant percent of the mesh renderers would cause the lag to stop. I was not able to reproduce that locally. In my reproducer, the lag seemed to be unchanged whether the objects were on or off, as long as they're not actually being rendered (e.g. getting frustum culled).
Log In
This post was marked as
available in future release
_
_tau_
Thanks for the report, would it be possible to provide a world ID where this issue can be reproduced? We can try getting the package you shared to work too, but if you have a world that you know experiences the performance drop it's easier to work from there.
aurycat
_tau_ Sure thing!
Here's the demo world:
And there's an interaction box in the middle to click which transfers ownership between players so you can easily see the framerate drop whenever you lose ownership.
Thanks!
_
_tau_
aurycat: We've reproduced the issue in that world, and have some optimizations we can ship eventually. However, there will still be a noticable performance overhead. Some context:
This is happening because of a feature that prioritizes network events from visible objects. As such, syncing components such as Udon Behaviours with
SyncMode != None
or VRCObjectSync
components will check all child renderers for if they are visible. This itself is something we cannot remove, even if we can optimize it a bit more. Frankly, having tens of thousands of Renderer components underneath a synced object is simply not a supported scenario.As a suggestion, some alternatives that may work in your case and would not experience this issue, or limit it:
- Don't have these child objects exist in the scene at all time, use Instantiate to spawn them in on-demand instead. Either from a base prefab, or I suppose you could have 60k prefabs and reference them, but that might run into its own kind of trouble.
- Have the large hierarchy live outside of any synced objects. I.e. put a an empty GameObject directly in the scene root, have the 60k objects underneath, and reference them not from a parent Udon Behaviour but from a sibling.
- Split the hierarchy into multiple groups of, say, 60x1000 objects, and disable the root GameObject with the syncing behaviour when none of the children are active. Disabled UdonBehaviours will not scan the hierarchy for Renderers. Might run into sync issues with a disabled UdonBehaviour though, we recommend keeping syncing behaviours active at all times. To mitigate that part, you could sync a sibling object and send data into the group root behaviours via events or similar.
Hope that helps!
StormRel
tracked