Massive hitching, freezing and performance issues since a couple updates.
needs more information
Desmoulins
Roughly since a month (maybe a bit longer, shortly before the 3p open beta build was introduced i think) we saw a massive decline in performance stability.
Not the average performance (at least not that i can tell) is affected here.
Anytime something happens, random things, onscreen effects from games, toggles, switches, anything really, there doesn't seem to be any pattern behind it (that i could notice) VRChat will completely freeze, not just hitch for a moment but completely lock up entirely, up to several seconds sometimes. I can see in the Steam Overlay that Steam (and everything else) continues to work just fine with no hitches at all. I cannot determine why exactly it has been happening though i noticed that turning down (to Errors Only) or entirely off Debugging does help a bit, it does not fix the constant freezes however.
Several many people i know have reported this degrade in performance so i'm confident it isn't just me.
I've tested the live build (since a month), the open beta and the third person beta. All of them have this issue and it makes playing quite frankly almost impossible.
Things that can cause these massive freezes:
- Turning off lights (Blackout, Murder, Prison Escape, Among Us)
- Some avatar effects (Scaling bones + playing a short sound + toggling physbones)
- Practically any new onscreen effect (text or effects popping up on screen, fade-to-backs when dying)
- Turning around for a bit and then turning back to face the same avatars
- Opening any menu that loads a couple images (Groups, Friendlist and several others)
- Spawning some very light items in some worlds (Pens, Objects, Furniture)
- The usual (Avatars loading in, lights toggling etc) are amplified (although weirdly enough not as bad as everything else)
I would have suspected an issue with logging (though when turning logging off it still happens) and/or the way the strings are gathered and printed but that seemed only part of the issue. Game lobbies are nigh unplayable with the constant freezes.
It's not an issue with firewall, defender or antivirus (tested all of them, turned all of them off). The effect is similar as to when you've been logged out of the API while still being online (API calls causing massive stalls of up to 10 seconds).
I honestly don't know what to point to other than it wasn't happening ~2 months ago. Its happening everywhere with as little as a single other player in the instance, its hard to miss it. Hardware has not changed, drivers and software has not changed, its repeatedly reported by many people with hardware reaching into the high-end bracket.
My only wild guess i have at this point is that the avatar optimization (single pass shaders) is getting reapplied aggressively after a short while. Usually when something requires shaders to switch to two passes (thus undoing the optimization), such as being lit by a realtime light, it stays unoptimized until (presumably) being reloaded (or possibly only after switching/rejoining worlds). My guess is that the switch back to single pass is being reapplied very aggressively here, which (if the previously compiled single pass or two pass shaders aren't cached) would cause a massive freeze as shaders are getting recompiled live. This would explain most if not all the cases in which these freezes seem to happen.
Log In
Desmoulins
Adding to the above according to some tests that were conducted.
This does not seem to happen in worlds with realtime lighting and shadows enabled.
It does happen in worlds without realtime lighting and shadows.
It can be reproduced reliably and instantly by reloading/rejoining the world or by fully reloading (reuploading a new version which causes a re-download) an avatar that can trigger this.
It does not seem to be sound related, all sounds were manually checked and properly optimized to not cause a hitch on load. It did not change anything.
It seems to be a compounding issue, it is unknown what exactly due to the complexity the avatar had that was tested with but removing absolutely everything except the one feature that caused the freeze did solve the freeze. This makes it look like an issue that only appears in combination with something else or is a faulty interaction with other components or possibly invalid components (null references).
It is not avatar specific, can be caused by any avatar, effect or world regardless of complexity. Many highly complex avatars and worlds do not cause this, some very simple ones will further putting the focus on a potential component interaction issue.
The past few months a lot of things regarding physbones, constraints and colliders has changed, particularly their order of execution. It could be related to that, in the reworks something might have been overlooked that causes a certain interaction to hang.. though it is weird that it only happens once and then only after a few minutes again.
I believe it might be an overlay aggressive de- and re-optimization process. As far as i'm aware single-pass shaders are live-switched and recompiled to a two-pass shader if necessary (when touched by realtime important lights or anything that needs depth for instance). This de-optimization was previously "permanent" until rejoining the world, it could be that it switches back (or recompiles) the single-pass shader, which could cause a hitch if these shaders are not cached. If this is true, once in a while it would cause seemingly random freezes as shaders switch around either after not requiring two-pass anymore for a while or requiring two pass due to something being enabled.
StormRel
marked this post as
needs more information
Is this still occurring?
If so, can you provide your hardware specs and logs?
Desmoulins
StormRel
It is indeed still happening, although at a much less extreme intensity.
- It might be just that i was lucky not to have people or effects around much that triggers it but i definitely did notice it still happening several times.
- Or the changes to process priority and affinity i made (which were previously left at default) may have impacted this positively, i did notice slightly higher framerates (on Desktop)
- Or the latest changes on open beta (most likely the physbone related ones) may have had a positive impact.
Hardware Specs:
CPU: Ryzen 5 3600
RAM: 64GB DDR4
GPU: RTX 3070TI (8GB)
Several people with much more powerful hardware (and even X3D Ryzen have also noticed it although less long freezes).
I attached the log of yesterday's session where it happened several times.
I'm currently running the Third Person Beta. These issues happen in all betas and live (since 2-3 months, it started basically right after Steam Audio went live, i was using the Steam Audio Beta for a long time and it did not have these issues).
Desmoulins
StormRelI
I will try again after today's open beta update (the one that fixes hitches every few minutes. I did notice that too, after like ~30 minutes it was getting so bad it was freezing for 2-5 seconds...
That is on PCVR tho.
I'll have a big test on Saturday and report back.
Desmoulins
StormRel So Saturday test is complete.
I tested on the latest 3rd Person Open Beta which also included the fixes for freezes.
I saw a significant improvement over time. I'm fairly confident the increasingly worsening stalls are gone. However the original issue reported here that lots of smaller things that shouldn't be causing hangs or freezes is still up and has not been resolved. I saw a slight improvement in that regard but its still not gone. As more people join a lobby and as average framerate drops, very tiny stutters/stalls increase all the way back to full 2-5 second freezes. Things like dying in basically any game that shows you a black screen (Prison Escape, Among Us) and other smaller interactions between avatars sometimes cause it still.