Add Support for Streaming Mipmaps for Avatars and Worlds
complete
DarkSwordsman
This feature request is to add support for streaming mipmaps for avatars and worlds.
Steaming mipmaps (unity docs) allows textures to be downscaled and dynamically loaded into VRAM. It does increase the size of each mipmapped texture by 33%, so package sizes will be larger. However, it allows textures to be loaded dynamically based on distance.
This would mean that a texture that traditionally may be 2.8 MB DXT1 at 2048x2048, may be under 44 KB if it is reduced to 256x256 in VRAM. So in worlds, this means significantly reduced VRAM usage when texutures are a certain distance away. Same with avatars if they are a certain distance away.
If tuned properly, this could mean that avatars that may traditionally take up 200 MB of VRAM, can instead take up magnitudes less. Often, 2K textures look just fine on most avatars from even 0.25 meters away. So being able to unload them for a smaller one would be significantly useful, especially for large meetup events.
Fax did mention that VRChat currently does not have mipmap streaming. The reasons are the following:
> Most notably - some user avatars may lack mipmaps, have them disabled, or have a maximum priority. This would adversely affect mipmaps with streaming enabled: Once you run out of VRAM, they’d appear blurry and stuck at a lower mipmap. Users might run into this issue maliciously, or accidentally.
Considering that the VRChat application currently allows avatars to be created with mipmaps, but they aren't shown, perhaps there is a way to allow the client to choose if they want them enabled? I do know many games have texture streaming as a togglable option, so perhaps VRChat can do this here? I would even be happy if this was an experimental feature that had risks associated with it, if it's for the sake of testing.
Fax also mentioned:
> Unfortunately, that wouldn’t fully solve the problem, as avatars uploaded without streaming mipmaps enabled effectively have an ‘infinite’ priority.
I see this as a non-issue, unless there are some other contingencies. Unity docs mention that it can load non-mipmapped textures alongside mipmapped textures. Considering VRChat doesn't have mipmap streaming already, this means that the game will be identical with or without mipmap streaming if all avatars don't have mipmap streaming. But in the event that half the lobby does, then there will be at least that many avatars that
will
save on resources, even if the others don't.Also, perhaps (idk if this can work), but this may also mean that the avatar hider can unload textures properly? At least in theory, if the avatar is not loaded, it may load the smallest mipmap? This would save a LOT of VRAM for users that want to use the avatar hider.
This feature would help a lot for people that do not want to optimize their avatars. I do think VRAM over-utilization is one of the worst offenders in terms of VRChat performance. Solving this issue would help many, especially those with less VRAM.
Log In
Fax
complete
Thank you for your request!
We released VRChat version 2024.4.2, which enables mipmap streaming.🎉
Please check out release notes for additional information: https://docs.vrchat.com/docs/vrchat-202442
DarkSwordsman
Fax I appreciate you 💜
Fax
DarkSwordsman: Thank you! I'm just the messenger - the team did all the hard work!
Fairplex
Fax Hey fax, is it really enabled? Because on both textures import and graphic settings its OFF. Is there an SDK update coming soon for us?
Fax
Fairplex: Yes, it's enabled!
If an avatar has mipmaps, VRChat uses mipmap texture streaming.
Fairplex
Fax Ok I see what you mean so VRC client have this feature, but what I am asking is that we also need texture streaming in both SDK, not only on the client. So if we want to debug the texture streaming feature, what do we do, modify the SDK?
Fax
Fairplex: Are you referring to "Texture Streaming" being disabled by default in "Project Settings" > "Quality" > "Textures"?
You
should
be able to test texture streaming by enabling this option and enabling "Mip Streaming" on your imported textures.Please feel free to create a feature request for enabling these options by default!
Fairplex
Fax Yes! I am referring to this option. But the issue it that VRCSDK (EnvConfig.cs) is currently coded to disable it entirely when Unity loads, and when scripts are recompiled. Do you think its possible to update this script so the SDK reflects the texture streaming setting that is being used in the client?
Fairplex
Fax well, I already opened it 2 years ago! 🙃
LunnaBug
Vrchat already has mipmaps for avarars, but now vrchat is forcing it even when we don't want it. This breaks quest avatar features like glitter. Please go back to the old system where users have choice!
DarkSwordsman
LunnaBug Sorry but this is irrelevant to my post. Please make a new bug report post if you want them to fix an issue you have.
Fax
To clarify (because I've seen a few avatar creators get confused about this)..
- VRChat doessupport mipmaps for avatars and world! Mipmaps make your textures look better from far away. Click "Generate Mipmaps" to enable them.
- VRChat does notsupport Unity's mipmap streaming system yet. This is an optimization that would reduce VRChat's VRAM usage. That's what this Canny is about!
DarkSwordsman
Fax Thanks for posting that clarification.
To add: From what I have experienced in the SDK, most or all textures have mipmaps enabled by default when imported to Unity. Further, the SDK forces users to enable mipmap streaming on the textures if mipmaps are enabled.
I believe this means that, surely, at least 90% of (modern, active) avatars already have mipmap streaming enabled with default settings. This is exciting to think about.
Thanks again for considering and working on this feature! I see the work you guys are putting in. <3
Fax
interested
We're interested in this! This has become easier to implement thanks to Unity 2022,
DarkSwordsman
Fax I am very happy! Thank you <3
benaclejames
Problem with this is that it'd break tons of avatars by retroactively turning it on. There may be a possibility for this in a future unity update, but as of now it's not really possible due to unity weirdness
DarkSwordsman
benaclejames I don't understand this argument. At least for the past 2 years that I've been uploading content to VRChat, the SDK has always enforced mipmap streaming to be turned on, because mipmaps are on by default.
I highly doubt that any more than 5% of SDK3 avatars do not have it turned on. And again: This can be a setting the user enables, like many other titles on Unity that use Mipmap Streaming.
Fax also said that the worst thing that will happen is that when it gets full, any avatar with it on will go to the lowest mipmap setting.
There's a lot of solutions here, namely it being a toggle that can be off by default, and be experimental like a few existing VRC features. I don't think this is a valid or backed-up reason to not have this feature.
I would rather it be an experimental option that you have to manually turn on, than one that doesn't exist, because this is the last major hurdle to avatar performance in VRC.
benaclejames
DarkSwordsman Interesting. I've never had the SDK tell me to turn on the setting per-project. I have, however, seen it ask me to turn on the option per-texture. The problem with this is that, while Unity will generate the mipmaps for the avatar on upload, it seems as if there's a problem with Unity's assetbundle format in which bundles created in a project without Texture Streaming turned on will look significantly lower resolution when loaded into a project with the option turned on.
This could just be errors within my own testing, but it does happen to add up with the majority of information I've received from other unity developers facing the same issue.
I do agree with you partly though. Just having an experimental option to enable it at the cost of potentially worse texture quality on legacy avatars would be nice, but the development cost of maintaining two builds simultaneously seems too much for VRChat (source: the outright removal of the quest pro native eye tracking branch)
DarkSwordsman
benaclejames They wouldn't be maintaining two builds. They would add mipmap streaming to their main project, but have the option turned off by default. Many other Unity games do this.
I'd rather see a commitment from them that they will try to implement this with a plan, than to not have the feature at all. It is unacceptable that this game basically requires a 3090/4090 or 7900XT/X just to enjoy dense instances without seeing robots or fallbacks everywhere.
This is an easily solvable problem, and ideally they should have had the fore-thought that a game with user-generated content should probably have texture streaming enabled.
Laser
benaclejames so be it then though.
Vrchat really is cripled by vram issues.. prompting people to get uncessairly expensive graphics cards just because of their vram :(
my 2080 super is outperformed by a laptop gpu simply because it has more vram.
owlboy
benaclejames when you say break, what situations will break?
And if the solution is for everyone to start turning off mipmaps, where does that leave us?
Rokk
DarkSwordsman the problem in Unity versions before 2022 is that if a single avatar (or even the world) doesn't have it turned on, it tends to break content that DOES have it on pretty quickly. If they could guarantee that every avatar and world has it then they could just do it, but right now that isn't the case due to older content.
Judging by the newest reply to this issue apparently they might be able to do it now that Unity 2022 is here, we'll see.
Laser
benaclejames this is one of those evils thats for the greater good. For the platform to grow it needs to work on the hard hitters in performance. Old legacy avatars should be updated and if they arent then they likely werent worth it...
A lot will work.. some old stuff will break but so be it.
McArdellje
Sorry if this is a stupid question because I don't understand too much about how unity handles textures but when you upload an avatar doesn't the SDK prevent you from uploading mipmapped textures with the "Streaming Mipmaps" setting disabled?
(I know you can upload textures with no mipmaps but I thought that for all mipmapped texture streaming mipmaps had to be on)
DarkSwordsman
McArdellje: You're correct. But apparently the game doesn't actually support mipmap streaming. So it really does nothing.
Demirramon
DarkSwordsman: it's so weird that they force you to do it but don't actually use it. Maybe they want to do that eventually?
DarkSwordsman
Demirramon: That does make sense to me, yeah. It's something that can kinda push for now, so when they do swap, a bunch of avatars already have it setup.
Fairplex
McArdellje: Totally agree, I opened a canny about that almost 1 year ago: