[1431] Estimated memory usage appears to show pre-crunch-decompression size
tracked
bd_
When a texture is compressed using crunch compression, the new estimated memory usage field in the avatar stats shows the crunched size, not the actual size in memory after crunch decompression. This is not ideal, as it will encourage users to juice these numbers by crunch-compressing textures, when this doesn't actually help memory usage, and makes load hitching worse.
As a concrete example, this avatar shows a texture memory of 10.67 MB, but an estimates memory usage of 0.48 MB. The avatar contains a single 4k all-white texture that has been crunch-compressed. https://vrchat.com/home/avatar/avtr_ef3447a0-5fea-4a7d-aa5d-d4d9a75aeb74
Log In
Desmoulins
Crunch Compression does not impact VRAM or RAM usage. It ONLY affects file size (upload size), NOTHING ELSE.
Crunch compressed textures also have a bigger impact on load than normal compressed textures. DO NOT use crunch compression unless you must absolutely crunch the file size down by just that tiny little bit more to get it in upload range (for Android as example)
Smash-ter
Desmoulins the reason why it's a bigger load is due to how Crunch works, which relies on your CPU to decompress it before sending it over to your GPU. Crunch exists because games were stored on HDDs and to reduce load times this was the solution at that time. Nowadays SSDs are cheap and as it stands the assetbundler compresses textures when compiling the avatar
Desmoulins
Smash-ter Right, but a much bigger issue with crunch is that it was also designed for static games that load assets prior to the game or level. VRChat loads everything dynamically during gameplay which makes the impact a lot worse because it now happens during active gameplay rather than on instance load.
DarkSwordsman
This needs to be fixed. It makes the features almost useless since a lot of users have hundreds of MB of texture memory hidden behind crunch textures.
_
_tau_
tracked
Sayamame
_tau_ (Sorry for pinging!)
Are there any updates for this?
The time for Reducing Avatar Download and Uncompressed Size Limit is approaching...
Although it may seem like a trivial issue, this difference should actually have a somewhat large impact on users' behavior.
＃かなたん
If they release crunch-compressed items without treating them at their decompressed size, wouldn't it make no sense at all to provide a separate limit on download size?
As a user who has been waiting for a limit on VRAM usage, I cannot welcome it...
＃かなたん
We just hope that the newly added "Uncompressed size" will be calculated correctly using the decompressed size for crunch-compressed textures as well.
That's all we want.
lilxyzw
Thank you for providing a workaround for the limitation! I'm going to recommend Crunch Compression to a friend who wants to reduce memory size!
...This is a joke, but users will think so if it is released to live with this specification.
Sayamame
If OpenBeta(1434) is released to Live without fixing this problem, the use of Crunch compression may be encouraged among users, starting with users who observed how the value of Uncompressed Size changes, and this is really not good.
bd_
There were some comments in VRCPrefabs suggesting that crunched textures might not be resident in main memory. After doing some investigation, I can confirm that there are cases where crunched textures are decompressed into main memory - in particular, I was able to access the native color data via GetPixelData in a standalone Unity application. When I did this, the buffer was not visible on the Unity memory profiler (or perhaps counted to graphics memory) so it’s difficult to confirm if this is always the case or only when requested.
AirGamer
bd_ iirc, the default import settings for textures is Read/Write disabled, implying that no copy in main memory would be kept.
At least in the unity editor, I got an exception when using GetPixelData on a texture that was not read/write enabled (or otherwise IsReadable)
Did you test with a texture that is read/write enabled?
AirGamer
"and makes load hitching worse" I was under the impression that crunching made virtually no difference? Is there performance testing on this?
bd_
AirGamer This was based on some comments by tupper re: crunch requiring more resources in load, but I’ve not benchmarked this in detail myself (and, looking back, it’s not clear if he meant to refer to hitching or just cpu consumption in general)
Tupper - VRChat Head of Community
bd_: Anecdotally, I've definitely noted hitching in extreme use cases of Crunch. But it's anecdotal -- I haven't benchmarked. I might be totally wrong.
DarkSwordsman
AirGamer There's two separate things:
- Bundle Compression
- Crunch Compression
This isn't based 100% on fact because we don't have access to the Unity source code, but based on feel/testing/docs:
- Texture gets crunched.
- Bundle compression (LZMA)
- Bundle decompresses on download
- Crunch textures converted to VRAM-compatible textures
To my knowledge: normal DXT1/5 and BC7 (i.e: "Texture Formats") are already in a format the GPU can read from VRAM as soon as the bundle decompresses. However, "Crunch" textures can't be accessed directly by the GPU, so it has to convert it to normal texture formats. Hence the "Crunch doesn't affect VRAM" thing we know.
You can think of it similar to a monitor needing a special high-bandwidth cable (HDMI/DP). It's why monitors don't need any extra processors or chips, just a basic display controller to draw to the pixels. The conversion of JPG, PNG, UI elements, etc. are all done earlier in the chain by the GPU and CPU, and all the monitor gets is a raw RGB feed (excluding Display Stream Compression, a technology that has a similar goal to to how texture formats work).
AirGamer
DarkSwordsman I'm aware that crunched textures required an extra decompression step
I'm also aware that Crunch Compression + Bundle Compression achieves a better then just a Bundle Compressed texture
What I however don't know is;
Is the cost of uncrunching a texture significant?
Assuming a smaller bundle will unpack faster, does that counter the extra uncrunch cost?
Is the uncrunching done in a way that makes it more likely to cause hitching (i.e. is uncrunching performed on the main thread? and is bundle decompression done on another thread?)
DarkSwordsman
AirGamer the entire process of converting it into the correct texture is what causes the hitching. It's primarily when people have a lot of textures. Even if there was no hitching, no one can actually really recommend crunch compression because it barely reduces download size and does not reduce VRAM usage. So all you are doing by using crunch is reducing the quality of your textures because it's fun.
AirGamer
DarkSwordsman You can achieve significant download savings with crunch compression, a topic that was discussed to great degree when it was brought up in one of the developer blogs comments.
Re hitching, have you performance tested this yourself? do you have side by side comparisons or hard numbers?
DarkSwordsman
AirGamer I have not done vacuumed tests myself, but have done observations based on experience with certain avatars, and trust the word of others who have done tests.
Yes, it "can", but the cases where it "can" are slim, and savings are negligible in comparison to the quality loss, possible hitching, and the fact that it does not change VRAM size. For example, I do not use crunch compression and my Texture Memory is about 45 MB for a specific avatar, but the download is only 7 MB. There is no point for me to use Crunch.
If you are at the point where crunch compression may have a notable impact on download size, you are probably destroying your textures and/or already have way too much texture memory usage. Also, instead of using crunch compression, you could just drop the resolution 1 step and use DXT1/BC7. Actual quality is more important than resolution in most cases.
AirGamer
DarkSwordsman "certain avatars"
Are you sure that crunch compression was a common feature of all those avatars?
Are you sure that crunch compression was absent in similar complexity avatars that didn't hitch?
Are you sure that those hitching avatars don't share a different aspect that might be causing the hitching, and you are misidentifying the cause?
"cases where it "can" are slim" More often then not it's given me a noticeable reduction.
"instead of using crunch compression, you could just drop resolution 1 step and use DXT1/BC7." Doesn't always give you better quality.
The last two points are, however, tangential to my request for more substantial evidence of that crunch is a common cause of hitching.
I'm pretty sure I've seen hitching due to complex shaders, variables like that need to be eliminated/controlled to get clear confirmation one way of the other
Edit: and continuing the discussion of if crunch is worthwhile is just repeating the discussion that was in https://ask.vrchat.com/t/developer-update-16-march-2023/16950 and was not what was asked here
DarkSwordsman
AirGamer To our knowledge, crunch causes lag spikes because it needs to be converted on decompression.
Whether it does or not, I don't think it matters, because crunch drastically reduces quality for a generally negligible bundle size reduction. That's all we have.
So no, to my knowledge, no one has done extensive testing. It's been all anecdotal. But people I trust to have good knowledge and testing of things have said the same things, and the logic makes sense, so I believe it, especially since there is virtually no reason to actually use crunch, even if it didn't cause lag spikes on avatar load.
At the end of the day: NEVER use crunch compression unless you have a very, very specific reason to.
AirGamer
DarkSwordsman "To our knowledge, crunch causes lag spikes" I'm asking for numbers, proof, what is the foundation for this knowledge
You can't keep reiterating this as fact and ignoring my request provide the backing information.
Anecdotally, I've seen people lag loading avatars that do not appear to be crunch compressed at all.
Crunch compression can and has (for me) outperformed dropping the resolution. You shouldn't wholesale advising against it. it is another tool for download optimisation.
I also see that you have also ignored my hint to stop derailing away from the original topic of my question, just stop it.
If you aren't going to respond with actual performance data, and instead waste our time arguing trying to against the use of crunch compression, then don't bother.
DarkSwordsman
AirGamer I gave you an answer and you can't accept it. Sorry.
If not for the unproven lag, just DO NOT USE CRUNCH AT ALL becuase it provides NO BENEFIT besides MAYBE a reduction in bundle size.
AirGamer
DarkSwordsman I had specifically asked if the performance of crunching was tested
The statements tupper and others have stated conflict with unity's own statements regarding the performance of crunch compression.
tupper stated they did not benchmark to confirm their suspicion
You had responded with
Explaining that crunch compression is different from bundle compression, and state that crunch doesn’t save VRAM
I didn't ask for this.
Your next post states the following
"the entire process of converting it into the correct texture is what causes the hitching"
This is the statement I was asking if this has been performance tested
This doesn’t answer my question.
The rest of the post contains statements unrelated to my question.
Your 3rd post;
"I have not done vacuumed tests myself, but ... trust the word of others who have done tests."
That does answer my question, with a both a no and a yes
I guess I missed this because up to this point, a whole 88% of what you had written so far was unrelated to this question.
I do, however, respond asking for more substantial evidence
I was more so directing this against your observations as without 'vacuumed tests' there would be too many uncontrolled factors to say this is evidence (and not an unproven link).
You respond with;
"To our knowledge, crunch causes lag spikes because it needs to be converted on decompression."
This does not answer my original question, it does not state where your knowledge comes from.
This does not answer my follow up question, if does not provide me with the source evidence for this knowledge.
I respond by making my request to provide evidence more explicit.
You respond with
"I gave you an answer and you can't accept it. Sorry."
Well I guess you did answer my 1st question. but you had bundled it with so much unrelated information that I had missed it.
You, however, have failed to answer my follow on question
"If not for the unproven lag"
You then contradict you own answer you provided earlier.
I'll guess I'll consider your answer as per you latest message, "no"
That explains your inability to provide evidence
Maybe stop spreading unproven statements
If you are wondering why I'm so critical of this
I had seen people claim WD-off gave better performance, until a benchmark proved the exact opposite
Your claims regarding crunch loading contradict unity's, and until proven otherwise I'm going to believe unity.
A side note:
"MAYBE a reduction in bundle size"
From the comments in the dev post, over 60% bundle size reduction is possible, 20-30% appeared to be typical.
Some quest avatars exist that need crunch compression to be able use reasonable quality textures, as the next resolution drop losses to much information
And second side note:
Less than 20% of what you have written is even indirectly answering the 1st questions I had asked
And you've somehow managed to contradict you self in doing so
The amount of effort you have expended with unrelated information is absurd
I can summarise the information you have provided that I had asked for down to this:
A: "Has the performance of crunch compression been tested?"
D: "I haven't, but others I trust have told me they have"
A: "Can you provide their results?"
...
...
D: "No, It's unproven"
Your almost 500 words spread across 5 posts could have been condensed into 14 words (in 2 posts) (contradiction included) of information that I actually asked for
It makes it a lot easier to receive information when it's not surrounded by unrelated opinions.
bonus points for also being less heated.
This is the single largest post I have ever sent in the feedback board, and it only exists to pick though your posts
Next time you try to answer a question, don't throw in information or opinions that was never asked for
For the effort of completeness, this post can also be summed down to
"Less than 20% of what you have sent me is what I had asked for, which ended up hiding your answer"
"I guess you answered my 1st question, but you managed to contradict yourself later, treating the answer as no based on your last post"
"You never provided evidence I then asked for in a follow up post"
DarkSwordsman
AirGamer k
bd_
As a concrete recommendation for improvement: It appears that these servers are calculated on the server side (since the estimated memory usage doesn't appear on local test avatars). If it's difficult to determine the uncrunched size based on server side analysis for whatever reason, if would be good to recalculate them on the client side as well based on decompressed texture memory, just to avoid creating an incentive to crunch things unnecessarily.
bd_
Perf stats for that test avatar
Load More
→