Counting hidden objects when calculating performance rank is a terrible idea!
tracked
Raccoonteur
I'm an avatar creator. And I don't mean to brag, but the avatars I create are popular enough that people often pop on on me to ask to clone them off me because I haven't created a world with them all yet.
The reason people like my avatars so much though is not because I'm an amazing modeler, but because I tend to put a lot of fun props and sound effects on them.
For example, for St. Patrick's day I created a leprechaun best boi avatar which had multiple songs, a box of lucky charms, three different types of alcohol, and a riverdance animation.
Now here's the problem:
Under the new system, despite my avatar only ever showing one of those props at a time, it would now be ranked very poor because the polygons for every prop would be counted in the rank!
But it gets worse!
The only way to do a dance routine with a gesture as the game is currently designed (and you want to use a gesture for this because you can't move around if you're using an emote, and you can't stop a long emote in the middle of the animation) is to create a duplicate of your avatar, hide the original, and play the animation on the duplicate.
The problem here is that under the new ranking system, any avatar that plays an animation on the avatar itself using a gesture will now have all of its polygons counted twice against it in the ranking system!
Worse still, if that animated avatar has props on it... because those props MUST be on the original avatar as well for the animation to reset properly after playing... all the props will be counted twice as well!
The end result of this is it will no longer be feasible to have an avatar which has many props and animations on it, because many people will hide it automatically based on an inaccurate performance rank calculation... and that's no fun!
But I'm not here just to complain. I'm here to offer a solution:
The correct way to implement this system, if you must implement it to prevent people from circumventing the limits, would be to determine performance dynamically at runtime. For example, if you need to read the meshes to determine how many polygons each has (which I assume is the case because you've changed it so any avatar with read/write disabled, which I assume would improve performance normally, is now ranked as very poor), do that, and store the count for each mesh in the avatar. Then calculate the performance of the avatar dynamically, and hide it if necessary, as parts of it are displayed and hidden, so that an avatar which has props or plays an animation on a gesture will only be hidden if the parts that are currently active on it exceed the performance ranking limits.
To do otherwise would be to punish every avatar creator who creates avatars which have lots of fun props and things on them!
It's already bad enough I have to tell people who aren't my friends to show my avatar before I demonstrate the fun things it does. Don't make it so they, along with my friends, won't be able to see my avatars at all even if I never toggle any props or animations!
Log In
ghostt
In response to this post and to my previous comment, I propose an even better idea. It would work in tandem with this feature: https://feedback.vrchat.com/feature-requests/p/switching-avatar-using-exmenu-andor-contacts
Linked avatars
Essentially, let people be able to select different versions of a single avatar using the action menu and then do the performance ranks for each version separately. This way an avatar could have, for example, multiple clothing toggles for different occasions which would actually be different avatars.
e.g.
I have a master Avatar with a sub-menu that contains different versions for different occasions. One ranked Good
for events filled with people, one ranked Medium
that is my main avatar, one ranked Poor
for experimental features that haven't been optimized yet, etc.Benefits:
- The performance ranks feel more "just". - More optimization as hidden objects/logic don't need to be loaded. - Enables way more complexity without affecting performance. - This system could be used to organize different avatar versions in the uploaded page.Drawbacks:
- Possibility of more loading of avatars in general if users like to toggle stuff on and off. - Possibility of crasher avatars existing within the avatar chain, however, this doesn't create the issue, but it d.Thinking ahead:
A way to reduce loading times using this system would be to download the avatar the user selects in the main menu and then download all the other versions in the background. Passing each from slow memory to fast memory when it is selected in the radial menu. Ideally, this preloading should be limited to a certain chain length and that should be an option as people have different cache sizes.Also, avatar authors shouldn't have the ability to put other authors' avatars on their own, as that would: Invalidate one of the biggest selling points of VRC+. Not every avatar creator would be OK with their public avatar being linked to someone else's avatar. It prevents crashers with multiple accounts from linking other crasher avatars from their other accounts.
It doesn't even need to be the same avatar. As an avatar creator and seller, I could use this system to have a master Avatar that has all of my public cloneable avatars that I could use to advertise my avatars and my Gumroad page. So, this system has multiple uses.
Scout - VRChat Head of Quality Assurance
tracked
ghostt
In the new system you propose, crashers could mask themselves as optimized avatars, despite being able to trigger spawn millions of polygons and particles. There's no way a user could detect those before they happen if they don't count torwards the stats.
I know where you're coming from, it sucks to be deterred from creating complex avatars. But this system was designed this way on purpose, to show the worst possible state.
In a new system I suggest it shows both the current state and the worst possible state, the latter one counting to the rank, but the former one also showing when clicking on "show avatar stats".
(Btw, unless you're creating avatars that aren't meant to be rendered in real time, there are a lot of ways to optimize models. If you work in the games' industry you would know this. Despite having different hidden meshes, that doesn't excuse them not being optimized themselves. You have to draw the line somewhere.)
LaFont
I've thought on this for a bit, and honestly I'm cool with the new ranking system, it allows you to see a relative "complexity level" for each avatar, which is nice when you're trying to figure out whose keyframes are killing your performance.
However, it doesn't allow you to figure out whether someone's default avatar (without any of their animations playing) might also be a performance issue.
I'd argue a two-column system would be more effective, the first column would be the full scan (as well as determine the overall grade), and the second column would be the "default state" scan that was in effect before.
Digitrevx
Sorry for the harsh criticism but this is one of many examples of why you should NOT push "unperfect" systems as you call it out till you have other systems in place that complement such issues. This is such a trademark method for how vrchat pushes "features". Please work on that before considering any other big brain solutions. "Smart" people can be stupid. Like sending designers out to be PR people. Or use programmers to make design decisions that effect social dynamics. It's been proven time and time again engineers are terrible at designing social platforms and they fail all the time because of this. Understanding your community is the best education the vrc team needs atm. The community is expecting you to make a fair weighted system to the profiler. What works on paper doesn't always work for people. Give people reasonable limits and people wont break it and stop taking unity's guide on poly count as gospel in an app that barely fits unities intended purpose in the first place.
g
gg67
Hi Raccoonteur, thanks for the post. Performance rank on avatars is a hard problem to solve, and you're right that runtime analysis would be a great solution. Unfortunately, accurate runtime analysis is very performance expensive - the obvious solution is to run the analysis every frame. It is possible there are other clever solutions that we haven't thought of yet.
In the meantime, an accurate, performant, obvious solution is to count disabled objects as well. In a perfect world, we could have spent much more time getting this perfect, but we needed to set the standard for avatar perf sooner rather than later. Every day we didn't set that standard, creators were uploading non-performant avatars, knowingly or unknowingly. The longer this went on, the more of a VRChat "standard" this would become. We needed to let our creators know that there are limits and they are going to be enforced. We acknowledge there are better solutions and we will continue to work towards those! Promise.
We do have an upcoming change that will simply disable the perf offending component (lights, particles, etc), but I don't think the change will help much with poly count (turning off polys wouldn't look good). Maybe there's something in this change that could help with your specific situation? I think it's slotted for the release after 2019.2.5. Let us know if it helps/doesn't help when it's out!
Also, from my understanding, the reason you have to make multiple copies of your avatar for animations is because that's the only way to get them to work with gesture overrides. If you created a regular emote and triggered from the quick menu, copies of the avatar mesh wouldn't be necessary. Is that your understanding as well? I believe we are exploring better ways to trigger those emotes instead of just via the quick menu.
We know the avatar perf rank system is not perfect. Like many of our systems, we will continue to iterate on them until they get to a place that we and our creators are happy with. This will take time, as we're spread pretty thin across a lot of systems and work. Believe it or not, most of our time for the rest of 2019 is iterating on existing systems! We appreciate your patience and support as we work through all of this. Your feedback really does help.
Floppiii
gg67: Would you also be able to check out why people get lagg locally when enabling objects with gesture?
We made a big canny with that already and sent Tupper and Avatar to test it.
We still heard nothing until now from him, it was 4-6 months ago.
Also Thanks for your reply and it's good to hear you are working on stuff. Sadly most of our animators quit already because of all the limits you set for us.
Also please take a close look at error.mdl's canny post about the audio limit, it's important because this will be the last straw.
g
gg67
Floppiii: Thanks for the response. Could you link me to the local lag post?
Hopefully we can get animations to a better place and win back all of those animators :)
And I've posted on the audio limit post! Hopefully we can come to good resolution.
Floppiii
This is the post.
My Friend Raivo then communicated with Tupper:
This last post is the most important one, i just got the information:
A friend also made a video:
Thanks for your help, you are finally a little hope for the developer team it seems like. :)
ExDe707
gg67: Hey, thanks for the response.
We use gestures because we would like to sometimes retain control of our avatar and still be able to move. Animations that change shader properties require a duplicate mesh under VRChat's limitations. For instance, I add music to my avatar and animate the shader to the music. This way I can dance myself and can skip on adding a lengthy custom dance animation.
rollthered
gg67: Would allowing for LOD's on avatar's help with the polygon issues? Perhaps VRChat could add in SDK tool's for atlasing and LOD generation in the future? Or even partner up with some pre existing tool's that exist in the Unity community.
Cibbi
gg67: wanna a better alternative?
First make a weighted system and not a "worst point is the base" system.
Second instead of counting like everything is active just consider what is activated as the actual rank BUT also count the total amount of objects in the avatar and if those exceed the limits just add an esclamation point on something on the side of the rank so the person knows that a specific avatar has some stuff disabled that can be enabled.
Then if you wanna go fancy you can make so the active objects statistics get updated on a specific avatar every time you check them, so at the end for each person the check gets done only once on load and once per every time you check that avatar. Is still probably not a perfect system, but at least would be miles better than the current one.
Kasaishiri
gg67: I appreciate this information and as a point of constructive criticism regarding your guys development this is the level of communication most of us want to see more often. keep it up please.
Wiks
gg67: I am genuinly not sure, but would it be hard to change it so one performance check is on load, which doesn't count deactivated objects, then do one check if a gesture or emote is activated, which would look for any activated gameobjects (and their child objects), save that result in a cache, so that if the gesture/emote is activated again, there wouldn't be a need for another check, just to roll the previous result.
This way the performance rating displays at fake runtime, where the checks have already been done and the stats are just saved and change depending on which gesture is currently used, displaying the object active only when the current animation is used.
Kizzy
gg67: I think the problem is that the standard that VRChat wants to set with this isn't actually the standard you think is going to happen. The people who have hidden stuff on their avatar such as props or other meshes or etc. aren't going to delete all that stuff so they can be a better avatar ranking, they're instead going to shrug and say well if my avatars are going to keep being poor ranking, nothing I'm able to do will change that without ruining the things on my avatar so I just won't.
And that is the general sentiment I see now with avatars. The system is set up in such a silly and arbitrary way that does not accurately represent how optimized a person is. So naturally, if the system is unfair people are going to try to get around it. The solution is not to make the system even more unfair like this change does. It just makes people even less incentivized to optimize. "Poor" is now the new standard. The majority of my friends are labeled poor including myself. Is it because their avatars and my avatars are highly unoptimized? Nope. The vast majority of our avatar stats are in the green, but just one thing in the red, such as polycount, now makes us poor. All this change did was reinforce how unfair the system was to begin with by making it even more unfair, and have the complete opposite result on trying to get people to optimize. I don't even try to make avatars that are able to get past the "Poor" rank anymore because of how little that rank and that title actually means and reflects upon the optimization of my avatar.
Raccoonteur
gg67: "Unfortunately, accurate runtime analysis is very performance expensive - the obvious solution is to run the analysis every frame. It is possible there are other clever solutions that we haven't thought of yet."
Two alternative methods come to mind:
- Don't run the analysis every frame. It's not like the game's animation is frame-perfect as it is. If a 500K poly avatar appears for 1/10th of a second before being hidden, that temporary hiccup is not a big deal, as the avatar will continue to remain hidden until it changes again.
- Analyze all of the animations the avatar can play... once, when it first loads into the world. Check which objects are made active by each animation and determine from that what the maximum poly count will be for the avatar during that animation. Store that value, and now you know if you should not allow that animation to play for those who have set their max poly count below that limit.
Raccoonteur
gg67: "In the meantime, an accurate, performant, obvious solution is to count disabled objects as well."
Performant? I'll give you that. But accurate? If I have an avatar which 99.9% of the time is displaying 10K polygons,and 0.1% of the time is playing an emote with 90K polygons, it could hardly be called accurate to declare that avatar to be too unoptimized to ever be allowed to be seen by anyone.
"In a perfect world, we could have spent much more time getting this perfect, but we needed to set the standard for avatar perf sooner rather than later. Every day we didn't set that standard, creators were uploading non-performant avatars, knowingly or unknowingly. The longer this went on, the more of a VRChat "standard" this would become."
Except your system as implemented is having the exact opposite effect. If I have these animations on my avatars they will be marked poor. So what incentive is there in optimizing my avatar so that in general, when I am not playing these animations, my avatar would be ranked good? There is none, aside from a desire not to piss off my friends, because my avatar will now always be marked poor regardless, unless I remove my animations entirely, which I won't do, because being creative with my avatars is half the joy of VRChat for me, and the limits imposed are too strict. You're not giving me incentive to optimize my avatar, you've simply resigned me to the fact that my avatars will now always be ranked Very Poor.
"We do have an upcoming change that will simply disable the perf offending component (lights, particles, etc), but I don't think the change will help much with poly count (turning off polys wouldn't look good). Maybe there's something in this change that could help with your specific situation?"
Unless you're disabling objects that I have hidden which are displayed by animations, your changes will have no effect for me. My avatars are not poor performing because I have tons of lights and particles. I don't use lights, and I try to keep my particle counts reasonable. My problem is that I use a lot of props in my animations, and they're not always the best optimized because beggars can't be choosers, and free 3D models aren't usually designed with VRChat's strict polygon limits in mind.
I mean if I download a car someone built with half a million polygons, there's only so much I can do in Blender without literally recreating the model from scratch. Get it down to 100K polys and 10 materials? Sure, I can do that. Get it down to 25K? Not likely. And it's not reasonable to expect avatar creators to make everything from scratch. I'm not getting paid to create content for your game after all, so if it's going to take me more than a week to create a new avatar, I'm simply not going to do it.
"Also, from my understanding, the reason you have to make multiple copies of your avatar for animations is because that's the only way to get them to work with gesture overrides. If you created a regular emote and triggered from the quick menu, copies of the avatar mesh wouldn't be necessary. Is that your understanding as well? I believe we are exploring better ways to trigger those emotes instead of just via the quick menu."
Yes, I'm aware of that. However triggering an emote means you're stuck in one place, which is often not desirable. I want to be able to make my avatar dance, and move about while I'm doing it, so I can continue to interact with people. If I had full body perhaps I wouldn't care about this so much as I could dance wherever and whenever I like, but at the moment, I don't, and having to stand there doing nothing while an emote plays just sucks. Even with my bigger animations, I've been looking into ways to play them in one place while I continue to move around because having to stand still for 30 seconds to a minute as they play is awful. I've seen others do this with emotes, so I know it's possible, but it's complicated to set up and one has to manually edit the animation files to add special keyframes so animators can disable themselves to achieve it.
✩Frisk✩
gg67: consider fixing the Mirror so that when animations changes materials ... It Updates on the Same Mesh ..... having to run another mesh just increases my PolyCount for materials to compensate for that problem never fixed.
H
Hugo Zink
gg67: the performance analysis isn't actually all that expensive. The most expensive part is finding the game objects. You can cache the found objects and rescan this same list every 5 seconds or so.
Kizzy
Nearly a month since this Canny was made and now has nearly 450 upvotes and 100 comments, more than the vast majority of canny posts on this site. Still, there has not even been one word about it from the VRChat team.
This is why people complain more and more that VRChat does not listen to its community and does not communicate with us. People are getting sick of the silent treatment and the "we know better than what the players want" attitude.
What is the point of even making these canny posts for feedback or bugs when one can get this many upvotes and comments and still not even receive acknowledgement about the issue from the team? Not even bothering with a response to this issue that has received gigantic backlash shows to me how little the VRChat dev team actually cares about the people who play their game.
Raccoonteur
While it's true the owner of that group asked people to vote, this post had around 300 upvotes before that and there wasn't an immediate surge of votes after their post - indicating most of the votes are legit.
Kizzy
This system does not work, and in fact does the complete opposite of what you are saying. The general sentiment I have seen is not "Oh no, I need to optimize now", it's "Oh, everyone else is also labeled poor and very poor now, so why bother optimizing if everyone is already poor?". In fact, some people already have optimized models to begin with, but with hidden things like props and such on them, that is now marking their optimized avatar as poor just because of hidden accessories that do not even render for other people unless it's unhidden.
This ranking system is bad, and it needs a rework. It should have always been on a weighted 1-100 scale, not arbitrary tiers based on arbitrary numbers. The change to count hidden objects now makes an already bad ranking system even worse and more pointless and inaccurate.
This will do absolutely nothing to fix the issue of optimization in the game, if anything it now makes people less incentivized to optimize, like I said before. All this does is screw over content creators.
The backlash to this issue is legit, you can't just say it's from some discord raid. You can't just say that the only people complaining are 14 year olds with instance wide shader effects either. To do so is just completely ignoring the main issue and is strawmanning it.
Raccoonteur
"Just because he started posting telling people outside his community to upvote it at 300 doesn't mean it started then."
You misunderstand. He invited me to his discord one day because he wanted to post about it and tag me in the post. I didn't know what sort of avatars he made at the time, so I agreed to join. And that was the first post I saw about it when I checked his discord, so that was likely the first time he'd mentioned it within his community.
Raccoonteur
"This system has successfully ingrained the mindset of "oh I guess we should optimize now since there can be consequences if we don't"
People don't create avatars so quickly that you would immediately see such an effect, and the setting defaults to being off, with most users being completely unaware it exists at the moment, so they would not know to design around it.
You only think you're seeing results because the problem wasn't as big a deal as people made it out to be in the first place.
UnknownAK
To be honest this change is killing my inspiration for new avatars. Like I think of an idea for an avatar and all the cool props I could have and then I just think "oh well its gonna be end up just being classed as Poor anyway so what's the point?" So many of my current avatars are instantly Poor now so I don't even want to update them anymore....
Personally I see avatar props as part of the avatar's quality and to be honest this just seems to punish creators who make good quality avatars. I think a much better workaround for this would be to just add a safety option related to avatar props.
Jasper Raine
Normally I would've disagreed at removing this feature, but the solution you proposed actually makes sense so I have no reason to disagree. This would fix issues with normal avatars and still limit avatars built for crashing.
✩Frisk✩
Jasper Raine: fail to notice that this dont apply to hacked clients that bypasses safety overall
Jasper Raine
✩Frisk✩: True. But it's possible if this is controlled client side as well, only the crasher would be seeing it and not the people with safety settings on.
✩Frisk✩
Jasper Raine: seen some who bypassed Shader safety >.> and even tested it on max to make sure im not seeing things .... months ago
Jasper Raine
✩Frisk✩: oof
moondevourer
I am in full agreement. Don't take away important, creator-enabling features whilst not actually addressing any of the issues or bugs VRC is plagued with. None of this 'ranking' stuff seems to be a solution to anything unless you hate having a fun, functional game that's kind to creators :/ VRC team I just wanna be seen, I put so much work into my avatars and so does everyone else who bothers to bring fresh new content into this increasingly awful game!!
Load More
→