Selfie Expression Causes Some Avatar Jaws to Close Too Much
tracked
Salbug
When using Selfie Expression, some avatar jaws will close too much and clip through the mesh.
Example avatars:
Photo Viewer
View photos in a modal
Log In
StormRel
Merged in a post:
Desktop Webcam - Lower Jaw clips through upper mouth on all avatars.
Kudalyn
Basically every avatar I use (particularly furry avatars, but this issue happens with EVERY avatar I've tried) have issues with the basic 'face tracking' that the desktop webcam has. Closing the mouth and in particular doing "ou" or pucker-lip style face shapes without talking makes the bottom jaw shove through the face - or if the face tracking decides it's being difficult.
Clip of this in action: https://www.twitch.tv/kudalyn/clip/PunchyIgnorantTruffleTakeNRG-AInYsC-PZoVw9URb
I do enjoy using the Desktop Webcam for not just occasionally 'vtubing' - but hanging out at conventions or events without having to be in VR constantly. This really should be fixed to encourage even more desktop webcam usage. It just looks plain awkward currently, for even when just relaxed or not talking, the jaw clips.
This has been an issue since the VRC desktop was in beta and behind VRC+.
TotallyUnInvisable
i experience this the other way around where when i have my mouth closed it keeps the mouth on the avatar a little bit open
StormRel
marked this post as
tracked
Jared
I also had this issue, alongside other friends of mine.
We had no way of knowing what was causing it since the avatar debug menu doesn't show anything related to the selfie expressions.
I have included a screenshot of my friend who made an avatar to specifically find the blendshapes it was effecting, and another of me wearing the same avatar showing my default facial expression without moving my face. The picture of me is the one without a nameplate. In the graph. the higher the bar is for each visme, the more it is being used. The upper rows of blocks are the measurements for %50 and %100.
You can see that the "aa" blendshape is being used to negate the opening of the jaw when making your avatar purse your lips when you arent opening your mouth. It achieves this effect by putting the blendshape into a negative state.
The problem with this is that it seems to being going into the negative at a constant state for some people, and is affecting every facial expression being used by the Selfie Expressions, thus making it harder to open the mouth and make a clean "ou" shape (not a pursed-lips state) without it clipping through the lips.
Also the "ih" blendshape is roughly %50 active when I'm not making an expression, so idk what's up with that. I guess my face is weird.
So at the very least, we now know that the "aa" blendshapes is at fault here.
Perhaps a way to give us a bit more freedom with how much the Selfie Expression feature affects each blendshape, or a way to calibrate the feature to our faces could be useful to counter this.
Photo Viewer
View photos in a modal
Kung
Jared Do you have any examples of AA going negative while other vowels aren't active?
The screenshot appears to be the system working as intended (as you noted) to separately extract only lip shapes (and not mouth opening) from other vowel sounds by blending vowels other than AA with a negative AA. This means that in all cases, the negative AA should be blending with another active viseme and negative AA should not be active alone.
For why some avatars may show issues, I'd guess that they may be authored such that the mouth doesn't open much for (at least some of) the other vowels. When searching for the problem shape, it probably wouldn't be AA, but rather another vowel that blended at around 0.5 with AA at -0.3~ish produces issues, and a potential solution would be to open the mouth a bit more for the offending vowel shape.
Jared
Kung The most I can do to get it at a flat level is to just have it not track my face at all. In fact, my most neutral expression causes it to think that I'm making the "ih" blendshape alongside putting the "aa" at negative.
It seems that slightly pursing my lips counters it and makes it go back to a "neutral" state, thus making the 'aa" blendshape return closer to 0. Though of course, if I purse them too much, the avatar will begin to purse theirs as well. Opening my mouth half an inch will also put it back to a more neutral state, and going further will finally make the mouth start to open and then reach its maximum.
Furthermore, I have found that some avatars do not have this issue whatsoever. GabSith's Atena avatar does not allow the bottom of the lips to go anywhere past the upper lips, no matter how much I try. It might be important to note that most of the blendshapes are not very exaggerated, but I do not believe that the exaggeration of those blenshapes would affect the "aa" blendshapes ability to become negative at all. I would have expected it to at least go in the negative direction by some margin or degree, but it seems to go literally nowhere.
I am still at a loss at the root cause.
I have provided more images, along side a couple with an avatar that shows the jaw issue. The picture of the Atena avatar, the blue one, is using my face (with no expression) and shows that it does not have the jaw issue.
The one labeled "Base Expression" is the one that is not tracking my face.
Pardon my mouse handwriting:
Photo Viewer
View photos in a modal