Control Selfie Expressions via OSC, or otherwise over the network
cubee_cb
For devices that don't have or can't use a webcam.
In particular, Proton currently doesn't work with webcams, and thus Steam Deck and Linux users like myself cannot utilise Selfie Expression at all even with a webcam plugged in and selected.
Software such as iFacialMocap can track head rotation and send it over the network.
I ask to have at least the avatar head position/rotation controls available to control through OSC so that external programs like VRCFT (and VRCFT.Avalonia on Linux) can pass through that pose info to avatars.
OSC for the hands would be neat for integrating Leap Motion and similar hand tracking solutions "natively" for Desktop users as well, instead of them relying on specialised avatar setups.
Alternatively, a way to run the Selfie Expressions tracking algorithm on another device (for example, a special mode in the VRChat mobile app) and send it in to the desktop app would be nice.
---
All I really want is to be able to nod when I'm listening to people, I do it subconsciously and get a little upset when I remember I have to grab my mouse to nod again. Selfie Expression is almost there for me.
Log In
cubee_cb
it seems this is a Proton issue due to poor Media Foundation support; the way VRChat accesses the camera is simply not supported?
someone is working on implementing the appropriate support on Proton's Issue for VRChat - https://github.com/ValveSoftware/Proton/issues/1199#issuecomment-3981022802
lnnercat
+1 for this. I'd really love to be able to move my head around and nod and boop people on my linux machine.
The two solutions I can see are :
- Make hands, head rotation, shoulders controllable via OSC instead of selfie expression directly acting on avatar. This would be the most "open" and useful change. It would also allow people (in a cross-platform way) to have custom alternatives to selfie expression, maybe more precise control, etc
- Make VRChat able to stream a video input in a way that's linux/proton compatible, i.e. as aricchi1998 said accept video stream URLS, or change video capture implementation
aricchi1998
for the Linux webcam issue specifically, I think it'd be sufficient to accept a local video stream URL as an alternative to a webcam device and have it otherwise work the exact same way... would be really nice to have this working on proton