Player Voice variables for Udon (e.g. isTalking, loudness, viseme)
tracked
DrBlackRat
Issue:
Currently it's not possible to react to a players voice at all. We don't know when someone is talking or how loud they are for example. This limits us from making games where someone needs to be quiet to hide from an enemy for example.
For security reasons it makes sense that we can't just listen to a player voice, which is why I think only giving limited access to it would be fine.
Suggestion:
Add an
isTalking
bool to the VRC Player API which is true when voice is detected and false when not.Add a
loudness
float to the VRC Player API which is a value between 0 and 1 depending on how loud a player is. (Basically how the current Voice parameter works on Avatars)Add a
viseme
integer to the VRC Player API which would indicate what viseme is currently used. (Basically how the current Viseme parameter works on Avatars)These would allow world creators to integrate player voice into game play while also not just directly giving them access to player voices for security.
Log In
vavassor
Visemes could be fun for something like a singing or karaoke game. Or a puppet or possession mechanic, where a character mimics the player's speech.
This post was marked as
tracked
Fax
Merged in a post:
World Sdk accessing some Avatar Sdk variables
_Chronic_
For a long time I've been desiring a process that allows me to get the variable within debug which shows a float value for visemes. I had hoped to make horror based Udon mechanics centered around player's voice volume. Would it be possible to add this to VRCplayerApi?
Scena
I really do hope this is added in the future, it's already possible on avatars, but even just to get a loudness float from use in worlds, would already be a huge advantage.
M
Mantibro
I've always wanted to have this feature in my new horror game, for the killer to be able to hear player's voices. I was VERY disappointed when it was not an option to me.
Widget365
This could have so many fun use cases! This would be incredibly helpful in my latest project Neoworlds, right now its very confusing at a given time who's talking when you can't see the default vrc nametags.
Fax
Merged in a post:
Expose voice volume to Udon
Jar
For a game world idea I'm working on, it would be useful to know whether another player is talking. You know how you can select a player, and see the amplitude of their voice going up and down, on the slider that lets you control a player's voice? (For the local player, there seems to be a similar bar that goes up and down underneath your microphone volume slider.) Perhaps this is the same value that makes their name tag light up? I would love if that percentage could be exposed to Udon. Perhaps something like "VRCPlayerApi.GetVoiceAmplitude()" and it returns a float.
In tandem with this, it would be nice if you could get the microphone mute state (bool) for the local player.
I can see this being useful and fun!
-You could have a prop in the world animate with your voice chat volume (that would be pretty fun)
-Have player names light up on Udon-based scoreboards or spectator screens when they are talking
-Games where NPC enemies can react to people communicating nearby (stealth game)
-Reminding you to unmute yourself in activities that have a microphone/stage/speaking role
Thanks for your consideration!
Omega22
This could be great for horror worlds
DrBlackRat
Omega22
That's exactly what I was thinking about when I originally made this canny :p
Mr․NoBοdy
Omega22
Agreed, especially games like SlashCo, Backrooms, and other horror games with AI or games that would fit perfectly with audio detections