RTSP URL passthrough in VRC_SyncVideoStream
complete
よしたか
When trying to play rtspt:// URL in VRC_SyncVideoStream, it failed to play and an error 'failed to sanitize URL' appears in the log.
Could you add support for RTSP stream in VRC_SyncVideoStream?
I confirmed that current VRChat client can play RTSP stream via 'AVProVideo for Windows' asset. But we can't change the URL of AVProVideo scripts in the VRChat world without C# scripting, so it will be great if VRC_SyncVideoStream can play RTSP stream (it can change URL by UIText).
I'm running Wowza Streaming Engine server on AWS and providing audio streaming software for free to VRChat musician. Our RTSP stream has ultra low latency (0.3 seconds in Tokyo) and AAC 320kbps high quality audio. Listening musical instruments through voice chat is not satisfactory because of bad quality audio, but the experience of RTSP streaming is so good. We have many music live events every weekend by Japanese community using our RTSP streaming service 'TopazChat'.
TopazChat download pages and instructions are currently Japanese only and English translation is TBD. These links are for your reference.
- 'TopazChat 2.0 Player' for VRChat world creators: https://tyounanmoti.booth.pm/items/1752066
- 'TopazChat 2.0 Streamer' for VRChat musicians: https://tyounanmoti.booth.pm/items/1756789
Thanks.
Log In
Tupper - VRChat Head of Community
complete
This appears to have been completed in a previous update.
Momo the Monster
in progress
Momo the Monster
tracked
Momo the Monster
よしたか - can you provide a link to an RTSP stream that works in AVPro? I've got a branch that allows RTSP through but cannot find or create a stream that AVPro natively supports on Windows.
よしたか
Momo the Monster: Thanks! There is no always up-time stream, so please stream from OBS to our TopazChat server.
Example setup of OBS:
- Custom Server: rtmp://topaz.chat/live
- Stream Key: momo
- Video bitrate: 1500 kbps
- Audio bitrate: 160 kbps
RTSP stream link for AVPro will be as following;
rtspt://topaz.chat/live/momo
You can use any URL suitable string as Stream Key. Total bitrate limitation is 2000kbps with some buffer. When using x264 zerolatency and NVENC low latency profile, video will be corrupted color in AVPro.
Momo the Monster
よしたか: thanks so much for the info on how to use your streaming server. We've implemented a bypass for RTSPT URLs so they can be input by players in the next release.
よしたか
Momo the Monster: Thank you very much! We are looking forward that.
よしたか
Momo the Monster : If you have 6 seconds or more latency, please enable "Use Low Latency" option in MediaPlayer component. And this option is now only on before 1.11.5 version. I requested RenderHeads to come it back. They responsed it will back as this issue: https://github.com/RenderHeads/UnityPlugin-AVProVideo/issues/491
よしたか
Thanks for every world creators and musicians, TopazChat concurrent connections have huge grows in last month. We achieved almost 2TB network transfer since this canny original post (actually second graph counts are started at June 1st). The interest is not only from Japanese musicians, but also from global, Jean-Michel Jarre concert had great success with TopazChat (for 60 people in VIP room and YouTube live streaming).
Anyway, our main concern is that we hears most world creators saying "we can't publish our world to public because we have only one or limited number of stream keys". Because we can't change the URL of AVProVideo MediaPlayer component in VRChat client, we can't change stream key per instance, and world creators have to restrict access for TopazChat Player UI by PIN code or something to prevent malicious occupation.
This problem will be solved by RTSP URL passthrough in VRC_SyncVideoStream. I tested with creating a webpage to youtube-dl returns rtspt:// URL, but VRChat log says no URL has been passed to AVProVideo. It seems that only http:// or https:// URL is picked up from youtube-dl standard output. If you add regex pattern to pick up rtspt:// and rtspu://, this will be solved. Or if you bypass rtspt:// rtspu:// URL sanitization and youtube-dl, it's easy to us because we do not have to prepare webpage for youtube-dl.
Thanks.
Lhun
Hi よしたか,, I've done some testing on this, and m3u8 playlist with multiple TS streaming options actually work "okay" in vrchat. For example, using vlc from a personal computer, you can create a url such as: http://xxx.xxx.xxx.xx:8080/my.stream and it will load on the video player. There is a BUFFER, that's true, but with a low enough bitrate and locals in the room, it's reasonable, you must also open an inbound port for request (in this case 8080). A very low video bitrate and a high audio bitrate are possible here. Have fun experimenting with various options, the main thing is that m3u8 style playlist does, indeed, work.
よしたか
Lhun: Yes, I’ve tried HLS streaming and already provided as service TopazChat 1.0 legacy version on AWS. But it has 5 seconds latency on VRChat. I’ve fine tuned segment duration to almost 1 second latency on the Chrome browser, but it has longer latency on VRC_SyncVideoStream.
How about latency on your try with VLC?
Lhun
よしたか: it can be fairly good, but the main issue is the buffer and translation between vrchat's systems, and windows media foundation.
there's another issue with unity itself running at around 44.1khz during runtime, so you should take advantage of that knowledge and tweak the bitrate.
the main difference here is that you do not need any intermediary server, it's direct from your pc if you open the outbound/inbound port.
so, there's no server cost.
よしたか
Lhun: Yes, streaming from your PC will be no server cost. Streaming 320kbps for 40 people will require 12.8Mbps bandwidth, it will be OK in most broadband connections.
But the problem is that the URL is fixed in your world, so you should have DNS domain record becase global IP will change in most connections for consumers (or upload your world everytime when your IP changed).
Our most usage is open mic bar style music live. Difficult setup should be avoided but self hosting on your PC requires router configurations or UPnP. Maybe it will be achieved by developing dedicated streamer with functions of DDNS, UPnP, RTSP server and AAC encoder. But it's high development cost and we don't have stable RTSP server implementation when I tested at last year.
Regarding sampling rate, it seems that Unity and AVProVideo handles it automatically. We streams 48kHz AAC from TopazChat Streamer and 44.1kHz AAC from OBS but everything is OK.
よしたか
P.S.
HLS streaming from your PC can use VRC_SyncVideoStream, but it has at least 5 seconds latency. We use RTSP to get sub-second latency for avatar movement matching and communicating through audio streams.
FloydianSound
greatly support this.
let us be our own media distribution, stop relying on youtube and twitch
Disnowned
FloydianSound: As other posts have probably noted, it's dangerous to others to have self hosting content unless VRChat is going to proxy all that traffic, preventing IP exposure.
FloydianSound
Disnowned: how about making a RTP/RTMP server that is not hosted locally
or in my case, i have a second machine i intend to run as a NAS as a media distribution, redirected from a DNS and/or protected via VPN
Lhun
FloydianSound: latency.