What appears to be happening is yt-dlp is always picking the first applicable HLS stream, recursing until it finds something with actual video. As a result, it can result in selecting an m3u8 that is invalid. It is also part of what causes VRChat to be unable to play youtube videos at high quality, thus needing a custom yt-dlp fork.

One solution I can suggest is having yt-dlp standard (or a fork still, but closer to standard) be used to construct multiple URLs - --get-url results in multiple lines returned in these cases, still abiding by filters, though -J is likely more ideal here as it will output all info at once, you'll just need to filter in your own logic rather than passing the filter into yt-dlp's command line.

For actually playing back - and then constructing a temporary .m3u8 in the format of:

This has been tested on windows and it plays back correctly. For example:

Using yt-dlp standard with JSON, the video bitrate can be extracted from the tbr field - it's in kbps, so just multiply by 1000 and make it an integer and HLS will take it.

Using AVPro's MediaPlayer::OpenMediaFromBuffer() call can take this HLS playlist directly, which is useful for both security (time-of-use related stuff) and general performance.

The proposed HLS playlists are the minimum required to get proper consistent playback. As per spec, BANDWIDTH needs to be set to the maximum bitrate, not the average, to allow buffers to be allocated large enough for smooth playback, and is a required tag to operate - but it'll survive if it's not perfectly accurate, tbr is a good enough value to use.