Get the oblique camera matrices for each eye
TapGhoul
The new 3.8.1-beta.1 SDK adds a bunch of new camera accessors. However, something that is missing that would be incredible for effects that involve funkier render passes and effects that need udon-side help is the ability to read the per-eye oblique matrix. While there are ways to get this as-is, it requires some manual math to derive and requires digging into unity innards. It'd be great to be able to get these values directly off of the camera.
An example of this is to make a custom VRCMirror that has an arbitrary origin to more easily make better portal-like effects, as this requires being able to put a camera on the receiving side - doing this in a stereo-correct way is currently a tricky task. Having these values as getters or methods on the new
VRCCameraSettings
API would be incredible.Worth a note here: while we have accessors for eye position/rotation, this is not enough to correctly do stereo rendering. See the VRC Mirror component for details.
Log In
aurycat
At least in my case, the only method I'd need exposed is GetStereoProjectionMatrix.
An API for it could easily go along with VRCCameraSettings.GetEyePosition and GetEyeRotation: VRCCameraSettings.GetEyeProjectionMatrix, or something like that.
I'm not sure if TapGhoul wants something more than that, though, given they've said _oblique_ camera matrices. If they're referring to Camera.CalculateObliqueMatrix, no additional API is needed for that if GetStereoProjectionMatrix is already exposed: you can just use a dummy camera, copy the main camera's projection matrix to the dummy camera, and then call CalculateObliqueMatrix on the dummy camera. Of course, directly exposing CalculateObliqueMatrix would make that easier, but if I had to pick one to expose, it would definitely be GetStereoProjectionMatrix!
TapGhoul
aurycat All of these values can be derived at runtime via various methods, including the ones you've mentioned. But given we are being given a singleton for it, I feel it'd make sense to expose these on the singleton directly. There have been methods to detect some of the other options on the singleton previously, even getting the position of other types of player-controlled cameras - but given this is a nicer API, I feel it would be ideal to expose these directly.
As for stereo vs oblique - I agree the stereo matrix would be more useful in the general case, however the use case for using the oblique matrix directly makes portal-like effects easier (take a peek at the source of the VRC mirror component)
But either way, these methods being exposed would be nice, and I don't see a reason not to expose both if both are okay to expose.
aurycat
In UdonPortals I solve this issue by creating dummy cameras which don't actually ever render, but are set to render to the stereo left and right eye. That causes Unity to update its stereo projection matrices. Then I position and rotate them to where I want to render, and then call GetStereoProjectionMatrix on the dummy camera to copy the projection matrix to the actual camera which renders to a render texture. See this code:
It's a silly hack, but it works!
The new API provides all those properties I'm copying from the dummy camera, except the projection matrix. I think if the new API provided the projection matrix too, I wouldn't need the dummy cameras.