Advanced Camera Control System: For Virtual Filmmaking & Photography
Gerzybow
VRChat’s camera overhaul update has added a very solid number of features, such as adjustable depth of field, masking, greenscreen, and overall ease of use. This has expanded the capabilities of virtual content creation, such as VRChat cinematic films and streaming quality. However, I feel it is merely scratching the surface of the possibilities that can be achieved. As someone who works in photography and filmmaking, I can help provide some feedback based on practical experience (both in the field and in VRChat) that can help really push the boundaries of what can be done with the camera system in VRChat and promote higher quality VRC content creation.
1. Frame Guides
Having visual guides to help frame your shot will assist in making more visually compelling images. Guides such as the Rule of Thirds, Golden Ratio, and other visual guides can help the image maker compose their shots more carefully and precisely.
2. Aspect Ratio Guides
In filmmaking, there are a wide variety of aspect ratios that a film can be projected, such as 4:3, 16:9, 1.85:1, 2:1, 2.39:1, and so on. For someone who is shooting a film or creating video content in VRChat, it can be incredibly useful to have a selection of aspect ratio guides for them to help frame their shot for the final look of the film. Having the option to crop in on the aspect ratio guides can also be nice as well.
3. Programmable Camera Moves
This I feel can have a much bigger impact on VRChat content creation. Being able to program how a camera moves through a shot opens up so many possibilities for virtual filmmaking and streaming. You can program how fast the camera moves, where and when it should be positioned, as well as the angles, zooming, and adjusting focus.
4. Give the VRChat Camera a Sensor Size (Or at Least Clarify What It Is)
Digital cameras typically have a digital sensor that captures the image. This can range from a variety of sizes, such as Full-Frame, APS-C, Super 35mm, Micro Four Thirds, Super 16mm, etc. Each sensor size gives you a different look, mainly being field of view.
Right now, it is very arbitrary to figure out exactly how you want your image to look because there is no set standard for the VRC camera’s sensor. The VRC Lens plugin allows you to configure the camera to a variety of sensor sizes.
Having this feature allows virtual photographers and filmmakers to have a better sense of how they want their image to look. This also translates well into the next point!
5. Focal Lengths
One of the most important aspects to creating the look of an image is its focal length, from wide angle to telephoto (ex. 24mm, 35mm, 85mm, etc.). Because it isn’t clear what the sensor size of the VRC camera is, it becomes really difficult to determine what focal length you are at and this can impact what your final look will be.
Virtual photographers and cinematographers will heavily benefit from this feature so that they have more control over how they compose their shots.
6. Exposure Monitoring Controls
On a professional cinema camera or monitor, there are often a set of exposure controls to better gauge how well your image is exposed. These controls are things like False Color, Zebras, Waveform, RGB Parade, Vectorscopes, Histogram, and even an EV+/- Scale. These are very useful for determining when your image is over or underexposed (or too bright or too dark).
This may be unlikely to be implemented as not everyone is familiar with these sorts of controls and VRChat is a virtual environment with a lot of variables. However, perhaps maybe one of these features like the Waveform can still serve a very useful purpose for any virtual cinematographer or photographer.
7. Focus Peaking
The ability to have depth of field in VRChat is absolutely stunning. However, it can be a challenge to figure out if your shot is in focus or not when there aren’t any visual cues. Cinema cameras have this feature where it will highlight the parts of the image that are in focus. This is called “focus peaking”. This can be really useful for ensuring confidence that your images are in focus. This is also a great transition into the next part about focusing!
8. Better Manual Focusing
Nowadays, autofocus in today’s cameras have gotten much more reliable. However, in the filmmaking world, pulling focus manually is still the standard because of how much control you have.
You may suggest using the Semi-Auto mode in VRChat and it does work for most scenarios. However, for more complex shots such as pulling focus while filming a shot, keeping focus on a moving subject, or you want to focus on a specific object in photography, then that’s when the Manual mode comes in. But the problem is that the only way to pull focus manually is by using the slider on the camera, which can be unreliable and very uncomfortable to use.
My suggestion is being able to manually pull focus by turning a wheel, much like how a follow focus system in filmmaking works. Having the ability to also control how sensitive the manual focusing can be will also be extremely useful. This way, the user will have much more control and better quality of life for manually pulling focus to achieve critical focus in their shots.
9. Anamorphic Shooting
I’m not just talkin’ lens flares. Anamorphic offers a special kind of look in the image that makes it feel that much more cinematic. Being able to emulate the characteristics of anamorphic can offer a truly unique look that you simply cannot get with regular shooting (we call that “spherical”). I really do emphasize “emulate”. I do not recommend an anamorphic “filter” if possible.
Characteristics such as squeeze factor (ie. 1.33x, 1.8x, 2x, etc.), the oval bokeh, lens distortion, chromatic aberration, as well as anamorphic lens flares all come together to create that sort of look.
Bonus Suggestion: Being able to control how much lens flares you want in the image (or how responsive you want the lens flares to be towards light sources in the image.)
10. Shutter Speed/Shutter Angle
In photography, the shutter speed is basically how fast the shutter moves when it takes the photo. With a fast shutter speed, everything is frozen in time in the image. However, with a slow shutter speed, you can get motion blur which allows for some unique effects like playing with light for long exposure shots.
On the filmmaking side of things, shutter speed is what allows for more natural motion blur in the image and makes it feel the way our eyes see things when objects move.
11. Film Grain
This may sound rather odd for a virtual space. However, what is great about film grain is that it adds an element of texture in the image that makes it feel a lil’ more organic and less artificial.
Having the option to choose different types of film grain, from fine grain to coarse grain, gives the photographer or filmmaker the ability to further customize how they want their images to look. This is especially useful if they are going for a more vintage look.
12. Record Video In VRChat Internally
Currently, the only way to record video in VRChat is through an external program, like the OBS streaming platform. Having the option to do so internally without one, as well as determine where the files will be stored, will really help streamline the workflow much more easily. This transitions well into the next one!
13. Recording Codecs & Framerates
Speaking of recording video in VRChat, it would be insanely useful to have the option to choose how you want your video to be compressed. It would be nice to be able to choose H.264, H.265, or even ProRes 422 for recording (but ProRes may be unlikely).
As long as the bitrates of the codecs are high enough to avoid blocky artifacts in compression, this would be nice to have. Perhaps have different versions of H.265 with different recording bitrates for how high quality you want your footage to be.
Being able to choose your framerate opens up the ability to shoot in 24fps for filmmaking or shoot slow motion in VRChat and have it playback.
-----------
Alrighties! I know that was quite a long-winded list and thank you so much if you’ve made it this far. I understand that I may be speaking for a niche audience in the VRC community. However, I do strongly believe that if VR content creation were to become much higher quality and more mainstream, then suggestions such as these can help heavily influence the direction the VR platform may eventually lead.
In the meantime, thank you all so much and I’ll see ya round!
Log In
Tony Benson
Basically meaning for the camera option to have a video recorder on there
Gerzybow
Tony Benson: Much more than that really! This expands on the current capabilities of the default camera in VRC.
The photographic side of things are also addressed as well. In fact, I can even add onto that list even more actually with some lens effects for photography. However, the filmmaking world consists of a lot of features and areas to cover while shooting video. And with VRC video content growing more and more, people may begin to ask for more additional features to help them pull off their creative ideas.
VRCLens does do a pretty solid job of assisting VRC photographers and filmmakers, but I would love to see VRChat themselves be able to heavily overhaul their current camera again to deliver and expand on what VRCLens can do.
If you have any ideas that can help add to the list, feel free to share! I really do appreciate it and would love to hear from ya.
Wolveeps
Adding on to this:
Ability of Adding Custom Lenses Through Local Only Camera Shaders.
- By allowing users to add custom local shaders to the camera. It allows the VR film maker community to add custom lenses, lens flares, tilt shift, Bokeh lens effects or any other unique camera effects in game through the shaders system. By making this part of the camera system. This will help eliminate with currant issues of Camera vs VR view conflicts that some camera shaders do have with currant avatar system methods. Furthermore sense this is local only, the shaders performance and it's effects only effect the user who installed it to there camera.
An Option to See Pickupable or Non static Object in Alpha Transparency Mode (currant Environment Removal Mode)
- As Creator, there are times that I like to use pickupable world objects for a scene, but the world/background need to be removed for special effect or layered scene effect. But the issue today is, that is almost not possible without days of rotoscoping every frame, or hours of creating greenscreen words for selective object. By providing an option to keep pickupables or non static object in the scene while the world/background is removed. This greatly speed up the process and be able open gateway for more creators do more specific effects in VRChat and enhancing the viewing experiences of the VR films.
For International Recording Idea Above by Gerzybow:
- (1) I personality like to see .Mov codec to be add to the list for it ability to save the alpha transparence layer that VRChat already has built in to the camera system. As this current time of writing this, there is no way to for OBS or any other outside recording software able to be save alpha layer to video file or ability be used outside of livestream software. By enabling us to record the transparent layer within Vrchat. This will expand and streamline VR films, skits, music videos and any complex scene to be able to recorded for special effects. Without any worry or bugs that may happen with green screens bleads on fast past scene, fine edges or odd green shaders on actors. With this, this will streamline the film process without missing with green screen removal.
- (2) Ability to choose what codec, frame rate, resolution, aspects ratios, and bitrate that are able to record in game. Ether through the quick menu - under camera tab, or by adding pro menu in the camera system itself. This will help pro VR film makers ability to save how they really want capture experience or shot with out worry of odd compression ratios in file presets options
Even though a lot of this is advance or complex features, but I do agree with Gerzybow. By allowing the community and the creators ability to expand there the toolsets more officially and streamline. This will help us to expand our creativity and spread more awareness with ease for this never ending adapting VR platform that many of us call home. Who knows there might be more VR film festivals & creative works within VRChat
catboy1357
Adding on to this, it would be nice if the camera were exposed through the animator.
As the camera is more closely related to the player, it would make sense if we had a VRC Camera Controller component acting as a proxy. Having access to the transform alone would allow creators to do very creative things, like a screen space shader ball over the camera to make their own post processing effects, letting osc control programmed movements, or letting other players grab a physbone with a high amount of immobile that has the camera attached.
I'm sure people could come up with more ideas if we had the camera settings exposed to the sdk.