An introduction to virtual production

Virtual production is revolutionising the way film and video content are being created, providing far more scope for innovation, rapid development, reduced costs and a sustainability benefit. Nor is virtual production just for movies, advertising and AV pros: it opens the door for more organisations to explore the visual arts to engage with employees and external audiences.

Accelerated by the pandemic, virtual production is fast becoming mainstream. It is expected to be a huge growth market in the coming years, with multiple studios and vendors already on board. According to research from Technavio published in April 2022, the virtual production market share is expected to grow by USD 1.85 billion from 2021 to 2026, and the market's growth is anticipated to accelerate at a CAGR of 16.85%.

Virtual production has been around for a few years, with probably the most famous milestone being the release of the Mandalorian in late 2019. In addition to wowing viewers, that film also made a massive impression on film and video professionals who realised that by combining software with on and off-set components, the only limitation now was the imagination.

Then came along the pandemic, and to carry on making films, TV ads, and other video content, more studios turned their attention to virtual production. One of the earliest pioneers was Final Pixel, a global creative studio founded in 2020 by film & TV industry veteran Michael McKenna, his brother Christopher, and executive producer Monica Hinden. Today, the studio has an impressive virtual production track record, including projects for Shutterstock and Dancing With The Stars.

On its website, Final Pixel states “We believe that we are on the verge of a technological shift of greater magnitude than the move from celluloid film to digital cinema. The advantages to virtual production are so numerous it is inevitable that a significant amount of film and television production will soon become virtual.”

Big benefits

The benefits of virtual production are powerful. Barriers to creativity are removed (as Final Pixel says on its website, ‘If you can think it, you can build it’.) Also, as the crew can see in-camera VFX, post-production time is saved, and it becomes easier to iterate, with revisions made earlier in the process while actors are still on set. This leads to less rework and reduced post-production effort. And in post-production, since virtual production supports remote contributors, team members and vendors from literally anywhere in the world can access assets to work on.

Multiple ‘locations’ can be covered in one day (simply swap out the background), and what the film industry calls ‘the golden hour’ is no longer dependent on whether or not there is natural light. The turnaround between virtual locations can be fast, with less travel and expensive physical sets required. Furthermore, compared to physical sets, virtual set components can be reused time and again.

Plus, virtual content can now be created by decentralised teams around the globe without delays. Filmmakers can build up their teams with creatives and technical specialists anywhere they are located. The possibilities widen without increasing the cost.

Virtual production seems a natural for film and advertising studios, but it also opens up the potential for businesses to be more creative with AV. For example, create more engaging training material for employees, communicate content to customers in a visually compelling way, and even launch a new product range.

Virtual production components

Virtual production has multiple elements, but the main three are camera tracking, an LED wall, and a  game engine, which runs on a high-specification PC to render and animate a 3D model in real-time. Games engines have become the bedrock of virtual production because the game industry’s use of high-quality 3D rendering was ahead of anything else. Unreal and Unity seem to be the most popular for virtual production so far.

The environment is played from the game engine software and displayed on the LED wall. Each camera’s position is tracked in 3D, changing the perspective in the video game environment displayed on the LED wall as it moves. When done well, the net effect is that the viewer’s perspective changes and everything is related, creating a realistic effect.

Data-dependent

Virtual production depends on talented people, great ideas, high-quality cameras and LED walls, and choosing the right game engine. However, given its digital nature, virtual production is also very dependent on managing data assets, of which there could be a vast volume dispersed across multiple locations.

Having simplified and universal visibility and control over all these assets is vital. This is why several techniques and tools well-established in the enterprise IT world are now being used in virtual production.

For example, adopting a hybrid or cloud model will make it easier for teams to quickly access virtual production assets they need without the wide-area network (WAN) wait that can easily create delays. Cloud provides file access, but virtual production teams also need data management tools: putting files into Dropbox is not sustainable in this environment. Data asset management tools (such as Helix DAM by Perforce) secure assets and assist with collaboration.

In addition, version control is used to make it easier for teams to catalogue assets and track their evolution, regardless of where people are located or what tools they use (such as Photoshop, Maya, and 3DS Max to build visual assets).

For Final Pixel, version control can also dramatically speed up real-time changes to game engine files. For instance, the team can be on-set, and the director says they need an object in the background to be moved. An off-set artist can make the change to the file, sync the file change to Helix Core by Perforce (the version control system), and then render it on the LED wall within minutes. Final Pixel’s team of remote contributors can work with the on-set team, making it feel like everyone is in the same room. With the changes loaded, they can keep on filming.

Another helpful category of tools is project management, which can be used to keep the objectives on course and people on task. Global artists, supervisors, producers and engineers can be united to help track, manage, discuss and review visuals in real-time.

While virtual production has many benefits, there is also much to take on board, representing a learning curve for teams. Fortunately, there is an increasing number of knowledge resources, including Perforce U College of Virtual Production, a free online resource of training content. Again, there may be a lot to learn, but given the potential benefits, especially the creativity that can be unleashed.