Virtual Reality Design, Storyboarding and Animatic Tools for Stop Motion Film Productions

Harryhausen's Cyclops Puppet with VR Headset

Traditional film planning has not changed much since its beginnings. Take a piece of paper, draw a simple scene, add notes, dialogue and perhaps lines indicating motion. Essentially what is created is a comic strip of old, we just call it a storyboard when it relates to film. It's a tried and true method but lacks elements that are inherent to stop motion. Mainly that a stop motion film is full 3-D!

Why does this matter? Because a story teller working with the director has to convey what that want. Sometimes that's not easy to do with simple 2-D sketches. The process can be slow when the writer has to verbally explain something and sketch out their ideas. Then a back and forth process happens and the idea is honed sharper and sharper. But what if this whole process could be done almost instantly in VR?

Another aspect of design is the creation of sets. Again, sketches can be made on paper or digitally in Photoshop. Directors will try to figure out what should be built or what should not depending on the storyboard and camera locations. The element of distance, scale and forced perspective are all a little difficult to convey using traditional methods. Again, the process requires a lot of back and forth with the director and set designer to make it all work. Traditionally, small-scale set mock ups are done in the physical with styrofoam, poster board and other cheap materials to save money. This can all be done much quicker in VR today and can be done in any virtual scale at no cost.

What about full 3-D animatics? Some studios have pre-animated their stop motion films in CG using expensive programs like Maya. This is a slow process and very labor intensive. What if there was a way to animate virtual mock-puppets using both puppeteering instead? Or with simple pose-to-pose keyframe animation that took only seconds to achieve? It is all possible now in VR which can speed up all productions. Pop-throughs (pre-animating a scene to test the timing) with wire armatured puppets can also get a boost in longevity if virtual stop motion puppets are animated instead.

There are some prerequisites that I want to cover here before we get into actual technology and software. First, programs and tools need to be cheap! Second, these tools need to work in a more efficient and faster way than traditional techniques. If you can't create a storyboard or an animatic faster than how things are done now - it's not worth it. Luckily all the tools available are either as fast or faster than current methods. In fact many are free and everything can be bought for under 100 dollars USD! All you need is an investment into an Oculus Rift or HTC Vive system.

Tvori in use

First, let's talk about the most powerful of these programs called Tvori. Tvori is a great tool for both storyboards and animatics all in one. With this program you get a set of simple puppets that you can modify that resemble your actual puppets. They also have many props, sets, skies and elements that you can plop into a scene that you build. Basic shapes can be plucked from a palette and inserted into a scene. Doors, windows, trees can be plopped into a virtual stop motion set and scaled up or down to any size instantly. In essence you can build a mock-up of your stop motion stages within it. Lighting can also be put into scenes with different colored lights.

For directors, the camera has simple controls - one of which is the zoom level of your camera. This information can be instantly conveyed to the director of photography to match on the real stop motion set. Just share your Oculus headset with the next person to show where things need to go in full 3-D in your mock set. Testing out different ideas can be done on the fly by anyone you are collaborating with.

How to animate your virtual puppets in Tvori

In Tvori you can animate a character in two ways. One is to press your record button and puppeteer your character while the motion is recorded in real time. Another way is to use pose-to-pose animation and capture those poses using keyframes. All objects can be picked up and animated too. Or if you are only making a storyboard, setting the pose and camera to take a snap shot takes seconds to do.

Sculptor Glen Southern shows how to he uses Oculus Medium

The next question is - can you digitally create your stop motion puppets quickly and insert them into Tvori for your storyboards or animatics? Yes! This can be done with the virtual sculpting tool called Oculus Medium. So if you want to digitally re-create your stop motion puppets for use in your animatics this program is quick and simple to use. Everything from simple sculpting and even painting of the model for export into Tvori is done in no time.

Tvori allows sculptures exported as .OBJ's in Medium to be rigged for use. If you are into stop motion - this is the equivalent of a puppets armature or bones. Rigs are what you grab in Tvori and the body of your digital puppet sculpt follows where they are moved. The work flow looks like this:

2-D character design is made --> 3-D digital re-creations of the puppet are made in Oculus Medium --> Digital puppet is rigged for use in Tvori --> Digital puppet is posed for storyboards and animatics in Tvori

This video demonstrates how Medium sculpted characters can be rigged for Tvori animation.

Google Blocks and Tilt Brush in use

What about set designs? Two programs that are great for use in tandem are Google Tilt Brush and Google Blocks. Tilt Brush should be used as the main method of getting the feel for a set and scene through the colors, lighting and tone of the virtual 3D brush strokes. Tilt Brush allows you to walk into a scene and view it at any angle.

In Tvori you are getting more in tune with the actual action and timing of the characters and shots with roughly designed sets. Tilt Brush is best used in the concept stage where ideas are pitched early on. It can replace Photoshop with a 3-D version that may best suits stop motion productions. Whatever is created in Google Blocks or Oculus Medium can be exported as .OBJ files and imported into Tilt Brush which is perfect for visualizing spaces. You paint your set with brush strokes in 3-D which gives a sense of depth. Lights, fog, fire, twinkling stars, smoke and simple geometric lines can be quickly applied to the space you are in. Forced perspective aspects that are unique to stop motion can be plopped into a scene and different scales tested from different view points.

Once you complete a design in Tilt Brush, the files can be shared with other people privately over a network and on 3-D social sharing sites such as Sketchfab.com. This allows smaller productions outside a studio setting to send ideas in 3-D to be turned and examined by set designers and directors. Within Sketchfab, anyone who views the design can also do so in virtual reality.

One bad aspect to pre-building a set in VR is translating the measurements to the actual sets. I would recommend using any digitally sculpted puppets you make in Oculus Medium be imported into Tilt Brush for the sake of scale. This is one way of measuring how much of a set needs to be built which currently is a little difficult to translate from VR into the real world. Creating measuring sticks and rulers digitally can help if you know the height of your stop motion puppet. Just create an elongated block in Google Blocks and put it next to your stop motion puppet. If the puppet is 12 inches tall you can then duplicate your stick height digitally and lay them across your painted design to translate into the real world applications.

Another point to make here is that there are several virtual ways to build your sets. If you plan to use Tvori for animatics or storyboards then the importation of low poly geometric set pieces from Google Blocks is going to be a great solution. If speed and performance are what you want, this is the fastest way to go. Simply build your mock set pieces in Google Blocks and export them as .OBJ files (if needed) for importation into Tvori. You can then plop them into your scene and scale them or rotate them into place with your digital puppets.

Flipside Studio in action

Finally there is Flipside Studio. It's a VR application that allows an animator to act out scenes through virtual avatars. These virtual avatars can be digital reproductions of actual stop motion puppets they will later animate. Will Vinton Studios was famous for hiring actors to act scenes out so animators could go through frame by frame and match the movements. Flipside Studio allows them to do the same by using exported videos made in this program and then imported into stop motion capture programs like Stop Motion Pro or Dragonframe. This solution is a little more complex and requires the use of the Unity game engine as well as digitally sculpting and rigging puppet mock-ups. However it is worth mentioning as a potential pre-production tool. This video explains the process of importing custom designed sets and assets.

Hopefully this gives some insight into the many ways virtual reality can potentially speed productions up. While hard core stop motion animators of old may scoff at these new tools, they are things that in my opinion can help rather than hinder the future of stop motion animation.

Links:

Oculus (now by Adobe) Medium https://www.oculus.com/medium/

Google Blocks https://vr.google.com/blocks/

Google Tilt Brush https://www.tiltbrush.com/

Tvori http://tvori.co/

Flipside Studio https://www.flipsidexr.com/