Morphing 2 Clips Together

(Max Ellinger) #1

Hi DK!

Say I have a clip of a character idling and a clip of a character saying something, and I want to go into a sequence like:

Idle > Action > Idle

Or even

Idle > Action > Walk somewhere > Idle

If those were shot as 3 or 4 separate clips, is there a way I could morph them together to create the illusion of a single shot? The aesthetic is already pretty abstract so I’m not concerned with photorealism or melting faces at this point.

(Alexander Porter) #2

Hi Max! Great question.
There are a few ways to create transitions between two Depthkit clips in Unity.

If you’re intending to seamlessly stitch two clips together so that you can load them as a single clip in Unity you are best off doing that in a NLE like After Effects or Premiere. There are three primary ways to do that.

  1. The simplest way is to simply crossfade the two clips with each other. The result will look like a volumetric version of a crossfade where one dissolves into the other. If there is a good overlap between the two clips spatially – they’re in the same location – it can work quite well. If they don’t overlap well it can look a bit ghostly.
  2. The more complex, automatic way is to use a morph cut in Creative Suite. It will analyze the cut and attempt to morph between the two clips automatically. This works surprisingly well if the two clips are not too dissimilar. But sometimes it just fails outright. Be aware that it doesn’t know the difference between the depth and the color, so sometimes it might transition one effectively but not the other.
  3. Then there’s always the manual and fussy option – in this case it is to manually morph one clip towards the other.

Always be aware of your export settings when re-encoding Depthkit clips to avoid introducing artifacts in Unity.

If your goal is not necessarily to stitch them together, there is a dynamic way to do it that makes use of Depthkit Looks. If your idle loop uses the photo Look, when you need to trigger a transition into an action, you fade the Depthkit clip into particles then reduce the opacity, while simultaneously starting your second clip and fading it in using the inverse transition. This is the more technically complex approach because it would require you to build the transitions in code.

These techniques are useful for loop-closure in order to create an idle loop. In order to do that, you would put a single clip end to end in the timeline and create the transition in and out of itself.

These techniques can also be used if you’re trying to morph in or out of an idle loop to a specific action like you asked (e.g. Idle => Action => Idle) where each of those is a different Depthkit video clip. In that situation, you’ll not always be guaranteed that the action will match if you trigger it dynamically – especially if you have a very lively idle loop – but it can allow you to film your actions piecemeal and stitch them together dynamically in the scene. To do that, you would create a transition on your Action that starts in a general/central pose in your Idle loop and morphs into the Action, then back into the idle loop again at the end.

The way you would accomplish that is to lay your Loop clip, then your Action clip, then your Loop clip again in a timeline and create transitions at the cuts. Then you export only the Action clip which should now have transitions baked into the beginning and end.