Unity Timeline playback

Depthkit Integrations
1

Hey there,

I’m running in to some timeline playback issues -

So playing the Depthkit clip back directly I have no problems. E.g. just play on awake.

But then hooking up the clip into a timeline following the instructions here the clip stutters/sometimes doesn’t play at all (On MacbookPro).

Is there something that anyone has come across? Is it down to the video codec? Any advice on what the best encoding is for working with timeline?

Thanks

2

Hi @JonathanForder2 , great to hear from you again!

Sorry to hear you are having trouble with the timeline playback.

Can you provide a few more details to help us narrow down what could be causing the problem:

  • How are you encoding the videos? Are you exporting videos from Depthkit, or using FFMpeg to encode from image sequences, or another way? Let us know the encoding settings if you are using something other than Depthkit.
  • If you just play the video back as a 2D texture applied to a Material on a quad, do you see the same issue? This will help understand if its an interaction with Depthkit playing or a result of the video timeline playback independently.

It may also be worth trying AVPro 2’s timeline system instead of Unity’s Video Player. You can see how we set that up here:

There is a also a little further discussion here on how to set it up

3

Thanks James, good to hear from you too!

Looks like it was down to some encoding, I’m picking up an old project that had some rouge videos in there. After a re-encode the timeline is playing ball again.

Thanks for sharing those links, useful for future reference.

p.s. encoding I used below - in case I end up here again :slight_smile:

ffmpeg -i input.mp4 -c:v libx264 -pix_fmt yuv420p -profile:v baseline -level:v 3.0 -b:v 5M -maxrate 5M -bufsize 10M -r 30 -g 60 -keyint_min 60 -x264opts no-scenecut -colorspace bt709 -c:a aac -b:a 128k -ac 2 output.mp4
4

Might need to backtrack on what I previously posted! Seems I am still getting the same results after some more testing.

I setup a quad video timeline and that is playing back as expected (Playing smoothly).

I’m using the Depthkit Studio Lite (built-in RP) for my assets but also tested on studio main and getting the same stutter results in timeline.

With the AVPro option is there much of a difference? Can I use the free trial to test beforehand as it’s a pricey asset.

Looking over the profiler as well and can’t see anything to out the ordinary.

5

Hi @JonathanForder2

Thanks for conducting the further tests. If the video playback is smooth on a quad, and then stuttering on Depthkit, it would indicate an issue with some resources getting overwhelmed such as GPU memory or pixel fill.

As a test, can you try reducing your Studio Lite Mesh Density to the lowest setting, and having it show just one perspective via the Perspective Limit set to 1? If this results in smooth playback, then it shows that it’s a Depthkit optimization problem.

Do you see the same issues in build as in Editor?

Is your target delivery platform macOS, or are you using a mac to develop for iOS/Android or other mobile platform? If the latter, what is your target platform and do you see the same results when building to there?

Can you confirm that your framerate is running smoothly (60 - 90hz) even when there is stuttering in the Depthkit capture?

The AVPro test with the free watermarked version may be helpful as a test to see if it resolves the issue, then decide if it’s worth the price of upgrade once you confirmed it works.

Let’s hope some of these additional tests/info help us get to the bottom of the issue.

6

Hey @James

So a bit more digging, here is what I’ve done so far -

In a clean scene I have a timeline version of the clip and a play on awake plus a sphere doing some ping pong lerping to test for performance.

Each clip is on the lowest settings, low mesh density, disabled any filtering, and downscale is low.

Profiling -
Run the play on awake clip, I get between 60 - 70 fps. Looking at the profiler everything is happy.

Then switch to the Timeline version - fps drops to around 15-30~, I get big spikes in the editorloop in the profiler (30 - 60ms). And the clip is stuttering, etc

Building to iOS (ARKIT) running on an iPhone12ProMax, play on awake version is running smoothly, timeline I get the same stutter results.

I can run the timeline sequence on my PC but that has a A4000 graphics card so wouldn’t be surprised! So my guess is that there is something on the graphics side on the macbook pro / iOS that doesn’t have enough GPU processing power to run via timeline.

I’ll trail the AVPro free version and see if I get any issues there.

7

Update: Tested out the AVpro plugin and getting smooth playback from the same clip/asset.

Now do I take the leap or work out why Unity Video timeline doesn’t work… :sweat_smile:

8

Hey @JonathanForder2

Great sleuthing. The Unity Timeline issues is likely a result of some frequent seeking during playback - something that AVPro likely is smarter at avoiding.
There may be ways of avoiding the issue with encoding, such as all i-frames, but you’ll need to test on your side.

FWIW in our opinion, AVPro 2 is a great product and worth the investment.