Smooth DK clip playback on Quest 2 using Unity

Hi everyone,

has anyone got advise for smooth playback for DK clips on the Quest 2 using Unity? I’m aware there’s the post from Jillian here (Depthkit for the Oculus Quest).

As far as I know I have most of it set up as directed by DK, Oculus etc. and I’ve found using Vulkan helped my playback a lot. I’ve got it working and my clips run smoothly to a point, but then freeze for a moment, and then continue. I need to get rid of this staggered playback.

I’ve tried a few Garbage Collection tweaks (forcing collection every 30 frames in Update) which made no difference, after a look in the Unity profiler I think it may be due to the Video rendering needing the next frame…?

The clips were shot on a Kinect Azure. I’ve crunched down the clips as modest 720x1200 mp4s using AVC x264 and the Depthkit quality is set to “low” in the Editor but I’m running out of ideas for getting this working.

Anyone been able to get DK clips running smoothly on Quest?

Thanks

1 Like

I’m wondering if loading DK clips from StreamingAssets using WebRequest to set the URL would help? Maybe I’ll try that…

Hi @Psicon_Lab, thanks for sharing these tests. If using Depthkit Plugin v0.2.7 you are doing all the right things with reduced mesh density and video resolution. We have not yet tested if loading assets from StreamingAssets will improve performance specifically for the Quest 2.

I’m curious, are you trying to run many Depthkit clips at one time?

We are committed to optimizing performance in future developments for the Quest and we look forward to keeping you posted!

Hi @jillianmorrow thanks for your support. No, this is a single depth kit clip in an empty environment.

Actually, I have had DK clips running smoothly on Quest before but only single clips, one at a time shot with the older, lower res Kinect V2 so seems linked to the slight increase in quality of the clips from the Azure.

Would a change in Render Pipeline in Unity help maybe?

I’m starting to think this is all about video playback performance on the Quest 2…?

Ok, for the record, I’ve just built for Quest2 with 6x DK clips running simultaneously and playback was smooth. The framerate was low but I did not experience the staggered playback I’d seen before (which is bizarre as I used an almost identical project, but anyway it’s clearly something in my project that’s causing issues).

I’ve found that tidying up the clips in post and removing some unwanted background has had big performance benefits. I’m guessing the mesh is more cohesive and spans a lot less physical space if there are no random artefacts pulling it out of shape here and there.

1 Like

@Psicon_Lab ensuring you are applying a crop within the Depthkit Refinement workflow should help, in addition to reducing the clip resolution. For additional support, let me know if jumping on a screen share or call would be helpful. That way we can discuss your asset specifics.