A problem with Image Sequence Player

Hi Everyone,
I discovered the DK Image Sequence Player and can see that it could solve a bunch of synching problems I’ve been having with some of my outputs, so I installed it using Unity version 2020.3.24f1 along with the other associated packages. It immediately threw up some errors that I’m not sure how to fix and I was wondering if you knew what I was doing wrong:

C:\AV_Weaps\Depthkit\Depthkit_Studio_Expansion_Package_Phase8_030722\depthkit.streamingimagesequenceplayer\Runtime\StreamingImageSequencePlayer.cs(201,25): error CS0433: The type ‘GammaCorrection’ exists in both ‘Depthkit.Core, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null’ and ‘Depthkit, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null’

Any help from the group is most appreciated!
Keith

Hi @KeithJones ,

This shader error is likey due to conflicts created when upgrading Unity package versions or accidentally adding multiple instances of the Depthkit Unity SDK

Please first confirm that you only have one version of Depthkit Unity expansion package included in your project. If so, the error may be cleared by right-clicking your packages and selecting “reimport all”

Failing both of those, start a clean project and import the packages just once into the project.

Let us know if this helps!

Hi James,
Thanks for getting back to me. This is actually the fourth time that I’ve rebuilt my project after believing I was doing something wrong. I did, however, upgrade the first time I did this, but each time afterwards I’ve gone from having a totally clean project with the new expansion package. I just tried the “reimport all” method you suggested but had no joy.
This error only occurs when the Image Sequence Player and Studio Mesh Sequence are installed, once removed all errors disappear.

Have there been any changes in the way or order that I’m meant to install the expansion packages?

Cheers
Keith

Hi @KeithJones

I just tested on my side and verified a clean project I get no errors adding Depthkit Core and Depthkit StreamingImageSequencePlayer

Steps I took:

• Download a clean version of the latest expansion package from Depthkit :: Downloads
• Create a new project in Unity 2020.3 - one slight difference is I’m using .39f1
• Import Depthkit Core via Package Manger > Add Package From Disk
• Import Depthkit Streaming Image Sequence Player via Package Manager > Add Package From Disk
• No errors observed

The following screenshot shows my configuration

If you can follow those steps and still see an error, perhaps upload a screen recording so we may be able to spot the discrpency.

Thanks!

Hi James,
Thanks for this, it’s super helpful. My footage was shot using multiple sensors and works using your Depthkit Studio Expansion. The set-up you’ve kindly put together is based on your DK Core settings, right? After setting up as you’ve shown me should I then install my Studio Expansion elements?

Cheers
Keith

Hey @KeithJones

Sure no problem. After importing the Core and Image Sequence Player, you should be able to import the Studio package next.

I’ve updated the screenshot to show Studio components, again with no errors on my side from a clean project.

The entire dependency tree can be found in the documentation here:

Wicked! I’m gonna get neck-deep into this when I get to my other machine later. I’ll let you know how I get along.
Thanks so much!

Great! Let me know how it goes, if you encounter any errors post a screen grab or a screen recording of the steps to get there and we’ll be sure to sort it out together.

Hi James,
Sorry for the delay in getting back to you. I did what you suggested and got the image sequence plugin installed, I then went on to install Studio, which was fine too (nice one!). But when I tried installing Studio Shader and Studio VFX I got the errors in the screenshot and failed the installation.

Cheers
Keith

@KeithJones Have you installed the Core Shader Graph and VFX Graph plugins prior to installing the Studio Shader Graph and VFX Graph ones? See this dependency chart to make sure you’re adding them in the proper sequence.

Can you also confirm which render pipeline you’re using? The Shader Graph and VFX Graph looks only support the scriptable render pipelines (URP & HDRP).

Hi Cory, I can confirm that I’m using HDRP. I followed the steps in the dependency chart which showed no errors in installing them. I wanted to check that just the normal DK studio clips were working so I added the Depthkit Clip + Studio VFX Look and added my clip to it and it was still cool. But when I added the metadata file it got VERY grumpy

I then just tried with the Depthkit Clip + Studio Shader Graph Look Prefab and it didn’t like that either:

This is a totally blank canvas project with just one clip in there so I can test before I migrate everything over. This is also about the 8th time I’ve had to rebuild the project and has become an incredibly stressful affair. If you guys weren’t around and so helpful in offering this technical help I think I’d have to consider throwing in the towel with DK.

@KeithJones Can you share the clip, metadata, and poster file you are trying to render? I can try to load it on my side to see what’s causing the error.

Hey Cory, I’ve sent a short clip along with the image sequence via wetransfer.
Cheers
Keith

@KeithJones , I was able to load your clip into a fresh HDRP scene without any console errors, so I am not sure what’s happening on your end. Here are a couple of observations that may help:

• The resolution of this video is very high, beyond the spec of h264. Some players and playback hardware fail to play videos of this size. As of Depthkit v0.6.0, the multiperspective video exporter now includes automatic scaling and stacking to keep the output resolution within 4096x4096, eliminating the need for workflows like encoding with FFMPEG and our row-stacking Python script. Alternatively, you can use our published FFMPEG workflow to re-encode your image sequence into a video that is scaled down to this spec:

ffmpeg -r 30 -f image2 -start_number 0 -i RODNEY_standing_rear_08_26_15_43_00_Export_09_30_09_48_19_%06d.png -c:v libx264 -x264-params mvrange=511 -vf "scale='min(4096,iw)':'min(ih,4096)':force_divisible_by=2:out_color_matrix=bt709:out_range=full,setsar=1:1" -colorspace bt709 -color_primaries bt709 -color_trc bt709 -color_range pc -b:v 5M -pix_fmt yuv420p output.mp4

• The following tranforms needed to be applied to move and rotate the asset up above the floorplane:
  
  

  image980×283 28.7 KB
• I also noticed that the near- and far-planes for this asset were set well outside the volume that your subject occupies, as evidenced by the depth maps of each sensor only containing yellow-green hues - They should use more of the hue spectrum, with the nearest pixels in the orange range, and the furthest in the purple range. Pulling the near- and far-planes in prior to exporting out of Depthkit will not only improve depth fidelity once the asset is reconstructed, but will also reduce the default volume bounds size. The default bounds are currently defining a volume that’s much larger than needed by default, and that can cause issues on some systems. Be sure to pull in the bounds to better fit your subject’s performance, and set the Surface Buffer Capacity once these changes have been made.
  
  

  image1014×1364 124 KB
  
  
  

  image914×1088 95.9 KB

My recommendation is to try a variation of the asset that has the near- and far-planes brought in, and is scaled down within h264 spec and see if that produces the same errors. Let us know how it goes.

Hey Cory, thanks for this.
I know of the issues you flagged with the video as this was the very first (and only) multi-sensor capture session I was able to do before I ran out of budget for DK multi-sensor outputs. I also got this up and running without issue, apart from lip-synch. This was why I wanted to use use the image sequence player (the image sequence is also in the wetransfer), this was where I was getting all the errors. Were you able to import this too?

@KeithJones I am looking at this now. Can you confirm which version of Unity’s Streaming Image Sequence package is installed in the project? Unity v2020.3 only supports 0.16.0-preview, which I was able to add using the Git URL: com.unity.streaming-image-sequence@0.16.0-preview

With the right version of Unity’s Streaming Image Sequence package, I was able to play the clip using the image sequence player without any errors.