Replacing recorded RGB footage with upscaled / different resolution

Hello @James and everyone else,

I have been experimenting with integrating NVIDIA’s new AI SuperResolution tools into my Depthkit workflow to test out ways in which I can improve captured texture quality.

I have been testing the SuperResolution Upscale and Artefact/Noise reduction to create 3840x2160 textures, from 1920x1080.

Th epipeline is working nicely, but I wouldlike to know how to update the DK project file so that it expects the larger resolution footage - as currently it is misaligned with the Depth footage.

Cheers,

O.emphasized text

@OliverEllmers I haven’t swapped out record media with different resolutions before, but this is what you would need to do:

• First, for each sensor you’re using, generate an “auto-calibration” profile for the desired resolution in the deviceConfigurations.<serial number>.sources.calibrations portion of the JSON. If your target resolution is supported by the Kinect, you can generate these simply by making a short recording at that resolution. If the target resolution is not supported, you’ll need to duplicate and modify an existing auto-calibration object, and extend out any of the differences between the properties of the other profiles to match the new resolution, and give it a unique name like “auto-calibration-7680x4320”.

Annotation 2023-01-23 111157611×1195 88.3 KB

• Then, you need to apply this new “auto-calibration” to your recording under the recordings.<recording name>.streams.<serial number>[0].calibration - The ‘0’ index is the color stream object.

Annotation 2023-01-23 111206771×534 50.5 KB

Let us know if this works, and what your results look like!

Hi @CoryAllen ,

Thanks for this!

I have followed your instructions above, but the editor keeps crashig when I go into edit mode. I am seeing a ‘render dimensions not supported’ error or something of the like in the console window before it crashes if that is any hint?

json file

I have attached a link to download the project config .json file below for you to take a look at in case I am missing anything important?

image1783×860 77.4 KB

Cheers,

O.

@OliverEllmers We took a look at the JSON you provided, and it looks like there’s only an "auto-calibration-3840x2160" object for serial number 000014201312 - The other 9 sensors also need auto-calibrations for this resolution. Try making a very short Studio recording using all 10 sensors set to 2160p, and that should generate the calibration profiles for all of them.

@OliverEllmers Following up - Were you able to generate the auto-calibrations for the other 9 sensors, and did this eliminate the error?

Hey Cory,

I was able to do it in another project we were doing this week - but haven’t revisited the upscale pipeline yet as I have been a bit busy. I am off to NZ for the month, so will need to pick up again on my return. Hopefully it can help to increase quality in future pipelines!

Cheers,

O.