Low quality in unity

Depthkit Depthkit Expansion Package for Unity
1

I followed a livestreaming tutorial to create a local livestreaming project in Unity2020.3.
However, I found that the quality in Unity seems lower than in Depthkit.
For example, here’s a part of the view in Depthkit:

image
image299×252 67.2 KB

And this one is in Unity. The object’s boundaries appear much blurrier.
image
image258×193 39.6 KB

I’ve tried adjust the Live Output Texture Maximum Height & Width, and the settings in the Depthkit Mesh Source.
Are there any other ways to improve the quality in Unity?

2

@ChengYunLiu - Thanks for sharing the screenshots.

Which renderer are you using in Unity? From what looks like dithered edges, it appears you are using the Depthkit Studio Lite renderer. This renderer is designed to run performantly on lower-spec mobile devices, and therefore makes some compromises on visual quality compared to the standard Depthkit Studio renderer.

Are you able to capture a few seconds of video (or even a couple of frames) of the Depthkit Spout stream at native resolution and share it along with the livestream metadata file generated by Depthkit? (Software tools like OBS with the Spout plugin installed can be helpful in recording troubleshooting materials like this.) Looking at the Combined-Per-Pixel data itself may help reveal where any additional quality is being lost.

Note: As of the release of the Depthkit Expansion Packages for Unity (Phase 10), we recommend using this version with Unity version 2022.3 (LTS).

3

Thank you for your help! I’ve already switched from the Depthkit Studio Lite renderer to the Depthkit Studio renderer, and the results do look better.
However, the visuals in Unity are still not as good. For example, there are more holes in areas like the chair back.

image
image348×417 27.8 KB

image
image386×405 20 KB

I’ve noticed that adjusting some settings in Unity’s Mesh Source can improve the results, but there are so many parameters that I’m not sure how to tweak them to achieve the best effect. Could you help me find the optimal settings or explain what each parameter means?

4

@ChengYunLiu - It looks like there are a few things which might be contributing to the difference in quality between the asset within the Depthkit 3D viewport and Unity:

Surface Compatibility Note: Depthkit is principally designed to capture people, so objects like the chair you are trying to reconstruct can be challenging due to surface materials which are hard for the sensors to detect (perhaps in your case the arms of the chair), and thin surfaces (the mesh material that comprises the chair back). Consider swapping your test object which follows our guidance for surface materials, and repositioning your sensors to capture multiple sides of an object.

image
image544×1088 84.5 KB

  • First, in the Depthkit app, ensure that the near and far planes for each sensor in Depthkit are set correctly. These should be positioned as close as possible to the subject/object you are capturing without cutting off any part, and is important for later steps in Unity. Repeat this for each connected sensor.
  • Next, in Unity, set the Volume Density to the highest value possible without affecting playback performance - If you are building to a mobile device, this setting will have to be low (40~60) to maintain playback performance, but if you are playing back on a higher-performance device, use a higher value (100~200) which will better retain quality.
  • Then, set the volume bounds to be as close as possible to the subject/object you are capturing without cutting off any part.
  • In the previous two steps, some geometry may flicker or disappear, so make sure the Surface Buffer is large enough to accommodate all of the geometry within the Depthkit Asset. A quick way to do this is, with the Livestream running, click the ‘Set Surface Buffer Capacity’ - This will sample all of the geometry in the asset and automatically set a surface buffer large enough for it. You can manually adjust the buffer size below to increase the buffer size for additional capacity for variable amounts of geometry in the asset.
  • Finally, adjust the Experimental Volume Settings - In particular:
  • Surface Sensitivity only needs to be adjusted higher if parts of the surface are missing, or lower if there’s a lot of extraneous geometry.
  • Depth Bias Compensation should remain close to its default value of 0.8 for Depthkit Livestreams.
  • Weight Unknown should be increased to remove extraneous geometry, like that seen in your Unity screenshot.
  • Each of the additional settings can be adjusted to see if they make any improvement, but their impact will likely be less than the settings above.

Try these recommendations, and let us know if/how the quality of the Unity asset improves.