Hi, is there any way (or hack) to export the raw point cloud view look of a capture? To a 3D format that I could use in Unity? Thanks! Michel
Hi Michel @doubleA
There are a few ways to playback Depthkit as raw pointclouds in Unity.
I’ll acknowledge first that the “right” way to do this for Scatter to implement a “Look” for pointclouds in our Unity plugin, it’s something we have played around and a few customers have asked, but haven’t prioritized it
In lieu of an official solution from us, here are a few hacks that can get you there:
Option 1: Modify Depthkit Core to draw Points instead of Triangles
This quick approach is how we have achieved this before, it’s easy but has the drawback of not rendering all the points at full density.
- Ensure you have Phase 8 (depthkit.core package version 0.12.2)
- Apply the Depthkit Core Look (not Studio Look) to your incoming clip. It should visualize all perspectives as separate meshes with correct calibration.
- Under Packages/Depthkit Core/Runtime/Looks open ProceduralLook.cs
- Change line 48 from MeshTopology.Triangles to MeshTopology.Points
- Your render will show up as a point cloud!
- Make sure you set the Mesh Density to Full in the Depthkit Core Mesh Source component settings
Option 2: Export OBJs and compress them into Alembic sequences
This one requires some tricky 3D workflows, but no shader hacking
When you export an OBJ of a take captured with a Depthkit Studio calibration, the calibration is applied to the origin of the OBJ. You can export each perspective from a capture, then combine all the OBJS together into an alembic sequence and render it as a point cloud.
Check out this tutorial on calibration validation that explains the OBJ export process with Depthkit Studio captures
This approach will require some complex 3D trickery but will give you all the points and control over the geometry.
Let us know how it goes and we can continue to work on a good solution for this.
2 Likes
Great, thanks James!
Hey James
Keen to do the same look with Studio Lite - Am I understanding correctly that you change the code in CORE then set up your studio lite as usual and because it is dependent on core it will automatically apply the cloud point look to studio lite? Or can this only be achieved in CORE?
Never mind - it worked - and it is incredibly cool!!!
1 Like
May have spoke too soon. Looks great in unity but when I built to mobile the points are huge - there was a warning that there was no specification of sizes of points in the project - any ideas?
Hey guys any ideas on the above?
@CoryAllen any thoughts?
Keen to get mesh look built to mobile with studio lite!
Hi, @RobertGraham. Unfortunately, I am not familiar enough to help with workflow to generate particles in the Built-in Render Pipeline, and was hoping someone else could weigh in with an answer for you, but I’ll ask around and see if I can get more information.
Ok please let us know at a bit of an empasse at the minute until we can sort this out
Cheers
R
@RobertGraham Is this project explicitly constrained to the Built-in Render Pipeline? If you’re able to switch to a scriptable pipeline like URP, it is much easier to achieve a similar look with our VFX Graph integration.
Hey Cory - we were trying to achieve the look with the studio lite pipeline in order to keep the multi cam assets as performative as possible on mobile devices. VFX graph plus studio assets tends to crash straight away
@RobertGraham I’d like to better understand what’s causing the URP VFX Graph project to crash. We have seen other users successfully render Depthkit Studio assets performantly as VFX particles in AR on mobile devices - See this example from @DaneChristensen - so it should be possible to accomplish your desired look this way.
Some top-level considerations: Start with a minimal (30) Volume Density in the Depthkit Studio Mesh Source, and ensure your Volume Bounds are pulled into your subject, set your surface buffer, and see if the build still crashes.
- If it doesn’t crash, you can incrementally increase the Volume Density until you see the performance drop.
- If it does crash, please share any console errors you are seeing.
Hey Cory,
Yes we have developed AR assets for mobile with VFX graph also here - https://southpawdancecompany.co.uk/download-southpaw-digital-app/
I think it is with the multi cam and the number of particles we would wish to animate in order to achieve the desired effect - the point look from the Depthkit preview / calibration view is the perfect look for our project - We will revisit if the point look isn’t accessible in LITE but if it was would be much preferable given the duration of our captures and the capacity to then incorporate multiple captures into a scene
Let me know if you think there is a fix - If @James has seen this work previously and there is a known workaround it would be amazing. Or if you guys can work into a lite feature even better
R
Worth saying the point cloud view works perfectly in Unity, only on export it is going sideways!
Just wanted to say - gorgeous work, Robert!
Thanks Catherine - works in progress. It can check our other work here -
1 Like
So beautiful. Is this a Unity or native app?