Adding a little to what @MartnD shared for specific considerations and guidance on shooting with instruments
First, for the sake of other readers on the forums, here is the double-bass example you are referencing. This was captured with 6 cameras during or Museum of Moving Image “Tales of the Holoverse” project. then processed to WebXR using Arcturus HoloEdit and HoloStream.
The Problems
-
Thin surfaces with small details and complex occlusion are difficult to reconstruct instruments often have very thin details like the neck of the bass, or a trumpet’s woven tubes. These details often are much smaller than the rated error levels of the Azure Kinect’s sensors, or the density of the reconstruction volume once captured, leading to them turning into noise or disappearing entirely. If these details are tightly packed, they often will occlude one another from the camera’s view leading to further difficulties.
-
Shiny Surfaces reflect the infrared lasers used for depth sensing The Azure Kinects uses active lasers emitted front he cameras to sense the surface. These light signals work best when hitting matte surfaces, and get disrupted when hitting reflective, transparent or very dark surfaces that absorb the light from returning to the camera. In these cases, the depth data streams will just read as blank, further challenging the reconstruction.
The Solutions
-
More cameras is better with props like instruments When shooting subjects instruments, we strongly recommend increasing the number of cameras from 6 to 10. This helps mitigate both issues above. First shiny surfaces will often pick up differently when seen from different angles, giving a better chance to recover the instrument when more cameras are capturing the subject from a variety of angles. Secondly, more cameras will help avoid occlusions allowing smaller details to be captured.
-
Experiment with surface treatments We’ve had a lot of success using matte spray like dry shampoo on difficult materials like instruments and human hair. We touch on this in our wardrobe recommendations.
Test!!
We recommend doing some low stakes practical tests with the instruments prior to capturing a critical performance. Having a chance to try different camera placements, matte treatments, etc, can significantly increase the quality of the final production.
Hope this is helpful! we’d love to hear from any community members who have also worked with musicians and instruments to hear what has worked before.
Best