@Julien-RobertLegault , I discussed this a bit with Patrice yesterday, but here’s a brief recap:
First, combining/editing clips together only works with image sequences which have been exported with the same calibration and near- and far-plane settings per sensor, as the resulting sequence can only reference one metadata file.
FFMPEG is the only only workflow we have validated to preserve the color in the depth map accurately to avoid distorted geometry in Unity. The only way to “edit” with FFMPEG is with the source image sequences (e.g. combining image sequences from two recordings into one folder, and batch renaming one of the sequences so that they share the same prefix, and that the suffix is one unterupted sequence of numbers - tedious and grueling, but it works). You can use this technique to combine multiple clips with hard cuts, extend one clip, and even reverse sections (by reversing the sequence numbers with the renamer), but that’s about it.
Alternatively, some NLE’s (Premiere, Resolve, Media Composer, etc.) may have the export options necessary to preserve the color and resolution, though we haven’t investigated or validated them ourselves. If you would like to test any of these workflows yourself, import your CPP Image Sequence into the software of your choice and put it on a timeline which:
- matches the precise resolution of the image sequence
- contains no scaling or positional transformations
- has a framerate which matches your source footage (usually 30.0fps)
When exporting, look for options that specify the following:
- Color Range: BT.709 (Full), sometimes called Rec709 - This one is critical to accurate geometry reconstruction!
- H.264’s Level 5.2 spec
- Pixel aspect ratio of 1:1 - Square
- Bitrate of at least 5 Mbps (or possibly higher to avoid compression artifacts)
To verify the color space of your exported video, use the ffprobe command in our documentation.
If your exported video meets all of the required specifications, feel free to use that NLE with those export settings to edit your footage, add audio, use effects (be careful with the depth maps!) including transiotions like dissolves and morphs, etc. Be sure to share any of these workflows so we can validate them and add them to our documentation.