Hello! This is Eugene Kuznetsov currently writing. I work as the manager overseeing the Emerging Media Lab where we have a 5 Azure Kinect sensor volumetric capturing setup. I hope you have been doing well. I am writing to get some insight about an issue I am seeing with some of our Azure Kinect sensors and/or troubleshooting their connection to our capture computer.
The video feed on some of the sensors (usually one or two) ends up briefly freezing/stuttering/glitching out rather randomly. I see this when looking through them individually through Depthkit Studio or the Azure Kinect Viewer. Sometimes, all the video feeds are fine but we may still see intermittent flashing/glitching on the actual spatial capture when recording. Usually we swap which USB port the Azure Kinects are plugged into (we have them all plugged in with USB-C to USB 3.0 cables, into Tripp Lite ASB-A 3.0 Extension Cables). We have a Precision 3650 computer that was specced out accordingly, originally for a 5-sensor setup. Specs provided below:
I have noticed that the feed glitches more regularly/frequently when plugged into the Startech’s USB ports, and as such we have been avoiding plugging any of the sensors into it. None of the old Renesas drivers are installed, and I’ve reinstalled the drivers for that USB expansion card, to no avail in addressing the glitched feed issue. The other USB ports usually work without issue when examining the sensors one-by-one. We start noticing the sensors intermittently glitching out during Streaming, while recording has also been dropping frames whether the sensors are set to 1440p or 1080p. Previously we have not seen frame dropping with this resolution mode set. Thank you for your time and help.
@EmersonCollege Apologies that you’re experiencing this issue, and thank you for providing all of these details.
A visual glitch is usually the result of an incomplete or torn frame from the sensor, and usually indicates a bandwidth constraint somewhere upstream. Are you able to record this artifact and identify it in the recorded media to verify which sensors the issue is affecting? What usage percentages are reported in Windows Task Manager for CPU, GPU, and Memory? Are there any messages in the console log which may help us identify the cause?
Have you been able to narrow down the issue to specific points in the signal path by bypassing or swapping components and seeing if the issue follows a particular component across changes? (E.g. sensor serial number(s), factory USB cable, Tripp Lite USB extender cable, USB ports, PCIe slots…) Though from what I can find on the Dell, it looks like there’s only one PCIe x16 slot for the GPU and one PCIe x4 for the StarTech card - Is this correct?
@EmersonCollege - Following up to see if you’ve been able to narrow down the cause based on the above guidance. Let us know if this issue persists - Otherwise, we will move to close the ticket.
Thank you for the follow up, apologies for the delay. We had another 5 sensor setup later in the semester (April 20th). We did not seem to encounter the feed glitch issue; however, we had been encountering a separate ongoing issue where either the application freezes (and crashes due to a sensor sync issue) or the entire computer freezes, forcing a hard reboot. This can occur whether during calibration or, at times, in the middle of recording. I’ll follow up with screenshots/logs; however, I do want to rekindle this thread in the event this issue had been seen elsewhere. The freezing issue has been one we’ve encountered consistently, though hasn’t changed with new OS builds or versions of depthkit Studio. Thank you again for your time.
@EmersonCollege Unfortunately, we can’t provide any more guidance on the glitched video issue you were experiencing until you have tried narrowing down the issue using the methods listed above and reported back which recordings are affected, which resources are reported to be at capacity, and if the issue follows a particular hardware component when you try different combinations. If you are able to provide any of this information, we’ll be happy to take another look.
I’ll also keep an eye out for a post about the other isssue and its supporting materials.
...
libusb: warning [handle_timeout] async cancel failed -5 errno=0
libusb: warning [handle_timeout] async cancel failed -5 errno=0
libusb: warning [handle_timeout] async cancel failed -5 errno=0
libusb: warning [handle_timeout] async cancel failed -5 errno=0
...
WARN [9348][Depthkit::DepthkitSensorStage::onSensorEvent@378] [Azure Kinect 000981312712] Timed out waiting for a capture
WARN [3384][Depthkit::DepthkitSensorStage::onSensorEvent@378] [Azure Kinect 000896512712] Timed out waiting for a capture
WARN [9348][Depthkit::DepthkitSensorStage::onSensorEvent@378] [Azure Kinect 000981312712] Timed out waiting for a capture (2)
WARN [5524][Depthkit::DepthkitSensorStage::onSensorEvent@378][Azure Kinect 000044693712] Time out waiting for a capture.
WARN [5072][Depthkit::DepthkitSensorStage::onSensorEvent@378][Azure Kinect 000774312712] Timed out waiting for a capture.
...
This was a one off issue I encountered yesterday, depthkit Studio stopped responding during video clip playback, though the clip itself is uncorrupted and can be played back on a subsequent relaunch.
Error [8668][Depthkit::IO::VideoPlayerWMFSourceReader::seekToTime1358] Error seeking video
Error [2584][Depthkit::IO::VideoPlayerWMFSourceReader::seekToTime@1358] Error seeking video
Error [2680][Depthkit::IO::VideoPlayerWMFSourceReader::seekToTime@1358] Error seeking video
Exporting depthkit Studio CPP files off our capture computer will freeze on a random frame, causing clips to “pause” at a random moment. The current workaround is to move the project file onto a separate computer with depthkit Studio installed and export from there.
I have also attached the project file. Please let us know if any further information would be helpful. I’ll continue keeping an eye out for and/or see if I can re-produce the video glitch issue.
@EmersonCollege thanks for the additional info and the project.
The CPU usage show in image 1 is much higher than we expect. Do you recall which color and depth resolutions the cameras were set to while this was occurring? The calibration process doesn’t use the data from the color cameras, so you can set them all to 720p color before starting calibration to improve performance and reduce CPU load. Similarly, you can switch the depth mode to NFOV raw - Based on the sensor positions shown in the 3D viewport, your sensors will still be able to see the calibration chart in the positions you are sampling. Then, when recording, also test different resolutions to see how that affects CPU load and system stability.
The issues shown in the console errors can usually be narrowed down to specific sensors (or the cables / ports / USB controllers they are connected to). If these continue, try selectively removing one sensor at a time to see if the issue goes away, and if it does, trying that sensor with different cables or USB ports until you can identify which component is causing the problem.
Unclear if this is related to this issue, but I noticed the Camera app and OBS running in your task list. Best practices avoid having any application open on the computer which may attempt to connect to the Kinects, and therefore may result in Depthkit failing to connect to the Kinects.
I am unable to reproduce your playback issue in the project you shared. Is there any other information you can share to help us reproduce it?
I am seeing some issues exporting CPP video from the project you shared. Can you break this out into a separate topic, and share any screenshots and steps to reproduce so that I can more easily share it with my team?
@EmersonCollege Following up - Were you able to reduce/eliminate the impact of the issue with different stream resolutions, or narrow anything down to a specific component.
For the export issue, please post the details of this issue - error messages, console logs, screenshots, and anything else that can help us understand and reproduce the issue - in a new topic.
