The ZED Open Capture is a multi-platform, open-source C++ library for low-level camera and sensor capture for the ZED stereo camera family. It doesn't require CUDA and therefore can be used on many desktop and embedded platforms.
The open-source library provides methods to access raw video frames, calibration data, camera controls and raw data from the camera sensors (on ZED 2, ZED 2i, and ZED Mini). A synchronization mechanism is provided to get the correct sensor data associated to a video frame.
Note: While in the ZED SDK all output data is calibrated and compensated, here the extracted raw data is not corrected by the camera and sensor calibration parameters. You can retrieve camera and sensor calibration data using the ZED SDK to correct your camera data [see
On some embedded devices, like Raspberry pi 4, the depth extraction example can crash with the following error:
vtkShaderProgram.cxx:438 ERR| vtkShaderProgram (0x23a611c0): 0:1(10): error: GLSL 1.50 is not supported. Supported versions are: 1.10, 1.20, 1.00 ES, and 3.00 ES
to correctly execute the example application it is necessary to change the default OpenGL version:
you can permanently add this configuration by adding the above command as last line of the
Install GCC compiler and build tools
$ sudo apt install build-essential
Install CMake build system
$ sudo apt install cmake
Install HIDAPI and LIBUSB libraries:
$ sudo apt install libusb-1.0-0-dev libhidapi-libusb0 libhidapi-dev
Install OpenCV to build the examples (optional)
$ sudo apt install libopencv-dev libopencv-viz-dev
$ git clone https://github.com/stereolabs/zed-open-capture.git $ cd zed-open-capture
Stereo cameras such as ZED 2 and ZED Mini have built-in sensors (e.g. IMU) that are identified as USB HID devices.
To be able to access the USB HID device, you must add a udev rule contained in the
$ cd udev $ bash install_udev_rule.sh $ cd ..
$ mkdir build $ cd build $ cmake .. $ make -j$(nproc)
$ mkdir build $ cd build $ cmake .. -DBUILD_EXAMPLES=OFF $ make -j$(nproc)
$ mkdir build $ cd build $ cmake .. -DBUILD_SENSORS=OFF -DBUILD_EXAMPLES=OFF $ make -j$(nproc)
$ mkdir build $ cd build $ cmake .. -DBUILD_VIDEO=OFF -DBUILD_EXAMPLES=OFF $ make -j$(nproc)
To install the library, go to the
build folder and launch the following commands:
$ sudo make install $ sudo ldconfig
videocapture.hpp header, declare a
VideoCapture object and retrieve a video frame (in YUV 4:2:2 format) with
#include "videocapture.hpp" sl_oc::video::VideoCapture cap; cap.initializeVideo(); const sl_oc::video::Frame frame = cap.getLastFrame();
SensorCapture header, declare a
SensorCapture object, get a list of available devices, initialize the first one and finally retrieve sensors data:
#include "sensorcapture.hpp" sl_oc::sensors::SensorCapture sens; std::vector<int> devs = sens.getDeviceList(); sens.initializeSensors( devs ); const sl_oc::sensors::data::Imu imuData = sens.getLastIMUData(5000); const sl_oc::sensors::data::Magnetometer magData = sens.getLastMagnetometerData(100); const sl_oc::sensors::data::Environment envData = sens.getLastEnvironmentData(100); const sl_oc::sensors::data::Temperature tempData = sens.getLastCameraTemperatureData(100);
After installing the library and examples, you will have the following sample applications in your
SensorCaptureobject and displays on the command console the values of camera sensors acquired at full rate.
SensorCaptureobject, initialize the camera/sensors synchronization and displays on screen the video stream with the synchronized IMU data.
To run the examples, open a terminal console and enter the following commands:
$ zed_open_capture_video_example $ zed_open_capture_control_example $ zed_open_capture_rectify_example $ zed_open_capture_sensors_example $ zed_open_capture_sync_example $ zed_open_capture_depth_example $ zed_open_capture_depth_tune_stereo
Note: OpenCV is used in the examples for controls, display, and depth extraction.
The API is documented in the Include.h files. It is also generated as a Doxygen for simpler navigation: https://stereolabs.github.io/zed-open-capture
You can also generate the documentation locally in HTML format (with Doxygen) using the commands below. Access the docs by opening
doc/html/index.html in your web browser.
$ sudo apt-get install -y doxygen # if not previously installed $ cd doc $ ./generate_doc.sh
The coordinate system is only used for sensors data. The given IMU and Magnetometer data are expressed in the RAW coordinate system as show below
This library is licensed under the MIT License.
If you need assistance go to our Community site at https://community.stereolabs.com/