These examples, written in Python, will provide a good starting point for a lot, and the most common, applications of GStreamer and OpenCV. The snippets mainly use OpenCV's VideoWriter and VideoCapture object, and include the following functionalities:
import cv2 # Cam properties fps = 30. frame_width = 1920 frame_height = 1080 # Create capture cap = cv2.VideoCapture(0) # Set camera properties cap.set(cv2.CAP_PROP_FRAME_WIDTH, frame_width) cap.set(cv2.CAP_PROP_FRAME_HEIGHT, frame_height) cap.set(cv2.CAP_PROP_FPS, fps)
# The following string usually works on most webcams webcam2appsink_YUY2_640_480 = "v4l2src device=/dev/video0 ! video/x-raw, format=YUY2, width=640, height=480, pixel-aspect-ratio=1/1, framerate=30/1 ! videoconvert ! appsink"
gst_str = "appsrc ! videoconvert ! shmsink socket-path=/tmp/foo sync=true wait-for-connection=false shm-size=10000000"
cap = cv2.VideoCapture("shmsrc socket-path=/tmp/foo ! video/x-raw, format=BGR, width=640, height=480, pixel-aspect-ratio=1/1, framerate=30/1 ! decodebin ! videoconvert ! appsink")
gst_str_rtp = "appsrc ! videoconvert ! x264enc noise-reduction=10000 tune=zerolatency byte-stream=true threads=4 " \ " ! h264parse ! mpegtsmux ! rtpmp2tpay ! udpsink host=127.0.0.1 port=5000"
# mfxh264enc does all the HW encoding on the INTEL HD GPU appsink2file = "appsrc ! videoconvert ! mfxh264enc ! \ video/x-h264, profile=baseline ! \ matroskamux ! filesink location=the_gstreamer_enjoyer.mkv"
Since you are here, you probably know why you want to use GStreamer and OpenCV and I'm not gonna list all the advantages that GStreamer brings to the table. However, if you find this repo helpful or even remotely funny, consider leaving a star. Or not. Your choice.
Be a unix enthusiast. The GPU encoding pipelines in GStreamer are extremly powerful, but are hard to install.
Correct drivers for GPU (CUDA + CUDNN if necessary)
sudo apt-get install gstreamer1.0* libgstreamer-plugins-bad1.0-0 libgstreamer-plugins-base1.0-0 libgstreamer-plugins-base1.0-dev libgstreamer1.0 libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libgtk-3-dev
optional FFMPEG, QT (+ OpenGL)
Supported platforms: Intel Haswell / Broadwell / Skylake with Intel HD / Iris Pro graphics / Apollo Lake) (>= Gen4)
Installation process (Honestly this is pretty frustrating Intel. I got very mad followed by getting drunk and yelling at the PC, but as far as i remember the process went something like this):
Detailed examples here
Achieved speedup for writing a Full-HD frame (1920x1080px) to NVMe SSD:
/usr/lib/x86_64-linux-gnu/gstreamer-1.0/, or in
/usr/local/lib/gstreamer-1.0/or even in your local path if you don't have
sudo. Truth is that i copipasta'd the files until it matched. There might be a more elegant way to do this.
sudo apt-get install cmake-qt-gui); search and click the features you want to have enabled (even after your exec'd a usual
$ gst-launch-1.0 v4l2src ! xvimagesink
$ gst-inspect-1.0 | grep mfx mfx: mfxh264dec: MFX H264 decoder mfx: mfxhevcdec: MFX HEVC decoder mfx: mfxh264enc: MFX H.264 encoder mfx: mfxhevcenc: MFX H.265 encoder
$ gst-inspect-1.0 | grep nvenc nvenc: nvh264enc: NVENC H.264 Video Encoder
VideoCapture(0)and puts the raw image in a shared memory. Mind here to define your own webcam properties.
VideoCapture(0),encodes the frame and streams it to
v4l2src /dev/video0(usually webcam) to OpenCV format and writes it as an h264 encoded file.
v4l2src /dev/video0(usually webcam) to OpenCV format and writes it as an h264 encoded rtp stream. Use the provided
*.sdpfiles for VLC viewer.
Inspired by https://github.com/tik0/mat2gstreamer