libav_Mat: C++ library for getting video stream with ffmpeg libav, and return cv::Mat for OpenCV users.

# FFmpeg libav to OpenCV(ROS ver.)
C++ library for getting video stream with ffmpeg libav, and return with ```cv::Mat``` for OpenCV users.  
Lower latency than using ```cv::VideoCapture``` with FFmpeg cap(```CAP_FFMPEG```).

Change options in CMakeLists.txt for ROS version. Image data from video stream will be published to topic(`[node_name]/image` by default). The [nodelet](http://wiki.ros.org/nodelet) package is used to run multiple nodes in the same process with zero copy transport between them.

## Requirements

- OpenCV  
    Install OpenCV [here](https://opencv.org/releases/).
- libavcodec
- libavformat
- libavutil
- libavdevice
- libswscale  
Install other libs with command:  
```shell
$ sudo apt-get install libavcodec-dev libavformat-dev libavutil-dev libavdevice-dev libswscale-dev
```

## Usage
### Build and test without ROS

1. Clone the code

```shell
   git clone git@gitee.com/sensors_and_external_devices_drive/libav_Mat.git
   ```

2. build

```shell
   $ mkdir build && cd build
   $ cmake .. && make
   ```

3. test

```shell
   ./test_libav <test stream addr>
   ```

### Build as ROS package
1. Clone the code to ```<catkin_ws>/src```.

```shell
    git clone git@gitee.com/sensors_and_external_devices_drive/libav_Mat.git
    ```

2. Change settings for ROS

Open `CMakeLists.txt` and modify `WITH_ROS` option to ON.

3. Build ROS

```
    catkin_make --pkg libav_nodelet
    ```

### Test with your own stream(ROS)

1. Change the stream address in `launch/send_nodelet.launch` at line:

```xml
   <param name="addr" value="rtsp://127.0.0.1:8554/" />
   ```

2. Run send launch(Remember to source your workspace)

```shell
   roslaunch libav_nodelet send_nodelet.launch
   ```

3. Run receive launch

The test will simply show the video. Here we provide node and nodelet version of receiving example.

- node ver.

```
     rosrun libav_nodelet test_node
     ```

- nodelet ver.

```
     roslaunch libav_nodelet receive_nodelet.launch
     ```

### Test with your own receive node/nodelet(ROS)

1. The nodelet version support zero copy transport between nodes. For converting your own code to nodelet, follow the tutorial [here](http://wiki.ros.org/nodelet/Tutorials/Porting%20nodes%20to%20nodelets). As a wrapping example code, you can check file `test_nodelet.cpp`.

2. If you'd prefer using node instead of nodelet, it's also supported. Check example code `test_node.cpp` for receiving image with `image_transport`.

3. If you want to change the topic name, just uncomment the `remap` option in `send_nodelet.launch` and remap to the topic name you want.

```xml
   <remap from="/camera/image" to="/topic" />
   ```

## Delay Test

### Local network(eth) UDP stream(h264)

Streaming camera video via UDP(no rtp or rtsp protocol) wih h264 encoded in local subnetwork.

- OpenCV ```cv::VideoCapture("udp://...", cv::CAP_FFMPEG)```: ~8100ms

![image-20200411000132685](./assets/image-20200411000132685.png)

- libav ```LibAV("udp://...")```: 320ms

![image-20200411000252110](./assets/image-20200411000252110.png)

### Localhost rtsp stream

Locally streaming desktop via rtsp using VLC.

- OpenCV ```cv::VideoCapture("rtsp://...")```:~1146ms

Need to set environment ```export OPENCV_FFMPEG_CAPTURE_OPTIONS="rtsp_transport;udp"```

![image-20200410235224041](./assets/image-20200410235224041.png)

- Libav  ```LibAV("rtsp://...")```: ~678ms

![image-20200410235059385](./assets/image-20200410235059385.png)

传感器和外部设备驱动/libav_Mat

简介

发行版

贡献者

近期动态

传感器和外部设备驱动/libav_Mat .gitee-modal { width: 500px !important; }

简介

发行版

贡献者

近期动态

搜索帮助

传感器和外部设备驱动/libav_Mat