基于多模态的车辆行人检测技术研究

Research on vehicle pedestrian detection technology based on multimodal image

摘要:

针对单一传感器在复杂路况以及恶劣天气情况下车辆行人检测效果不佳的问题,本文搭建了一套可见光、可见光偏振、短波红外、长波红外多模态数据采集系统,构建一个多模态数据集；提出了一种多模态车辆行人检测算法。首先,提出了一种基于改进型SIFT特征点的多尺度部分强度不变特征的异源图像配准算法；然后,提出基于YOLOv5多模态数据目标检测网络。最终实现了平均精度在日间数据集1.0%的提升,日间夜间混合数据集10.9%的提升。

Abstract:

As one of the bottlenecks restricting the vehicle pedestrian detection algorithm, visual perception is difficult to cope with complex road conditions and weather conditions when a single visual sensor detects road surface objects. Therefore, this paper uses multimodal data to improve the performance of the detection algorithm. Firstly, this paper builds a multi-modal data acquisition system through the visible light camera, visible polarization movement, short-wave infrared movement, and long-wave infrared movement, and constructs a multi-modal data set to fill the gap in the data. Secondly, for heterologous image registration, a heterologous image registration algorithm based on the multi-scale partial intensity invariant features of improved SIFT feature points is proposed. Then, for target detection, a multimodal data object detection network based on YOLOv5 is proposed. Finally, the average accuracy was improved by 1.0% in the daytime dataset and 10.9% in the daytime and nighttime mixed dataset.