北京邮电大学学报

  • EI核心期刊

北京邮电大学学报 ›› 2023, Vol. 46 ›› Issue (3): 103-108.

• 论文 • 上一篇    下一篇

复杂环境下多模态指导的点云补全方法

靳梦凡1,2,黄智濒1,储志强3   

  1. 1. 北京邮电大学 计算机学院 2. 国家计算机网络与信息安全管理中心 青海分中心 3. 国家市场监督管理总局信息中心

  • 收稿日期:2022-03-23 修回日期:2022-07-04 出版日期:2023-06-28 发布日期:2023-06-05
  • 通讯作者: 黄智濒 E-mail:huangzb@bupt.edu.cn

Multi-Mode Guided Point Cloud Completion Method in Complex Environment

JIN Mengfan1,2, HUANG Zhibin1 , CHU Zhiqiang3   

  • Received:2022-03-23 Revised:2022-07-04 Online:2023-06-28 Published:2023-06-05

摘要:

在现实复杂环境中,往往使用RGB-D相机直接获取点云,会受到外界复杂场景的影响,例如:光线、透明物体、遮挡和阴影等,导致点云出现大规模缺失,甚至无法表示物体真实的三维特征。不完整的点云将会对目标识别和路径规划等多个计算机视觉领域的重要研究造成影响。对此,提出了使用RGB图片语义信息等多模态下的数据来指导补全点云的方法。该方法先使用一种基于“编码器-解码器”结构的RGB图片语义分割网络,获取RGB图片的语义分割结果,然后将RGB图片、RGB图片语义分割结果和残缺的稀疏深度图作为算法输入,输出补全好的点云。经过在真实复杂场景中进行大量实验,实验结果表明,所提方法在补全效果和运行效率等方面均取得不错效果。

关键词: 点云补全, 计算机视觉, 深度学习, 语义分割

Abstract:

In complex environments, the point cloud obtained directly by RGB-D camera is often affected by complex external scenes, such as light, transparent objects, shadows, etc., resulting in large-scale loss of point cloud, and even being unable to represent the real 3D features of the object. Incomplete point cloud affects many important applications in computer vision fields, such as object detection, path planning, etc. To solve this issue, a method is proposed to complete the point cloud using multimodal data such as RGB image semantic information. This method first uses a semantic segmentation network of RGB image, which is based on the "encoder-decoder" structure to obtain the semantic segmentation results of RGB image, and then takes RGB image, semantic segmentation results, and incomplete sparse depth map as the input of algorithm, and outputs completed point cloud. After a large number of experiments in complex scenes, the experiments show that this method is effective in completion effect and operation efficiency.

Key words: point cloud completion, computer vision, deep learning, semantic segmentation

中图分类号: