基于轴向空间注意力和中间融合表示的单图像三维重建模型

A Single-image 3D Reconstruction Model Based on Axial Spatial Attention and Intermediate Fusion Representation

摘要:

提出一种基于注意力和中间融合表示的三维重建模型,旨在重建具有精细化结构的三维模型。该方法利用轴向空间注意力机制学习不同方向的信息,将其嵌入编码器中以捕获局部结构特征；并基于双流网络推测深度图和三维平均形状以设计中间融合表示模块,该模块能够有效地融合可见表面细节信息,从而更好地描绘对象的三维空间结构。实验结果表明:所提出的轴向空间注意力机制和中间融合表示模块增强了特征提取的能力,IoU和F-score比PixVox++分别提升了1.3%和0.4%,三维重建效果更优。

Abstract:

A 3D reconstruction model based on attention and intermediate fusion representation is proposed, aiming to reconstruct a 3D model with refined structure. This method used the axial spatial attention mechanism to learn information in different directions, and embeded it in the encoder to capture local structural features. Then, based on the two-stream network, the depth map and 3D average shape could be inferred to design an intermediate fusion representation module which could effectively fuse visible surface details to better describe the 3D spatial structure of the objects. The experimental results show that the axial spatial attention mechanism and the intermediate fusion representation module proposed in this paper enhance the ability of feature extraction. The IoU and F-score are 1.3% and 0.4% higher than PixVox++, respectively, proving that the 3D reconstruction effect is better.