文本生成图像的DF-GAN增强模型研究

DF-GAN Enhancement Model for Text-to-Image

摘要:

针对文本生成图像任务中生成图像的语义关联性低,细节较模糊以及结构完整性较差等问题,提出了一种结合自注意力机制的DF-GAN增强模型。首先,利用BERT模型挖掘文本上下文语义特征,同时结合语义深度融合模块,实现深层的文本语义与图像区域特征的匹配。其次,在模型架构层面引入一种自注意力模块作为卷积模块的补充,目的是能够更好地建立长距离、多层次依赖关系。实验显示本文提出的增强模型不仅加强了文本与图片之间的语义联系,而且保证了生成图像的细粒度和完整性。

Abstract:

A DF-GAN enhancement model combined with self-attention mechanism was proposed for low semantic relevance, fuzzy details and inadequate structural integrity in text-to-image tasks. Firstly, the BERT model is used to mine the semantic features of text context, and the ? deep text-image fusion block is combined to realize the matching of deep text semantics and image regional features. Secondly, a self-attention mechanism module is introduced as a supplement to the convolution module at the model architecture level, aiming to better establish long-distance and multi-level dependencies. The experimental results show that the proposed enhancement model not only strengthens the semantic relationship between the text and the image, but also ensures the precise details and overall integrity of the generated image.