结合可逆网络的超高分辨率图像的风格迁移

Super-resolution image style transfer combined with reversible network

摘要:

针对处理超高分辨率图像时面临的内存成本问题和风格迁移过程中过度风格化的问题,提出了一种结合可逆网络的超高分辨率图像的风格迁移方法,能够以较小的内存成本实现任意尺寸图像的风格迁移。所提算法采用可逆的Glow模块作为基本单元构建可逆神经网络模块,通过将图像划分成小块的方式避免超高分辨率图像引起的内存问题。为了更好地提取输入图像特征以及确保不同块之间的风格一致性,在风格迁移模块使用引入通道注意力机制的残差模块和缩略图实例化归一化模块(TIN)。实验结果表明,相较于当前通用的各种神经风格迁移网络,所提算法不仅能够避免图像在编码和解码过程中的信息丢失问题,而且以更低的内存成本实现了相当的风格迁移性能。

Abstract:

Aiming at the problem of memory cost when dealing with ultra-high resolution images and the problem of over-stylization in the process of style transfer, a style transfer method for ultra-high resolution images combined with a reversible network is proposed, which can be realized with a small memory cost style transfer for images of any size. The proposed algorithm uses the reversible Glow module as the basic unit to construct a reversible neural network module, and avoids memory problems caused by ultra-high resolution images by dividing the image into small blocks. In order to better extract the features of the input image and ensure the style consistency between different blocks, the residual module and the thumbnail instantiation normalization module (TIN) which introduces the channel attention mechanism are used in the style transfer module. Experimental results show that compared with the current general-purpose neural style transfer network, the proposed algorithm can not only avoid the information loss problem during image encoding and decoding, but also achieve comparable style transfer performance at a lower memory cost.