Real-time image super-resolution reconstruction for system on chip FPGA

Abstract

In recent years, image super-resolution (SR) techniques based on Convolutional Neural Network (CNN) have achieved impressive attention from computer vision scholars and artificial intelligence (AI) companies. Due to the necessity of using the SR algorithms in real-world applications, designing an efficient and lightweight SR algorithm that improves the sharpness and visual quality of the SR results is a critical issue in real-time hardware implementation. To address these issues, we proposed the Multi-FusNet of Cross Channel Network (MFCC) network by constructing the groups of Residual-in-Residual architecture under the multi-path cascading framework. Additionally, a residual connection is used to transfer the low-level features of the early layer to the reconstructed SR image. The proposed SR model is initially trained with the GPU's training image dataset. To implement our trained model in System on Chip FPGA, the size of the proposed model is required to reduce. We convert the floating-point checkpoint into a fixed-point integer checkpoint in the quantization procedure. According to the experimental results, the proposed method reduces the number of network parameters significantly (8.4 times compared to RCAN), can execute fast in System On Chip FPGA, around 30 frames per second, and image accuracy of the proposed method in terms of PSNR value does not decrease over 1 dB.