Genetic algorithm based coding for high efficiency video coding

Abstract

High efficiency video coding (HEVC) is the newest video coding standard to greatly increase the coding efficiency of its ancestor H.264/AVC with the aids of its new features such as the quadtree-based coding unit (CU) partitioning, a simple deblocking filter, and other advanced coding techniques. However, HEVC delivers a highly increased computation complexity, which is mainly due to the exhaustive rate distortion optimization search of quadtree-based CU partitioning. Firstly, a feature reduction approach is proposed on a fuzzy support vector machine (SVM) based CU size decision method. The proposed feature reduction approach with rate control (RC) can reduce computational complexity by eliminating some correlated features of a fuzzy SVM-based CU size decision method under a similar coding efficiency. According to the empirical results, our approach can achieve up to 3% of complexity reduction under the same rate distortion (RD) performance over a fuzzy SVM-based approach. Secondly, instead of machine learning (ML) based fast algorithm approach, a CU partitioning pattern optimization method based on genetic algorithm (GA) is proposed to save the computational complexity of a hierarchical quadtree-based CU partitioning. The required coding unit partitioning pattern for an exhaustive partitioning and the rate distortion cost are efficiently considered as the chromosome and the fitness function of the genetic algorithm, respectively. To reduce the computational time, CU partitioning patterns of the key frame is searched and shared to other consecutive frames by taking into account the highly temporal correlation. Our evaluation results show that the proposed method can achieve 62.5% and 16.7% computational complexity reduction on average at 8 Mbps with a negligible average quality degradation compared with HM16.5 and state-of-the-art support vector machine-based fast algorithm, respectively, under low delay P configuration with rate control while 64.1% and 15.1% under low delay configuration with rate control.