Current Issue Cover


摘 要
目的 未来视频编码(Future video coding, FVC)是在高效视频编码标准(High Efficiency Video Coding, HEVC)的基础上提出的新一代编码技术,复杂度极高。现有的基于HEVC的快速编码方法不适用于FVC中的四叉树加二叉树编码结构或者节省时间有限。本文提出了一种结合随机森林的FVC帧内编码单元(Coding Unit, CU)快速划分算法。方法 该算法针对FVC中的四叉树加二叉树结构进行优化。首先,提取视频编码过程中的各CU的图像纹理特征和划分结果;然后,分别使用各划分深度下的纹理特征和划分结果进行在线训练,建立多个随机森林模型,不同深度的CU对应于不同的模型;最后,使用模型对视频其余帧的CU进行划分结果预测,从而减少了划分模式遍历和率失真代价计算的次数,节省了编码时间。结果 实验结果表明,与原始平台算法相比,该算法能够节省44.1%的时间,在相同峰值信噪比的情况下,比特率仅上升2.6%。与当前先进的方法相比,能进一步节省20%以上的时间。结论 该算法通过提取图像的纹理特征,建立随机森林模型,对CU划分结果进行预测,在保证编码率失真性能的前提下,有效地降低了FVC的帧内CU划分复杂度。
Random forest based fast intra coding unit partition algorithm for FVC

Ren Yan,Peng Zongju,Cui Xin,Chen Fen,Chen Hua(Faculty of Information Science and Engineering)

Objective With the development of digital video technology, especially the emergence of ultra high definition (UHD) video technology, video compression faces enormous challenges. In order to solve the problem of large amount of data and high-speed transmission requirements caused by UHD video, the Joint Video Experts Team (JEVT) are exploring Future Video Coding (FVC) based on the High Efficiency Video Coding (HEVC) standard. FVC uses the hybrid coding framework of HEVC with new techniques. The compression efficiency of FVC is higher than that of HEVC, however, the coding complexity is extremely high. Therefore, it is of great significance to reduce the complexity of FVC. Among all the new techniques in FVC, the most effective but extremely time consuming one is the Quad Tree plus Binary Tree (QTBT) coding structure, which includes four partition modes: quad tree split, vertical split, horizontal split and no-split. The final split of CUs is decided after trying all the partition modes and calculating the Rate-Distortion cost. Hence the complexity of the QTBT is extremely high. The existing HEVC-based fast coding method is no longer suitable for FVC because of the QTBT coding structure and the recent work about low-complexity encoding methods are not sufficient for application of FVC. In order to reduce the high complexity of FVC, the complexity of QTBT structure should be considered first. The traversal process of CUs’ partition modes exists redundancy, and unnecessary attempts of partition modes should be avoided. To optimize the CUs’ split process, this paper proposes a random forest based fast intra coding unit partition algorithm for FVC. Method The algorithm is designed to optimize the QTBT structure in FVC. Compared with the traditional statistical based methods, the machine learning based approach is more applicable due to the elaborate split modes of QTBT structure. Among the many methods of machine learning, random forest has its unique advantages. It can handle the classification problem of multi-dimensional data and has strong resistance to over-fitting and estimation. Besides, it has good performance on classification issues, which is exactly suitable for the CU split. Therefore, a fast algorithm based on random forest is proposed. The problem of distinguishing different split results of CUs is considered to be a classification problem and random forest is used as the classifier. First, the image texture features and the split results of the coding unit (CU) in the first frame of video sequences are extracted. Image texture features have a strong correlation with split results, thus can be selected as the training data of the model. Various image texture features are used in the algorithm to achieve better performance, and they are carefully selected through the calculation of feature importance. Specifically, the features finally used in the proposed algorithm are the width and height of CU, Haar wavelet coefficients, angular second moment, entropy, contrast, inverse differential moment and standard deviation. After the data collecting process, four random forest models are established for different depth of CUs. CUs’ depth can be represented as the joint depth of the quad tree and the binary tree, and this representative method is used to collect data in the algorithm. Then, the texture features and the split result are set as multidimensional data, and they are trained online for each model separately. The training time is included in the whole encoding time, and they are relatively small compared with the encoding time. Finally, the trained models are used to predict the split result of the CUs of the remaining frames of the video sequences, thereby reducing the traversal of the partition modes, reducing the time of rate distortion cost calculation. To ensure the algorithm’s effectiveness, the accuracy of the models is tested online by using different video sequences. The algorithm is implemented on the JEM5.0 platform which is recently released. A total of 22 test sequences of different contents and resolutions from class A1 to class E are tested under the common test condition, which is full I-frame configuration mode with quantization parameters 22, 27, 32 and 37. The encoding performance of algorithm is evaluated using Bjontegaard Delta bitrate (BDBR) and average time saving between the proposed algorithm and the original platform. Result The experimental results show that compared with the original platform’s algorithm, the proposed algorithm can decrease the average encoding time by 44.1% while coding performance loss is negligible, and the BDBR only increases by 2.6%. Compared with the state-of-the-art methods, it can save more than 20% of the time, with BDBR slightly increases. This algorithm is suitable for various classes of video sequences with different resolutions and texture. Among all the sequences, the sequences with high resolution achieved more time saving than other sequences because of the online training time consumption. Besides, the coding performance of proposed algorithm is stable, which proves the effectiveness of the models. Conclusion A random forest based fast intra coding unit partition algorithm for FVC is proposed to reduce the complexity of the QTBT structure in FVC. By extracting the texture features of the image, the algorithm establishes random forest models to predict the CU partitioning result, and the unnecessary split modes traverse can be avoided to save encoding time. The proposed intra prediction coding algorithm can effectively reduce the complexity of FVC and maintain the encoding performance. The proposed algorithm is more suitable for video sequences which have high resolution. Furthermore, the proposed algorithm should be optimized in the future for more time reduction and less coding performance loss.The possibilities of the machine learning in FVC inter prediction will also be explored in the future.