A new HEVC-compliant perceptual video coding scheme based on Just Noticeable Difference

A new HEVC-compliant perceptual video coding (PVC) method has been developed that can improve coding efficiency through a significant number of bitrate reductions with negligible visual quality loss. Such a remarkable coding efficiency improvement comes from a novel quantization scheme based on an elaborate Just Noticeable Difference (JND) model. Due to the compliance to the normative of HEVC standard, our PVC scheme can be implemented in many applications based on HEVC standard such as smartphones, UHDTV, and digital cameras/camcorders.

Article  |  Fall 2015

Researchers have tried a variety of coding schemes to find perceptually effective video compression by using human visual characteristics, but most of them have limitations such as standard-incompliant PVC schemes, computationally complex PVC schemes, and small coding efficiency improvements in PVC schemes.

However, our research team has overcome such limitations by developing an HEVC-compliant PVC scheme that uses accurate but simple JND models and that performs a computationally efficient perceptual video coding, producing significant bitrate reductions at a maximum of 49.1% and an average of 16.1% with negligible visual quality loss. The proposed PVC scheme only causes about an 11.25% increase in execution time compared to the original reference software model of HEVC, which is computationally very efficient and seven times faster than the most recent other PVC works.

The proposed PVC scheme incorporates JND models in both the transform and pixel domains. A pixel-domain luminance adaptation JND model is applied for the non-transform mode of HEVC where the signal amounts above the JND threshold are considered to be perceptually redundant and are effectively removed. A novel transform-domain JND model is applied for transform modes of HEVC where the transform coefficient values are effectively JND-suppressed. The novel transform-domain JND model is a unified model that considers the JND characteristics such as contrast frequency sensitivity, background luminance adaptation, texture masking, and motion masking effects. Moreover, the proposed JND model newly incorporates the summation effect of JND suppression for individual transform coefficients to harmonize with variable block-sized transforms of 4´4, 8´8, 16´16, and 32´32 block sizes in HEVC.

The proposed PVC scheme has been designed by considering computational complexity and coding efficiency. In order to reduce psycho-visual redundancy, the incorporated JND suppression is accomplished by performing shift and subtraction operations for each transform coefficient at once, which is computationally very efficient. In contrast, the previous PVC schemes require heavy or iterative computation, thus preventing them from being applied to hardware encoders. To improve coding efficiency for JND-based suppression, a distortion compensation factor was also devised to reflect the perceptual distortion in the rate-distortion optimization based encoding process.

This novel HEVC-compliant PVC scheme was published in IEEE Transactions on Circuits and Systems for Video Technology and won the “Best Encoder Optimization for High Efficiency Video Coding” award in the Video Compression Grand Challenges contest of the Picture Coding Symposium held San Jose, USA in December 2013.

This work was supported by the IT R&D Program of the Ministry of Science, ICT and Future Planning (MSIP) and Institute for Information and Communication Technology Promotion (IITP) in Korea.