360° images and videos capture light coming from all directions and are widely used in fields such as virtual reality (VR), augmented reality, autonomous driving and intelligent video surveillance. In order to process and store such images with common image processing tools, the spherical content is typically projected onto one or multiple planes. To store and transmit the large amount of data, different compression algorithms can be considered. Recent advancements in learning-based image compression have been shown to outperform conventional image codecs in terms of compression efficiency. However, the learned transforms are not well-studied compared to the hand-crafted transforms used in conventional image codecs, especially for 360° images. Although various methods have been developed to analyze learned image codecs, this has not been carried out for 360° content. This work involves the application of channel impulse response analysis to understand what kind of features are captured by a learned image codec that codes 360° images.
The goal of this project is to investigate the transform properties for 360° image coding and identify specific directional properties across various projection formats. As a first step, the transform properties of a basic pretrained image compression model shall be investigated for the common equirectangular (ERP) and cubemap (CMP) projection for the luma component. Followed by this, the color information can be included in the analysis. Further investigations could include other projections or spherical sampling formats and the extension to recent learned image codecs.
Prerequisites:
- Course ’Image and Video Compression’
- Fluent in Python programming
- Good to have: Experience with learning-based image coding
Contact:
| M.Sc. Srivatsa Prativadibhayankaram | srivatsa.prativadibhayankaram@fau.de |
| M.Sc. Paul Wawerek-López | paul.wawerek-lopez@fau.de |
Prof. Dr.-Ing. A. Kaup