Efficient Learning-based Image Enhancement

K. I. Kim, Y. Kwon, J. Tompkin, J.-H. Kim, and C. Theobalt

About || Examples || References || Contact

Input JPEG 2000-encoded image

Enhanced image

 

About

Many computer vision and computational photography applications essentially solve an image enhancement problem. The image has been deteriorated by a specific noise process, such as aberrations from camera optics and compression artifacts, that we would like to remove. We describe a framework for learningbased image enhancement. At the core of our algorithm lies a generic regularization framework that comprises a prior on natural images, as well as an application-specific conditional model based on Gaussian processes. In contrast to prior learning-based approaches, our algorithm can instantly learn task-specific degradation models from sample images which enables users to easily adopt the algorithm to a specific problem and data set of interest. This is facilitated by our efficient approximation scheme of large-scale Gaussian processes. We demonstrate the efficiency and effectiveness of our approach by applying it to example enhancement applications including singleimage super-resolution, as well as artifact removal in JPEG- and JPEG 2000-encoded images.

Details can be found in [KK12][KK14].

Examples

JPEG 2000 compression artifact removal

JPEG 2000 (Comp. Ratio 0.15BPP)

[RB09]

[NO03]

[GW05]

[KK14] (our method)

 

JPEG compression artifact removal

JPEG (Q2, see [KK12])

[NO01]

[RB09]

[GW05]

[FK07]

[KK14] (our method)

 

Single-image super-resolution (generic)

 

 

 

 

 

 

N/A

Bicubic resampling; magnification factors 2 (left) and 3 (right)

[FJ02]

 

 

 

 

 

 

N/A

[CY04]

[YW10]

[KK10]

[KK14] (our method)

 

Single-image super-resolution (faces)

Bicubic resampling;
magnification factor 4

[KK10]
(generic model)

[KK14] (our method;
f
ace-specific model)

 

References

[KK10]

K. I. Kim and Y. Kwon, Single-image super-resolution using sparse regression and natural image prior, IEEE TPAMI, 2010.

[NO01]

A. Nosratinia. Denoising of JPEG images by re-application of JPEG, Journal of VLSI Signal Processing, 2001.

[FJ02]

W. T. Freeman, T. R. Jones, and E. C. Pasztor, Example-based super-resolution, IEEE CGA, 2002.

[CY04]

H. Chang, D.-Y. Yeung, and Y. Xiong, Super-resolution through neighbor embedding, CVPR, 2004.

[YW10]

J. Yang, J. Wright, T. S. Huang, and Y. Ma, Image super-resolution via sparse representation, IEEE TIP, 2010.

[NO03]

A. Nosratinia, Postprocessing of JPEG-2000 images to remove compression artifacts, IEEE SPL, 2003.

[FK07]

A. Foi, V. Katkovnik, and K. Egiazarian, Pointwise shape-adaptive DCT for high-quality denoising and deblocking of grayscale and color images, IEEE TIP, 2007.

[RB09]

S. Roth and M. J. Black, Fields of experts, IJCV, 2009.

[GW05]

P. V. Gehler and M.Welling, Product of “edge-perts”, NIPS, 2005.

[KK12]

Y. Kwon, K. I. Kim, J. H. Kim, and C. Theobalt, Efficient learning-based image enhancement: application to super-resolution and compression artifact removal, BMVC, 2012.

[KK14]

Y. Kwon, K. I. Kim, J. Tompkin, J. H. Kim, and C. Theobalt, Efficient learning of image super-resolution and compression artifact removal with semi-local Gaussian processes, TPAMI, Accepted.

Contact

Kwang In Kim: kkim at mpi-inf.mpg.de.

Younghee Kwon: youngheek at google.com.

James Tompkin: jtompkin at seas.harvard.edu.