max planck institut informatik
In Le Noeud Noir
, Georges Seurat introduces contour effects to enhance overall contrast and emphasize the woman figure against the background.
This thesis documents an investigation into the effect of contours on contrast perception and its application to computer depiction. Justified by findings in visual perception and demonstrated through novel graphics techniques, the conclusion is that local contrast alteration is a powerful Computer Graphics tool that is useful in a variety of situations. The tone of this thesis is unlike that of the papers from which it arose. In addition to championing individual applications, there is an emphasis on contrast perception and on brightness phenomena created by contours. The intended audience is anyone interested in applying perception to computer graphics, and researchers working on tone mapping, greyscale conversion or enhanced 3D rendering.
Contrast in photographic and computer-generated imagery communicates colour and lightness differences that would be perceived when viewing the represented scene. Due to depiction constraints, the amount of displayable contrast is limited, reducing the image's ability to accurately represent the scene. A local contrast enhancement technique called unsharp masking can overcome these constraints by adding high-frequency contours to an image that increase its apparent contrast. In three novel algorithms inspired by unsharp masking, specialized local contrast enhancements are shown to overcome constraints of a limited dynamic range, overcome an achromatic palette, and to improve the rendering of 3D shapes and scenes. The Beyond Tone Mapping approach restores original HDR contrast to its tone mapped LDR counterpart by adding high-frequency colour contours to the LDR image while preserving its luminance. Apparent Greyscale is a multi-scale two-step technique that first converts colour images and video to greyscale according to their chromatic lightness, then restores diminished colour contrast with high-frequency luminance contours. Finally, 3D Unsharp Masking performs scene coherent enhancement by introducing 3D high-frequency luminance contours to emphasize the details, shapes, tonal range and spatial organization of a 3D scene within the rendering pipeline. As a perceptual justification, it is argued that a local contrast enhancement made with unsharp masking is related to the Cornsweet illusion, and that this may explain its effect on apparent contrast.
Contrast in photographic and computer-generated imagery communicates colour and lightness differences that would be perceived when viewing the represented scene. Due to depiction constraints, the amount of displayable contrast is limited, reducing the image's ability to accurately represent the scene. Local contrast enhancement can overcome these constraints to produce improved imagery with higher information content, resulting in more efficient depictions. One such technique, called unsharp masking, influences the perception of contrast by adding high-frequency contours to an image.
In three novel algorithms inspired by unsharp masking, local contrast enhancement is shown to overcome a limited dynamic range, overcome an achromatic palette, and to improve the rendering of 3D shapes and scenes. The \emph{Beyond Tone Mapping} approach restores original HDR contrast to its tone mapped LDR counterpart by adding high-frequency colour contours to the LDR image but preserving its luminance. \emph{Apparent Greyscale} is a multi-scale two-step technique to convert colour imagery to greyscale that restores diminished colour contrast with high-frequency luminance contours. Finally, \emph{3D Unsharp Masking} performs scene coherent enhancement in a technique that introduces high-frequency 3D luminance contours to emphasize the details, shapes, tonal range and spatial organization of a 3D scene within the rendering pipeline.
The algorithms are justified by perceptual explanations of actual and apparent contrast. In particular, it is argued that local contrast enhancements made with unsharp masking can be explained by the Cornsweet illusion. This illusion creates a perceived brightening or darkening of regions that are adjacent to a specially shaped high-frequency contour, much like those that are introduced through unsharp masking. The same perceptual mechanism may explain why contours added to an image increase its apparent contrast.
High Dynamic Range (HDR) images capture the full range of luminance present in real world scenes, and unlike Low Dynamic Range (LDR) images, can simultaneously contain detailed information in the deepest of shadows and the brightest of light sources. For display or aesthetic purposes, it is often necessary to perform tone mapping, which creates LDR depictions of HDR images at the cost of contrast information loss. The Beyond Tone Mapping approach enhances an LDR depiction with colour adjustments that add base layer and detail layer contrast without modifying luminance. Its adaptation of traditional unsharp masking calculates the local HDR contrast lost during tone mapping, then restores it to the LDR chromatic channels.
Although colour printing has become common practice, artists and publishers continue to employ greyscale. The format is more dependable and often more evocative. One of the most basic tools in digital image editing software is the greyscale converter, which takes a colour image and produces a colourless version. The depiction challenge is to ensure that the original's chromatic contrasts are communicated even when no colour is present. Apparent Greyscale is a quick and simple method for converting complex images and video to perceptually accurate greyscale versions. It is a two-step approach to globally assign grey values and determine colour ordering, that then locally enhances the greyscale to reproduce the original contrast. The global mapping is image independent and incorporates the Helmholtz-Kohlrausch colour appearance effect for predicting differences between isoluminant colours. The multiscale local contrast enhancement reintroduces lost discontinuities only in regions that insufficiently represent original chromatic contrast. All operations are restricted so that they preserve the overall image appearance, lightness range and differences, colour ordering, and spatial details, resulting in perceptually accurate achromatic reproductions of the colour original.
In 3D rendering, the virtual camera settings are fixed, meaning that not all relevant scene information may be visible. The 3D Unsharp Masking approach enhances local scene contrast by unsharp masking over arbitrary surfaces under any form of illumination. The adaptation of the well-known 2D unsharp masking technique to 3D interactive scenarios is designed to aid viewers in tasks like understanding complex or detailed geometric models, medical visualization and navigation in virtual environments. Its holistic approach enhances the depiction of various visual cues, including gradients from surface shading, surface reflectance, shadows, and highlights, to ease estimation of viewpoint, lighting conditions, shapes of objects and their world-space organization. The operatation runs at real-time rates on a GPU and the effect is easily controlled interactively within the rendering pipeline. It is validated by psychophysical experiments showing that the enhanced images are perceived as having better contrast and are preferred over unenhanced originals.
First, I'd like to thank my co-authors Pierre-Edouard Landes and Tobias Ritschel for their readiness to try new ideas, their love for designing software and for indulging my insistance that images can always look better. To Thorsten Grosch for his careful reading and editing. To Matthias Ihrke for running a valuable user study on the Cornsweet effect in 3D.
I owe much to the many professors and researchers who I've had the fortune to work with. To Allison Klein, who introduced me to computer graphics and put me through my first SIGGRAPH challenge. To Victor Ostromoukhov, who encouraged me to continue my doctoral work and facilitated my move to MPI. To George Drettakis, whose guidance safely steered my research to publication and whose faith in my abilities was a much needed encouragement. To Joëlle Thollot, you are a remarkable woman and supervisor - you've selflessly nurtured my research, have accepted my failings, and shown by example what it is to be a scientist, artist, mentor and mediator. To Karol Myszkowski, my supervisor, for introducing me to the Cornsweet illusion and colour theory, and especially for his patience and sound advice. I am forever grateful for the freedom you have given me in my research - for never saying 'no'.
To Hans-Peter Seidel, MPI, IMPRS and all the administrative people I have irritated in Saarbrücken with mis-filled forms and special requests. I appreciate the travel opportunities, good working environment and leave for both my research visit to France and internship at Google. To the ARTIS/AERIS group and everyone at INRIA Grenoble, you are a lovely team: supportive, encouraging and fun. To everyone who edited something I've written, and to the reviewers of various computer graphics conferences and journals, your comments and advice are invaluable.
To Margaret Livingstone, whose book on vision and art inspired me to look at graphics with an artistic eye. To Floyd Raliff, whose pioneering work on lightness and colour phenomena and their connections to artistic techniques was, and still is, an inspiration. His writing is admirable, and he authored the only paper that, over three long years of paper reading, made me laugh.
To everyone on my email list who particiapated in many user studies, my apologies that none of that research is presented here. To Saschka Unseld for the mice and encouragement. To my Saarbrücken supporters, for yoga, dinners, travel and lifetime friendships. To Tom Annen for treats, laughs, German translation, and for being an amazing colleague and cherished friend. To my parents and sister, for their continued love and support of a perpetual adult student.