Compounding this situation is the wide variation of effects that contrast enhancing auxiliary components can introduce into the microscope optical system. Brightfield, darkfield, phase contrast, DIC, polarized light, Hoffman modulation contrast, and fluorescence illumination all present different manifestations to color balance correction, which must often be addressed by considering the specimen and illumination conditions on an individual basis. Presented in Figure 2 are several digital images captured under conditions of varying illumination color temperature and contrast enhancing methodology.
An eosin and hematoxylin stained thin section of human testicular cancer seminoma in tungsten-halogen illumination is illustrated in Figure 2 a. Note the overall yellow cast that pervades the entire image and renders the stained moieties off-color when compared to a properly color balanced image illustrated in Figure 7 b. This is a common error that occurs in brightfield microscopy when a daylight color conversion filter is not inserted into the light pathway.
Addition of a blue daylight filter to the optical path without correcting the digital camera for white balance can result in overall bluish tones to a digital image, as illustrated in Figure 2 b. This image of living HeLa cells in monolayer culture reveals the blue cast that occurs when the camera is not correctly color balanced. Applying white balance algorithms to the capture software will render the image in the grayscale values observed in the microscope eyepieces.
Images obtained with other contrast enhancing techniques yield similar problems when microscope illumination is not balanced for daylight color temperature and the camera system does not have the white balance properly adjusted. In Figures 2 c , 2 d , and 2 e , images captured with differential interference contrast DIC , polarized light, and Hoffman modulation contrast, respectively, all have color balance values shifted to warmer yellow hues. In the DIC image Figure 2 c , features appear muddy and normal grayscale tones are rendered in various shades of brown and red.
Likewise, the polarized light image of recrystallized urea Figure 2 d appears too green, while the Hoffman modulation contrast image of a radiolarian Figure 2 e has a decidedly green background and highlights. Fluorescence images Figure 2 f typically do not suffer from color balance problems, primarily because they are dominated by a small range of wavelengths. The phenomenon of variation in color balance or color rendition is well recognized by most individuals in everyday activities and is usually accepted as a natural occurrence not requiring any intervention.
For example, the golden quality of daylight near sunset is very familiar, as is the fact that colors appear dramatically different in candlelight as opposed to fluorescent office lighting. The human visual system functions by combining the sensory response of the eyes with interpretation of signals by the brain to accommodate variations in light color and intensity.
As a result, white objects are interpreted as white under widely varying conditions of illumination. Typically, if white is perceived correctly, other colors and hues fall into place as well. In contrast, image sensors, whether traditional film or a modern digital camera, produce a response to illumination that is fixed at the moment of exposure. The color qualities of the image produced will depend upon the specific response of color-sensitive layers designed into the film, or the sensitivity of individual color-sensing elements pixels of the solid-state sensor.
With either capture method, the color balance of the image can be modified by introduction of color filters into the illumination or imaging light paths, but the digital method has the distinct additional advantage of allowing electronic, precise adjustment of the sensor response. A sensor employed to record images, whether conventional photographic film or a digital imaging device, is generally designed or adjusted so that its base-line response matches a broad general category of illumination. Photographic films, for example, are most commonly manufactured in two major categories, suitable for use in either daylight or with tungsten illumination sources, and fine adjustments to the film response for critical applications are made by using appropriate filters.
Solid-state sensors, which are typically charge-coupled device CCD or complementary metal oxide semiconductor CMOS photodiode detectors, are capable of being adjusted electronically to match their response characteristics to a variety of illumination sources. The individual light-sensing elements of CCD or CMOS detectors are inherently monochromatic and achieve their color sensitivity either by sequentially passing the incident light through red, green, and blue filters onto the entire sensor producing separate images for each color, which are subsequently combined , or through miniature polymeric thin-film filters that are placed in a mosaic pattern over each pixel of the array.
The most common filter arrangement is an ordered mosaic array of red, green, and blue colored filters that repeats a G-R-G-B pattern over the entire sensor array. This arrangement, termed a Bayer filter pattern see Figure 3 a , incorporates twice as many green elements as red or blue. The additional green sensor pixels allow the imaging device to more closely approximate the color response of the human visual system, which peaks in sensitivity in the green spectral region approximately nanometer wavelength; Figure 3 b and, therefore, facilitates output of images having visually acceptable color balance.
Adjustment of the separate red, green, and blue signal amplitudes from the corresponding pixels or single-color images of the sensor array is implemented through the white balance control function to allow color balancing of the acquired image. Some camera systems execute these adjustments through software instead of, or in addition to, the hardware control. Digital cameras intended for general-purpose applications are familiar to many microscopists and are increasingly being adapted for attachment to microscopes as an economical alternative to dedicated research-grade imaging systems, although their capabilities are usually more limited.
Because the techniques involved in utilizing digital cameras for conventional purposes may be extended to understanding factors such as white balance adjustment in microscopy applications, it is useful to initially consider the non-technical situation. The basic concepts that govern white balance adjustment are the same for general photographic applications and for imaging in microscopy. Conventional digital cameras typically provide the user with a number of different white balance settings that are selectable as "presets".
These may correspond to broad lighting categories, such as daylight sunny or cloudy conditions , tungsten, fluorescent, or a variety of other illumination scenarios. Many cameras allow the preset values to be fine-tuned to achieve more precise color balance of images. Some cameras have the additional capability to adjust white balance by reference to a white card, a wall, or another object that should be represented as white if included in an image.
In practice, the camera is positioned so that the white object fills the field of view, and white balance adjustment is initiated by switch settings or selection in an operation menu depending upon the specific camera , after which the camera makes appropriate sensor adjustments to render the target as white. Adjustment by reference to a defined white object is conducted under the same illumination conditions employed during acquisition of the image and can provide highly accurate color balance calibration.
The procedure must, however, be repeated if the illumination changes. Advanced photographers often choose to modify their images by employing white balance settings other than the ones that match the illumination in order to achieve a desired aesthetic effect. For example, an image can be made to appear cooler or warmer in tone than it would if acquired with the "correct" white balance.
Such effects are, of course, considered errors if accurate scene rendition is the intention, similar to using daylight-balanced film in tungsten illumination, or vice versa. A popular technique of color-balancing, which should generally be avoided in critical applications, is commonly referred to in consumer cameras as automatic white balance adjustment.
This method is intended to be applied to the image field as the image is acquired and functions by evaluating the overall field of view, averaging the light values present with respect to hue, and attempting to average, or zero-out, any overall color bias. The shortcoming of automatic balancing techniques is that the color values present in any viewfield represent an "average" distribution of hue, which are combined to produce a neutral gray or white. In effect, if the summed pixel response is not similar to the programmed expected overall average, the white balance adjustment made by the camera will not produce accurate color rendition.
The typical specimens viewed in the microscope vary widely in color distribution and often exhibit a single predominant color especially in fluorescence. It is likely that an automatic white balance adjustment performed on a specimen exhibiting predominantly red tissue stain will produce substantially different color balance than the same procedure applied to a blue-stained preparation. Neither is likely to result in an accurate specimen representation. Attempts by the camera circuitry to balance the detector response to output an averaged overall color value will produce substantially different results on different specimens, particularly if a given viewfield has strong or dominant colors.
There are, of course, examples of specimens that produce acceptable results with automatic white balancing most likely those with a large proportion of white or gray areas , but the technique lacks the reproducibility necessary to make it routinely useful. In considering the different approaches discussed for optimizing white balance, it is obvious that some techniques do not lend themselves to the constraints and demands of optical microscopy.
Utilizing preset values for specific illumination types assumes that the characteristics of the light source are fixed and have standard values of color temperature and other spectral qualities. When tungsten halogen lamps are employed, it is common practice in microscopy to vary the lamp voltage to control the light intensity or minimize heat generation. Doing so produces a variation in illumination color temperature, which leads to incorrect color balance if a standard preset value for tungsten-type lighting is used on the digital camera.
An additional source of color rendition variation results from the changes that occur in color output as lamps age during their useful life. Similar problems exist with light sources that are optimally balanced for the daylight approximately K color temperature region. Not only is the color temperature of "daylight" variable, but few artificial sources accurately mimic daylight spectral qualities.
These difficulties could, in theory, be at least partially overcome by allowing automatic circuitry to correct for minor illumination fluctuations, but other problems often make this approach undesirable. With automatic evaluation of the image field, localized specimen variations can produce substantial errors in color balance. In general, the best approach for most microscopy applications is to restrict the white balance evaluation to a carefully chosen image area or other suitable target. When digital capture devices are utilized to image color specimens in optical microscopy, obtaining correct color balance to provide a true representation of the specimen is usually the primary goal.
- Illumination, Colour and Imaging: Evaluation and Optimization of Visual Displays.
- Beginning ASP.NET 2.0.
- Book Illumination, Color And Imaging: Evaluation And Optimization Of Visual Displays 2012.
- Theory and Practice in the Bioarchaeology of Care.
- Journal Papers.
Intentional deviations from this strategy are usually made only to correct a problem with the specimen preparation that produces an undesirable color cast. Most scientific grade digital cameras, including those specifically designed for microscopy, rely on adjusting white balance by reference to a selected color value. In transmitted illumination, an appropriate region usually white or a neutral gray is chosen from the specimen field or the adjustment is performed on the illuminated field alone, with the specimen removed from the light path.
In order to perform white balance adjustment in a microscope utilizing reflected illumination, a white or neutral-gray card or section of paper can be positioned on the microscope stage in place of the specimen.
- Concurrent Engineering and Design for Manufacture of Electronics Products.
- Publications & Presentations.
- Spanish Dance in B-flat Major, Op. 12, No. 4.
- A Universe Of Consciousness: How Matter Becomes Imagination.
- Illumination, Colour and Imaging: Evaluation and Optimization of Visual Displays - Semantic Scholar.
- Color rendering index;
- Color rendering index - Wikipedia!
The white balance setting is subsequently acquired by measuring the light reflected from the surface of the white card. The majority of digital cameras designed for microscopy are controlled through software residing on a host computer and are often configured to interact with a number of microscope functions. The software interface for the Nikon DXM digital camera system, for example, is representative of commercial products currently available with respect to the manner in which white balance adjustment is achieved.
When the white balance adjustment window is activated in the user interface, options are made available for selecting an area in the viewfield for white balance evaluation by the camera system. The live image on the display monitor should be carefully evaluated for an appropriate white or neutral gray area to serve as a reference point for the image sensor. If the image displayed on the monitor has a color cast that differs from the color balance observed in the microscope eyepieces, the camera system must be adjusted for white balance in order to render an accurate image of the specimen.
Ideally, the displayed color cast will be removed by the color balance circuitry of the camera when the proper specimen area is selected for white balance adjustment. Several typical examples of specimen areas that can be employed to set digital camera white balance algorithms are presented in Figure 4. The specimens are a living culture of fibroblasts imaged with differential interference contrast Figure 4 a , a quadruple-stained thin section of starch granules in potato tissue under brightfield illumination Figure 4 b , and human red blood cells in phase contrast Figure 4 c.
The regions on each image that are suitable for white balance adjustment using the area selection technique are outlined in red, while the yellow arrows indicate specific points on the images that may produce satisfactory white balance calibration when selecting a single pixel. The region selected as a white balance reference should be as large as possible and free from the coloration effects of specimen stains that have bled into the mounting medium. The white balance adjustment software in many systems enables the selection of either a single point pixel in the image, or a larger area that may be designated by marquee selection with the mouse cursor.
Better results are generally obtained by selecting the largest possible region. A much wider variation in results can occur if a single point is chosen for adjustment, because fluctuating localized combinations of red, green, and blue pixel intensities can contribute to the overall visual effect of white. By selection of a larger area, an average is obtained over a larger number of pixels in the sensor array, with improved probability of accomplishing acceptable color balance. Following reference area selection, the white balance adjustment is initiated, and the camera system utilizes either an algorithm or look-up table LUT to set appropriate electronic values such as sensor gain for each of the component colors that produce a neutral or white color value.
As discussed previously, the color balance of a digital image is heavily influenced by the spectrum of wavelengths gathered by the CCD or CMOS image sensor, regardless of whether the sensor is housed in a camera, telescope, laser bench, or microscope. In color digital cameras that employ these solid-state devices, a range of balance adjustments is often necessary in order to produce acceptable color images that conform to the color temperature of the illumination source. Several guidelines for successfully achieving proper color balance should be considered:.
Situations are often encountered in which acceptable white balance cannot be achieved during image capture by following the usual protocol. In these cases, non-standard techniques can sometimes be employed that will effectively "deceive" the white balance function of the camera to produce a specific color balance, which may or may not be considered correct, but achieves the desired effect. If even this strategy fails to provide acceptable color rendition, or if existing images were initially acquired with poor color balance, post-acquisition image processing with digital image editing software such as Adobe Photoshop can provide some degree of correction.
The basic technique for "forcing" the white balance function to deviate from its normal behavior is to perform the white balancing on a color other than white. If a non-white hue is presented to the camera as being white, the sensor gain circuitry will attempt to push the output toward the opposite or complementary color to compensate for the non-white hue. In effect, the relative magnitude of the red, green, and blue channels is altered to reproduce the color as white, while at the same time pushing other colors in the image toward the same complementary hue.
For example, an image having a reddish cast that is not acceptably corrected by the camera's circuitry can often be balanced by calibrating on a red reference target. In an effort to reproduce the red target as white, the blue and green pixel output of the sensor are both increased, adding the complementary cyan hue necessary to compensate for the red.
By similar logic, color balancing on a yellow target would result in the addition of blue to the overall image. Application of this technique for color balancing requires careful analysis in order to determine which hue can effectively be added or subtracted to correct the image problem. Presented in Figure 5 are several digital images of specimens that suffer from color casts as a result of masks or preparation errors.
The integrated circuit imaged in reflected light differential interference contrast illustrated in Figure 5 a contains a silicon nitride passivation layer on the surface that acts similarly to a yellow filter. Wavefronts reflected from the surface of the chip must pass through the coating before reaching the objective, and many of the shorter wavelengths blue and green are blocked. By calibrating the digital camera white balance on the point indicated by the yellow arrow in Figure 5 a , the yellow cast created by the passivation layer is largely eliminated to yield an image having excellent color balance Figure 5 b.
Likewise, an overstained thin section of American basswood tree , depicted in Figure 5 c , can be corrected in the same manner. Selecting a pixel for white balance calibration in an area devoid of tissue see the yellow arrow; Figure 5 c yields an image with a clean, white background and nicely saturated colors Figure 5 d. Fluorescence specimen preparations can often bleed unbound fluorophores into the surrounding mounting medium to create a similar "overstaining" effect, as illustrated in Figure 5 e for a fern thin section.
Setting the white balance digital camera calibration with a pixel on a counterstained feature yellow arrow in Figure 5 e often reduces the amount of background fluorescence Figure 5 f in the final image. In order to apply the techniques described above, a variety of color filters is often useful for transmitted light configurations. Filter sets designed for color photographic printing are appropriate for this purpose.
The sets contain a range of densities in each primary color and can be combined to produce any hue required. For reflected light microscopy, suitable reflection color references are required for the non-white balancing. It is not necessary that the targets conform to any color standards, and experimentation is usually required to produce the desired result. Any colored reflective surface can be employed, but it is desirable to have available a wide range in variation of hue and saturation. The paint color sample cards available at home centers or paint stores are ideal for the purpose, as they are provided in nearly every conceivable color variation.
In some cases, selected regions of the specimen itself can be utilized to set off-color white balance calibrations. Whether utilizing filters in transmitted light or reflective targets such as the paint sample cards in reflected light, the concept of manipulating the camera's white balance function is the same: white balance calibration on a non-white color will cause the camera's circuitry to remove the target color and render it as a more neutral gray hue.
In most cases, only a subtle change is desired, and experimentation will determine the hue and saturation of target that will produce the necessary change in overall image balance. Balancing on a pale blue color will cause an overall warming effect, or a shift toward red. Conversely, employing a pale red color as the reference will produce a bluish shift toward cooler color balance. Other color corrections follow the same general logic. Performing white balance on any given color will tend to cause the camera's circuitry to shift the color balance toward the complementary color.
It is important to emphasize that these non-conventional color balancing techniques are a potential mechanism for achieving the desired result when the usual methods have failed for reasons related to a particular specimen preparation, light source, or imaging device.
In these cases, it still may be possible to acquire acceptable images by offsetting the camera's response to the specimen color palette. While it is always preferable to initially acquire images with proper color balance, some degree of correction is possible after acquisition by application of post-processing operations through image editing software. These procedures are not a substitute for proper in-camera white balancing and must be used judiciously in order to avoid unacceptable changes in specimen rendition.
General alterations to color balance will affect all areas of the image, but this is sometimes an acceptable compromise since minor changes will produce comparatively large variations in background tone while having a smaller effect on more intensely colored specimen features. Adjustments to color balance made in image editing programs can take on different forms depending upon the level of control desired. However, the estimation of material properties, such as glossiness, is a classic ill-posed problem.
Image cues that we rely on to estimate gloss are also affected by shape, illumination and, in visual displays, tone-mapping. Here, we focus on the latter two. We manipulate the illumination field to violate statistical regularities of natural illumination, such that light comes from below, or the luminance distribution is no longer skewed. These manipulations result in errors in perceived gloss.
Illumination, Colour and Imaging: Evaluation and Optimization of Visual Displays
Similarly, tone mapping has a dramatic effect on perceived gloss. However, when objects are viewed against an informative rather than plain gray background that reflects these manipulations, there are some improvements to gloss constancy: in particular, observers are far less susceptible to the effects of tone mapping when judging gloss. We suggest that observers are sensitive to some very simple statistics of the environment when judging gloss.
Purchase this article with an account. Jump To Introduction Research questions Light-field analyses Experiment 1: Gloss constancy across changes in illumination Experiment 2: Gloss constancy across tone mapping Discussion Acknowledgments References. Open Access. Adams ; Gizem Kucukoglu ; Michael S. Author Affiliations Wendy J.
Connected Content. Errata: Corrections. Journal of Vision December , Vol. Alerts User Alerts. Naturally glossy: Gloss perception, illumination statistics, and tone mapping. You will receive an email whenever this article is corrected, updated, or cited in the literature. You can manage this and all other alerts in My Account. This feature is available to authenticated users only.
Get Citation Citation. Get Permissions. We identify a wide range of materials e. We can usually predict how an object will feel before we touch it; we are able to judge material qualities, including surface gloss from a single static image. Glossy reflections are commonly modeled in computer graphics by the proportion of incoming illumination that is reflected specularly i.
A surface that reflects a large proportion of incoming light in a perfect, mirror-like way will have bright, sharp specular highlights. Therefore, these are valid cues to gloss. Figure 1. View Original Download Slide. In each image one property illumination, reflectance, or tone mapping was modified while the other two were kept constant. Unfortunately for the observer, other factors also affect these gloss cues.
The pattern of specular highlights varies according to the shape of the reflecting surface and the pattern of incoming illumination.
IEEE Xplore Full-Text PDF:
For example, specular highlights are more spatially compressed and thus brighter at regions of high curvature. In addition, highlights will be brighter when a greater proportion of the incoming illumination is directional, rather than ambient e. For humans to achieve gloss constancy, they should estimate not only an object's reflectance, but also estimate and compensate for object shape and the illumination.
When viewing a glossy nectarine at a sunny picnic, we don't want to be confused when the sun goes behind thick clouds, and accuse our copicnickers of switching it for a matte peach Landy, To complicate the task further, most images that we view in print or via a screen have been tone mapped to accommodate the limited range of available luminance values. Luminance contrast in real scenes considerably outstrips that which can be displayed on a standard monitor. Previous work suggests that in the absence of information about illumination i. Fleming, Dror, and Adelson presented glossy spheres that had been rendered under various artificial illumination fields.
For highly unnatural illumination e. In the experiments of Fleming et al. When stimuli do not contain information about the current illumination, observers only have their prior knowledge of illumination to guide gloss perception. In a similar vein, Olkkonen and Brainard and Pont and te Pas found that while observers are able to match the specular reflectance of objects rendered under the same illumination, they show failures in gloss constancy when comparing objects rendered under different illumination fields.
Normal viewing situations usually provide observers with some information about the current illumination, which could be used to optimize gloss judgments. When such information is available, do observers make use of it, to achieve partial or full gloss constancy over illumination changes? Motoyoshi and Matoba presented scenes depicting a complex object a statue in a room amongst other objects, and varied the illumination.
They found failures in gloss constancy when observers were asked to match surface reflectance across different illumination environments. In addition, manipulating the object's background varying contrast or gamma had no effect on perceived gloss. In other words, observers failed to use the object's context to compensate for illumination changes and achieve gloss constancy. The authors propose that observers use simple image statistics from within the object's image to infer gloss, and ignore the background.
A few factors should be kept in mind when interpreting these results: First, the same object albeit from a slightly rotated viewpoint was compared across scenes, which may have encouraged image matching, rather than gloss matching. Second, the outdoor illumination fields used for rendering were inconsistent with the indoor scene presented, and third, all stimuli were presented in grayscale, which may have limited the observers' abilities to segment specular from diffuse reflectance.
Doerschner, Maloney, and Boyaci directly tested the effect of the mean luminance of an object's context using real scenes and found that this can affect perceived gloss. Observers viewed real spheres either matte black with a white painted dot, or glossy black in front of a black or a white background.
Objects were perceived as somewhat glossier when presented against a black, rather than a white background. In summary, when information about the illumination is unavailable, observers must and do rely on prior assumptions about the illumination structure. Some authors suggest that even when cues to the current illumination are available, they are not used. Instead, we rely on certain characteristics of natural illumination environments such as dynamic range, skew, distribution of wavelet coefficients, and the dominant direction of illumination Fleming et al. In Bayesian terms, this reliance would amount to giving all weight to the prior, and none to the available information about the current illumination.
For this approach to be successful i. In this case, observers could estimate gloss directly from the object's image ignoring the context and gloss constancy would come for free, without any compensatory mechanism. Thus, models of gloss perception that ignore this variation predict that our perception of an object's material will vary across different natural illumination fields as well as across artificially manipulated ones.
In other words, we would fail to show gloss constancy. In addition to changes in illumination, tone mapping can also alter the luminance and contrast of specular highlights. Studies of gloss perception often use a sigmoidal tone mapping Fleming et al. Phillips, Ferwerda, and Luka used a high-dynamic-range HDR display to compare the perceived gloss of stimuli with and without a sigmoidal tone mapping.
Tone-mapped stimuli were perceived to be substantially less glossy than HDR stimuli. Similarly to Fleming et al. An interesting question thus remains: Would observers be gloss constant over tone-mapping manipulations, if more contextual image information were available? We can compare gloss constancy with lightness and color constancy. Well-known demonstrations of lightness and color constancy show that the perceived hue or lightness of a surface patch is strongly affected by the context in which it is viewed Chevreul, ; Mollon, As information about the current illumination increases, lightness constancy improves.
Snyder, Doerschner and Maloney asked observers to judge the albedo of a surface patch within a 3D scene. Lightness constancy improved when specular spheres which provided information about the illumination context were added to the scene. Thus, for lightness judgments, observers act in a Bayesian manner, combining prior and current information about the illumination to optimize perception and improve constancy.
In contrast to the extensive work on color and lightness constancy, relatively little is known about how or the extent to which observers combine prior knowledge about illumination with online sensory cues to achieve gloss constancy. Here we investigate a set of related issues: First, we analyze the structure of a diverse set of natural illumination environments from the Southampton-York Natural Scenes SYNS dataset Adams, Elder, et al. Second, we render objects under natural and manipulated illumination environments to determine the effects on perceived gloss when statistical regularities are violated.
Third, we investigate the effects of tone mapping on perceived gloss. Finally, we ask whether observers can exploit contextual information within an image to compensate for changes in illumination, or tone mapping, in order to achieve gloss constancy. To preview our key results:. Manipulating two of these three characteristics affected perceived gloss: the luminance distribution and the dominant illumination direction. Providing explicit information about illumination, by presenting objects within their true environment, did have some effect on perceived gloss, but did not lead to gloss constancy.
Conversely, tone mapping had a substantial effect on perceived gloss, but only when objects were presented in isolation; when the whole image was present, tone mapping had a much smaller effect on perceived gloss.
Beyond the Numbers
In summary, observers are disappointingly vulnerable to biases in perceived gloss when substantial, salient changes are made to the illumination. However, as long as an object is viewed within the context of a larger image, tone-mapping has only a small effect on gloss judgments; some compensation occurs such that near gloss constancy is achieved.
The current analyses included 72 of these light-fields that were sampled at unique locations across Hampshire, UK, within a diverse array of scene categories 20 outdoor categories, six indoor categories. Figure 2. Eight example light-fields from the SYNS dataset. Some light-fields that were captured in full sun included an artifact below the sun—a bright narrow vertical strip—this has been removed by replacing the affected part of the image with corresponding pixels from a second image, captured a few minutes later.
Previous studies have also explored the statistical characteristics of natural illumination. For example, most pixels in each light-field were low luminance, with a few very high luminance points due to small bright sources. This allowed them to record light-fields in low resolution up to second-order spherical harmonics. Similarly, Morgenstern, Geisler, and Murray built a multidirectional photometer consisting of 64 evenly spaced photodiodes to collect relatively low-pass light-fields. Their analyses suggested that illumination fields are relatively diffuse, and that observers' reflectance estimates across changes in surface orientation are consistent with an assumption of similarly diffuse illumination.
The current analyses extend previous work by using the SYNS high-resolution, spherical illumination maps, sampled in a principled manner from diverse scene categories. One of the simplest statistics we can consider is the distribution of luminance across the light-field, summarized by the luminance histogram Figure 3 , left column. We confirm previous assertions that natural illumination tends to be skewed note the logarithmic x axis : there are a few very bright pixels.
Figure 3. Luminance distributions left and luminance as a function of elevation right. The luminance scale is arbitrary. Rows represent light-fields from indoor upper , outdoor cloudy middle , and outdoor sunny scenes lower. Each gray line corresponds to a single scene; black lines show the median, and green lines show the 10th and 90th percentiles. In agreement with previous work Dror et al.
This effect is especially pronounced for scenes captured during sunny, rather than overcast conditions Figure 3 , right column. We can analyze the power distribution within a spherical image the light-field in an analogous way using spherical harmonics. Luminance contrast decreases with increasing spherical harmonic order increasing frequency in angular terms. As noted previously Dror et al. Figure 4. Spectral power distribution. The averaged squared coefficients per order, as a function of spherical harmonic order in luminance left column or log luminance right column.
We ask whether observers rely on these fairly ubiquitous characteristics of illumination fields when estimating surface gloss.
Join Kobo & start eReading today
Moreover, when information about the illumination field is available, can observers maintain gloss constancy when the illumination deviates from these characteristics? Experiment 1 investigates these questions: We render glossy objects under natural and manipulated illumination fields and measure the effect on gloss perception. Experiment 1: Gloss constancy across changes in illumination. Eight outdoor light-fields were selected from the SYNS dataset as shown in Figure 2 to represent a range of scene categories and weather conditions sunny vs.
As noted above, natural light-fields tend to be highly skewed, with just a few bright pixels, and previous researchers have suggested that skew may contribute to perceived gloss Motoyoshi et al. For our uniform condition, we manipulated each light-field to have a uniform luminance distribution while preserving the spherical harmonic power spectrum.
To this end, we followed an iterative process inspired by Heeger and Bergen's procedure for texture synthesis. First, luminance values were adjusted preserving rank order to create a uniform luminance distribution. Then, spherical harmonic power at each order was reset to its original value. These two steps were repeated seven times, after which the image changes were minimal.
Figure 5. Above: an example light-field in its original form standard , and after the three different illumination manipulations. Below: example stimuli, rendered under standard illumination, to illustrate the nine different stimulus gloss levels. For our half-slope condition, the power distribution was manipulated to boost contrast at high, relative to low angular frequencies, while preserving the luminance histogram. We investigated the role of overhead illumination in gloss perception by reversing the direction of the first directional spherical harmonic component, to create our Halloween condition.
Following manipulations, light-fields were linearly scaled such that all light-fields, across all conditions, matched in mean luminance. Finally, the original hue was restored before reverting to RGB format.
Edit This Favorite
Complex and varying shapes were used instead of spheres to encourage subjects to judge gloss, rather than performing simple image comparisons. Stimuli were rendered as though impaled above a transparent pedestal to enhance the impression that the virtual object was embedded within the scene. Examples are shown in Figure 6. Figure 6.
Stimuli rendered under standard, uniform, half-slope, and Halloween light-fields. On the left, moderately glossy stimuli level 3 are presented with the background visible. On the right, high-gloss stimuli level 7 are presented against a gray background, rather than the true rendering context.
Below, the schematic shows the slow reveal used in Experiment 1. Stimuli were prerendered using Octane Render Version 1. Stimuli were rendered with nine gloss levels, defined by the parameters of the renderer that correspond to a specular strength, i. The diffuse parameter was fixed at 0. See Figure 5 for examples of the nine gloss levels, rendered under standard illumination.
The experiment was carried out at two locations: New York University and the University of Southampton. At the University of Southampton, the experiment ran on a in. The luminance response gamma was measured for the monitors in each location. All stimuli were tone-mapped using the same function Equation 1 and then, inverse gamma corrected separately for each monitor to maintain linearity when displayed. We prefer this approach over adjusting the monitor to have a linear output, since the latter produces visibly discretized luminance at lower levels.
The tone mapping was designed such that intensities up to the 99th percentile of all pixel values across all stimuli were linearly scaled. Observers were seated 55 cm from the display and viewed the stimuli monocularly with their head stabilized via a chin-rest. Twelve observers completed the study: six at New York University, six at the University of Southampton. All observers, except for GK one of the authors , were unaware of the purposes of the experiment. All observers had normal or corrected-to-normal vision. Subjects gave informed consent prior to testing. Each stimulus was gradually revealed within the first 1 s of display time Figure 6 to encourage the observers to attend to the whole scene rather than only comparing the central objects.
Following stimulus offset, a response screen displayed a response prompt with a reminder of the response keys. Observers were given unlimited time to respond and were not given any feedback. On the majority of the trials of total trials , stimuli were either both presented in context, i. In the latter case, the luminance of the background matched the mean luminance of all backgrounds for the background-present stimuli.
On a smaller number of trials 90 , one stimulus was shown with and the other was shown without its background. When the two stimuli shared a common manipulation condition, their gloss levels differed by 1 to 3. In other cases they differed by 0 to 4. For example, a Halloween stimulus of gloss level 1 was compared to gloss levels 2, 3, and 4 within the Halloween condition.
Stimulus comparisons were selected to maximize informative trials i. Data analyses followed Thurstone Case V scaling Thurstone, Thus, data were pooled across light field identity and object shape within each condition. For Thurstonian scaling, for every possible pair of conditions e. In addition, the relationship between conditions is assumed to be transitive, i. First, we consider the effects of the three different light-field manipulations on perceived gloss, in the absence of information about the prevailing illumination Figure 7 , top row.
Our data suggest that observers have internalized certain characteristics of natural light-fields luminance skew, and possibly illumination from above and rely on these for gloss estimation. Figure 7. Data from Experiment 1 for the background-absent top row and background-present bottom row conditions. Second column: Group data, averaged across all observers. Third column: Summary data: mean perceived gloss for each illumination manipulation, averaged across gloss levels. Fourth column: Sensitivity, defined as the difference in perceived gloss, in JND units, between the most glossy and most matte stimuli.
Curves are 2nd-order polynomial fits to the data. There was no significant effect of manipulating the power distribution half-slope condition on perceived gloss in the background-absent condition. Under this manipulation, the luminance distribution was held fixed while varying the spectral slope. Thus, although natural illumination fields differ very little in their distribution of spectral power across frequencies, we don't seem to rely on this characteristic when estimating gloss. In contrast, perceived gloss is affected when there is a change in the predominant illumination direction, or the luminance distribution e.
Recall that we manipulated the illumination field rather than directly manipulating the stimulus images. Next, we consider whether any simple image statistics might explain the reduction in perceived gloss under the Halloween and uniform conditions. The effects of our manipulations on simple statistics of the luminance distribution within the image of the judged object are presented in Figure 8.
A linear regression reveals that, in the background-absent conditions, variations in perceived gloss are well approximated by a combination of Michelson contrast and skew. Figure 8. Simple statistics characterizing the luminance distribution within the object's image top row or the background context bottom row , averaged across light fields and random variations in object shape. Our primary question, however, is whether observers can use information about the illumination field, when available, to improve gloss constancy.
If observers completely ignore the context that the object is presented in, the data would be identical for the background-absent and background-present conditions. In contrast, if observers use the background information to compensate for the current illumination conditions, i. Importantly, we can compare perceived gloss across the background-present and background-absent conditions to see whether contextual information is used when judging perceived gloss. However, it does not, as we hypothesized, improve gloss constancy. When the background is present, stimuli rendered under the half-slope condition are perceived as less glossy than those rendered under a normal light-field and less glossy than when the same stimuli were presented against an arbitrary gray background.
Why would the presence of the background reduce perceived gloss in the half-slope condition? Changing the slope of the power spectrum while preserving the luminance histogram produced multiple small bright spots within the visible background; the contrast of the background was increased relative to standard illumination , whereas the contrast within the object region was reduced. A parsimonious explanation of both the background-present and background-absent conditions is thus that observers are sensitive not only to contrast and skew within the object's image, but how these characteristics compare to the luminance profile of the background Figure 8.
This makes sense: high-contrast illumination fields with large skew i. In the half-slope condition, skew is increased, but contrast is decreased within the image of the target object relative to the standard condition. Accordingly, there is little change in perceived gloss following this manipulation in the background-absent condition. However, both contrast and skew are substantially elevated within the background, and this is accompanied by a decrease in perceived gloss in the background-present condition.
In the Halloween condition, skew within the object's image is similar to that of the standard stimulus whereas contrast is decreased. This combination is accompanied by a decrease in perceived gloss in the background-absent condition. Within the background, skew is increased relative to the standard condition, and Michelson contrast is approximately unchanged. Accordingly, when the background is present in the Halloween condition, there is a small reduction in perceived gloss relative to background-absent.
In the uniform condition, large decreases in skew and contrast within the object's image are accompanied by a large decrease in perceived gloss. Notably, however, the reduced contrast and skew in the background for this condition do not produce an increase in perceived gloss in the uniform, background-present condition relative to the background-absent condition.
It is possible that there is an asymmetry in the effects of contrast and skew within the surrounding context of a viewed object: Unusually large values have a suppressive effect on perceived gloss; however, a reduction in the highlight-inducing features of the background has little effect on perceived gloss. In fact, a similar observation can be made in relation to the effect of a uniform background in the background-absent conditions: A zero-contrast background does not inflate perceived gloss, as one might otherwise expect. Finally, we investigated whether the presence of the background improves gloss discrimination , indexed by the range of perceived gloss values in JND space for each condition Figure 7 , fourth column.
Background presence might improve gloss discrimination by giving observers explicit information about the illumination, which could serve a useful reference to correctly interpret specular highlights. To this end, we independently analyzed data from the background-present and absent conditions. This is consistent with the influence of luminance skew on gloss perception; skew increases more dramatically with stimulus gloss in this condition Figure 8 , top-right plot.