Exploiting Colorimetry for Fidelity in Data Visualization

02/27/2020 ∙ by M. J. Waters, et al. ∙ Northwestern University 0

Advances in multimodal characterization methods fuel a generation of increasing immense hyper-dimensional datasets. Color mapping is employed for conveying higher dimensional data in two-dimensional (2D) representations for human consumption without relying on multiple projections. How one constructs these color maps, however, critically affects how accurately one perceives data. For simple scalar fields, perceptually uniform color maps and color selection have been shown to improve data readability and interpretation across research fields. Here we review core concepts underlying the design of perceptually uniform color map and extend the concepts from scalar fields to two-dimensional vector fields and three-component composition fields frequently found in materials-chemistry research to enable high-fidelity visualization. We develop the software tools PAPUC and CMPUC to enable researchers to utilize these colorimetry principles and employ perceptually uniform color spaces for rigorously meaningful color mapping of higher dimensional data representations. Last, we demonstrate how these approaches deliver immediate improvements in data readability and interpretation in microscopies and spectroscopies routinely used in discerning materials structure, chemistry, and properties.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 3

page 5

page 10

page 11

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Scientific, engineering, and medical research are expensive endeavors and include studies ranging from fundamental materials chemistry to materials deployment. This broad community of materials chemists and its stakeholders seek to ensure that complex data sets generated in this enterprise are accurately perceived; however, researchers have become increasingly aware of how 2D representation, in the form of color maps or heat maps, which define the conversion of raw data to colors on a screen or in print, can be poorly constructed to obscure or over-represent data 1, 2, 3, 4, 5. Furthermore, 2-3% of the human population suffers from deuteranomaly—the most common form of red-green blindness. 6, 7 Which limits their ability to differentiate hues in the red-orange-yellow-green palette. Color maps, therefore, should be inclusive for broad accessibility while providing a faithful representation.

The current state of scientific data visualization uses Red-Green-Blue (RGB) values directly when creating visual representations of experimental or computationally acquired data. RGB color spaces, are conceptually easy to comprehend as 3D coordinate systems that approximate the tristimulus values, i.e. the stimulation of the red, green, and blue receptors in the human eye. Although there are multiple RGB standards and high-dynamic range (HDR) standards will improve their gamut, the fallback choice remains the sRGB standard.8 For data visualization, there are many issues with using the sRGB color space—some disadvantages are described below.

The main limitation being that it is not a perceptually uniform color space; if one specifies uniformly spaced hue values, the corresponding colors will not disperse linearly to the human eye. Thus, the aspirational goal of optimal color usage for achieving fidelity in data visualization requires moving away from sRGB to the more appropriate perceptually uniform colors spaces (UCSs).

Figure 1: (Left) A photograph of Nobel Laureate Walter Kohn and two modifications of the image where (center) yielding gray scale and (right) removing lightness information. Lightness information is generally much easier to process. Images adapted from Ref. 9.

Today, large selections of off-the-shelf, aesthetically pleasing, perceptually uniform color maps for scalar fields are both available and can be readily designed with available tools. They are designed such that the Euclidian distance between any two color coordinates are a perceptual distance. They are, in essence, models of how humans process color information. 10 As a demonstration, we provide an original (Figure 1a) and variations (Figure 1b and c) of a photograph of Nobel Laureate Walter Kohn (Figure 1), owing to the immense number of materials-chemistry publications that include electronic structure calculations (and derived properties) from density-functional theory. The modified photographs illustrate how colorimetry choices affect data (image) perception. In Figure 1b, we remove the hue and saturation data whereas we remove the lightness data in Figure 1c. These transformations are trivial in UCS. Figure 1 also demonstrates how lightness information alone is easy to comprehend, while hue and saturation data alone are more difficult. The importance for data representation lies in that perceived changes between colors can be mapped intuitively in a linear fashion to the changes in data they represent.

Figure 2: The (left) CIELAB and (right) sRGB color space axes and their hues. The sRGB color space forms a unit cube where the origin is black, the cube vertices on the axes are red, green, and blue primaries, and white is on furthest vertex from origin.

The CAM02-UCS and CAM16-UCS colorspaces11, 12 represent the current state-of-the-art and should be utilized for diverse scientific visualizations. However the simpler and more widely used CIELAB is sufficient for demonstration purposes while being easily adoptable. In CIELAB, is lightness and and are color channels, where lightness ranges from 0 to 100 (unitless) and the black-gray-white trajectory lies along (Figure 2). The perceptual distance provides a measure of how far apart, or rather, how much contrast exists between two colors. Thus, one can infer the utility of using UCSs for data representation so as not to obfuscate variations in color contrast with variations in data (e.g., scalar magnitudes).

For color mapping data, we further link the color contrast to the perceivable data contrast using the derivative of the perceptual distance with respect to the data, , as . Having a constant perceptual derivative is what defines a perceptually uniform color map. Since most people only recognize a ,13

the perceptual derivative allows us to compute the minimum perceivable change in the data with a certain user- or instrument-specified color mapping. If a non-perceptually uniform mapping is made, then the value of the derivative will vary with the data values. . For simple scalar fields, perceptually uniform color mapping can be as simple as linear interpolation between two points in a UCS. However, to increase the perceptual derivative and thus the data contrast without sacrificing perceptually uniformity, a helical path along the lightness axis is often used. Another benefit of this path is that it is linear luminant—the lightness derivative is constant. This feature ensures good readability for those with color-blindness or in gray scale, since hue differentiation is not required. An alternative simple way to address the deuteranomaly is to ensure that red and green do not appear next to each other in the color space.

Recently, Matplotlib, MATLAB®, and other scientific and engineering software changed their default color maps to be perceptually uniform, linear luminant, and compatible with red-green color blindness. Nonetheless, ad hoc and/or purely aesthetically designed color maps remain pervasive in figures rendered for both electronic and print publication. In contrast, well-designed, perceptually uniform color maps exhibit uniform contrast both in lightness and hue across the whole color map while avoiding the red-green section of the spectrum. These maps are excellent for simple scalar fields such as height or density data (Figure 3). For amplitude data, which has both positive and negative values, there are perceptually uniform diverging color maps 14. Likewise, for cyclic fields, such as phase angle, there are perceptually uniform cyclic colormaps 14.

Figure 3: Electron density slice through a copper phthalocyanine molecule, computed with GPAW15, is colormapped with the ubiquitous jet colormap (left) and the perceptually uniform and linear luminant viridis colormap of Matplotlib (right).

Here we describe two tools to facilitate the use of UCSs and demonstrate their use with real materials data. First, we demonstrate the Phase-Amplitude Perceptually Uniform Colormap Designer (PAPUC) tool, which is implemented as a small library for creating perceptually uniform color mappings of vector fields where vector orientations/phase angles are mapped onto hue and vector magnitudes/amplitudes are mapped onto lightness. Next, we demonstrate the Composition Mapping within Perceptually Uniform Colorspaces (CMPUC) tool for performing perceptually uniform color mappings of three component composition fields. A common use-case for this is energy-dispersive x-ray spectroscopy (EDS) mapping of elements in multicomponent materials where traditionally red, green, and blue are used to represent different comprising elements. We then showcase the use of these tools, both written in Python and are integrated into the SciPy/NumPy/Matplotlib ecosystem,16, 17, 18 for a 2D vector field in a complex oxide heterostructure and a multicomponent composition mapping in the common hydroxyapatite biomineral. The tools are freely available at Refs. 19, 20 and use colorspacious 21 for back-end color space conversions. The Colour-Science22 package will likely be added as another back-end option in the future. The concepts we demonstrate are useful to data acquired in various scientific and technological fields; both of the aforementioned examples use data collected with transmission electron microscopy (TEM), which is a broadly utilized multimodal characterization tool in materials chemistry.

2 Methods

2.1 Vector Fields

In representing 2D vector fields, it is common to use the HSV (hue, saturation, value) color space, which is a mathematically simple transformation of sRGB. The value of each pixel, i.e., brightness and hue, are determined by magnitude and angle, respectively. Here we use the CIELAB color space to map the direction and magnitude of the vectors (Figure 4). Using the perceptually uniform representation, as implemented in PAPUC, the vector angle , is mapped to the hue angle, , in the - plane (Figure 4a) and lightness is scaled to vector magnitude so that the perceptually uniform color wheel has a conceptually familiar layout as the HSV color wheel (Figure 4b). When visualized in CIELAB colorspace, input vector data are mapped on to the surface of an inverted cone with an axis along , cone tip at , cone base at , and radius of 40 which can be seen in Figure 4c. (More details may be found in Ref. 19.).

Figure 4: (a) A slice of the CIELAB color space at . The black circle of radius 40 forms the base of the cone used in our perceptually uniform color mapping. The region that can be represented in the sRGB gamut is colored to match. The outermost line is the path that the outer ring of HSV takes through CIELAB space and is shown with is solid line where and broken elsewhere. HSV passes through the sRGB primaries at , and secondaries at if . (b) The perceptually uniform color wheel is shown with the origin hue shifted to match the HSV color wheel. Vector magnitude and angle are shown in 10% and increments, respectively, (c) The cone in the uniform color space upon which vector angle and magnitude are mapped. (d) The perceptual derivatives of the HSV and perceptually uniform color wheels with respect to angle. (e) Lightness as a function of vector angle for both color wheels.

The angle contrast is quantified by the perceptual derivative with respect to angle, , at the maximum magnitude the HSV perceptual derivative varies by an order of magnitude and is discontinuous (Figure 4d). Further complicating the HSV representation, lightness varies wildly with respect to the angle , which leads some angles to have inherently higher perceived magnitude (Figure 4e). This is a result of HSV passing through the sRGB primaries, where red and green are 1.65X and 2.72X lighter than blue. The perceptual derivative discontinuities occur at the sRGB primaries and their binary combinations of cyan, magenta, and yellow. Assuming a just noticeable difference of 13, our proposed mapping permits a minimum discernible angle of 3.3.

Figure 5: A chemical color mapping where red, green, and blue represent three different elements (P, Pt, and Ca, respectively) where (a) is perceptually uniform and the three color points form an equilateral mixing triangle in the - plane while (b) depicts a linear mixing triangle of the sRGB primaries, i.e., permutations of (1,0,0). (c) The inverted trigonal pyramid formed by perceptually uniform chemical mapping in the uniform color space.

2.2 Compositional Mapping

In mapping elemental composition in materials, typically the three sRGB primaries or their binary pairs, cyan, magenta, and yellow, are used to represent the measured concentrations of three components. In our approach implemented in CMPUC, we scale lightness, , to the total intensity (concentration). For the relative composition, three color points are chosen such that they are perceptually equidistant and centered at white () forming an equilateral triangle (Figure 5a). Ideally, the equilateral triangle in the - plane should be as large as possible so that the smallest changes in composition are perceivable. Thus, we use the largest white centered triangle that fits into the sRGB color gamut; which occurs at , circumradius of 60.14, and hue angle of 33.73. The circumradius of the triangle is scaled to the total intensity since the sRGB gamut converges to () as approaches zero. The overall mapping in our CMPUC code is essentially an inverted trigonal pyramid which has a triangular base where each base vertex corresponds a maximum signal that is purely of one component (Figure 5c).

In traditional RGB mapping, the foremost issue with the approach is that the primaries exhibit different lightness values. The lightness variation is so large that the component represented by green will be perceived as nearly 3X more concentrated than the component represented by blue. Although the sRGB primaries have maximal perceptual distance and thus, compositional distinguishability, they are not perceptually equidistant, such that there are more perceivable differences between blue and green than either of the other pairs. Additionally, linear mixing between them yields low contrast near pure compositions.

3 Use-cases

3.1 Application to vector-field visualizations

First, we show how perceptually uniform color mappings give immediate improvements in data readability and interpretation using our software tool PAPUC for a vector field consisting of nanoscale vortex patterns of electrical polarization in a PbTiO/SrTiO superlattice measured from atomic-scale Scanning Transmission Electron Microscopy (STEM).23 The rotational structure and magnitude differences of this vector field provide an ideal test for the interpretability of color mapping methods as they rotationally and radially traverse the entire color space.

Figure 6 presents a dataset containing ferroelectric PbTiO layers with a spontaneous polarization that form a series of vortices when alternated with non-polar SrTiO layers (see inset). The electrical polarization vectors are mapped with our perceptually uniform representation (left) and the common HSV (right) representation. Most notably, the apparent magnitude of the vector is well preserved in the perceptually mapping but not in the HSV mapping. The vectors principally along the Y axis render in opposing extremes of lightness, either low lightness blue (-Y) or a high lightness yellow (+Y) for a given magnitude, making accurate visual inspection challenging. For the same reason, the rotation of the vector field is also better preserved, such as the low magnitude “hole” at the vortex centers and a higher rotation gradient along preferential planes creating a slight “X” pattern. Finally, there is less hue distortion when using the perceptually uniform mapping under simulated deuteranomaly.

Figure 6: (a) Ferroelectric polarization domains in a super lattice of PbTiO/SrTiO as visualized by angle as hue and intensity as lightness with our perceptually uniform mapping and with HSV. (b) The same two images simulated with the deuteranomaly. Pixel spacing of 3.94 Å/pixel.
Figure 7: Energy-dispersive X-ray spectroscopic chemical mapping of platinum deposited on hydroxyapatite. Red, green, and blue represent P, Pt, and Ca signals, respectively where (a) is perceptually uniform and (b) uses linear mixing of the sRGB primaries, i.e. permutations of (1,0,0). Lengths are given in pixels. (c) The average signal for the bottom 20 horizontal line scans. (d) The chemically mapped images with simulated deuteranomaly.

3.2 Application to compositional maps

Next, we examine the application of our perceptually uniform composition mapping using CMPUC on TEM EDS composition data collected from a cross section sample of hydroxyapatite deposited with platinum (Figure 7). As before, we compare the perceptually uniform visualization (Figure 7a) to the traditional mapping method, which in this case is R-G-B mixing (Figure 7b). When pairing colors to chemical elements in either mapping (Figure 5), we advocate for match-ups that minimizes spatial overlap between red and green, to aid those with red-green color blindness.

Using the R-G-B mixing color map, the platinum-rich region on the left side of Figure 7b appears to have higher signal; however, the total signal is lowest in the platinum-rich region as shown in Figure 7e. This inconsistency in the R-G-B mixing color map shows how our perceptually uniform color map yields a more accurate representation of the total signal while maintaining good compositional distinguishability as seen in Figure 7c. One might argue that the small amount of platinum signal visible on the right side of Figure 7b,c is not noticeable within our perceptually uniform color mappping; however, if the elemental color pairings were permuted and Pt was represented as blue, the small platinum signal would not be visible in R-G-B mixing color mapping. To that end, if emphasizing small signals in a quantitative manner is important, it is best to separate elemental signals into separate figure panels and use a scalar colormap such as viridis as it provides 30% more contrast than using a black-green colormap.

4 Conclusions

Using colorimetry to design better—in a scientific fidelity rather than aesthetic sense—data representations for authentic perception is gaining substantial traction. Yet many of the current off-the-shelf tools, may not cover the more complex data representations with perceptually uniform use of color. The two examples and their implementations in PAPUC and CMPUC we highlighted are only marginally more complex than normal data post-processing as the software implementation of -sRGB color conversion is trivial. Yet for minimal extra effort, we demonstrate significant improvements in interpretability over the common HSV color mapping of 2D vector fields. Most importantly, the fixed lightness-magnitude mapping no longer hides vectors mapped to blue and we can ensure uniform angle contrast. Last, within three-component chemical mapping, we demonstrate a fair lightness-concentration representation for all three components. We hope, above all else, that our work encourages others to use perceptually uniform color mappings since the community of scientific researchers all strive for the accurate conveyance of data.

M.J.W. was supported by the Office of Naval Research (ONR) (Contract No. N00014-16-1-2280). J.M.R. acknowledges support from an Alfred P. Sloan Foundation fellowship (Grant No. FG-2016-6469). C.T.N. was supported by the U.S. Department of Energy (DOE), Office of Science, Basic Energy Sciences, Materials Sciences and Engineering Division. We thank Prof. R. Ramesh for his feedback and financial support of C.T.N. during initial data collection. Human enamel sample data presented in Figure 7 were provided by the Procter & Gamble Company.

References

  • Borkin et al. 2011 Borkin, M.; Gajos, K.; Peters, A.; Mitsouras, D.; Melchionna, S.; Rybicki, F.; Feldman, C.; Pfister, H. Evaluation of Artery Visualizations for Heart Disease Diagnosis. IEEE Transactions on Visualization and Computer Graphics 2011, 17, 2479–2488.
  • Wong 2010 Wong, B. Color Coding. Nature Methods 2010, 8, 573.
  • Wong 2011 Wong, B. Points of view: avoiding color. Nature Methods 2011,
  • Gehlenborg and Wong 2012 Gehlenborg, N.; Wong, B. Points of View: Heat Maps. Nature Methods 2012,
  • Gehlenborg and Wong 2012 Gehlenborg, N.; Wong, B. Mapping quantitative data to color: data structure informs choice of color maps. Nature Methods 2012, 9, 769–770.
  • Birch 2012 Birch, J. Worldwide prevalence of red-green color deficiency. J. Opt. Soc. Am. A 2012, 29, 313–320.
  • Wong 2011 Wong, B. Points of view: Color blindness. Nature Methods 2011,
  • IEC 61966-2-1:1999/AMD1:2003 2003 Amendment 1 - Multimedia systems and equipment - Colour measurement and management - Part 2-1: Colour management - Default RGB colour space - sRGB; Standard, 2003; Vol. 61966.
  • 9 Pössel, M. Walter Kohn at the 62nd Lindau Nobel Laureate Meeting on July 3, 2012. https://commons.wikimedia.org/w/index.php?curid=20120170, License: CC BY-SA 3.0.
  • Thwaites et al. 2018 Thwaites, A.; Wingfield, C.; Wieser, E.; Soltan, A.; Marslen-Wilson, W. D.; Nimmo-Smith, I. Entrainment to the CIECAM02 and CIELAB colour appearance models in the human cortex. Vision Research 2018, 145, 1 – 10.
  • Li et al. 2000 Li, C. J.; Luo, M. R.; Hunt, R. W. G. A revision of the CIECAM97s model. Color Research & Application 2000, 25, 260–266.
  • Li et al. 2017 Li, C.; Li, Z.; Wang, Z.; Xu, Y.; Luo, M. R.; Cui, G.; Melgosa, M.; Brill, M. H.; Pointer, M. Comprehensive color solutions: CAM16, CAT16, and CAM16-UCS. Color Research & Application 2017, 42, 703–718.
  • Sharma 2002 Sharma, G. Digital Color Imaging Handbook; CRC Press, Inc.: Boca Raton, FL, USA, 2002.
  • Kovesi 2015 Kovesi, P. Good Colour Maps: How to Design Them. 2015.
  • Mortensen et al. 2005 Mortensen, J. J.; Hansen, L. B.; Jacobsen, K. W. Real-space grid implementation of the projector augmented wave method. Phys. Rev. B 2005, 71, 035109.
  • Virtanen et al. 2020 Virtanen, P. et al. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods 2020,
  • van der Walt et al. 2011 van der Walt, S.; Colbert, S. C.; Varoquaux, G. The NumPy Array: A Structure for Efficient Numerical Computation. Computing in Science Engineering 2011, 13, 22–30.
  • Hunter 2007 Hunter, J. D. Matplotlib: A 2D Graphics Environment. Computing in Science Engineering 2007, 9, 90–95.
  • Waters and Rondinelli 2020 Waters, M. J.; Rondinelli, J. MTD-group/papuc: First release of PAPUC. 2020; https://zenodo.org/record/3688502.
  • Waters and Rondinelli 2020 Waters, M. J.; Rondinelli, J. MTD-group/cmpuc: First release of CMPUC. 2020; https://zenodo.org/record/3688508.
  • Nathaniel J. Smith 2018 Nathaniel J. Smith, Colorspacious. 2018; https://github.com/njsmith/colorspacious.
  • Mansencal et al. 2018 Mansencal, T.; Mauderer, M.; Parsons, M.; Canavan, L.; Cooper, S.; Shaw, N.; Wheatley, K.; Crowson, K.; Lev, O.; Leinweber, K.; Vandenberg, J. D. Colour 0.3.11. 2018; https://doi.org/10.5281/zenodo.1175177.
  • Yadav et al. 2016 Yadav, A. K. et al. Observation of polar vortices in oxide superlattices. Nature 2016, 530, 198201.