Extraction of Color Information and Visualization of Color Differences between Digital Images through Pixel-by-Pixel Color-Difference Mapping

Abstract

A novel method of extracting color information on a pixel-by-pixel basis or by the average of the regions of interest (ROIs) from digital images is proposed and demonstrated using newly developed and customized image-processing/analysis software (PicMan). For quantitative and statistical analyses of color, the newly developed software can be used for digital archive or digital forensic applications in various fields. The color differences between unrelated, similar, or identical scenes and or objects were quantified in various formats of desired color spaces such as RGB, HSV, XYZ, CIE L*a*b*, Munsell color, and hexadecimal color values. The color differences were visualized as images of pixel-by-pixel mapping of the ΔL*, Δa*, Δb*, ΔE_RGB, ΔE_HSV, and ΔE*_L*a*b* values and block comparison images of desired block sizes. Various color analyses and color-difference mapping examples using an aged and damaged oil painting before and after restoration were introduced. The effects of the image file format differences between PNG and JPG on color distortion are demonstrated by statistics and pixel-by-pixel color-difference mapping. A portrait of Chuk-ki Yoo (兪拓基, 1691–1767) on silk from the 18th century from Korea was used for further color analysis for whole and selected areas. A collector’s ownership stamp of Chuk-ki Yoo stamped in red ink on the text areas in one of his book collections was extracted using the image-processing software and superimposed on the original image as a visualization enhancement example. Image analysis, processing, modification, enhancement, and highlighting, as well as statistical color analysis of digital images in most formats, can conveniently and efficiently be performed using one piece of dedicated software (PicMan). The pixel-by-pixel color information extraction and color comparison technique can be very effective for a variety of applications in art and cultural heritage objects.

1. Introduction

Although the true color reproduction of objects is still a challenge in digital photography, huge numbers of digital images are generated and shared on a daily basis. Colorimetric measurement devices have been introduced for quantitative color analyses and data-driven objective decision-making processes in many applications including food science, biology, pharmacy, medicine, agriculture, art, and cultural heritage, to name a few. However, conventional point-of-use colorimetric devices suffer from poor spatial resolution due to the minimum sampling area requirement (greater than a few millimeters in diameter) and require direct contact with objects under illumination from an internal light source [1,2,3,4]. The colorimetric mapping of any object regardless of size and shape requires significant effort for sampling and the manual processing of the measurement data afterward. Even with these efforts, the number of data points is limited to the actual number of measurements. It is impractical to manually acquire more than a few hundred discrete measurement data, which is usually far from satisfactory for data visualization to comfortably draw statistical analysis-based reasoning or conclusions.

A noncontact, telemetric, color measurement technique with increased spatial resolution is strongly desired to enable the instantaneous acquisition of vast amounts of color information pertaining to the object while providing ease of use. True color reproductions of objects and scenes are still quite a challenge in photography [5,6]. The ultimate goal is to create digital images that are not distinguishable from real-world objects and scenes. The color of objects or scenes varies depending on the lighting and viewing conditions (such as viewing angle, relative direction to the light source, etc.). Nevertheless, digital photography is one of the best options for instantly capturing the color information of objects and scenes under lateral and controlled lighting conditions. Document scanners with an internal light source can be an excellent resource to provide controlled lighting at a predetermined distance between the scanning line camera and a document [7,8]. Digital cameras can be used for the same purpose as document scanners. Compared to a dedicated scanner, a camera image is subject to a degree of distortion, reflections, and shadows and may exhibit low contrast. Image blur can happen when the camera is moved or disturbed during exposure [8].

Nowadays, most digital cameras including smartphone cameras can take very realistic photographs. The color differences between digital photographs and real objects cannot be easily noticed when digital photographs are taken under well-controlled lighting conditions and adhere to predetermined standards for the intensity, direction, quality, color, and spectrum of a light source. Many efforts have been made to utilize digital image files generated from digital cameras and scanners for the quantification of the color information of objects with higher spatial resolution without using conventional chroma meters [2,3,4,5,6,9,10,11].

Most commercially available image-editing software is designed for color, brightness, and contrast adjustments for viewing and printing. There is no image analysis software for quantitative, statistical, and dimensional analyses of the color, brightness, and contrast of regions of interest (ROIs). Due to the difficulties of image analysis and the labor-intensive nature of image analysis tasks using multiple types of software, only a very small fraction of (digital) image data is utilized. It is important and necessary to develop a new user-friendly image-processing/analysis software with the desired functions for quantitative, statistical, and dimensional analyses of the color, brightness, and contrast of ROIs to promote the efficient utilization of image data. This led the authors to develop user-friendly, unified image-processing software, which can support various formats of digital image and movie files [2,3,11].

In this paper, a novel method of the extraction of quantitative color information from digital images and visualization of color differences between digital images through pixel-by-pixel color-difference mapping is proposed. A newly developed and customized image analysis/processing software (PicMan) [2,3,11,12,13,14,15,16] was utilized for demonstrating the proposed concept of quantitative color characterization, image maneuvering, and statistical color analysis of full or partial areas of digital images of art and cultural heritage subjects.

2. Image Analysis/Processing Software (PicMan)

In the digital information age, we are surrounded by a flood of digital information in our daily lives. Digital photographs, images, videos, documents, and music can be reviewed, played, and edited for an intended purpose using more than one type of software. Depending on the complexity of the desired task, a handful of image-editing application software requiring tremendous resources (processing capability and memory) can run on PCs simultaneously in the background to perform very specific tasks. With the increase in image file sizes, the demand for computing resources is increasing exponentially. Image processing and analysis are very different tasks compared to image editing. If we can develop integrated software suitable for frequently needed image editing, processing, and analysis functions with image, video, and numerical data exporting capabilities, it will make image analysis and processing much simpler and more convenient. A regular PC with reasonable capabilities can handle a lot of tasks without opening multiple applications and switching between applications for specific tasks.

To address these deficiencies, the authors’ group has been developing user-friendly software (PicMan, WaferMasters, Inc. Dublin, CA, USA) for image-based dimensional analysis, quantitative color analysis, statistical analysis, and various image-maneuvering (pattern selection, highlighting, editing, etc.) functions. A few of these functions have been applied to archaeology, cultural heritage, and conservation science studies, as have been reported previously [2,3]. PicMan can process and analyze various formats of digital image and video files such as JPG, BMP, GIF, PNG, TGA, TIFF, PDF, CR2, DM3, ND2, MIRAX, MOV, MP4, AVI, WMV, etc.

From the image and video files containing plentiful information generated digitally, only a very small fraction of data is ever utilized. This is mainly due to the difficulties in image analysis and the labor-intensive nature of the image analysis task. All image data contain coordinate, color, brightness, contrast, dimension, and time-related information that can be extracted for useful applications. They are ready to be extracted for further analysis and interpretation of their uniqueness and significance. To promote the efficient utilization of image data, it is important to develop user-friendly image-processing/analysis software with application-specific, customized functions that can significantly simplify workflow and enhance productivity with accuracy and reliability.

All images have a set of pixel information on the x and y coordinates and RGB brightness. Combinations of RGB brightness values determine the color and overall brightness. The pixel-to-pixel RGB brightness difference determines the contrast and shape of an object. Everything starts from the pixel information. Combining the RGB brightness (8 bits per channel) values is very useful in that it can generate millions of colors (2⁸ brightness values × 3 channels = 2^8×3 colors = 2²⁴ colors = 16,777,216 colors) in displays and monitor screens. However, it is less useful for us to understand and interpret how we recognize and feel colors from visual stimulation by differences in the spectrum of light (wavelength dependence of brightness in the visible wavelength range of 400–700 nm or 400–780 nm). Thus, many different concepts of color spaces have been introduced to characterize and classify colors in quantitative and traceable manners.

The most popular color spaces used across disciplines are RGB, HSV, L*a*b*, and Munsell color. The RGB and L*a*b* color spaces are based on cartesian coordinates. The HSV and Munsell color spaces use cylindrical coordinates. Since all photographic images are based on combinations of RGB channel brightness, the color information in the other color spaces is calculated from the RGB values using corresponding transformation formulas. For conversion between RGB and L*a*b* color spaces, a new cartesian coordinate of XYZ is introduced. Details of color space transformations can be found elsewhere [17,18,19].

All 16,777,216 colors are assigned to six-digit hexadecimal color codes for computer graphics. The capability of pixel-by-pixel color extraction, as well as the extraction of the average color for selected areas, can be very useful for quantitative color characterization. All colors can be quantified as numerical values or corresponding codes in all color spaces for many applications including archaeology, cultural heritage, and conservation science studies.

In the following section, the extraction of color information from sample digital images using the newly developed image-processing/analysis software (PicMan) is introduced using sample images and discussions of possible applications in cultural heritage characterization and conservation science are presented.

3. Results and Discussion

Two photographic images of an aged and damaged oil painting on canvas before and after restoration from the website of a gallery located in Franklin, Massachusetts [20], were selected as example images for color information extraction, color analysis, and image comparisons using PicMan. A photograph of a portrait of Chuk-ki Yoo (兪拓基, 1691–1767) on silk from the 18th century during the Joseon Dynasty (1392–1910) in Korea [21] was also tested. A seal image from a book named Garyewonryu (가례원류: 家禮源流) belonging to Chuk-ki Yoo (兪拓基, 1691--1767) [22] was used for color analysis, selection of pixels with a specific color range, and image modification as an application example of the modified image.

For best results, images should be photographed or scanned under standard lighting and image acquisition conditions. The validity of L*a*b* values converted from RGB values should be confirmed by colorimeter measurements. Since the colorimeters simply do not have pixel-by-pixel spatial resolution and provide the average L*a*b* values within a few millimeters of the diameter area [1], a separate color checkerboard may be used for color calibration and validation.

3.1. Block-by-Block Image Comparisons before and after Restoration

Figure 1 shows the combined images of a girl in an aged and damaged oil painting on canvas before and after restoration in the PNG image format [20]. Craquelure reduction, canvas stabilization, lining the painting, and oil painting cleaning were carried out through restoration work. The color became very vivid after restoration. The two images looked identical in terms of the size and proportion of the girl. However, the two images did not perfectly overlap. The image after restoration was cropped slightly and shifted to the right and the width was slightly narrower than the image before restoration. One other important thing to note is that the image before restoration was rotated approximately 1^° in the counterclockwise direction. To compare the two images properly, the image′s crop size and rotation must be corrected before proper pixel-by-pixel image comparisons. The image shift, cropping, and rotation were done manually by comparing the overlapped images. The image size was matched to 442 × 652 pixels by adjusting the rotation and cropping only the common areas between the two images. The size and rotation-adjusted images were saved as PNG image files, the same as the source image file, so as not to lose or modify the original color information.

A screen capture of PicMan comparing the two images before and after restoration is shown in Figure 2. The two images on the left and in the center are the source image files for the image comparison. The image on the right shows the block-by-block image comparison results. The block size can be varied depending on the image size and degree of pattern complexity of the images for optimum visualization results. The block size applied in Figure 2 was 50 × 50 pixels. The top-left triangles show the image before restoration and the bottom-right triangles show the image after restoration. The color differences between the before and after images can be easily recognized without alternately looking at two images and committing them to memory. The imaginary differential operation of the two images is not quite accurate and cannot communicate one’s findings to others. Our memory and observation are not as good as we might think. In addition, we cannot record our findings from this mental exercise. The block comparison results of different block sizes can be exported as separate images for record-keeping and clear communication with others.

The color difference analysis did not consider/differentiate the glossy appearance of the painting surface due to the varnish (as seen in the upper-left area of the artwork in Figure 2). The computed color difference was not a difference, possibly due to changes in terms of the degradation of the painting materials but most likely due to the removal of aged and altered varnish in the restoration process. In this study, the image comparison technique is given as an example of the color-difference characterization of images taken under controlled lighting and photography conditions after calibration. Proper lighting, photography, and calibration are required for proper color-difference characterization. The practice for accurate colorimetry measurements from digital cameras can be found in previous reports [5,23,24].

3.2. Pixel-by-Pixel Image Comparison before and after Restoration

Figure 3 shows a summary of the pixel-by-pixel image comparisons of the images of a girl before and after restoration. The pixel-by-pixel color differences were mapped in terms of the CIE ΔL*, CIE Δa*, CIE Δb*, and CIE ΔE* values at full scale (100% scale), which can be seen in the top row, and at quarter scale (25% of full scale), which can be seen in the bottom row. Since the color differences along the a* and b* axes were small compared to the lightness L* axis, the full-scale pixel-by-pixel color-difference mapping images for the Δa* and Δb* values did not show significant color shifts. The Δa*, Δb*, and ΔE* values became much more recognizable at a 4x magnified scale (equivalent to 25% of full scale).

The CIE ΔL* and CIE ΔE* values only have a magnitude in the brightness and overall color difference. Thus, the resulting image becomes grayscale. The Δa* and Δb* values can be negative, zero (i.e., no color difference along the a* or b* axis), or positive. The CIE L*a*b* color space expresses color as three values: L* for perceptual lightness and a* and b* for the four unique colors of human vision, red, green, blue, and yellow. The lightness value L* defines black as 0 and white as 100. The a* axis is relative to the green-red opponent colors, green (negative a* values) to red (positive a* value). Similarly, the b* axis is relative to the blue-yellow opponent colors, blue (negative b* values) to yellow (positive b* value). The a* and b* axes are independent in the range of −128 to 127.

In Figure 3, the degrees of the color shifts along the a* and b* axes can be easily recognized pixel by pixel. The color difference ΔE* can easily be determined pixel by pixel. This type of detailed color difference characterization cannot be measured or visualized by conventional chroma meter measurements [1,2,3,4,5,6].

3.3. Effect of Image File Format Conversion on Color

Image data can be saved in many different formats. Raster images, such as photographs and scanned documents, are constructed by a series of pixels, or individual blocks, to form an image. We often recklessly save images in JPG format for convenience and file size. The JPG format is designed for the lossy compression of images in return for a file-size reduction. The JPG compression algorithm is designed to operate well on photographs and paintings of realistic scenes with smooth variations of tone and color. For applications where the trueness and traceability of the original color of images are of great concern, it is important to understand and set rules for the image file format of choice. The image file format conversion should be avoided as much as possible because it may modify or damage the integrity of the original images in terms of color, shape, and sharpness. To verify the effect of the JPG compression of photographic images on image quality, the PNG images of an oil painting of a girl (a) before and (b) after restoration were saved as JPG images (c and d), as seen in Figure 4.

The image size did not change before and after file conversion. However, the file sizes of the JPG images were reduced to a fraction of the original sizes of the PNG images (365 KB → 59.4 KB before restoration and 481 KB → 73.4 KB after restoration). The color differences between the PNG and JPG images were minimal. Without paying great attention, it is difficult to notice any differences between them. The only area with noticeable color changes was the top-left corner of the images before restoration (Figure 4a,c). The JPG image is a little brighter than the original PNG image. This is all that we can say about the differences between the PNG and JPG files.

To evaluate the differences between the PNG and JPG files statistically, the color information from all the pixels (442 × 652 pixels = 288,184 pixels) of the four images was exported as CSV files to generate a histogram for the RGB, HSV, and L*a*b* color spaces. For simplicity, the RGB and L*a*b* histograms and statistics are discussed in this paper. Figure 5 shows the histograms of the RGB brightness for the PNG (a and d) and JPG (b and e) images before and after restoration and the differences (c and f) between the PNG and JPG files of the same images. The distribution of the RGB brightness before restoration was skewed left (darker side), whereas it was spread more broadly after restoration. The effect of restoration on the apparent color of the images can also be recognized in the histograms. The area under the individual RGB histogram curves should be equal to the total number of pixels (288,184 pixels). As seen in Figure 5a–c, sharp RGB peaks below the brightness level of 96 in the PNG image (a) became dull in the JPG image (b) before restoration and the differences in histograms between the PNG and JPG images appeared to be quite large (c), suggesting a significant modification of colors from the PNG → JPG conversion.

The RBG statistics for all pixels (442 × 652 = 288,184 pixels) of the PNG and JPG files of the oil painting of a girl before and after restoration are summarized in

Table 1. Statistical summary of RGB of all pixels (442 × 652 = 288,184 pixels) of PNG and JPG files of the oil painting of a girl before and after restoration.

Restoration Status		Before			After			Difference
Color Variable		Red	Green	Blue	Red	Green	Blue	Red	Green	Blue
PNG	Minimum	15	15	4	0	4	2	15	11	2
	Average	92.6	76.3	55.2	113.3	88.4	80.0	−20.7	−12.1	−24.8
	Maximum	227	227	225	255	255	245	−28	−28	−20
	Range	212	212	221	255	251	243	−43	−39	−22
	Std. Dev.	32.0	30.5	30.1	71.3	65.1	53.1	−39.3	−34.6	−23.1
JPG	Minimum	15	15	2	0	0	1	15	15	1
	Average	92.5	76.3	55.4	113.3	88.4	80.1	−20.8	−12.1	−24.6
	Maximum	227	227	225	255	255	246	−28	−28	−21
	Range	212	212	223	255	255	245	−43	−43	−22
	Std. Dev.	31.9	30.5	30.0	71.2	65.1	53.1	−39.3	−34.6	−23.1
PNG-JPG	Minimum	0	0	2	0	4	1
	Average	0.0	−0.1	−0.2	0.0	0.0	−0.1
	Maximum	0	0	0	0	0	−1
	Range	0	0	−2	0	−4	−2
	Std. Dev.	0.1	0.0	0.0	0.1	0.0	0.0

. The minimum, average, maximum, range, and standard deviation of the RGB brightness of the 288,184 pixels from each image (PNG and JPG images before and after restoration) are shown. The differences between the before and after restoration are summarized in the three columns on the right in Table 1. The differences between the PNG and JPG images are summarized in the five rows at the bottom of the table. From a statistics point of view, the differences between the PNG and JPG files seem to be very small because they were averaged from the total number of pixels (288,184 pixels) per image. The differences between the PNG and JPG images before and after restoration are very small for the same reason. It is obvious that pixel-by-pixel color-difference mapping has the highest sensitivity to the exact location information of the affected pixels. A histogram can be considered a projected graph of RGB brightness binning ignoring the pixel location information. The dimension of the image data was significantly reduced, as seen in the histogram and the statistics summarized in Table 1. The importance of pixel-by-pixel color-difference mapping cannot be overemphasized.

For completeness, the histograms and statistics of the CIE L*a*b* values of all pixels (442 × 652 = 288,184 pixels) of the PNG and JPG files of the oil painting of a girl before and after restoration are summarized in Figure 6 and Figure 7 and

Table 2. Statistical summary of CIE L*a*b* values of all pixels (442 × 652 = 288,184 pixels) of PNG and JPG files of the oil painting of a girl before and after restoration. Overall ΔE* between the PNG and JPG files before and after restoration (columns on the right) are also calculated (bottom row).

Restoration Status		Before			After			Difference
Color Variable		L*	a*	b*	L*	a*	b*	ΔL*	Δa*	Δb*	ΔE*
PNG	Minimum	4.3	−12.9	−7.0	1.5	−12.0	−31.0	2.8	−0.9	24.0	24.2
	Average	33.4	4.1	15.2	39.5	10.2	9.6	−6.1	−6.1	5.7	10.3
	Maximum	90.2	23.3	38.5	99.5	50.8	52.0	−9.4	−27.5	−13.5	32.0
	Range	85.9	36.2	45.5	98.0	62.8	83.0	−12.2	−26.6	−37.4	47.5
	Std. Dev.	12.4	6.7	10.2	25.1	13.7	16.4	−12.6	−7.0	−6.2	15.7
JPG	Minimum	4.4	−10.8	−6.8	1.4	−13.6	−28.7	3.0	2.8	21.9	22.3
	Average	33.4	4.1	15.1	39.5	10.3	9.5	−6.1	−6.2	5.6	10.3
	Maximum	90.2	23.4	37.4	99.6	52.7	51.1	−9.4	−29.4	−13.7	33.8
	Range	85.8	34.1	44.2	98.2	66.3	79.9	−12.4	−32.2	−35.6	49.6
	Std. Dev.	12.4	6.6	10.2	25.1	13.7	16.4	−12.6	−7.0	−6.2	15.7
PNG-JPG	Minimum	−0.1	−2.1	−0.2	0.2	1.6	−2.3
	Average	0.0	0.0	0.1	0.0	0.0	0.0
	Maximum	0.0	−0.1	1.1	0.0	−1.9	0.8
	Range	0.1	2.0	1.3	−0.2	−3.5	3.1
	Std. Dev.	0.0	0.1	0.0	0.0	0.1	0.1
Overall ΔE*PNG-JPG		0.1			0.0

. The area under the individual L*a*b* histogram curves should be equal to the total number of pixels (288,184 pixels). The color difference ΔE* and overall color difference between the PNG and JPG files ($\mathsf{\Delta }{E}_{\mathit{PNG-JPG}}^{*}$: $\mathsf{\Delta }{E}_{\mathit{ab}}^{*}$ or $\mathsf{\Delta }{E}^{*}$) were also calculated to understand the impact of restoration and image file format conversion. The ΔE* was calculated using the following CIE76 formula:

$$\mathsf{\Delta }{E}_{\mathit{ab}}^{*}=\sqrt{(L_2^{*}-L_1^{*}{)}^2+(a_2^{*}-a_1^{*}{)}^2+(b_2^{*}-b_1^{*}{)}^2}$$

where $\mathsf{\Delta }{E}_{\mathit{ab}}^{*}\approx 2.3$ corresponds to a JND (just noticeable difference). There are other revisions for color difference calculation formulas such as CIE94 and CIEΔE2000 [3,4,25,26].

The lightness L* histograms of the images before (Figure 6a) and after (Figure 7a) restoration show significant differences in distribution reflected from the source images, as shown in Figure 4. The a* and b* axes’ distributions also showed significant differences before and after restoration (Figure 6b,c, and Figure 7b,c). The differences between the PNG and JPG files are also quite noticeable in Figure 6d–f and Figure 7d–f. This verifies that the colors of the JPG files were different from those of the source PNG files even though the changes were not large enough to be distinguished by the human eye without very careful observation.

Compared to the RGB expression of colors, the L*a*b* expression is more insightful for understanding the lightness, as well as color balance, in the a*b* plane of RGYB. The HSV (hue, saturation, value) color space only makes assumptions based on the RGB balance in the cylindrical coordinates. It is an alternative representation of the RGB color model and was designed in the 1970s by computer graphics researchers to more closely align with the way human vision perceives and the color-making attributes from the additive colors of RGB used in monitors and displays [27].

3.4. Application Examples using a Portrait and a Book Ownership Seal

The extraction of the color information and the statistical analysis of the extracted color information were applied to a photograph of a portrait of Chuk-ki Yoo (兪拓基, 1691–1767) on silk from the 18th century during the Joseon Dynasty (1392–1910) in Korea, as shown in Figure 8 [21]. The portrait PNG image file was 740 × 972 pixels (=719,280 pixels) in physical dimensions and 1.37 MB in file size. The RGB and CIE L*a*b* histograms of the image were plotted after extracting the RGB and L*a*b* values of all pixels using PicMan and are shown in Figure 9. The area below the RGB and L*a*b* curves should be the same and equal to 719,280 pixels. The brightness distribution of the RGB colors was visualized by graphing the binning results of the RGB brightness range of 0 to 255 in increments of 1. The distribution of the lightness and relative position of color in the RGYB plane was visualized by plotting the L*a*b* binning results in increments of 0.1. The statistical summary of the RGB and CIE L*a*b* of all pixels (740 × 972 pixels = 719,280 pixels) comprising the portrait image is shown in

Table 3. Statistical summary of RGB and CIE L*a*b* values of all pixels (740 × 972 pixels = 719,280 pixels) of a PNG image of a portrait of Chuk-ki Yoo.

Restoration		Color Statistics
Color Variable		Red	Green	Blue	L*	a*	b*
PNG	Minimum	15	15	2	4.4	−10.8	−6.8
	Average	92.5	76.3	55.4	33.4	4.1	15.1
	Maximum	227	227	225	90.2	23.4	37.4
	Range	212	212	223	85.8	34.1	44.2
	Std. Dev.	31.9	30.5	30.0	12.4	6.6	10.2

.

Figure 10 shows a screen capture from PicMan displaying the line intensity graphs of the RGB and its average intensity across A-A′ on the top of the screen and the quantified color information of the individual pixels and averages of the selected square- or circle-shaped regions of interest (ROIs) on the image in RGB, HSV, and L*a*b* formats. The width and diameter of the square can be selected up to 100 pixels in 1-pixel increments. As for the selected ROIs, 40 × 40 pixels (=1600 pixels) were used for averaging the color information, as seen in Figure 10. The single-pixel color information can be considered an interactive chroma meter with a pixel resolution. The color information can also be displayed in hexadecimal color codes or the Munsell color format, whichever is more convenient. All displayed values and graphical data can be exported as CSV or image files of desired formats for record-keeping and further analyses and/or comparisons.

The upper row in Figure 11 shows the color inversion and grayscale conversion of the original image. The background areas removed from the portrait and the outline image of the portrait using the color selection and illustration functions in PicMan are shown in the lower row in Figure 11. The total number of pixels in the background-removed image was 270,707 out of 719,280 pixels, which was 37.64% of the total area of the portrait. The area of the silk background was measured at 448,573 pixels, which is equal to 62.36% of the total area of the portrait. Outline images can also be easily generated by detecting the pixels with significant color and/or brightness changes compared to the neighboring pixels in the original image.

Figure 12 shows the RGB and CIE L*a*b* histograms for the silk area (448,573 pixels) of the portrait of Chuk-ki Yoo (兪拓基, 1691–1767), former Prime Minister during part of the Joseon Dynasty (1392–1910) in Korea. Since the color of the silk was relatively uniform, the RGB and L*a*b* histograms were quite narrow and symmetrical. By comparing them with the RGB and CIE L*a*b* histograms for the entire portrait shown in Figure 9, the distribution of the RGB and L*a*b* values of the 270,707 pixels (for the background-removed portrait) can be distinguished.

Other useful applications of color-based selection/elimination functions with color homogenization and transparency capabilities are shown in Figure 13. For color analysis, the selection of pixels within specific color ranges, image modifications, and application examples are shown. The modified image of a seal from a book named Garyewonryu (가례원류: 家禮源流) belonging to Chuk-ki Yoo (兪拓基, 1691–1767) [22] was used. The seal-image highlighting was carried out with a series of actions: background removal, color homogenization, background transparency, and superposition of the modified image on the original image. Similarly, paper color homogenization can be applied if needed.

For the physical restoration of paper-based cultural heritage items, any mistakes can result in permanent damage to the item. Photoprints of paper-based cultural heritage documents and paintings are frequently created for introduction to the public. A large portion of paper-based cultural heritage and rare books are in poor condition due to physical damage, decoloring, darkening, foxing, and so on. A photocopy of the as-is condition may not be suitable for the purpose intended. Digital restoration can be an excellent alternative without risking damage to the original. The evaluation of the quantitative color information and digital filtering of noise by color can greatly improve legibility. Newly developed and added functions of the image-processing/analysis software (PicMan) have made these rather complex tasks very simple without the need to use other software. PicMan is designed as an execution file that is <20 MB in size and no installation is required. The small file size of PicMan also significantly frees up computing resources and memory. In addition, there is no need to open other software that requires CPU and memory sharing of the computing device. This greatly reduces the requirement for computing resources such as hardware specifications and memory requirements. Almost all tasks can be performed using tablets or laptops. Relatively large image files can be handled using tablets and laptops with general specifications. This provides great freedom in the working environment. Complex tasks can be accomplished anywhere at any time.

Image capture using the integrated cameras of laptops and tablets can be accomplished using PicMan and the resulting digital photographs and video files can be instantly analyzed and processed as needed. The software allows for the use of a USB camera. The camera can be utilized as an image-capturing device and controlled through PicMan. Digital photographs, images, and video files from other devices or the internet can be analyzed in the same way.

4. Conclusions and Recommendations

In our daily lives, our eyes are stimulated by photons, energized particles, or electromagnetic waves and we perceive them as color and brightness in the visible energy (3.1–1.77 eV or 3.1–1.59 eV) or wavelength (400–700 nm or 400–780 nm) ranges. Each individual has a different experience with the same colors and color information, which cannot be accurately communicated using descriptive language. If a different language is used, the translation between languages adds more distortions to the true meaning of the speaker. The meaning and expression of colors differ from language to language, culture to culture, region to region, country to country, industry to industry, and person to person. To avoid or at least reduce uncertainties and communication/interpretation errors, color information should be described in quantitative terms. In this digital era, this is not an impossible task, even though we still need to improve how it is defined and utilized. All digital images are made up of pixels with color information in a quantitative manner. Indeed, all pixels are expressed as hexadecimal code containing RGB brightness information in the range of 0–255, assuming 24-bit (8-bit per channel) images. As long as we select a hexadecimal color code, it would indicate the same color around the world. Of course, the color representation on different monitors may differ slightly. However, the integrity of the color information is stored in the image file. In the past, the only quantitative way to characterize and express color was by measuring the average color of a sampling area (typically a few mm in diameter) using a chroma meter. Due to the poor spatial resolution of the chroma meter measurement, the color of a small area or sampling area with non-uniform colors cannot be measured and recorded correctly. By using digital photographs and pixel-by-pixel color extraction techniques, the pixel resolution of the color information can be extracted and utilized for further analysis, reproduction, and utilization.

In this paper, a novel method for extracting color information on a pixel-by-pixel basis or by the average of the regions of interest (ROI) from digital images has been proposed. The proposed concept has been demonstrated using newly developed and customized image-processing/analysis software (PicMan). The software can be used for quantitative analysis, statistical analysis, digital archiving of color, and digital forensic applications in various fields. The color differences between images, which were of unrelated, similar, or identical scenes and objects, were quantified in various formats of desired color spaces. The color information of individual pixels was extracted as RGB, HSV, XYZ, CIE L*a*b*, Munsell color, and hexadecimal color values. The color differences were also visualized as images of pixel-by-pixel mapping of ΔL*, Δa*, Δb*, ΔE_RGB, ΔE_HSV, and ΔE_L*a*b* values and block comparison images of desired block sizes. The effects of the image file format differences between PNG and JPG on color distortion were investigated and demonstrated by statistics and pixel-by-pixel color-difference mapping. Image analysis, processing, modification, enhancement, and highlighting, as well as statistical color analysis of any digital images in most formats, can be conveniently and efficiently performed using one dedicated piece of software (PicMan). It can be used for a variety of applications for art and cultural heritage objects.

Art restoration is intended to preserve the integrity and value of an original work of art. Many artworks are in need of repair because they have decayed over long periods of time due to storage and climatic conditions, which can have a significant impact on the condition of the artwork [28]. Art restoration is any attempt to preserve and repair drawings, paintings, sculptures, architecture, or other objects of fine and decorative art, whose condition has been negatively altered due to various causes. Quantitative color characterizations from digital images have become increasingly important for recording and documenting color changes over time to properly plan art restoration strategies as museums and cultural institutions work toward protecting invaluable cultural collections.

Recently, in medical fields, attempts to link pain to individual artistic expression have been reported in an objective and quantitative fashion using Frida Kahlo (1907–1954)’s artworks [29]. The intention was to use the linkage to integrate art into the therapeutic armory for pain and pain-related disorders. Frida Kahlo was a Mexican artist who is remembered for her self-portraits expressing pain and passion with bold and vibrant colors. Her artworks were used to examine the effects of physical and emotional pain (rage) on artistic expression with quantitative color analysis results. Quantitative color analyses are required in many areas, including medicine, pharmaceutical, biology, food science, agriculture, animal science, fashion, painting, design, and many other fields. When the proposed method becomes available and has been practiced, the more active usage of quantitative color information in various fields is expected [11,30]. No color information should be overlooked due to a lack of convenient image analysis/processing software.