Additional Information Delivery to Image Content via Improved Unseen–Visible Watermarking

Abstract

In a practical watermark scenario, watermarks are used to provide auxiliary information; in this way, an analogous digital approach called unseen–visible watermark has been introduced to deliver auxiliary information. In this algorithm, the embedding stage takes advantage of the visible and invisible watermarking to embed an owner logotype or barcodes as watermarks; in the exhibition stage, the equipped functions of the display devices are used to reveal the watermark to the naked eyes, eliminating any watermark exhibition algorithm. In this paper, a watermark complement strategy for unseen–visible watermarking is proposed to improve the embedding stage, reducing the histogram distortion and the visual degradation of the watermarked image. The presented algorithm exhibits the following contributions: first, the algorithm can be applied to any class of images with large smooth regions of low or high intensity; second, a watermark complement strategy is introduced to reduce the visual degradation and histogram distortion of the watermarked image; and third, an embedding error measurement is proposed. Evaluation results show that the proposed strategy has high performance in comparison with other algorithms, providing a high visual quality of the exhibited watermark and preserving its robustness in terms of readability and imperceptibility against geometric and processing attacks.

1. Introduction

In the future, the current growth of technological development will be compared only with the period called the industrial revolution. This development allows people easy access to electronic devices that are capable of capturing and displaying different multimedia files, especially images and videos, which could be easily copied, edited, and distributed without any protection, turning these practices into a problem of copyright protection and intellectual property. To solve this problem, in the last decade, several digital watermarking algorithms have been proposed [1,2,3,4,5]. However, in a real-life scenario, the watermarks are used to deliver auxiliary information, such as banknotes, official documents, etc., where the watermarked document delivers information about its authenticity when its visualization is enhanced, seeing it against light. In this context, several watermarking algorithms have been proposed to deliver auxiliary information about the visual content of images [6,7,8,9,10,11]; opening with this target a new research field that is denoted as invisible–visible watermarking, which can be divided according to the exhibition stage in unseen–visible watermarking (UVW) [6,7,8], and imperceptible–visible watermarking (IVW) [9,11]. The main idea of invisible–visible watermarking algorithms applied to digital color images is based on the digital reproduction of the real-life scenario of the watermarks, that in addition to providing copyright and intellectual property protection, are capable of delivering information to the end-users via owner logotype, 2D barcode or quick response (QR) codes [12]. To make this possible, these algorithms are focused on the observation of how some digital image information that is invisible to the naked eyes becomes visible after applying some image enhancement operations such as Gamma Correction Function (GCF) [6,7,8,13], Histogram Modulation (HM) [9] or Binarization Function (BF) [10,11].

In the IVW algorithms [9,10,11], the image processing capabilities of display and capture images devices such as cameras, laptops or mobile phones are used to embed and exhibit a binary watermark pattern such as owner logotype, 2D barcode or QR code. In the embedding stage, the algorithm takes advantage of the imperceptibility and visibility of the invisible and visible watermarking algorithms, respectively, to imperceptibly embed a binary watermark pattern into the largest homogeneous region of the corresponding host image. Until now, in the literature, two ways have been proposed to exhibit the watermark pattern and deliver information to the end-users. In the first [9], the watermark pattern is exhibited in color after applying a histogram modulation to the watermarked image; in the second [10,11], a binarization function is used to exhibit, to the end-users, the watermark in a black–white color. In contrast with the IVW algorithms, the embedding and the exhibition stages of UVW algorithms work independently of each other. In a general scenario, the entity who wants to provide additional information about the visual content embeds in an imperceptible manner a logotype, 2D barcode or QR code, into the image or video; after the transmission, when the end-user requires extra information about the visual content, he/she executes an exhibition stage, taking advantage of the equipped functions into the display devices, such as GCF [6,7,8,13], and contrast or brightness [14].

In this proposal, additional information delivery to the image content refers to the application of the general scenario of the UVW algorithms in daily life; when the end-users exhibit the QR watermark, the decoded information could be a company website, information of singers, dress designer, location or another kind of information that has or does not have relation to the image content. Additionally, it is not limited to extend the applicability to information security tasks such as medical imaging [15,16], deepfake prevention [17], 3D-video protection [18], face recognition authentication [19] and other related fields [20]. However, to obtain better results in all those applications, several drawbacks of the UVW algorithms must be improved: first, the algorithms only require large image regions with low intensity [6,7,8,14] second, embedding strategies increase histogram distortion and visual degradation of the watermarked image, impacting the legibility of the visual content and the watermark imperceptibility. To improve UVW algorithms in the context of imperceptibility and visibility of the watermark pattern, this paper proposes a strategy that is distinguished by three main goals:

• First, the strategy to select the embedding region considers large smooth regions of low or high intensity, extending the applicability of the proposed algorithms to any class of images;
• Second, the watermark complement strategy is capable of reducing the histogram distortion and the visual degradation of the watermarked image;
• Third, a methodology to quantify the embedding error induced by the watermark embedding strategy is introduced.

To evaluate the performance of the proposed algorithm, the experimental results are compared with previously reported algorithms [6,7,8], showing that the proposal can be implemented efficiently in images with different characteristics and that the histogram distortion and visual degradation of the watermarked image are reduced.

The rest of this paper is organized as follows. A brief description of the previously reported works and the motivation is given in Section 2. Then, in Section 3, the materials and methods of the proposed watermarking algorithm are detailed. In Section 4, the experimental results, including a performance comparison with the previously reported algorithms [6,7,8], are shown. Finally, Section 5 concludes this work.

2. Literature Survey

In this section, related works to UVW algorithms [6,7,8] and the motivation of this work are described briefly. The binary watermark pattern and the host image are defined in all this work as W = {W(i, j) = 0, 1 | i = 1, …, ww, j = 1, …, hw}, where ww and hw are the width and height, respectively, and the host image is defined as Ih = {Ih_k(x, y) | 1 ≤ x ≤ M, 1 ≤ y ≤ N}, where M and N are the width and height of the host image, respectively, and 0 ≤ Ih_k(x, y) ≤ 255 is the pixel in the corresponding (x, y) position of the color channel k ∈ {R,G,B}.

2.1. Unseen–Visible Watermarking

In [6], Huang et al., inspired by a real-life scenario of the watermarks, propose a pioneer UVW algorithm based on [21,22] and the observation of how several pieces of image information that are invisible to the naked eyes become visible after applying an image enhancement function equipped into display devices such as Gamma Correction Function [13], as is shown in Figure 1.

In this algorithm, the best intensity value i* is obtained as the maximum gradient of the GCF, which is considered as the watermark exhibition strategy. Once the best intensity value i* is obtained, an adequate embedding region is selected from the host image according to the following equation.

$$(x*,y*)=\underset{(x_0,y_0)}{\arg \max }{\displaystyle \sum _{x=x_0}^{x_0+ww+1}{\displaystyle \sum _{y=y_0}^{y_0+hw+1}\left|Ih(x,y)-i*\right|}},$$

(1)

where (x*, y*) is the upper-corner left position of the adequate embedding region R that is selected from the host image as the darkness plain region, and (x₀, y₀) is the region position in Ih. To prevent a quality degradation of the exhibited watermark, a denoising operation is applied interactively to the adequate embedding region before being watermarked. The denoising operation is controlled in each n iteration by a threshold δ_n ∈ [0,1], which represents the pixels’ percentage of the embedding region with a similar value to the intensity i*.

Finally, once the denoised host images are obtained, Ih′, the watermark W is embedded by a slight adjusting of the pixels intensity value, where according to [6], the degree of adjustment has an empirical value Δ = 3. Then, the watermarked image is obtained as follows:

$$Iw\left(x,y\right)=\{\begin{array}{l}I{h}^{\prime }(x,y)+\Delta ,ifW(x-x*+1,y-y*+1)=1,\\ I{h}^{\prime }(x,y),otherwise.\end{array}$$

(2)

2.2. Improved Unseen–Visible Watermarking

In [7], Juarez et al. propose an improved version of [6], where two methodologies are adapted to improve the reported UVW algorithm. To increase the applicability of images without any darkness plain region, the Total-Variation L1-Norm [23] is introduced, which decomposes the input image luminance into texture and cartoon images since the adequate embedding region can be selected from the cartoon image that contains several dark plain regions. To embed the binary watermark, the same proposed methodology is used with the difference that the embedding strength, Δ, is calculated according to the human visual system (HVS) through a methodology called Just Noticeable Distortion (JND) [24,25,26,27,28]. In the exhibition stage, the shifted GCF is used to reveal the watermark to the naked eyes.

2.3. Camouflaged Unseen–Visible Watermarking

In [8], a camouflage unseen–visible watermarking algorithm was proposed to deliver information, claiming the copyright protection and ownership authentication of color images under two exhibition strategies: a composed strategy of logarithmic and negative transformations, and the image enhancement functions equipped into the mobile display devices, as is shown in Figure 2. In this algorithm, an adequate embedding region R with the minimum mean value is selected from the color image luminance according to (3) and (4).

$$(\widehat{x},\widehat{y})=\underset{(x,y)}{\min }\left(\frac{1}{ww\times hw}{\displaystyle \sum _{i,j\in \mathsf{\Omega }}W(i,j)}\right),$$

(3)

$$\mathsf{\Omega }=\{\begin{array}{c}i\in \left[x-\frac{ww}{2},x+\frac{ww}{2}\right]\\ j\in \left[y-\frac{hw}{2},y+\frac{hw}{2}\right]\end{array},x=1,\dots ,M;y=1,\dots ,N,$$

(4)

where (x, y) denotes the appropriate position in the image luminance to find the central pixel $\left(\widehat{x},\widehat{y}\right)$ of R. Then, an image texture classification process in the DCT domain [29] is used to classify each non-overlapping block of size (8 × 8) of R, where the dark and weak texture blocks correspond to class 1, semi-dark and strong texture blocks correspond to class 3 and the remaining blocks correspond to class 2.

Once R is obtained and its blocks are classified, a binary watermark with the same size of R is embedded adaptively according to its block classification. If the b_ζ pixel of the ζ-th candidate region corresponds to class 1, the embedding process follows the process given by (5); otherwise, if the b_ζ pixel corresponds to class 2 or 3, the embedding process is given by (6).

$$b_{\varsigma }^{*}(x,y)=\{\begin{array}{l}{b}_{\varsigma }(x,y)+\beta ,ifW(i,j)=0,\\ b_{\varsigma }(x,y)-\beta ,otherwise,\end{array}$$

(5)

$$b_{\varsigma }^{*}(x,y)=\{\begin{array}{l}{b}_{\varsigma }(x,y)+0.5b_{\varsigma }(x,y)·\alpha ,ifW(i,j)=0,\\ b_{\varsigma }(x,y)-0.5b_{\varsigma }(x,y)·\alpha ,otherwise,\end{array}$$

(6)

where $b_{\varsigma }^{*}(x,y)$ is the watermarked pixel and β and α are the corresponding strength factor to class 1, and class 2 or 3, respectively. After the embedding process, the watermarked image Iw is reconstructed. Finally, to exhibit the watermark image to the HVS, two strategies are adopted: in the first, a logarithmic transformation given by c log(1 + Iw) is applied to the watermarked image, where c and Iw are a constant and the watermarked image, respectively; subsequently, a negative transformation is applied to the logarithmic representation of the watermarked image. In the second strategy, a mobile device is superimposing to the watermarked image, and only varying the brightness and contrast equipped into the mobile devices, the watermark can be exhibited to the HVS; more details about exhibition strategies can be accessed in [8].

2.4. Motivation

The UVW and the IVW are developed to deliver information to the end-users via owner logotype, 2D barcodes or QR codes. In the IVW algorithms [9,10], the embedding stage is based on a histogram accumulation strategy that modifies the histogram of the embedding region in the upper and lower boundaries with respect to its mean value [11]; its imperceptibility is reported in the range of 40 dB–60 dB. To exhibit the watermark, the processing capabilities equipped into capture and display devices are used. The UVW algorithms [6,7,8] embed a binary watermark by a slight adjustment of the embedding region pixels and guarantee a watermark imperceptibility above 40 dB; however, they do not offer better performance in images without a darkness plain region [15,16]. To exhibit the watermark, these algorithms take advantage of the image enhancement function equipped into the display devices such as GCF [13], brightness [30], contrast [14] and the angle of vision [21,22], such that a complex algorithm is not required. According to the analysis, in both cases, the embedding stage increases the histogram distortion and visual degradation of the watermarked image, disserving watermark imperceptibility. These alterations are a consequence of its corresponding embedding strategy that does not consider the watermark characteristics. To solve these drawbacks, the watermark features are considered in the proposed algorithm, which is applied to any images with large smooth regions, adopting a watermark complement strategy to reduce the visual degradation and the histogram distortion, increasing the imperceptibility of the watermark and the watermarked image quality. Additionally, to quantify the embedding error, a measurement strategy is introduced.

3. Materials and Methods

In this section, the main requirement and the key contribution of the proposed algorithm are described in detail.

3.1. Just Noticeable Distortion (JND)

The luminance-based JND [24] used in the proposed algorithm is a numerical representation of the human eyes’ ability to perceive intensity variations into an image, which has been employed on several applications of video [24], image processing [25], data hiding [26], image compression [27] and image watermarking [28]. According to the proposed methodology, if the embedding region ensures a minimum variance, its mean value can be used to obtain an embedding strength based on the HVS perception; this methodology is calculated in the spatial domain by (7). As is shown in Figure 3, the HVS has a weak perception to intensity variations in the smooth region within mean value in the range (0, 32), whereas a strong perception is given in the range (33, 208) and a middle perception is given in the range of (209, 255).

$$JND\left({\mu }_R\right)=\{\begin{array}{c}-\frac{1}{8}{\mu }_R+6,0\le {\mu }_R\le 32\\ -\frac{1}{32}{\mu }_R+3,33\le {\mu }_R\le 64\\ \frac{1}{96}{\mu }_R+\frac{1}{3},65\le {\mu }_R\le 255\end{array}.$$

(7)

3.2. Watermark Complement Strategy (WCS)

Inspired by [31,32] and how several applications can scan inverted QR codes, a watermark complement strategy based on the watermark features is introduced to reduce the histogram distortion and the visual degradation of the UVW algorithms.

In addition to the owner logotype and 2D barcode as watermark, the UVW algorithms also propose the use of QR codes, which can be obtained in a binary representation [12], with features ‘0s’ and ‘1s’, such that its absolute watermark length is given by:

$$\left|W\right|=\left|W_0\right|+\left|W_1\right|,$$

(8)

$$\{\begin{array}{ll}\left|W_0\right|={\displaystyle \sum _{i=1}^{ww}{\displaystyle \sum _{j=1}^{hw}W(i,j)+1,}}& ifW(i,j)=0,\\ \left|W_1\right|={\displaystyle \sum _{i=1}^{ww}{\displaystyle \sum _{j=1}^{hw}W(i,j)},}& ifW(i,j)=1,\end{array}$$

(9)

where according to Figure 4a, |W₀| = 7029 and |W₁| = 15,471 are the absolute number of ‘0s’ and ‘1s’ of the binary watermark W, respectively; and $\left|\overline{W_0}\right|=\mathrm{15,471}$, $\left|{\overline{W}}_1\right|=7029$ correspond to the watermark complement, Figure 4b. To embed the watermark or its corresponding complement, the appropriate embedding region R has an absolute length given by:

$$\left|R\right|=\left|R_1\right|+\left|R_0\right|,$$

(10)

$$\{\begin{array}{ll}\left|R_0\right|={\displaystyle \sum _{x=1}^{wr}{\displaystyle \sum _{y=1}^{hr}R(x,y)}}+1& |R(x,y):\to W(i,j)=0,\\ \left|R_1\right|={\displaystyle \sum _{x=1}^{wr}{\displaystyle \sum _{y=1}^{hr}R(x,y)}}& |R(x,y):\to W(i,j)=1,\end{array}$$

(11)

where |R₀| = 7029 and |R₁| = 15,471 are the absolute number of pixels of the embedding region that match with the ‘0s’ and ‘1s’ of watermark, respectively. Considering the embedding strategies mentioned in the previous reported UVW algorithms, a general embedding strategy is obtained, formulated by (12).

$$R_w(x,y)=\{\begin{array}{l}R(x,y)+\Delta ,ifW(i,j)=1,\\ R(x,y),\hspace{1em}otherwise,\end{array}$$

(12)

where R_w is the watermarked region and Δ is the embedding strength obtained from the JND in the range of (1, 6), where Δ = 1 and Δ = 6 represent the minimum and the maximum histogram distortion. According to the previously mentioned, the binary watermark of Figure 4a is embedded into the image lena.bmp by the general embedding strategy and a calculated embedding strength Δ = 1; as a result, Figure 5a,b show a visual degradation with a PSNR = 34.1075 dB and a histogram distortion with 15,471 modified pixels, respectively.

With the aim to reduce the drawbacks of Figure 5a,b, the proposed watermark complement strategy is based on the evaluation of the pixels within the embedding region with respect to its mean value. This evaluation determines if the original watermark W or its corresponding complement $\overline{W}=1-W$ is the suitable watermark W′ to be embedded; this evaluation is satisfied by:

$$W^{\prime }=\{\begin{array}{l}\overline{W},if\left|{\displaystyle \sum _{x=1}^{ww}{\displaystyle \sum _{y=1}^{hw}\Gamma \left(R_1\left(x,y\right)<{\mu }_R\right)}}\right|>\left|{\displaystyle \sum _{x=1}^{ww}{\displaystyle \sum _{y=1}^{hw}\Gamma \left(R_0\left(x,y\right)\ge {\mu }_R\right)}}\right|,\\ W,if\left|{\displaystyle \sum _{x=1}^{ww}{\displaystyle \sum _{y=1}^{hw}\Gamma \left(R_1\left(x,y\right)\le {\mu }_R\right)}}\right|\le \left|{\displaystyle \sum _{x=1}^{ww}{\displaystyle \sum _{y=1}^{hw}\Gamma \left(R_0\left(x,y\right)>{\mu }_R\right)}}\right|,\end{array}$$

(13)

$$\Gamma (s)=\{\begin{array}{l}1,ifs\ne 0,\\ 0,otherwise,\end{array}$$

(14)

where R₀ and R₁ are the pixels of the embedding region that match with the ‘0s’ and ‘1s’ of the watermark, respectively, and Γ(s) is a conditional function. Finally, based on the previous evaluation, the binary watermark must be embedded into R according to its mean value µ_R; in this context, the watermark’s ‘0s’ are embedded into the pixels of the embedding region in the range of (0, µ_R-1), f:W = 0→(0, µ_R-1), and the ‘1s’ in the range of (µ_R, 255), f:W = 1→(µ_R, 255), where f corresponds to the general embedding strategy.

According to the proposed strategy, an embedding error (EE) measurement is obtained; it considers all the pixels in the ranges (0, µ_R-1) and (µ_R, 255) where the ‘0s’ and ‘1s’ of the watermark were embedded, respectively, as given by (15).

$$EE=\left(\frac{{\displaystyle \sum _{x=1}^{ww}{\displaystyle \sum _{y=1}^{hw}\left(\Gamma \left(R_1\left(x,y\right)<{\mu }_R\right)+\Gamma \left(R_0\left(x,y\right)\ge {\mu }_R\right)\right)}}\times 100}{ww\times hw}\right)$$

(15)

To observe the feasibility of the proposed strategy, the watermark complement, Figure 4b, is embedded into the image lena.bmp. The resulting Figure 5c,d shows a visual degradation with a PSNR = 42.23 dB and a histogram distortion with 7029 modified pixels, respectively. Finally, the watermark complement strategy generates an EE = 35.64% in comparison with EE = 44.68% obtained by the original watermark. In addition to the previous analysis, the relation (|W′₀|>>|W′₁|) must be satisfied during the embedding stage to ensure a reduction of visual degradation and histogram distortion.

3.3. Proposed Watermarking Algorithm

In this section, the unseen–visible watermarking algorithm in combination with the proposed strategy is described by two stages— embedding and exhibition [33].

3.3.1. Embedding Stage

1.

The host image

Ih with RGB color model is the corresponding input to the embedding stage, as shown and listed in Algorithms 1 and Figure 6, respectively. The major problem in the UVW algorithms is the selection of the color model and the embedding region where the watermark will be embedded. The RGB color model has strongly correlated components; however, the alteration into one or more components is not perceived to the naked eyes because image colors are dependent on each other. Conversely, the YCbCr color model has different advantages that should be considered; the most important are the low correlation between luminance and its chrominance components, the image information accumulated into the luminance Y and any alteration in Y observed in each RGB color channel [34,35]. In this way, the host image becomes the YCbCr color model [30], and the luminance Y is isolated.

2.

Adequate embedding region

To make this algorithm suitable to any class of images with large smooth regions of low or high intensity, the adequate embedding region R—with an upper-left corner pixel position (x, y), where x = 1, 2, 3, …, w_R and y = 1, 2, 3,…, h_R are the width and height, respectively—is obtained from the luminance Y of the host image Ih, satisfying the minimum variance given by (16). An empirical analysis indicates that to obtain better results, the range of intensity where the adequate embedding region must be selected corresponding to (0, 50) and (200, 255), low and high intensity, respectively. In this way, the mean value of the region is satisfied by 0 ≤ µ_R ≤ 50 for images with low intensity and 200 ≤ µ_R ≤ 255 for images with high intensity.

$$\underset{R\in Y}{\min }{\left(\frac{1}{\left(w_R\times h_R\right)-1}{\displaystyle \sum _{x=1}^{w_R}{\displaystyle \sum _{y=1}^{h_R}\left|R(x,y)-{\mu }_R\right|}}\right)}^2.$$

(16)

3.

Embedding Strength

As mentioned in Section 3.1, to obtain an embedding strength in terms of the JND, refs. [7,8,10,11,33] as well as mean value µ_R that guarantee high watermark imperceptibility to HVS, the embedding region must ensure a minimum variance. However, when µ_R ≥ 253 and JND(µ_R) = 3, an overflow can be generated by the embedding strategy into the pixels of the watermarked region Rw. In this way, to avoid this overflow, the embedding strength is finally obtained by a soft adjustment, as is formulated in (17).

$$\Delta =\{\begin{array}{l}JND\left({\mu }_R\right)-1,if{\mu }_R\ge 253,\\ JND\left({\mu }_R\right),otherwise.\end{array}$$

(17)

4.

Embedding Strategy

As is shown in Figure 6, once the embedding region R and the embedding strength Δ are obtained, the watermark is evaluated by (13) to determine the suitable watermark W′, which is embedded by (18).

$$R_w(x,y)=\{\begin{array}{l}R(x,y)+\lfloor \Delta \rfloor ,if{W}^{\prime }(i,j)=1,\\ R(x,y),otherwise,\end{array}$$

(18)

where R_w corresponds to the watermarked region. Along with the procedure of the embedding stage, Algorithms 1 summarizes and details each step to embed a watermark W′ into the adequate embedding region R of the host image Ih.

Algorithms 1. Embedding survey.

Input: Host Image Ih, watermark W.

Step 1:  The host imagen Ih becomes the RGB color model to the YCbCr, and the luminance Y is isolated. Step 2:  According to (16), the adequate embedding region R is obtained from the luminance Y. Step 3:  The watermark is evaluated by (13) to select the suitable watermark W′, ensuring the minimum embedding error given by (15). Step 4:  The JND of the mean value µR and the embedding strength are obtained according to (7) and (17), respectively. Step 5:  The watermark is embedded into the adequate embedding region R, satisfying (18). Step 6:  Once the watermarked region Rw is obtained, the luminance is rebuilt YM. Step 7:  Finally, the watermarked image is obtained by the addition of the chrominance components to the watermarked luminance YM, becoming the color model YCbCr to the RGB.

Output: Watermarked image Iw.

3.3.2. Exhibition Stage

To exhibit the watermark to the HVS, an extra complex computational algorithm is not required [33]; this algorithm takes advantage of the image enhancement function equipped into display devices, mainly, the image enhancement function GCF. In this section, the GCF and other feasible exhibition strategies based on the GCF, contrast, brightness and the angle of vision are described in detail.

1.

Gamma Correction Function (GCF)

Recently, the GCF is incorporated as a function to improve the visual content of images into display devices and is formulated by:

$$P_{out}=C{P}_{\mathrm{int}}^{\gamma }$$

(19)

where Pint and Pout are the intensity value of the input pixel and its corresponding mapping output pixel, respectively, γ and C are defined by the system where it is implemented.

In this way, if γ = 1, then intensity values of the output pixels are mapped with the same value of the corresponding input pixel, as is shown in Figure 7a; when γ < 1, the input pixels with low intensity are mapped to high-intensity values, and the input pixels with high intensity are saturated as shown in Figure 7b. Finally, when γ > 1, the input pixels are mapped conversely to γ < 1, as shown in Figure 7c.

Additionally, to the GCF as an exhibition strategy, another option capable of improving the exhibited watermark readability on large smooth regions is the shift operation of the GCF that can be applied to the watermarked image with respect to the mean value of the adequate embedding region, as shown in Figure 8.

2.

Other feasible exhibition strategies

As mentioned above, the most common image enhancement function used through the display devices to exhibit the watermark to the HVS is the GCF; however, the GCF is not the only image enhancement function integrated into the display devices capable of exhibiting the watermark to the HVS. In this context, other feasible exhibition strategies based on the pixels intensity variation are contrast, brightness and angle of vision.

The most popular contrast system adopted by display devices manufacturers [14] defines the contrast and the brightness as the absolute difference between the white peak and the black luminance levels and the way to perceive the light intensity by the HVS, respectively. However, in an image processing context, both functions are pixel operations where the output pixel depends on the input pixel. The combination of contrast and the brightness functions, which can be applied to the watermarked image through display devices, is given by:

$$I_R(x,y)=a{I}_w(x,y)+b,$$

(20)

where a and b are the corresponding parameters to adjust the contrast and the brightness, respectively.

The reported algorithms [21,22] are based on LCD technology [36], where the angle of vision is used to show multiple images through display devices. The idea to take advantage of the angle of vision as another feasible exhibition strategy is focused on how the display devices modify the contrast and the brightness perception [14] to the HVS when it has changed in azimuth and elevation, as shown in Figure 9.

4. Experimental Results and Analysis

In this section, the watermarked image quality is evaluated by the Peak Signal to Noise Ratio (PSNR) given by (21), and the performance of the proposed algorithm is evaluated in terms of imperceptibility by the Structural Similarity Index Measure (SSIM), [37], and the Normalized Color Difference (NCD), [38], that are formulated by (22) and (23), respectively. A PSNR > 37 dB shows a high quality between the original host image and its corresponding watermarked image. Besides the evaluation of the imperceptibility, SSIM and NCD metrics also are used to evaluate the similarity between host and watermarked image. An SSIM value nearest to ‘1’ corresponds to identical images, whereas the CIELAB-based color space NCD metric is used to measure the difference in terms of color between the host and the watermarked image. An NCD value nearest to ‘0’ corresponds to identical images. The robustness of the proposed algorithm is evaluated against common geometric and image processing attacks listed in

Table 1. Common geometric attacks used to the experimental.

Attack	Specification
Rotation	Angle 45°
Translation with cropping	x = 15, y = 50
Affine transformation	(1,0.1,0;0.1,1,0;0,0,1)
Aspect ratio	(2,0,0;0,1.0,0;0,0,1)
Flip transformation	Horizontal direction Vertical direction
Shearing	x-direction (1 0 0; 0.5 1 0; 0 0 1) y-direction (1 0.5 0; 0 1 0; 0 0 1)
Rescaling	Rescaling 50%
Barrel	-------

and

Table 2. Image processing attacks.

Attack	Specification
Noise contamination	Impulsive, density = 0.4 Gaussian, µ = 0, σ2 = 0.001
Sharpening	Radius = 2; Amount = 1
Blurring	Radio = 3
Average filter	3 × 3
Compression	JPEG QF = 75 JPEG QF = 50 JPEG2000 CR = 20 JPEG2000 CR = 10

, respectively. Finally, the performance of the proposed algorithm is mainly contrasted with three previously reported algorithms [6,7,8]. To present a fair comparison, the host images are obtained from the uncompressed color image dataset (UCID) [39], which contains 1338 uncompressed TIFF color images of 512 × 384 size. Watermark patterns are based on two binary images given by an owner logotype and the QR code with sizes 117 × 117 and 150 × 150, respectively.

$$PSNR(dB)=10{\log }_{10}\frac{MP{V}^2}{\frac{1}{N\times M}\left({\displaystyle \sum _{x=1}^M{\displaystyle \sum _{y=1}^N{\left(Ih(x,y)-I_W(x,y)\right)}^2}}\right)},$$

(21)

$$SSIM\left(Ih,I_W\right)=\frac{\left(2{\mu }_{Ih}{\mu }_{I_W}+C_1\right)\left(2{\sigma }_{Ih{I}_W}+C_2\right)}{\left({\mu }_{Ih}^2+{\mu }_{I_W}^2+C_1\right)\left({\sigma }_{Ih}^2{\sigma }_{I_W}^2+C_2\right)},$$

(22)

$$NCD=\frac{{\displaystyle \sum _{x=1}^h{\displaystyle \sum _{y=1}^w\left(\sqrt{{\left(\Delta L(x,y)\right)}^2+{\left(\Delta a(x,y)\right)}^2+{\left(\Delta b(x,y)\right)}^2}\right)}}}{{\displaystyle \sum _{x=1}^h{\displaystyle \sum _{y=1}^w\left(\sqrt{{\left(L(x,y)\right)}^2+{\left(a(x,y)\right)}^2+{\left(b(x,y)\right)}^2}\right)}}}.$$

(23)

In the PSNR expression, the MPV represents the maximum pixel value of the host image. In the SSIM, µ_X, ${\sigma }_X^2$ and ${\sigma }_{XY}$ are the mean value, the variance and the covariance of the corresponding Ih and I_W, respectively, C₁ and C₂ are two constants with the same value defined in [37]. Finally, In the NCD, L represents the luminance, a and b are the color variation between the luminance of green–red and blue–yellow, respectively.

4.1. General Performance Validation

In addition to the use of the 2D barcodes and the owner logotype as watermark patterns, the UVW algorithms also propose the incorporation of the QR code in a binary representation as a novel watermark pattern to deliver auxiliary information. This is based on the observation that the QR code is the most popular mechanism to store and share a high amount of information among end-users and a wide variety of industry sectors. The QR code used in the performance evaluation of the proposed algorithm corresponds to the classification L with 7% of data restoration.

Table 3. Average quality and imperceptibility of the proposed algorithm.

Watermark Complement Strategy	Watermark Size	Low-Intensity Embedding Region			High-Intensity Embedding Region
		PSNR (dB)	SSIM	NCD	PSNR (dB)	SSIM	NCD
Incorporated	QR code Logotype	51.52 52.45	0.9975 0.9985	0.0158 0.0105	50.64 51.58	0.9890 0.9984	0.0109 0.0097
Non-Incorporated	QR code Logotype	46.59 47.18	0.9932 0.9950	0.2010 0.1508	47.67 47.12	0.9857 0.9938	0.0981 0.1548

shows the average quality and imperceptibility of the proposed algorithm applied to images with large smooth regions with low and high intensity. To appreciate the performance of the proposed algorithm, the results are obtained with and without the incorporation of the proposed watermark complement strategy that also reduces the histogram distortion and visual degradation, highlighted by its suitability to any class of images, where more than the 74% of images used in this test were watermarked using the proposed watermark complement strategy.

Figure 10, Figure 11, Figure 12 and Figure 13 show a sample set of images that are marked by a QR code. The watermarked images show, under normal view conditions, the same visual quality as its corresponding original image. At the same time, these set of images show results of the QR code exhibited by the GCF, Shift GCF operation, contrast and brightness combination and the angle of vision, where the watermark in terms of readability and imperceptibility is recognized in high quality by the HVS. Figure 10b shows a watermarked image where a QR code is embedded into an embedding region located on a smooth area with low intensity. As shown in Figure 10c, after enhancing the watermarked image by the GCF, the QR code is exhibited with enough quality to be decoded by any QR code reader application. Figure 11b and Figure 12b show a watermarked image where the QR code is embedded into a region located in a middle intensity. Although the proposed algorithm does not consider middle intensities to embed and exhibit a watermark pattern, the proposed algorithm is able to exhibit it to the HVS with high visual quality after enhancing the watermarked image by the proposed exhibition strategies. In Figure 11c and Figure 12c, the watermark is exhibited by the Shift GCF and the combination of contrast and brightness, respectively. In Figure 13a–c, the watermark is exhibited to the HVS by the angle of vision, GCF and the combination of the contrast and brightness, respectively, that are functions integrated into a Samsung display device model UN32J4300AF; moreover, the obtained results can be improved using another display device with better resources.

4.2. Watermark Robustness

To evaluate the robustness of the proposed algorithm against common geometric and image processing attacks listed in Table 1 and Table 2, respectively, two sample images, ucid00327 and ucid00539, obtained from the UCID dataset, were marked by a QR code and the owner logotype.

4.2.1. Robustness to Image Geometric Attacks

Figure 14 shows a sample image set, where, to demonstrate that the proposed algorithm is not limited to embed only one watermark pattern, Figure 14a is used to embed a complement representation of the QR code and the owner logotype. Figure 14b shows the result of the double-watermarked image that, in terms of imperceptibility, maintains a high visual quality to the HVS; additionally, Figure 14c shows both watermarks exhibited to the HVS after being enhanced by the Shift GCF operation.

To evaluate the performance of the proposed algorithm against common geometric attacks, the double-watermarked image Figure 14b is processed by the attacks listed in Table 1; after that, the watermark is exhibited by the proposed image enhancement operation. Figure 15 shows several images with good robustness against the attacks listed in Table 1, including rotation 45°, affine transformation, an aggressive aspect ratio, shearing in x and y directions and barrel transformation. In all cases, the watermarks are exhibited with high visual quality to the HVS; moreover, the quality of the exhibited QR code is sufficient to decode the information by any QR code reader application.

4.2.2. Robustness to Image Processing Attacks

Figure 16 shows a sample image set used to evaluate the performance of the proposed algorithm against image processing attacks. Figure 16a shows the original image, whereas Figure 16b corresponds to the watermarked image, where a QR code is embedded into the smooth region with minimum variance and low intensity. Finally, Figure 16c corresponds to the enhanced representation of Figure 16b after being treated by the shift GCF exhibition strategy, and where the exhibited QR code has enough quantity to be read by any application focused on reading QR codes. Figure 17 shows the robustness against several image processing attacks, listed in Table 2. To obtain the experimental results, first, the watermarked image of Figure 16b is attacked with the maximum attack factors which allow reading and decode the QR code; second, the attacked representation of the watermarked image is treated by the shift GCF to exhibit the watermark pattern. The attacks include two of the most popular compression formats, JPEG and JPEG2000. The robustness of the proposed algorithm to different compression factors of the JPEG and JPEG2000 formats are shown in Figure 18a,b, respectively; in both images, the compression factor and the ability to be read and decode the information of the QR code are compared in terms of the data restoration, that correspond to 7%, 15%, 25% and 30% to the QR code classification L, M, Q and H, respectively. From the experimental results, two main observations are obtained. First, robustness to JPEG and JPEG2000 of the proposed algorithm can exhibit the QR code with high visual quality to be read by any application. Second, in the case of using an owner logotype as a watermark, the proposed algorithm can overcome a compression factor QF = 40, as shown in Figure 19a to JPEG and Figure 19b to JPEG2000.

4.2.3. Performance Comparison and Discussion

In

Table 4. Comparison of embedding error, quality and imperceptibility.

Algorithm	Watermark	EE (%)		PSNR (dB)		SSIM		NCD
		High-Intensity Embedding Region	Low-Intensity Embedding Region	High-Intensity Embedding Region	Low-Intensity Embedding Region	High-Intensity Embedding Region	Low-Intensity Embedding Region	High-Intensity Embedding Region	Low-Intensity Embedding Region
Unseen–Visible Watermarking [6]	QR Code Logotype	33.54 32.98	37.86 36.45	46.95 49.08	45.05 49.14	0.9850 0.9945	0.9810 0.9975	0.0154 0.0160	0.0102 0.0198
Improved Unseen–Visible Watermarking [7]	QR Code Logotype	34.01 38.46	39.47 37.15	48.91 51.83	45.78 50.30	0.9881 0.9986	0.9881 0.9990	0.0237 0.0285	0.0192 0.0159
Camouflaged Unseen–Visible Watermarking [8]	QR Code Logotype	36.12 39.04	38.80 37.69	45.19 41.40	42.94 43.58	0.9613 0. 9489	0.9564 0.9452	0.0178 0.0204	0.0189 0.0177
Proposed	QR Code Logotype	32.12 32.25	35.42 35.02	50.64 51.58	51.52 52.45	0.9890 0.9984	0.9975 0.9985	0.0102 0.0097	0.0158 0.0105

, the performance in terms of the visual quality and imperceptibility of the proposed algorithm is compared with the previously reported algorithms [6,7,8], under the same features such as image watermark, hardware capabilities and image dataset. To obtain trustworthy results, a set of 800 images of the UCID dataset that had been selected randomly was split into 465 and 335 images with embedding regions located in low and high intensity, respectively. Due to the incorporation of the watermark complement strategy to reduce the histogram distortion and the visual degradation, the applicability of the proposed algorithm is not limited to images with low-intensity regions as reported in [6,7,8], an extra denoising operation [6,7], and complex computational algorithms to embed or exhibit the watermark [8,9,10].

The experimental results in terms of embedding error, visual quality and imperceptibility with EE ≤ 35.42%, PSNR ≥ 50.64 dB and SSIM ≥ 0.9890, NCD ≤ 0.0158, respectively, show that the proposed algorithm improves the performance of the previously reported algorithms [6,7,8]; in addition, the results in [6,7] are nearest to the proposed algorithm since their embedding strategy is similar to the proposed one. In contrast, the reported results in [8] have weaker performance than the proposal, as shown in Table 4, because the embedding strategy increases the histogram distortion and visual degradation.

Table 5. Global comparison among previously reported algorithms and the proposed.

Parameter	Proposed	Unseen–Visible Watermarking			Imperceptible Visible Watermarking
		Unseen–Visible Watermarking [6]	Improved Unseen–Visible Watermarking [7]	Camouflaged Unseen–Visible Watermarking [8]	Imperceptible Visible Watermarking [9]	Improved Imperceptible Visible Watermarking [10]
Watermark	Owner logotype, 2D barcode and QR code	Owner logotype, 2D barcode and QR code	Owner logotype, 2D barcode and QR code	Owner Logotype	Owner logotype and QR code	Owner logotype, 2D barcode and QR code
Universality	Images with large smooth regions with low or high intensity	Images with large smooth region with low intensity	Images with large smooth region with low intensity	Images with large smooth region with low intensity	Images with large smooth region	Images with large smooth region
Invisible watermark	Yes	Yes	Yes	Yes	Yes	Yes
Visible watermark	No	No	No	No	No	No
Embedding strength	JND-based	Empiric	JND-based	JND-based	Empiric	JND Based
Visual degradation	Low	High	High	High	High	High
Histogram distortion	Low	High	High	High	High	High
Watermarked image quality	High	High	High	High	High	High
Extra exhibition information	Not required	Gamma/Shift value	Shift value	Not required	Mean value	K-th Mean value and color channel
Exhibition procedure	Gamma/shift gamma/contrast and brightness combination/angle of vision/histogram modulation	Gamma	Shift gamma	Logarithmic transformation and negative function/image enhancement by a mobile device	Histogram modulation	Binarization function
Quality of the exhibited watermark	High	Media	Media	Low	High	High
Exhibited watermark nature	Grayscale/Color	Grayscale	Grayscale	Grayscale	Grayscale/Color	Binary
Multiple watermarks	Yes	No	No	Yes	Yes	Yes
Provide auxiliary information	Yes	Yes	Yes	Yes	Yes	Yes
Computational complexity	Low	Low	High	High	High	High
Intellectual property protection	Yes	Yes	Yes	Yes	Yes	Yes
Copy right protection	Yes	Yes	Yes	Yes	Yes	Yes
Prevention of non-authorized duplicity	Yes	Yes	Yes	Yes	Yes	Yes
Additional DCT algorithm	No required	No required	No required	Required	No required	Required
Robustness to JPEG	High	Low	Low	Media	Low	High
Robustness to JPEG2000	High	Low	Low	Media	Low	High

shows a global comparison of the proposed algorithm against previously reported UVW algorithms [6,7,8]; additionally, for a broader overview of the invisible–visible watermarking algorithms, the most relevant IVW algorithms [9,10] are considered in this table.

Considering Table 4 and several aspects of each algorithm shown in Table 5, the proposal is more suitable to be used in practical applications of additional information delivery, information security tasks [15,16,17,18,19] and other related fields [20], such as is mentioned in Section 1.

According to the mechanism to exhibit the watermark pattern, nowadays, traditional electronic devices with the purpose of giving the end-users a comfortable visual experience via a high quality and resolution [40,41] have been developed with several image and video processing capabilities [41,42,43,44], but exhibiting watermarks focused to additional information delivery can be conditioned by manufactures with dedicated functions or applications.

The angle of vision as another strategy to exhibit the watermark pattern can be considered as a new research subject based on imperfect systems [45,46] because the strategy could be considered as a consequence of an imperfect physical process.

In comparison with the IVW algorithms, the proposed algorithm shows similar results to [9], but with minimum computational complexity and a high amount of exhibition strategies integrated into common display devices; in comparison with [10], the proposed algorithm does not require an additional DCT-based algorithm in the embedding and exhibition stages.

5. Conclusions

In this paper, a novel complement unseen–visible watermark algorithm is proposed to deliver information via color images and display devices. The conclusions of the proposed algorithm are highlighted according to its three main contributions: first, the minimum variance formulated in Equation (16), and the mean value in the range μR ∈ (0,50) and μR ∈ (200, 255) is not limited to select the appropriate embedding region with middle intensity, as shown in Figure 11 and Figure 12. Second, the watermark complement strategy given by (13) reduces the histogram distortion and the visual degradation of the watermarked image using an embedding strategy proposed in (18) that satisfies the relation (|W′0|>>|W′1|). The third contribution is the strategy to quantify the embedding error by using (15), being a parameter that ensures minimal distortion during the embedding stage.

The experimental results show that the proposed algorithm improves several aspects of the previously reported algorithm in [6,7,8]; with an SSIM value close to ‘1’ and NDC value close to ‘0’, the watermarked images maintain a high imperceptibility of the watermark pattern that can be an owner logotype, 2D barcodes or QR code. The high visual quality of the watermarked image is obtained with a PSNR ≥ 50.64 dB and a mean value of EE = 35.42% for any class of images. Additionally, the proposed algorithm can exhibit the watermark pattern via several exhibition strategies incorporated within common display devices; in this way, empirical non-reported results show that histogram modulation can be used as another feasible exhibition strategy; however, this function is not incorporated within display devices.

The addition of the histogram modulation as another exhibition strategy, in combination with several aspects listed in Table 5, makes possible a general comparison between the proposed one and two most common IVW algorithms [9,10]. The proposed algorithm provides higher robustness against common geometrics and image processing attacks listed in Table 1 and Table 2, respectively. The most relevant results are obtained for JPEG and JPEG2000 compression, where, after applying the JPEG and JPEG2000 compression to the watermarked image, the QR code as watermark can be decoded by common reader application as shown to different QR-classification in Figure 18. In this way, the features of an owner logotype as a watermark increase the robustness to the compression attacks, as shown in Figure 19.

In general, the robustness of the proposed algorithm and the several watermark exhibition strategies incorporated into common display devices make it suitable in practical multimedia applications. In the future, the presented work could be extended to video processing and information security and other applications related to electronic display devices.