Flickering-Free Distance-Independent Modulation Scheme for OCC

Abstract

Communications for IoT sensor networks in Smart City applications are demanding new technologies to avoid the radio frequency spectrum saturation and support the massive nodes situation required in this kind of network. Visible Light Communications are gaining interest as an alternative scheme for solving this problem. In particular, Optical Camera Communication (OCC) takes advantage of the presence of cameras in a wide range of devices (smartphones, surveillance systems). Several transmission and modulation schemes, specifically developed for OCC, have been proposed but they present different problems as the communication range, the flickering effects in the transmitted signal, or the system complexity. In this work, an Optical Camera Communication modulation scheme that addresses the considerable variability of the transmitter–receiver distance is proposed. As will be shown, the modulation scheme works properly independently of this separation for a wide range of distances. Additionally, it performs a flickering free transmission to avoid undesired blinking effects for humans and uses a simplified transmission and reception scheme, reducing the system complexity and cost. Results of the modulation’s performance for short (<20 m), medium (20–60 m), and long (>60 m) ranges are presented in the paper, showing a 16 bps transmission rate with Bit Error Rate below the Forward Error Correction (FEC) limit (3.8 · 10⁻³).

1. Introduction

A Smart City is usually defined as an urban area that employs devices (general-purpose sensors) to collect data and manage assets, resources, and services by applying data analytic tools in several applications such as governance, environmental impact, security, or emergency management. These different sensors gathering information can be considered Internet of Things (IoT) elements that generally connect and communicate with other devices over short, medium, and long distances using wired or wireless networks. Due to the number of sensor devices that a Smart City would require for making accurate decisions, traditional wireless communication links, based on Radio Frequency (RF), may be insufficient. This type of communication presents issues related to the current spectrum saturation and the inter-systems interference based on the extensive use of RF links, so other complementary wireless communication techniques are needed.

On account of the advantages of the Light Emitting Diodes (LED), such as energy efficiency, extended life span, cost-efficiency, and high available bandwidth, there is a trend in the use of LED-based devices for illumination, advertising, and decoration in both indoor and outdoor environments. This massive utilization of LED-based devices impulses the development and implementation of Visible Light Communications (VLC) systems [1], an optical wireless communication technology that works with the visible light portion of the electromagnetic spectrum (430 THz to 790 THz).

However, photodiode-based VLC system implementations imply using potentially expensive elements and the necessity of equipment modifications. In addition, many day-to-day devices such as smartphones, wearables, and tablets, have embedded cameras presenting large resolutions and high frame acquisition speeds. In the same way, camera-based security and monitoring systems have been implemented extensively. This context derived into the development of Optical Camera Communications (OCC) [2], which is a specific VLC-based technique that uses image sensors for receiving optical signals. The use of pre-existing camera-based infrastructures such as traffic-camera networks or security systems decreases OCC deployment costs. In addition, spatially-separated sources allow the design of Multple Input Multiple Output (MIMO) systems [3,4], increasing OCC systems’ data rate. In this way, OCC technology is suitable for deploying IoT applications and services.

The use of VLC for Smart City applications has been previously proposed. For instance, a Smart City’s public lighting system for ubiquitous communication ensuring broad coverage was presented in [5]. Nonetheless, the work left aside attractive possible emitters, such as traffic lights or advertising screens, and used modified devices as receivers. In [6], traffic lights and car lights were studied as potential transmitters for MIMO inter-vehicular communication solutions within a short distance range. An OCC system for rolling shutter cameras was proposed in [7], achieving a 2 m link. In [8], a long-distance OCC system, above 300 m, for slow data rate applications in Smart Cities was introduced, using pre-existing infrastructure over heterogeneous networks. However, this solution was affected by the flickering problem.

Nevertheless, the use of OCC for implementations within urban environments requires an extended analysis. The distance between emitters and receivers may vary ranging from some centimeters to several tens of meters, affecting the transmitter’s projection on the image sensor, and eventually the communications performance. Moreover, the majority of IoT devices have memory and processing limitations that should be taken into account. In summary, these deployments require a low-complexity communication algorithm capable of correctly functioning for a wide range of distances while maintaining human-friendly illumination.

In this work, a novel flickering-free distance-independent modulation scheme is proposed. It consists of an OCC-based modulation which can work adequately for short, medium, and long distances without the necessity of previous calibration or specialized equipment in both indoor and outdoor scenarios, which is a critical aspect of future OCC systems for IoT within urban surroundings. The modulation technique compares the similarity between consecutive frames in terms of 2D statistical values. The paper is organized as follows. A brief analysis of state-of-the-art modulations for OCC is presented in Section 2, while the theoretical fundamentals of the modulation are presented in Section 3. Then the modulation and demodulation schemes are detailed in Section 4, the experimental evaluation of the proposed scheme is detailed in Section 5, and the obtained results are presented and discussed in Section 6. Finally, some conclusions are provided in Section 7.

2. Related Work

In this section, the current state-of-the-art solutions for OCC systems are analyzed focusing on the proposed modulation techniques to verify the feasibility of employing OCC for outdoor long-distance links, including Smart City applications. Several modulation schemes for OCC have been proposed and can be mainly divided into four categories: based on OOK, polychromatic modulations, undersampling techniques, and based on space-time coding.

At first, the researchers adapted OOK modulation directly from VLC, where the on-light state represented a “1” bit whilst the off-light state corresponded to a “0” bit. In general, the implementation of this method is simple. In [9] the LED’s switching frequency was defined as half the camera’s frame rate, and two consecutive frames were required for representing each symbol, but the implementation involved a synchronization stage. For avoiding synchronization delimiters, an asynchronous OOK was presented [10], two out of three consecutive frames that represented two bits were selected for the demodulation process based on a predefined initial sequence. Later on, the authors improved this technique [11] by using an adaptive threshold for flickering-free links within $0.2$ m. In [12,13,14], the OOK modulation was combined with Manchester codification and the rolling shutter effect for representing several bits per frame. However, these modulations did not tolerate environmental noise or distances greater than 1 m. Rajagopal et al. [15,16] introduced a binary Frequency Shift Keying (FSK) scheme (frequencies separated by 200 Hz) that supported several transmitters with different pair of frequencies. This technique required a synchronization delimiter with a flag and a pilot signal for each transmission decreasing the throughput. In [17], frequencies from a predefined cluster depending on the camera’s characteristics were assigned for each symbol for improving the performance. Nevertheless, communication was feasible for short distances (less than 1 m) and required a fixed frequency splitter before each symbol, a parity symbol, and two delimiters. To improve link range, Li et al. [18] proposed the application of image processing in the receiver for using the illuminated area outside the light source. However, the maximum achieved distance was $0.4$ m. Finally, in [7] the application of an adaptive thresholding technique (Bradley’s algorithm) for Non-Line-Of-Sight (NLOS) links was carried out with successful results in a 2-m link, although the packets required to be transmitted twice including a 12 bit header, ultimately decreasing the throughput.

Several authors worked with polychromatic systems for taking advantage of Red-Green-Blue (RGB) cameras. At the beginning, the schemes were based on Color Shift Keying (CSK) modulation, which is primarily used for VLC. For example, in [19] a 4-CSK modulation was presented for short range links (less than 1 m), while Hu et al. [20] introduced a high level constellation modulation (8-CSK, 16-CSK, and 32-CSK), that needed a synchronization delimiter and periodic calibration packets during the transmission, and complex image processing for the demodulation. In [21,22], an asynchronous CSK scheme was presented for distances below 1 m, and an 8-CSK modulation [23] was presented along with a beacon-jointed packet reconstruction scheme for longer packets, but the system’s distance range was strongly limited (less than 5 cm). In [24] the authors defined their technique as Binary CSK. The modulation scheme is based on a light polarization change that was perceived by the receiver as two different colors due to the use of dispersion films. This technique achieved 10 m range but required physical modifications on the receivers. Another polychromatic modulation is Generalized Color Modulation (GCM), which is a generalization of CSK [25,26]. This scheme is based on creating different color constellations to achieve a specific target color working in the CIELUV space for long-range links (up to 75 m). Nevertheless, each packet required two delimiters, header, verification bits, and the decoding process involved high-complexity algorithms.

Modulation schemes based on undersampling transform the data signal into a passband signal used for switching the LED with a frequency high enough to avoid flickering. Depending on the applied shift keying, they are classified in: frequency, phase, and amplitude shifting. For example, the Undersampled Frequency Shift On-Off Keying (UFSOOK) modulation [27] used one specific frequency, greater than 100 Hz and a multiple of the camera’s frame rate for each state (one or zero), whilst Le et al. [28] defined the two subcarrier frequencies as a discrete sampling of the camera’s frame rate taking the shutter speed as a limitation. Chow et al. in [29] proposed an RGB modulation that combined frequency shifting with OOK signals by comparing two consecutive frames. For each frame, only one band was analyzed per color channel, and two consecutive frames in the same state (on or off) represented a “1” bit. The experimental results showed a 100 m link with $3.8·{10}^{-3}$ Bit Error Rate (BER). However, the system’s performance was very sensitive to the threshold definition and calibration. In [30], the authors applied a similar scheme over two transmitters, the second emitter’s signal was phase-shifted by ${90}^{\circ }$. The receiver selected the best emitter for each communication process. A BER of $1.7·{10}^{-3}$ was achieved for a 160 m. Nevertheless, the thresholding and calibration issues were not taken into account. In [31], the phase was shifted instead of the frequency, for representing complementary bits separated ${180}^{\circ }$. This technique reached distances up to 12 m, but it required a synchronization delimiter and the transmission of a “1” bit to define the system’s demodulation logic. A modification of this modulation was presented in [32], the signal shifted the phase ${180}^{\circ }$ in the second frame for sending a bit “1”, avoiding the previous restriction on the demodulation logic. The results showed a feasible $0.5$ m link. The light amplitude was changed in [33,34], avoiding the flickering problem while reaching long distances (60 m). Nevertheless, this method demanded the transmission of all intensity light levels in a predefined order. This technique’s data rate was improved by using a Quadrature Amplitude Modulation (QAM) [35], but for distances lower than $1.5$ m. Finally, a combination of Pulse Width Modulation (PWM) and amplitude shifting was introduced in [36]. The PWM signals modulated the LED’s brightness levels by changing its duty cycle instead of its frequency, while the bit stream was mapped onto an m-ary amplitude modulation constellation. This scheme is highly complex compared to other implementations and reached a modest distance (1 m).

Intelligent Transportation System (ITS) applications have been the target of a significant amount of contributions in the field. Amano et al. presented a space-time coding [37] based on a modified Alamouti code [38] which avoid the use of negative signals. For distances where each LED is represented by at least 1 pixel, the demodulation was simple, but longer distances require extra information and complex processing. During the validation, a 48 m link with a BER less than $1·{10}^{-5}$ was achieved. For improving this method, Ebihara et al. [39] introduced a layered space-time coding: the matrix was divided into submatrices where the diagonal matrices represented the first layer bitstream. A 1 kbps link was established for 210 m with $3·{10}^{-1}$ BER using a 3-layer bitstream. Similarly, Masuda et al. [40] presented a layered space-time coding for spatial modulation with different interpolation techniques. The system sent data successfully over a 240 m link. However, both methods required specialized software and hardware due to the process’ complexity.

In general, OOK-based modulations have been shown to be low-complexity techniques that work correctly with moderate data rates exclusively for short distances. Therefore, this type of OCC modulation can be used for specific IoT applications within the range of 5 m, supporting low data rates. Since polychromatic modulations take advantage of RGB channels for simultaneous transmissions, the data rate may be increased. Nevertheless, the achieved link distances were short, and the system’s complexities were moderate for CSK. Nevertheless, the polychromatic version of other modulations showed better results in terms of data rate and should be considered. On the other hand, undersampled techniques provided the fastest transmissions rates with moderate-to-high complexity and medium-range distances. Modulations based on space-time coding reached longer distances with relatively low data rate, but presenting very high complexity. Finally, in all the cases, the modulation schemes that compared consecutive frames reached the most extended distances without a significant impact on complexity. Figure 1 summarizes the analyzed modulations schemes in terms of complexity and distance range.

3. Theoretical Description of the Proposed Modulation

As it was aforementioned, there exist several modulation schemes developed or adapted for OCC. However, there are three main constraints for these modulation techniques: data rate, limited distance range, and algorithm complexity. Furthermore, OCC implementations should avoid the flickering problem since the lights may be used also for illumination, decoration or advertising purposes without disturbing people. Flickering-free modulations with low complexity algorithms only perform adequately for medium distance ranges since the number of bits that can be allocated in a single frame decreases with distance. In the case of long-distance modulation schemes [24,25,26,37,39,40], these techniques rely on high-complexity algorithms that require adaptive thresholding method and a calibration process for identifying the light state. Moreover, the most advanced schemes use several bits in each transmission header for assuring the calibration, decreasing the throughput [29,30]. In general terms, the modulation methods presented in the previous section work correctly for either short or long distances but not for both cases simultaneously. In this work, a flickering-free and distance-independent modulation based on frame comparison that overcome these constraints is proposed and evaluated. This scheme can be defined as a rolling shutter effect-based modulation that requires a reduced minimum number of light bands for extracting the information. Hence, it can work for short, medium, and long distances. As a brief description, the proposed technique selects different signals as symbols for the transmitted data (in this work, 2 bits per symbol were chosen). These signals use specific switching frequencies, as shown in Figure 2, based on the system’s characteristics to improve the link capabilities (flickering avoidance and operation range enhancement). Additionally, the system employs constant voltage as input signal of the emitter when it is not transmitting. In this was, the proposed modulation reduces the detection process’s complexity, since the 2D standard deviation is directly used during the frame synchronization, to detect a change from constant light to switching signal. Similarly, this scheme mitigates the problems related to the system’s sensitivity to background light by using other statistical function, Pearson’s Correlation Coefficient (PCC), for symbol detection, instead of determining the frame illumination level which usually requires thresholding techniques and an additional calibration process. At this point, the scheme is similar to a traditional Frequency Shift Keying (FSK) modulation, but the demodulation procedure is based on two main assumptions that ease the detection procedure and reduce the needed frequency variation on the transmitter side. On the one hand, thanks to the rolling shutter scanning mode of CMOS cameras, the apparent movement of a given frequency between two consecutive frames is determined by the modulo operation with respect to the frame acquisition rate of the camera. In the same way, the likeliness of consecutive frames, obtained from the PCC value, can be used for determining the represented symbol. In the next section, all the procedures will be in-depth described and explained.

3.1. Statistical Measures in Digital Image Processing

The working principles of the proposed system are based on some traditional statistical measures. Concretely, the 2D mean, the 2D standard deviation, and the correlation coefficient are the fundamental operations that conform the procedure. The 2D mean (${\mu }_A(x,y)$) is a basic statistical measurement and it represents the average value of the pixels intensity or the expected value of a pixel in a given region. In the same way, the 2D standard deviation (${\sigma }_A(x,y)$) is a variability measure or diversity of the pixels’ intensities. It shows how much dispersion exists in the image from the average value. A low standard deviation indicates that the pixels tend to be very similar. Finally, the correlation coefficient assesses the degree of linear correlation between two image matrices A and B. In this work, Pearson Correlation Coefficient (PCC) is employed (although other similarity measures could be used, extending this procedure), and it is calculated using Equation (1), which is bounded in the interval $[-1,1]$.

$$PCC(A,B)=\frac{{\sum }_x{\sum }_y(A_{xy}-\overline{A})(B_{xy}-\overline{B})}{\sqrt{\left({\sum }_x{\sum }_y{(A_{xy}-\overline{A})}^2\right)\left({\sum }_x{\sum }_y{(B_{xy}-\overline{B})}^2\right)}}$$

(1)

$\overline{A}$ and $\overline{B}$ are the expected values of the input matrices A and B, respectively. If the two images are equal, they are perfectly correlated and therefore the coefficient would be 1. In the case of similar images, as the ones shown in Figure 3, the value of the coefficient is close to one.

On the contrary, if the two images are exactly opposite, as Figure 4 illustrates, they are perfectly anti-correlated so the coefficient would be −1.

If there is no linear correlation between the images, the coefficient would be close to 0. For example, the images in Figure 5 are not similar nor opposite, therefore resulting in a coefficient close to zero. It must be noted that a low linear correlation does not mean necessarily low correlation in general terms. This could happen in nonlinear systems, which would require other types of correlation measures, but this work assumes that the signals are linearly distorted.

These tools are the building blocks of the proposed system, which consume relatively low computational resources. Hence, this reception process could be implemented in real-time applications suitable for IoT systems. For real-time operation, the PCC calculation should be carried out before the arrival of the next frame. In conventional cameras, this time ranges between $33.33\mathrm{ms}$ and $16.66\mathrm{ms}$, which is enough time for a simple PCC calculation that requires less than $10.5\mathrm{ms}$ using an interpreted programming language such as MATLAB. Nevertheless, this calculation time can be reduced significantly by using adequate software and hardware.

3.2. Theoretical Fundamentals

When a rolling shutter digital image sensor is employed, one scan-line of photosensors is exposed to the light simultaneously during a given time window (exposure time) and the output is immediately read-out. Therefore, the first line starts its exposure at $T_0$ and the next scan-line is shifted by a fixed period $T_1$, acquiring different light information. For this reason, if a light source is switching on and off at a frequency $f_{\mathrm{SW}}$ greater than the camera’s frame rate $f_{\mathrm{CAM}}$, the resulting image will include dark and light bands, as shown in Figure 6. The pixel’s intensity in those bands depends on the environmental conditions (background illumination), the emitter’s optical power, and the camera’s settings. Furthermore, the width of the light strips depends on the camera’s characteristics and the transmitter’s switching frequency.

It is important to deduce an equation for calculating the width of the light bands obtained due to the rolling shutter effect. At first, the light source’s location is determined and the corresponding Region of Interest (ROI) is extracted. Regardless of the emitters shape, the ROI is considered as a rectangle within the light source containing the maximum number of available rows. In this way, the resulting image only contains the light and dark lines eliminating the background and therefore limiting possible elements that could affect system performance. From the definition of the camera’s frame rate, it is deduced that each frame is acquired over the same time $T_{\mathrm{CAM}}$. Therefore, the total amount of exposed lines, pixels’ resolution in that direction ($N_y$), correspond to that time. In the same way, the switching signal period $T_{\mathrm{SW}}$ is related to the amount of pixels in a dark band and a light strip combined. Assuming that the switching signal has a $50\%$ duty cycle, the band’s pixel resolution, $N_{\mathrm{BAND}}$, is approximately the same for dark and light strips and it is related to half the time $T_{\mathrm{SW}}$, as shown in Equation (2).

$$\begin{array}{cc}\hfill \begin{array}{c}{N}_y\to T_{\mathrm{CAM}}\\ N_{\mathrm{BAND}}\to T_{\mathrm{SW}}/2\end{array}\}\Rightarrow N_{\mathrm{BAND}}& =\frac{N_y{T}_{\mathrm{SW}}/2}{T_{\mathrm{CAM}}}\hfill \end{array}$$

(2)

Since $T_{\mathrm{CAM}}=1/f_{\mathrm{CAM}}$ and $T_{\mathrm{SW}}=1/f_{\mathrm{SW}}$, the band’s pixel resolution can be calculated in function of the camera’s frame rate and the switching signal’s frequency, well-known values, using Equation (3).

$$\begin{array}{cc}\hfill N_{\mathrm{BAND}}& =\frac{N_y{f}_{\mathrm{CAM}}}{2f_{\mathrm{SW}}}\hfill \end{array}$$

(3)

Additionally, depending on the relation between the switching frequency and the camera’s frame rate, the relation between the lighter and darker bands on consecutive frames change. These changes will be used as the data codification scheme in the proposed flickering free, distance independent modulation. Since, the relation between the bands on consecutive frames can be calculated using the PCC, the switching frequency can be deduced from this value. There are three possible cases for the relation between the switching frequency and the camera’s frame rate:

• The signal frequency is a multiple of the camera’s frame rate.

  $$f_{\mathrm{SW}}=\alpha f_{\mathrm{CAM}}$$
• The signal frequency is perfectly uncoupled.

  $$f_{\mathrm{SW}}=(\alpha +0.5)f_{\mathrm{CAM}}$$
• The signal frequency is between the previous values.

  $$\alpha f_{\mathrm{CAM}}<f_{\mathrm{SW}}<(\alpha +0.5)f_{\mathrm{CAM}}$$

In the first case, the signal’s period will repeat exactly $\alpha$ times during the frame’s acquisition. Therefore, if the light is turned on at $T_i$ of the capturing process of frame m, the light will be also turned on at $T_i$ of frame $m+1$ acquisition. Furthermore, consecutive frames of the light source’s ROI will be almost equal. In other words, when $f_{\mathrm{SW}}=\alpha ,f_{\mathrm{CAM}}$ the consecutive frames are highly correlated and therefore $PCC\approx 1$.

In the second case, the signal’s period will repeat exactly $(\alpha +0.5)$ times during the frame’s acquisition. In other words, when the camera start the next frame’s capture, the signal will be shifted $50\%$ of its period. If the light is turned on at $T_i$ of the capturing process of frame m, the light will be turned off at $T_i$ of frame $m+1$ acquisition due to the $50\%$ duty cycle of this signal. Furthermore, consecutive frames of the light source will be close to inverted images. Therefore, when $f_{\mathrm{SW}}=(\alpha +0.5)f_{\mathrm{CAM}}$ the consecutive frames are almost anti-correlated and therefore $PCC\approx -1$.

In the third case, the signal’s period will repeat exactly $(\alpha +k)$ times during the frame’s acquisition. In other words, when the camera start the next frame’s capture, the signal will be shifted between $0\%$ and $50\%$ of its period, depending on the selected switching frequency. The resulting images of consecutive frames varies from highly correlated to almost anti-correlated in a quasi linear relation, and therefore $-1<PCC<1$.

The inspection of these consecutive frames gives information related to the frequency of the switching signal. However, the PCC is not the only statistical measure that provides information about the signal transmitted over the light. The 2D mean (${\mu }_{ROI}(x,y)$) and the 2D standard deviation (${\sigma }_{ROI}(x,y)$) also present interesting relations with the light source input signal.

The 2D mean is related to the average pixel’s intensity, therefore this value will be affected by the environmental conditions, the cameras exposition setup, and the light intensity. Two frames of switching lamps, captured by the same camera under similar environmental conditions will have alike intensity values, even if the frequency varies. Nevertheless, in the case of a continuous input signal, the 2D mean value will increase due to the elimination of the dark strips.

In the same way, since the 2D standard deviation (${\sigma }_{ROI}(x,y)$) represents the variability of the pixels intensity, its value do not change for the blinking images in which pixels diversity is similar (dark and light strips). However, the 2D standard deviation of the image where the light is continuously turned on is close to zero.

These two statistic measurements provide important information related to the light source’s input signal, specially by comparison. If a continuous signal is employed, ${\sigma }_{ROI}(x,y)$ will be close to zero while ${\mu }_{ROI}(x,y)$ will directly represent the lamp optical power. If a switching signal is used, ${\sigma }_{ROI}(x,y)$ will increase while ${\mu }_{ROI}(x,y)$ will decrease. Therefore, by comparing the 2D mean and standard deviation of two frames, a possible input signal change can be detected. Consequently, these elements can be used for implementing a simple wake-up process.

In summary, the proposed process employs two different input signals for the emitter: a continuous one when the system is waiting for bits to transmit; and a square signal during the emitting stage. The selection of the square signal switching frequency depends on the symbol to be represented and the system’s characteristics. Additionally, each symbol is transmitted for m frames. In the receiver side, the transmission start is detected based on the 2D standard deviation of the frame (${\sigma }_{ROI}(x,y)$), as seen in Figure 7. Then the corresponding symbols are obtained from the calculated average PCC. Each process will be described in detail in the upcoming sub-sections.

3.3. Wake up Process

As stated in the previous section, the 2D mean and 2D standard deviation can be employed for determining a change in the light input signal. Consequently, these values can detect the transmission start as the base of the system’s wake up procedure.

Since the emitters can start the transmission at any time, the camera is constantly acquiring frames. However, there must be a mark that triggers the data extraction, also known as the wake up signal. For this scheme, a change in the LED’s input signal was selected as the token. If the LED-based lamp is not emitting data, it will be constantly turned on. Therefore, ${\sigma }_{ROI}(x,y)$ will be close to zero and ${\mu }_{ROI}(x,y)$ will have its maximum value. When the transmission starts, the LED will vary its state from on to off continuously. As a result of this change, ${\sigma }_{ROI}(x,y)$ increases while ${\mu }_{ROI}(x,y)$ decreases, triggering the data extraction on the camera’s communication side. Similarly, when the transmission ends, the emitter receives once again a constant signal at its input. Therefore, ${\sigma }_{ROI}(x,y)$ returns to the minimum value and ${\mu }_{ROI}(x,y)$ increases, stopping the information’s extraction process.

As can be observed on the Rx side of Figure 7, the wake up process is based on a comparison of consecutive frames’ 2D mean, and analysis of actual frame’s 2D standard deviation. Since the procedure started assuming that the system is not transmitting data (State = waiting), the maximum possible value was assigned to the 2D mean as initial value, ${\mu }_{RO{I}_0}(x,y)=255$. In the case of the 2D standard deviation, a threshold value is needed and $3\%$ of the maximum value was selected (${\sigma }_{Th}(x,y)=6.3750$), based on experimental intermediate results.

When a frame ($F_N$) is captured, the ROI that contains the emitter’s projection is extracted for calculating the 2D mean (${\mu }_{RO{I}_N}(x,y)$) and standard deviation (${\sigma }_{RO{I}_N}(x,y)$). Then the obtained values are compared with the previous frame’s mean and the threshold standard deviation. Depending on the comparison’s outcome and the current system’s state, $F_N$’s ROI can be stored for data extraction or ignored. If the algorithm was waiting for a transmission, the standard deviation is greater than the threshold (${\sigma }_{RO{I}_N}(x,y)>{\sigma }_{Th}(x,y)$), and the mean decreases (${\mu }_{RO{I}_N}(x,y)<{\mu }_{RO{I}_{N-1}}(x,y)$), then a transmission has started. Therefore, the system’s state change to transmitting and the frame’s ROI is stored.

In the same way, if the algorithm is in the “emitting data” state, the frame’s ROI is stored except when the standard deviation is lower than the threshold (${\sigma }_{RO{I}_N}(x,y)<{\sigma }_{Th}(x,y)$), and the mean increases (${\mu }_{RO{I}_N}(x,y)>{\mu }_{RO{I}_{N-1}}(x,y)$). In this case, the transmission is assumed to be stopped, the system’s state changes to waiting and the frame is ignored. In

Table 1. Waking up examples from transmissions shown in Figure 6. The case (a) corresponds to gray-scale images of $640\times 480$ taken in a room with the light turned on. For the case (b), the camera acquired color images of $1280\times 720$ with default characteristics in a room where the light was turned off. In the last case (c), the camera’s settings were high contrast, low brightness, and low exposure time for $1280\times 720$ color images taken in a room with the light turned off.

State	${\mathit{\mu }}_{\mathit{ROI}\left(\mathit{a}\right)}$	${\mathit{\sigma }}_{\mathit{ROI}\left(\mathit{a}\right)}$	${\mathit{\mu }}_{\mathit{ROI}\left(\mathit{b}\right)}$	${\mathit{\sigma }}_{\mathit{ROI}\left(\mathit{b}\right)}$	${\mathit{\mu }}_{\mathit{ROI}\left(\mathit{c}\right)}$	${\mathit{\sigma }}_{\mathit{ROI}\left(\mathit{c}\right)}$
Waiting	223.87	4.22	252.52	4.26	168.95	4.66
Transmitting	208.39	18.76	231.43	27.28	120.74	55.93

, the obtained values from three experiments are shown. In all the examples, the camera was acquiring frames at a rate of $25\mathrm{fps}$. In the first case, the original frames $F_{\left(a\right)}$ were gray-scale images of $640\times 480$ that were taken in a room with the light turned on. The camera’s exposure time was low. For the second case, $F_{\left(b\right)}$ were color images of $1280\times 720$ taken by a camera with default characteristics in a room where the light was turned off. In the last case, the camera’s setting were high contrast, low brightness, and low exposure time for $1280\times 720$ color images $F_{\left(c\right)}$, taken in a room with the light turned off.

With this procedure, the transmission is detected. However, when the communication starts there are three options regarding the acquisition of the emitter’s projection (ROI):

• The data transmission start before the ROI scanning, Figure 8a.
• The data transmission start after the ROI scanning, Figure 8b.
• The data transmission start in the middle of the ROI scanning, Figure 8c.

For this particular explanation, illustrated in Figure 8, the number of consecutive frames used to represent each symbol was set to three ($m=3$), therefore $t_{symbol}=3T_{\mathrm{CAM}}$. In the same way, the first transmitted symbol corresponds to the perfectly correlated frequency ($f_{\mathrm{SW}}=\alpha f_{\mathrm{CAM}}$). In the first case (Figure 8a), the emission began previous to the emitter’s signal acquisition by the receiver, therefore the 2D standard deviation of the ROI in the first frame, $F_1$, is bigger than the defined threshold (${\sigma }_{F_1}>3\%$) and the transmission is detected on that frame. The ROIs extracted from $F_1$, $F_2$, and $F_3$ are stored for demodulation. The data can be obtained without problem from those images, the calculated average PCC of the three stored ROI is equal to one ($PCC(F_1,F_2)=PCC(F_2,F_3)=1\Rightarrow \overline{PCC}=1$), so the symbol is perfectly identified.

In the second case (Figure 8b), the emission started after the emitter’s acquisition, therefore the 2D standard deviation of the ROI from frame $F_1$ is lower than the defined threshold (${\sigma }_{F_1}<3\%$) and the transmission is not detected on that frame. However, the calculated 2D standard deviation of frame $F_2$ is bigger than the threshold (${\sigma }_{F_2}>3\%$) and the transmission is detected on the second frame. Since each symbol is emitted for three times the frames’ acquisition period ($3T_{\mathrm{CAM}}$), the ROIs extracted from $F_2$, $F_3$ and $F_4$ are stored for demodulation. The data can be obtained without problem from those images. The calculated average PCC of those three stored ROIs is equal to one ($PCC(F_2,F_3)=PCC(F_3,F_4)=1\Rightarrow \overline{PCC}=1$), so the symbol is perfectly identified.

In these two scenarios, the information is successfully extracted from all the stored frames. Nevertheless, the third case (Figure 8c) is different. Since the transmission starts during the acquisition process, one part of the ROI from frame F1 has a wide bright strip. This band corresponds to the time previous to the data transmission, when the emitter is continuously turned on. Depending on the remaining acquisition time, there are two possibilities: the ROI has dark strips; or the ROI has only bright rows. In the first scenario, due to the presence of darker strips, the 2D standard deviation of the ROI is bigger than the threshold (${\sigma }_{F_1}>3\%$) and the transmission is detected on this frame. The ROI extracted from $F_1$, $F_2$ and $F_3$ should be stored for demodulation. Nevertheless, the data can not be appropriately obtained from those images. As seen in Figure 8c, the calculated PCC of the two first stored ROIs is positive and less than one ($0<PCC(F_1,F_2)<1$), so the symbol is wrongly identified, while the PCC of the remaining stored ROI is equal to one ($PCC(F_2,F_3)=1$) and the symbol is identified. The second scenario is similar to the case where the emission started after the emitter’s acquisition. All the rows are bright pixels so the 2D standard deviation of the ROI is lower than the threshold (${\sigma }_{F_1}<3\%$) and the transmission is not detected on $F_1$. The ROIs collection is triggered on frame $F_2$, so the ROI extracted from $F_2$, $F_3$, and $F_4$ should be stored for demodulation. Unlike the previous scenario, the demodulation from frames $F_2$ and $F_3$ is correct ($PCC(F_2,F_3)=1$), but the accuracy of the symbol extracted from frames $F_3$ and $F_4$ can not be assured ($-1<PCC(F_3,F_4)<1$), due to the presence of the next transmitted symbol. Therefore, when the transmission starts during the acquisition process, an additional operation is required.

In order to avoid the demodulation uncertainty that a transmission starting during the acquisition process may produce, all the emissions begin with a known symbol. For this first symbol, the decodification is done independently from the possible frame’s combination. If the decoded symbol from the first two ROIs, based on $PCC(F_1,F_2)$, is different to the defined one, the emission started during the acquisition process, as shown in Figure 8c, generating the first scenario where the ROI has dark strips. In this case, the first ROI of each frame group is ignored during the average PCC calculation ($\overline{PCC}=average(PCC(F_2,F_3),\dots ,PCC(F_{m-1},F_m))$). However, if the decoded symbol from the last two ROIs, based on $PCC(F_{m-1},F_m)$, is different to the defined one, the emission started during the acquisition process without generating black bands (second scenario). In this case, the last frame’s ROI of each frame group is ignored, and the data extraction is applied to the remaining frames ($\overline{PCC}=average(PCC(F_1,F_2),\dots ,PCC(F_{m-2},F_{m-1}))$).

As it was shown above, the number of successive frames (m) used for transmitting each symbol can improve the detection results in the reception stage. By employing the average PCC from m consecutive frames, the impact of noise on the PCC calculation is mitigated. However, the use of several consecutive frames for transmitting one symbol generates throughput degradation. Therefore, three consecutive frames have been defined as a trade off between reception performance and system’s throughput. Still, a higher number of frames used for the transmission of each symbol can improve the success probability. To prove this, 3 to 5 frames were used to determine the system’s improvement during the experimental evaluation.

After explaining the wake up procedure that determines whether the transmission started, and selecting which frames should be analyzed for data extraction, the modulation and demodulation techniques will be introduced.

4. Modulation/Demodulation Scheme

As was detailed in Section 3.2, depending on the switching signal’s frequency value there is a specific correlation between consecutive frames, with three main cases:

• If the frequency is a multiple of the camera’s frame rate ($f_{\mathrm{SW}}=\alpha f_{\mathrm{CAM}}$), the consecutive frames are highly correlated. Therefore, as stated in Section 3.1, the PCC is close to one.
• If the frequency is perfectly uncoupled ($f_{\mathrm{SW}}=(\alpha +0.5)f_{\mathrm{CAM}}$), the consecutive frames are almost anti-correlated, therefore the PCC is almost minus one.
• For the frequencies between these two cases ($\alpha f_{\mathrm{CAM}}<f_{\mathrm{SW}}<(\alpha +0.5)f_{\mathrm{CAM}}$), the resulting images of consecutive frames vary from highly correlated to almost anti-correlated in a quasi linear relation, in other words, the PCC is between minus one and one.

These correlation situations are the flickering free distance independent of modulation’s basis. During the data transmission state, the emitter is switched on and off by applying a square signal with $50\%$ duty cycle. This signal’s frequency varies depending on the corresponding data symbol. Since each frequency generates specific correlation between consecutive frames, the symbol can be identified by calculating the PCC value. The PCC represents the linear correlation between two images using Equation (1). These values vary from −1 (opposite images) to 1 (equal images). The correct frequency selection assures four possible outcomes from the consecutive frames’ comparison and therefore a 2 bits per symbol codification:

• Consecutive frames highly correlated ($PCC>0.7$), when the frequency is a multiple of the camera’s frame rate ($f_{\mathrm{SW}}=\alpha f_{\mathrm{CAM}}$).
• Consecutive frames highly anti-correlated ($PCC<-0.7$), when the frequency is perfectly uncoupled ($f_{\mathrm{SW}}=(\alpha +0.5)f_{\mathrm{CAM}}$).
• Consecutive frames moderately correlated ($0<PCC<0.65$), when the frequency is greater than but close to a multiple of the camera’s frame rate.
• Consecutive frames moderately anti-correlated ($-0.65<PCC<0$), when the frequency is lower than but close to the uncoupled frequency.

Each two-bits symbol is emitted for a multiple of the frame’s time ($t_{symbol}=m{T}_{\mathrm{CAM}}$). For assuring the data extraction accuracy even when the transmission starts during the emitter’s acquisition process, the emission should last at least three frames ($m\ge 3$).

When the system is in the “Transmitting” state, from each frame the rectangular ROI containing only the light and dark bands from the emitter is obtained and stored. During the data extraction process, as shown in Figure 7, these frames’ ROI is normalized using the “max-min” technique. In this way, the pixel values of consecutive frames are in the same range (from 0 to 1). Therefore, any possible effect of an environmental light change is mitigated. Then the PCC is calculated from the adequate consecutive frames’ ROI.

As stated in the previous section, if the transmission started during the acquisition of the emitter’s pixel rows, the frame’s ROI, which contains the transition between symbols, is eliminated from the frame’s group. In this case, the PCC calculation is done $m-2$ times. Otherwise, when the emission began before or after the emitter acquisition, this process is done $m-1$ times ($PCC(F_1,F_2),\dots ,PCC(F_{m-1},F_m)$). Then the average PCC ($\overline{PCC}$) of the frame’s group is determined from those PCC values. Finally, based on that $\overline{PCC}$ value, the data are decoded using Equation (4).

$$DATA=\left\{\begin{array}{cc}10\hfill & \overline{PCC}>0.7\hfill \\ 11\hfill & 0<\overline{PCC}<0.65\hfill \\ 01\hfill & -0.65<\overline{PCC}<0\hfill \\ 00\hfill & \overline{PCC}<-0.7\hfill \end{array}\right.$$

(4)

Since each PCC value is obtained from the consecutive frames’ comparison, there is no need of a thresholding techniques for identifying the light state. Moreover, the pixel’s intensity value by itself is irrelevant so an initial calibration is not required. Additionally, these correlation’s values can be evaluated with an image’s ROI containing two strips or more. Therefore, the theoretical distance range of the system depends on the emitter’s projected size and the band’s pixel size. These dimensions can be easily calculated using Equations (3), (5) and (6). Then the obtained values can be used for the frequency selection in the system design process.

$$x=2\frac{N_x}{FO{V}_x}arctan\left(\frac{d_x}{2D}\right)$$

(5)

$$y=2\frac{N_y}{FO{V}_y}arctan\left(\frac{d_y}{2D}\right)$$

(6)

These equations estimate the 2D pixel projection (x, y) in a picture of a real distance ($d_x$, $d_y$) in function of the separation between the object and the camera (D), and the camera’s FOV and resolution in pixels ($N_{x,y}$), as shown in Figure 9.

Therefore, equations for determining the adequate frequencies for each implementation must be deduced. These equations are based on the $\alpha$ parameter which relates $f_{\mathrm{SW}}$ and $f_{\mathrm{CAM}}$, called the frequency multiplier. This parameter has specific requirements related to the camera’s characteristics, the emitter’s size, and the distance range of the system. In the next subsection, the selection of this value will be explained.

4.1. Frequency Multiplier’s Selection

As was shown before, for calculating the adequate symbols’ codification frequencies, the frequency multiplier ($\alpha$) must be determined. The ($\alpha$) value depends on several system characteristics: the distance range, the selected camera’s vertical resolution, the vertical FOV, and the selected emitter’s dimensions. This frequency multiplier is an integer parameter that multiplies the camera’s frame rate ($f_{\mathrm{CAM}}$) to obtain the coupled frequency, defined as $f_{\mathrm{SW}}=\alpha f_{\mathrm{CAM}}$. Using this relation, Equation (3) can be modified to calculate the band’s size in terms of the camera’s resolution and the frequency multiplier, as shown in Equation (7).

$$\begin{array}{cc}\hfill N_{\mathrm{BAND}}& =\frac{N_y{f}_{\mathrm{CAM}}}{2\alpha f_{\mathrm{CAM}}}\hfill \\ \hfill N_{\mathrm{BAND}}& =\frac{N_y}{2\alpha }\hfill \end{array}$$

(7)

As has been stated previously, the PCC requires that each image contain at least two strips. Therefore, the projected vertical size of the light source, $y_{\mathrm{Source}}$, must be greater than the double of the strip’s vertical size. Additionally, in order to locate the transmitter in the frame, recover the corresponding ROI, and adequately process the cropped image, the band’s vertical size should be greater than five pixels. The theoretical value is three pixels for assuring the four possible PCC values. However, the five pixel requirement was determined from the execution of several experiments related with the data extraction process for OCC. Therefore, the boundaries of the number of light bands are expressed as:

$$\begin{array}{cc}\hfill y_{\mathrm{Source}}>\upsilon N_{\mathrm{BAND}}/N_{\mathrm{BAND}}\ge 5,\forall \upsilon \ge 2& \hfill \\ \hfill \frac{2}{{\varphi }_y}\left[arctan\left(\frac{d_y}{2D}\right)\right]>\upsilon \left(\frac{N_Y}{2\alpha }\right)>5\upsilon & \\ \hfill \frac{N_y}{10}>\alpha >\frac{{\varphi }_y{N}_y}{2arctan\left(\frac{d_y}{2D}\right)}\frac{\upsilon }{2}& \hfill \end{array}$$

(8)

Then, the frequency multiplier $\alpha$ can be obtained based on the camera’s characteristics (${\varphi }_y$, $N_y$, ${\mathrm{FoV}}_y$), the distance between the emitter and the receiver (D), and the light source’s size ($d_y$) using Equation (9)

$$\begin{array}{cc}\hfill \frac{N_y}{10}& >\alpha >\frac{\upsilon {\mathrm{FoV}}_y}{4\left[arctan\left(\frac{d_y}{2D}\right)\right]}\hfill \end{array}$$

(9)

As previously stated, this frequency multiplier is the base for selecting the switching frequencies under specific conditions. The selection process is presented in the following subsection.

4.2. Switching Frequencies’ Selection

The frequency multiplier, $\alpha$, directly provides two frequencies for the system: coupled frequency ($f_1=\alpha f_{\mathrm{CAM}}$) and uncoupled frequency ($f_2=(\alpha +0.5)f_{\mathrm{CAM}}$). The next step is to determine the other two frequencies needed for the two-bits symbols generation.

Based on the quasi linear relation between the frequencies in the range from $\alpha f_{\mathrm{CAM}}$ to $(\alpha +0.5)f_{\mathrm{CAM}}$ and the consecutive frames’ ROI correlation coefficient, a fixed step between the frequencies is assumed. This relation is depicted in Equation (10) and can be observed in Figure 10.

$$\begin{array}{cc}\hfill \mathrm{step}& =\frac{(\alpha +0.5)f_{\mathrm{CAM}}-\alpha f_{\mathrm{CAM}}}{3}\hfill \\ \hfill & =\frac{\alpha f_{\mathrm{CAM}}+0.5f_{\mathrm{CAM}}-\alpha f_{\mathrm{CAM}}}{3}\hfill \\ \hfill & =\frac{f_{\mathrm{CAM}}}{6}\hfill \end{array}$$

(10)

The remaining frequencies are calculated using Equation (10) as follows:

$$\begin{array}{cc}\hfill f_3& =f_1+\mathrm{step}\hfill \\ \hfill & =\alpha f_{\mathrm{CAM}}+\frac{f_{\mathrm{CAM}}}{6}\hfill \\ \hfill & =\left(\alpha +\frac{1}{6}\right)f_{\mathrm{CAM}}\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill f_4& =f_1+2\mathrm{step}\hfill \\ \hfill & =\alpha f_{\mathrm{CAM}}+\frac{2f_{\mathrm{CAM}}}{6}\hfill \\ \hfill & =\left(\alpha +\frac{1}{3}\right)f_{\mathrm{CAM}}\hfill \end{array}$$

(12)

Therefore, the switching signal frequency can be determined using Equation (13)

$$f_{\mathrm{SW}}=\left\{\begin{array}{cc}\alpha f_{\mathrm{CAM}}\hfill & \mathrm{DATA}=10\hfill \\ (\alpha +1/6)f_{\mathrm{CAM}}\hfill & \mathrm{DATA}=11\hfill \\ (\alpha +1/3)f_{\mathrm{CAM}}\hfill & \mathrm{DATA}=01\hfill \\ (\alpha +1/2)f_{\mathrm{CAM}}\hfill & \mathrm{DATA}=00\hfill \end{array}\right.$$

(13)

4.3. Modulation’s Advantages

The flickering free, distance independent modulation scheme proposed in this work overcomes two of the main constrains for OCC modulation techniques: limited distance range and high algorithm’s complexity. Since at least three consecutive frames are required for transmitting two bits symbols, the data rate is limited to $2f_{\mathrm{CAM}}/3$. These data rate is greater than the value obtained by other similar modulations which reach transmissions of $f_{\mathrm{CAM}}/2$. However, a $2f_{\mathrm{CAM}}/3$ transmission is considered slow but usable for IoT applications. Therefore, the data rate issue is partially addressed.

Nevertheless, the proposed procedure increases the regular distance range of the modulations schemes, and is intended for its implementation for short, medium, and long links. This modulation is defined as a rolling shutter effect-based modulation. The distance constraint for this particular type of modulation is that the number of recovered bands decreases with the distance. However, for this technique, the data extraction requires a minimum number of light bands. Therefore, it can perform without problems even for long distances where only two bands are recovered.

Additionally, the modulation is based on signals changing the switching frequency for different symbols (similar to a frequency shift keying modulation, FSK). The demodulation process relies on the simple calculation of the PCC value from consecutive frames, which requires a low complexity and computational cost processing. The use of the PCC value also eliminated the necessity of an adaptive thresholding algorithm for obtaining the correct light and black bands from the transmitted signals. Similarly, the pixel’s intensity value is irrelevant due to the frame normalization performed at the beginning of the demodulation process. Therefore, an initial calibration is not required, neither is the system affected by changes in the environmental conditions. On the other hand, the data are coded using square signals with different frequencies. Those frequencies are high enough to assure no flickering, and the square shape maintains the duty cycle constant, which reduces even more the possible flickering effect.

Due to these advantages, the proposed modulation scheme can be easily implemented for a wide range of OCC applications independently of the distance between the emitters and the camera. Since the complexity is minor, non-specialized hardware can be used for the deployment of the OCC systems without impacting its performance. Therefore, this modulation scheme is suitable for IoT applications in urban environments where the distances varies from less than one meter to several hectometers.

The previous system explanation does not consider additional channel effects because they have a low impact on the system performance, or they can be compensated with further processing independently to the reception stages. For example, cross-talk interference between RGB emissions can be reduced by channel compensation techniques. Multi-path phenomena have a negligible effect in this work due to the small used bandwidth compared to the channel capacity. Source motion aspects are not considered in this paper. Still, they can be addressed by implementing light source discovery and tracking processes to maintain the emitter source into the camera’s field of view. Finally, the blurring effect does not change the bands on the image since they depend only on the time variation of the data signal nor on the shape of the picture.

For observing the performance of the proposed systems, simulations and experimental implementations have been prepared. The main results are presented in the next sections.

5. Methodology

To prove that the proposed modulation works properly for short, medium, and long distance ranges, a two phases procedure was used. During the first stage, a simulation was designed for validating the switching frequencies’ selection procedure by using Equations (9) and (13). For this purpose, the modulated signals using the proposed scheme were generated using the switching frequencies obtained by applying Equation (13) for different values of the frequency multiplier $\alpha$. Then these signals were used as the input of the light emitters and in the reception stage the corresponding PCC values of the acquired consecutive frames were calculated to demonstrate the effectiveness of the calculated frequency multiplier $\alpha$, the resulting frames from the cameras should have at least $\upsilon =2$ light bands, and each light strip should be greater than five pixels wide. Additionally, for validating the viability of the switching frequencies selection using Equation (9), the obtained PCC for each data symbol must fit within the range defined in Equation (4).

The second stage was designed for calculating the system’s BER, the succeed probability and the transmission failure probability of the system. The experimental implementation of the proposed scheme was done using switching frequencies for three different distance ranges in order to validate the independence of the scheme from the distance. The experimental testing was designed to send data using the whole modulation scheme including the wake-up process. Over all the transmissions, the received data were compared to the sent information to obtain the number of symbols appropriately decoded, and the amount of bits that the system failed to recover. In this way, the BER was calculated along with the probability of transmitting each symbol successfully.

5.1. Simulation Setup

In this case, the frame acquisition process was simulated for a camera with diagonal Field of View FOV$={67}^{\circ }$. The camera’s frame rate was set to 30 fps. To emphasize the light bands, a high contrast (255), low brightness (zero), and short exposure time ($1/1000$ s) were assumed. For speeding up the simulations, a basic image resolution of $640\times 480$ pixels was selected. The emitter was a $10.00$ cm $\times 10.00$ cm white LED-based lamp with square shape. The distance between the camera and the emitter was set to $0.25$ m for assuring that the whole frame represents the light source while the environmental conditions can be neglected, therefore the greatest range of $\alpha$ values can be evaluated.

From Equation (9), the maximum value of the frequency multiplier $\alpha$ for the selected resolution ($N_y=640$) is 64 ($\alpha <N_y/10$). On the other hand, since the whole frame represents the light source and the minimum value of $\alpha$ is one ($f_{\mathrm{SW}}=f_{\mathrm{CAM}}$), the frame has at least two bands for all the cases in the simulation. When $f_{\mathrm{SW}}=f_{\mathrm{CAM}}$ the signal period corresponds to the frame’s acquisition time ($t_{\mathrm{SW}}=t_{\mathrm{CAM}}$), therefore the frame has a complete light band and a complete dark strip. Taking this fact into account, for the simulation, the frequency multiplier was set from 10 to 70. For all the considered cases, the number of extracted light strips should be greater than two. However, the number of projected pixels for each band at $\alpha =70$ is less than five, the amount of pixels that assures the data extraction for long distances.

Using this range of $\alpha$, the coupled switching signal’s frequency varies from 300 Hz to 2100 Hz. For all the cases, it is expected that the PCC of consecutive frames for the different calculated switching frequencies are distinguishable. The PCC values should not overlap, allowing the correct demodulation of the four symbols. A total of 7500 frames (corresponding to the transmission of 5000 bits represented by 2500 symbols) for each value of $\alpha$ were generated.

5.2. Experimental Setup

As stated previously, a field test is required for validating the proposed modulation scheme under realistic conditions. All the experiments were held during sunlight days with similar environmental conditions: temperature below 30 °C, low wind-speed, without the presence of haze or rain. However, as explained in the system advantages, the background illumination effect is eliminated due to the normalization and the calculation of the average PCC.

For this experiment, the emitter was a $60\times 60$ cm² commercial LED-based square panel with effective vertical distance of $d_y=0.595$. During transmission, this lamp was activated by a squared signal with duty cycle of $50\%$. However, when the panel was on waiting state, the device transmits a continuous signal. Since, the signal normalization processes allow the receiver to work properly from a minimum emitted power level, the experimental setup established this transmission level in the lamp (a 10 V amplitude driving signal in this work was experimentally determined).

On the receiver side, a Raspi Camera Module V2 was selected. This camera was mounted on a Raspberry Pi B for managing its characteristics and controlling the video acquisition. During the experiments, the resolution was set to $1920\times 1080$, while the frame rate was left as default (25 fps). For this combination, the vertical FOV is partial with a value of 34.65° (0.6048 radians) instead of 48.8°, resulting in a camera’s FOV and resolution ratio of ${\varphi }_y=5.5996·{10}^{-4}$. Additionally, the camera has high contrast, low brightness, focus to infinity, and automatic white balance. The camera was activated for recording videos of three minutes duration before the emitter started the data transmission. For each video, the resulting 4500 frames were stored in the Raspberry Pi memory card for off-line processing. For these experiments, three values of consecutive frames for representing each symbol were used ($m=3$, $m=4$, and $m=5$). Therefore, the 4500 frames transmitted 3000 (1500 symbols), 2250 (1125 symbols), and 1800 bits (900 symbols) in each case.

Since the proposed modulation scheme requires at least two projected bands, for all the calculations $\upsilon$ was assumed to be 2 for the worst case scenario. Additionally, the frequency multiplier was determined by applying Equation (9), resulting in a maximum value of 108 for the applied resolution $N_y=1080$. However, the minimum frequency multiplier depends on the projected source size and therefore on the distance D between the emitter and the receiver. Then, this experiment was divided into three trial sets:

• Short range: D < 20 m.
• Medium range: 20 < D < 60 m.
• Long range: D > 60 m.

In all the trials, the distance between the emitter and the receiver changes. This separation started at 5 m and the receiver was moved lineally 5 m at the time. Additionally, each symbol was repeated for three, four and five consecutive frames.

For the short range case, the target distance was $D=15$ m. For this separation, the selected light source ($d_y=0.595$) is represented by approximately 72 vertical pixels (calculated using Equation (6)). Therefore, the minimum frequency multiplier is calculated as follows:

$$\begin{array}{cc}\hfill \alpha & >\frac{\upsilon {\mathrm{FoV}}_y}{4\left[arctan\left(\frac{d_y}{2D}\right)\right]}\hfill \\ \hfill \alpha & >\frac{2\ast 0.6048}{4\left[arctan\left(\frac{0.595}{2\ast 15}\right)\right]}\hfill \\ \hfill \alpha & >14.99\hfill \end{array}$$

This value was selected for the experimental implementation. The system’s frequencies were determined with Equation (13), while by applying Equation (7) a strip’s size of 36 pixels was calculated. The calculated frequencies are shown in Equation (14). For experimental reasons, the frequencies were set to their approximations.

$$f_{\mathrm{SW}}=\left\{\begin{array}{cc}375.00\mathrm{Hz}\approx 375.0\mathrm{Hz}\hfill & \mathrm{DATA}=10\hfill \\ 379.17\mathrm{Hz}\approx 379.0\mathrm{Hz}\hfill & \mathrm{DATA}=11\hfill \\ 383.33\mathrm{Hz}\approx 383.5\mathrm{Hz}\hfill & \mathrm{DATA}=01\hfill \\ 387.50\mathrm{Hz}\approx 387.5\mathrm{Hz}\hfill & \mathrm{DATA}=00\hfill \end{array}\right.$$

(14)

In the medium range case, the target distance was $D=30$ m. For this separation, the selected emitter ($d_y=0.595$) has at least 36 vertical pixels (calculated using Equation (6)). Then, the minimum frequency multiplier is calculated as follows:

$$\begin{array}{cc}\hfill \alpha & >\frac{2\ast 0.6048}{4\left[arctan\left(\frac{0.595}{2\ast 30}\right)\right]}\hfill \\ \hfill \alpha & >29.98\hfill \end{array}$$

This value was selected for the experimental implementation. Based on the distance and the frequency multiplier, the system’s frequencies were determined using Equation (13). In the same way, by applying Equation (7), a strips’ vertical width of 18 pixels was calculated. The calculated frequencies were:

$$f_{\mathrm{SW}}=\left\{\begin{array}{cc}750.00\mathrm{Hz}\approx 750.0\mathrm{Hz}\hfill & \mathrm{DATA}=10\hfill \\ 754.17\mathrm{Hz}\approx 754.0\mathrm{Hz}\hfill & \mathrm{DATA}=11\hfill \\ 758.33\mathrm{Hz}\approx 758.5\mathrm{Hz}\hfill & \mathrm{DATA}=01\hfill \\ 762.50\mathrm{Hz}\approx 762.5\mathrm{Hz}\hfill & \mathrm{DATA}=00\hfill \end{array}\right.$$

(15)

Finally, for the long range case, a distance $D=70$ m was selected and the lamp ($d_y=0.595$) in the frames has 15 vertical pixels (calculated using Equation (6)). Therefore, the minimum frequency multiplier is calculated as follows:

$$\begin{array}{cc}\hfill \alpha & >\frac{2\ast 0.6048}{4\left[arctan\left(\frac{0.595}{2\ast 70}\right)\right]}\hfill \\ \hfill \alpha & >71.14\hfill \end{array}$$

The value of $\alpha =75$ was selected for the frequency multiplier. Using Equation (13) the system’s frequencies were determined, while by applying Equation (7) a strip’s size of $7.2$ pixels was calculated. The frequencies were:

$$f_{\mathrm{SW}}=\left\{\begin{array}{cc}1875.00\mathrm{Hz}\approx 1875.0\mathrm{Hz}\hfill & \mathrm{DATA}=10\hfill \\ 1879.17\mathrm{Hz}\approx 1879.0\mathrm{Hz}\hfill & \mathrm{DATA}=11\hfill \\ 1883.33\mathrm{Hz}\approx 1883.5\mathrm{Hz}\hfill & \mathrm{DATA}=01\hfill \\ 1887.50\mathrm{Hz}\approx 1887.5\mathrm{Hz}\hfill & \mathrm{DATA}=00\hfill \end{array}\right.$$

(16)

At each location, before starting the transmission, a photo of the lamp turned on was taken. These frames were used to locate the light source for each distance D. For this purpose, at first the image was manually masked around the lamp location for eliminating the great part of the background. Then the resulting frame was binarized using Otsu’s thresholding technique. Therefore, the binary image was segmented to obtain the lamp’s centroid. By applying Equations (5) and (6), the pixel distances of the panel were calculated. With these data, the exact ROI containing only the emitter was obtained and used for cropping the video frames.

Afterwards the videos were processed using the cropped frames. At first, the 2D mean and standard deviation of the image were computed for performing the wake up process described in Section 3.3. In this particular case, the symbol “10” was selected as the header of the transmissions, for determining if the transmission started before, during, or after the acquisition of the light source pixels. Once the system changed its status to transmitting, the images were grouped together depending on the number of consecutive frames that represented each symbol.

Finally, each group of m frames were put through the data extraction procedure described in Figure 2. At first a min-max normalization was applied for assuring the same pixel’s intensity range in all the frames. Then the PCC was calculated for the corresponding consecutive frames. For example, in the case of $m=3$ and a synchronized signal, two PCC values would be computed: $PCC(F_n,F_{n+1})$ and $PCC(F_{n+1},F_{n+2})$. For the demodulation process, the average value of PCC was used. The obtained symbols were stored gradually in a vector.

When the system detected the transmission’s end employing the wake up process, the stored data vector was compared to the transmitted data. From this comparison, the BER of the communication was directly extracted. Additionally, the successful transmission probability was calculated based on two elements: frame error rate and symbol error rate. The first part took into account the relation between the frames that have not been correctly received and processed, and the total transmitted frames. The second part is defined as the probability of receiving the symbol A given that another symbol was transmitted. Nevertheless, it is possible to obtain a correct symbol even when some frames are not adequately received or processed. In this cases, an error for the success rate calculation is assumed.

6. Results

In this section, the results obtained from the simulation and experimental setups described in Section 5.1 and Section 5.2 are presented.

6.1. Simulation’s Results

As a result of the simulation, 52,500 frames were obtained. Figure 11 illustrates two of the obtained binary images for $\alpha =10;F_{\mathrm{SW}}=300$ Hz and $\alpha =50;F_{\mathrm{SW}}=1500$ Hz, where we must highlight two things: there are at least 19 strip lines, and each band has at least 7 pixels. In fact, since the minimum $\alpha$ value was set to 10, in all the frames at least 18 complete light bands were projected on the image. In the case of the bands’ pixel width, with $\alpha =70$, the frames showed values between four and five pixels, while lower frequency multipliers ($\alpha <64$) produced frames with bands of at least five pixels that can be easily extracted from longer distances. With this result, the proposed limits for the frequency multiplier have been validated.

On the other hand, as can be seen in Figure 12, for all the selected values of the frequency multiplier $\alpha$, the obtained PCC values of the consecutive frames for the selected frequencies corresponding to the data symbols were within the proposed ranges, without overlapping. In this way, the signals that correspond to the different symbols can be separated adequately. These results validate the calculated switching frequencies and therefore Equation (13). Nevertheless, this simulation was done without external interference, and for a short distance. For validating the modulation scheme, a field experimentation is needed. In this way, the impact that the environmental conditions, including external interference such as other illumination sources, the camera’s frame rate fluctuations and changes in the distance could have over the modulation scheme can be tested.

As an example of the system, a sample of the practical ROIs extracted from consecutive frames (obtained from the experimental videos) which represented the four possible transmitted symbols are presented in Figure 13. As can be observed, for all the switching frequencies the calculated PCC confirmed the proposed value ranges previously validated with the simulations. For example, in the case of the symbol “10” ($f_{\mathrm{SW}}=\alpha f_{\mathrm{CAM}}=375$ HZ), the images are almost equal and the calculated PCC is greater than 0.7 ($PCC=0.994>0.7$). In the same way, for the symbol “11” ($f_{\mathrm{SW}}=(\alpha +1/6)f_{\mathrm{CAM}}=379$ HZ), the light bands are slighted delayed and the calculated PCC is within the proposed range ($0<PCC=0.535<0.6$). Similarly, from the frames that represent the symbol “01” ($f_{\mathrm{SW}}=(\alpha +1/3)f_{\mathrm{CAM}}=383.5$ HZ), the light bands are significantly delayed and therefore calculated PCC is within the proposed range ($-0.6<PCC=-0.219<0$). Finally, in the case of the symbol “00” ($f_{\mathrm{SW}}=(\alpha +1/2)f_{\mathrm{CAM}}=387.5$ HZ), the obtained images are inverted and the calculated PCC is less than −0.7 ($PCC=-0.992<-0.7$).

6.2. Experimental Results

In this subsection, the results for the three different proposed cases are presented, for all the cases 2500 2-bit symbols were transmitted:

• Short range: D $=15$ m, $\alpha =15$.
• Medium range: D $=30$ m, $\alpha =30$.
• Long range: D $=70$ m, $\alpha =75$.

In this work, we have focused on the effects of the camera system (frame rate variations, lens distortion) and the PCC-based demodulation scheme on the detection process. In this way, we have excluded the noise level as a parameter in the experiments. Obviously, the noise level influences the system performance, particularly it introduces variations in the PCC obtained values. For this reason, decision regions for the symbols estimation have been established instead of specific values. Additionally, the use of several frames in the PCC value estimation has improved the obtained average value and reduced the noise effects. The main aim of the experimental setup is to analyze the characteristics of the proposed modulation without other external effects involved. However, this low noise setup is not ideal or theoretical at all, since the emitter and the receiver are already affected by thermal noise. Anyways, the noise influence over the system can be easily achieved in practice using an appropriate transmission power, camera gain, or optical zoom for example.

Results from the first experimental case ($\alpha =15$ and target distance D $=15$ m) are depicted in Figure 14. The link presents a BER of $5·{10}^{-4}$ when 3 consecutive frames are used in the PCC calculation algorithm. The same happened for a 20 m link, where the lamp’s image in the camera was represented by 53 vertical pixels (1.5 bands). However, at 25 m where the lamp was represented by 43 vertical pixels (1.2 bands) the BER increases to $2·{10}^{-2}$ for 3 consecutive frames, but stays at $9·{10}^{-4}$ for 4 consecutive frames. Additionally, in all the cases, the succeed probabilities are above $99.8\%$ and the BER values are below the FEC limit ($3.8·{10}^{-3}$) for distances up to 20 m.

Figure 15 shows the results corresponding to the $\alpha =30$ and D $=30$ m case. In this case, a BER of $1.2·{10}^{-3}$ for the 3 consecutive frames based on PCC processing and $1.1·{10}^{-3}$ for 4 consecutive frames were obtained. Then at 35 m and 40 m, where each frame captured 1.5 light bands, the link increased its BER to $1.3·{10}^{-3}$ and $1.8·{10}^{-3}$ correspondingly for the PCC estimation with 3 consecutive frames. However, in the case of 4 consecutive frames, the BER value did not change. While the BER was less than $1·{10}^{-4}$ for 5 consecutive frames. Moreover, at 45 m where the lamp was represented by 24 vertical pixels (1.3 bands) the BER reaches $4·{10}^{-2}$ for 3 consecutive frames, but $2.9·{10}^{-3}$ for 4 consecutive frames, and $1.5·{10}^{-3}$ for 5 consecutive frames. Additionally, in all the cases, the succeed probabilities are above $99.6\%$ and the BER values are below the FEC limit for distances up to 40 m.

Finally, as shown in Figure 16, for the $\alpha =75$ and target distance D $=70$ m case, an up to 90 m link was successful with BER of $6·{10}^{-4}$ for 3 and 4 consecutive frames’ PCC estimation. While at 95 m, where the lamp was represented by 11 vertical pixels (1.5 bands), the link has a BER of $7·{10}^{-4}$ for 3 and 4 consecutive frames. Moreover, at 100 m, the distance where each frame captured 1.4 light bands, the BER increases to $2·{10}^{-3}$ for 3 consecutive frames, and up to $7·{10}^{-4}$ for 4 consecutive frames. Additionally, in all the cases, the succeed probabilities are above $99.6\%$ and the BER values are below the FEC limit for distances under 100 m, and $99.8\%$ for maximum 95 m.

Since the probability of decoding the symbols successfully is above $99.6\%$ for the three distance’s ranges, the experiment demonstrated that the modulation scheme is functional, even when only 1.5 light bands are extracted from the frame. Additionally, the BER results for three consecutive frames are comparable to the outcomes presented in [29,30], which are considered viable modulations schemes. Since each transmitted symbol contains two bits, the throughput of the system is $f_{cam}\ast 2/3=16.67$ bps.

7. Conclusions

In this work, a flickering-free distance-independent modulation scheme was proposed and validated. From the detailed analysis of its theoretical fundamentals, a procedure for selecting the appropriated switching frequencies, based on the receiver’s characteristics, the emitters’ sizes, and the maximum distance of the system were developed. The validity of that strategy was proved through a set of simulations detailed in Section 6. At the same time, the communication’s performance when applying the modulation technique was determined with some field experiments presented in the same section. The obtained BER and success rate for the implementation with the minimum amount of consecutive frames are comparable to the outcomes of the target modulation schemes proposed in [29,30]. Additionally, the system’s throughput ($f_{cam}\ast 2/3$ bps) is better without the necessity of calibration procedures or overhead since the proposed modulation method is based on the analysis of the consecutive frames PCC. BER and throughput results outperform previous works, although they are not so high as conventional optical links. Nonetheless, in some cases where the BER values are under the FEC limit, the use of these error correction techniques could improve the link quality, but with a cost in the data rate that needs to be considered.

On the other hand, the simplicity of the wake up process and the modulation method assures an overall minor complexity of the system, an improvement compared to other long-distance modulation techniques which presented significant or high complexity [25,26,33,39,40] for similar distance’s ranges. Moreover, the experimental results showed a reliable 100 m link using a frequency multiplier $\alpha$ below the maximum value calculated for the camera’s characteristics. When a greater $\alpha$ is used for the system, the switching frequencies increase, producing thinner light bands requiring fewer pixels for fulfilling the one and a half strips limitation required for proper demodulation. Therefore, longer distances can be reached. In the detailed field experiment, the maximum achievable distance is 134 m for the selected vertical resolution working with a switching frequency of 2700 Hz. Finally, the experimental results concluded that this modulation technique’s main constraint is the number of lighter and darker bands extracted from each frame, since the communication’s performance is directly proportional to this number. Therefore, the maximum value of $\alpha$ should be applied to the system independently of the target distance.

The proposed modulation technique mitigates the problem of light’s condition instability during data transmission by working on the similarity of normalized images. However, more research on Signal to Noise ratio (SNR) effect over the system is required to propose a mitigation process.