A Survey on Energy Harvesting Wireless Networks: Channel Capacity, Scheduling, and Transmission Power Optimization

Abstract

This paper presents a survey on energy harvesting (EH) wireless communication networks focusing on channel capacity, transmission schemes, and power optimization. While many network researchers focus on energy management policies addressing the intermittency and randomness of the EH processes, but the channel capacity, and transmission power optimization have not been fully explored yet. In this paper, we provide a review and analysis of channel capacity, offline and online transmission schemes, and power optimization from an information theory perspective. By reviewing and analyzing wireless networking literature, we found that EH is a technologically feasible and economically viable paradigm for cost-effectiveness in the design and deployment of next-generation wireless networks. Finally, we identify open research problems and future research directions in the emerging field of EH wireless networks. We expect this study to stimulate more research endeavors to build energy-efficient scalable next-generation wireless network systems.

1. Introduction

The exponential growth of wireless networks has highlighted the issue of energy consumption into sustainability (e.g., reducing carbon footprint) and operating expenditure (OPEX) perspectives. Wireless energy harvesting (WEH) is one of the key techniques in achieving energy harvesting (EH) in wireless communication networks [1]. Harvesting energy from natural and ambient sources is a promising means of increasing energy efficiency and improving sustainability in wireless communication systems. In effect, we need to address the fundamental bandwidth-power and delay-power trade-off to reduce energy consumption ensuring that network performance continues to meet the traffic demand [2].

Energy harvesting devices (EHDs) are of great interest in the contexts of energy efficiency in wireless communication networks. The deployment of EHDs is on the rise, often replacing battery-operated counterparts as they offer several advantages over grid or battery-powered systems [3]. The potential of energy self-sufficiency, self-sustainability, and perpetual lifetime limited hardware components can improve energy sustainability. Various alternative energy sources such as solar, vibrational, thermoelectric, and electromagnetic can be harvested to power EHDs. Therefore, EHDs facilitate perpetual operation and self-sustainability as far as energy efficiency is concerned [4].

The EH is a promising technique to achieve a cost-effective solution for green computing and networking [5,6]. Although there is a long history of EH focus at the device and circuit level, it has only very recently received engineering attention at the system, communication, and network level. The availability of powerful and very low-power electronics provides the computational capability and operational longevity needed by EHD. Network, system, and communication have traditionally incorporated energy consciousness as a design discipline focusing on either minimizing energy consumption against a given performance metric or optimizing performance given a fixed amount of energy. The additional constraints due to the inherent intermittent and stochastic nature of the energy harvesting process introduce novel dimensions of uncertainty to the wireless communication problem. The EH system design demands a reevaluation [7,8] of the communication paradigm aimed at maintaining reliable communication in the light of additional constraints. The research methodology we adopted in the literature review is discussed next.

1.1. Research Methodology Adopted

To formulate this survey paper, we collected and reviewed about 35 relevant papers in the broad field of energy harvesting wireless communication networks. These papers were collected from various refereed journals and conference proceedings obtained from well-known databases including IEEE Xplore, ACM, Springer, and ScienceDirect. We also looked at industry white papers from online credible sources. We used the analysis and synthesis approach in reviewing the literature to come to a meaningful conclusion. A summary of related work (literature review) is presented next.

1.2. Summary of Related Research

There are various research topics related to EH such as mobile charger scheduling, message broadcast, and optimal coverage [9,10,11,12,13]. In the review of literature, we mainly focused on aspects of channel capacity, transmission schemes, and power optimization in the context of EH. A summary of related work is presented in

Table 1. Summary of related research.

Survey Paper	Scope	Year	Remark/Limitation
Rashidi et al. [14]	Epidemic routing, delay-tolerant network, non-sparse network.	2020	Developed an infection rate routing model for the supercritical network as a function of time.
Thomas et al. [11]	Quality of Service (QoS) in Wireless Sensor Networks (WSNs)	2020	Energy management technique in WSNs with a focus on node scheduling.
Wang et al. [15]	Energy harvesting technologies, tools, and techniques.	2018	Conducted a comprehensive review on EH in wireless communication networks.
Yang & Chin [9]	Energy harvesting node placement for energy-neutral coverage and connectivity.	2017	Determined the locations to place the minimal number of nodes used for sensing and relaying deployed nodes.
Liu et al. [10]	Assuring coverage quality for rechargeable Wireless Sensor Networks.	2017	The study endeavors to determine the minimum number of sensor nodes to deploy to ensure a given coverage quality.
Djenouri & Bagaa [12]	Communication coverage for sustainable data forwarding.	2017	Proposed an energy-aware deployment model for Relay Nodes (Rns).
Baroudi [13]	Battery maintenance in WSN.	2017	Proposed a framework for battery maintenance in WSNs through recharging sensor batteries using mobile robots.
Mao et al. [3]	Mobile-edge computing, EH devices.	2016	Proposed a computing offloading strategy for EH devices for a mobile-edge computing system.
Ulukus et al. [2]	Information-theoretic performance limits, medium access, network issue.	2015	Addressed the design issues of EH wireless communication protocols.
Blasco et al. [16]	Reinforcement learning technique, transition probabilities.	2013	The assumption of independence of the stochastic processes governing the data and energy arrivals might not hold at a network level.
Tutuncuoglu et al. [17]	Binary noiseless channel mode, AWGN binary channel input, Naïve IID (NIID).	2013	The achievable rate is based on mutual information between defined channel input and output.
	Equivalent timing channel, Optimized IID (OIID).	2013	Channel capacity definition is equivalent to the state-dependent channel.
	Auxiliary variable with finite cardinality, equivalent timing channel.	2013	No provision in the scheme/method to utilize energy packets that arrive when the battery is full.
Ozel & Ulukus [5]	Energy-saving phase, save and transmit method.	2012	Delay averages out and energy fluctuates.
	Energy constraints, best-effort transmit.	2012	Increased encoder/decoder complexity.
Jing & Ulukus [18]	Energy causality constraint.	2012	Not applicable to applications with delay constraint as no data arrival during transmissions.
Ozel & Ulukus [19]	Optimal capacity for the coding scheme	2011	The system is limited to a single access scenario.
Ozel et al. [20]	Finite battery size, static channel, point-to-point data/energy constraints.	2011	The monotonically increasing condition on transmit powers may be violated.
	Fading channel, continuous-time model.	2011	Epoch durations are only constrained by channel fade events and decoupled from energy arrival events.
Shuguang et al. [21]	Adaptive coding and modulation scheme, battery-powered, WSNs.	2005	Applicable for WSNs but missing for EH field.

. The selected published papers are listed in column 1. The scope of each survey paper, year of publication, and remark/limitations are presented in columns 2 to 4, respectively.

1.3. Contribution

The main contributions of this paper are highlighted below.

• The main contribution and strength of this paper is the emphasis that energy harvesting (EH) in wireless networks is crucial for the successful design and deployment of next-generation wireless networks. To this end, we summarize recent results in EH wireless networks focusing on aspects of data scheduling and transmission power optimization. This necessitates analyzing communication protocols at the physical and medium access layers from information theory, communication theory, and signal processing perspectives.
• We present a comprehensive review and analysis of channel capacity limits of the physical channel for various battery capacities, including infinite battery, no battery, and finite battery scenarios.
• We provide an in-depth study on offline transmission optimization with energy causality constraint, data arrival, finite battery, and fading channels. In addition, online optimization strategies are reviewed using a Markov decision process (MDP) formulation.

The rest of the paper is organized as follows: Section 2 introduces an abstraction of a generic EH node and a mathematical model. Section 3 identifies information theory-based efforts at modeling channel capacity for three battery capacity constraints. Section 4 surveys recent results on offline optimization of transmission policy for various constraints. Online transmission schemes are reviewed in Section 5. Future research directions and open research challenges are discussed in Section 6. Finally, a brief conclusion in Section 7 concludes the paper.

2. Mathematical Abstraction of Energy Harvesting

An analytic model must represent features common to all EH systems: An intermittent energy supply, finite storage capacity, imperfections of storage devices, and complexity constraints. We examine a mathematical abstraction for energy harvesting devices (EHDs) that briefly introduces key concepts needed for the subsequent discussion and analysis.

An EHD consists of sub-systems that implement energy harvesting and storage, data processing, and sensing and communication, as shown in Figure 1 (top part) [22]. The bottom part of Figure 1 represents a mathematical abstraction of an EHD. Incident ambient energy is captured and converted by the EH module. The electrical energy is stored by the storage element (SE), which may consist of a battery in conjunction with a capacitor. The SE powers the microprocessor ($\mu P$), sensor, and radio. The $\mu P$ implements the energy management policy by duty cycling the sensor and radio sub-systems and managing data storage.

The block diagram of an EHD in Figure 1 can be conceptualized as consisting of finite energy and data queues determined by corresponding models of energy arrival process $H\left(t\right)$ and data arrival process $I\left(t\right)$. The data buffer has a static capacity $D_{max}$, while the capacity of the energy buffer is a function of time (reflecting imperfections of the SE and non-deterministic EH process). The $\mu P$ controls the switches determining the flow of data and energy according to a set of rules determined by the energy management policy. We assumed that energy and data arrivals are independent of each other and described by independent and identically distributed (i.i.d.) discrete-time stochastic processes. The optimal energy management policy, in turn, can be conceptualized as one that seeks the best strategy to empty the data queue, given some $H\left(t\right)$ and $I\left(t\right)$ processes.

3. Analysis of Channel Capacity

Channel capacity in a system powered by EHDs is a function of the rate at which harvested energy is available for data transmission, the data arrival rate along with conditions on transmission, and the correlation between these two stochastic processes.

Energy harvesting process $H\left(t\right)$, depends on assumptions on battery size. Every channel use depletes the stored energy and must not violate energy causality. Generally, the classic additive white Gaussian noise (AWGN) channel is considered for analytic tractability with input $X$, zero mean unit variance Gaussian noise $Z$, and output $Y=X+Z$.

The energy arrival at the energy harvesting battery located at the transmitter is modeled as an ergodic, stationary stochastic process $H$, with mean value $P$ (${\rm E}\left(H\right) =P$). The average power constraint per codeword in this setting is denoted by$: \frac{1}{n}{{\displaystyle \sum }}_{i=1}^n{X}_i^2\le P$. At each channel use $X_i^2$ units of energy are depleted and H_i units arrive via the harvesting process. The energy constraint that needs to be satisfied by codewords on every channel use is given by

$${\displaystyle \sum }_{i=1}^k{X}_i^2<{\displaystyle \sum }_{i=1}^k{H}_i$$

(1)

where $i=1,\cdots ,k$.

The EHD term on the right of the inequality shown in (1) determines the available energy per-codeword. Therefore, optimal transmission policy would vary the design, rate, and scheme of codewords, to satisfy (1). The different schemes are qualified on battery sizes categorized as infinite, finite, and zero.

The channel input at discrete instant i occurs if $H\left(i\right)\ge X_i^2$. The energy constraint satisfied per code symbol is $X_i\in \left(X_1,..,X_n\right)$. For equivalent AWGN channel featuring average transmit power, the capacity-constrained by average recharge rate is given by

$$\complement =\frac{1}{2}log\left(1+P\right)$$

(2)

The above analysis is valid for ideal battery, no energy processing cost, and assumed independence of energy arrival process.

Figure 2 shows a schema of EHD channel capacity for three battery-size scenarios. The three battery scenarios (infinite, no battery, and finite) are discussed below.

(A)

Infinite battery scenario: Channel capacity findings with infinite battery for two methods—namely, save and transmit, and best-effort transmit [16], are compared based on energy arrival profile, channel coding scheme, among other features. The arrival process is assumed to be an independent i.i.d. sequence of random variables. The save-and-transmit method assumes an infinite battery and features an energy-saving phase where it transmits zeros, followed by a data transmission phase. The best-effort transmit method, on the other hand, begins transmission right away and discards a symbol that requires more energy than is available. As a result, it requires greater encoder/decoder complexity. The main feature of the save-and-transmit method is that more symbols carry information than the best-effort transmit method.

(B)

No battery scenario: Channel capacity findings with no battery ($E_{max}=0$) for AWGN channel assumes time-varying amplitude constraints ($\alpha 1,\alpha 2$) with causal information available only at the transmitter [19]. The channel coding scheme, in turn, is determined by the support set for optimal cumulative distribution function (CDF) F of energy arrivals. The channel capacity is given by

$$C=\underset{F\in \Omega }{max}{I}_F\left(T;Y\right)$$

(3)

where $\Omega$ is the space of compact and convex joint probability density function (pdf) over $\left[-a1,a1\right]\times \left[-a2,a2\right]$. The expression for mutual information $I_F\left(T;Y\right)$ in terms of a convex optimization problem with a unique solution. The energy arrival profile independent and identically distributed (i.i.d) sequence with finite realizations. The code symbol amplitude constrained to available energy at each channel use can be expressed as

$$X_i^2\le E_i$$

(4)

The energy arrival profile ($E_i$) is an i.i.d. Bernoulli process under the constraints of the current state of the battery and the last channel use. The expressions are given by

$$E_i\in \left\{0,1\right\}$$

(5)

$$Pr\left[E_i=1\right]=q$$

(6)

However, the above analysis is valid for the following assumptions. Transmitter has causal knowledge of $S_i$ and has feedback knowledge of past channels used. The receiver does not know history or $S_i$. The rate does not account for the energy arrival process, and the receiver exploits the knowledge of system memory.

Optimal coding scheme scales with the magnitude of $\pm \sqrt{E_i}$. $S_F$ denotes the finite optimal support set for F. Members of $S_F$ determine the optimal rate capacity for the coding scheme in place [19]. Though limited to a single-access scenario, this study demonstrates the capacity for a stochastic, amplitude-constrained channel with a discrete input distribution. Furthermore, it demonstrates that an infinite battery-size assumption lends itself to higher achievable capacity.

(C)

Finite battery scenario: Channel capacity findings with finite battery for two methods—namely, naïve i.i.d. (NIID) and optimized i.i.d. (OIID) Shannon strategy for the state-dependent channel [17]. For binary channel input, the number of channels uses between the transmission of successive 1’s may be used as the basis for encoding and decoding. Let us suppose that the number of channels uses between (n − 1)th and nth-transmitted 1 is given by

$$T_n\in \left[1,2,\dots \right]$$

(7)

Idle time between (n − 1)th-transmitted 1 and the next energy arrival is given by

$$Z_n\in \left[1,2,\dots \right]$$

(8)

The number of channels uses the transmitter waits to transmit 1 after energy arrival following (n − 1)th transmission of 1 is given by

$$V_n\in \left[1,2,\dots \right]$$

(9)

The equivalent timing channel can be written as

$$T_n=V_n+Z_n$$

(10)

The above analysis is valid for the following assumptions. Transmitter has causal knowledge of $Z_n$ and feedback knowledge of $T^{n-1}$ outputs. The receiver can only calculate $T_n$. Message: $W\in \left[1,2,..M\right]$. The maximum channel uses to transmit $W$ for a given $E_i$.

Table 2. Comparison of energy arrival, transmit, encoder complexity, and capacity for infinite, finite, and no battery.

Battery Condition	Arrival Process	Transmit Method	Encoder/Decoder Complexity	Achievable Capacity
Infinite	Independent	Save and best effort transmit	Greater	High
No battery	Bernoulli process	Casual knowledge/information	Lower	Low
Finite	Binary arrivals	Naïve and optimized i.i.d	Medium	Low to high

compares the energy arrival processes, transmit method, encoder/decoder complexity, and achievable capacity for the three battery scenarios (infinite, no battery, and finite). The battery condition is listed in Column 1, and the corresponding arrival process, transmit method, encoder complexity, and achievable channel capacity are presented in Columns 2 to 5, respectively. We observe that the infinite battery can offer high channel capacity than the finite battery conditions. However, this high capacity is achieved at the expense of greater encoder/decoder complexity. In case of no battery condition, the achievable channel capacity is low, as expected.

(D)

Summary of Findings and Discussion

The performance superiority of this energy management scheme is demonstrated, which is agnostic to battery degradation. We observed that battery capacity is a critical determinant of channel capacity. The capacity for the Gaussian channel with an infinite battery is achievable using Gaussian codebooks employing save-and-transmit and best-effort transmit schemes. The amplitude-constraint discrete and time-varying inputs in the case of no battery achieve a strictly smaller capacity than the former case. Simpler assumptions of a binary, noiseless channel employed for the case of finite battery (unit size considered) yielded a correspondence with the timing channel. The resulting single letter capacity expression required solving for the cardinality of an auxiliary variable. The issues with the capacity for general channel and energy arrivals in the finite battery case are partially solved.

4. Analysis of Offline Transmission Schemes

Offline energy management of the transmission power and data rate assumes precise knowledge of the harvested energy and data arrival processes at the transmitter before the transmission begins [24]. The goal is to schedule the delivery of the data packets to the destination given the available energy profile while optimizing performance metrics such as minimum time to complete transmission, delay constraints, and throughput maximization while incorporating practical concerns such as processing cost, channel fading, and non-ideal battery. We reviewed recent results for single-user and multi-user scenarios. Most solutions emerge out of a convex optimization framing of the problem. Figure 3 shows a classic directional water-filling algorithm. It forms the basis with several variants, such as directional, staircase, and glue-pouring that represent various combinations of energy, data, and battery constraints. The top part of Figure 3 shows a scenario in which all the taps are turned OFF, and the epochs are determined by the channel-fading events. The bottom part of Figure 3 shows a scenario where all the taps are turned ON, and consequently, no energy flow to previous epochs. However, the energy arrivals due to E1 flow into the next epoch.

L1, L2, …, L9 represent epochs, whereas E0, E1, E2, and E3 represent energy arrivals per epoch. The $E_{max}$ indicates the maximum size of the energy buffer (i.e., storage capacity). More details about offline energy management strategies can be found in [24,25,26].

The assumptions for monotonic increase, concavity, and continuously differentiability associated with the rate power function $g\left(p\right)\left(r\right)$ generally hold. These assumptions imply optimality of constant power during transmission of a single bit and the existence of $g^{-1}\left(r\right)$. Assuming an AWGN setup and the associated rate power relationship $R=\frac{1}{2}log\left(1+hP\right)$.

Therefore, every transmission of power $p$ and duration $l$ sends $\frac{l}{2}log\left(1+hP\right)$ bits of data using $lp$ units of energy. Our problem is to determine the optimal time-varying power scheme $P\left(t\right)$ that minimizes transmission time for all packets as a function of the random energy arrival, data backlog, and fluctuating channel fading profiles. It is noted that the transmission minimization problem is a dual of the throughput maximization [20].

We assume $B_i$ and $E_i$ represent the ith bit and amount of energy arrival into the data and energy queues, respectively, of the EH transmitter at different epochs in addition to backlogged data and already stored energy. The transmitter can adapt transmission power and rates to suit average energy and remaining bits. Power changes occur at every epoch [18]. The analytical models for energy and transmission power optimization are presented next.

(A)

Energy optimization: Given the power sequence $p=\left\{p_1,p_2,\cdots ,p_N\right\}$ and duration $l=\left\{l_1,l_2,\cdots ,l_N\right\}$.

The energy arrivals in epochs $\left\{s_1,s_2,\cdots ,s_k\cdots \right\}$ with amounts $\left\{E_1,E_2,\cdots ,E_k\cdots \right\}$ are given by

$$\underset{p,l}{min}Ts.t.E\left(t\right){\displaystyle \sum }_{i:s_{i<t}}{E}_i$$

(11)

$$B\left(T\right)=B_{o}0\le t\le T$$

(12)

The energy causality constraint can be expressed as

$${\displaystyle \sum }_{i=1}^k{l}_i{p}_i\le {\displaystyle \sum }_{i=0}^{k-1}{E}_i$$

(13)

where $i=1,2,\cdots ,k$.

The optimal policy can be achieved as

$${\displaystyle \sum }_{n=1}^{N}g\left(p_n\right)l_n=B_o$$

(14)

for $n=1,2,\cdots ,N$.

$$i_n=arg\underset{i:s_{i\le T}}{min}\left(\frac{{{\displaystyle \sum }}_{j=i_{n-1}}^{i-1}{E}_j}{s_i-s_{i_{n-1}}}\right)$$

(15)

$$p_n=\frac{{{\displaystyle \sum }}_{j=i_{n-1}}^{i_n-1}{E}_j}{s_{i_n}-s_{i_{n-1}}}$$

(16)

$$l_n=s_{i_n}-s_{i_{n-1}}$$

(17)

where $T={{\displaystyle \sum }}_{n=1}^N{l}_n$.

(B)

Optimization of data arrivals: At $t=0$, $B_0$ bits available and data arrive in amounts $B_1,B_2,\cdots ,B_{M }$at times $t_1,t_2,\cdots ,t_M$.

Now the optimization problem is

$$\underset{p,l}{min}Ts.t.E\left(t\right)\le {\displaystyle \sum }_{i:s_{i<t}}{E}_i$$

(18)

$$B\left(t\right)\le {\displaystyle \sum }_{i:s_{i<t}}{B}_{i}0\le t\le T$$

(19)

$$B\left(T\right)={\displaystyle \sum }_{i=0}^M{B}_{i}0\le t\le T$$

(20)

(C)

Finite battery optimization: The system optimization problem for finite battery constraint can be written as

$$\underset{p_i\ge 0}{max}\frac{L_i}{2}log\left(1+p_i\right)$$

(21)

$$s.t.{\displaystyle \sum }_{i=1}^l{l}_i{p}_i\le {\displaystyle \sum }_{i=0}^{l-1}{E}_i$$

(22)

where $l=1,2,\cdots ,N+1$.

$${\displaystyle \sum }_{i=0}^{l-1}{E}_i-{\displaystyle \sum }_{i=1}^l{l}_i{p}_i\le E_{max}$$

(23)

where $l=1, 2,\cdots ,N$.

The concavity of the objective function along with the linear convexity of the constraints implies a unique maximal and suitability to a Lagrangian formulation. Associate Lagrange multipliers ${\lambda }_i\wedge {\mu }_i$ with the two constraints. Express the optimal power levels by applying the Karush–Kuhn–Tucker (KKT) optimality conditions.

$$p_i=\frac{1}{{{\displaystyle \sum }}_{j=i}^{N+1}{\lambda }_j-{{\displaystyle \sum }}_{j=i}^N{\mu }_j}-1$$

(24)

where $i=1, 2,\cdots ,N$.

(D)

Fading channel optimization: Suppose there are $N$ changes in power levels and $M$ changes in the channel transfer function during. Therefore, $T=L+M+N+1$.

The optimization problem becomes

$$\underset{p_i\ge 0}{max}{\displaystyle \sum }_{i=1}^{N+M+1}\frac{L_i}{2}log\left(1+p_i{h}_i\right)$$

(25)

$$s.t.{\displaystyle \sum }_{i=1}^l{l}_i{p}_i\le {\displaystyle \sum }_{i=0}^{l-1}E\left(i\right)$$

(26)

where $l=1,2,\cdots ,N+M+1$.

$${\displaystyle \sum }_{i=0}^{l-1}E\left(i\right)-{\displaystyle \sum }_{i=1}^l{l}_i{p}_i\le E_{max}$$

(27)

where $l=1, 2,\cdots ,N$.

The Lagrange multipliers now consist of three sets: ${\lambda }_i,{\mu }_i\wedge {\eta }_i$. The third multiplier accounts for the scenario where the current fading implies an optimal policy that dictates no transmission during that epoch.

(A)

Summary of Findings and Discussion

When the transmission distances are small (typically for wireless sensor networks), the processing energy cost needs to be considered. Youssef-Massaad et al. [27] develop optimal rate regions for an EH transmitter considering fixed (𝜖) processing cost for the single-user, multiple channels and multi-access AWGN static channels. They have found bursty transmissions with the same average signal power for a fraction 𝛳 of an epoch are optimal. The processing cost overhead of using a new channel in the case of multiple parallel channels is conditioned on the signal-to-noise ratio (SNR) level of the unused channels being lower than the channel already in use by a given threshold. They call this “glue pouring” variant of the water-pouring algorithm (Figure 4). The multiple access case considered for two users reduces to the single-user case when there is no contention and the parallel channel scenario when users compete. The main results that are associated with the offline transmission optimization schemes (glue-pouring algorithms) are illustrated in Figure 4.

Figure 5 illustrates the energy optimizations solution using the directional glue-pouring algorithm. It starts with defining the remaining energy maximization (REM) problem and packet scheduling for EH transmission. Then, the problem set is defined. The online transmission policies (heuristic solution) are then set up to deal with transmitting all the data. The glue-pouring algorithms are executed to provide energy to transmit data packets fulfilling the energy causality.

Orhan et al. [28] extend the results of fading channels. Optimal offline schemes are developed for the REM, minimizing transmission completion time (TCT), and an online heuristic using the REM solution.

5. Analysis of Online Transmission Schemes

Online transmission policy assumes that the underlying stochastic processes are ill-defined. Therefore, it is better to define an iterative procedure for an optimal policy on a sequence of past decisions. A Markov decision process (MDP) permits exactly this. An MDP is built on the state-based formalism described by a Markov chain, by first associating a reward function with each state transition in the Markov chain to achieve a Markov reward process (MRP), and finally associating an action with each reward, such that a sequence of actions determines a policy. The resulting MDP may be employed to evaluate a fixed policy by the process of value iteration that determines the value of each state for the policy or may be used to determine the optimal policy that maximizes the reward.

Figure 6 shows a schema of various MDP optimizations. The MDP framework is used to associate instantaneous importance with the transmission of each packet from an EHD to maximize long-term average reward. An i.i.d. harvesting process restricts the current system state to the contents of the battery with state transition probabilities determined by the set of permissible actions assigned to each state (transmit/drop and associated powers).

Sharma et al. [29] develop optimal and sub-optimal transmission policies to maximize throughput or minimize delay and extend the results to the time-slotted media access control (MAC) level. The treatment is clear, comprehensive, and free of restricting assumptions making it a good base for design reference.

Michelusi et al. [30] consider a threshold structure per transmission policy that determines the importance of transmitting the sensed data packets. A strict deadline policy ensures the packets that are not transmitted are dropped. EH follows a Bernoulli i.i.d. process, independent of the stochastic process generating the importance values {$V_k$}. It is assumed that the amount of energy harvested during each slot is greater than the amount required to transmission. The transmitter assumes perfect knowledge of the battery state of charge, and battery degradation is not considered. The evolution of the battery state $B_k$ over interval $\left[k,k+1\right),k\in Z^{+}$ is described by $B_{k+1}=min\left\{\left[B_k-T_k\right]+E_k,E_{max}\right\}$, where $T_k$ represents the amount of energy utilized for transmission during slot $k$, $E_k$ is the amount harvested. The policy $\lambda$ manifests as a probability measure over the binary action space and aims to maximize the long-term throughput. Given an initial state $S_0$ the expected reward gathered is defined by $R\left(\lambda ,S_0\right)=\underset{k\to \infty }{\lim }inf\frac{1}{K}E\left\{{{\displaystyle \sum }}_{k=0}^{K-1}{T}_k{V}_k|S_0\right\}$ and $\lambda \left(opt\right)$ solves the optimization problem $\lambda \left(opt\right)=arg\underset{\lambda }{max}R\left(\lambda ,S_0\right)$.

Figure 7 illustrates the system models for online transmission policies with correlated energy supply. It consists of a system model, MDP model base, continuous-time deterministic model (CDM), and discrete-time state model (DSM).

Three classes of policy are developed including optimal (using linear programming), balanced policy (BP), and heuristic policy. BP transmits every permissible energy state on an average balancing the energy consumption and harvesting rates. The heuristic policy is conservative when energy levels are low, BP-like at average energy levels, and aggressive when stored energy approaches $E_{max}$. The research presented in [31] extends the analysis for the case where the underlying EH process {$A_k$} is a two-state (Good, Bad) Markov chain. The optimal policy is the transmission probability $\eta \left(e,a\right)$ and power to depletion ratio $\rho \triangleq \frac{e_{max}}{D_B}$ defined that determines the performance of the various algorithms developed in the discrete and continuous time-space. The continuous-time model (CTM) involves a deterministic EH process known to the EHD controller. The optimal policy is found to be determined by the value of $\rho$.

6. Open Research Problems and Future Directions

The mathematical model for EHDs needs to be augmented with statistical data from real-world scenarios. We need a base power consumption and storage models on actual devices as well as account for leakage, degradation, and capacity depletion. This is often seen in storage elements and uses this information to inform energy management policies. An intelligent energy management scheme that responds to battery capacity degradation within the Markov chain framework is developed in [32].

6.1. Open Research Problems

There are several open research problems on EH wireless networks. First, the random intermittency of EH along with the transmitter’s action result in uncertainty of energy available at the transmitter. The transmitter energy is coupled with the non-availability of energy state information at the receiver posing significant challenges in determining the channel capacity. Certainly, the processing of the received signal and determining energy allocations to various system components become a complicated problem. A signal processing perspective is best suited to this problem. The problem of generic coding schemes remains open. The information-theoretic capacity of a general channel with a general finite-size battery remains unsolved, as does accounting for correlated side information of communicating EHDs harvesting energy from the same source.

Second, a priori knowledge of EH and data arrival processes assumed by offline transmission schemes are ideal and unrealistic. Furthermore, even the knowledge of future states (online schemes) may not remain consistent over time or available during deployment. The reinforcement learning (RL) algorithms could be used to estimate stochastic process parameters in real-time to adapt transmission schemes. Given the state space of EHD correlated with the transmission, the policy is not large, owing to the binary nature of the actions. The RL techniques can be employed to evaluate the policy and system optimization.

Third, Blasco et al. [16] employ a reinforcement learning technique called Q-learning to learn the transition probabilities of the underlying Markov processes (fading states, EH, data arrivals). Simulations indicate that the algorithm converges within an acceptable learning period. An interesting research problem would be to explore such solutions and integration of learning algorithms and sub-optimal low complexity solutions. The assumption of independence of the stochastic processes governing the data and energy arrivals might not hold at a network level. For example, in a multi-hop setting the transmissions scheme of one node couples with the data arrival process of the receiving/relaying node. Additionally, the energy arrivals at spatially proximate EH nodes from a common source may be jointly distributed. Though these challenges exacerbate analytic complexity they also present an opportunity for greater integration of local optimization schemes. Although reference [21] derives throughput maximizing adaptive coding and modulation schemes for lifetime extension of battery-powered WSNs, a similar treatment is missing in the EH field.

Fourth, the efficiency of EH devices is determined by the efficiency of energy transfer and thus with the circuits and devices that harvest, covert, and transfer the energy. Energy harvesting devices typically operate over short distances where circuit-level energy costs are comparable to transmission costs. We saw that with processing energy cost the optimal transmission is bursty. However, the energy required for burstiness scales exponentially with capacity. A detailed model of energy consumption calls for an inter-disciplinary approach integrating communication strategy with circuit design would be a good research problem for investigation.

6.2. Future Research Directions

Future wireless networks are expected to employ a portfolio of technologies, architectures, devices, components, algorithms, and protocols targeting energy harvesting (EH) in wireless systems. We highlight and discuss (with supporting literature) the following four future research areas that are worthwhile to pursue: (1) energy harvesting framework for WSNs; (2) network-wide green measures toward EH framework; (3) energy outage probability analysis toward EH; (4) energy harvesting delay-tolerant integrated framework. These four future research directions are discussed below.

Energy harvesting framework for WSNs: Yang and Chin [9] present a study for WSN to determine the locations to place the minimal number of nodes used for sensing and relaying deployed nodes. In Liu et al. [10], given a set of locations, the study endeavors to determine the minimum number of wireless sensor nodes required for good coverage. Thomas et al. [11] discuss energy management techniques in WSNs focusing on node scheduling and deployment. For WSNS, Djenouri and Bagaa [12] present a study within the context of communication coverage for sustainable data delivery. Baroudi [13] proposes a framework for battery maintenance and recharging in WSNs using mobile robots. Developing an energy harvesting framework for WSNs is suggested as future research.

Network-wide green measures toward the EH framework: Figure 8 shows an overview of the network-wide green measures EH framework. There may be various future research projects that can be derived from Figure 8. Examples of new research projects include green network architecture, energy harvesting, and monitoring, and system-level modeling for energy consumption.

Wireless sensor networks (WSNs) form a subset of the overall architecture yet there is a wide variety of MDPs applied to different aspects [33]. Figure 9 shows various applications of Markov decision processes related to WSNs. A hybrid approach combining centralized and distributed control facilitated by the separation of data and control planes powered by software-defined networking (SDN) and network function virtualization (NFV). These technological realizations are needed to manage the anticipated uncertainties and performance requirements that emerge from the system-level integration of EH.

Energy outage probability analysis toward EH: Existing literature on EHD restricts evaluating performance measures such as energy outage probability (EOP) to the device level. There is a gap in policies examining the reconfiguration of topology and routing to minimize the impact of individual device-level energy outages on system EOP. The weather systems form part of a larger eco-sphere of balanced energy exchanges that determine climatic patterns. If all of the significant energy needs of wireless systems [34,35] are derived from natural sources, one can expect a downstream effect on the encompassing systems the energy is depleted from, especially given the scale and pervasiveness of wireless communication systems. Research on energy outage probability modeling and analysis would a good one to pursue.

Energy harvesting delay-tolerant integrated framework: References [36,37,38] are some examples of research that have been analyzed routing in delay tolerant networks (DTNs). A multi-paradigm approach to routing and system design that switches to a DTN typesetting in the event of transient/long-term energy depletion or failure of EH nodes leading to disconnected sub-graph topologies. A mobile (aerial/terrestrial) sink can be deployed for periodic data collection from the sub-graphs. Such means decouple network lifetime from individual node failures while accounting for temporary energy shortages, extending the utility of deployments. EH has the potential to produce clean and renewable energy [15], but at the same time, it presents novel theoretical challenges that need to incorporate challenges emanating from practical considerations. Research on developing an efficient EH delay-tolerant integrated framework using an interdisciplinary approach (information theory, coding, and wireless communication with practical considerations) would be worthwhile to pursue.

7. Concluding Remarks

The energy harvesting (EH) wireless networks focusing on channel capacity, transmission schemes, and power optimization were surveyed. We explored recent advances in EH approaches for wireless communication systems. We looked at information-theoretic frameworks describing channel capacity at the physical protocol layer for different scenarios of battery capacity. Algorithmic formulations relating to energy and data constraints, as well as processing costs, were considered in the offline energy optimization schemes. The energy feasibility tunnel and various flavors of the water-filling algorithms were discussed. Online frameworks employing Markov decision process variants were also explored. Various research issues in the design of EH wireless networks and channel capacity were discussed. New research programs are required to create an EH Reference model, architecture, and protocols for addressing issues and design challenges of green wireless communication systems. Hence, we need a concerted effort among industry and academia and close cooperation with various government agencies including energy and communication sectors, and regulatory bodies. Thus, EH can be the next technological shifting paradigm that provides technologically feasible and economically viable solutions for next-generation wireless network systems. The integration of EH algorithms for a sub-optimal low complexity solution is suggested as future work.