The Effects of Dynamic Pricing of Electric Power on Consumer Behavior: A Propensity Score Analysis for Empirical Study on Nushima Island, Japan

Abstract

This study aimed to investigate the change of consumer behavior in electric power consumption after the application of dynamic pricing via real-time feedback. Afield experiment of dynamic pricing was carried out on Nushima Island, which is located in Hyogo Prefecture in central Japan. The panel data of hourly electric power consumption among 50 households (including 22 control households and 28 treated households) were collected from a baseline survey (14 days before the dynamic pricing experiment was conducted) and during the 14-day experimental period. Propensity score analysis with local linear matching was employed to analyze the average treatment effects of dynamic pricing on consumer behavior. The results report that dynamic pricing plays a crucial role in reducing consumers’ electric power consumption—by 9.6% compared to the pre-experimental period.

1. Introduction

Japan significantly lacks domestic reserves of fossil fuels and heavily depends on the import of substantial amounts of crude oil, natural gas, and other energy sources to meet its energy demands. Furthermore, Japan previously relied on nuclear power to meet up to one third of its electricity demands before the serious 2011 earthquake and tsunami at Fukushima Daiichi Nuclear Power Station. Consequently, Japan has been facing some issues in the decline in its energy self-sufficiency ratio, the increase of electric power costs, and increasing amounts of greenhouse gas emissions from the increase of imported crude oil, coal, and liquefied natural gas (LNG) (Agency for Natural Resources and Energy [1]). Particularly, the energy self-sufficiency ratio of Japan dramatically declined from 19.9% in 2010 to 6.0% in 2014, which is low compared to other OECD (The Organization for Economic Co-operation and Development) countries. The amount of greenhouse gas emissions from the electric power field increased by 83 million tons, from 374 million tons in 2010 to 457 million tons in 2014. Consequently, much attention has shifted to the promotion of renewable energy as an emergent response to satisfy Japan’s future energy needs. Renewable energy is an important source that potentially has low CO₂ emissions during electricity generation and contributes to the energy self-sufficiency ratio. Inf 2014, the renewable energy ratio in the amount of generated electric power in Japan was 3.2% (excluding hydroelectric power), representing an increase from 0.03% in 1973 and 1.1% in 2010, but is still low compared to other major countries (e.g., Germany, Spain, United Kingdom, United States, and France) [1]. However, the supply of renewable energy in Japan is unstable due to low capacity. Furthermore, the problems of limitation of nuclear power plants and maximization of the usage of existing LNG-based thermal power plants are not easy to solve for the immediate balancing and enhancing of the electricity supply. Therefore, it is necessary to focus on the perspective of electricity demand—one of the potential ways for managing electricity needs by lowering electric power consumption. This can also help to reduce not only the dependence on imported energy resources from foreign countries (e.g., fossil fuels) but also the problems of environmental degradation or greenhouse gas emissions (e.g., CO₂). Demand response is expected to reduce consumers’ electric power consumption through incentives of dynamic pricing and real-time information feedback via smart meters.

In addition, the introduction of electric power market liberalization from the Japanese Government since April 2016 allows customers to select from multiple supplier companies competitively selling electricity (Agency for Natural Resources and Energy [2]). The new market system can provide great advantages for customers in lowering the prices and empowering their choices of electricity suppliers; however, many of these suppliers only sell locally and mainly in large cities.

Nushima Island (Figure 1) was selected as the experimental study site because it is a remote island located in the south of Hyogo Prefecture, where the demand for electric power heavily relies on the Kansai Electric Power Company in Osaka, Japan. The total population of this island is about 500 people with 231 households (Statistical data in 2012, Hyogo Prefecture Website [3]). The current problem for Nushima Island is that it is far from the mainland, thus it is difficult for supplier companies to deliver electricity to its communities. Electric power shortages might occur on Nushima as well as more than 6500 other islands that have the same characteristics as Nushima. Therefore, it is important for consumers in these regions to adjust their consumption in the case of electric shortages and accommodate the fluctuations of renewable energy sources such as solar photovoltaic (PV) generation for future energy needs.

To the best of the authors’ knowledge, there is no previous study that has explored the treatment effects of dynamic pricing on energy-saving behavior using the propensity score analysis with panel data in field experiments. Therefore, the fundamental aim of the study was to assess the consumer behavior in electric consumption through the experiment of dynamic pricing using the propensity score analysis approach. The main differences between this study and Thoa et al. [4] include the objectives and data analysis methodology. The former difference means that this study focused on relatively short-term response by dynamic pricing while Thoa et al. [4] emphasized the persistence of dynamic pricing effects. The latter difference means that this study employed propensity score matching to solve a selection bias issue while Thoa et al. [4] used conventional panel analysis by dividing the households into control and treatment groups randomly.

The following hypotheses were considered: the incentive of dynamic pricing has negative effects on electric power consumption and the tariff or specific deduction rate related to dynamic pricing has a positive correlation with energy-saving effect. With respect to the results of this experiment, a smart-energy community’s model that is both environmentally friendly and resistant to the electric power market’s instability may be established.

2. Conceptual Framework

This section presents a short discussion of demand response, dynamic pricing, as well as the effects of dynamic pricing on demand response in the electricity sector in turn.

2.1. Demand Response

The term demand response (DR) in electricity markets can be considered to mean intentional changes in behavior related to electricity usage by end-use customers from their normal consumption patterns to adopt to change in the price of electricity over time, or to incentive payments to lower consumption. Demand response programs, therefore, should be designed to encourage consumers to shift their electric power consumption from peak to off-peak periods or reduce their peak demand in urgent or high-peak situations. Hence, demand response can be considered as a cost-effective alternative compared to adding generation capability to meet the peak or occasional demand spikes. The performance of demand response programs is measured by peak load reduction and demand elasticity.

There are two main components of demand response programs involving Incentive Based Programs (IBP) and Priced Based Programs (PBP) (Figure 2). Particularly, IBP are further classified into classical programs (including Direct Load Control and Interruptible/Curtailable Programs) and market-based programs (involving Demand Bidding, Emergency DR, Capacity Market, and Ancillary Services Market). In terms of classical programs, participants receive participation payments (e.g., a bill credit or discount rate) for their participation in the programs. In terms of market-based programs, participating customers are rewarded with money for their performance, depending on the amount of load reduction during critical conditions. Furthermore, PBP are mainly based on dynamic pricing in which electricity tariffs are not flat. PBP include the Time of Use (TOU) rate, Critical Peak Pricing (CPP), Extreme Day Pricing (EDP), Extreme Day CPP (ED-CPP), and Real Time Pricing (RTP). The fundamental objective of these programs is to flatten the demand curve by charging high prices during peak periods and lower prices during off-peak periods. The rate during peak periods is higher than the rate during other off-peak periods.

2.2. Dynamic Pricing

Dynamic pricing is considered as a demand-side management tool that can reduce peak load by charging different prices at different times in term of demand. For instance, dynamic pricing can reduce electric power demand by increasing the electricity rates when electric power demand is strong and decrease when electric power demand weakens. This implies that dynamic pricing can stimulate demand response and shift the demand from peak to off-peak. In addition, dynamic pricing also provides each consumer with an opportunity to reduce his or her electricity bill at a constant consumption level, just by changing the consumption pattern and by shifting the load.

Some previous studies related to field experiments reported that dynamic pricing can contribute to a reduction in peak-time electric power demand [6,7,8]. In addition, the empirical results from Faruqui [9] indicate that real-time pricing can induce a peak demand reduction of 10–14%, a resource cost reduction of 3–6%, a market-based customer cost reduction of 2–5%, and a social welfare increase of $141–403 million per year. Furthermore, Desai and Dutta [10] pointed out that dynamic pricing was economically more efficient than traditional flat rate prices since it absorbs consumer surplus, thereby enhancing total revenue at existing cost and reduced peak load.

3. Experimental Design

The field experiment of dynamic pricing on Nushima Island aimed to assess the possibility of the community’s self-control of its electric energy demand through dynamic pricing as well as the possibility of dynamic pricing policy according to solar PV generation potential.

The experiment was carried out on Nushima starting in 2012 through a five-year project. Fifty households were randomly assigned in five districts including South district, Central district, North district, East district, and Tomari district. In December 2012, smart meters were first installed in these fifty participating homes (Figure 3).

In May 2013, tablet PCs that could provide real-time feedback on electric power consumption were distributed to those participants. A variety of real-time feedback information types, which calculated each household’s electric power consumption and per-capita electric power consumption, as well as ranking of the levels of electric power consumption among participating households in the experiment, was displayed on the tablet PCs (Figure 4).

From the summer (August and September) of 2015 and the winter (January and February) of 2016, dynamic pricing was introduced. In this stage, the control and treatment groups were randomly selected based on the official household list of each district guided by local leaders. A control subject would receive a smart meter, a tablet PC, and a reward of 5000 Japanese Yen in summer at the end of the experiment while a treatment subject would receive a smart meter, a tablet PC, and a reward of 5000 Japanese Yen as well as a monetary incentive for energy conservation. In details regarding the monetary incentive, each treated participant was allocated 7000 points and points were then subtracted according to their electric power consumption. They could exchange their remaining points into cash at the end of the experiment (one point was equal to one Japanese Yen). There were three subtraction rates of dynamic pricing of 20, 30 and 40 points. These rates changed daily based on the weather forecast and were differentiated based on the PV power generation potential. The rate of 20-point deduction (20 points per kWh per person) was defined when weather forecast for both the preceding and current days included “sunny”. The rate of 30-point deduction (30 points per kWh per person) was defined when weather forecast for either the preceding or current day included “sunny”. The rate of 40-point deduction (40 points per kWh per person) was defined when weather forecast for neither the preceding nor current day included “sunny”. The reason we included the weather condition on the preceding day to decide those deduction rates is that it would have some influence on the remainder of a virtual battery to be charged by photovoltaic solar generation. Furthermore, data regarding hourly electric power consumption were collected based on the one-second interval data from the installed digital smart meters. In addition, the household characteristics of treatment and control groups were collected through a pre-experimental questionnaire survey. Secondary data from recorded meteorological data were also included. The result of this experiment is reported in Thoa et al. [4].

In the final stage of the project, dynamic pricing was continuously conducted from 20 July to 16 August 2016. The panel data collected from a baseline survey (14 days before the dynamic pricing experiment was introduced), during the 14-day experimental period, and a follow-up survey (14 days after the treatment) spanned treated and control subjects. We implemented the experiment for relatively short period because it aimed to investigate three short-term effect of dynamic pricing under the hottest and sunniest weather condition when the gap between photovoltaic solar electricity supply and demand is usually widened.

The experiment proceeded as follows:

-

Firstly, all confirmed participants were told in advance about the experiment and were asked to freely select to be a control or treatment subject after receiving a detailed explanation. A control subject would receive a smart meter, a tablet PC at the beginning of the experiment, and a reward of 2000 Japanese Yen at the end of the experiment while a treatment subject would receive a smart meter, a tablet PC, and an initial 7000 points at the beginning, which would then be subtracted from based on their actual electric power consumption during the experiment. The treatment subjects could exchange their remaining points into cash at the end of the experiment (one point was equal to one Japanese Yen). We took a different approach to divide them into control and treatment group compared to the previous experiments in summer 2015 and winter 2016 when the group was randomly divided. This selection strategy was chosen considering that some households in Japan can choose either conventional fix electricity tariff system or time-variant dynamic tariff system. We aimed to investigate the different effects on the two groups taking both current choice options and selection bias into account.

-

In terms of monetary incentive, namely dynamic pricing, three deduction points or tariff rates of 20, 40 and 80 were set up. These rates changed daily based on the weather forecast and were assumed based on the PV power generation potential (high tariff on rainy days and low tariff on sunny days). The rate of 20-point deduction (20 points per kWh per person) was defined when weather forecast for both the preceding and current days included “sunny”. The rate of 40-point deduction (40 points per kWh per person) was defined when weather forecast for either the preceding or current day included “sunny”. The rate of 80-point deduction (80 points per kWh per person) was defined when weather forecast for neither the preceding nor current day included “sunny”.

-

Then, experimental data of 22 control participants and 28 treated participants regarding hourly electric power consumption, frequency of access to tablet PCs, and weather data were recorded. In addition, household characteristics were collected through a pre-experimental questionnaire survey. Figure 5 depicts the experimental procedure in July and August 2016.

4. Research Method and Data Description

The previous field experiment in the summer of 2015 and winter of 2016, which was assigned by means of administrator selection or local leaders who decided which households should get control or treatment, was defined as a quasi-experiment with the lack of random assignment. Meanwhile, the experiment in summer of 2016 divided those confirmed households again into control and treated groups based on their intentions, which could be motivated by some individual or economic factors. This could meet a challenge of selection bias or hidden bias that may lead to bias estimations, since true experimental designs are not always possible. (“Hidden bias is essentially a problem created by the omission in statistical analysis of important variables, and omission renders nonrandom the unobserved heterogeneity reflected by an error term in regression equations” [11] (p. 357).) Therefore, this study applied the propensity score analysis with non-parametric regression developed by Heckman et al. [12,13] with the fundamental aim to improve the problems of selection bias in investigating changes of a consumer’s behavior by an application of monetary incentive namely dynamic pricing.

This method is also called kernel-based matching which allows estimations of average treatment effects for the treated using information from all possible control subjects within a predetermined span. A three-step analytic process was employed. The best conditioning variables that are speculated to be causing an imbalance between treated and control groups and the propensity scores P(X) were investigated in the first step. Then, an analysis of weighted mean differences using kernel or local linear matching through the non-parametric regression were employed to match on P(X). Finally, sensitivity analyses and balancing test based on the matched samples were conducted.

Firstly, the propensity scores were estimated based on the predicted probability for all observations derived from the fitted regression model or propensity score model. In detail, the propensity score model of binary logistic regression was conducted. A binary logistic regression describes the conditional probability of receiving treatment as follows:

$$P\left(D_i/X_i=x_i\right)=\frac{e^{{\beta }_i{X}_i}}{1+e^{{\beta }_i{X}_i}}=\frac{1}{1+e^{-{\beta }_i{X}_i}}$$

(1)

where D_i denotes treatment variables including Treateffect, Dum_20_elas, Dum_40_elas, and Dum_80_elas. X_i is the observable vector of control variables.

After an estimation of propensity scores, a matching algorithm must be defined to estimate the missing counterfactual for each treated observation. In this study, to take advantage of the panel data, the kernel-based matching algorithm (including kernel and local linear matching), which was developed from non-parametric regression methods, was used to identify the treatment effect for the treated (ATT). Specifically, kernel matching uses a kernel estimator for constructing the weighted mean for a focal point while the local linear matching using local linear regression (or lowess) with a tricube kernel function for constructing a smooth local linear regression to produce the smooth curve [12,13]. These approaches allow one-to-many matching by calculating the weighted average of the outcome variable for all control cases and then comparing that weighted average with the outcome of the treated cases. The difference between two terms provides an estimate of the average treatment effect for the treated (ATT) which is given by:

ATT= (1/n)∑[Y_ij − ∑W (i, j)Y_0j]

(2)

To estimate a treatment effect for each treated case i of treatment subjects, the average of the outcome Y_1i (denoting the outcome for treatment group) was compared with an average of the outcome Y_0i (denoting the outcome for the control group) for matched case j of the control subjects in the untreated sample. Matches were constructed based on the term of W (i, j), which is defined as the weight derived from the distance of propensity score P(X) estimated by the binary logistic regression on covariates X between a treated case i and each untreated case j. The W (i, j) was determined by non-parametric regression methods. According to Fan [14], local linear regression is expected to have more promising sampling properties and a higher minimax efficiency compared with kernel matching. Therefore, the local linear regression estimator was deployed in this study to determine the value of W (i, j) and then the average treatment effect for the treated after matching on P(X). In general, this method uses propensity scores derived from multiple matches to calculate a weighted mean that is used as a counterfactual. This implies that kernel-based matching using local linear regression constructs matches using all individuals in the potential control sample in such a way that it takes more information from those who are close to matches and down-weights more distal observations. Furthermore, the matching procedure ensures that the treated subjects will be matched to the control subjects that are most similar to them in terms of characteristics and, therefore, dissimilar subjects and outliers have no influence on the treatment effects.

Upon completing the matching estimation, sensitivity analyses and balancing test for propensity score matching were employed in the final step to check the robustness and adequacy of the results. Particularly, sensitivity analyses of different bandwidth specifications and different trimming schedules were used to confirm the results and test the sensitivity of findings to variations. In terms of bandwidth analysis (which is defined as the fraction that is used to determine the number of observations that falls into a span), three values of 0.01, 0.05, and 0.8 were used. Regarding trimming analysis (which is considered to impose a common support by dropping treated observations whose propensity scores fall outside the lower end of the common support region and non-treated observations whose propensity scores fall outside the upper end of the common support region), three trimming schedules to discard 2%, 5%, and 10% of study observations at the two ends were used [11]. Furthermore, the balancing test was applied to check whether the propensity score is an adequate balancing score or the overall quality of estimation. Among the variety of balance tests, the standardized test of differences was employed in this study. The test was first mentioned by Rosenbaum and Rubin [15] to check the balance between the treated and control group using the following formula for the standardized differences:

$$B_{before}\left(X\right)=\frac{\overline{X_T}-\overline{X_C}}{\sqrt{\frac{V_T\left(X\right)+V_C\left(X\right)}{2}}}\times 100$$

(3)

$$B_{after}\left(X\right)=\frac{\overline{X_{TM}}-\overline{X_{CM}}}{\sqrt{\frac{V_T\left(X\right)+V_C\left(X\right)}{2}}}\times 100$$

(4)

where, for each covariate, $\overline{X_T}$ and $\overline{X_C}$ are the sample means for the full treated and control groups, $\overline{X_{TM}}$ and $\overline{X_{TM}}$ are the sample means for the matched treated and control groups, and V_T(X) and V_C(X) are the corresponding sample variances. B_before(X) and B_after(X) are defined as the percentage of the standardized difference or bias between the treated and control groups before and after matching, respectively. The standardized difference is considered the size of the difference in means of a conditioning variable X_i between the treated and control group, divided by the square root of the variances in the original samples, which allows comparisons in the differences in X before and after matching. They also suggest that the matching quality can be evaluated by a reduction in the standardized difference. If the differences remain, then either the propensity score model should be estimated using a different approach, or a different matching algorithm should be used, or both.

The authors believe that using all observations in full panel data may provide the best matching since subjects with different demographic characteristics may reach the same action or behavior in hourly electric consumption in the same experimental period while subjects with the same demographic characteristics may vary in their action or behavior during the pre-experimental period and during the experimental period.

4.1. Data Description

This study used panel data of hourly electricity consumption among households 14 days before the experiment and 14 days during the experiment.

Table 1. Description of the outcome variable, treatment variables and control variables in the study.

Abbreviation	Brief Definition	Unit	Source
Lneleccon	Logarithm of hourly electric power consumption	Wh	Smart meter
Treatgroup	Dummy variable, 1 denotes treated group and 0 denotes control group	-	-
Withintreat	Dummy variable, 1 denotes experimental period (20th July to 2nd August) and 0 denotes pre-experimental period (from 6th July to 19th July)	-	-
Treateffect	= Withintreat∙Treatgroup, 1 denotes the treated group during experimental period and 0 denotes otherwise	-	-
Dum_20	20-point deduction day dummy during the experimental period	-	-
Dum_20_elas	= Dum_20∙Treatgroup, 1 denotes treated group in 20-point deduction days during the experimental period and 0 denotes otherwise	-	-
Dum_40	40-point deduction day dummy during the experimental period	-	-
Dum_40_elas	= Dum_40∙Treatgroup, 1 denotes treated group in 40-point deduction days during the experimental period and 0 denotes otherwise	-	-
Dum_80	80-point deduction day dummy during the experimental period	-	-
Dum_80_elas	= Dum_80∙Treatgroup, 1 denotes treated group in 80-point deduction days during the experimental period and 0 denotes otherwise	-	-
Happy-e	Happy-e contract dummy (discount after 10 p.m.)	-	Questionnaire survey
Aircon	Number of air-conditioners	Unit	Questionnaire survey
Refrigerator	Number of refrigerators	Unit	Questionnaire survey
Com_refrigerator	Number of commercial refrigerators	Unit	Questionnaire survey
Wood	Wooden house dummy	-	Questionnaire survey
Member	Household members	Person	Questionnaire survey
Accesstimes	Frequency of access to tablet PC	Times/hour	Smart meter
District_n	Regional dummy variables	District 1~4	Questionnaire survey
Period_n	Hour dummy variables	Period 1~7	Questionnaire survey
Wind	Hourly mean wind speed	m/s	Japan Meteorological Agency
Cool_d	Cooling degree hour	Degree	Japan Meteorological Agency
Temp	Hourly mean temperature	Degree	Japan Meteorological Agency

presents the brief definition and source of measurement of outcome variable, treatment variables, and control variables in the study.

Regarding the outcome variable of Lneleccon, the logarithm of hourly electric power consumption was used to estimate the percentage of change in electricity consumption between the control and treated group.

With respects to treatment variables, the study used four main treatment variables of Treateffect, Dum_20_elas, Dum_40_elas, and Dum_80_elas to estimate the treatment effects of dynamic pricing with specific deduction rate on consumer behavior change in electric power consumption.

In terms of control variables, some demographic variables associated with the number of people in family, the number of air conditioners, the number of refrigerators and commercial refrigerators were expected to have positive impacts on electric power consumption. In addition, electric power consumption was expected to be more when households are living in wooden houses. Moreover, the variable related to real-time feedback (i.e., frequency of access to tablet PCs) was hypothesized to have a negative impact on electric power consumption. This means that the more a household accessed the tablet PCs to check the real-time feedback information and real consumption, the more energy they saved. Some other variables refer to weather conditions, namely the cooling degree hour, hourly mean wind speed, and hourly mean temperature, that were also expected to affect consumers’ electric power consumption.

The statistical descriptions of the hourly electric power consumption, the frequency of access, and demographic characteristics between control and treated groups in the study are shown in

Table 2. Descriptive statistics of experimental data.

Variables	Control Group			Treated Group
	Observations	Mean	S. D.a	Observations	Mean	S. D.
Hourly electric power consumption	14,589	783.0077	722.8344	18,761	615.5162	438.9045
Happy-e contract	22	0.2713	0.4447	28	0.3932	0.4885
Air conditioners	21	2.6217	0.6513	28	2.3572	0.6105
Refrigerators	20	1.3498	0.5725	27	1.3338	0.5447
Commercial refrigerators	21	0.5253	0.8527	28	0.2863	0.5253
Wooden house	19	0.8427	0.3641	28	0.7858	0.4103
Household members	22	2.5457	1.0332	27	2.9261	1.1202
Frequency of access	14,742	0.0016	0.0473	18,775	0.0180	0.1655

Note: ^a Standard deviation.

.

To estimate the differences between the control and treated participants, the difference test in means was used. The p-value of t-statistics in

Table 3. Differences between the control and treatment groups.

Variables	Difference (Control vs. Treatment)	p-Value
		H: Diff < 0	H: Diff ≠ 0	H: Diff > 0
Hourly electric power consumption	167.4915 (6.4073)	1.0000	0.0000	0.0000
Happy-e contract	−0.1219 (0.0052)	0.0000	0.0000	1.0000
Air conditioners	0.2645 (0.0070)	1.0000	0.0000	0.0000
Refrigerators	0.0160 (0.0063)	0.9941	0.0118	0.0059
Commercial refrigerators	0.2389 (0.0076)	1.0000	0.0000	0.0000
Wooden house	0.0569 (0.0045)	1.0000	0.0000	0.0000
Household members	−0.3804 (0.0120)	0.0000	0.0000	1.0000
Frequency of access	−0.0164 (0.0014)	0.0000	0.0000	1.0000

Note: Standard errors in parentheses.

shows that there is a statistically significant difference between the control and treated participants in major variable of hourly electric power consumption and all observable variables at the either significant level of 0.01 or 0.05. More specifically, the average hourly consumption of the control participants is significantly higher than the treated participants. The numbers of air conditioners, refrigerators, and commercial refrigerators used by the control participants are substantially more than those used by the treated participants. Additionally, the control participants tend to live in wooden houses compared to the treated participants. On the other hand, the number of household members in the control group and the frequency of access to tablet PCs are clearly smaller than those in the treated group. Furthermore, the ownership of a happy-e contract (discount after 22:00) within the control group is significantly smaller than that of the treated group. Consequently, these differences imply that there is a clear existence of selection bias in the experimental design. The employment of the propensity score analysis approach is therefore necessary to mitigate the problem of selection bias in estimating the effects of treatment in the study.

4.2. Difference Test in Means among Treatment Variables

In term of the treatment variable of Treateffect, the result in

Table 4. Difference test of means in logarithm of hourly consumption by Treateffect variable.

Treateffect 0 (n=23,972)	Treateffect 1 (n= 9378)	Difference (0 vs. 1)	p-Value
Mean	Mean		H: Diff < 0	H: Diff ≠ 0	H: Diff > 0
6.2858	6.1909	0.0949 (0.0090)	1.0000	0.0000	0.0000

Note: Standard errors in parentheses.

reports that the average difference in the hourly electric power consumption between the treated participants during the experimental period and of both during the pre-experimental period and the control participants is statistically positive at the significant level of 0.01. This means that the treated participants’ hourly electric power consumption during the experimental period is 9.5% lower than theirs during the pre-experimental period and the control participants’ consumption.

In terms of the effects of dynamic pricing with three different rates of deduction point, the result in

Table 5. Difference test of means in logarithm of hourly consumption by Dum_20_elas variable.

Dum_20_elas 0 (n=18,775)	Dum_20_elas 1 (n= 2680)	Difference (0 vs. 1)	p-Value
Mean	Mean		H: Diff < 0	H: Diff ≠ 0	H: Diff > 0
6.2568	6.2173	0.0395 (0.0152)	0.9953	0.0093	0.0047

Note: Standard errors in parentheses.

shows that the average difference in the hourly electric power consumption between the treated participants in 20-point deduction days during the experimental period and both themselves during the pre-experimental period and the control participants is statistically positive at the significant level of 0.01. This implies that the treated participants’ hourly electric power consumption in the 20-point deduction days during the experimental period is less than theirs during the pre-experimental period as well as the control participants’ consumption by approximately 4%.

Similarly, the result in

Table 6. Difference test of means in logarithm of hourly consumption by Dum_40_elas variable.

Dum_40_elas 0 (n= 18,782)	Dum_40_elas 1 (n= 2677)	Difference (0 vs. 1)	p-Value
Mean	Mean		H: Diff < 0	H: Diff ≠ 0	H: Diff > 0
6.2538	6.1836	0.0702 (0.0152)	1.0000	0.0000	0.0000

Note: Standard errors in parentheses.

indicates that the average hourly electric power consumption of the treated participants in 40-point deduction days during the experimental period is significantly lower than that of themselves during the pre-experimental period as well as that of the control participants by 7%.

The result in

Table 7. Difference test of means in logarithm of hourly consumption by Dum_80_elas variable.

Dum_80_elas 0 (n= 19,831)	Dum_80_elas 1 (n=4021)	Difference (0 vs. 1)	p-Value
Mean	Mean		H: Diff < 0	H: Diff ≠ 0	H: Diff > 0
6.2632	6.1781	0.0851 (0.0128)	1.0000	0.0000	0.0000

Note: Standard errors in parentheses.

presents that the average hourly electric power consumption of the treated participants in 80-point deduction day during the experimental period is significantly lower than that of themselves during the pre-experimental period as well as that of the control participants by 8.5%.

According to the results of the difference test in means, the incentive of dynamic pricing via real-time feedback with specific deduction rate demonstrates significant impacts on participating consumers’ energy-saving or reduction of their electric power consumption.

5. Empirical Results and Discussion

5.1. Binary Logistic Models

To reduce the problem of selection bias in the field experiment, the propensity score analysis with local linear matching was employed to estimate the treatment effects of dynamic pricing on consumer behavior in electric power consumption.

The binary logistic models are used to provide propensity score estimates in the first step.

Table 8. Binary logistic models for propensity score estimates.

Variable	Model 1 (Treateffect)		Model 2 (Dum_20_elas)		Model 3 (Dum_40_elas)		Model 4 (Dum_80_elas)
	Coefficient	Marginal Effects	Coefficient	Marginal Effects	Coefficient	Marginal Effects	Coefficient	Marginal Effects
happy-e	0.3328 *** (0.0293)	0.0618 ***	0.2560 *** (0.0484)	0.0267 ***	0.2503 *** (0.0480)	0.0266 ***	0.2730 *** (0.0409)	0.0360 ***
aircon	−0.2716 *** (0.0226)	−0.0504 ***	−0.2176 *** (0.0374)	−0.0227 ***	−0.2188 *** (0.0370)	−0.0233 ***	−0.2321 *** (0.0316)	−0.0306 ***
refrigerator	−0.1072 *** (0.0279)	−0.0199 ***	−0.0966 ** (0.0459)	−0.0101 ***	−0.0883 * (0.0455)	-0.0094 *	−0.0942 *** (0.0389)	−0.0124 ***
com_refrigerator	−0.2739 *** (0.0261)	−0.0508 ***	−0.2202 *** (0.0439)	−0.0230 ***	−0.2188 *** (0.0435)	−0.0233 ***	−0.2383 *** (0.0370)	−0.0314 ***
wood	−0.7220 *** (0.0404)	−0.1340 ***	−0.5681(0.0660)	−0.0593	−0.5529 *** (0.0654)	−0.0589 ***	−0.6081 *** (0.0559)	−0.0803 ***
member	0.3060 *** (0.0145)	0.0568 ***	0.2514 *** (0.0245)	0.0263 ***	0.2485 *** (0.0243)	0.0265 ***	0.2648 *** (0.0206)	0.0349 ***
accesstimes	2.0016 *** (0.1466)	0.3716 ***	1.6018 *** (0.1887)	0.1673 ***	1.8024 *** (0.1872)	0.1919 ***	1.8812 *** (0.1713)	0.2483 ***
district1 (South)	0.6538 *** (0.0546)	0.1214 ***	0.5032 *** (0.0917)	0.0526 ***	0.4957 *** (0.0910)	0.0528 ***	0.5506 *** (0.0773)	0.0727 ***
district2 (Central)	0.3962 *** (0.0486)	0.0735 ***	0.3054 *** (0.0818)	0.0319 ***	0.3144 *** (0.0811)	0.0335 ***	0.3455 *** (0.0689)	0.0456 ***
district3 (North)	0.3314 *** (0.0529)	0.0615 ***	0.2616 *** (0.0897)	0.0273 ***	0.2711 *** (0.0890)	0.0289 ***	0.2855 *** (0.0755)	0.0377 ***
district4 (East)	0.7444 *** (0.0515)	0.1382 ***	0.5949 *** (0.0858)	0.0621 ***	0.5820 *** (0.0851)	0.0620 ***	0.6405 *** (0.0725)	0.0845 ***
period1 (0–3 a.m.)	0.1413 *** (0.0542)	0.0262 ***	0.0971 (0.0902)	0.0101	0.0038 (0.0894)	0.0004	0.0684 (0.0761)	0.0090
period2 (3–6 a.m.)	0.3114 *** (0.0558)	0.0578 ***	0.3108 *** (0.0926)	0.0325 ***	0.0413(0.0916)	0.0044	0.3746 *** (0.0788)	0.0494 ***
period3 (6–9 a.m.)	−0.4632 *** (0.0565)	−0.0860 ***	−0.6736 *** (0.0954)	−0.0704 ***	−0.6515 *** (0.0951)	−0.0694 ***	−0.5733 *** (0.0794)	−0.0757 ***
period4 (9 a.m.–12 p.m.)	−1.3135 *** (0.0666)	−0.243 8***	−1.7584 *** (0.1126)	−0.1837 ***	−1.7074 *** (0.1163)	−0.1818 ***	−1.4801 *** (0.0931)	−0.1953 ***
period5 (12–15 p.m.)	−1.5480 *** (0.0695)	−0.2874 ***	−1.9676 *** (0.1145)	−0.2055 ***	−1.8591 *** (0.1167)	−0.1979 ***	−1.7313 *** (0.0988)	−0.2285 ***
period6 (15–18 p.m.)	−1.1691 *** (0.0634)	−0.2170 ***	−1.5705 *** (0.1065)	−0.1640 ***	−1.4415 *** (0.1071)	−0.1535 ***	−1.3223 *** (0.0899)	−0.1745 ***
period7 (18–21 p.m.)	−0.2920 *** (0.0545)	−0.0542 ***	−0.3953 *** (0.0906)	−0.0413 ***	−0.2956 *** (0.0900)	−0.0315 ***	−0.3664 *** (0.0764)	−0.0484 ***
wind	0.2505 *** (0.0114)	0.0465 ***	0.2805 *** (0.0168)	0.0293 ***	0.2163 *** (0.0179)	0.0230 ***	0.3255 *** (0.0157)	0.0430 ***
cool_d	0.0092 (0.0522)	0.0017	−0.3293 *** (0.1143)	−0.0344 ***	0.8315 *** (0.0668)	0.0885 ***	−1.0513 *** (0.1188)	−0.1387 ***
temp	0.3054 *** (0.0483)	0.0567 ***	0.7554 *** (0.1073)	0.0789 ***	−0.3796 *** (0.0578)	−0.0404 ***	1.3660 *** (0.1154)	0.1803 ***
constant	−8.9565 *** (1.1551)	-	−20.8767 *** (2.5691)	-	6.3469 *** (1.3769)	-	−35.0739 *** (2.7451)	-
Number of observations	29,446	-	18,916	-	18,915	-	21,027	-
LR chi2 (21)	3280.93	-	1507.21	-	1164.17	-	1996.10	-
Pseudo R2	0.0917	-	0.1022	-	0.0791	-	0.1014	-
Probability > chi2	0.0000	-	0.0000	-	0.0000	-	0.0000	-

Note: Standard error in the parentheses. *, **, and *** mean significance with confidence interval at 90%, 95%, and 99%, respectively.

shows the results of the binary logistic models for four major treatment variables.

According to household characteristics, the variables of air conditioners, refrigerators, and commercial refrigerators are negatively significant in the binary logistic models. This means that the control households often have more air conditioners, refrigerators, and commercial refrigerators than the treated group. Additionally, the control households are more likely to live in wooden houses than the treated group.

On the other hand, the treated households are more likely to have a happy-e contract with Kansai Electric Power Company than the control group.

The frequency of access to tablet PCs has a positive significance. For instance, the treated households during the experimental period have higher frequency of access to tablet PCs than themselves during the pre-experimental period and the control group by 37.16% (Model 1 in Table 8). According to surveyed data, the treated group are more conscious of energy saving than the control group, thus the frequency of access to tablet PCs is different between two groups (Figure 6). Particularly, the treatment group increased the frequency of access to tablet PCs since they were more conscious of energy saving. Meanwhile, the control group increased the frequency of access to tablet PCs because they tended to think about the large amount of electricity consumption.

5.2. The Average Treatment Effects for the Treated by Propensity Score Matching

After propensity scores are estimated, the matching algorithm of local linear regression is chosen for the estimation of treatment effects by calculating the difference between the weighted average of the outcome variable for treated subjects and the weighted average of the outcome variable for all control subjects.

Table 9. Estimated average treatment effects for the treated electric power consumption: Propensity score analysis with local linear regression.

Group and Comparison	Dynamic Pricing Effect (Treateffect)	Deduction Rate of 20-point (Dum_20_elas)	Deduction Rate of 40-point (Dum_40_elas)	Deduction Rate of 80-point (Dum_80_elas)
Treated group	6.2393	6.2687	6.2331	6.2240
Control group	6.3307	6.3024	6.2993	6.3091
Unadjusted mean difference (=treated − control)	−0.0913 **	−0.0338 **	−0.0662 **	−0.0851 **
Adjusted mean difference
DID point estimate (bias correct 95% confidence interval)	−0.0957 ** (−0.1178; −0.0774)	−0.0494 ** (−0.0776; −0.0211)	−0.0739 ** (−0.1069; −0.0409)	−0.0980 ** (−0.1195; −0.0766)
Changing bandwidth
Small bandwidth = 0.01	−0.0951 ** (−0.1154; −0.0748)	−0.0529 ** (−0.0870; −0.0189)	−0.0747 ** (−0.1064; −0.0430)	−0.0976 ** (−0.1222; −0.0730)
Small bandwidth = 0.05	−0.0951 ** (−0.1127; −0.0776)	−0.0496 ** (−0.0781; −0.0212)	−0.0738 ** (−0.1048; −0.0428)	−0.0984 ** (−0.1278; −0.0690)
Large bandwidth = 0.8	−0.0969 ** (−0.1164; −0.0775)	−0.0470 ** (−0.0752; −0.0019)	−0.0773 ** (−0.1040; −0.0505)	−0.0925 ** (−0.1165; −0.0685)
Trimming
2%	−0.0966 ** (−0.1135; −0.0797)	−0.0490 ** (−0.0820; −0.0160)	−0.0749 ** (−0.1029; −0.0469)	−0.0982 ** (−0.1228; −0.0735)
5%	−0.0986 ** (−0.1168; −0.0804)	−0.0534 ** (−0.0788; −0.0279)	−0.0761 ** (−0.1104; −0.0417)	−0.0937 ** (−0.1215; −0.0658)
10%	−0.1059 ** (−0.1232; −0.0886)	−0.0607 ** (−0.0905; −0.0309)	−0.0865 ** (−0.1171; −0.0559)	−0.1097 ** (−0.1378; −0.0816)

Note: ** The 95% confidence interval does not include a zero, or p < 0.05 for a two-tailed test.

shows the estimated average treatment effects for the treated group (ATT) on electric power consumption.

More specifically, the result of the point estimate of the ATT of dynamic pricing (based on Treateffect variable) on electric power consumption (Table 9) is −0.0957, which falls into a 95% bootstrap confidence interval bounded by −0.1178 and −0.0774. That means there is a significant difference on electric power consumption between treated participants and others. More specifically, the treated participants tended to reduce their electric power consumption during the experimental period by 9.6% compared to theirs during the pre-experimental period as well as the control participants. This result is consistent with the result from Thoa et al. [4] which pointed out the dynamic pricing had a significant effect on constraining the energy demand of consumers by reducing their electric power consumption by 13.7% in the same field experiment on Nushima Island in the summer of 2015 and the winter of 2016 through a panel random analysis (i.e., the difference in differences method).

Regarding consumption response among various deduction rates, the point estimates of the ATT of the three specific deduction rates on electric power consumption (Table 9) fall into a 95% bootstrap confidence interval. This means that the incentive of dynamic pricing with the three specific deduction rates is significantly effective in reducing consumers’ electric power consumption. Conversely, the deduction rates during this experimental period have the same effects as tariffs, which can encourage an energy-saving effect. More specifically, the higher the tariff (i.e., deduction rate) is, the less electric power the household consumes. In terms of the incentive of the 20-point-deduction rate, treated households are more likely to reduce their consumption by 5.0% during the experimental period than theirs during the pre-experimental period as well as the control households. With respect to the incentive of the 40-point-deduction rate, treated households tend to reduce their consumption by 7.2% during the experimental period than theirs during the pre-experimental period as well as the control households. Regarding the 80-point-deduction rate, treated households are also more likely to reduce their consumption, by 9.8%, during the experimental period than theirs during the pre-experimental period as well as the control households. This result is in line with the theoretical framework and that reported by Shimada et al. [16], which demonstrated that there was a positive correlation between the energy-saving effect and the deduction rate in the South District of Nushima Island, Japan when the frequency of access to tablet PCs was double the average or more. On the other hand, Thoa et al. [4] showed the opposite relation between the deduction rate and energy-saving effect, which might be caused by the experiment period difference.

The values of adjusted mean difference (in Table 9) from the propensity score analysis with local linear matching (i.e., the values of adjusted mean difference in Table 9) produce the same substantive findings as both of the values from the difference test in means (Table 4, Table 5, Table 6 and Table 7) and the values of unadjusted mean differences (in Table 9) for four treatment variables of Treateffect, Dum_20_elas, Dum_40_elas, and Dum_80_elas. Although the estimation of propensity score analysis with local linear matching is slightly larger, all three estimators find the treatment effects of dynamic pricing statistically significant and thus lead to a consistent conclusion regarding research hypotheses. This also implies that the propensity score analysis approach with non-parametric regression is proper for estimating the average treatment effects of the dynamic pricing incentive on electric power consumption.

The positive relation between the dynamic pricing with specific deduction rate and the energy-saving effect is shown in Figure 7.

In practice, in terms of the dynamic pricing incentive with specific deduction rates, the majority of participants tended to check and calculate the remaining points every day (17.9%) or sometimes (37.9%) (Figure 8). This may lead to some activities related to energy-saving to manage electric power consumption among participants. Particularly, many participating households (46.7%) preferred to reduce their electric power consumption on high-point rate days. Others (26.7%) planned to shift their work and actions related to high electricity consumption to low-point rate days (Figure 9).

5.3. Sensitivity Analyses and Balancing Test

5.3.1. Sensitivity Analyses

Finally, sensitivity analyses of different bandwidth specifications and different trimming schedules were employed to confirm the results of the study. In terms of specific bandwidth levels (0.01, 0.05, and 0.8), all analyses show a 95% bootstrap confidence interval bounded by non-zero difference-in-differences estimates. Similarly, with respects to specific trimming schedules (by dropping 2%, 5%, and 10% of treated observations whose propensity scores are higher than the maximum or less than the minimum propensity scores of the non-treated observation), all analyses depict a 95% bootstrap confidence interval bounded by non-zero difference-in-differences estimates. The two sensitivity analyses imply that meaningful effects are reasonably unlikely to occur by chance as indicated by a 95% confidence interval that does not include a zero. These also confirm the importance of analyzing the behavioral change in electric power consumption between the control and treated groups in different periods of the experiment using a corrective procedure such as propensity score analysis with non-parametric regression (e.g., local linear regression).

5.3.2. Balancing Test

Regarding the balancing test, the test of standardized differences is used to illustrate the reduction in difference or bias of each covariate before and after matching for four propensity score models (

Table A1. Result of balancing test of propensity score model of Treateffect variable.

Variables	Sample	Mean			Percent Reduction Bias	t-Test
		Treated	Control	% Bias		t	p-Value
happy-e	Unmatched	0.4233	0.3396	17.30		13.66	0.0000
	Matched	0.4233	0.4216	0.40	97.90	0.23	0.8180
aircon	Unmatched	2.3467	2.4667	−18.70		−14.43	0.0000
	Matched	2.3467	2.3093	5.80	68.80 ***	3.64	0.0000
refrigerator	Unmatched	1.3466	1.3407	1.10		0.82	0.4120
	Matched	1.3466	1.3318	2.60	−151.10 *	1.75	0.0800
com_refrigerator	Unmatched	0.3085	0.3909	−12.90		−9.51	0.0000
	Matched	0.3085	0.2678	6.30	50.70 ***	4.63	0.0000
wood	Unmatched	0.7693	0.8388	−17.60		−14.14	0.0000
	Matched	0.7693	0.7740	−1.20	93.20	−0.74	0.4590
member	Unmatched	3.0001	2.8123	17.50		13.65	0.0000
	Matched	3.0001	2.9768	2.2	87.60	1.45	0.1480
accesstimes	Unmatched	0.0331	0.0024	18.70		18.18	0.0000
	Matched	0.0331	0.0250	4.90	73.80 **	2.46	0.0140
district1	Unmatched	0.2307	0.1939	9.00		7.13	0.0000
	Matched	0.2307	0.2295	0.30	96.90	0.18	0.8570
district2	Unmatched	0.2308	0.2273	0.80		0.65	0.5140
	Matched	0.2308	0.2371	−1.50	−80.30	−0.98	0.3250
district3	Unmatched	0.1928	0.2379	−11.00		−8.47	0.0000
	Matched	0.1928	0.2012	−2.00	81.40	−1.39	0.1640
district4	Unmatched	0.2308	0.1946	8.90		7.02	0.0000
	Matched	0.2308	0.2294	0.30	96.20	0.22	0.8290
period1	Unmatched	0.1252	0.1259	−0.20		−0.17	0.8620
	Matched	0.1252	0.1352	−3.00	−1258.70 **	−1.96	0.0500
period2	Unmatched	0.1252	0.1262	−0.30		−0.23	0.8180
	Matched	0.1252	0.1283	−0.90	−216.90	−0.61	0.5390
period3	Unmatched	0.1251	0.1261	−0.30		−0.25	0.8050
	Matched	0.1251	0.1358	−3.20	−922.30 **	−2.09	0.0360
period4	Unmatched	0.1249	0.1240	0.20		0.19	0.8460
	Matched	0.1249	0.1168	2.40	−883.80	1.63	0.1040
period5	Unmatched	0.1249	0.1236	0.40		0.31	0.7570
	Matched	0.1249	0.1152	2.90	−640.10 **	1.96	0.0500
period6	Unmatched	0.1249	0.1242	0.20		0.15	0.8830
	Matched	0.1249	0.1191	1.70	−822.40	1.16	0.2470
period7	Unmatched	0.1249	0.1260	−0.40		−0.28	0.7810
	Matched	0.1249	0.1200	1.50	−309.70	0.97	0.3320
wind	Unmatched	2.3529	1.9699	29.40		22.67	0.0000
	Matched	2.3529	2.3515	0.10	99.60	0.07	0.9440
cool_d	Unmatched	2.7000	2.1364	28.00		22.24	0.0000
	Matched	2.7000	2.6127	4.30	84.50 ***	2.84	0.0040
temp	Unmatched	26.6260	25.9960	29.20		22.96	0.0000
	Matched	26.6260	26.5310	4.40	85.00 ***	2.91	0.0040

Note: *, **, and *** mean significance with confidence interval at 90%, 95%, and 99%, respectively.

,

Table A2. Result of balancing test of propensity score model of Dum_20_elas variable.

Variables	Sample	Mean			Percent Reduction Bias	t-Test
		Treated	Control	% Bias		t	p-Value
happy-e	Unmatched	0.4234	0.3548	14.10		6.63	0.0000
	Matched	0.4234	0.4226	0.20	98.80	0.06	0.9540
aircon	Unmatched	2.3467	2.4446	−15.40		−6.98	0.0000
	Matched	2.3467	2.3323	2.30	85.20	0.76	0.4450
refrigerator	Unmatched	1.3467	1.3414	1.00		0.44	0.6590
	Matched	1.3467	1.3579	−2.00	−110.50	−0.7	0.4810
com_refrigerator	Unmatched	0.3082	0.3748	−10.70		−4.56	0.0000
	Matched	0.3082	0.2913	2.70	74.70	1.04	0.3000
wood	Unmatched	0.7693	0.8259	−14.10		−6.84	0.0000
	Matched	0.7693	0.7568	3.10	78.00	1.03	0.3020
member	Unmatched	3.0008	2.8470	14.30		6.63	0.0000
	Matched	3.0008	3.0064	−0.50	96.30	−0.19	0.8530
accesstimes	Unmatched	0.0321	0.0027	16.90		13.03	0.0000
	Matched	0.0321	0.0249	4.10	75.40	1.09	0.2750
district1	Unmatched	0.2307	0.2007	7.30		3.47	0.0010
	Matched	0.2307	0.2496	−4.60	37.30	−1.56	0.1190
district2	Unmatched	0.2299	0.2276	0.50		0.25	0.7990
	Matched	0.2299	0.2332	−0.80	−40.00	−0.27	0.7880
district3	Unmatched	0.1926	0.2303	−9.20		−4.2	0.0000
	Matched	0.1926	0.1782	3.50	61.70	1.31	0.1900
district4	Unmatched	0.2311	0.2012	7.30		3.45	0.0010
	Matched	0.2311	0.2227	2.00	71.90	0.71	0.4780
period1	Unmatched	0.1252	0.1256	−0.10		−0.06	0.9550
	Matched	0.1252	0.1425	−5.20	−4173.10 *	−1.79	0.0740
period2	Unmatched	0.1252	0.1260	−0.30		−0.12	0.9070
	Matched	0.1252	0.1336	−2.50	−912.50	−0.89	0.3760
period3	Unmatched	0.1248	0.1259	−0.30		−0.16	0.8760
	Matched	0.1248	0.1332	−2.50	−658.40	−0.89	0.3750
period4	Unmatched	0.1252	0.1246	0.20		0.09	0.9290
	Matched	0.1252	0.1108	4.40	−2168.70	1.58	0.1140
period5	Unmatched	0.1248	0.1241	0.20		0.09	0.9250
	Matched	0.1248	0.1188	1.80	−806.60	0.65	0.5160
period6	Unmatched	0.1244	0.1249	−0.10		−0.07	0.9470
	Matched	0.1244	0.1164	2.4	−1600.90	0.87	0.3840
period7	Unmatched	0.1252	0.1258	−0.20		−0.08	0.9340
	Matched	0.1252	0.1200	1.60	−788.00	0.56	0.5750
wind	Unmatched	2.4347	1.8814	37.80		18.23	0.0000
	Matched	2.4347	2.4308	0.30	99.30	0.09	0.9280
cool_d	Unmatched	2.7865	2.0104	40.70		19.15	0.0000
	Matched	2.7865	2.6690	6.20	84.90 **	2.17	0.0300
temp	Unmatched	26.7360	25.8550	43.00		19.78	0.0000
	Matched	26.7360	26.6140	6.00	86.10 **	2.16	0.0310

Note: * and ** mean significance with confidence interval at 90%and 95%, respectively.

,

Table A3. Result of balancing test of propensity score model of Dum_40_elas variable.

Variables	Sample	Mean			Percent Reduction Bias	t-Test
		Treated	Control	% Bias		t	p-Value
happy-e	Unmatched	0.4238	0.3553	14.10		6.62	0.0000
	Matched	0.4238	0.4246	−0.20	98.80	−0.06	0.9540
aircon	Unmatched	2.3466	2.4441	−15.30		−6.95	0.0000
	Matched	2.3466	2.3185	4.40	71.10	1.49	0.1360
refrigerator	Unmatched	1.3466	1.3415	0.90		0.43	0.6710
	Matched	1.3466	1.3317	2.70	−188.80	0.95	0.3420
com_refrigerator	Unmatched	0.3088	0.3747	−10.60		−4.51	0.0000
	Matched	0.3088	0.2774	5.00	52.40*	1.94	0.0530
wood	Unmatched	0.7700	0.8259	−13.90		−6.74	0.0000
	Matched	0.7700	0.7696	0.10	99.30	0.03	0.9730
member	Unmatched	3.0004	2.8468	14.30		6.61	0.0000
	Matched	3.0004	2.9891	1.00	92.70	0.38	0.7080
accesstimes	Unmatched	0.0306	0.0028	18.80		13.75	0.0000
	Matched	0.0306	0.0249	3.80	79.70	0.99	0.3230
district1	Unmatched	0.2300	0.2007	7.10		3.37	0.0010
	Matched	0.2300	0.2376	−1.90	73.90	−0.64	0.5250
district2	Unmatched	0.2312	0.2276	0.90		0.4	0.6880
	Matched	0.2312	0.2376	−1.50	−77.30	−0.54	0.5920
district3	Unmatched	0.1930	0.2301	−9.10		−4.13	0.0000
	Matched	0.1930	0.1998	−1.70	81.60	−0.61	0.5440
district4	Unmatched	0.2308	0.2014	7.10		3.38	0.0010
	Matched	0.2308	0.2163	3.50	50.70	1.23	0.2210
period1	Unmatched	0.1251	0.1257	−0.20		−0.09	0.9290
	Matched	0.1251	0.1275	−0.70	−278.10	−0.26	0.7980
period2	Unmatched	0.1251	0.1260	−0.30		−0.13	0.8950
	Matched	0.1251	0.1319	−2.10	−624.00	−0.72	0.4710
period3	Unmatched	0.1255	0.1259	−0.10		−0.06	0.9530
	Matched	0.1255	0.1275	−0.60	−379.10	−0.21	0.8310
period4	Unmatched	0.1247	0.1247	0.00		0.00	1.0000
	Matched	0.1247	0.1267	−0.60	−8,200,000.00	−0.21	0.8310
period5	Unmatched	0.1247	0.1240	0.20		0.09	0.9240
	Matched	0.1247	0.1066	5.50	−2588.00**	2.00	0.0460
period6	Unmatched	0.1255	0.1247	0.20		0.11	0.9100
	Matched	0.1255	0.1230	0.70	−200.00	0.26	0.7970
period7	Unmatched	0.1247	0.1259	−0.40		−0.18	0.8570
	Matched	0.1247	0.1166	2.40	−525.80	0.87	0.3840
wind	Unmatched	2.2466	1.8614	30.10		13.42	0.0000
	Matched	2.2466	2.1922	4.30	85.90	1.51	0.1320
cool_d	Unmatched	2.6571	1.9957	34.10		16.22	0.0000
	Matched	2.6571	2.5767	4.10	87.90	1.40	0.1610
temp	Unmatched	26.5030	25.8300	31.10		14.82	0.0000
	Matched	26.5030	26.3920	5.20	83.40*	1.73	0.0840

Note: *and ** mean significance with confidence interval at 90%and 95%, respectively.

and

Table A4. Result of balancing test of propensity score model of Dum_80_elas variable.

Variables	Sample	Mean			Percent Reduction Bias	t-Test
		Treated	Control	% Bias		t	p-Value
happy-e	Unmatched	0.4230	0.3512	14.80		8.28	0.0000
	Matched	0.4230	0.4265	−0.70	95.20	−0.30	0.7610
aircon	Unmatched	2.3467	2.4499	−16.20		−8.77	0.0000
	Matched	2.3467	2.3258	3.30	79.80	1.35	0.1760
refrigerator	Unmatched	1.3465	1.3411	1.00		0.53	0.5960
	Matched	1.3465	1.3395	1.20	−29.20	0.53	0.5940
com_refrigerator	Unmatched	0.3084	0.3784	−11.10		−5.71	0.0000
	Matched	0.3084	0.2999	1.40	87.80	0.64	0.5250
wood	Unmatched	0.7689	0.8291	−15.00		−8.65	0.0000
	Matched	0.7689	0.7762	−1.80	88.00	−0.75	0.4560
member	Unmatched	2.9995	2.8382	15.00		8.28	0.0000
	Matched	2.9995	3.0067	−0.70	95.50	−0.29	0.7710
accesstimes	Unmatched	0.0353	0.0026	19.50		16.19	0.0000
	Matched	0.0353	0.0206	8.70	55.10 ***	2.97	0.0030
district1	Unmatched	0.2311	0.1990	7.80		4.41	0.0000
	Matched	0.2311	0.2305	0.10	98.30	0.05	0.9560
district2	Unmatched	0.2311	0.2275	0.90		0.48	0.6340
	Matched	0.2311	0.2268	1.00	−18.90	0.44	0.6600
district3	Unmatched	0.1928	0.2323	−9.70		−5.25	0.0000
	Matched	0.1928	0.2035	−2.60	72.90	−1.16	0.2460
district4	Unmatched	0.2305	0.1997	7.50		4.23	0.0000
	Matched	0.2305	0.2174	3.20	57.50	1.36	0.1740
period1	Unmatched	0.1253	0.1257	−0.10		−0.06	0.9540
	Matched	0.1253	0.1151	3.10	−2843.60	1.35	0.1760
period2	Unmatched	0.1253	0.1259	−0.20		−0.11	0.9150
	Matched	0.1253	0.1261	−0.20	−26.20	−0.10	0.9170
period3	Unmatched	0.1250	0.1258	−0.20		−0.12	0.9030
	Matched	0.1250	0.1320	−2.10	−854.00	−0.90	0.3690
period4	Unmatched	0.1248	0.1247	0.00		0.02	0.9860
	Matched	0.1248	0.1170	2.30	−7134.90	1.03	0.3030
period5	Unmatched	0.1250	0.1242	0.30		0.14	0.8880
	Matched	0.1250	0.1138	3.40	−1238.40	1.50	0.1340
period6	Unmatched	0.1248	0.1246	0.10		0.03	0.9780
	Matched	0.1248	0.1167	2.40	−4754.40	1.07	0.2870
period7	Unmatched	0.1248	0.1259	−0.40		−0.20	0.8450
	Matched	0.1248	0.1336	−2.70	−654.00	−1.14	0.2550
wind	Unmatched	2.3691	1.8999	39.00		20.17	0.0000
	Matched	2.3691	2.4173	−4.00	89.70 *	−1.68	0.0930
cool_d	Unmatched	2.6710	2.0289	31.10		18.01	0.0000
	Matched	2.6710	2.5683	5.00	84.00 **	2.15	0.0320
temp	Unmatched	26.6330	25.8810	34.40		19.50	0.0000
	Matched	26.6330	26.5390	4.30	87.40 *	1.93	0.0530

Note: *, **, and *** mean significance with confidence interval at 90%, 95%, and 99%, respectively.

in Appendix A).

Before matching, there are significant differences between the treated and control groups because the p-value of most covariates is less than the significant level of 0.01, 0.05, or 0.10. This means that the null hypothesis of the mean value of two groups significantly differ before matching.

These differences are considerably reduced after matching with many of covariates receiving the p-value of the t-test larger than the significant level of 0.01, 0.05, or 0.10. This means that the null hypothesis of the mean value of each covariate being the same between the treated and control group after matching cannot be rejected. However, some other covariate differences are not eliminated. More specifically, in term of the propensity score model of Treateffect variable, the differences of some covariates—aircon, refrigerator, com_refrigerator, accesstimes, period1, period3, period5, temp, and cool_d—are statistically significant (Table A1 in Appendix A). Regarding the propensity score model of the Dum_20_elas variable, the differences of a few covariates—period1, temp, and cool_d—are statistically significant (Table A2 in Appendix A). Similarly, the differences of a few covariates—com_refrigerator, period5, and temp—are statistically significant in the propensity score model of the Dum_40_elas variable (Table A3 in Appendix A). In addition, the differences of a few covariates—accesstimes, wind, temp, and cool_d—are statistically significant in the propensity score model of the Dum_80_elas variable (Table A4 in Appendix A). Nevertheless, the bias was already rather small before matching.

In this sense, the propensity score analysis may provide proper propensity score models and adequate balancing scores. Conversely, this confirms that the overall quality of estimation and matching are sufficiently robust because of a substantial reduction in bias. This suggests that the full panel data with large observations can provide enough comparators for estimating the average treatment effects via the propensity score analysis with local linear matching.

In summary, the monetary incentive of dynamic pricing via real-time feedback was found to be effective in controlling electric power consumption. Particularly, the treatment group tended to reduce their electric power consumption during the experimental period. Furthermore, the higher the disincentive or tariff (i.e., point deduction rates) was, the less electric power treated participants consumed during the experimental period. It is worth noting that the analysis using the propensity score approach with local linear matching is accurate and robust in terms of handling the measurement errors by eliminating temporarily invariant sources of selection bias.

6. Conclusions and Policy Implications

The study estimated the effects of dynamic pricing via real-time information feedback on electric consumer behavior on a remote island of Japan and explored the potential utility of solar resources as a basis for energy policy.

Using a panel data of 50 households in the pre-experimental and experimental period, propensity score analysis with non-parametric regression (i.e., local linear regression) was employed to measure the treatment effects of the monetary incentive of dynamic pricing via real-time feedback on hourly electric power consumption. The results obtained from the propensity score analysis approach reveal that the incentive of dynamic pricing had statistically significant effects on the reduction of consumers’ electric power consumption. More specifically, treated participants tended to reduce their electric power consumption during the experimental period by 9.6% compared to both themselves during the pre-experimental period and the control participants.

Furthermore, the results confirmed that the higher the tariff (e.g., point deduction rates) is, the less electric power the household consumes. Particularly, in terms of the incentive of the 20-point-deduction rate, treated households are more likely to reduce their consumption, by 5.0%, during the experimental period than both themselves during the pre-experimental period and the control households. With respect to the incentive of the 40-point-deduction rate, treated households tend to reduce their consumption by 7.2% during the experimental period compared to themselves during the pre-experimental period and the control households. Regarding the 80-point-deduction rate, treated households are also more likely to reduce their consumption, by 9.8%, during the experimental period than themselves during the pre-experimental period and the control households.

In addition, the propensity score analysis approach with local linear matching has been reported to precisely estimate the treatment effects of dynamic pricing with specific deduction rates when using panel data with non-randomization due to the utility of information from all control participants for matching with treated participants. These major results are also consistent with previously mentioned literature that show dynamic pricing has substantial effects on consumer behavior change by reducing their electric power consumption.

The results of this study suggest the policy implications of a demand management system to accommodate the solar energy output fluctuation by shifting consumption to days having more solar radiation and reducing electric power consumption at the same time. The limitations of this study include a relatively short experiment period and a small sample size. An additional larger and longer experiment is required to accurately depict the real-life behavior of the consumers.