Comparative Evaluation of Non-Intrusive Load Monitoring Methods Using Relevant Features and Transfer Learning

Abstract

Non-Intrusive Load Monitoring (NILM) refers to the analysis of the aggregated current and voltage measurements of Home Electrical Appliances (HEAs) recorded by the house electrical panel. Such methods aim to identify each HEA for a better control of the energy consumption and for future smart grid applications. Here, we are interested in an event-based NILM pipeline, and particularly in the HEAs’ recognition step. This paper focuses on the selection of relevant and understandable features for efficiently discriminating distinct HEAs. Our contributions are manifold. First, we introduce a new publicly available annotated dataset of individual HEAs described by a large set of electrical features computed from current and voltage measurements in steady-state conditions. Second, we investigate through a comparative evaluation a large number of new methods resulting from the combination of different feature selection techniques with several classification algorithms. To this end, we also investigate an original feature selection method based on a deep neural network architecture. Then, through a machine learning framework, we study the benefits of these methods for improving Home Electrical Appliance (HEA) identification in a supervised classification scenario. Finally, we introduce new transfer learning results, which confirm the relevance and the robustness of the selected features learned from our proposed dataset when they are transferred to a larger dataset. As a result, the best investigated methods outperform the previous state-of-the-art results and reach a maximum recognition accuracy above 99% on the PLAID evaluation dataset.

1. Introduction

During the last decades, the electricity consumption in the residential sector increased steadily with the worldwide population growth and became a major ecological issue. Prior studies show that a real-time feedback down to the HEA level can help to effectively reduce consumption, with almost 15% of energy savings [1]. For consumers, the main advantages are the control and the understanding of their electricity consumption through a transparent access and a promptly forwarded information. For utilities, it can improve the load-forecasting accuracy and provide a basic scheme to set up energy management strategies [2]. In this context, Non Intrusive Load Monitoring (NILM) methods offer an efficient answer, since they provide a breakdown of the residential energy consumption without instrumenting each HEA.

Here, we are interested in event-based NILM systems, where the current and voltage measurements are recorded using a single sensor connected to the house electrical panel [3,4,5,6,7]. An event detection method is used to predict the changes in the aggregated power signals that occur at each HEA’s operating state [6]. Then, relevant features that meet the additive criterion [8] (which is required for the subsequent steps) are computed to recognize the HEA electrical signature that triggered an event using a pattern matching method.

NILM encounters several challenges that concern the correct identification of HEAs. Indeed, in a house, a broad range of HEAs can be present, and many of them can rely on the same electrical behavior [9,10,11]. Hence, discerning the most relevant and informative features is of paramount importance for any NILM application whose performances depend on the HEA signature uniqueness [4,9,12]. This is the reason why an HEA must be described by a reduced number of relevant features. To date, prior studies have focused on HEA recognition performance and often involve non-physics-related and difficult to interpret features. This is the case for the features provided by deep convolutional neural networks (CNN), which can suffer from the robustness issue with adversarial examples [13,14], which require the use of attention mechanisms [15]. However, only a few works investigate in detail the role and the meaning of Feature Selection (FS) methods in NILM problems when addressed through a pattern recognition approach [16,17,18,19,20].

Thus, this study aims at filling the gap by investigating different FS techniques to tackle the HEA identification problem in a supervised machine learning scenario when a large number of electrical features are available and when HEAs from distinct manufacturers belonging to the same device category are considered. The goal is to show the benefits provided by FS in terms of HEAs’ classification performance and interpretability, and for enhancing the generalization capability by reducing overfitting of the trained models. Thus, transfer learning [21], which deals with the generalization capability of the selected features for different datasets, is also an important NILM challenge that is investigated in this study.

This paper is organized as follows. In Section 2, the addressed problem is formulated. In Section 3, we introduce the two HEAs’ datasets considered in this study. In particular, we present a novel dataset of HEAs’ current and voltage measurements in steady-state conditions, and we detail the electrical features computed from these measurements. In Section 4, several FS methods are presented. Two of them that are novel are detailed: a heuristic forward method and a method based on a trained Dense neural network. Finally, in Section 5, the selected features are used in combination with several classification algorithms on both considered datasets to demonstrate the importance of selecting the suitable combination of features and classification algorithm. The paper is concluded with future work directions in Section 6.

2. Problem Statement

2.1. Supervised HEA Identification

From the values of a set of features describing a unique HEA signature, we aim at identifying the closest pre-registered HEA, referred to (according to the machine learning literature) as a class. To this end, we use a supervised machine learning framework trained on an annotated dataset. The overall HEAs’ recognition problem can be illustrated in the flowchart of Figure 1, which also includes references to the paper sections where each method is detailed. During the training step, several features (or descriptors) deduced from voltage and current HEAs’ measurements are first computed and used to derive a discriminative model for each HEA. Then, the most discriminant features are automatically selected to obtain the optimal classification accuracy. During the test and operating procedures, the selected features are computed from the observed measurements to predict the class of the corresponding HEA.

Since this work is motivated by HEA recognition for energy consumption estimation and prediction, the same appliance type can thus be considered from a different class when the HEA is recorded at different power levels, or when its energy consumption sufficiently differs due to a different manufacturing brand. Hence, we use a larger and more complicated classification taxonomy than those commonly used for HEA identification in the literature. From another hand, we only consider loads in steady-state condition because switching on (or off) introduces transient signals or fluctuations which are often not sufficient to accurately characterize an HEA. In fact, transients are also affected by HEA-independent factors such as the time on the voltage waveform where the HEA is switched on/off, the network impedance, supply voltage distortion, the sampling frequency or the switching on/off mechanism [22]. To deal with transient signals and state change detection for multiple HEA recognition, the reader can for example refer to a multivariate statistical approach recently proposed in [6,23].

2.2. Features Selection for HEA Identification

Let $F=\{f_1,f_2,\dots ,f_p\}$, with $f_i\in \mathbb{R},\forall i\in [1,p]$ be the overall set of p features. The FS process aims at finding the optimal subset of features $F^{\prime }\subseteq F$ that maximizes the identification accuracy such that its cardinality verifies $card\left(F^{\prime }\right)=d\le p$ [24]. This process induces several benefits such as avoiding the curse of dimensionality and overfitting phenomenon [25], improving the classification performance with the removal of non-discriminating features and decreasing the computational cost by only collecting the needed features. In contrast to the other dimensionality reduction techniques [24], FS does not change the original meaning of features and therefore allows further interpretation by a domain expert. Our work consists in evaluating several feature selection techniques when applied to an HEA recognition scenario, in terms of recognition accuracy and in terms of robustness by investigating two distinct datasets and several classification methods.

2.3. Transfer Learning

The last challenge addressed in this study concerns transfer learning [21], which consists in using the knowledge extracted from a given dataset to tackle a new problem based on a different dataset with a different setup and classification taxonomy. The investigated datasets can be of different natures since they use different recording protocols, different grid properties and different annotation taxonomies. This main motivation is to show the validity and the generalization capability of the selected features from one dataset to another. To this end, we introduce a novel dataset recorded on a French grid (utility frequency at $50\mathrm{Hz}$) which is different from the other existing publicly available datasets such as PLAID [26], which was recorded recorded in the USA grid with utility frequency at $60\mathrm{Hz}$. Hence, our new proposed dataset contains a large variety of HEA types including new ones such as LCD and Plasma TV, Coffee maker, and oven.

3. Materials

3.1. HEAs Datasets

To test the performance of an NILM technique, it is important to consider real data. Due to the challenges in the creation of such datasets, mainly related to the required time and the high costs involved, we investigated in this study two data sets that are freely available: the PLAID dataset [26], which is shared online and is used as common reference, and our novel publicly available dataset freely accessible at http://dx.doi.org/10.21227/ww76-d733). Both datasets contain individual HEAs measurements which are convenient for extracting features, training models, conducting performance evaluation and performing benchmarking on a common basis. Indeed, some existing datasets include scenarios of multiple simultaneous loads [27]. However, before conducting disaggregation (i.e., decomposing the whole energy consumption of a dwelling into the energy usage of individual HEAs), it is important to build an initial signature database that is key to many NILM techniques.

For both datasets considered in this study, a class of HEA corresponds to a brand of a category of HEA, e.g., the class “Incandescent light bulb-Electrix-soft white”, which is distinct from the HEA class “Incandescent light bulb-Philips Durama”. Furthermore, we only consider steady-state conditions, and we extracted for each class of HEAs in both datasets several periods of the current and voltage steady-state waveforms. Indeed, during HEAs’ switching On/Off, momentary fluctuations of the current and voltage signals occur before settling in to a steady-state value. These fluctuations are called transients and can characterize a given HEA. However, one major drawback of the switching transients is their reproducibility, since they can be affected by HEA-independent factors.

3.1.1. PLAID Dataset

This dataset [26] contains current and voltage measurements sampled at 30 kHz from 11 different HEA types present in more than 60 households in Pittsburgh, Pennsylvania, USA. The goal of this dataset is to provide a public library for high-frequency measurements that can be used to assess existing or novel HEA classification algorithms. There are 11 categories of HEAs, which correspond to: air conditioner, compact fluorescent lamp, fridge, hairdryer, laptop, microwave, washing machine, bulb, vacuum, fan, and heater. Each category of HEA is represented by more than ten different instances. For each HEA, three to six measurements are collected for each state transition (i.e., on/off changing state). As mentioned, we only consider the steady-state operations, and we extracted for each class of HEAs in the PLAID dataset several periods of the current and voltage steady-state waveforms such that we have a total of 71 HEAs (with different categories and brands) and $n=36,720$ distinct recordings (also called individuals in the statistical terminology).

3.1.2. New Proposed Dataset

We introduce a novel publicly availabledataset containing 24 categories of HEAs (e.g., fans, fridges, washers, etc.) considered with distinct brands (35 types) and that were recorded at several power levels to lead us to a total of 61 considered HEAs. For HEAs with a wide variety of operational programs or adjustable settings such as temperature or intensity, we recorded all power consumption patterns and considered that an HEA power level corresponds to a specific class of HEA, although power consumption patterns refer to the same device. Figure 2 lists the considered categories of HEAs.

The HEAs have been recorded in steady-state conditions in a French 50 Hz electrical grid. The measurement setup consists of an AC current probe (E3N Chauvin Arnoux) with a 10 mV/A sensitivity and a differential voltage probe with a 1/100 attenuation. Voltage and current waveforms were captured by an 8-bit resolution digital oscilloscope (RIGOL DS1104Z). The sampling rate was set at $f_{s_1}=250$ kHz for some of the recordings and $f_{s_2}=50$ kHz for the other part. Electrical assembly and instrumentation are depicted in Figure 3. Hence, each HEA of this dataset is described by 8 periods of the current and voltage steady-state waveforms, resulting consequently in a set of $n=8\times 61=488$ distinct individuals.

3.2. Electrical Features Computed from Current and Voltage Measurements

We can describe the different HEAs using 90 features extracted at each voltage period summarized in

Table 1. Summary of the proposed electrical features  [19]. The features colored in red meet the additivity criterion given by Equation (13).

Electrical Features Name	Number of Computed Features
Effective current I and its harmonics $I_k$ for $k\in \{1,\dots ,15\}$ and $I_H$ (A)	17
Active power P and its harmonics $P_k$ for $k\in \{1,\dots ,15\}$ and $P_H$ (W)	17
Reactive power Q and its harmonics $Q_k$ for $k\in \{1,\dots ,15\}$ and $Q_H$ (VAR)	17
Apparent power S and its harmonics $S_k$ for $k\in \{1,\dots ,15\}$ and $S_H$, $S_N$ (VA)	18
Current harmonic distortion ${\mathrm{THD}}_{\mathrm{I}}$	1
Distortion power D, $D_I$, $D_V$ (VAD)	3
Power factor $F_p$ and $F_{pk}$ for $k\in \{1,\dots ,15\}$	16
Current crest factor ${\mathrm{F}}_{\mathrm{CI}}$	1
Total	90

. The detail of their computation was introduced in [19], based on the latest IEEE 1459-2010 standard for the definition of single phase physical components under non-sinusoidal conditions [28,29]. From the voltage $v\left(t\right)$ and current $i\left(t\right)$ signals, we compute the Fourier coefficients $v_{ak}$, $v_{bk}$, $i_{ak}$ and $i_{bk}$, using the following formulas:

$$x_{ak}=\frac{2}{M}\sum _{m=0}^{M-1}x\left[m\right]cos(2\pi mk{f}_0/f_s),x_{bk}=\frac{2}{M}\sum _{m=0}^{M-1}x\left[m\right]sin(2\pi mk{f}_0/f_s)$$

(1)

where $x\left[m\right]=x(m/f_s)$ is the sampled discrete-time signal, $f_s$ is the sampling frequency, $f_0$ the utility frequency (e.g., 50 Hz in France) and $M=f_s/f_0$. From these Fourier coefficients, the following features can be computed for any harmonic rank $k\in \mathbb{N}$.

• The Root Mean Squared (RMS) value of the $k^{th}$ harmonic component of the voltage $V_k$ and currents $I_k$, and their sum $V_H$ and $I_H$:

  $$V_k=\sqrt{{\displaystyle \frac{v_{ak}^2+v_{bk}^2}{2}}},I_k=\sqrt{\frac{i_{ak}^2+i_{bk}^2}{2}},V_H=\sqrt{{\displaystyle \sum _{k=2}^{15}{V}_k^2}},I_H=\sqrt{{\displaystyle \sum _{k=2}^{15}{I}_k^2}}$$

  (2)
• The RMS voltage V and current I:

  $$\begin{array}{ccc}\hfill V& =& \sqrt{V_1^2+V_H^2},I=\sqrt{I_1^2+I_H^2}\hfill \end{array}$$

  (3)
• The $k^{th}$ harmonic component of the active, reactive, and apparent powers $P_k$, $Q_k$, $S_k$, and their sums $P_H$, $Q_H$, $S_H$:

  $$\begin{array}{ccc}& P_k=\frac{1}{2}(v_{ak}{i}_{ak}+v_{bk}{i}_{bk})& {\displaystyle P_H=\sum _{k=2}^{15}{P}_k}\hfill \end{array}$$

  (4)

  $$\begin{array}{ccc}& Q_k=\frac{1}{2}(v_{ak}{i}_{bk}-v_{bk}{i}_{ak})& {\displaystyle Q_H=\sum _{k=2}^{15}{Q}_k}\hfill \end{array}$$

  (5)

  $$\begin{array}{ccc}& S_k=V_k\times I_k& S_H=V_H\times I_H\hfill \end{array}$$

  (6)
• The active, reactive apparent, and distorsion powers P, Q, S, and D:

  $$P=P_1+P_H,Q=Q_1+Q_H,S=V\times I,D=\sqrt{S^2-P^2-Q^2}$$

  (7)
• The voltage and current total harmonic distortion ${\mathrm{THD}}_V$ and ${\mathrm{THD}}_I$:

  $${\mathrm{THD}}_V=\frac{V_H}{V_1},{\mathrm{THD}}_I=\frac{I_H}{I_1}$$

  (8)
• The voltage and current distortion powers $D_V$ and $D_I$:

  $$D_V=V_H{I}_1,D_I=V_1{I}_H$$

  (9)
• The non-fundamental apparent power $S_N$:

  $$S_N=\sqrt{D_I^2+D_V^2+S_H^2}$$

  (10)
• The voltage and current crest factors for $m\in [0,M-1]$:

  $$F_{CV}=\frac{max\left|v\right[m\left]\right|}{V},F_{CI}=\frac{max\left|i\right[m\left]\right|}{I}$$

  (11)
• Finally, the global and harmonic power factors $F_p$ and $F_{pk}$:

  $$F_p=\frac{P}{S},F_{pk}=\frac{P_k}{S_k}$$

  (12)

Since the NILM problem consists in the breakdown of an unknown mixture of HEAs into a set of identifiable HEAs signatures possibly belonging to a database, it is important to consider the electrical features that meet the additivity criterion [8] such that:

$$\begin{array}{c}\hfill {\displaystyle f(\mathbf{v},\mathbf{i})=\sum _{n=1}^{N_s}f(\mathbf{v},{\mathbf{i}}_n)\mathrm{for}\mathbf{i}=\sum _{n=1}^{N_s}{\mathbf{i}}_n}\end{array}$$

(13)

where $f(\mathbf{v},\mathbf{i})$ is an electrical feature, $\mathbf{v}$ is a vector of voltage, $\mathbf{i}$ a vector of current samples acquired during one voltage period and $N_s$ is the number of HEAs that are simultaneously switched on. Thus, when an HEA is connected (resp. disconnected) to the power network, an “additive” feature is increased (resp. decreased) by an amount equal to that produced by this HEA operating individually. Among the 90 features, $p=34$ features meet the additivity criterion and are reported in red in Table 1. This property is required for extracting HEAs features from an aggregated signal and for comparing them with the dataset of individual HEAs signatures. For example, a change detection method as proposed in [6] can be used in order to separate the distinct contribution of each HEA when several ones are switched on simultaneously. Hence, this study only focuses on the additive features computed for a unique observed HEA. Both datasets are stored in an $n\times p$ normalized matrix $\overline{X}$, where the $p=34$ features detailed earlier are in columns and have been normalized with a zero mean and a unit standard deviation.

4. Feature Selection

Existing approaches for feature selection can be categorized into filter-based methods and wrapper-based methods [30,31]. Filter-based methods perform FS independently of the classification process. Features are individually assigned to relevance scores, which are assumed to reflect their usefulness in the classification task. Hence, the features are sorted by descending order of the obtained score of relevance [32]. Filter-based methods are often computationally faster but are known to be less accurate than other approaches. On the other hand, wrapper-based methods use the classifier of interest to score feature subsets according to the classification accuracy. This allows selecting an optimal subset of features that maximizes the classification accuracy with an improvement of the computational cost [32].

4.1. Investigated Feature Selection Methods

This paper investigates the methods listed below, which are comparatively assessed in the remainder. More details are given to the sequential forward FS method and our original contributions concerning the DNN methods, in light of the other methods’ literature review.

4.1.1. Existing Methods

• Principal Component Analysis (PCA) can be used as a filter-based method based on the maximization of the dataset dispersion [33,34].
• Linear Discriminant Analysis (LDA) can be used as a supervised filter-based method maximizing the separation between classes (see Section 5.1).
• Mutual Information (MI) is a filter-based method measuring the amount of information each feature conveys from the class labels [35].

4.1.2. New Proposed Sequential Forward Method

This approach is an iterative wrapper-based FS method that aims at maximizing the classifier accuracy when adding each feature one-by-one [30,36,37,38,39]. The first iteration starts with an empty set of selected features, and each feature $f\in F$, where F is the set of p features, is tested individually using a given classifier (in this study, we used LDA and K-nearest-neighbor (KNN) presented in Section 5). The feature that maximizes the classification accuracy ($Acc$) [40] of the training dataset is the first one selected. Then, each remaining feature is added to the set of previously selected features, and the same process is applied to find again the highest accuracy. The iterative feature selection ends once further feature addition yields no accuracy improvement. The proposed method is presented hereafter in Algorithm 1.

Algorithm 1: Sequential forward FS algorithm

Input:  dataset $X_{n,p}$, whole set of features F, ground-truth labels Y   Output:  Set of sorted features $F_p$  1: Initialization: $F_0\leftarrow \varnothing$  2: for  $k\leftarrow 0\mathrm{to}p-1$  do 3: for  $f\in F\setminus F_k$  do 4: $F_{+}\leftarrow F_k\cup \left\{f\right\}$   5: evaluate (LDA or KNN classifier, $X_n\left(F_{+}\right)$, Y)   6: compute $Acc\left(F_{+}\right)$   7: end for  8: Select the best remaining feature: ${\displaystyle f_{\mathrm{k}}\leftarrow arg\underset{f\in F\setminus F_k}{max}Acc(F_k\cup \left\{f\right\})}$   9: Update feature set: $F_{k+1}\leftarrow F_k\cup \left\{f_{\mathrm{k}}\right\}$   10: end for  11: return$F_p$

4.1.3. New Proposed Deep Neural Network (DNN) Feature Selection Method

This method is inspired by [41], which is a filter-based approach that uses our proposed neural architecture. It uses the sum of the trained weights of the first layer neurons as a score of relevance of the input features.

Deep learning uses a combination of agents (the neurons) to learn high-level non-linear relationships and correlations in the analyzed data. Such methods can tackle complicated problems and become a promising approach for smart grid applications [42]. Here, we use a dense fully connected DNN architecture [43], where each neuron of the input layer is associated with a feature $f_i$ as presented in Figure 4. Our architecture is made of one input layer with p neurons and 2 hidden blocks containing one layer of 128 dense formal neurons including a Batch Normalization (BN) combined with a dropout layer. The output y of each neuron can be expressed as a function of an input vector $x\in {\mathbb{R}}^N$ such as:

$$y=g\left(w_0+\sum _{i=1}^N{w}_i{x}_i\right)$$

(14)

where g is the neuron activation function chosen as REctified Linear Unit (RELU) defined by $g\left(x\right)=max(0,x)$ except for the last output layer, which uses the softmax activation function [44]. The $w_i$ coefficients ($w_0$ being the bias) are the synaptic weights, which are learned during the training process. In our implementation, we choose $w_0=0$ and we use a BN of the output, which was shown to improve the training performances [45] when applied on the activation of each neuron. Thus, we have $\mathrm{BN}\left(y\right)=\frac{y-\mu }{\sigma }$, where $\mu$ and $\sigma$ are the mean and the standard deviation computed from the processed training batch defined with a size equal to 64. Each dropout layer is defined to randomly discard 10 % of the connected input layer and is used in order to reduce overfitting [46]. The last layer of our proposed DNN model applies a softmax activation function on each output neuron (each output neuron is associated with a prediction class) returning output values in $[0,1]$. This value corresponds to the probability of predicting the analyzed input as a member of class i. Hence, the final output label corresponds to the class index which maximizes the resulting probability of the last layer such as $\widehat{y}={argmax}_i{y}_i$, $y_i$ being the output of neuron i of the last layer. The proposed DNN is designed for classification or regression problems; however, we propose here a new feature selection method based on the trained weights of each neuron. Following this idea, we consider the sum of the weights of all the neurons of the first layer linked to the same input feature as a score of relevance for this feature. The DNN is trained on the studied datasets using cross-entropy [43] as a loss function.

4.2. Feature Selection Results

In the context of event-based NILM [6], the selection of features meeting the additivity criterion matters. Each FS method, among the five ones previously presented in Section 4.1, is then applied on the two datasets considered in this study (see Section 3.1), where each HEA is represented by a vector of $p=34$ features meeting the additivity criterion (see Section 3.2). Datasets are centered and reduced in order to get an $(n\times p)$ matrix $\overline{X}$ with a zero mean and a unit standard deviation. This is obtained by applying the same operation as used for the Batch Normalization described above (subtracting the mean and dividing by the standard deviation of each individual). For the FS methods based on a feature relevance score, we sort the features by descending order of relevance, and then we select the subset before the highest decrease in the relevancy score. The results obtained on our own dataset and the PLAID dataset are reported in

Table 2. Results of the features selection methods applied on the proposed dataset.

Method	Features Meeting the Additive Criterion
	# Features	Selected Features
		$P_1,Q_1,P_7,Q_3,Q,P,P_3,$
KNN based sequential forward FS method	12	$P_H,P_5,Q_5,Q_H,Q_9$
		$P_1,Q,P,Q_9,Q_3,P_3,P_2,$
LDA based sequential forward FS method	18	$P_{10},Q_4,P_4,P_6,P_9,Q_8,P_{13},$
		$P_8,Q_{15},Q_5,Q_{11}$
		$P_1,P,P_5,Q,Q_1,Q_H,P_H,$
MI	20	$P_7,P_{15},P_9,Q_5,Q_7,P_{13},Q_3,$
		$Q_9,P_{11},Q_{13},P_3,Q_{15},Q_{11}$
		$Q,Q_1,P_3,P_9,P_7,P_H,Q_5,$
PCA	17	$P_{11},Q_9,P_6,Q_3,Q_{11},Q_7,P_{15},$
		$P_{12},Q_6,P_8$
		$P_7,Q,Q_1,P_9,P_3,Q_5,P_H,$
LDA	14	$Q_7,P_{11},P_5,Q_9,P,P_1,Q_{11}$
		$Q_9,P_3,P_H,P_{12},Q_{15},Q_5,Q_{12},$
DNN	27	$P_6,P_9,Q_8,P_7,Q_4,P_5,Q_H,$
		$P_{11},Q_2,Q_3,Q_{11},Q_{14},P_{10},P_1,$
		$P,P_2,P_4,P_8,P_{13},P_{14}$

and

Table 3. Results of the features selection methods applied on PLAID dataset.

Method	Features Meeting the Additive Criterion
	# Features	Selected Features
		$P_H,Q,P_1,P_3,Q_H,Q_5,P_5,$
KNN based sequential forward FS method	25	$Q_3,Q_1,Q_7,P_{15},Q_2,P,Q_{13},$
		$P_{12},P_7,Q_{15},Q_9,P_{13},P_9,P_{10},$
		$Q_{10},Q_{11},Q_{12},Q_8$
LDA based sequential forward FS method	33	All features except $Q_9$
		$P_3,P_H,P_1,P,Q_1,Q,Q_9,$
MI	20	$Q_7,P_7,P_5,Q_5,Q_H,Q_3,P_9,$
		$Q_{11},P_{11},Q_{13},Q_{15},P_{15},P_{13}$
PCA	31	All features except $P_{14},Q,Q_1$
		$P_3,P_H,P_{15},Q_{13},Q_5,Q_H,Q_{15},$
LDA	12	$P_9,P_7,Q_3,Q_9,Q_7$
		$P_H,Q_{11},P_3,P_9,Q_9,P_{15},P_5,$
DNN	17	$Q_7,P_7,P_{10},Q_3,Q_H,Q_{12},Q_{13},$
		$Q_{15},P_{13},P_6$

.

The following observations can be made according to the obtained results:

• For both datasets, some features such as $P,P_1,P_H,Q,Q_1$, or $Q_H$ are present regardless of the used FS method;
• For both datasets, the features selected by the DNN method for FS and by the PCA method are diversified in terms of harmonic orders;
• For both datasets, the features selected by the MI and LDA methods are related to odd-order harmonics, which describe the power supply structures included in most of the HEAs;
• For the sequential forward FS method, our experiments compare the results provided by the Euclidean-based KNN classifier (where the neighborhood parameter is set to $K=7$) and to the LDA classifier. The number of nearest neighbors is set to 7 because it is the closest odd number to the number of instances in a class in the proposed dataset, so that for each neighborhood, there is a majority vote. The selected features are those that reach the maximum accuracy [47] reported in Table 2 and Table 3. For our dataset, only 12 features allow maximizing the KNN classifier accuracy and 18 features maximize the accuracy of the LDA classifier (see Figure 5). For the PLAID dataset, 25 features allow maximizing the KNN classifier accuracy and 33 features maximize the LDA classifier accuracy (see Figure 6). The low accuracy reached by the LDA classifier in the PLAID dataset can be explained by the unbalanced training sets (where one or several classes outnumber the other classes) [48]. Indeed, LDA is known to not provide good performances in this setting since classification is generally biased towards the majority classes. It is observed that for the LDA classifier, the accuracies less rapidly reach a plateau than for the KNN. Indeed, the KNN is known to be affected by the overfitting phenomenon [49,50], which increases the distance between individuals of the same class and decreases the accuracy. The classifier cannot deal with the feature relevance and is more sensitive to FS than other classifiers such as LDA, which can handle irrelevant features [51].

5. Home Electrical Appliances Classification Results

5.1. Investigated Classification Methods

We investigate four classification methods that can be combined with the different feature subsets provided by the FS methods presented in Section 4.

• The KNN method is widely used by the NILM community for HEAs’ identification [12,52]. We use the Euclidean distance and $K=7$, which corresponds to the closest odd number to the number of instances in a class in the proposed dataset. Hence, the predicted class corresponds to the most represented one in the neighborhood through majority voting.
• The LDA method estimates the optimal linear combination between features using the eigenvectors of the projection matrix ${(B+S)}^{-1}B$ of dimension $p\times p$, where ${\displaystyle B=\frac{1}{n}\sum _{k=1}^K{n}_k(g_k-g){(g_k-g)}^T}$ and ${\displaystyle S=\frac{1}{n}\sum _{k=1}^K{n}_k{V}_k}$, where $V_k$ are the covariance matrices built from the corresponding $n_k$ individuals (number of individuals of class $k\in \{1,\dots ,K\}$); ${\displaystyle g=\frac{1}{n}\sum _{i=1}^n{\overline{x}}_i}$ and ${\displaystyle g_k=\frac{1}{n_k}\sum _{i=1}^{n_k}{\overline{x}}_i}$ correspond to the mean over all the individuals of the whole dataset $\overline{X}$ and the mean over all the individuals in the class k, respectively. Then, the tested individuals are projected into the discriminative linear space before being assigned to the class whose centroid is the closest in terms of the Euclidean distance.
• The proposed DNN classification method uses the same fully connected DNN architecture as presented in Section 4.1.3 for FS. Our implementation is based on tensorflow/keras (https://keras.io/). The training is completed with a batch size equal to 64 and a maximal number of 350 epochs (one epoch is reached each time the whole training dataset is processed once). The optimization is completed using the RMSprop algorithm [43] with a learning rate set to $\eta ={10}^{-3}$.
• The Random Forest (RF) classification method creates a set of decision trees and aggregates the votes from the decision trees to predict the class of the tested individuals [53]. The number of trees was set to 5 after experimental tuning to get the best results.

Other classification methods such as Support Vector Machine (SVM) [8] or Adaboost [54] were not evaluated in this study because they require a very high computation cost, and our preliminary results did not reveal a significant accuracy improvement in comparison to the four investigated methods.

5.2. Classification Test Procedure

The evaluation of the classification performances uses an 8-fold cross validation methodology, which randomly splits the dataset into eight equal partitions, which are individually tested using the seven remaining partitions for training. This process is repeated eight times, until all subsamples have been used for cross validation [55]. The number of folds was arbitrarily set to 8 so that 12.5% of the studied dataset corresponds to the test set and the remaining 87.5% to the training set. The final metrics for classification performances were then computed by merging the results of each partition.

Our results are expressed in terms of the classical classification metrics used for NILM—Accuracy ($Acc$), ${\mathrm{F}}_{\mathrm{Measure}}$ ($F_M$), Recall ($Rec$) and Precision ($Pre$) [40,47]—which are deduced from the computed confusion matrices. Thus, if we denote C a resulting confusion matrix of dimension $I\times I$ (I being the considered number of classes), where $C_{ij}$ corresponds to the number of individuals of the true class i (row) predicted as being in the class j (column), the Accuracy, Precision, and Recall are computed as:

$$\mathrm{Acc}=\frac{{\sum }_i{C}_{ii}}{{\sum }_i{\sum }_j{C}_{ij}}\mathrm{Prec}=\frac{1}{I}\sum _i\frac{C_{ii}}{{\sum }_j{C}_{ji}}\mathrm{Rec}=\frac{1}{I}\sum _i\frac{C_{ii}}{{\sum }_j{C}_{ij}}$$

(15)

where $n={\sum }_i{\sum }_j{C}_{ij}$ is the total number of individuals. The F-measure (also called F- or $F_1$-score) is the harmonic mean of the Precision and Recall, computed as:

$$F_M=2\frac{\mathrm{Prec}·\mathrm{Rec}}{\mathrm{Prec}+\mathrm{Rec}}$$

(16)

We also present the ratio $\frac{Acc}{\#features}$, which measures the efficiency and allows us to judge on the right combination of FS method and classifier. Indeed, the goal is to achieve the highest accuracy for a small number of descriptors. A large ratio means that a high accuracy is reached with a small number of selected features. Each method is evaluated on both datasets.

5.3. Self-Database Results

Section 5.3.1 and Section 5.3.2 present the classification success rate as a function of the provided subset of selected features used to describe each HEA. Overall results using our new proposed dataset are summarized in

Table 4. Average $F_M$ scores obtained for the several different feature subsets selected for the proposed dataset (61 classes, n = 488) over all considered classifiers. Results are sorted in descending order of F-measure.

Feature Selection Method	${\mathbf{F}}_{\mathbf{M}}$ Average (%)
MI	92.96
KNN based Seq. forw. FS method	92.58
LDA	92.42
All features	90.92
LDA-based Seq. forw. FS method	90.14
PCA	87.64
P,Q features	86.62
DNN	84.87

for which the details are provided in

Table 5. Performance (in percentage) of the classification methods applied on the proposed dataset using different feature subsets from the additive feature set (61 classes, n = 488). Results are sorted in descending order of F-measure.

Methods (Selected Features/Classifier)				Results
Feat.	# feat.	Classifier	D.A	Acc.	${\mathbf{F}}_{\mathbf{M}}$	Rec.	Pre.	$\frac{\mathbf{Acc}.}{\#\mathbf{feat}.}$
MI	20	R.F	Yes	99.18	99.17	99.18	99.30	4.95
LDA	14	R.F	Yes	98.56	98.52	98.56	98.71	7.04
P,Q features	2	R.F	Yes	98.15	98.15	98.15	98.24	49.07
All features	34	R.F	Yes	98.15	98.14	98.15	98.28	2.88
KNN-based Seq. forw. FS (K = 7)	12	R.F	Yes	97.74	97.71	97.74	97.99	8.14
LDA	14	R.F	No	98.15	97.67	97.43	98.15	7.01
KNN-based Seq. forw. FS (K = 7)	12	R.F	No	98.15	97.60	97.33	98.15	8.17
MI	20	R.F	No	98.15	97.54	97.23	98.15	4.90
DNN	27	R.F	Yes	97.34	97.32	97.34	97.60	3.60
P,Q features	2	R.F	No	97.74	97.06	96.72	97.74	48.87
MI	20	LDA	Yes	96.92	96.83	96.92	97.55	4.84
LDA-based Seq. forw. FS	18	LDA	No	97.54	96.72	96.31	97.54	5.41
DNN	27	LDA	Yes	96.72	96.62	96.72	97.49	3.58
LDA-based Seq. forw. FS	18	LDA	Yes	96.72	96.57	96.72	97.43	5.37
LDA-based Seq. forw. FS	18	R.F	Yes	96.31	96.29	96.31	96.77	5.35
PCA	17	R.F	Yes	96.31	96.25	96.31	96.70	5.66
All features	34	LDA	Yes	96.31	96.18	96.31	96.84	2.83
DNN	27	LDA	No	96.93	96.14	95.77	96.91	3.59
LDA	14	LDA	Yes	95.69	95.59	95.69	96.32	6.83
KNN-based Seq. forw. FS (K = 7)	12	KNN	No	96.72	95.76	95.29	96.72	8.06
All features	34	KNN	Yes	95.69	95.53	95.69	96.82	2.81
MI	20	KNN	Yes	95.49	95.27	95.49	96.73	4.77
DNN	27	R.F	No	96.31	95.22	94.67	96.31	3.57
MI	20	LDA	No	96.31	95.21	94.67	96.31	4.81
LDA-based Seq. forw. FS	18	R.F	No	96.31	95.21	94.67	96.31	5.35
MI	20	DNN	No	96.31	95.08	94.47	96.31	4.81
KNN-based Seq. forw. FS (K = 7)	12	KNN	Yes	95.28	95.02	95.32	96.68	7.94
LDA	14	KNN	Yes	95.28	95.01	95.28	96.65	6.80
LDA-based Seq. forw. FS	18	KNN	Yes	95.27	95.01	95.28	96.49	5.29
P,Q features	2	KNN	Yes	95.28	94.97	95.28	96.66	47.64
All features	34	LDA	No	96.10	94.94	94.36	96.10	2.82
All features	34	R.F	No	96.10	94.91	94.33	96.10	2.82
PCA	17	R.F	No	95.90	94.87	94.39	95.90	5.64
KNN-based Seq. forw. FS (K = 7)	12	LDA	Yes	95.28	94.86	95.28	96.34	7.94
KNN-based Seq. forw. FS (K = 7)	12	LDA	No	95.90	94.60	93.95	95.90	7.99
PCA	17	LDA	Yes	94.46	94.37	94.46	95.48	5.55
LDA	14	LDA	No	95.69	94.33	93.65	95.69	6.83
P,Q features	2	KNN	No	95.08	93.44	92.62	95.08	47.54
PCA	17	LDA	No	94.67	93.06	92.28	94.67	5.57
Feat.	# feat.	Classifier	D.A	Acc.	${\mathbf{F}}_{\mathbf{M}}$	Rec.	Pre.	$\frac{\mathbf{Acc}.}{\#\mathbf{feat}.}$
LDA	14	KNN	No	94.46	92.96	92.21	94.46	6.74
PCA	17	DNN	No	93.64	92.15	91.48	93.65	5.51
P,Q features	2	LDA	Yes	93.23	91.85	93.23	93.69	46.61
All features	34	DNN	No	93.24	91.34	90.44	93.24	2.74
P,Q features	2	LDA	No	93.23	90.98	89.85	93.23	46.61
KNN-based Seq. forw. FS (K = 7)	12	DNN	No	92.62	90.40	89.34	92.62	7.72
MI	20	KNN	No	91.60	89.21	88.01	91.59	4.58
LDA-based Seq. forw. FS	18	DNN	No	90.98	88.48	87.39	90.98	5.05
LDA	14	DNN	No	92.01	89.89	88.99	92.01	6.57
DNN	27	KNN	Yes	87.09	86.18	87.09	89.46	3.23
LDA-based Seq. forw. FS	18	KNN	No	88.11	84.35	82.51	88.11	4.89
DNN	27	DNN	No	86.27	82.70	88.99	92.01	3.20
DNN	27	KNN	No	85.86	81.39	79.17	85.86	3.18
All features	34	KNN	No	84.43	80.31	78.36	84.43	2.48
PCA	17	KNN	Yes	81.14	79.87	81.14	82.58	4.77
PCA	17	KNN	No	82.37	78.18	76.25	82.37	4.84
All features	34	DNN	Yes	77.05	76.07	77.05	79.51	2.27
LDA	14	DNN	Yes	76.84	75.40	76.84	77.41	5.49
MI	20	DNN	Yes	76.84	75.33	76.84	77.18	3.84
KNN-based Seq. forw. FS (K = 7)	12	DNN	Yes	76.02	74.70	76.02	77.51	6.33
PCA	17	DNN	Yes	74.59	72.35	74.59	75.47	4.39
LDA-based Seq. forw. FS	18	DNN	Yes	70.70	68.45	70.70	70.50	3.93
P,Q features	2	DNN	No	71.52	66.43	64.28	71.52	35.76
P,Q features	2	DNN	Yes	63.32	60.05	63.32	59.19	31.66
DNN	27	DNN	Yes	45.08	43.36	45.08	45.34	1.67

. The results obtained using the PLAID dataset are summarized in

Table 6. Average $F_M$ scores obtained for the several different feature subsets selected for the PLAID dataset (71 classes, n = 36,720) over all considered classifiers. Results are sorted in descending order of F-measure.

Feature Selection Method	${\mathbf{F}}_{\mathbf{M}}$ Average (%)
MI	85.80
LDA based Seq. forw. FS method	85.07
KNN based Seq. forw. FS method	85.05
All features	84.56
PCA	83.53
DNN	82.52
LDA	81.61
P,Q features	58.31

where the details are presented in

Table 7. Performance (in percentage) of the classification methods applied on PLAID dataset using different feature subsets from the additive feature set (71 classes, n = 36,720). Results are sorted in descending order of F-measure.

Methods (Selected Features/Classifier)				Results
Feat.	# Feat.	Classifier	D.A	Acc.	${\mathbf{F}}_{\mathbf{M}}$	Rec.	Pre.	$\frac{\mathbf{Acc}.}{\#\mathbf{feat}.}$
KNN-based Seq. forw. FS (K = 7)	25	KNN	No	99.19	98.63	98.81	98.56	3.97
KNN-based Seq. forw. FS (K = 7)	25	KNN	Yes	99.09	98.54	98.25	98.90	3.96
LDA-based Seq. forw. FS	33	KNN	No	99.07	98.50	98.71	98.39	3.00
LDA-based Seq. forw. FS	33	KNN	Yes	99.02	98.50	98.21	98.83	3.00
All features	34	KNN	No	99.07	98.49	98.69	98.40	2.91
PCA	30	KNN	No	99.00	98.46	98.60	98.41	3.30
DNN	17	KNN	No	98.99	98.45	98.62	98.41	5.82
MI	20	KNN	No	99.13	98.43	98.43	98.53	4.96
DNN	17	KNN	Yes	98.94	98.41	98.14	98.74	5.82
All features	34	KNN	Yes	99.07	98.38	98.09	98.81	2.91
Feat.	# Feat.	Classifier	D.A	Acc.	${\mathbf{F}}_{\mathbf{M}}$	Rec.	Pre.	$\frac{\mathbf{Acc}.}{\#\mathbf{feat}.}$
MI	20	KNN	Yes	99.03	98.33	98.23	98.48	4.95
PCA	30	KNN	Yes	98.95	98.28	98.00	98.69	3.30
MI	20	R.F	No	98.91	98.27	98.50	98.18	4.95
LDA	12	KNN	No	98.92	98.09	98.06	98.18	8.24
LDA-based Seq. forw. FS	33	R.F	No	98.75	97.97	98.20	97.91	2.99
All features	34	R.F	No	98.79	97.88	98.09	97.89	2.91
KNN-based Seq. forw. FS (K = 7)	25	R.F	No	98.80	97.85	98.00	97.85	3.95
LDA	12	KNN	Yes	98.76	97.84	97.64	98.06	8.23
DNN	17	R.F	No	98.58	97.74	98.14	97.54	5.80
PCA	30	R.F	No	98.67	97.66	97.84	97.60	3.29
LDA	12	R.F	No	98.60	97.62	97.99	97.51	8.22
LDA-based Seq. forw. FS	33	R.F	Yes	98.23	97.22	96.88	97.70	2.98
KNN-based Seq. forw. FS (K = 7)	25	R.F	Yes	98.19	97.14	96.69	97.78	3.93
All features	34	R.F	Yes	98.25	97.04	96.67	97.53	2.89
PCA	30	R.F	Yes	98.16	96.90	95.53	97.43	3.27
MI	20	R.F	Yes	98.19	96.89	96.48	97.34	4.91
LDA	12	R.F	Yes	97.88	96.86	96.43	97.40	8.16
MI	20	DNN	Yes	98.32	96.78	96.38	97.63	4.92
DNN	17	R.F	Yes	97.96	96.61	96.07	97.62	5.76
All features	34	DNN	No	98.04	96.35	96.64	96.67	2.88
KNN-based Seq. forw. FS (K = 7)	25	DNN	No	97.56	96.30	96.59	96.58	3.90
MI	20	DNN	No	96.26	96.05	96.99	96.28	4.81
LDA-based Seq. forw. FS	33	DNN	No	96.29	94.40	95.42	94.85	2.89
LDA	12	DNN	No	95.52	94.10	95.10	94.09	7.96
LDA	12	DNN	Yes	95.51	92.88	91.57	95.88	7.96
DNN	17	DNN	No	94.55	92.41	94.03	92.52	5.56
PCA	30	DNN	No	94.45	92.15	92.85	92.63	3.15
LDA-based Seq. forw. FS	33	DNN	Yes	94.81	91.66	90.72	94.43	2.87
KNN-based Seq. forw. FS (K = 7)	25	DNN	Yes	94.87	91.22	90.29	93.27	3.79
P,Q features	2	KNN	Yes	94.43	90.91	91.75	91.68	47.22
P,Q features	2	KNN	No	94.58	90.82	91.39	90.86	47.29
PCA	30	DNN	Yes	91.19	90.82	88.99	94.33	3.04
DNN	17	DNN	Yes	93.28	90.52	88.14	94.84	5.49
P,Q features	2	R.F	No	93.79	90.31	90.50	90.60	46.90
All features	34	DNN	Yes	90.33	86.33	85.00	89.60	2.66
P,Q features	2	R.F	Yes	90.63	85.84	86.14	85.70	45.32
LDA-based Seq. forw. FS	33	LDA	No	45.52	53.46	59.45	54.25	1.38
All features	34	LDA	No	45.31	53.25	59.42	54.05	1.33
MI	20	LDA	No	44.55	53.11	58.49	53.84	2.23
KNN-based Seq. forw. FS (K = 7)	25	LDA	No	44.37	51.50	57.62	52.32	1.77
Feat.	# Feat.	Classifier	D.A	Acc.	${\mathbf{F}}_{\mathbf{M}}$	Rec.	Pre.	$\frac{\mathbf{Acc}.}{\#\mathbf{feat}.}$
P,Q features	2	DNN	Yes	65.56	50.48	51.27	55.29	32.79
P,Q features	2	DNN	No	65.52	49.75	52.58	52.45	32.76
PCA	30	LDA	No	42.54	48.58	54.28	49.23	1.42
LDA-based Seq. forw. FS	33	LDA	Yes	42.89	48.87	49.68	57.03	1.30
All features	34	LDA	Yes	42.91	48.78	49.75	56.29	1.26
KNN-based Seq. forw. FS (K = 7)	25	LDA	Yes	42.72	48.67	49.61	55.40	1.71
MI	20	LDA	Yes	42.45	48.50	49.49	53.77	2.12
DNN	17	LDA	No	40.65	46.79	54.21	46.85	2.39
PCA	30	LDA	Yes	40.59	45.42	45.64	52.09	1.35
LDA	12	LDA	No	38.66	40.40	47.43	40.75	3.22
DNN	17	LDA	Yes	37.09	39.20	39.66	45.50	2.18
LDA	12	LDA	Yes	36.08	35.88	35.49	41.93	3.00
P,Q features	2	LDA	Yes	12.12	4.29	7.11	4.19	6.06
P,Q features	2	LDA	No	11.59	4.05	3.65	7.55	5.80

. The confusion matrices of the best methods are also presented in Appendix A. We also compare the results with the active and reactive powers (P,Q) which are the usual features proposed in the NILM literature [56]. The results show the clear improvement provided by the FS methods on each evaluated dataset. In addition, to study how noise influences our training, we have trained the studied classifiers using the selected features and Data Augmentation (DA) [57]. For this, both studied datasets are 100 % augmented by adding a white Gaussian noise to the current signals of each HEA class, to obtain an Signal to Noise Ratio (SNR) = 20 dB (defined as $10{log}_{10}\left(\frac{{\left|\right|x\left|\right|}^2}{{\left|\right|b\left|\right|}^2}\right)$, with x the original signal amplitude and b the noise signal). This leads to new augmented datasets containing noisy current signals, from which we compute the 34 features meeting the additive criterion. Then, for each original dataset, we apply cross-validation for classification evaluation using the selected features in Section 4.2 and the classification methods used in Section 5. The experiment setup also consists of an 8-fold cross-validation experiment, where the original datasets were partitioned into eight. For each of the 8 simulations, 20% of each HEA class of the noisy generated dataset was added to the training set. No noise was added to the test part that was used for performance generalization for each of the eight simulations in our experiments. The success rate of a classification method for a specific subset of selected features was obtained by calculating the average scores of the performances metrics used in Section 5. All the results presented in the following tables are ranked in descending order of the best ${\mathrm{F}}_{\mathrm{M}}$ scores.

5.3.1. Proposed Dataset

Table 5 shows that the RF classifier outperforms all the other classifiers. This is consistent with the findings of X. Wu et al. in [53], where an accuracy equal to 98.0% was found using eight steady-state features (which do not meet the additive criterion). These authors showed the advantages of RF classifier over KNN. In our experiment, RF classifier obtains the best recognition rate equal to 99.18% with the features selected by the MI method and DA, and 98.15% of accuracy when combined with LDA feature selection method without DA. This leads to a ratio (Acc/# feat.) of 7.01 and a computational time that is very low. It can be observed that the classifier performances are also better when data is artificially augmented during training. The confusion matrix depicted in Figure A1 shows that 20% of tested individuals that belong to class index 4 “Fan-Coala level 1” are classified as class index 5 “Fan -Coala Level 2” and 10% of the tested individuals belonging to class index 1 “Electric mixer Moulinex” are classified as class index 7 “Fan-Coala Level 3”. Interestingly, the performance of the DNN method is improved by a suitable choice of relevant descriptors as confirmed by the usage of the MI and the DNN featurs selection methods. Indeed, DNN classifier usually obtains the best results when using all the considered features. This point is of interest to develop new strategies to improve the training efficiency of DNN when used on a small training dataset. Finally, it can be observed that the results obtained with the DNN classifier are lower than those obtained for the PLAID dataset in Table 7. This can be explained by the small size of the training dataset. Indeed, DNN is known to require a large number of data to efficiently be trained. It can also be observed that for most of the classifiers, data augmentation by the addition of white Gaussian noise can significantly improve the results. Table 4 depicts the average ${\mathrm{F}}_{\mathrm{M}}$ scores obtained for each FS method considering all the studied classifiers. It can be seen that the odd order harmonics features selected by the MI method are the ones for which the best average ${\mathrm{F}}_{\mathrm{M}}$ score is reached.

5.3.2. PLAID Dataset

In Table 7, we can notice the improvement brought by FS approaches combined with the KNN classifier. Indeed, the obtained ratios (Acc/# feat.) are all greater, which denotes the fact that KNN classifier needs a small number of features to reach excellent accuracy. The best ratio (Acc/# feat.) of 47.29 is obtained when using KNN with P, Q features, which are usually considered for HEAs’ identification in the NILM literature, but the reached accuracy is only of 94.58%. The best accuracy of 99.19% is reached for only 25 features selected by the KNN-based sequential forward-FS method with and without data augmentation (according to the confusion matrix depicted in Figure A2, 10% of the tested individuals belonging to class index 7 “Incandescent light bulb-Electrix-soft white” are classified as class index 8 “Incandescent light bulb-Philips Duramax”). However, an accuracy of 99.13% with a very good ratio $\frac{Acc}{\#feat}=4.96$ is obtained when considering the subset of 20 features selected by the MI method with the KNN classifier. As we seek to achieve the best identification rates (close to 99 % of accuracy) for the smallest number of selected features, this combination classifier/selected features is a good trade-off. This allows the KNN classifier to slightly exceed the performances reached by the RF classifier with the 20 features selected by the MI method (98.91 % of accuracy).

In contrast to our dataset, the high number of individuals ($n=36720$) eases the DNN to learn the features from the dataset and to figure out that an important number of features is reliable and should be used. Our results outperform the previous ones reported in [58] for the PLAID dataset using the VI trajectories features, where an F-measure of $F_M\approx 77\%$ is given. This validates the efficiency of our approach consisting in carefully selecting features before classification. Combining RF classifier with features selected by the KNN-based sequential forward FS method also allows us to exceed the performances obtained in [18], where authors also used an RF classifier applied on PLAID dataset using an optimal subset of 20 steady-state and transient features (selected through a systematic feature elimination process) and were able to reach an accuracy of 93.2%. The performances obtained by the LDA are the worst ones. As mentioned, this can be explained by the fact that the LDA classifier struggles when the number of individuals is very large, resulting in individuals overlapping between classes. In almost all the cases, data augmentation improves or slightly modifies the classification performance for all methods (exception for the KNN classifier). Finally, Table 6 depicts the average ${\mathrm{F}}_{\mathrm{M}}$ scores obtained for each FS method considering all the studied classifiers. The same observation as the one made for the proposed dataset can be made: the odd order harmonics features selected by the MI method are the ones for which the best average ${\mathrm{F}}_{\mathrm{M}}$ score is reached.

5.4. Transfer Learning Results

Now, we propose evaluating if the features selected from a dataset are able to be transferred to another dataset. For this, a cross-learning strategy is adopted for both studied datasets. The goal is to study to what extent the features selected for a particular dataset are invariant across HEAs and can be used to get good classification performances in another dataset. Indeed, as several common features are selected from each dataset separately, we assume that they convey common information that can be transferred from one dataset to another. This approach allows us to reduce the number of training samples from unknown HEAs. First, the subsets of selected features from the proposed dataset are tested using the several classifiers presented previously and applied to the PLAID dataset. The results are presented in

Table 8. Performance (in percentage) of the classification methods applied on the PLAID dataset using the different feature subsets selected in the proposed dataset. Results are sorted in descending order of F-measure.

Methods (Selected Features/Classifier)			Results
Feat.	# Feat.	Classifier	Acc.	${\mathbf{F}}_{\mathbf{M}}$	Rec.	Pre.	$\frac{\mathbf{Acc}.}{\#\mathbf{feat}.}$
MI	20	KNN	99.12	98.42	98.40	98.53	4.96
LDA	14	KNN	99.10	98.36	98.53	98.31	7.08
KNN-based Seq. forw. FS (K = 7)	12	R.F	98.99	98.36	98.48	98.35	8.25
KNN-based Seq. forw. FS (K = 7)	12	KNN	99.04	98.29	98.39	98.30	8.25
PCA	17	KNN	98.95	98.28	98.54	98.17	5.82
DNN	27	KNN	98.95	98.20	98.42	98.09	3.66
LDA-based Seq. forw. FS	18	KNN	98.92	98.14	98.331	98.13	5.49
LDA	14	R.F	98.96	97.95	98.09	97.94	7.07
MI	20	R.F	98.89	97.86	97.90	9791	4.94
PCA	17	R.F	98.77	97.84	98.04	97.79	5.81
LDA-based Seq. forw. FS	18	R.F	98.76	97.64	97.79	97.62	5.49
DNN	27	R.F	98.68	97.61	97.84	97.53	3.65
KNN-based Seq. forw. FS (K = 7)	12	DNN	98.08	96.83	97.40	96.87	8.17
LDA	14	DNN	97.85	96.35	97.10	96.32	6.99
MI	20	DNN	94.85	94.82	95.86	95.10	4.74
PCA	17	DNN	96.87	94.68	95.55	94.89	5.70
DNN	27	DNN	93.76	93.66	94.86	94.17	3.47
LDA-based Seq. forw. FS	18	DNN	95.76	93.18	94.22	93.43	6.32
MI	20	LDA	44.14	52.43	58.21	53.18	2.21
DNN	27	LDA	42.36	48.29	54.22	48.48	1.57
PCA	17	LDA	41.84	43.63	50.10	43.87	2.46
LDA	14	LDA	40.75	43.54	48.95	44.3	2.46
KNN-based Seq. forw. FS (K = 7)	12	LDA	37.73	38.23	42.81	40.30	3.14
LDA-based Seq. forw. FS	18	LDA	38.65	37.70	43.20	38.74	2.15

. Very good results are obtained with the KNN, the DNN, and the RF classifiers with the different subsets of selected features. The best ones are obtained with the KNN classifier and particularly when combined to the MI feature selection method, which allows us to obtain the best performances with only 20 features. Figure A3 shows the tested individuals that were misclassified, such as the following: 10% of the individuals belonging to class index 7 “Incandescent bulb light - Electrix soft white” were classified as class index 8 “Incandescent light bulb- Philips Duramax” or 10% of the individuals in class index 39 “Laptop-HP-C24” were classified as class index 40 “Laptop-Apple macbook air”. Second, the subsets of selected features from the PLAID dataset were tested using the several classifiers presented previously and applied to the proposed dataset. The results are presented in

Table 9. Performance (in percentage) of the classification methods applied on the proposed dataset using the different feature subsets selected in PLAID dataset. Results are sorted in descending order of F-measure.

Methods (Selected features/Classifier)			Results
Feat.	# Feat.	Classifier	Acc.	$F_M$	Rec.	Pre.	$\frac{\mathbf{Acc}.}{\#\mathbf{feat}.}$
LDA-based Seq. forw. FS	33	R.F	98.77	98.36	98.15	98.77	2.99
MI	20	R.F	98.76	98.36	98.15	98.77	4.93
KNN-based Seq. forw. FS (K = 7)	25	R.F	97.74	96.99	96.61	97.74	3.90
PCA	30	LDA	96.51	95.46	94.95	96.51	3.21
LDA	12	R.F	96.51	95.45	94.94	96.51	8.04
LDA-based Seq. forw. FS	33	LDA	96.31	95.18	94.63	96.31	2.92
KNN-based Seq. forw. FS (K = 7)	25	LDA	96.31	95.15	94.56	96.31	3.85
Feat.	# Feat.	Classifier	Acc.	$F_M$	Rec.	Pre.	$\frac{\mathbf{Acc}.}{\#\mathbf{feat}.}$
MI	20	LDA	96.31	95.15	94.56	96.31	4.81
DNN	17	R.F	95.70	94.33	93.65	95.70	5.63
PCA	30	R.F	95.49	93.98	93.23	95.49	3.18
MI	20	DNN	93.85	92.14	91.32	93.85	4.69
LDA	12	LDA	93.64	92.07	91.29	93.64	7.80
DNN	17	LDA	93.44	91.67	90.78	93.44	5.50
LDA	12	LDA	92.62	90.50	89.48	92.62	7.72
KNN-based Seq. forw. FS (K = 7)	25	DNN	93.24	91.02	89.92	93.24	3.73
LDA-based Seq. forw. FS	33	DNN	92.62	90.67	89.75	92.62	2.80
MI	20	KNN	91.80	89.27	88.01	91.80	4.59
DNN	17	DNN	90.98	88.38	87.16	90.98	5.35
PCA	30	DNN	90.37	87.56	86.27	90.37	3.01
PCA	30	KNN	85.04	80.66	78.58	85.04	2.83
LDA-based Seq. forw. FS	33	KNN	84.42	79.84	77.66	84.42	2.55
LDA	12	KNN	83.81	79.57	77.52	83.81	6.98
KNN-based Seq. forw. FS (K = 7)	25	KNN	84.01	79.37	77.11	84.01	3.36
DNN	17	KNN	77.87	72.96	70.65	77.87	4.58

. Very good results were obtained with the RF and the LDA classifiers with the different subsets of selected features. An accuracy of 98.77% was obtained when combining the RF classifier to the subset of features selected with the LDA sequential forward-FS method. The confusion matrix in Figure A4 shows that 50% of the individuals belonging to class index 5 “Fan-Coala Level 2” were classified as class index 4 “Fan-Coala Level 1”, and 50% of the individuals that belong to class index 53 “Washing machine-LG state (b)” were classified as class index 58 “Washing machine-LG state (g)”. Through cross transfer learning, we show that in both situations, the features selected from another dataset improve the classification rate. There is therefore transfer of knowledge on the discriminating “power” of the selected features.

6. Conclusions

In this paper, we addressed one of the main challenges of the NILM problem consisting in the HEA identification from electrical measurements. To this end, we covered a broad sweep of existing supervised FS and classification methods to show the efficiency of this approach for identifying distinct HEAs using the suitable set of relevant features.

As a first contribution, in addition to a novel publicly available dataset, we introduced a comparative evaluation of a large number of methods in an event-based NILM context, involving all the possible combinations of these techniques applied on two distinct annotated HEA datasets where each HEA signature made of relevant features is extracted from different categories and manufacturers.

Second, thanks to our proposed data augmentation and feature selection, we have improved the best HEA identification results on the PLAID dataset by obtaining a resulting classification rate above 99%. To our knowledge, this result outperforms the best available results obtained with the PLAID dataset using state-of-the-art methods. Furthermore, in this regard, validating our solution using two datasets has helped in (i) showing its high performance despite the data collection procedure being different and (ii) proving its capability to give very good results even if HEAs are from different categories and manufacturers.

Moreover, our results show that the number of extracted features can significantly be reduced to efficiently perform HEAs recognition. A cross transfer learning strategy was adopted by using the subsets of features selected by FS approaches applied to our dataset to classify HEAs of the PLAID dataset and conversely. Very good results were obtained, which confirms that the features selected with one dataset can be transferred to another one.

Several conclusions can be safely drawn from this study. First, the selected electrical features can be justified by the power supply topologies included in an HEA (the front-end circuitry that connects them to the power grid), which affect their current waveforms. Each subset of selected features contains for the most part odd-order harmonics related to power components. The performance of a designed classifier can be improved by the use of an optimal subset of features. Some features, such as $P,P_1,P_H,Q,Q_1,and{Q}_H$, are retrieved in most of the subsets of selected features, which shows their importance for HEAs’ identification. Secondly, several observations can be made according to the chosen classifier: DNN requires many training data and works poorly on the small proposed dataset (and works better on the PLAID dataset); LDA is sensitive to unbalanced datasets, and it works less well on PLAID; KNN method is efficient and sensitive to the choice of descriptors; RF is a state-of-the art method in automatic classification before the arrival of deep learning and is robust and gives good results in most cases. In addition, the augmentation of data shows that it is possible to improve in almost all cases classification performance for all methods (exception for PLAID with KNN classifier). Finally, features selected through FS methods in a dataset could be used to correctly identify unknown HEAs from another dataset using, for example, an unsupervised statistical modeling approach [6]. This addresses one of the biggest NILM challenges: generalization.