Biomass Higher Heating Value Estimation: A Comparative Analysis of Machine Learning Models

Abstract

The research conducted focused on the capabilities of various non-linear and machine learning (ML) models in estimating the higher heating value (HHV) of biomass using proximate analysis data as inputs. The research was carried out to identify the most appropriate model for the estimation of HHV, which was determined by a statistical analysis of the modeling error. In this sense, artificial neural networks (ANNs), support vector machine (SVM), random forest regression (RFR), and higher-degree polynomial models were compared. After statistical analysis of the modeling error, the ANN model was found to be the most suitable for estimating the HHV biomass and showed the highest specific regression coefficient, with an R² of 0.92. SVM (R² = 0.81), RFR, and polynomial models (R² = 0.84), on the other hand, also exhibit a high degree of estimation, albeit with somewhat larger modelling errors. The study conducted suggests that ANN models are best suited for the non-linear modeling of HHV of biomass, as they can generalize and search for links between input and output data that are more robust but also more complex in structure.

1. Introduction

Renewable energy sources are gaining more attention, especially when they are used in sectors that want a stable and sustainable energy supply system. One of the most important of these sources is biomass, which includes organic matter of plant or animal origin that can be used directly as fuel or useful energy [1]. The basic products that can be derived from biomass include various feedstocks, fuels for transportation and the production of heat (energy) through direct combustion [2]. Biofuels derived from the biomass of agricultural and forestry production or from the cultivation of energy crops are considered promising sources for replacing existing conventional fuel sources and thus influencing the reduction in greenhouse gases and the increase in negative climate change [3]. For this biomass to be utilized as a fuel source through various conversions, the calorific value must be determined, which is the most important parameter in quality assessment but also promotes the use of feedstock as a fuel source [4].

The higher heating value is used in the design of energy systems of different sizes and is considered an extremely important parameter for the processes [5]. Aghel et al. (2023) [6] state that the relationship between the variables of biomass proximity analysis is not linear, so nonlinear modeling is a better alternative in finding a solution. In addition to the analysis of the HHV as a measure of fuel quality, proximate analysis can provide a detailed insight into the physicochemical composition of the biomass, which is of crucial importance for further modeling. Proximate analysis provides an insight into the physicochemical composition of the feedstock, considering the proportion of the main components of fixed carbon (FC), volatile matter (VM), and ash [4]. Velázquez Martí et al. (2023) [7] state that proximate analysis is one of the least economically demanding (laboratory) analyses for the determination and characterization of biomass as a fuel. For this reason, the datasets of the above analysis are suitable for modeling and processing in a machine learning (ML) environment. Understanding proximate analysis enables the use of advanced analysis techniques, such as the application of deep learning in modeling. In the field of modeling, the application of deep learning (DL) is becoming increasingly popular as an ML approach, which achieves significantly better results in modeling compared to less complex models and whose main feature is learning from data [8]. ML models should be used when it comes to understanding the interaction and “intelligent” analysis of large datasets. ML algorithms can be supervised, semi-supervised, unsupervised, and reinforcement learning [9]. Using ML models, it is possible to develop more accurate and reliable predictive models that are used as regression models for calculating the desired output values [10]. Classical ML models such as artificial neural networks (ANNs), random forest regression (RFR), support vector machines (SVMs), extreme learning machines (ELMs), K-nearest neighbors (KNNs), and decision trees (DTs) find extensive use across various scientific domains for modeling purposes. Among these, SVM stands out as a widely employed discriminant technique rooted in statistical learning theory, renowned for its robust generalization capabilities. Achieving the optimal network involves striking a balance between model complexity and training error, as highlighted by Ma et al. (2022) [11]. ELM, on the other hand, constructs a single-layer feedforward network through the random generation of input weights and biases for hidden layers, as elucidated by Wang et al. (2022) [12]. For sequence data, a plethora of state-of-the-art machine learning techniques, including ensemble learning models like XGBoost [13], LightGBM [14], and CatBoost, offer viable options. XGBoost excels in prediction accuracy and interpretability, particularly for high-dimensional datasets. Conversely, LightGBM accommodates large datasets and facilitates GPU training, demonstrating superior accuracy and speed compared to XGBoost. Data fusion further enhances forecasting accuracy by integrating gradient boosting-based categorical attributes, a capability supported by the CatBoost algorithm, as discussed by Dutta and Roy (2022) [15].

More specialized machine learning methods, such as artificial neural networks (ANNs), enable more complex analyses and predictions in the energy sector [16]. ANNs, as one of the ML techniques, can be successfully used for predictions and work on the principle of the human brain. This enables the processing of a large amount of data and the analysis of more complex relationships [17]. Although ANN models have numerous advantages in modeling, the optimization of the model is particularly challenging due to the complexity of the structure [18], but due to the ability to generalize and implement multidimensional nonlinearity, the system can also be used in real-time predictions [19].

Besides ANN models that handle complex data, random forest regression (RFR) appears to be a simpler but effective alternative in regression modeling and is recognized as one of the most effective tools in the field of regression modeling. Due to its simplicity, it is easier to optimize, which facilitates the learning process of models and predictions [20]. Despite its practical application, RFR shows high efficiency compared to other ML models [21]. Although RFR offers simplicity and efficiency, support vector machines (SVMs) represent a different approach that focuses on data classification in energy modeling [22]. SVM is a method for classifying data into two groups and searches for the hypersurface that best separates the data into two classes, positive and negative [23]. The support vector machine (SVM) method shows high performance when the edge of the class separation is clear. It is more effective in spaces with large dimensions, and it is also widely used in forecasting methods for reasons of “efficient” storage [24]. Polynomials are often used in regression models because they can approximate a variety of functions and describe complex patterns in data. In the context of regression, polynomials enable model flexibility as they can model non-linear relationships between variables [25]. The application of polynomial regression is particularly useful in situations where simple linear models do not provide sufficiently accurate predictions, making polynomial regression a powerful tool in data analysis and machine learning [26,27].

Qian et al. (2018) [28] developed different regression models in their research for predicting the calorific value based on the inputs of proximal analysis. The authors find that the most accurate model contains a combination of linear and polynomial expressions and interaction effects. The developed method showed a high degree of accuracy (R² > 0.91). In the study conducted by Aghel et al. (2023) [6], a dynamic Elman recurrent neural network (ENN) was developed to predict biomass HHV based on the input data from the terminal and proximate analysis. It was found that the ENN model built from a hidden layer with four nodes was the most accurate and had a relatively low modeling error. Matveeva and Bychkov (2022) [29] highlight the effectiveness of using MLP ANNs for datasets with pronounced heterogeneity, achieving an R² value of 0.880 ± 0.025.

A structure with the rectified linear unit (ReLU) activation function and the Adam training algorithm is recommended. It is emphasized that the amount of data and their preprocessing, including the rejection of dependent and noisy values, significantly improves the accuracy of the predictions.

This research aims to build several ML models (ANNs, RFR, and SVM) and a higher order polynomial regression model to determine the possibility of estimating the HHV value based on the biomass proximate analysis dataset. Reliability metrics such as the coefficient of determination (R²), root mean square error (RMSE), and mean absolute error (MAE) are used to compare models to determine accuracy and modeling error.

2. Materials and Methods

2.1. Data Collection and Preprocessing

Data on FC, VM, ash, and HHV were collected from the available database of scientific literature [30,31,32,33,34]. Data were collected in semi-structured form, and a total of 872 data points were used for modeling, with all missing data and duplicate values removed. Data processing included cleaning, normalization, coding, and feature selection to improve the data structure and remove any obstacles that could affect the model’s ability to learn effectively. The collected data can be found in Supplementary Table S1.

2.2. Data Analysis

In the context of the ML pipeline used, data analysis involved performing descriptive statistics, visualizing the data distribution through graphs, and creating correlation diagrams to understand the relationships between variables before proceeding with feature engineering and model building. The Python programming language was used for data analysis and the creation of ML models, including the Juyper notebook IDE environment with the associated software packages.

2.3. Model Selection

Four machine learning models were selected for modeling. An ANN was used due to its ability to model complex non-linear relationships and interactions between features [35]. The basic structure of this model consists of layers with artificial neurons (nodes) and activation functions between them. The model is configured with multiple layers to adapt to the specifics of the dataset. The ANN model was created as a sequential model with 2 layers. The first layer consisted of 10 artificial neurons and a ReLu function, while the second layer consisted of 1 neuron and a linear function. To optimize the model, the optimizer “Adam” and the mean square error (MSE) were used as loss functions. The model was trained with 4000 epochs and a batch size of 100, with the remaining dataset used to measure performance.

SVM is also included for robustness and efficiency in regression tasks. The model used radial basis function (RBF) to transform the data multidimensionally for easier separation (regression). The input data for the SVM model are split in a ratio of 70% for training and 30% for testing, with a “random seed” of 42. At the beginning of the pipeline, the Python standardization function (“StandardScaler”) and “GridSearchCV” were used to automatically adjust the parameters of the model C, gamma, and kernel by cross-validation (5-fold). The values 1, 0.1, 0.01, and 0.001 were tried out for the gamma value.

The RFR model uses an ensemble of multiple decision trees, which provides greater precision and stability in the estimation. The model also evaluates the importance of certain features in the modeling process, which provides additional insights during the analysis. An important hyperparameter in the random forest (RF) algorithm is the size of the feature set used to determine the optimal partitioning rule at each tree node [36]. The RFR model was set up with 200 trees, with a maximum depth of 20 per tree and a seed generator of 42 to ensure the reproducibility of the results.

Using polynomial regression, it is possible to fit data with non-linear trends by transforming features in the equation. Due to its “flexibility”, the model can be adapted to different degrees of polynomials depending on the complexity of the data. Regularized polynomials are implemented using a pipeline that integrates polynomial features up to the fourth degree together with ridge regression that applies L2 regularization with a strength parameter α = 0.5, effectively controlling model complexity and preventing overtraining.

2.4. Model Evaluation

The model evaluation procedure assessed the performance of the models created. The statistical metrics chosen were indicators of the coefficient of determination (R²) (1), which is used as a measure of how much variation in the real data can be represented by the model. The root mean square error (RMSE) (2) was used to measure the mean square error of the prediction. The mean absolute error (MAE) (3) parameter is used to calculate the absolute value of the error in the modeling. The calculated metrics were used to compare the models in terms of their predictive ability. In addition, the execution time of the “execution code” was measured for all models to determine the modeling speed. The formulas mentioned are presented as follows [37]:

$$R^2=1-\frac{{\displaystyle \sum _{i=1}^n{(X_i-Y_i)}^2}}{{\displaystyle \sum _{i=1}^n{(Y-Y_i)}^2}}$$

(1)

$$RMSE=\sqrt{\frac{1}{n}{\displaystyle \sum _{i=1}^n(y_i}-y_p{)}^2}$$

(2)

$$MAE=\frac{1}{n}{\displaystyle \sum _{i=1}^n\left|y_i-y_u\right|}$$

(3)

2.5. Model Optimization and Hyperparameter Tuning

Model optimization included adjustments to achieve the “best possible” performance and included calibration of the algorithm, adjustments to the learning process, and changes to the model architecture. By adjusting the hyperparameters, Tziachris et al. (2020) [38] found the settings that led to the best model performance on the test set. By optimizing the hyperparameters of the model, it is possible to achieve greater efficiency in modeling [38]. The configuration of the hyperparameters included a change in the learning rate, the number of trees in the RFR model, the number of layers and artificial neurons in the ANN model, and a change in the gamma value in the SVM model. Methods were also used to systematically search for parameters and determine the optimal combination.

3. Results

Figure 1 shows the distribution diagram and the descriptive statistics of the observed variables of the samples of the proximate analysis and the HHV biomass.

Figure 2 shows a scatter plot to determine the relationship between the input variables (FC, VM, and ash) and the output variables of the model (HHV).

Figure 3 shows the correlation matrix of the analyzed variables with the corresponding Pearson correlation coefficients of the input and output variables of the model.

The correlation matrix (Figure 3) shows the relationships (strength of the relationship according to Pearson’s correlation coefficient) between the four variables examined. For research purposes, it is particularly important to examine the correlation between the HHV and the variables of the proximate analysis.

Table 1. Performance of developed ML models used in this research.

Model Type	R2	MSE	MAE
ANN	0.92	1.33	0.77
RFR	0.84	1.61	1.03
SVM	0.81	1.75	1.25
Polynomial	0.72	2.14	1.53

ANNs—artificial neural networks; RFR—random forest regression; SVM—support vector machine; R²—coefficient of determination; MSE—mean squared error; MAE—mean average error.

shows the results of the statistical analysis of the ML model in terms of modeling error (MSE and MAE) and the degree of data overlap (R²).

Table 2. Execution time for developed models.

Model	Execution Time (s)
ANN	222.33
SVM	3.82
RFR	0.13
Polynomial	0.03

ANNs—artificial neural networks; SVM—support vector machine; RFR—random forest regression.

shows the execution time required for modeling the exposed values of the models created.

Figure 4 shows a scatter plot of the developed models in terms of regression representativeness.

4. Discussion

After statistically analyzing the variables needed to build the ML models, the mean values and standard deviation for FC, VM, ash, and HHV biomass were determined. The results of the above values can be seen in Figure 1. From this analysis, it can be concluded that there is considerable variability between the samples. The average value of FC is 20.81% and the standard deviation is 18.00%, indicating a wide range of values in the samples. For VM, most of the samples have a significant share, with a mean value of 71.73% and a standard deviation of 17.11%. The ash content varies with a mean value of 5.26% and a standard deviation of 5.08%, which indicates less variation compared to other measurements. For the HHV, the average is 20.63 MJ kg⁻¹ with a standard deviation of 4.67 MJ/kg, indicating moderate variability between samples. For the development of new models for the prediction of biomass components, Park et al. (2023) [39] gave the values for VM in the range of 59.07–87.3%, FC 10.44–38.64%, and ash 0.25–10.03%, indicating that the dataset collected is consistent with the range in the literature. For the HHV values, Harun et al. (2018) [40] reported values in the range of 17.58–19.15 MJ kg⁻¹. The average HHV value in this research is slightly higher. The reason for this is that certain biomass samples (from the collected data) were subjected to different treatments that affected the change in energy value. The displayed scatter diagrams (Figure 2) illustrate the relationships between the input characteristics (FC, VM, and ash) and the HHV within the analyzed dataset. Plotting FC against HHV indicates a potential correlation, suggesting that FC content may be an important factor in predicting HHV. Plotting VM against HHV shows more scattered points, suggesting a weaker relationship between VM and HHV. The plot of variable ash with HHV also shows a scattered pattern, suggesting that ash content may not have a strong or clear influence on HHV. To develop the model, Qian et al. (2018) [28] also analyzed the relationship between individual components of the proximate analysis and the HHV. The scatter plot showed that FC has the highest coefficient of determination (R² = 0.62) with HHV, while the variables ash and VM have a weaker relationship with HHV. Observing the strength of the relationship using the Pearson correlation (Figure 3), the variables ash (−0.39) and VM (−0.40) are negatively correlated, while the variable FC (0.40) is positively correlated with HHV. The creation of correlation matrices [41] shows that the variables VM and FC show no significant statistical correlation with HHV, while the variables VM and FC show a strong linear relationship. The best-fit ML model was the ANN model developed with 3 artificial neurons in the input layer and 10 in the hidden layer, and the ReLU activation function was used because it is considered the most suitable activation function for neural networks [42]. A linear activation function was used to calculate the output data. The model was optimized with the Adam optimizer using the MSE parameter as the loss function. The SVM model created used the RBF (radial basis function) kernel. With the RBF function, it is possible to achieve better results to minimize the computer load by separating the samples from different classes [43]. The model also included a regularization parameter (C = 100) and a gamma parameter. The RFR regression model was created with 200 estimators and a maximum depth of the trees of 20, and the model is based on the construction of multiple decision trees that calculate the output data based on the average prediction of each tree [5]. The polynomial regression was created with three levels, and power regularization was included (alpha = 0.5) to prevent overfitting of the model to the data. Overfitting occurs when a model learns from the training set and tests improvements using errors from the training samples, leading to a disproportionate impact on the model’s performance with original data [44]. When comparing the performance of four ML models on the regression task of predicting the value of the HHV variable, the ANN achieved the best results in all metrics. An R² value of 0.92, an RMSE of 1.33, and an MAE of 0.77 show that the ANN can predict the value of the HHV variable well. The SVM also performed well with an R² of 0.81, an RMSE of 1.75, and an MAE of 1.25. RFR and polynomial showed satisfactory performance with an R² of 0.84, an RMSE of 1.61, and an MAE of 1.03 and an R² of 0.72, an RMSE of 2.14, and an MAE of 1.53, respectively. These results show that the ANN is the best model for predicting the value of the HHV variable. Aghel et al. (2023) [6] reported the model accuracy for various developed ANN models as R² 0.83–0.88 and MAE 0.66–0.85, while Afolabi et al. (2022) [5] reported 1.21 for the ANN model and 1.01 for the RF model.

Based on the statistical indicators used, the ANN model showed the highest efficiency, which indicates high precision and reliability in prediction. The ANN model had the longest execution time (222.33 s), which is a consequence of the complexity of the model, the number of neurons, and the learning process, which involves a multi-layered structure and optimization. The shortest execution time is seen for third-degree polynomials due to the relative simplicity and non-complex regression tasks. The SVM and RFR models also have a shorter execution time due to the lower complexity of the model and the complexity of the optimization process. The experiments conducted by Guimarães et al. (2023) [45] demonstrate that predicting the training time of models like decision trees and neural networks is achievable with reasonable accuracy. The average prediction error is 0.103 s for decision trees and 21.263 s for ANNs, which is acceptable considering their training times span up to 14 s and over 1400 s, respectively. The application of the ML model in the estimation of HHV biomass shows clear advantages in terms of accuracy and reliability of predictions. Models such as ANNs often achieve high R² values (>0.90) and lower RMSE and MAE values, demonstrating their ability to estimate by managing the data [46,47]. On the other hand, it should be noted that although high accuracy and lower modeling errors are achieved, longer runtimes are often required due to higher learning complexity [48], which can be a limiting factor when computational resources are limited. On the other hand, other ML models that have simpler structures provide faster results due to their (relative) simplicity, although they are not as accurate in estimation [48].

5. Conclusions

The artificial neural network (ANN) demonstrated superior performance across all evaluated metrics in this scientific study. The model achieved an R² value of 0.92, an RMSE (root mean square error) of 1.33, and an MAE (mean absolute error) of 0.77, indicating its high accuracy in predicting the HHV variable. This proficiency is attributed to the ANN’s complex structure, which involves a multi-layered configuration and an optimization process, encompassing numerous neurons and a comprehensive learning procedure. However, it is noteworthy that the ANN model had the longest execution time, clocking in at 222.33 s. This extended duration is primarily due to the model’s complexity, the extensive number of neurons, and the intricate learning process that includes a multi-layer structure and optimization. In contrast, the polynomial regression model of the third degree demonstrated the shortest execution time at 0.03 s. This efficiency stems from its relative simplicity and the less complex nature of the regression tasks it handles. Despite this, the polynomial model exhibited the lowest accuracy and the highest error in modeling, with an R² value of 0.72, an RMSE of 2.14, and an MAE of 1.54. The most effective ANN model was achieved with a configuration of 4000 epochs, with 3 artificial neurons in the input layer and 10 in the hidden layer. Additionally, the ReLU (rectified linear unit) activation function was employed in this model. Furthermore, a diagram and correlation analysis concerning the fixed carbon (FC) content of the HHV suggested a potential correlation. This indicates that the content of fixed carbon might be a significant factor in predicting the higher heating value, highlighting an area for further exploration in the field.