Machine Learning Design for High-Entropy Alloys: Models and Algorithms

Abstract

High-entropy alloys (HEAs) have attracted worldwide interest due to their excellent properties and vast compositional space for design. However, obtaining HEAs with low density and high properties through experimental trial-and-error methods results in low efficiency and high costs. Although high-throughput calculation (HTC) improves the design efficiency of HEAs, the accuracy of prediction is limited owing to the indirect correlation between the theoretical calculation values and performances. Recently, machine learning (ML) from real data has attracted increasing attention to assist in material design, which is closely related to performance. This review introduces common and advanced ML models and algorithms which are used in current HEA design. The advantages and limitations of these ML models and algorithms are analyzed and their potential weaknesses and corresponding optimization strategies are discussed as well. This review suggests that the acquisition, utilization, and generation of effective data are the key issues for the development of ML models and algorithms for future HEA design.

1. Introduction

The concept of high-entropy alloys (HEAs) has been raised by Cantor [1] and Yeh [2] since 2004. HEAs usually consist of four or five elements with atomic percentages (at.%) that are equal or nearly equal. Usually, the atomic fraction of each component is greater than five percent [3]. Their configurational entropy of mixing is high which is beneficial for the formation of the solid-solution phase [4]. They mainly possess Face-Centered Cubic (FCC), Body-Centered Cubic (BCC), and Hexagonal Close-Packed (HCP) structures [5]. Unlike conventional alloys, the complex compositions of HEAs lead to exceptional effects. HEAs usually exhibit outstanding physical and chemical properties, i.e., high mechanical properties, superior fatigue and wear resistance, good ferromagnetic and superparamagnetic properties, and excellent irradiation and corrosion resistance, etc. [6,7,8,9,10]. Using optimized composition design, a lighter density and better performance of HEAs can be obtained to achieve the purpose of lightweight HEAs [7]. However, due to the flexible compositions and ample performance tuning space, obtaining HEAs with low density and high properties solely through experimental trial-and-error methods requires a substantial investment of time and labor, resulting in low efficiency and high costs.

In recent years, the use of a computer-assisted design method has made significant progress in the field of HEAs. High-throughput calculation (HTC) is one promising computer-assisted design method, which is characterized by concurrent calculations and an automated workflow, enabling efficient computations for tasks at a high scale, rather than sequentially processing multiple tasks [11]. It initially focuses on the quantum scale, effectively meeting the demand for expediting the discovery of new materials and exceptional performance. In recent years, the concept of HTC has been applied to micro-thermodynamic scales, becoming a rapid method for obtaining phase information in metal structural materials [12]. High-throughput first-principles calculations and thermodynamics calculations are two main technologies of the HTC methods. High-throughput first-principles calculations, which do not rely on empirical parameters, can predict material property data by inputting element types and atomic coordinates [13]. They play an indispensable role in understanding and designing target materials from a microscopic perspective and enable the quantitative prediction of composition optimization, phase composition, and the structure–property relationship of materials. High-throughput first-principles calculations demonstrate specific roles and advantages for three aspects of HEAs: (1) the accurate construction of long-range disordered and short-range ordered structures; (2) the precise prediction of the stability of HEA phases; (3) the accurate calculation of the mechanical properties of HEAs [14]. The process of screening HEAs based on high-throughput thermodynamics calculations combines equilibrium calculations with non-equilibrium Scheil solidification calculations [15]. Using high-throughput calculation to predict the melting point, phase composition, and thermodynamic properties of HEAs after processing, it rapidly obtains the alloy composition space that satisfies criteria such as the melting point and phase volume fraction [16]. This assists in the quick analysis of effective alloy compositions, reducing the frequency of experimental trial and error. High-throughput thermodynamics calculations demonstrate specific functions and advantages in three ways: (1) the accurate acquisition of the phase diagrams and thermodynamic properties of HEAs; (2) the rapid retrieval of key microstructural parameters for HEAs; (3) the implementation of cross-scale analysis [12]. However, HTC technology mainly uses the theoretical calculation values such as phases, melting points, and various energies as data sources. Although the amount of data used in HTC calculation is huge, the direct correlation with the performance of HEAs and the accuracy of performance prediction are far from satisfactory. Therefore, the current HTC method can only be used as a reference criterion for HEA design. A certain number of experiments are still needed to verify the accuracy of the HTC design results.

In the past decade, the rapid ascent of artificial intelligence (AI) has brought a transformative revolution [17]. This revolution has not only fundamentally reshaped various domains of computer science, including computer vision and natural language processing, but has also made a significant impact on numerous scientific fields, including materials science. AI success comes from its ability to comprehend complicated patterns, and these complicated AI models and algorithms can be systematically refined through learning from real data, which is closely related to performance [18]. This capability is further enhanced by the availability of computational resources, efficient algorithms, and substantial data collected from experiments or simulations. The exponential increase in relevant publications is indicative of this trend. In essence, with a sufficiently large dataset of high quality, AI can effectively capture the intricate atomic interactions through standard procedures of training, validation, and testing [19]. Additionally, AI models and algorithms can identify non-linear structure–property relationships, which are challenging to determine through human observation [20]. These attributes position AI as an effective tool to tackle the challenges associated with the theoretical modeling of materials [21]. Machine learning (ML) is one of the most important technologies used for the AI design of materials [22]. This method, based on comprehensive experimental and theoretical studies, enables rapid data mining, revealing underlying information and patterns, and accurately predicting material properties for target material selection [23]. However, a small number of datasets becomes a key issue in HEA design, leading to high requirements for accuracy and the generalization ability of ML models and algorithms. This review aims to provide assistance for in-depth ML models and algorithms and their optimization strategies for HEA design by summarizing and sorting out the current research.

2. Machine Learning (ML) in HEA Design

ML is a multidisciplinary field involving probability theory, statistics, approximation theory, and algorithmic complexity theory [24]. The concept of ML, first introduced by Samuel in 1959, has evolved into a cross-disciplinary field spanning computer science, statistics, and other fields. Due to its efficient computational and predictive capabilities, ML has been gradually applied in materials science research [25]. In recent years, ML has gained widespread attention and demonstrated outstanding capabilities in the development of new materials and the prediction of material properties in the field of materials science [26]. A notable example is the 2016 article published in Nature, titled “Machine-learning-assisted materials discovery using failed experiments”, which successfully predicted chemical reactions and the formation of new compounds by mining a large dataset of failed experimental data, further fueling the momentum of research related to the application of ML to materials [27].

With an in-depth understanding of the concept of materials engineering, ML has found extensive applications in the design, screening, and performance prediction and optimization of materials [28]. Data-driven methods significantly expedite the research and development process, reducing time and computational costs. Whether on the micro or macro scale, this approach can be applied to new material discovery and the prediction of material properties in the field of materials science.

The discussion on the rules of phase formation has always accompanied the research on HEAs. The role of phases has been crucial in the design of HEAs. In the design strategy of HEAs, predicting the composition and phase stability of unknown alloy components is an essential aspect [29]. Widely used descriptors for phase prediction include entropy of mixing, enthalpy of mixing, elastic constants, melting temperature, valence electron concentration, electronegativity, etc. [30]. As research advances, the development involves utilizing differentiating alloy elemental contents as inputs or various combinations of the intrinsic properties of monatomic elements such as their physical and mechanical features. Examples of these features include atomic radius difference, valence electron count, configuration entropy, mixing enthalpy, etc. By directly modeling, relationships between the element combinations and phase formation can be obtained.

Besides phase formation, exploring the relationship between the compositions and properties of HEAs is also an essential task. By establishing a correlation model between feature parameters and properties such as strength, it is possible to achieve the rapid prediction of material performance based on chemical composition. This method, supplemented by a substantial amount of experimental data, offers valuable guidance for alloy composition design.

3. Common ML Models and Algorithms in HEA Design

So far, commonly used ML models and algorithms in HEA design include neural networks (NNs) [31,32,33,34,35,36,37,38,39,40,41,42,43], support vector machine (SVM) [44,45,46,47,48,49,50,51,52,53,54], Gaussian process (GP) [36,55,56,57,58,59,60,61], k-nearest neighbors (KNN) [62,63,64,65,66], and random forests (RFs) models and algorithms [67,68] etc.

3.1. Neural Networks (NNs)

NNs are computational models and algorithms inspired by the structure of the human brain [69]. The basic units of a NN are neurons, which simulate the connections and information transmission between biological neurons [70]. They are organized into layers such as the input layer, hidden layer, and output layer [71]. Each neuron receives inputs from neurons in the previous layer, applies weights to these inputs, and then produces an output through an activation function [72]. This output serves as the input for neurons in the next layer. By adjusting the weights of connections, the NNs can learn and adapt to patterns in the input data, enabling it to perform tasks such as learning and prediction. NNs have achieved significant success in areas such as image recognition, natural language processing, and speech recognition, demonstrating powerful performance in various applications. However, the data requirement of NN models and algorithms is huge.

J. Wang et al. [73] developed ensemble NN models and algorithms to test input data in order to design HEAs with a higher yield strength (YS) and ultimate tensile strength (UTS) (see Figure 1). They collected 501 data points from previous studies to be used for NN model training and validation. The data include the chemical composition, process conditions, and tensile mechanical properties of HEAs. The basic models and algorithms included in their experiment are the simple deep neural network (DNN) and the concatenated DNN and conventional neural network (CNN) model. They performed an inverse prediction and selected a random search and designed two HEAs, HEA1 and HEA2. The results are measured using multiple tensile tests. The results show that the combinations of the UTS and total elongation (T.EL.) of the present HEAs are better than the input data, as well as the HEAs from previous ML studies. This research demonstrated the effectiveness of the model in HEA design. This alloy design approach, specialized in finding multiple local optima, could help researchers design an infinite number of new alloys with interesting properties.

3.2. Support Vector Machine (SVM) Algorithm

In the ML model, the SVM algorithm has a good decision boundary [74]. The SVM algorithm was introduced by VaPnik as a supervised learning method falling under the category of binary classification models [75]. Its primary objective is to identify a separation hyperplane in the feature space that maximizes the margin, ensuring the correct classification of samples. This process is eventually transformed into a convex optimization problem. The segmentation principle of the SVM algorithm revolves around maximizing the interval. The SVM algorithm demonstrates significant advantages in addressing non-linear, high-dimensional, and small-sample problems [76]. Originally applied to linear classification, the SVM algorithm was later extended to handle non-linear examples and was further adapted for high-dimensional spaces [77]. It can also solve the problem of overfitting.

In the research of W. Zhang et al. [78], to elucidate the varying prediction accuracies of a singular characteristic parameter within amorphous alloys (AM), solid-solution alloys (SS), and high-entropy alloys containing intermetallic compounds (IM), three approaches were introduced. The first involves qualitatively explaining the high or low prediction accuracies of characteristic parameters in the three types of AM, SS, and HEA-IM alloys by employing a simple division across the entire range of the characteristic parameter. They chose the SVM algorithm as a predictive model and used atomic size difference (δ), mixing enthalpy (${\Delta H}_{mix}$), electronegativity difference (Δχ) and mixing entropy (${\Delta S}_{mix}$) as descriptors. Figure 2 illustrates the 4 × 4 scatter matrix representing the data distribution, aiding in comprehending the relationship among various features of the data [28]. In Figure 2, the correlation degree between 330 data points and four features is depicted. The diagonal scatter plot reveals the association between a single feature and the phase distribution, suggesting that a lone feature is insufficient for distinguishing phases. This is also the reason why the past research on phase prediction with a single feature has been ineffective. In the scatter plot involving two features, the distinct distribution of various features indicates their influence on the phase formation. Considering the average prediction accuracies of the four parameters, δ and ${\Delta S}_{mix}$ exhibit higher accuracies, whereas ${\Delta H}_{mix}$ and Δχ show lower accuracies. However, the prediction accuracies of Δχ in SS and HEA-IM alloys are the highest, reaching values of 83.1% and 72.0%, respectively, which are shown in Figure 3. On the other hand, the parameter ${\Delta H}_{mix}$ demonstrates the lowest prediction accuracies for AM, SS, and HEA-IM alloys, with values of 73.9%, 66.5%, and 65.3%, respectively.

In addition to W. Zhang, Nguyen et al. [79] also conducted research on SVM prediction. They employed the SVM method with hyperparameter tuning and the use of weighted values for the prediction of the alloy’s phase. They wrote a Python program to create a multi-principal element alloy (MPEA) and HEA dataset. Search operations are conducted to optimize the SVM hyperparameters. Finally, cross-validation is used to evaluate the accuracy of the prediction models utilizing the SVM algorithm with the optimized hyperparameters. In addition, they compared their SVM solution with the artificial neural networks (ANN) method, demonstrating that the SVM approach outperforms or is comparable to alternative methods employing the ANN method. Through experimental validation, they showed that incorporating the average melting point and standard deviation of melting point variables into the original dataset can enhance the prediction accuracy for MPEAs and HEAs. The conclusion drawn is that accurately predicting the structure of alloys contributes to an efficient search for new materials, providing feasible candidate materials for various applications that require materials with specific phases. Consequently, combining the SVM method with other ML algorithms is worthwhile for predicting the phases of MPEAs.

The above studies collectively demonstrate the superiority of using the SVM model for predicting and designing HEAs. In comparison to other ML models and algorithms, the SVM model proves to be more effective in handling multi-parameter issues, providing significant assistance in the design of HEAs.

3.3. Gaussian Process (GP) Model

The Gaussian process (GP) model is a statistical model that defines a distribution over functions, embodying a collection of random variables where any finite subset exhibits a joint Gaussian distribution [80]. Within the realm of ML, GP models find extensive application in regression, classification, and optimization tasks [81]. The fundamental concept underlying GP models is the representation of functions as random variables, characterized by a mean function and a covariance function. The mean function delineates the expected value of the function at each point, while the covariance function captures interdependencies between distinct points in the input space.

Tancret et al. [36] design HEAs employing GP statistical analysis (see Figure 4). The datasets include 322 alloys reported in the literature. In the realm of HEA design, the solitary application of any single method proves insufficient for the dependable prediction of a singular solid-solution formation. Rather, a robust strategy is introduced, grounded in a critical evaluation of existing criteria and a statistical analysis leveraging GP models. This innovative approach concurrently considers a multitude of previously proposed criteria, providing a comprehensive method for predicting the emergence of a single solid solution. Thus, it stands as an invaluable guide for the design of novel HEAs.

3.4. K-Nearest Neighbors (KNN) Model

The k-nearest neighbors (KNN) model is a machine learning model used for classification and regression tasks [82]. In the context of this study, the KNN model is employed to predict mechanical properties, specifically tensile strength and hardness [83]. The KNN model operates based on the principle of similarity [84]. Given a new data point, the algorithm identifies the ‘k’ nearest data points from the training dataset in the feature space. The prediction for the new data point is then determined by the average or weighted average of the outcomes of its k-nearest neighbors [85].

Raheleh et al. [65] employed a graph-based KNN approach to predict the phase of HEAs (see Figure 5). Each HEA compound has its distinct phase, falling into five categories: FCC, BCC, HCP, multiphase, and amorphous. A composition phase signifies a material state with a specific energy level. The effectiveness of the phase prediction lies in determining the practical applications of the material. Within the network, each compound has neighboring counterparts, and the phase of a new compound can be predicted based on the phase of its most similar neighbors. The proposed approach was implemented on the HEA network. The experimental results demonstrate that the accuracy of this method in predicting the phase of new alloys is 88.88%, surpassing that of other machine learning methods.

3.5. Random Forests (RFs) Algorithm

The random forests (RFs) algorithm is a ML algorithm that belongs to the ensemble learning category, utilized for both classification and regression tasks [86]. It constructs multiple decision trees during the training phase and aggregates their predictions for robust and accurate results. The algorithm initiates the process by creating numerous bootstrap samples from the original dataset. Each sample is then employed to train an individual decision tree [87]. To introduce diversity and prevent overfitting, a random subset of features is considered at each node of the decision tree. After training the decision trees, the algorithm combines their predictions. For regression tasks, the final prediction is the average of the individual tree predictions [88]. In classification tasks, the mode of the predictions is considered. The RFs algorithm is renowned for its resilience, adaptability, and effectiveness in handling high-dimensional datasets [89]. It is particularly valuable due to its ability to mitigate overfitting compared to individual decision trees, contributing to enhanced predictive accuracy.

Krishna et al. [90] utilized a ML approach to predict the multiphase alloy system, characterized by a combination of solid-solution and intermetallic phases (SS + IM), using a dataset of 636 alloys (see Figure 6). In the investigation of the RF classifier, parameters are varied for n estimators, representing the number of trees in the forest, and maximum depth, ranging from the root node to the leaf node. The range for n estimators is set from 10 to 190 with a 20-unit interval, while the maximum depth varies from three to fourteen. It is determined that, for the current investigation, the optimal parameter values for n estimators are 50, with a maximum depth of 13. These parameter values yield an average cross-validation score of 0.788 based on a set of five cross-validation folds.

However, every model has both advantages and limitations. Four problems in the use of NNs in data modelling are overfitting, chance effects, overtraining, and interpretation [91]. Jack V. Tu draws a similar conclusion in his work [92] and depicts the typical relationship between the network error and training duration, as shown in Figure 7.

As training progresses, the network error gradually decreases until reaching a minimum in the training set. However, the error in the test set may initially decrease and then begin to rise as the network starts overfitting the training data. It is common practice among neural network developers to periodically cross-validate the network on the test set during training and to save the network weight configuration based on either of two criteria: (1) the network with the minimum error in the training set (point B in Figure 7) or (2) the network with the minimum error in the test set (point A in Figure 7). The latter technique is often used to prevent the network from overtraining and overfitting.

The weaknesses of the SVM algorithm include algorithmic complexity, inefficiency in multi-classification, and imbalanced datasets [93]. J. Cervantes et al. also show some of the approaches used to improve the training time of the SVM algorithm. Eliminating data that are less likely to be support vectors is a crucial step. In addition, it is more efficient to decompose the dataset into multiple chunks and optimize each chunk separately.

The limitation of the GP model is mainly the inefficiency in dealing with high-dimensional data and non-stationary data. The KNN model shows slow prediction on large-scale datasets. The RFs algorithm performs well on large-scale datasets but may perform poorly when dealing with highly correlated features [94]. All the advantages and limitations of these five models are shown in

Table 1. Advantages and limitations of NNs, SVM, GP, KNN, and RFs. Data from Refs. [93,94].

Model	Advantages	Limitations
NNs	(1) Powerful for complex, non-linear relationships. (2) Robust to noisy data. (3) Ability to learn from large datasets.	(1) Prone to overfitting, especially with small datasets. (2) Requires careful tuning of parameters. (3) Black-box nature makes interpretation difficult.
SVM [93]	(1) Effective in high-dimensional spaces. (2) Works well with small to medium-sized datasets. (3) Versatile due to kernel trick for non-linear classification.	(1) Can be slow to train on large datasets. (2) Sensitivity to choice of kernel parameters. (3) Memory-intensive for large-scale problems.
GP	(1) Provides uncertainty estimates for predictions. (2) Flexible and interpretable modeling. (3) Can handle small datasets effectively.	(1) Provides uncertainty estimates for predictions. (2) Flexible and interpretable modeling. (3) Can handle small datasets effectively.
KNN [94]	(1) Simple and easy to understand. (2) No training phase, making it fast for inference. (3) Robust to noisy data and outliers.	(1) Simple and easy to understand. (2) No training phase, making it fast for inference. (3) Robust to noisy data and outliers.
RF	(1) High accuracy and robustness. (2) Works well with high-dimensional data. (3) Handles missing values and maintains accuracy.	(1) Can be slow to predict on large datasets. (2) Lack of interpretability due to ensemble nature. (3) May overfit noisy datasets if not tuned properly.

. Researchers need to select the appropriate model or combination of models based on the specific characteristics of the research subject.

In addition to the five ML models and algorithms above, some other common ML models and algorithms, such as principal component analysis (PCA) [95,96] and logistic regression (LR) [97], are used in design of HEAs as well. However, it should be pointed out that a large amount of high-quality data is still needed for the establishment and generalizing of the common ML models and algorithms. Advanced ML models and algorithms should be further explored.

4. Advanced ML Models and Algorithms in HEA Design

Recently, some ML models and algorithms with optimization strategies from underlying logical concepts have been raised up due to their high adaptability of small sample datasets, which are beneficial for HEA design. The active learning (AL) algorithm, the genetic algorithm (GA), the deep learning (DL) algorithm, and the transfer learning (TL) algorithm have achieved remarkable results in this field.

4.1. Active Learning (AL) Algorithm

The general idea of the AL algorithm [98] is to employ ML methods to identify samples that are challenging for classification. These samples are then subjected to human confirmation and review. Subsequently, the human-annotated data are reintegrated into the training process using supervised or semi-supervised learning models. This iterative process aims to progressively enhance the performance of the model by incorporating human expertise into the ML model. Compared to other ML methods, the AL algorithm excels in efficiently reducing data annotation costs [99]. Additionally, in scenarios characterized by class imbalance or label noise within the dataset, the AL algorithm can enhance data quality, enabling the model to focus more effectively on crucial samples for performance improvement, thereby enhancing the generalization capability of the model.

The framework of the AL algorithm has attained significant success in composition design and the performance prediction of HEAs. With the continuous influx of high-quality data, the effectiveness of AL models would be better. In addressing the challenge of simultaneously optimizing competing properties in machine learning-based material design, a constrained AL loop that incorporates domain knowledge was proposed by Li et al. [100] (see Figure 8). This approach aims to design high-entropy alloys with optimized strength and ductility by narrowing down the unexplored space using the valence electron concentration criterion. The active learning loop underwent six iterations, resulting in the synthesis of an alloy with an ultimate strength of 1258 MPa and an elongation of 17.3%. To uncover the underlying mechanism for synergistic optimization, the researchers characterized the phase structure and eutectic microstructure, delving into potential origins from the perspectives of strain hardening and crack initiation. The proposed framework, which integrates domain knowledge with machine learning, has the potential to facilitate the design of target materials with the coordinated optimization of competing properties. However, the most significant challenge that AL frameworks still face is the data source.

4.2. Genetic Algorithm (GA)

The GA, originating from computer-simulated studies of biological systems, is a stochastic global optimization method that emulates phenomena such as natural selection, replication, crossover, and mutation observed in genetics [101,102]. Starting from an initial population, the GA operates through random selection, crossover, and mutation operations to generate a group of individuals better suited to the environment [103]. This evolutionary process leads the population to progressively explore and converge toward more favorable regions within the search space. Iteration by iteration, the population undergoes continuous reproduction and evolution until it converges to a group of individuals that are most adapted to the environment, ultimately providing high-quality solutions to the problem at hand. This algorithm is widely adopted due to its global search capabilities and computational efficiency.

Moses et al. [104] conducted research to develop HEA powders with good control of grain-and-particle sizes (GPSs) for improved flowability (see Figure 9). First, the optimal process control agent type is determined through experimental exploration. Subsequently, a Taguchi experimental design is employed to plan and investigate the impact of parameters on the response variables and GPSs. This is followed by the development of a regression model to create predictive process models, and finally, the implementation of multi-objective optimization using a GA. Both grain size and particle size are simultaneously treated as optimization objectives. The simultaneous optimization of multiple objectives poses a significant challenge in genetic algorithms, as objectives typically conflict with each other. Consequently, multi-objective optimization problems often involve trade-offs. For example, increasing the milling speed to reduce the grain size could concurrently increase particle size, when finer particle sizes are desirable. The curve showing this optimal trade-off solution between objectives is recognized as the Pareto front. The Pareto front is a set of non-dominated solutions where each objective is considered equally favorable. The results obtained from the genetic algorithm align well with those from the Taguchi optimization.

Menou et al. [105] also employed a multi-objective optimization GA to devise HEAs. This methodology seeks an optimal compromise among factors such as single-phase stability, solid-solution hardening, and density. Through this approach, thousands of Pareto-optimal BCC HEAs have been designed. After casting and characterization, the alloy exhibits a microstructure comprising a singular disordered solid solution, uniquely combining outstanding hardness with moderate density. The GA generated 3155 alloys that exhibit Pareto optimality or non-domination. This implies that, for each alloy among them, there is no other alloy that simultaneously possesses a higher likelihood of forming a single solid solution, a greater Specific Solvent Heat (SSH), and a lower density. The outcomes are visually represented on a plot.

4.3. Deep Learning (DL) Algorithm

The deep learning (DL) algorithm is a subset of ML models that involves the use of deep neural networks (DNNs, a special type of neural network used in deep learning (DL) models, consist of multiple hidden layers and can more effectively learn and represent complex features and patterns [106]) to enable machines to learn and make decisions without explicit programming. It is inspired by the structure and function of the human brain, with layers of interconnected neurons that process information. The DL algorithm involves the process of uncovering the inherent patterns and hierarchical representations within sample data [107]. The information acquired through these learning processes significantly aids in interpreting data such as text, images, and sounds.

In the study of Zhu et al. [108], a DNN structure utilizing a residual network (RESNET) was introduced for predicting the phase formation of HEAs. It demonstrated an impressive overall accuracy of 81.9%. Compared to conventional machine learning models such as ANNs and standard DNNs, its Micro-F1 score highlighted its superiority in HEA phase prediction. Notably, this approach effectively mitigated the network degradation and enhanced the algorithmic accuracy. This work presents a novel avenue for developing phase formation prediction models using deep learning models, which holds broad relevance in facilitating the design of HEAs with innovative chemical compositions. The schematic diagram of the DNN model is shown in Figure 10.

4.4. Transfer Learning (TL) Algorithm

The TL algorithm is an algorithm that transfers knowledge from a source domain to a target domain, enabling the target domain to achieve better learning outcomes [109]. Generally, the source domain has an abundant amount of data while the target domain has limited data; TL aims to migrate the knowledge learned from the data-rich environment to the new environment with a smaller dataset.

In the study of Feng et al. [110], researchers aimed to utilize CNNs in conjunction with the TL algorithm to predict the crystalline structures of inorganic substances. The well-trained CNNs’ feature extractors were repurposed to extract features from a phase prototypes dataset, which comprised 17,000 inorganic substances and included 170 crystal structures, as well as two datasets on HEAs. These extracted features were subsequently fed into a random forest classifier as inputs. Exceptionally high classification accuracy, exceeding 0.9, was achieved across all three datasets. The visualization of the extracted features conclusively demonstrated the effectiveness of transferable feature extractors. This method presents a straightforward approach to rapidly constructing ML models with strong performance, eliminating the need for time-consuming manual feature engineering processes. Figure 11 shows the process of the research.

When developers aim for accurate property prediction through the creation of DL models, it is sometimes overlooked that certain physical properties of materials exhibit insensitivity to the local atomic environment. In response to this, Elemental Convolution Neural Networks (ECNets) were introduced by Wang et al. [111] to derive more generalized and globally applicable element-wise representations for the precise modeling of material properties. ECNets demonstrate improved prediction capabilities for properties such as band gaps, refractive indices, and elastic moduli in crystals. In its application to HEAs, the focus is specifically on the FeNiCoCrMn/Pd systems using data derived from Density Functional Theory (DFT) calculations. Leveraging knowledge gained from less-principal element alloys, performance in HEAs is enhanced through the TL technique. Additionally, the element-wise features extracted from the parent model, serving as universal descriptors, maintain accuracy even under limited data conditions. Employing this framework, concentration-dependent formation energy, magnetic moment, and local displacement are successfully obtained, thereby enriching the understanding of the physics behind these HEAs. Figure 12 shows the architecture for the ECNet model. This framework has the ability to acquire material representations from elemental embeddings trained on its own dataset. Through elemental convolution operations, the element-wise features serve as intermediary and final descriptors, extracting knowledge related to both atomic information and crystal structures, while being adaptable through the learning process of target material properties.

The advantages and limitations are shown in

Table 2. Advantages and limitations of AL, GA, DL, and TL.

Models	Advantages	Limitations
AL	(1) Reduces labeling effort (2) Improves model performance with limited data (3) Allows for adaptive training	(1) Requires expert query strategies (2) Can be computationally expensive (3) Depends on query strategy quality
GA [112]	(1) Optimizes complex problems (2) Searches across wide spaces (3) Handles multi-objective tasks	(1) No guaranteed global optimum (2) Complexity increases with dimensions (3) Sensitive to noisy objectives
DL [113,114,115]	(1) Learns complex patterns (2) Automatically extracts features (3) Excels in various tasks	(1) Needs large, labeled data (2) Prone to overfitting (3) Requires powerful hardware
TL [116,117]	(1) Leverages related knowledge (2) Reduces data need for new tasks (3) Speeds up training, improves performance	(1) Performance depends on domain similarity (2) Domain shift may affect transferability (3) Fine-tuning may still be necessary

.

To sum up, although these advanced ML models and algorithms possess a high adaptability to small sample datasets, which are beneficial for HEA design, nevertheless, these ML models and algorithms have higher requirements for data quality and discreteness.

5. Potential Weakness of ML Models and Optimization Strategies

However, the ML models used in HEA design have potential weaknesses such as data dependence, model complexity, generalization, interpretability, etc. For these weaknesses, some researchers have raised corresponding countermeasures and conducted optimization strategies.

5.1. Data Dependence and Generalization

The performance of ML models in HEA design is greatly influenced by both the quality and quantity of data. Limited or biased datasets can constrain the accuracy and applicability of predictions. Researchers usually use date pre-processing and data augmentation techniques to weaken the negative impact of data dependence.

Raw data are vulnerable to noisy, corrupted, missing, and inconsistent data; it is necessary to perform pre-processing steps, which is performed using classification, clustering, association and many other pre-processing techniques. This makes the knowledge extraction from the dataset much easier with cleaning, integration, transformation, and reduction methods [118]. The most common steps are data cleaning and noise handling.

Data cleansing is a process of identifying and correcting incorrect data or removing them from the dataset. It typically involves detecting and replacing incomplete, inaccurate, irrelevant, or noisy data and records. Generally, it follows these steps: (1) removing irrelevant or duplicate data; (2) correcting structural errors; (3) filling missing values using interpolation methods; and (4) assessing the consistency of the data. Various methods exist for handling noisy data. One option is to ignore the noise if the model can handle overfitting. Alternatively, the noise can be filtered, modified, or relabeled. Techniques like filtering erroneous values, removing them, or imputation can help clean noisy data. Additionally, binning can reduce minor errors by replacing values with general values derived for each bin, lowering overfitting risks, especially in shorter datasets. Binning methods include equal frequency or equal width binning.

Data pre-processing is crucial for the generalization performance of supervised ML algorithms. As the dimensionality of the input space increases, the quantity of training data grows exponentially. It is estimated that pre-processing may consume 50% to 80% of the entire classification process, underscoring its significance in model construction. Enhancing data quality is equally imperative.

Many global scholars have made progress in image data augmentation, discovering several methods for expanding sample size, thereby enhancing the generalization capability of neural networks in use. Utilizing the AlexNet model based on CNN architecture, several authors evaluated and compared various augmentation strategies. ImageNet and CIFAR10 datasets were employed by these authors. Several studies also assessed the effectiveness of different augmentation processes, such as flipping, rotating, adding noise, shifting, and cropping. Additionally, investigations have been conducted on the semantic segmentation of images and videos using deep learning techniques [119].

Additionally, the models may have a limited capability to generalize to novel alloys, especially when the training data do not encompass a diverse range of material properties and compositions. The generalization ability of models is mainly influenced by the quality and variety of data. Therefore, data augmentation also enhances the generalization capability of models.

5.2. Model Complexity

Advanced ML models, although potent, often come with complexity and high computational demands. These characteristics can restrict their applicability in practical scenarios where computational resources are limited or constrained. Additionally, the complexity of a model directly influences its generalization capability. Overly complex models may perform well on training data but poorly on unseen data because they may overfit noise and specific patterns in the training data.

Model simplification is a method aimed at enhancing interpretability by reducing the complexity of a model. It can be achieved through various approaches: (1) Reducing the number of model parameters. By eliminating certain parameters from the model, its complexity can be reduced. This approach is commonly referred to as “Model Compression”. (2) Reducing the structure of the model. By removing certain layers or nodes from the model, its structural complexity can be reduced. This approach is often called “Model Pruning”.

Model simplification can aid in better understanding the functioning of the model but may also lead to a decrease in performance. Therefore, when undertaking model simplification, it is essential to strike a balance between interpretability and performance.

5.3. Interpretability

Model interpretability refers to the ability of a model to explain its predictions or decision-making process in a simple and intuitive manner. In the field of deep learning, models are often considered black boxes because their internal workings are typically complex and difficult to understand. The improvement of model interpretability means that we can better understand how the model makes predictions or decisions based on input data, thereby enhancing trust and an understanding of the behavior of the model.

For models with weak interpretability, techniques such as feature importance analysis, local interpretability methods, and visualization can be used to enhance model interpretability. Local interpretable model-agnostic explanation (LIME) is a commonly used method.

The basic idea of LIME is to generate a set of local samples around a specific data point, and then use a simple interpretable model to explain the predictions of the original model on these local samples. By analyzing these explanatory models, one can understand how the original model makes decisions about specific predictions, thereby enhancing the understanding of the behavior of the model. The main advantage of LIME is that it is model-agnostic, allowing it to be applied to various types of models, including deep learning models and black-box models.

By improving the interpretability of the model, we can better understand its behavior, thereby increasing trust in the model and making it more actionable in practical applications.

5.4. Integration of Computational Theory and Experiment

Using experimental data to validate ML models helps determine the effectiveness of the model in practical applications and identify potential shortcomings. Additionally, based on the results of experimental validation, improvements can be made to the ML model. This may involve adjusting model parameters, optimizing algorithms, or adding new features to enhance the predictive performance of the model. Establishing a feedback loop is essential to continuously improve and iterate on a model, thereby enhancing its accuracy and reliability over time.

The research of Lee et al. [29] is a great example. It encompasses optimization, generation, and interpretation; the aim is to enhance performance and identify crucial design parameters for predicting phases of HEAs. Initially, a regularized deep neural network is established to predict HEA phases, optimizing model architecture, training, and regularization hyperparameters. To address the data scarcity issue in HEAs, the focus then shifts to the development of a conditional generative adversarial network for generating additional HEA samples. A significant improvement in model performance is observed through augmentation from the generative model, achieving a prediction accuracy of 93.17%. Finally, concentration is placed on understanding the contribution of design parameters to identifying the solid-solution phase. The work not only provides guidance for developing a reliable deep learning-based phase prediction model but also offers insights into explaining crucial design parameters to facilitate the design of novel HEAs. In particular, researchers have employed several regularization methods to optimize the DNN model. Regularization is a strategy that fine-tunes the balance of complexity in the neural network between trained and untrained cases. The DNN model is illustrated in Figure 13.

6. Prospect

With the deepening of research on HEAs and the exploration of potential applications, advanced tools used in HEA discovery and mechanistic exploration will be fully utilized in future studies. However, the methods of using ML to design HEAs still face some challenges.

The first challenge is related to data acquisition. Data are the foundation for the ML method. The greatest challenge for the entire field of ML is high-quality data. Currently, various research groups obtain data under different experimental conditions, necessitating careful calibration when merging the data. The acquisition of this data is time-consuming, and in the case of HEA issues, typically only small datasets are available for use. Meanwhile, computer simulation can generate a large amount of data, but these data are not directly related to performance. Thus, how to effectively extract these simulation data and combine them with experimental data is one of the key issues for the development of ML models and algorithms.

The second challenge is related to data compatibility. The results of ML predictions cannot be blindly trusted when the quantity, quality, and discreteness of data are not desirable. Uncertainty quantification is necessary in such research. In practice, there are different methods to assess the uncertainty of models, including Dempster–Shafer theory, fuzzy sets, interval methods, probability methods, Bayesian methods, etc. Using uncertainty quantification, it is possible to derive the model confidence intervals. This not only helps in understanding the reliability of predictions but also enables ML by adding data in regions with high uncertainty. With the optimization of methods, model accuracy has been significantly improved. For example, using adaptive design can enhance the accuracy of the model. Improving the data compatibility of ML models and algorithms can be beneficial for the future of HEA design.

The third challenge is related to data generation. In addition to the existing methods, other cutting-edge ML approaches may also potentially assist in the future design of HEAs. For example, self-supervised learning [120] models can address issues of small datasets. Meta-learning [121] models can achieve more advanced generalization capabilities. Quantum ML [122] models have the potential to address complex issues in HEAs. The efficient verification, iteration, and generation of data are essential to the future of ML methods.

7. Conclusions

ML has been an efficient and effective tool to assist in HEA design and is considered a promising approach. Commonly used ML models and algorithms in HEA design include ANN, SVM, GP, KNN, PCA, RFs, etc. These models and algorithms are trained using previous data to learn patterns and correlations and can then be used to predict or classify new data. Such models and algorithms can automatically make decisions by learning features and patterns in the data, facilitating the design of HEA compositions and performance predictions. The common ML models and algorithms have a high dependence on the quantity of data. However, a small number of datasets becomes a key issue in HEA design, leading to high requirements in accuracy and the generalization ability of ML models and algorithms.

Recently, some advanced ML models and algorithms with optimization strategies have been raised up due to their high adaptability of small-sample datasets, which are beneficial for HEA design. Nevertheless, these ML models and algorithms have higher requirements for data quality and discreteness. In addition, the ML models used in HEA design have potential weaknesses as well, such as data dependence, model complexity, generalization, interpretability, etc. For these weaknesses, some researchers have raised corresponding countermeasures and introduced optimized strategies.

Overall, the effective acquisition of experimental data and computer simulation data, the high compatibility in data quantity, quality, and discreteness, and the efficient verification, iteration, and generation of data are three development directions for ML models and algorithms in the future of HEA design.