Machine Learning and Knowledge Extraction

This study investigates the relationship between consumer personality traits, specifically openness, and responses to product designs. Consumers are categorized based on their levels of openness, and their affective responses to nine vase designs, varying in curvature and line quantity, are evaluated. The study then introduces the inverse clustering approach, which prioritizes maximizing predictive model accuracy over within-cluster similarity. This method iteratively refines cluster assignments to optimize prediction performance, minimizing errors in forecasting consumer design preferences. The results demonstrate that the inverse clustering approach yields more effective clusters than personality-based clustering. Moreover, while there is some overlap between personality-based and accuracy-based clustering, the inverse clustering method captures additional individual characteristics, extending beyond personality traits and improving the understanding of consumer product design response. The practical implications of this study are significant for product designers, as it enables the development of more personalized designs and optimization of product features to enhance specific consumer perceptions, such as robustness or esthetic appeal. Full article

Representing and interpreting human opinions within an unstructured framework is inherently complex. Hesitant fuzzy linguistic term sets offer a comprehensive context that facilitates a nuanced understanding of diverse perspectives. This study introduces a methodology that integrates sentiment analysis with hesitant fuzzy linguistic term sets to effectively aggregate and compare news from diverse sources. By employing linguistic scales, our approach enhances the interpretation of various perceptions and attitudes, facilitating comprehensive knowledge extraction and representation. The main objective of this research is to conduct a comparative analysis of news coverage across European countries in relation to the Israel–Gaza war. This analysis aims to capture the multifaceted sensitivities surrounding the ongoing situation, highlighting how different nations perceive the conflict. Full article

Interdisciplinary research (IDR) is essential for addressing complex global challenges that surpass the capabilities of any single discipline. However, measuring interdisciplinarity remains challenging due to conceptual ambiguities and inconsistent methodologies. To overcome these challenges, we propose a deep learning approach that quantifies interdisciplinarity in scientific articles through semantic analysis of titles and abstracts. Utilizing the Semantic Scholar Open Research Corpus (S2ORC), we leveraged metadata field tags to categorize papers as either interdisciplinary or monodisciplinary, establishing the foundation for supervised learning in our model. Specifically, we preprocessed the textual data and employed a Text Convolutional Neural Network (Text CNN) architecture to identify semantic patterns indicative of interdisciplinarity. Our model achieved an F1 score of 0.82, surpassing baseline machine learning models. By directly analyzing semantic content and incorporating metadata for training, our method addresses the limitations of previous approaches that rely solely on bibliometric features such as citations and co-authorship. Furthermore, our large-scale analysis of 136 million abstracts revealed that approximately 25% of the literature within the specified disciplines is interdisciplinary. Additionally, we outline how our quantification method can be integrated into a TRIZ-based (Theory of Inventive Problem Solving) methodological framework for cross-disciplinary innovation, providing a foundation for systematic knowledge transfer and inventive problem solving across domains. Overall, this approach not only offers a scalable measurement of interdisciplinarity but also contributes to a framework for facilitating innovation through structured cross-domain knowledge integration. Full article

Artificial intelligence (AI) is increasingly applied in a wide range of healthcare and Intensive Care Unit (ICU) areas to serve—among others—as a tool for disease detection and prediction, as well as for healthcare resources’ management. Since sepsis is a high mortality and rapidly developing organ dysfunction disease afflicting millions in ICUs and costing huge amounts to treat, the area can benefit from the use of AI tools for early and informed diagnosis and antibiotic administration. Additionally, resource allocation plays a crucial role when patient flow is increased, and resources are limited. At the same time, sensitive data use raises the need for ethical guidelines and reflective datasets. Additionally, explainable AI is applied to handle AI opaqueness. This study aims to present existing clinical approaches for infection assessment in terms of scoring systems and diagnostic biomarkers, along with their limitations, and an extensive overview of AI applications in healthcare and ICUs in terms of (a) sepsis detection/prediction and sepsis mortality prediction, (b) length of ICU/hospital stay prediction, and (c) ICU admission/hospitalization prediction after Emergency Department admission, each constituting an important factor towards either prompt interventions and improved patient wellbeing or efficient resource management. Challenges of AI applications in ICU are addressed, along with useful recommendations to mitigate them. Explainable AI applications in ICU are described, and their value in validating, and translating predictions in the clinical setting is highlighted. The most important findings and future directions including multimodal data use and Transformer-based models are discussed. The goal is to make research in AI advances in ICU and particularly sepsis prediction more accessible and provide useful directions on future work. Full article

Balancing the accuracy and the complexity of models is a well established and ongoing challenge. Models can be misleading if they are not accurate, but models may be incomprehensible if their accuracy depends upon their being complex. In this paper, semilattices are examined as an option for balancing the accuracy and the complexity of machine learning models. This is done with a type of machine learning that is based on semilattices: algebraic machine learning. Unlike trees, semilattices can include connections between elements that are in different hierarchies. Trees are a subclass of semilattices. Hence, semilattices have higher expressive potential than trees. The explanation provided here encompasses diagrammatic semilattices, algebraic semilattices, and interrelationships between them. Machine learning based on semilattices is explained with the practical example of urban food access landscapes, comprising food deserts, food oases, and food swamps. This explanation describes how to formulate an algebraic machine learning model. Overall, it is argued that semilattices are better for balancing the accuracy and complexity of models than trees, and it is explained how algebraic semilattices can be the basis for machine learning models. Full article

Urban happiness prediction presents a complex challenge, due to the nonlinear and multifaceted relationships among socio-economic, environmental, and infrastructural factors. This study introduces an advanced hybrid model combining a gradient boosting machine (GBM) and neural network (NN) to address these complexities. Unlike traditional approaches, this hybrid leverages a GBM to handle structured data features and an NN to extract deeper nonlinear relationships. The model was evaluated against various baseline machine learning and deep learning models, including a random forest, CNN, LSTM, CatBoost, and TabNet, using metrics such as RMSE, MAE, R², and MAPE. The GBM + NN hybrid achieved superior performance, with the lowest RMSE of 0.3332, an R² of 0.9673, and an MAPE of 7.0082%. The model also revealed significant insights into urban indicators, such as a 10% improvement in air quality correlating to a 5% increase in happiness. These findings underscore the potential of hybrid models in urban analytics, offering both predictive accuracy and actionable insights for urban planners. Full article

The now-globally recognized concerns of AI’s environmental implications resulted in a growing awareness of the need to reduce AI carbon footprints, as well as to carry out AI processes responsibly and in an environmentally friendly manner. Benchmarking, a critical step when evaluating AI solutions with machine learning models, particularly with language models, has recently become a focal point of research aimed at reducing AI carbon emissions. Contemporary approaches to AI model benchmarking, however, do not enforce (nor do they assume) a model initial selection process. Consequently, modern model benchmarking is no different from a “brute force” testing of all candidate models before the best-performing one could be deployed. Obviously, the latter approach is inefficient and environmentally harmful. To address the carbon footprint challenges associated with language model selection, this study presents an original benchmarking approach with a model initial selection on a proxy evaluative task. The proposed approach, referred to as Language Model-Dataset Fit (LMDFit) benchmarking, is devised to complement the standard model benchmarking process with a procedure that would eliminate underperforming models from computationally extensive and, therefore, environmentally unfriendly tests. The LMDFit approach draws parallels from the organizational personnel selection process, where job candidates are first evaluated by conducting a number of basic skill assessments before they would be hired, thus mitigating the consequences of hiring unfit candidates for the organization. LMDFit benchmarking compares candidate model performances on a target-task small dataset to disqualify less-relevant models from further testing. A semantic similarity assessment of random texts is used as the proxy task for the initial selection, and the approach is explicated in the context of various text classification assignments. Extensive experiments across eight text classification tasks (both single- and multi-class) from diverse domains are conducted with seven popular pre-trained language models (both general-purpose and domain-specific). The results obtained demonstrate the efficiency of the proposed LMDFit approach in terms of the overall benchmarking time as well as estimated emissions (a 37% reduction, on average) in comparison to the conventional benchmarking process. Full article

Data drift can significantly impact the outcome of a model. Early detection of data drift is crucial for ensuring user confidence in predictions. It allows the user to check if a particular model needs retraining using updated data to adapt to the evolving process dynamics. This study compares five different statistical tests, namely four unidimensional and a new multidimensional test (MSPC), to identify data drift in both mean and deviation. While some are designed to detect drift in mean only, like our multidimensional proposal, others respond to changes in both mean and deviation. However, our Hotelling multidimensional method can be trained once and then applied in a single stage to any data stream with several attributes, and it can identify the most relevant variables causing a data drift with one execution, thus avoiding the need for a single univariate test for each attribute. Moreover, our method yields the relative importance of each attribute for drift and allows users to increase or decrease the relative weight of each variable regarding drift detection. It also may be capable of detecting drift due to changes in multivariate interactions. This behavior is especially suitable for real-world scenarios, such as industry, finance, or healthcare environments. Full article

Deep learning models are widely used for medical image analysis and require large datasets, while sufficient high-quality medical data for training are scarce. Data augmentation has been used to improve the performance of these models. The lack of transparency of complex deep-learning models raises ethical and judicial concerns inducing a lack of trust by both medical experts and patients. In this paper, we focus on evaluating the impact of different data augmentation methods on the explainability of deep learning models used for medical image classification. We investigated the performance of different traditional, mixing-based, and search-based data augmentation techniques with DenseNet121 trained on chest X-ray datasets. We evaluated how the explainability of the model through correctness and coherence can be impacted by these data augmentation techniques. Local Interpretable Model-agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP) methods were used. Sanity checks and overlapping scores were applied to confirm the correctness and coherence of explainability. The results indicate that both LIME and SHAP passed the sanity check regardless of the type of data augmentation method used. Overall, TrivialAugment performs the best on completeness and coherence. Flipping + cropping performs better on coherence using LIME. Generally, the overlapping scores for SHAP were lower than those for LIME, indicating that LIME has a better performance in terms of coherence. Full article

Graph neural networks (GNNs) have been effectively implemented in a variety of real-world applications, although their underlying work mechanisms remain a mystery. To unveil this mystery and advocate for trustworthy decision-making, many GNN explainers have been proposed. However, existing explainers often face significant challenges, such as the following: (1) explanations being tied to specific instances; (2) limited generalisability to unseen graphs; (3) potential generation of invalid graph structures; and (4) restrictions to particular tasks (e.g., node classification, graph classification). To address these challenges, we propose a novel explainer, GAN-GNNExplainer, which employs a generator to produce explanations and a discriminator to oversee the generation process, enhancing the reliability of the outputs. Despite its advantages, GAN-GNNExplainer still struggles with generating faithful explanations and underperforms on real-world datasets. To overcome these shortcomings, we introduce ACGAN-GNNExplainer, an approach that improves upon GAN-GNNExplainer by using a more robust discriminator that consistently monitors the generation process, thereby producing explanations that are both reliable and faithful. Extensive experiments on both synthetic and real-world graph datasets demonstrate the superiority of our proposed methods over existing GNN explainers. Full article

The deep learning model has attracted widespread attention in the field of rolling bearing remaining useful life (RUL) prediction due to its advantages of less reliance on prior knowledge, high accuracy, and strong generalization. However, a large number of prediction models use very complicated artificial feature extraction and selection methods to build the original input features of the deep learning model and health indicator. These approaches do not fully exploit the capabilities of deep learning models as they continue to heavily rely on prior knowledge, The accuracy of their predictions largely hinges on the quality of the input features, and the generalization of manually crafted features remains uncertain. To address these challenges, in this paper, an end-to-end prediction model for the remaining useful life of rolling bearings is proposed, which is divided into three modules. First, a short-term Fourier transform module is incorporated into the model to automatically obtain the time–frequency information of the signal. Then, the convolutional next (ConvNext) module, which is a simple and efficient pure convolutional neural network, is utilized to extract features from the spectrogram. Finally, we capture the short-term dependence and long-term dependence by two parallel channels Transformer and self-attention convolutional long short-term memory (SA-ConvLSTM), and the self-attention mechanism is employed for the adaptive prediction of the bearing’s remaining useful life. Through integration with artificial intelligence, this method proposes a high-performance solution for predicting the remaining useful life of bearings. It has minimal reliance on manual labor, stronger fitting capabilities, and can be widely used for predicting the remaining useful life of bearings. Full article

Large language models (LLMs) achieve remarkable predictive capabilities but remain opaque in their internal reasoning, creating a pressing need for more interpretable artificial intelligence. Here, we propose bridging this explanatory gap by drawing on concepts from topological quantum computing (TQC), specifically the anyonic frameworks arising from $SU{\left(2\right)}_k$ theories. Anyons interpolate between fermions and bosons, offering a mathematical language that may illuminate the latent structure and decision-making processes within LLMs. By examining how these topological constructs relate to token interactions and contextual dependencies in neural architectures, we aim to provide a fresh perspective on how meaning and coherence emerge. After eliciting insights from ChatGPT and exploring low-level cases of $SU{\left(2\right)}_k$ models, we argue that the machinery of modular tensor categories and topological phases could inform more transparent, stable, and robust AI systems. This interdisciplinary approach suggests that quantum-theoretic principles may underpin a novel understanding of explainable AI. Full article

Red-light running at signalised intersections poses a significant safety risk, necessitating advanced predictive technologies to predict red-light violation behaviour, especially for advanced red-light warning (ARLW) systems. This research leverages Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) models to forecast the red-light running and stopping behaviours of drivers in connected vehicles. We utilised data from the Ipswich Connected Vehicle Pilot (ICVP) in Queensland, Australia, which gathered naturalistic driving data from 355 connected vehicles at 29 signalised intersections. These vehicles broadcast Cooperative Awareness Messages (CAM) within the Cooperative Intelligent Transport Systems (C-ITS), providing kinematic inputs such as vehicle speed, speed limits, longitudinal and lateral accelerations, and yaw rate. These variables were monitored at 100-millisecond intervals for durations from 1 to 4 s before reaching various distances from the stop line. Our results indicate that the LSTM model outperforms the GRU in predicting both red-light running and stopping behaviours with high accuracy. However, the pre-trained GRU model performs better in predicting red-light running specifically, making it valuable in applications requiring early violation prediction. Implementing these models can enhance red-light violation countermeasures, such as dynamic all-red extension (DARE), decreasing the likelihood of severe collisions and enhancing road users’ safety. Full article

Detecting malware has become extremely important with the increasing exposure of computational systems and mobile devices to online services. However, the rapidly evolving nature of malicious software makes this task particularly challenging. Despite the significant number of machine learning works for malware detection proposed in the last few years, limited interest has been devoted to continual learning approaches, which could allow models to showcase effective performance in challenging and dynamic scenarios while being computationally efficient. Moreover, most of the research works proposed thus far adopt a fully supervised setting, which relies on fully labelled data and appears to be impractical in a rapidly evolving malware landscape. In this paper, we address malware detection from a continual semi-supervised one-class learning perspective, which only requires normal/benign data and empowers models with a greater degree of flexibility, allowing them to detect multiple malware types with different morphology. Specifically, we assess the effectiveness of two replay strategies on anomaly detection models and analyze their performance in continual learning scenarios with three popular malware detection datasets (CIC-AndMal2017, CIC-MalMem-2022, and CIC-Evasive-PDFMal2022). Our evaluation shows that replay-based strategies can achieve competitive performance in terms of continual ROC-AUC with respect to the considered baselines and bring new perspectives and insights on this topic. Full article

Implementing machine learning technologies in manufacturing environment relies heavily on human expertise in terms of domain and machine learning knowledge. Yet, the required machine learning knowledge is often not available in manufacturing companies. A possible solution to overcome this competence gap and let domain experts with limited machine learning programming skills build viable applications are digital assistance systems that support the implementation. At the present, there is no comprehensive overview over corresponding assistance systems. Thus, within this study a systematic literature review based on the PRISMA-P process was conducted. Twenty-nine papers were identified and analyzed in depth regarding machine learning use case, required resources and research outlook. Six key findings as well as requirements for future developments are derived from the investigation. As such, the existing assistance systems basically focus on technical aspects whereas the integration of the users as well as validation in industrial environments lack behind. Future assistance systems should put more emphasis on the users and integrate them both in development and validation. Full article

Research and applications in artificial intelligence have recently shifted with the rise of large pretrained models, which deliver state-of-the-art results across numerous tasks. However, the substantial increase in parameters introduces a need for parameter-efficient training strategies. Despite significant advancements, limited research has explored parameter-efficient fine-tuning (PEFT) methods in the context of transformer-based models for instance segmentation. Addressing this gap, this study investigates the effectiveness of PEFT methods, specifically adapters and Low-Rank Adaptation (LoRA), applied to two models across four benchmark datasets. Integrating sequentially arranged adapter modules and applying LoRA to deformable attention—explored here for the first time—achieves competitive performance while fine-tuning only about 1–6% of model parameters, a marked improvement over the 40–55% required in traditional fine-tuning. Key findings indicate that using 2–3 adapters per transformer block offers an optimal balance of performance and efficiency. Furthermore, LoRA, exhibits strong parameter efficiency when applied to deformable attention, and in certain cases surpasses adapter configurations. These results show that the impact of PEFT techniques varies based on dataset complexity and model architecture, underscoring the importance of context-specific tuning. Overall, this work demonstrates the potential of PEFT to enable scalable, customizable, and computationally efficient transfer learning for instance segmentation tasks. Full article

In this tutorial, we present a compact and holistic discussion of Deep Learning with a focus on Convolutional Neural Networks (CNNs) and supervised regression. While there are numerous books and articles on the individual topics we cover, comprehensive and detailed tutorials that address deep learning from a foundational yet rigorous and accessible perspective are rare. Most resources on CNNs are either too advanced, focusing on cutting-edge architectures, or too narrow, addressing only specific applications like image classification. This tutorial not only summarizes the most relevant concepts but also provides an in-depth exploration of each, offering a complete yet agile set of ideas. Moreover, we highlight the powerful synergy between learning theory, statistics, and machine learning, which together underpin the deep learning and CNN frameworks. We aim for this tutorial to serve as an optimal resource for students, professors, and anyone interested in understanding the foundations of deep learning. Full article

In the emerging field of computational gastronomy, aligning culinary practices with scientifically supported nutritional goals is increasingly important. This study explores how large language models (LLMs) can be applied to optimize ingredient substitutions in recipes, specifically to enhance the phytochemical content of meals. Phytochemicals are bioactive compounds found in plants, which, based on preclinical studies, may offer potential health benefits. We fine-tuned models, including OpenAI’s GPT-3.5-Turbo, DaVinci-002, and Meta’s TinyLlama-1.1B, using an ingredient substitution dataset. These models were used to predict substitutions that enhance the phytochemical content and to create a corresponding enriched recipe dataset. Our approach improved the top ingredient prediction accuracy on substitution tasks, from the baseline 34.53 ± 0.10% to 38.03 ± 0.28% on the original substitution dataset and from 40.24 ± 0.36% to 54.46 ± 0.29% on a refined version of the same dataset. These substitutions led to the creation of 1951 phytochemically enriched ingredient pairings and 1639 unique recipes. While this approach demonstrates potential in optimizing ingredient substitutions, caution must be taken when drawing conclusions about health benefits, as the claims are based on preclinical evidence. This research represents a step forward in using AI to promote healthier eating practices, providing potential pathways for integrating computational methods with nutritional science. Full article

In the era of rapidly expanding graph-based applications, efficiently managing large-scale graphs has become a critical challenge. This paper introduces an innovative graph reduction technique, Node-Centric Pruning (NCP), designed to simplify complex graphs while preserving their essential structural properties, thereby enhancing the scalability and maintaining performance of downstream Graph Neural Networks (GNNs). Our proposed approach strategically prunes less significant nodes and refines the graph structure, ensuring that critical topological properties are maintained. By carefully evaluating node significance based on advanced connectivity metrics, our method preserves the topology and ensures high performance in downstream machine learning tasks. Extensive experimentation demonstrates that our proposed method not only maintains the integrity and functionality of the original graph but also significantly improves the computational efficiency and preserves the classification performance of GNNs. These enhancements in computational efficiency and resource management make our technique particularly valuable for deploying GNNs in real-world applications, where handling large, complex datasets effectively is crucial. This advancement represents a significant step toward making GNNs more practical and effective for a wide range of applications in both industry and academia. Full article