Review of Time Domain Electronic Medical Record Taxonomies in the Application of Machine Learning

Abstract

Electronic medical records (EMRs) help in identifying disease archetypes and progression. A very important part of EMRs is the presence of time domain data because these help with identifying trends and monitoring changes through time. Most time-series data come from wearable devices monitoring real-time health trends. This review focuses on the time-series data needed to construct complete EMRs by identifying paradigms that fall within the scope of the application of artificial intelligence (AI) based on the principles of translational medicine. (1) Background: The question addressed in this study is: What are the taxonomies present in the field of the application of machine learning on EMRs? (2) Methods: Scopus, Web of Science, and PubMed were searched for relevant records. The records were then filtered based on a PRISMA review process. The taxonomies were then identified after reviewing the selected documents; (3) Results: A total of five main topics were identified, and the subheadings are discussed in this review; (4) Conclusions: Each aspect of the medical data pipeline needs constant collaboration and update for the proposed solutions to be useful and adaptable in real-world scenarios.

1. Introduction

Translational medicine (TM) includes collaboration between medical clinicians, biomedical engineers, and scientists to develop artificial intelligence (AI) models that account for differing data sources, data collection methods, and other real-world factors. In some cases, TM reduces the time from solution development to deployment. It does so by allowing effective communication between different players for shared goals. It is characterized by integrating digital biomarkers, multi-omics profiling, model-based data, AI, biomarker-guided trial designs, and patient-centric companion diagnostics. Therefore, the taxonomies identified in this review are guided by translational medicine [1]. Ref [2] presents the following components of the complete medical record including: connected fitness devices, patient-focused personal health records, individual behavioral patterns, pharmacy-focused medical adherence data, provider-focused medical records, connected medical devices, and genomic information.

In these complete electronic medical records (EMRs), the use of time-series data is essential because most of the biomarkers are tracked as trends in time [3]. EMRs help in disease monitoring, pandemic monitoring, adjustment of lifestyles, hospitals, intensive care units, and integration of healthcare services. Machine learning (ML) is “The ability of computers to advise decisions based on the available data” [4]. ML has been used in applied clinical studies for some time now [5], and recent renewed interest has been driven by increased data availability [6] and an increase in computational capacity [7]. The most common medical data types are images, such as computerized tomography (CT) scans, time-series data, and blood, urine, and metabolic panels. Some noteworthy literature reviews cover computer vision techniques to assist clinical decision-making [8,9,10]. However, there is a need to review the broader research landscape relating to the application of ML in time-series data in EMRs to give the relevant taxonomies, patterns in literature, emerging trends, and knowledge streams in this field. Time series can be defined as repeated readings from a device over a period of time. The frequency of the readings can be periodic or non-periodic and range from thousands of times per second (e.g., accellerometry) to a few times a day (e.g., glucose monitoring).

Our review of the relevant studies has found numerous pertinent publications [11], including a previous work by Davy et al. (2015) that investigates the effectiveness of chronic care AI models employed in primary healthcare [12]. A more recent work by Chen et al. (2020) reviews the use of probabilistic ML models applied to healthcare data [13], while Wang et al. (2021) examined the latest advancements in graph-based analytics in healthcare [14]. The application of telemedicine in maintaining electronic health records was recently reviewed by Gu et al. (2019) using a cite space analysis [15]. Cite space was also used in a scientometric review of the application of latent discriminant analysis in healthcare data by Tean et al. (2019) [16]. Emerging challenges when using unstructured EMRs were deliberated by Adnan et al. (2020) in the context of using big unstructured data in healthcare [17].

In comparison to the studies above our study has three novel features:

• It identifies taxonomies within the field after a systemic search of research databases.
• It finds these taxonomies based on the principles of translational medicine so that the reader may find all the information needed for a translational solution in one place.
• It identifies the core challenges and advancements in each taxonomy and provides a rigorous volume of the literature to serve as a baseline.

The following are the research questions we will try to address:

• What paradigms fall under the umbrella of AI in time series and graph-based healthcare data?
• What are the latest advances in these domains?
• What are the latest challenges in these taxonomies?

Although earlier literature reviews have covered the application of specific algorithms and problems in EMRs, as shown in

Table 1. Comparison with earlier works.

References	Time Series	Disease Specific	Translational Medicine
[12]	✓	X	X
[18]	✓	✓	X
[19]	X	✓	X
[16]	X	✓	X
[15]	✓	✓	X
[17]	✓	✓	X
Our work	✓	X	✓

, a need exists to develop taxonomies and discuss recent findings.

2. Materials and Methods

A paradigm is a collection of elements based on a common lexeme [20]. Identifying the paradigms of fields interacting with each other is imperative to achieving overarching solutions. In this review, these paradigms are based on an industry perspective of TM [1] and TM principles. The three Ts in TM are developing treatments and interventions, testing the proposed interventions’ effectiveness, and deploying these applications in the real world [21]. In this review, all the paradigms in the field considered are presented by Figure 1, and the challenges in applying AI-based solutions to TM are also discussed.

In this review, the literature was searched from the following databases: PubMed, Scopus, and Web of Science, and the papers between 2015–2022 were selected. The search terms were ((“Physiological sensors” OR “Biomedical sensors” OR “Bio-medical Sensors”) AND (“Machine Learning” OR “ML” OR “Artificial Intelligence” OR “AI” OR “Deep learning” OR “DL” OR “Reinforcement learning” OR “Electronic health records”) NOT (“Security and Privacy”) NOT (“images”) NOT (“Robot”)). The search was limited from 2015 to November 2022. Only articles were included, and reviews were not made part of the search criteria. The proceedings of various conferences were excluded from the search. Applying these filters and only selecting English language records resulted in 320 articles being removed, resulting in the number of articles selected for review totaling 164. These records were then collected and thoroughly read to answer the following questions:

• What is the type of data?
• What kind of algorithm is used?
• What pre-processing methods are used?
• What post-processing methods are used?
• What data privacy standards are observed?
• What interoperability or fusion techniques are used?

By answering these questions in the form of a table, the top three methods are identified within each paradigm. However, if the top three topics in a paradigm are repeated in any other paradigm, the next three topics are also discussed in the section to give a holistic overview of the topic. For example, the top three topics in the subtopic: time-series data and structured data are the same and, therefore, positioned in the structured data section.

An Ishikawa Fishbone Diagram (Figure 2) presents the different paradigms on the topic.

Figure 2 is a representation of the paradigms found in the literature under the topic at hand. The inclined axes represent the major topics in the field, for example, types of data and levels of automation. These inclined axes are also divided, shown as horizontal arrows, to identify the relevant subtopics.

3. Results

After collecting the records through the process previously described, then reading their methods and results, using TM principles, we enabled the paradigms to then be identified. The subsequent discussion is arranged as follows; first, we identify the most commonly occurring topics in the records that we have collected within each paradigm, and then the challenges within the paradigms are elaborated upon. This review is a combination of a narrative review built on systematic data collection.

3.1. Types of Data

One of the most obvious choices of paradigms is the type of data, which can depend upon the source of data (different types of sensors), organization of data (structured, unstructured), and representation of data (time series, images, graph representations).

Representation of Data

The way that the data are represented makes them suitable for a particular type of analysis to then be undertaken. The inductive biases of different ML models can be matched to different types of data representation. For example, as we will see in later parts of this review, the graph-based data representations are widely used for phenotyping (phenotype of the collection of observable effects of a disease), since the inductive bias of a graph-based ML model suits the relational nature of the problem of phenotyping. Thus, the first axis, along which we separated the data, is according to how it is being represented. This can take tabular, time series, or graph forms.

1.

Time series (tabular representation): Time-series data contain information from physiological events in the form of time-varying biomarkers. Three leading solutions are specific to this data type: motif or pattern detection, data generation/imputation, and time-series forecasting. Generative AI models can potentially overcome the lack of access to time-series data by synthetically producing the missing and unknown data; however, accuracy by patient needs to be proven for each application, where missing data can be missing readings within a time series or the complete absence of a time-series recording in the EHR. Guided evolutionary networks (GENs) combine artificial neural networks and optimization algorithms such as genetic algorithms. These are used to fuse various information sources [22,23]. GENs are also used to discover time-series motifs in ECG data [24]. Ref. [25] uses a multilayer perceptron for time-series forecasting in healthcare data. The following

Table 2. Comparison of the time-series solution.

References	Applications	Sensors	Generative	Predictive	Clinical	Imputation
[26]	Motif Discovery	ECG	✓	X	✓	X
[22]	Motif Discovery	ECG and EEG	✓	✓	X	✓
[24]	Anomaly Detection	ECG	X	✓	✓	X
[25]	Expenditure Calculation	Healthcare data	X	✓	✓	X
[27]	Benchmarking	MIMIC-III	X	✓	✓	X
[28]	Imputation	ECG, MIMIC	X	✓	✓	✓

presents a comparison of the representative literature.

2.

Graph representation: Healthcare data are relational, which makes them suitable for graphical representation. Relational data are characterized by the relations or dependence that exists amongst the rows and columns [29]. Graph-based techniques are used for developing graph-based representations of healthcare data, identifying clinical pathways and phenotypes of disease, and performing predictive modelling of disease and interventions. For example, refs. [30,31] are some typical graph representations of healthcare data. Ref. [32] determines the temporal phenotypes based on graph representations of healthcare data. Ref. [33] is a fog-based temporal network graph analysis for the Chikungunya virus in India. Ref. [34] uses a proximity-preserving graph embedding to represent electronic health records for hypertension. Ref. [35] incorporates metadata of the patients along with their vitals and lab results to learn a graph representation of electronic healthcare data. Ref. [36] is a study that employs cryptographic techniques for information embedding in the healthcare data. Ref. [37] is another knowledge-graph-based phenotyping technique for subarachnoid hemorrhage. Ref. [38] is a graph-based visualization for sensitive outcomes in medicine for healthcare data. Ref. [39] is a graph-based channel fusion for wrist pulse detection. Ref. [40] uses graphs for learning a lower dimensional representation of drug–disease interaction. As illustrated in [41], the main applications of graphs in medical interventions are drug–drug interaction, drug–disease interaction, protein–protein interaction, medical term classification, and protein function prediction. The three main methods to realize these ends are matrix factorization, random walk, and neural network-based methods. These include Laplacian methods, as demonstrated in [42], deep walk methods, as shown in [43], and neural networks, as illustrated in [40]. Graph algorithms commonly used can be categorized into temporal data mining [44], causal and contextual [45], and patient enteric graphs [46].

It is worth noting that there is no unique graph representation for sensor data or electronic medical records. Hence, most research focuses on developing graph-based presentations. One crucial research area is benchmarking and creating a numeric qualitative marker of adequate representation. There are several limitations of time-series- and graph-based healthcare data; these include data sparsity [47], noise [48], limited generalizability [49], and lack of context [49].

The following

Table 3. Comparison of graph-based solutions.

References	Application	Techniques Used	Data	Contributions	Predictive	Descriptive
[32]	Temporal Phenotyping	Attention Models	MIMIC-III	10% greater than RNN in disease prediction and 3% improved areas under ROC	✓	✓
[38]		Hinge Loss		Predicted congestive health failure with an 80% accuracy. The area under the curve for patient readmission increased by over 50% from the spectral clustering	✓	✓
[36]	Graph representation	Note Binning	STRIDE	Developed term and concept mappings	X	✓
[39]	Feature fusion	Multi-Channel feature fusion	Pressure and Photo–electric Sensors	93.1% accuracy in predicting diabetes from pulse detection data.	X	X

presents a comparison of the representative literature.

3.2. Structure of Data

Another way to classify the type of healthcare data is the structure of available data. Most healthcare data are not structured against a set of rules. The structure of data dictates the kind of preprocessing required or the kind of algorithms that can be used.

• Structured data: This follows a definite set of rules or schemes [50]. The main issues when using ML and structured data are data generation, data fusion, pattern detection, privacy preservation, and prediction of outcomes. Privacy preservation is guided by HIPAA rules [51]. Generative algorithms are used extensively to impute the missing data in the structured datasets [52,53]. Data fusion is another typical application of ML for combining two different kinds of structured data [54,55]. Federated learning that trains the models based on data from various decentralized devices is used extensively for privacy preservation of healthcare data [56,57,58,59]. ML and structured data are also valuable in predicting the outcomes of interventions, for example, [60] analyzes the user’s choice in the event of alerts from clinical decision systems for potential drug–drug interference. Ref. [61] uses structured and unstructured data to find the social determinants of health characterized by social behavior, demographic features, and environmental factors of medical status and health care access.
  Ref. [62] is a systemic review of records from PubMed and Web of Science on the detection of strokes from structured data that found the leading keyword to be mortality and the most used algorithms to be neural networks, support vector machines, and XGBoost. Ref. [63] is another review that looked at the statistical and predictive machine learning models for cancer risk and found the cox model [64] is the most commonly used algorithm for predicting disease onset based on the input features. Ref. [65] used AI to auto-complete structured clinical records based on context. Ref. [66] is a model to detect probable cases of dementia using structured and unstructured data that uses a latent Dirichlet algorithm for feature extraction and a logistic regression model. The key issues of research for structured data in healthcare are detecting phenotypes from electronic health records [67,68], privacy and encoding of information [69,70,71,72], data harmonization from various sources [72], synthetic data generation for research [73,74,75], and fairness and bias in the structured data [76].
• Semi-structured data: These EMRs have no specific structure, enabling categorical data, meta-data, and numerical data to be entered in any field. The key areas in application of ML in unstructured data is in the conversion to structured data, predictive modeling, and interoperability of different kinds of data sources. For example, an application of ML with unstructured data for predictive modeling is used [77] to derive contextual information to generate semi-structured data from electronic medical records. Ref. [78] is a method to allocate resources from the knowledge of semi-structured healthcare data. Ref. [79] uses HL7 standards to develop the interoperability of structured, semi-structured, and unstructured data to develop obesity phenotypes. Ref. [80] is another such system that uses open EMRs to this end. Ref. [81] detects autism from semi-structured and unstructured data using a combination of skip-gram models.
• Unstructured data: Most EMRs are unstructured [82]. Key research areas for ML applications in unstructured data are conversion amongst the various kinds of data structure and predictive modeling. An example of predictive modeling using unstructured data [83] employs unstructured EMRs to phenotype depression in youth. Latent Dirichlet Analysis (LDA) and other dimensionality reduction methods are used to obtain the hidden information between different kinds of data and then leverage it for predictive modeling [84,85,86,87]. A priori algorithms and other Bayesian methods are used to convert unstructured data to structured data [37,88,89], and in so doing, these works can also combine with structured data to make predictions [90,91]. Another technique that is relevant to the conversion of unstructured data to structured data is distant supervision. Distant supervision is a method for labeling the data by utilizing the known structures of similar data [92,93]. Exploratory text analysis is also used for pattern analysis for predictive modeling in this [94,95].

The following

Table 4. Comparison of unstructured data.

References	Application	Techniques Used	Evaluation Metrics	Structured Data
[83]	Detection of clinical depression	NLP	Specificity:97%. Sensitivity:45%	X
[84]	Disease prediction	LDA	AUC 0.94, Sensitivity 0.87 and Specificity 0.87	✓
[94]	HPV detection	NLP	AUC: 0.861	X
[92]	Breast cancer detection	NLP	AUC 0.91, Sensitivity: 0.861, Specificity 0.878, Accuracy 0.870.	✓

compares these techniques:

The different ML techniques used in conjunction with unstructured data are clustering, classification, boosting, and a combination of these three. Clustering can help with phenotyping and grouping together different clinical pathways. Classification requires labeling the data, which can be taxing for a large volume of clinical notes. Boosting models can leverage the different structures present in unstructured data to make meaningful predictions, especially, risk and mortality.

Natural language processing techniques are extensively applied to unstructured data to detect disease onset. Data harmonization and standardization is also an essential topic of discussion in unstructured healthcare.

In the healthcare context, structured and semi-structured data are typically easier to work with and analyze because they have some inherent structure. Unstructured data, such as free-text notes in electronic health records, can be more challenging to work with because they require more effort to extract meaningful information.

3.3. Types of Sensing Elements

Types of data are dependent on the types of sensing elements used. There are many types of sensing elements including wearable sensors, mobile device sensors, ingestible sensors, medical devices from hospitals, and a combination of all or some of the factors mentioned earlier.

• Wearable sensors: These bridge the gap between assessment and onset prediction. The data sources measure the biomarkers from the physiological signals in real-time, making this a vital component of multi-omics profiling [96].
• Mobile devices: Along with real-time monitoring using mobile sensors, mobile devices also allow for input from the user, making them helpful in tracking medical adherence [97].
• Ingestible sensors: Drug adherence [98] and monitoring [99] are some applications of ingestible sensors.
• Medical devices from hospitals: These include connected medical devices intended to enhance healthcare quality for people in the hospital [100].
• Combinations: The combination of the sensors enables the Internet of Medical Devices [101].

The critical limitations of wearable sensors are the contextualization of data and integration with the existing clinical care pathways; hence, a challenge exists to show clinical efficacy. Most historic clinical data are taken from a patient at rest (e.g., resting heart rate) with the assumption that only disease can shift homeostasis, and most wearable data are ambulatory (e.g., heart rate during a workout) with confounders such as physical activity making traditional clinical interpretation challenging. Interoperability is another crucial aspect that needs to be addressed when deciding on different sensing elements. This will help increase the generalizability of models by allowing them access to various kinds of data.

3.4. Data Preprocessing

As we have seen previously, data can come from various sources and in various forms. For the successful application of ML, these data must be harmonized and standardized. Data harmonization standards and intelligent interoperability techniques are the two classes along the knowledge stream. Another axis to classify data preprocessing techniques is the data fusion methods, which include feature level, data level, and decision level fusion. One more way to organize the data prior to analysis is through preprocessing techniques. These include filtering, feature extraction, and natural language processing techniques.

• Data harmonization standards: These standards describe the preprocessing technique that prepares different kinds of data to become compatible with each other. It allows the AI to access a diversity of information through access to researcher and institution knowledge [102]. Some standards are specific to the medical cases they deal with [103,104,105]; however, there exists a set of medical means to ensure interoperability. The most common standards are Health Level 7 (HL7), openEHR, and ISO/IEEE 11073 Personal Health Data (PHD) standards [106], International Statistical Classification of Diseases version 10 (ICD-10) [107] and Current Procedural Terminology (CPT) codes [108]
• Intelligent interoperability: Here, ML or other algorithms are used to combine the information from different data sources, and particularly EMRs. In intelligent interoperability of healthcare components, artificial intelligence or some other rule-based systems are used to automatically draw the relevant information from the EMRs or sensor data. These systems use different algorithms to ensure the interoperability of various data sources. The following

  Table 5. Comparison of interoperability techniques.

  Name	Properties	References
  Blockchain technology	Focused on patients rather than healthcare providers. Data are linked to the patient, aggregated, and then sensitive information such as allergies is published on the blockchain, ensuring privacy and data immutability.	[109,110]
  Internet of Things	It employs the principles of the internet of things for data interoperability. It uses the protocols of Message Queuing Telemetry Transport (MQTT) to publish the relevant patient information.	[111]
  Dynamic Semantic Web services	It uses the dynamic semantic web to convert the data into the HL7 framework.	[112]
  Cloud Based Interoperability	It uses cloud-based models, for example, amazon web services, Microsoft Azure, and IBM Watson, to convert it into an openEHR or HL7 standard.	[113]
  Knowledge Graphs	Knowledge graphs are used for the interoperability of biomedical data.	[37]

  elucidates such strategies. Although these systems allow for effective data communication while ensuring information integrity, one key issue is allowing for the encoding of categorical features so that the information is stored effectively.

The methods used for interoperability include NLP, data mapping and transformation, data quality assessment, predictive analytics, and anomaly detection. They are used to promote one or more of these: Standardization of data, using application programming interface (API), using middleware and frameworks such as the Da Vinci project, and health information exchanges (HIEs). While effective, NLP techniques are very resource intensive. Data mapping and transformations can be very narrow in application. Data quality assessments can be used to compare inconsistencies but require constant updates and maintenance. Predictive analytics can help improve care coordination and resource allocation, but this is also effective in a narrow range of situations. Anomaly detection can identify unusual or unexpected patterns in healthcare data, potentially flagging issues that may need to be addressed, but can suffer from alert fatigue if the sensitivity is too high. However, it requires certain contextual information to be more effective.

3.

Data Fusion: A physiological event can be observed with the help of various sensors, each sensing a unique aspect of the physiological event. The system has to fuse or combine information from different sensing elements for a holistic understanding of the event. This is done at multiple levels. In industry 4.0, healthcare systems, these sensing elements are spread across time and space (wearable sensors, ambulances, and hospitals). Fusing information from multiple sensors provides a more holistic picture of healthcare, including detection, phenotyping, disease progression, and other related data-powered solutions.

Ref. [114] exhibits a combination of different layers of data fusion in connected healthcare, from individual sensors to detect medical events, to a network of connected devices, and finally, fusing information amongst various institutions. Ref. [115] displays a sensor fusion model between communication systems. Ref. [116] defines different levels of data fusion. These include signal level fusion, feature level fusion, and decision level fusion. Kalman Filtering is a popular statistics method for signal level fusion and is widely used in biomedical sensor networks. Weighted averages are also widely used to penalize sensors with more noise in a sensor network [117,118,119]. Particle filtering, amongst various other variants, is also used extensively for signal level fusion in sensor networks in healthcare [120]. Ref. [121] uses temporal evidence theory for signal level fusion for activity recognition. Feature level fusion means each sensing element’s features are calculated and fused. Ref. [122] calculates a linear combination of features to obtain a new feature. Ref. [123] is a weakly supervised program for feature-level fusion. Decision level fusion is a way to fuse decisions based on different information streams. There exist many such systems in the context of healthcare [124,125].

The critical issue in all these is developing a plastic nature of fusion techniques. A plastic fusion technique would be flexible to change with the emerging problem because different features or data may have other significance for each model.

There are several key limitations to data harmonization standards for electronic medical records for example:

• Complexity—Data harmonization standards can be complex and may require significant resources to implement and maintain.
• Limited adoption—Not all electronic medical record systems may adopt the same data harmonization standards, which can limit the ability to exchange data between systems.
• Changing standards—Data standards can change over time, which can make it difficult to maintain compatibility with other systems.
• Privacy and security concerns—The exchange of patient data between systems can raise concerns about privacy and security. Careful measures must be taken to ensure that patient data are protected when they are shared between systems.
• Cost—Implementing and maintaining data harmonization standards can be expensive, particularly for smaller healthcare organizations.
• Intended use—some coding is designed for a different reason than it is used for, e.g., reimbursement versus treatment.

3.5. Decision Systems

The nature of decision systems is specific to the problem they deal with. One axis along which the decision systems can be classified is the medical problems they solve, which include data quality, phenotyping, medication adherence, graph representation of data, detection of disease, and mortality prediction. One more axis along which the decision systems can be classified is the nature of algorithms, natural language processing, time series analysis, and graph neural networks.

1.

Data Quality: The quality of the data acquired in healthcare is essential for the credibility of the predicted outcomes. Data quality issues are hard to identify in data with varying structures, shapes, dimensions, and sources. The dimensions of data quality, as elaborated by [126], are completeness (whether the relevant information is present), correctness (are the data correct), concordance (are they relatable to other data sources), plausibility (is any element in the EHRs making sense in the presence of other evidence), and currency (meaning how old are the data). These solutions will help to identify data quality issues, log them, encode them in metadata for datasets, help develop exclusion criteria of data based on its quality, and record the number of such problems. Ref. [127] is one such work that creates a framework to carry out all the tasks and uses probabilistic models to detect temporal stability and plausibility in biomedical data. It employs probabilistic change detection using Jensen–Shannon distance principles of statistical control of posterior beta distribution. Ref. [128] uses probability distribution distance to the same end. Ref. [129] is a measure of completeness by flagging incomplete data sources using the Delphi method. It also measures the same DQ dimension using patterns in the number of patients and compares them. Ref. [130] considers the data quality of radio frequency identification (RFID) in nine phases within healthcare systems.

2.

Phenotypes: Phenotypes are the combination of an individual’s observable disease traits. The data from the electronic health record are a set of data points related to interventions and the change in the states measured in lab tests. The data help align heterogeneous disease progression into temporal phenotypes. This allows data science techniques to find the relation between disease, symptoms, and interventions. These are also linked to mortality prediction, disease progression, and observation of medically complex phenotypes. Most temporal phenotype identification methods deploy clustering techniques. Phenotypes are also used to identify rare diseases [131,132,133]. These methods are rule-based [133] and graph-theory-based [134].

One of the critical challenges in AI-based phenotype is the representation of data. The data are being presented to domain experts, but developing a metric that identifies the visual tools’ efficacy to represent the temporal phenotypes is worthwhile. For example, in encoding information in the edges and nodes of a graph, silhouette diagrams [135] are very different in richness and application compared to graphs. Graph theory is widely used in these systems as it is very suitable for the relational nature of phenotypes. One key concept is called category theory, which is a directed graph. The data used in temporal phenotypes are time-based (hence, directed), comprising nodes and morphisms. It is different from other graph representations as the morphisms encode the information of the mappings [136]. Very little focused work in this domain comes from EMRs.

3.

Deidentification: De-identification of electronic medical records in an automatic manner is an active area of research where blockchain has recently been widely used [137,138]. Ref. [139] compares deep learning, rule-based systems, and shallow learning for de-identifying EMRs and argues that stacked learning is the most efficient ensemble technique. Ref. [140] deploys self-attention networks and stacked recurrent neural networks to de-identify the medical records. The main de-identification methods are neural networks, blockchain technology, and rule-based systems [140]. Some Internet of Medical Things (IoMT) schemes uses IoT protocols to preserve privacy while ensuring that critical information is relayed to the relevant stakeholder [141].

Challenges in this field remain the interplay of structured, unstructured, and semi-structured data. These data come from various sources and categories and, in the case of categorical features with other features, must be collated before solutions can be designed.

4.

Adherence: Adherence to suggested and prescribed medical regimens is a crucial component of healthcare. Healthcare is an integrated process; hence, adherence is monitored by different sensing and AI techniques to ensure the efficacy of the interventions. The following

Table 6. Recent Works: AI in Adherence.

Name	Summary	Application	References
Conversational Robot	Chatbot used for drug adherence	Drug Adherence	[142,143]
Ethics	Deliberates over the ethical questions arising from the usage of AI in Norm Adherence	Ethics	[144]
Lifestyle Modification	It uses a web app to help monitor adherence, lifestyle modifications, for Example, in the case of cancer.	Drug Adherence	[145]
Medication Adherence	It uses machine learning to perform binary classification of the medication adherence for Parkinson’s disease patients.	Remote Monitoring	[146]
Excercise Adherence	Uses machine learning models to estimate likelihood to adhere to a physical exercise regimen using accelerators and other data sources.	Predictive healthcare	[147]
Medication Adherence	Uses machine learning models to identify the likelihood of non-adherence to medication from electronic health records	Predictive healthcare	[148]
Medication Adherence	Uses data from wearable sensors to measure drug adherence for a specific cause.	Remote Monitoring	[149]
Medication Adherence	Uses cloud-based applications for medication adherence in home hospitalizations	Remote Monitoring	[150]

represents the various AI methods used to this end.

The key challenge in this domain is access to relevant data as the disease progresses. Here, the importance of different features coming from the same sensors and additional sensors can change as the condition changes its phase.

5.

Diagnosis and mortality prediction: Disease prediction can help speed up the process of health care and increase the prediction accuracy, leading to the correct treatment being administered earlier. In the case of critical systems, the idea of mortality prediction and their interplay with demographic information and phenotype can help save lives. It can also help in understanding the progression of the disease and can direct healthcare resources in the right direction.

Ref. [151] contains a process for disease prediction using electronic health records. It uses convolutional neural networks (CNN) to this end. Ref. [152] uses hybrid machine learning techniques to predict cardiovascular diseases. It uses a combination of random forest and linear classification models. Ref. [153] develops a naive Bayes analytic model for disease prediction using electronic health records.

Machine learning has been used to predict mortality for some time [154]. It has significant implications for different phenotypes [155,156]. These algorithms are used widely in brain injuries [157,158].

The critical challenges in disease and mortality prediction are the development of explainable machine learning models. As these models make predictions, they need to be explainable and validated as accurate for each patient prediction. Another crucial issue in this domain is the development of ethical frameworks to enable them to be deployed in the real world. Moral dilemmas such as those explained in [159] for self-driving cars should be identified.

3.6. Explainability

The literature uses many exciting techniques in time-series, EHR, and graph-based data. These techniques include feature significance and their interplay-based methods. Deep learning important features, or DeepLIFT, is widely used to this end [160,161] as it combines the importance of a feature as it passes through the layers of the neural network. local interpretable model agnostic explanation, or LIME, introduced in [162], is also widely used in electronic health records [163,164].

Attention mechanisms find the relevant neurons or the dataset components that are the most pertinent information needed for classification. DeepSOFA, DeepHINT, and Grad-CAM are such systems [165,166,167].

Least absolute shrinkage and selection operator (LASSO) is an explainability technique that uses dimensionality reduction techniques to explain the outcomes of a neural network. They are also used to describe healthcare outcomes [168].

Some explainability techniques draw the rules from the networks, and these systems are also applied in healthcare [169,170].

Deep Taylor decomposition is one such explainability technique used in these systems [171]. Shapley values are also used in such scenarios. The key challenges in developing these systems for graph neural networks are primarily encountered when this method is used for phenotyping.

There is currently a lack of standardized evaluation methods for interpretability techniques, making it difficult to compare and contrast the effectiveness of different approaches. Clinical relevance at present is limited to the identification of which traditional inputs are significant.

3.7. Levels of Automation

Levels of automation for the said topic are discussed as follows:

• Human Only: Here, there is no AI involved, for example, the calculation of muscle atrophy using electromyogram (EMG) signals [172]. This process, however, involves the signal processing techniques for the representation of data.
• Shadow Mode: In shadow mode, the data generated by the interaction of the medical practitioner and other sources are logged, and the data are labeled using the judgment of a qualified physician. These data are used to train a machine learning or an optimization algorithm. One such system developed by the ICL team is a reinforcement learning framework optimizing interventions retrospectively that allows a regulatory compliant pathway to clinical testing. This technique is used for sepsis treatment in the ICU [54].
• AI Assistant: This level of decision making assistance provides the physician with suggestions. Some systems use these to detect cancers; for example, one such system uses biomedical images and structured data to detect hepatocellular carcinoma in the AI assistant model [173].
• Partial Solutions: Based on the data, the AI comes up with a diagnosis independently, but needs a physician’s input.
• Full Automation: All the tasks in healthcare are provided by AI alone.

4. Conclusions

This review presents a paradigm of the application of AI in times-series and graph-based healthcare data that is driven by translational medicine. It looks at the complete pipeline, starting from data collection, harmonization, and quality dimensions. The decision systems are deliberated over, including various kinds of phenotyping, mortality detection, and other methods. We looked at the components related to the data, classifying them into multiple axes. Recent advances and state of the art technology in the various lexemes of the paradigms found were also reviewed.

Data can be classified along multiple axes, including structure, source, and dimension. Most healthcare data are unstructured, which has been used in conjunction with structured data to predict healthcare outcomes. Data preprocessing techniques can help combine different types of data, denoising and harmonizing to increase the reusability.

Another issue that needs to be tackled in data collection and preprocessing is interoperability of various devices and sensors, and this review has elaborated on different interoperability methods.

There are issues where collected data are fed to different decision systems. This part of the pipeline was discussed by this review, especially where the graph-based solutions, such as temporal phenotyping used to help identify risks for various morbidities and help cluster disease presentation into various groups, are concerned. The most recent works and reviewed literature focus more on applying these solutions in the real world. This application becomes easier when the AI process making predictions can be explained, hence, different explainability and interpretability techniques are compared here while highlighting the lack of standard metrics of evaluation for such methods. The validation of accuracy for each individual patient is an open area of research.

Based on the advances mentioned in this review, any future review may include the identification of ethical dilemmas in healthcare interventions and personalized healthcare: continuous healthcare monitoring and better intervention methods. Clinical use of ambulatory data continues to be a challenge for traditional medical practice. The debate between generalizable AI models with the required precision to achieve individual specific health outcomes will likely continue. This work can influence the current healthcare system in a positive manner; however, a way of combining these issues can first develop individual specific models, then explain them using explainability techniques, and then cluster them for general exploratory studies.

There are many challenges associated with healthcare data collection for the so-called disease X [174,175]. The more evolved diseases can be stopped from progressing in real-time using multi-omics-profiling and outlier detection [176]. Another challenge in dealing with data derived from time-based sensor data is the integration of advancements in real-time systems. To this end, translational medicine is already defining some solutions. Another major challenge is generating data for groups for which these data are unavailable using generative AI.