1. Introduction
Motor disorders—epileptic seizures with convulsive patterns, for example—can have a debilitating impact on the lives of the affected patients. In cases where therapy has failed, constant safeguarding may be necessary to prevent major complications, injuries, or even death as a result of sudden attacks or seizures [
1,
2]. This requires around-the-clock monitoring in both dedicated facilities and home situations. Although remote video monitoring is now readily available and incorporated in many care systems, it poses certain challenges when operated by human observers. It is extremely labor-intensive, and operators have to pay uninterrupted attention to multiple screens. In addition, privacy issues may prevent the use of this modality in a variety of cases. Alternative solutions based on attachable devices are also available on the market [
3,
4], but they also require frequent attention and control for charging, proper positioning, etc. Certain patients, due to the stigmatizing effect of wearing them, can also reject the attachable devices.
All of the above arguments motivated us to develop an automated video-input-based alerting system that is capable of detecting a variety of adverse events such as convulsive motor seizures, falls, and nonobstructive apnea, as well as possible warnings for post-seizure cortical electrographic depression. Such an approach combines the advantages of remote nonobstructive monitoring, a holistic representation of the patient state, and automated privacy-respecting alerting.
A central issue for such a development is the wide variety of clinical conditions, camera parameters, and other environmental factors that can challenge the universality of the algorithms used to provide real-time alerting. To address these challenges, we developed an adaptive reinforcement concept that can account for nonstationary applications and optimize the detection process accordingly.
In an earlier work, we addressed the problem of reducing false-positive detections by restricting the detection to only validated true-positive events. This, however, left open the question of missing events, or false negatives. Here, a solution is proposed that can also expand the detection space and gradually reduce the chance of undetected true events.
To verify the effectiveness of our algorithms, synthetic data that can account for a variety of dynamic situations and nonstationary environments are used.
This paper is organized as follows. In the Materials and Methods (
Section 2), the general concept of TOREADA (Topological Reinforcement Adaptive Algorithm) is described. Then, we present its particular application to convulsive seizure alerting. We introduce the concept of dual detection, which allows for the accounting of missed true seizures and expansion of the detection parameters.
A brief introduction to the MC (Monte Carlo) type of simulating input data is included, and its role in the validation of the classifier and adaptive algorithms is explained.
In the Results (
Section 3), an array of trials demonstrating the effectiveness of our approach for cases of different initial detector states and different levels of separation between events and background signal controlled by the signal-generation parameters is presented.
In the Discussion (
Section 4), we state the main motivations for our concept and the assumptions behind it. We also acknowledge certain limitations of the approach and propose some directions for further extensions.
Finally, our major findings are summarized in the Conclusions (
Section 5).
2. Materials and Methods
2.1. General Concept of TOREADA
Our approach is of the type of reinforcement learning that uses constant assessments of the detector performance and optimizes the parameters accordingly. For a general overview and full description of this type of machine learning, we refer to a few of the numerous publications available [
5,
6,
7,
8,
9,
10], and here, we describe our specific methodology. We illustrate the overall process for the proposed adaptive classification of events in
Figure 1.
The generic description of our approach is illustrated in
Figure 1. The green blocks represent the input and the output of an automated system. The blue block “Decision Manifold” represents a direct (without any adaptive reinforcement) reactive process. The reinforcement learning loop consists of an embedding (brown oval containing “Embedding”) step where the decision criteria are relaxed and define the “Observation Manifold” (blue box) in order to account for a larger variety of input events (the training data). An assessment procedure (brown rectangle containing “Assessment”) that can be either supervised or automated, labels the events as true or false (blue box containing “Labelled Data”). The labeled set of events provides an input for constructing an envelope of decision criteria (brown oval containing “Envelope”) that optimizes the detection of the true events and the rejection of the false ones. This decision envelope forms the reinforced, adapted, new decision manifold. The assumption is that our system detects a certain class of events from an input stream of data. Upon a detection, the system will proceed with a predefined action, like raising an alert or simply logging the event. The detection algorithm is encapsulated as
, the “Detection Manifold”. The latter can be explicit or implicit (like in the case of trained classifiers), but in all cases, we assume that the event features fall into a certain area, or manifold, that defines them. The adaptive reinforcement involves on-the-move deforming or adapting
according to the performance of the detector. This process should lead to both decreasing the number of false detections and avoiding undetected events. As our training process includes only events that are detected and not on screening the whole data history, we introduce an embedding procedure:
This step expands the decision manifold into a larger manifold that may include events missed by the initial detection. The extra events will enter the dynamic training set for the adaptive detector but will not affect directly the output (for example, they will be logged separately and no alerts will be generated).
The assessment of the whole set of events is outside the scope of this work. In an earlier work [
11], a clustering-based algorithm for separating the true from the false events was proposed. Alternatively, a trained operator can perform the assessment. Here, we assume that a labeled set of events is available for the next phase of the training process, the construction of the envelope manifold:
Finally, the reinforcement cycle closes by substituting the initial detection manifold with the envelope from (2):
The above procedure repeats with every new event or in blocks of predefined number of events. It is a learning during the operation scheme that does not require precollected datasets.
Equations (1)–(3) are very abstract and do not reveal any specific algorithm of performing the process. In
Section 2.2 and
Section 2.3, we provide the specific application to the task of alerting for convulsive epileptic seizures.
2.2. Detection of Convulsive Seizures
For a complete description of seizure detection algorithms, we refer to previous work [
11] and the references therein. Here, we note that an optical flow technique processes live feed from an optical camera and, subsequently, a set of Gabor filters defines a single time series signal that represents the “epileptic content” in the video sequence. Our focus here is only on the last stage of the detection techniques, the decision of whether in a given time window a convulsive seizure has taken place. We illustrate the decision algorithm in
Figure 2.
The following condition defines a time interval preceding any time point
as containing a seizure event:
Here, is the epileptic marker as an input signal, is the selected threshold, is the retrospective time interval (seconds or sample points), and is the selected minimal amount of time (or number of samples) when the epileptic marker is above the threshold. Note that in (4) we allow for the marker to move up and down across the threshold during the observation period as long as the total time above is longer than .
According to (4), the three parameters define a seizure event. It is clear that the higher the threshold, the more conservative will be the detection. Also, the longer the interval for a given , the more “liberal” is the detection; more events will fulfil the condition (4). A larger for a fixed defines a stronger condition for an event to be detected as a seizure. Finally, a “black-out” period is introduced in order to avoid multiple detections of the same event. After a detection, the algorithm skips the next samples. In all our detections, we selected a black-out period of 90 samples.
Presented in
Section 2.3 are the embedding and the envelope implementations for this case.
2.3. Dual Detection Approach
The enlargement or embedding of the decision space formed by the parameters
in inequality (4) is postulated by the following substitution:
In the transition (5), the union means that if either of the triple parameters fulfils the condition (4), the event is considered as a training event. The parameter is the relative threshold decrease, and a reasonable choice is This choice is a compromise between taking too many fault positive events into the training set ( is very small) that will make the validation difficult, or taking it to close to one; this will make the adaptation too slow. The embedding (5) is by no means unique but it provides a minimal extension of the decision manifold that is close to the detection one. Larger embedding may speed up the inclusion process but may also introduce too many training events which will make the validation phase challenging.
To complete the enforcement procedure, the envelope-construction algorithm is also needed. In earlier work [
11], this stage is introduced in detail. Here, we recall that after the assessment of the training events, we introduce the cost function:
In the second equation, the value of the parameter
, the filling number, is derived for each pair of
values as the minimal number of threshold-exceeding epochs, n, over all detected and validated seizure events. This way, we can guarantee that, for any choice of the pair
, all the previously detected true seizure events will be preserved. Therefore, we can postulate the optimized choice of parameters as follows:
2.4. Generating Synthetic Data
Commonly referred to as Monte Carlo simulation, the synthetic data-generating algorithm consists of a random process driven by a known probability distribution over the states of the system [
12,
13].
Figure 3 introduces our implementation that generates a synthetic epileptic marker according to probabilistic rules with variable predefined parameters.
The stochastic model introduces the following probabilities for the epimarker during seizures and outside seizure periods:
In Equation (8),
are the thresholds for, correspondingly, the ictal and normal periods, and
are the associated probabilities. We also postulate random ictal onset times. The seizure duration is a random number in the interval
, where
is a chosen maximal duration. The generated events are used as a ground truth in order to assess the performance of the detection and reinforcement processes.
Figure 4 illustrates a trace of the epimarker from a real seizure as observed by our seizure-detection algorithm (top frame) and from a generated event (bottom frame). The parameters for the latter are
,
,
.
In the results below, a variety of scenarios are considered with both stationary and nonstationary seizure-generation parameters , , , L}.
To quantify the separation between normal and seizure states, one can use the “confusion” metric between two distributions
defined over the states of any system:
Assuming in our case that
, we obtain from Equation (9)
2.5. Simulation Protocol and Performance Quantification
Here, the different seizure-generating scenarios and initial detection parameters are introduced.
Unless otherwise stated, the typical stimulation session includes a number of epochs, each of a length of 3600 simulation steps. In each epoch, we randomly generate five seizure events according to the stochastic rules explained in the previous subsection. After each epoch, we detect the seizures with the current detector parameters according to
Section 2.2. Based on the data-generation log, only the true positive events are selected to enter the parameter optimization algorithm. Next, a minimum number of 20 retrospectively detected true events is required in order to start performing the dual detection parameter optimization procedure explained in
Section 2.3. If the number of events is lower than 20, the current parameters are kept. The update is performed after each new simulation epoch; no more than 60 previously detected true events are taken, in order to account for possible nonstationary conditions.
After each epoch, the detector performance is evaluated by collecting the statistics from the last 8 epochs (40 generated true events) according to the following standard definitions for the sensitivity and specificity:
In Equation (11), are the true positive, false positive, and false negative (undetected events) detections, correspondingly. The ground truth for the events is in accordance with the simulation log.
All the calculus is performed in Matlab®, version 2024a. The statistical distribution plots are produced by the standard “boxplot” function from the “Statistics and Machine Learning Toolbox” version 24.1.
4. Discussion
In this work, a general method for adaptive detection and reinforcement is introduced. We call it TOREADA (from TOpological REinforcement Detection Algorithm) and we explain here, in short, the role of the term “topology”. In addition to the formal description in
Section 2, a more intuitive insight might be helpful. Our major assumption, or axiom, is that the events that are to be detected form upon appropriate parameterization of a connected continuous manifold. An ad hoc detector setting, or the detection manifold, may partially cover the “true” event manifold, therefore resulting in false positive (FP) detections and false negative (FN) detections in the detected nonevents and the undetected true events, correspondingly. Therefore, the reinforcement process, or learning paradigm, is associated with deforming the detection manifold so that it can possibly cover as precisely as possible the true event manifold. To do so, two complimentary transformations are introduced: embedding and envelope. The embedding step expands the detection manifold to an “embedding” one that contains the former. All events that fall into the larger, embedding manifold form the training set of events. In this way, if the detector is not initially detecting certain true events, they may still fall into the training set. The second stage constructs an envelope around all classified true positive events. This step will form a “tighter” manifold within the embedding one that will possibly help to reject false positive detections. Finally, the envelope manifold is taken as the new, updated detection manifold. Subsequently, the process is iteratively repeated according to the appropriate schedule.
The two steps of embedding and envelope forming are by far not unique. Taking a too-wide embedding manifold around the current detection one, for example, may speed up the process of reducing the FN events, but at the same time it may introduce too many FP events, which would make the assessment of the true events quite complicated. A very tight envelope may reduce the FP detections but may also counteract the embedding process and, consequently, slow down the reduction in undetected true events.
We also left outside the scope of this work the procedure of validating the detected events. Here, both supervised and automated labeling are possible, using for the latter the clustering algorithm, as proposed, for example, in our previous work [
11]. Including this form of automated validation into the simulation tests will be a subject of our future research. Another extension of the current work may involve simulation of the optical flow output signals and not just the resulting epifactor. This provides the possibility of testing our real-world adaptive algorithm, also presented in [
11], that also involves the adaptive reinforcement of the frequency interval relevant for the particular seizure-generating system. We remind the reader that our objective here was to concentrate on the steps described by Equations (5) and (6).
Another hypothesis used here is that seizure events are characterized by an epimarker that, during a seizure, takes values from a different distribution than during “normal” periods. More specifically, we assumed that the ictal distribution favors higher values. As this is a highly nonparametric assumption, we implemented the simulation model with a couple of thresholds and couple of probabilities in order to explore various levels of separation between the states. In a more general scenario, the epimarker may define a seizure event not by its value but by some features of its distribution statistics. It can be, for example, the fluctuation rate or, more generally, certain momenta of the distribution. We believe that the topological reinforcement paradigm proposed here can be applicable to those cases by properly parameterizing the embedding and envelope manifolds and algorithms. As for the simulation of data, one can apply a similar technique to the one used here to generate synthetic signals, by selecting two model distributions representing seizures and normal states. This and possibly other generalized implementations of TOREADA will be considered in our future research.
The classic approaches to machine learning, including applications targeting detection of convulsive epileptic states, rely on available big collections of labeled data [
14]. The founding belief is that these collections contain the necessary information and that they can account for all the variability of the real-world scenarios. Our concept does not require, although it may use, a prerecorded labeled sets of events. It acquires its training sets along with the operation of the detector and, therefore, is a paradigm of continuous learning from experience. This way, the method is applicable to nonstationary situations and, more generally, can cope with the real-world variability of conditions. Moreover, in contrast to the static data collection paradigm, the TOREADA concept can potentially apply adaptive reinforcement to the data acquisition protocols. It can, for example, adjust spectral ranges, apertures, and other sensor properties in order to achieve optimal performance.
The above topological paradigm also presents certain restrictions. If the events that we are attempting to detect are of several different classes, corresponding to disconnected manifolds, our approach is unlikely to work. A possible solution would be to introduce multiple initial detection manifolds and apply the TOREADA algorithm to each one separately.
As a particular real-world application, we presented here our development of systems for convulsive seizures detection. This explicit example of TOREADA is an operational implementation undergoing field trials in clinical and patient care settings. We use this specific realization, called also the dual detection algorithm, to provide numerical validation and performance assessment of the general concept. In this particular application, the embedding stage consists of decreasing the detection threshold and simultaneously relaxing the duration criteria. There are no absolute rules to determine the boundaries of the embedding in general. The optimal choice may depend on the data variability. It might be interesting to explore a second level of reinforcement and to optimize the embedding parameters according to the rates of false positive versus negative training detections. This possibility may also be addressed in our future investigations.
In this work, instead of collected clinical data, data generated by the Monte Carlo (MC) type of numerical simulations were used. There are several reasons for this. First, validating and exploring the limits of adaptive and machine learning algorithms in clinical conditions may present certain ethical issues. If the alerting system departs from its optimal setting and fails to alert for convulsive events, this may place the patient at risk. Conversely, if an unsuccessful reinforcement trial leads to many false alerts, the caregiving personnel may switch the system off. Second, no amount of real-life examples can provide enough data for all possible scenarios and exceptions that the system may encounter in general. Finally, the use of simulated data avoids the issues of privacy, especially present in the cases of supervised video data validation.
Concerning our results from the trials described in
Section 3, we again point out the “asymmetry” between the performance boundaries of our approach with respect to the sensitivity and the specificity. The detector loses specificity earlier than it loses sensitivity from the decrease in seizure/normal separation. The latter can be either due to confusion increase or to seizure length decrease, as seen from
Figure 8,
Figure 9,
Figure 10 and
Figure 11. As mentioned earlier, it is a result of the “competition” between the embedding process described by Equation (5) and the envelope described by Equations (6) and (7) in this particular implementation. It is clear that if only the embedding is active, the system can only enlarge the detection manifold, for example, according to Equation (5) it can only decrease the threshold. The opposite process, the envelope making, can counter this and increase the threshold, as indeed seen from the results in
Figure 7. Other choices, for example, a more conservative embedding and tighter envelope construction, may reverse the balance, and the system will lose sensitivity while the reinforcement algorithm still achieves high specificity. Adding to this point, the current results are favorable in the view of clinical and care practice. The harm of undetected seizure events may be potentially higher than that of false alarming, without diminishing, of course, the need to reduce the latter as much as possible.
5. Conclusions
As demonstrated in
Section 3.1 and visually presented in
Figure 5, the proposed reinforcement parameter adaptation successfully converges to a detector with both high sensitivity and specificity performance. This is achieved from a variety of initial settings, some very “liberal”, allowing for many false positive detections, and others very “conservative”, skipping some real events. In all cases, here, we use the same event-generation parameters. The later correspond to a relatively good separation between normal and seizure epochs with a confusion factor of 0.02. From
Figure 7, it is clear that for all initial values, the detector parameters reach approximately the same optimal levels due to the reinforcement algorithm. As the process is continuously active, certain fluctuations are present, but the system recovers its target parameters and shows asymptotic stability.
To answer the question of how far one can trust the method with respect to the separation between seizures and normal periods, the experiments reported in
Section 3.2 are performed. From the results of the first test, shown in
Figure 8 and
Figure 9, it is clear that, independently of the initial detection parameters, the reinforcement adaptation provides stable sensitivity for generation parameters corresponding to confusion factors of less than 0.2. For the specificity, the adaptive process fails already for confusion factors greater than 0.1. The conclusion is that for systems with marginal or poor distinction between normal and seizure states, the detector can still adapt for proper detection of the events but will also produce false positive detections.
The final test explores the limits of the seizure event durations so that the approach will still be useful. The results shown in
Figure 10 and
Figure 11 show that, again, the sensitivity reaches high values independently of the initial detector settings for all maximal event durations ranging from 10 up to 60 simulation steps. The specificity trace, however, is not stable for short seizure events. For short seizures the algorithm will manage to restore an optimal detection sensitivity but at the expense of letting through false positive detections.