1. Introduction
In recent years, cognitive impairment and related diseases have increasingly engaged the attention of researchers, neurologists, and neuropsychologists [
1,
2]. Cognitive impairment might be neglected, especially by young people, but it has a significant social impact since it affects employment, daily activities, and living [
3,
4]. Therefore, its early diagnosis is an important factor for the proper prevention and treatment of cognitive diseases, which requires targeted neuropsychological testing. Neuropsychological assessment is still a time-consuming task based on paper tests that requires the involvement of highly specialized clinicians. The processing and analysis of results from such paper tests is difficult and requires time and considerable effort. In practice, clinicians work in isolation with separate groups of patients and limited healthy controls for comparison. Since the data are collected in a variety of formats and are structured in different ways, additional efforts for aggregation and digitalization are needed. This in turn hinders searching for certain patterns and dependencies in cognitive impairments. Although automated versions of the paper tests have already been developed, they primarily serve as result repositories, and the further analysis of collected data is not supported [
5]. Some of them are limited to the statistical processing of the results, and others apply machine learning algorithms to predict probable outcomes only for individual patients.
At the same time, the rapidly increasing volume and complexity of data foster the growth of big data technologies. All domains of everyday life continuously produce data, ranging from user-generated content of around 2.5 quintillion bytes every day [
6] to applications in healthcare [
7], education [
8], knowledge sharing, and others. The data and knowledge extracted from it become a key factor in the development of advanced solutions that support decision-making. Given the data value for applications in different domains, there is a need for data-driven holistic platforms with functionality for runtime adaptations across the whole data lifecycle.
This paper presents a platform for the evaluation of cognitive dysfunction in patients with Multiple Sclerosis (MS) called CogniSoft. The platform allows the assessment of cognitive status in a timely manner and enables clinicians to prescribe appropriate rehabilitation actions. It is registered as medical software by the Bulgarian Drug Agency (no. BG/CA01/MD-0037/28.09.2020). The main contributions of CogniSoft are as follows:
The improved collection, enrichment, and use of health data related to cognitive diseases by the automation of a set of diagnostic tests for the evaluation of memory and executive functions based on the nature of Brief International Cognitive Assessment for MS (BICAMS), as well as a set of games for cognitive rehabilitation based on BICAMS;
More precise diagnosis through the automation of a questionnaire for depression (Beck Depression Inventory—BDI-II) that can be filled before neuropsychological tests are conducted for the differentiation of possible depression, which could interfere with the results;
The implementation of two approaches for the evaluation of neuropsychological results, which allow the early detection of cognitive impairments in these patients and the initiation of timely cognitive rehabilitation.
The rest of the paper is organized as follows.
Section 2 briefly describes the related work in the problem area.
Section 3 describes the nature of diagnosis and rehabilitation tests for MS.
Section 4 is devoted to the CogniSoft platform’s architecture, while
Section 5 describes its implementation.
Section 6 provides experimental results. Finally,
Section 7 concludes the paper and gives directions for future work.
2. Related Work
In our previous work [
9], a review of the current state of the research on the problem area was performed, covering the automated tools and platforms for the assessment of patients with cognitive disorders such as MS, Alzheimer’s disease, early dementia, mild cognitive impairment, etc. It includes a summary of four literature reviews, as well as a comparison analysis of eight software solutions for the evaluation of cognitive status. The software solutions are compared according to the following characteristics: application area, type of device, means of interaction with the device, patient’s performance mode, number of tests and batteries, additional factors, output data, maintained cognitive domains, and diseases.
The systematic review of the diagnostic accuracy of automated tests for cognitive impairment performed by Aslam et al. includes studies from January 2005 to August 2015 [
10]. It evaluates the diagnostic precision of automated tests for the assessment of mild cognitive impairment or early dementia. Eleven tools for the automated testing of cognitive disorders are explored. The systematic review by Wild et al. covers studies published by 2007 that present 11 computer-based tests for the assessment or detection of changes in age-related cognitive status, early dementia, and mild cognitive impairment [
11]. The tests are evaluated against several characteristics, such as the availability, validity, and reliability of tests; range; and use. The review of computerized cognitive testing for older adults by Zygouris and Tsolaki explores 17 tests for the assessment and screening of cognitive status, published by 2012 [
12]. It presents a comparative analysis of tests according to the hardware, mode of execution and administration, runtime, and supported cognitive domains. The psychometric properties of the tests are also examined and conclusions are made regarding the tests’ validity and reliability. Another review on computerized cognitive testing for patients with MS was published by Lapshin et al. [
13]. It includes the following tests: Automated Neuropsychology Assessment Matrix (ANAM), the Mind streams Computerized Cognitive Battery (MCCB), the Amsterdam Neuropsychological Tasks (ANT), the Cognitive Stability Index (CSI), and the Cognitive Drug Research (CDR) battery.
To the best of our knowledge, there is no a standard for the cognitive rehabilitation of patients with MS, which is expected due to the individual progression of the disease in both physical and cognitive aspects. Currently, several programs are being developed with different cognitive focuses:
Six-week cognitive training program, which includes a set of cognitive training forms focused on memory functions, attention, and the planning of activities during oral training [
14].
A neuropsychological group training for the improvement of cognitive performance [
15].
BrainStim, which is a computerized training tool to improve specifically the key features of working memory [
16]. It consists of three modules: City Map, which trains visuospatial and verbal working memory; Find Pairs, for the training of visual short-term memory, working memory, and updating function of the central executive; and Memorize Numbers, which requires the patients to encode numbers and recall them from memory after an arithmetic distraction task.
CogniFit, which is a personalized, computer-based cognitive training program available online [
17]. It provides a battery of tasks for the cognitive assessment of divided attention, auditory short-term and visual short-term memory, working memory, naming, hand-eye coordination, processing speed, and shifting.
ProCof-SEP program, including 12 sessions with different language exercises for the training of verbal memory, visuospatial skills, verbal fluency, etc. [
18].
MAPSS-MS intervention program, which focuses on cognitive performance, memory strategy use, self-efficacy for the control of symptoms, and neuropsychological competence in the daily living activities of patients with MS [
19]. It consists of two components—namely, eight weekly group sessions, which aim to build efficacy for the use of cognitive compensatory strategies, and a computer-assisted rehabilitation program based on home training.
RehaCom, a modular, interactive program for training cognitive abilities. It is based on compensatory strategies, controlled stimuli, and immediate feedback. The program provides training procedures related to attention, memory, visuospatial processing, and executive functions [
20].
Attention Processing Training (ATP) program for cognitive rehabilitation intervention, which is related to focused, sustained, selective, alternating, and divided attention [
21].
3. Diagnostics and Rehabilitation Tests
In order to obtain domain knowledge and to collect the initial set of software requirements for the automation of diagnostic and rehabilitation tests, several interviews with two clinicians with expertise in MS are performed. As a result, a specification of test assessment processes is created, which describes how the platform will be used by both clinicians and patients. The first version of the platform implements an initial set of requirements by automating the following tests:
Beck Depression Inventory (BDI-II) test;
Diagnostic tests for the evaluation of memory and executive functions based on the nature of BICAMS—Symbol Digit Modalities Test (SDMT), Brief Visuospatial Memory Test (MVMT), and California Verbal Learning Test (CVLT);
Rehabilitation tests, based on BICAMS battery.
The SDMT is a well-known test that emerged in the world practice, since it provides a high receptiveness to cognitive impairment in MS. Its convenience of application is proven—the test takes only about 5 min for conduction and assessment. The attention and speed of information processing could be measured using the SDMT [
22]. The test results correlate with the findings of magnetic resonance imaging (MRI) [
23]. The lower scores correspond to the severity of diffuse abnormalities of the supratentorial pathways composed of white marrow and atrophy of the right anterior cingulate gyrus, right temporal medial gyrus, and left postcentral gyrus [
24]. Since the test is based only on geometric figures and numbers, it is considered as independent of cultural differences and can be applied in any language.
The episodic memory can be assessed using BVMT, which is a short test for the assessment of visual-spatial orientation and constructive praxis. Since its first edition, the test has been revised several times, with each new version showing increasing reliability and validity. The revised version of BVMT (BVMT-R) can be used as part of a large neuropsychological battery as well as a screening tool for baseline cognitive level and for monitoring the changes in cognitive status over time [
25]. The test is applicable in a wide age range, from 18 to 79 years. Since the patient’s age has a significant impact on the test results, norms in narrow age categories should be defined [
26].
The CVLT evaluates episodic verbal learning and is a widely used tool in worldwide neuropsychological practice. It is based on auditory perceived stimuli to predict employment in the long term. The test is effective in monitoring the advantages of treatment in MS and strongly correlates with MRI findings in regard to the lesion load and loss of volume of the gray matter [
27]. CVLT allows the further assessment of the individual strategies for learning as semantic, serial, and subjective clustering in a requirement for learning [
28]. It is also designed for a wide age range.
The above three tests are chosen for automation in the CogniSoft platform due to the following reasons:
High diagnostic value [
29,
30];
Possibility for international use due to the minimization of cultural differences and cross-country comparability [
29,
30,
31];
Objective monitoring of cognition and evaluation of the effects of immuno-modulating drugs and rehabilitation therapy;
Applicability for the assessment of cognition in healthy subjects;
Ease of application for both physicians and patients due to the relatively short time span for administration and avoiding cognitive fatigue that could affect the evaluation of the results;
Opportunity to conduct the tests at a convenient time for the patient;
Lack of need for assistance in most cases.
4. Software Architecture, Modules and Technologies
The architecture of CogniSoft is based on the Service-Oriented Architecture (SOA) paradigm. Its implementation consists of several software modules, which are described in this section.
4.1. Architecture of CogniSoft
The architecture of CogniSoft is shown in
Figure 1. The client layer provides a web-based user interface (UI) for both patients and clinicians. An angular web application framework is used for its implementation in order to provide a responsive user experience. Since AngularJS supports the Model-View-Controller (MVC) architecture, it is smoothly integrated with the server layer. The model and the view are automatically synchronized through two-way data binding. Thus, the user is always provided with an up-to-date view of the model.
The scope objects of AngularJS are related to the application model. In order to mimic the Document Object Model (DOM) structure of the application, they are arranged in a hierarchical structure. The AngularJS controllers set up the initial state and add behavior to the scope objects. The AngularJS directives attach a specific behavior to the DOM elements or transform them and their children.
The server layer, which is implemented using the Spring framework, provides a set of Application Programming Interfaces (APIs). The Spring web application framework fully supports the Representational State Transfer (REST) architecture. Thus, the functionality of the platform is composed by a set of REST services. The modularity of the Spring provides an opportunity for developers to pick the modules that are relevant to their server development.
The data layer provides a repository for the storage of patients’ data and the results from cognitive assessment tests, both diagnostic and rehabilitation. PostgreSQL Relational Database Management System (RDBMS) is used for the implementation of the data repository. Since the platform is designed to work in two modes—transactional and analytical—different instances are created depending on the working mode. Multiple transactional instances of the database are created for each medical center or hospital using the platform, while the analytical instance always remains single. The database’s instances use a unidirectional master-slave replication.
4.2. Modules of CogniSoft
CogniSoft consists of seven main modules, which are described in this section.
The security and privacy issues are handled by the security module. Since the patients’ data are sensitive information, anonymization as well as data separation techniques are applied to ensure data privacy and confidentiality. Thus, the user profiles are separated by the corresponding personal records. The server layer and client layer rely on the Spring and Angular Security libraries, respectively. Angular handles the most popular web application attacks by implementing best practices and built-in protections. Data access is controlled on both application and network levels. The data repository is on servers within an internal network. External users can access the platform by sending requests to a web server behind a proxy firewall using the SSL channel. The architecture of the application is stateless, meaning that the authorization is passed through a standard JSON Web Token (JWT) in the HTTP header. The header is issued during authentication and contains assertions signed by the server.
Users’ roles and profiles are implemented by the user module. The users’ roles are the same for the client and server layers. All system elements, including the audit module, are accessible for the administrators. They are allowed to create nomenclatures and to perform Create, Read, Update and Delete (CRUD) operations on every type of objects. The clinicians administrate the patients’ records and are allowed to create groups of patients, versions of the tests and group of tests in batteries. They assign and monitor the execution of the tests. The patients execute tests, which are assigned to them by the clinicians, and eventually to access the results from the test execution. The controls execute tests, which are assigned to them by the clinicians.
The nomenclature module is responsible for the definition of platform’s nomenclatures in a multilingual mode. The nomenclatures are grouped based on the type and the language.
The administration of the personal records of users’ roles clinicians, patients and controls is implemented by the personal records module. The platform stores different information for different roles. For example, the patient’s personal record describes the disease, nationality, education, affiliation, and other classification information about the patient. The clinician’s personal record contains information about the specialty, medical center or hospital, and participation in public healthcare projects.
The disease module keeps track of the patient’s disease progression. A dedicated JSON file is associated with the patient’s personal record to store the corresponding data. The test module implements the diagnostics and rehabilitation tests. The tests can be changed by the clinicians at both the diagnosis and rehabilitation stages. The test assignments are stored in the table “Assignment” of the database, while the test executions are recorded in the table “Execution”. Each executable test consists of a static part, which is common for all tests of the same type, and a dynamic part that defines how the test will be performed.
The audit of the platform’s actions, such as login and changes in the tracking of valuable objects, is implemented by the audit module. The audit information is related to the users who perform CRUD operations over the database.
5. Automation of Cognitive Assessment Tests
The cognitive assessment of MS using BICAMS covers the most vulnerable neuro-psychological domains of mental function, such as the speed of information processing, episodic memory, visual-perceptual functioning, and attention. CogniSoft automates BICAMS for both diagnosis and rehabilitation.
The rehabilitation’s versions of BICAMS tests are called games. Each game follows the nature of the specific BICAMS test, but its rules depend on a variety of parameters. The availability of different configurations enables numerous levels with various difficulties, numbers of attempts, or durations to be defined. The levels for each game are essential when a diverse rehabilitation program has to be prescribed. Such levels provide an opportunity for clinicians to create individual rehabilitation programs, which are specifically tailored to the current needs of their patients.
5.1. Symbol Digit Modalities Test
The patient is required to perform a symbolic substitution with a corresponding digit of a given nine-digit code (key) for a period of 90 s (see
Figure 2). The test is based on the paper version using:
Key field—it consists of nine symbol-digit pairs that illustrate the correspondence between a symbol and a digit (1 to 9). A standardized predefined characters pool is used.
Work field—contains six couples of rows, which are arranged in tables with 15 columns and 2 rows each. The top rows contain characters from the represented key in a random order. The bottom rows are blank and must be filled in by the patient in consecutive order with the corresponding (key) digits. The patient should complete the test as quickly as possible.
The computer version of the test consists of the following steps: (1) instruction, including text or audio directions for test execution; (2) demonstration, which provides a short video with a sample test execution; (3) training, which allows a one-time execution for gaining user experience; (4) execution, which is the actual test execution; and (5) end, which finalizes the test, showing a message for successful completion.
The test execution requires the consequent filling of numbers in the displayed screens (standard six screens). The final screen ends with a field where the patient can share additional information about any circumstances related to the test execution and his/her performance. The information related to performance outcomes, such as execution time, the number of correct answers, problematic key symbols (recurring errors), etc., can be seen only by the clinician.
Turning SDMT into a game with different levels of difficulty means that it provides variations related to key length, the number of lines, or the number of symbols per line. For example, by setting up a key of only four symbols, the game would be easier, less stressful, and thus more acceptable for patients in the more advanced stages of the disease. The difficulty of a game with a four-symbol key can be raised if the patient has to complete not three but 10 or 15 lines. Another advantage of the game configuration is the option for missing a time limit to achieve the goal, which lowers the pressure on the patient and gives him or her enough time to enjoy the rehabilitation.
Figure 3 shows the different levels of the SDMT rehabilitation game, where a more colorful set of symbols is used in comparison to the standard set. A random selection of symbols from a large pool of pictures is performed for every execution. This guarantees that two keys will never be the same, leading to uncertainty during the execution of the rehabilitation game by the patient.
5.2. Brief Visuospatial Memory Test
The computer version of BVMT provides specially designed forms containing six predefined geometric shapes, shown in a table with two rows and three columns (see
Figure 4).
The BVMT includes two main steps as follows:
Memorizing: For 10 s, a table 2 × 3 is shown with validated figures from a predefined pool.
Reproducing: Two tables are shown—one empty 2 × 3 table and second 3 × 4 table, which is filled with 12 figures. Six figures are taken from the previous screen (validated) and six figures are similar to those previously presented (not validated). The patient should drag and drop the figures to fill the empty table with the correct figures based on what is memorized from the screen.
This process is repeated three times and the best performance is taken into consideration. The computer version of the test consists of the following steps: (1) instruction, including text or audio directions for test execution; (2) execution of the test, consisting of two stages—memorizing and reproducing; and (3) end, which finalizes the test, showing a message for successful completion. The results are based on the number of correct figures and their correct placement in the table.
The BVMT rehabilitation game provides an option to set up different sizes for the key matrix and test matrix. Attaching a large pool of symbols to the configuration also makes sure that the key matrix will always be versatile and the patient will never use the same symbols.
Figure 5 shows BVMT rehabilitation games with different key matrixes and test matrixes.
5.3. California Verbal Learning Test
The CVLT is implemented based on three tables of words. The “validation”table includes four categories of words, such as sport, paper products, geographic objects, and sweets. The second table contains words that are similar to those in the first table and fall in the same categories. The third table contains words that fall in categories that are different from ones in the first table. During test execution, the words are randomly selected (eight words from the first table, six words from the second table, and two words form the third table).
The test allows five consecutive attempts, during which the same validated 16 words belonging to the selected categories are read. The patient should try to memorize and identify them. Statistics are collected about the number of memorized words and the categories to which they refer at each attempt. The number of correct answers is expected to increase after every attempt.
The computer version of the CVLT, shown in
Figure 6, adapts the paper version and consists of the following steps: (1) instruction, including text or audio directions for test execution; (2) demonstration, which provides a short video with a sample test execution; (3) execution of the test, consisting of five attempts; and (4) end, which finalizes the test, showing a message for successful completion.
The patient guesses words by pressing buttons labeled with the words heard. The test is performed in, two stages corresponding to two different screens. The final screen allows the patient to fill in additional information about the circumstances related to the tests execution and his/her performance. On the clinicians’ side, information related to the results of the performance could be displayed or not, including time of implementation, number of correct answers, problem categories.
The CVLT rehabilitation game provides an option to add pictures to the buttons, from which the patients choose the words to answer. This option turns the CVLT into an enjoyable game and makes rehabilitation more fun. In addition, the number of words per screen could be changed to create different levels of difficulty. The pools of words used in the game also vary. A larger pool of words reduces the chance for the patient to hear the same words while repeating the same level of the CVLT game.
5.4. Rehabilitation Program Based on Authored Games
CogniSoft provides an additional possibility for clinicians to define personalized rehabilitation programs by creating their own batteries with authored cognitive assessment tests or rehabilitation games related to a particular cognitive domain. The following rehabilitation program is an example of the assessment of emotional recognition skills. The rehabilitation program consists of the three rehabilitation games, as follows: visual empathy, face expressions, and emotion state.
During the visual empathy rehabilitation game, the patient sees on the screen pictures with eye expressions of men and women. For each picture, he or she has to select (using mouse or touch screen) a word that best describes what the person thinks or feels. Although more than one word may be appropriate, the patient should choose one as quickly as possible. Each correct answer brings one point. The total score, which is a sum of all received points, is evaluated according to a range with predefined lower and upper average limits, corresponding to the social profile of the patient. If the score is below the lower limit, the patient is recommended to proceed with the rehabilitation.
The face expressions rehabilitation game is similar to the previous one. The difference comes from the fact that, instead of eye expressions, the patients should recognize and evaluate the expressions of human faces. In fact, the whole face provides more information and creates a better chance of recognizing a particular emotion. The total score is based on the correctly given answers. It is compared with the average value of healthy controls with relevant social profiles.
The Emotion State Test is implemented based on the video materials provided by the Karolinska Institute [
32] and was reviewed and approved by neuropsychologists from the Clinic of neurology of MHAT “National Cardiology Hospital” in Sofia, Bulgaria. Based on specific storyline presented as a video, the patient must determine the leading emotion of the main actor by choosing from a set of predefined options. The format of this rehabilitation game is a novelty, since the static representation of emotions has been replaced by dynamic content. The level of difficulty is higher due to the need for complex emotional perception and information processing, as well as the closeness of the task to be solved to everyday life.
6. Experimental Results
CogniSoft is deployed as a software for the diagnosis and regabilitation of cognitive disorders in patients with MS in the Neurological Clinic of the National Hospital of Cardiology in Sofia, Bulgaria, according to a decision of the commission of medical and academic ethics, protocol number 108/20.03.2020.
CogniSoft implements two approaches for the evaluation of the test results, based on a norm built by control subjects [
33]. The first approach takes into account only the left boundary (minimum) of the 95% confidence interval (regardless of the type of distribution) of the results obtained from healthy controls when performing a neuro-psychological test (or the total score from a non-psychologic battery test), denoted as
BL:
where
is the mean value,
is the standard deviation, and
is the sample size of the cohort (number of the observed cases).
In addition to B
L, the second approach considers four additional parameters—namely, the percentage of MS patients who have mild cognitive impairment (L), the percentage of patients with MS who report moderate cognitive impairment (M), the percentage of patients with MS who report severe cognitive impairment (H), and the percentage of MS patients who report very severe cognitive impairment (V). The interpretation of the results is shown in
Table 1.
Seventy controls and 36 patients with MS were involved in the study [
34]. For each participant, an electronic health record was created. The following tests were performed by the participants: (1) test with digits for speed of information processing, (2) test for visual-spatial memory and orientation, and (3) test for verbal learning and delayed memory. The results were processed statistically using descriptive analysis, T-test, and correlation analysis.
Regarding the level of depression, no statistically significant difference was found between the controls and patients. Persons with MS experienced the most obvious difficulties in performing the test with digits for the speed of information processing (p < 0.05). Employment status correlated significantly with better results in subjects with MS. A proportional relationship between the disease duration and the degree of cognitive impairment was observed. The relationship between the severity of physical disability and the degree of neuropsychological decline was also proportional. As a result of the rehabilitation games, an improvement in the score was registered in each of the tests.
The results of the study expand the knowledge about cognition in patients with MS in the Bulgarian population, as well as providing an opportunity to improve the strategy for the rehabilitation of cognitive impairments in this contingent.
7. Conclusions
The preventive and therapeutic approaches, which are tailored to patient requirements, need a personalized medicine for early disease detection. It is a societal challenge to adjust to the further demands on health sectors due to the ageing of the population. Effective healthcare requires improved decision-making in prevention and treatment provision as well as in the identification and support of best practices.
The CogniSoft platform described in this paper contributes to precision medicine for MS. It reports not only the presence or absence of cognitive abnormality, but assesses also the degree of cognitive impairment. The test results support clinicians to make specific decisions about what rehabilitation program to prescribe using the rehabilitation module of CogniSoft. In addition, the computer-based form of the tests eliminates subjectivity in the administration and interpretation of results by different physicians.
Being web-based, CogniSoft guarantees maximal access to its functionality in different neurological departments, which saves time and financial resources as well as facilitating the collection and storing of large data volumes and their subsequent analysis. The bilingual implementation of the platform allows it to be used in English-speaking countries, which enables the comparison of the results according to national peculiarities and generally the enrichment of the knowledge of MS. Once integrated through a common platform, clinicians are provided with an opportunity to provide efficient diagnosis, prognostication, and timely decisions for patients. The presented implementation covers clinical questions that deal with the short-term management of patients. The diagnosis and rehabilitation tests covered by CogniSoft support intervention planning in hospital setting. The prognosis of the disease progression requires a longitudinal setup that is the subject of ongoing research and the focus of the analytical component of the platform. The potential of the platform is wider and could be expanded to provide long-term decision support and evidence-based outcome predictions in MS. CogniSoft is designed to provide a quick interpretation of patients’ data and a view of the probable outcomes of rehabilitation based on the past records of neurological departments, Electronic Health Records (EHRs) from other systems, open clinical databases, etc.
Author Contributions
Conceptualization, I.M.; methodology, S.I.; software, I.S. and Y.P.; validation, D.P.-A., Y.P., and I.M.; formal analysis, I.M.; investigation, D.P.-A.; writing—original draft preparation, D.P.-A.; writing—review and editing, I.M. and S.I.; visualization, Y.P.; project administration, I.M. All authors have read and agreed to the published version of the manuscript.
Funding
This research work was supported by the CogniSoft “Information System for Diagnosis and Prevention of Multiple Sclerosis Patients” project, funded by the Program for Innovation and Competitiveness, co-financed by the EU through the ERDF under agreement no. BG16RFOP002-1.005; GATE “Big Data for Smart Society” project, funded by the Horizon 2020 WIDESPREAD-2018-2020 TEAMING Phase 2 program under grant agreement No. 857155; and the CogniTwin “Digital twin modelling of patients with cognitive disorders” project, funded by the Bulgarian National Science fund, under agreement no. KP-06-N32/5.
Conflicts of Interest
The authors declare no conflict of interest.
References
- Cotter, J.; Vithanage, N.; Colville, S.; Lyle, D.; Cranley, D.; Cormack, F.; Barnett, J.H.; Murray, K.; Pal, S. Investigating domain-specific cognitive impairment among patients with multiple sclerosis using touchscreen cognitive testing in routine clinical care. Front. Neurol. 2018, 9, 331. [Google Scholar] [CrossRef] [Green Version]
- Donaldson, E.; Patel, V.P.; Shammi, P.; Feinstein, A. Why Sex Matters: A Cognitive Study of People with Multiple Sclerosis. Cogn. Behav. Neurol. 2019, 32, 39–45. [Google Scholar] [CrossRef] [PubMed]
- Honarmand, K.; Akbar, N.; Kou, N.; Feinstein, A. Predicting employment status in multiple sclerosis patients: The utility of the MS functional composite. J. Neurol. 2011, 258, 244–249. [Google Scholar] [CrossRef] [PubMed]
- Labiano-Fontcuberta, A.; Mitchell, A.J.; Moreno-Garcia, S.; Benito-Leon, J. Cognitive impairment in patients with multiple sclerosis predicts worse caregiver’s health-related quality of life. Mult. Scler. J. 2014, 20, 1769–1779. [Google Scholar] [CrossRef] [PubMed]
- Golan, D.; Wilken, J.; Doniger, G.M.; Fratto, T.; Kane, R.; Srinivasan, J.; Zarif, M.; Bumstead, B.; Buhse, M.; Fafard, L.; et al. Validity of a multi-domain computerized cognitive assessment battery for patients with multiple sclerosis, Multiple sclerosis and related disorders. Mult. Scler. Relat. Disord. 2019, 30, 154–162. [Google Scholar] [CrossRef]
- IBM. Bringing Big Data to the Enterprise. Available online: http://www-01.ibm.com/software/data/bigdata/what-is-big-data.html (accessed on 20 February 2020).
- MIT Technology Review. Available online: https://www.technologyreview.com/business-report/data-driven-health-care/free/ (accessed on 22 February 2020).
- Van Rijmenam, M. Four Ways Big Data Will Revolutionize Education. Available online: https://datafloq.com/read/big-data-will-revolutionize-learning/2016 (accessed on 22 February 2020).
- Petrova-Antonova, D.; Spasov, I.; Krasteva, I.; Manova, I.; Ilieva, S. A Digital Twin Platform for Diagnostics and Rehabilitation of Multiple Sclerosis. In Proceedings of the 20th International Conference on Computational Science and its Applications (ICCSA 2020), Cagliari, Italy, 1–4 June 2020; Gervasi, O., Murgante, B., Misra, S., Garau, C., Blečić, I., Taniar, D., Apduhan, B.O., Rocha, A.M.A.C., Tarantino, E., Torre, C.M., Eds.; Theoretical Computer Science and General Issues; Springer: Berlin/Heidelberg, Germany, 2020. [Google Scholar]
- Aslam, R.W.; Bates, V.; Dundar, Y.; Hounsome, J.; Richardson, M.; Krishan, A.; Dickson, R.; Boland, A.; Fisher, J.; Robinson, L.; et al. A systematic review of the diagnostic accuracy of automated tests for cognitive impairment. Int. J. Geriatr. Psychiatry 2018, 33, 561–575. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Wild, K.; Howieson, D.; Webbe, F.; Seelye, A.; Kaye, J. Status of computerized cognitive testing in aging: A systematic review. Alzheimers Dement. 2008, 4, 428–437. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Zygouris, S.; Tsolaki, M. Computerized cognitive testing for older adults: A review. Am. J. Alzheimers Dis. Other Dement. 2015, 30, 13–28. [Google Scholar] [CrossRef]
- Lapshin, H.; O’Connor, P.; Lanctot, K.L.; Feinstein, A. Computerized cognitive testing for patients with multiple sclerosis. Mult. Scler. Relat. Disord 2012, 1, 196–201. [Google Scholar] [CrossRef] [PubMed]
- Brenk, A.; Laun, K.; Haase, C.G. Short-term cognitive training improves mental efficiency and mood in patients with multiple sclerosis. Eur. Neurol. 2008, 60, 304–309. [Google Scholar] [CrossRef]
- Baller, G.; Kalbe, E.; Kaesberg, S.; Kessler, J. NEUROvitalis. Ein Neuropsychologisches Grup-Penprogramm zur Förderung der Geistigen Leistungsfähigket; NEUROvitalis. A neuro-psychological group training for the improvement of cognitive performance; ProLog: Köln, German, 2010. [Google Scholar]
- Penner, I.; Kobel, M.; Stöcklin, M.; Opwis, K.; Calabrese, P. BrainStim—hirnstimulation als Präventions- und Therapiemassnahme? Neurogeriatrie 2007, 4, 109–115. [Google Scholar]
- Shatil, E.; Metzerb, A.; Horvitzc, O.; Millerb, A. Home-based personalized cognitive training in MS patients: A study of adherence and cognitive performance. NeuroRehabilitation 2010, 26, 143–153. [Google Scholar] [CrossRef] [PubMed]
- Brissart, H.; Leroy, M.; Debouverie, M. Cognitive rehabilitation in multiple sclerosis: Preliminary results and presentation of a new program. Rev. Neurol. 2010, 166, 406–411. [Google Scholar] [CrossRef] [PubMed]
- Stuifbergen, A.K.; Becker, H.; Perez, F.; Morison, J.; Kullberg, V.; Todd, A. A randomized controlled trial of a cognitive rehabilitation intervention for persons with multiple sclerosis. Clin Rehabil. 2012, 26, 882–893. [Google Scholar] [CrossRef] [PubMed]
- RehaCom. Computer-Aided Cognitive Rehabilitation; HASOMED GmbH: Magdeburg, Germany, 2003. [Google Scholar]
- Amato, M.P.; Goretti, B.; Viterbo, R.; Portaccio, E.; Niccolai, C.; Hakiki, B.; Iaffaldano, P.; Trojano, M. Computer-assisted rehabilitation of attention in patients with MS: Results of randomized, double blind trial. Mult. Scler. 2014, 20, 91–98. [Google Scholar]
- Pascoe, M.; Alamri, Y.; Dalrymple-Alford, J.; Anderson, T.; MacAskill, M. The Symbol-Digit Modalities Test in mild cognitive impairment: Evidence from Parkinson’s disease patients. Eur. Neurol. 2018, 79, 206–210. [Google Scholar] [CrossRef]
- Parmenter, B.A.; Weinstock-Guttman, B.; Garg, N.; Munschauer, F.; Benedict, R.H.B. Screening for cognitive impairment in multiple sclerosis using the Symbol Digit Modalities Test. Mult. Scler. 2007, 13, 52–57. [Google Scholar] [CrossRef]
- Riccitelli, G.C.; Pagani, E.; Rodegher, M.; Colombo, B.; Preziosa, P.; Falini, A.; Comi, G.; Filippi, M.; Rocca, M.A. Imaging patterns of gray and white matter abnormalities associated with PASAT and SDMT performance in relapsing-remitting multiple sclerosis. Mult. Scler. 2019, 25, 204–216. [Google Scholar] [CrossRef]
- Benedict, R. Brief Visuospatial Memory Test-Revised: Professional Manual; Psychological Assessment Resources: Odessa, FL, USA, 1997. [Google Scholar]
- Smerbeck, A.; Benedict, R.H.B.; Eshaghi, A.; Vanotti, S.; Spedo, C.; Blahova Dusankova, J.; Sahraian, M.A.; Marques, V.D.; Langdon, D. Influence of nationality on the Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS). Clin. Neuropsychol. 2018, 32, 54–62. [Google Scholar] [CrossRef]
- Rocca, M.A.; Amato, M.P.; De Stefano, N.; Enzinger, C.; Geurts, J.J.; Penner, I.K.; Rovira, A.; Sumowski, J.F.; Valsasina, P.; Filippi, M. Clinical and imaging assessment of cognitive dysfunction in multiple sclerosis. Lancet Neurol. 2015, 14, 302–317. [Google Scholar] [CrossRef]
- Vlahou, C.H.; Kosmidis, M.H.; Dardagani, A.; Tsotsi, S.; Giannakou, M.; Giazkoulidou, A.; Zervoudakis, E.; Pontikakis, N. Development of the Greek Verbal Learning Test: Reliability, construct validity, and normative standards. Arch. Clin. Neuropsychol. 2012, 28, 52–64. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Dusankova, J.B.; Kalincik, T.; Havrdova, E.; Benedict, R.H. Cross cultural validation of the minimal assessment of cognitive function in multiple sclerosis (MACFIMS) and the brief international cognitive assessment for multiple sclerosis (BICAMS). Clin. Neuropsychol. 2012, 26, 1186–1200. [Google Scholar] [CrossRef] [PubMed]
- Langdon, D.W.; Amato, M.P.; Boringa, J.; Brochet, B.; Foley, F.; Fredrikson, S.; Hämäläinen, P.; Hartung, H.P.; Krupp, L.; Penner, I.K.; et al. Recommendations for a brief international cognitive assessment for multiple sclerosis (BICAMS). Mult. Scler. 2012, 18, 891–898. [Google Scholar] [CrossRef] [Green Version]
- Sousa, C.; Rigueiro-Neves, M.; Miranda, T.; Alegria, P.; Vale, J.; Passos, A.M.; Langdon, D.; Sa, M.J. Validation of the brief international cognitive assessment for multiple sclerosis (BICAMS) in the Portuguese population with multiple sclerosis. BMC Neurol. 2018, 18, 172. [Google Scholar] [CrossRef] [PubMed]
- Karolinska Institute. Available online: https://ki.se/en/kind/recognize-emotions-with-eu-emotion-video-material (accessed on 20 September 2020).
- Todorova, L.; Ignatova, V. Computerized neuropsychological test battery CogniSoft for assessment of cognition in patients with multiple sclerosis. Bioautomation. accepted.
- Ignatova, V.; Stoyanova, T.; Haralanov, L. Results from a current pilot study for approbation of a computer system CogniSoft for diagnosis and rehabilitation of cognitive impairments in persons with multiple sclerosis. In Proceedings of the IX International Congress of Psychology, Sofia, Bulgaria, 13–15 November 2020. accepted. [Google Scholar]
| Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).