Next Article in Journal
Print and Try Technique: 3D-Printing of Teeth with Complex Anatomy a Novel Endodontic Approach
Next Article in Special Issue
Machine Learning Driven Contouring of High-Frequency Four-Dimensional Cardiac Ultrasound Data
Previous Article in Journal
Study on Time Reversal Maximum Ratio Combining in Underwater Acoustic Communications
Previous Article in Special Issue
Augmented Reality Based Surgical Navigation of Complex Pelvic Osteotomies—A Feasibility Study on Cadavers
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Communication

Postural Control Disturbances Induced by Virtual Reality in Stroke Patients

by
Charles Morizio
1,
Maxime Billot
2,
Jean-Christophe Daviet
3,
Stéphane Baudry
4,
Christophe Barbanchon
4,
Maxence Compagnat
3 and
Anaick Perrochon
3,*
1
ILFOMER, Institut Limousin de Formation aux Métiers de la Réadaptation, 87000 Limoges, France
2
PRISMATICS Lab (Predictive Research in Spine/Neuromodulation Management and Thoracic Innovation/Cardiac Surgery), Poitiers University Hospital, 86021 Poitiers, France
3
HAVAE, Université de Limoges, EA 6310, 87000 Limoges, France
4
Laboratory of Applied Biology and Research Unit in Applied Neurophysiology, ULB Neuroscience Institute, Université libre de Bruxelles, 1050 Brussels, Belgium
*
Author to whom correspondence should be addressed.
Appl. Sci. 2021, 11(4), 1510; https://doi.org/10.3390/app11041510
Submission received: 20 December 2020 / Revised: 14 January 2021 / Accepted: 5 February 2021 / Published: 8 February 2021
(This article belongs to the Special Issue Artificial Intelligence (AI) and Virtual Reality (VR) in Biomechanics)

Abstract

:
People who survive a stroke are often left with long-term neurologic deficits that induce, among other impairments, balance disorders. While virtual reality (VR) is growing in popularity for postural control rehabilitation in post-stroke patients, studies on the effect of challenging virtual environments, simulating common daily situations on postural control in post-stroke patients, are scarce. This study is a first step to document the postural response of stroke patients to different challenging virtual environments. Five subacute stroke patients and fifteen age-matched healthy adults were included. All participants underwent posturographic tests in control conditions (open and closed eyes) and virtual environment without (one static condition) and with avatars (four dynamic conditions) using a head-mounted device for VR. In dynamic environments, we modulated the density of the virtual crowd (dense and light crowd) and the avoidance space with the avatars (near or far). Center of pressure velocity was collected by trial throughout randomized 30-s periods. Results showed that more challenging conditions (dynamic condition) induced greater postural disturbances in stroke patients than in healthy counterparts. Our study suggests that virtual reality environments should be adjusted in light of obtaining more or less challenging conditions.

1. Introduction

A stroke is a serious life-threatening medical condition characterized by the interruption of blood flow to a part of the central nervous system due to an ischemic or hemorrhagic vascular injury [1]. Patients who survive a stroke (SP) suffer from various somatosensory impairments, motor dysfunction, perceptual, visual disturbances, and altered spatial cognition with reference to the upright body position [2,3,4], resulting in balance disorders [2,3]. SPs show changes in motor strategies for postural control with delayed and reduced anticipatory postural adjustments [4] and a shift in sensory weighting, with an increased reliance on visual information [5]. Balance disorders are responsible for medical complications due to falls and are associated with the inability to perform activities of daily life [4].
Among the wide variety of motor rehabilitation programs, virtual reality (VR) has shown promising results as a means of improving postural control abilities [6,7]. VR is defined as “the use of interactive simulations created with computer hardware and software to present users with opportunities to engage in environments that appear and feel similar to real-world objects and events” [8] and features immersive systems such as head-mounted display [9]. Immersive VR using a head-mounted display seems to be safe and feasible in stroke patient rehabilitation [10]. VR technology offers the possibility to immerse people in a 3D environment that can mimic a real-life scenario, allowing assessment of postural control in more realistic situations with the possibility to manipulate, in a reproducible way, the task complexity [11].
Previous work indicates that static virtual environments (VE) do not appear to challenge postural control [12,13] whereas dynamic scenes induce postural perturbations in healthy young adults (HA) [12] and older adults [14]. However, D’Antonio et al. [15] reported lesser postural perturbations in SP and older HA compared with young HA in semi-immersive environment. To date, there exists no clear evidence on the effects of static and dynamic immersive VE on postural control in SP, specifically in ecological environments with irregular visual flows. Immersive VR, compared to semi-immersive or screen projection, improves the immersion and the presence (being in the virtual world) of the patients [16] and should be preferred in investigating the VR effect on postural control and in rehabilitation programs.
The current study aimed to investigate the effects of dynamic VE on the postural control in SP and HA. Dynamic VE consisted of visual flow modulation including a crowd of walking avatars. We hypothesized that stroke patients would show greater postural perturbations in dynamic VE than HA, given that they are strongly reliant to visual information to maintain postural control, with increased responses in more challenging VE.

2. Methods

Five age-matched SP (61.2 ± 7.05 years old, range: 51–69 years, two women) and 15 age-matched HA (59.1 ± 2.8 years old, range: 55–65 years, nine women) participated in the study conducted in the Physical Medicine and Rehabilitation department. The inclusion criteria were as follows: adults (>18 years old) with subacute paresis [14 days–6 months] having given informed agreement. The exclusion criteria were: (i) standing upright for less than 1 min without assistive device; (ii) cognitive impairment (Mini Mental State Examination < 24); (iii) inability to understand the instructions (Boston Diagnostic Aphasia Examination (BDAE) < 2); (iv) visual or auditory impairment; (v) motion sickness in VE by adverse events (e.g., vertigo); and (vi) neurological or orthopedic complications modifying quiet standing. All subjects gave their informed consent for inclusion before they participated in the study. The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved by the Ethics Committee of the University Hospital Center of Poitiers (CHU86-RECH-R2021-02-01).
A clinical assessment of fear of falling with the Falls Efficacy Scale-International (FES-I) and balance abilities with Postural Assessment Scale for Stroke Patients (PASS) was filled by the SP.
Before starting the postural control assessment, each participant discovered neutral VE in a seated position and carried out a 30-s familiarization trial on a force platform. Then, postural control assessment was performed while participants were positioned with bare feet oriented at 30° from the sagittal midline in two control conditions: open eyes (OE) and closed eyes (CE); and in two types of VE (Figure 1A): a static virtual environment where the participant is in the middle of a room without any avatars (Figure 1B), and dynamic virtual environments, which represent a commercial center with moving avatars walking at spontaneous velocity following random displacements (1.1 m·s−1) (Figure 1C–F). Dynamic virtual environments (DVEs) provided four visual conditions obtained by the modulation of virtual crowd density (sparse and dense) and avoidance space with avatars (wide: 0.8 m and narrow: 0.1 m). We thereby tested four DVE conditions: sparse and wide (SW); dense and wide (DW); sparse and narrow (SN); and dense and narrow (DN) (Figure 1C–F). Conditions were randomly assigned before the beginning of the session. After each condition, participants were asked to determine the difficulty in identifying feelings to maintain posture stability with a visual analogue scale (VAS) from 0 to 10 (0 no difficulty and 10 unable to perform).
During assessment, safety devices (i.e., human security, metal frame) were put in place to prevent falls (Figure 1A). Postural data were collected over 30 s per trial with a 1-min rest between trials. A WIN-POSTURO force platform (Medicapteurs®, Co. France) was used to measure the center of pressure (CoP) displacement (sampling rate: 40 Hz [17]), and an head-mounted display device (HTC® Vive, HTC Co., New Taipei City, China) with a large field of vision (110°) and a high resolution screen (1200 × 1080 px each eye) to display VE (Virtualis® software, Co. France). CoP velocity was expressed in mm·s−1 (corresponding to the cumulative distance over the sampling period) [18]. To investigate the effect of visual attention on postural control due to VE, we used the following equation: VRcost = (VE – OE)/OE∗100) (in%). Raw signals were imported in MATLAB (Mathworks Inc., Natick, MA, USA) and low-pass filtered (fourth order low-pass zero-phase lag Butterworth filter, 7 Hz cut-off frequency) prior to computing the CoP velocity.
The data are presented as median with data range. The Mann–Whitney U-test was performed to compare postural performance (i.e., CoP velocity and Vrcost) between the OA and HA groups. Statistical analysis was carried out with SPSS Statistics® 22 software (IBM Corp, Armonk, NY, USA) with a p-value < 0.05.

3. Results

Twenty-two SP and 15 HA were screened, where 17 SP were not included for the following reasons: six for wearing of an orthosis, six could not stand for one minute, two for visual trouble, one for motion sickness, one for hip surgery, and one for cognitive impairment. As shown in Table 1, the socio-demographic variables of both groups were homogeneous. Clinical characteristics of SP were a Barthel index of 87 [70–100]; PASS of 33.6 [32–36]; and FES-I of 26.6 [16–39]. There were four left and one right superficial sylvian ischemic strokes and three of them had aphasia. SP had a NIHSS (National Institutes of Health Stroke Scale) of 6 [6–9] at admission into the department.
Postural control data are presented in Figure 2. There were no significant differences of CoP velocity displacement between SP and HA in open eyes (p = 0.275), closed eyes (p = 0.266), and static virtual environment (p = 0.098) conditions. In contrast, our results revealed that CoP velocity was greater in SP than HA for all dynamic virtual environment conditions (p < 0.05).
Furthermore, the Vrcost appeared significantly higher for SP than HA in the dynamic virtual environment DN condition (51.9% for SP and 20.9% for HA, p = 0.033). Vrcost did not appear significantly different in any other conditions (p > 0.05). No group effect was observed for difficulty in identifying feelings (VAS) for all conditions.

4. Discussion

This study aimed to compare the impact of static and dynamic virtual environments (i.e., visual flow modulation including a crowd of walking avatars) on postural control in SP with HA. CoP velocity was significantly greater for SP than for HA in the dynamic virtual environment, and Vrcost was greater in SP than HA in the dynamic virtual environment of dense and narrow conditions, showing that the difficulty of VE more influenced the postural control in SP compared with HA.
While we found that the postural control of SP and HA was not differently altered in the SVE condition, it was more perturbed in SP than in HA under DVE conditions. In a recent study, Chiarovano et al. [14] reported that the percentage of postural control deficit in a virtual environment increased with age and amplitude of the moving scene. In addition, it has been shown that postural control of SP is affected by visual rather than proprioceptive or vestibular perturbation [5]. Many authors have suggested that in comparison with young adults, older adults and SP are not able to adapt their postural behavior to the external perturbations [15,18,19]. In our study, challenging VE led to more pronounced postural perturbation in SP than in HA, which is consistent with their strong reliance on visual information [5]. Furthermore, in contrast to other studies evaluating the effect of VR on postural control [12,13,15], we proposed a multiple complex task with various optical flow induced by changing the density and avoidance space of avatars in a crowded environment. All in all, we suggest that immersive and various ecological virtual environment conditions can be used in SP virtual reality rehabilitation programs with progressive challenging situations.
In addition, it has been reported that attentional demand increases in SP compared to HA, especially when conditions become more challenging [20]. In a near-infrared spectrometry study, Hinderaker et al. [21] showed that attentional cost, illustrated by prefrontal cortex activation modulation, rose in older adults under challenging balance conditions. This process may be exacerbated in SP and could be partially related to postural control alteration in a challenging VR environment. VR permits a standardized and reproducible environment that can be used to present ecological situations [11] and can provide an additional assessment by observing SP behavior in daily living activities.
While the VR environment provided clear postural impairment in SP, our study was not free of limitations. While the sample size of SP was small, our study showed that among 22 SPs, only five met our inclusion criteria. This limited enrolment is related to the fact that screening was performed in routine practice and we chose to recruit patients with high postural capabilities (PASS: 33.6 [32:36]) to test a wide range of VR conditions. Regardless of these limitations, our pilot study provides new insights that should be considered in future randomized controlled trials. Furthermore, investigation of the underlying mechanisms of postural control alteration in VR for both SP and HA needs to be undertaken to improve the VR rehabilitation program.

5. Conclusions

Our study showed that dynamic VE had a greater impact on postural control in stroke patients than in healthy adults, all the more so when challenging ecological situations are presented (crowded environment). While immersive VR is currently used to improve postural control in stroke rehabilitation, our study suggests that the level of visual perturbation should be adjusted to provide more or less challenging conditions for postural control.

Author Contributions

Conceptualization, M.B. and A.P.; methodology, C.M.; validation, all authors; formal analysis, C.M. and A.P.; investigation, C.M.; resources, J.-C.D. and M.C.; writing—original draft preparation, C.M., A.P. and M.B.; writing—review and editing, J.-C.D., M.C., S.B. and C.B.; supervision, M.B. and A.P. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Institutional Review Board of the University Hospital Center of Poitiers (CHU86-RECH-R2021-02-01).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Conflicts of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

  1. Stroke, R.H. Handbook of Clinical Neurology; Elsevier: Amsterdam, The Netherlands, 2013; pp. 427–433. [Google Scholar]
  2. Alfieri, F.M.; Riberto, M.; Lopes, J.A.F.; Filippo, T.R.; Imamura, M.; Battistella, L.R. Postural Control of Healthy Elderly Individuals Compared to Elderly Individuals with Stroke Sequelae. Open Neurol. J. 2016, 10, 1–8. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Zampogna, A.; Mileti, I.; Palermo, E.; Celletti, C.; Paoloni, M.; Manoni, A.; Mazzetta, I.; Costa, G.D.; Pérez-López, C.; Caerota, F.; et al. Fifteen Years of Wireless Sensors for Balance Assessment in Neurological Disorders. Sensors 2020, 20, 3247. [Google Scholar] [CrossRef] [PubMed]
  4. Tasseel-Ponche, S.; Yelnik, A.P.; Bonan, I.V. Motor strategies of postural control after hemispheric stroke. Neurophysiol. Clin./Clin. Neurophysiol. 2015, 45, 327–333. [Google Scholar] [CrossRef] [PubMed]
  5. Bonan, I.V.; Marquer, A.; Eskiizmirliler, S.; Yelnik, A.P.; Vidal, P.-P. Sensory reweighting in controls and stroke patients. Clin. Neurophysiol. 2013, 124, 713–722. [Google Scholar] [CrossRef] [PubMed]
  6. Mohammadi, R.; Semnani, A.V.; Mirmohammadkhani, M.; Grampurohit, N. Effects of Virtual Reality Compared to Conventional Therapy on Balance Poststroke: A Systematic Review and Meta-Analysis. J. Stroke Cerebrovasc. Dis. 2019, 28, 1787–1798. [Google Scholar] [CrossRef] [PubMed]
  7. Laver, K.E.; Lange, B.; George, S.; Deutsch, J.E.; Saposnik, G.; Crotty, M. Virtual reality for stroke rehabilitation. Cochrane Database Syst. Rev. 2017, 11, CD008349. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  8. Weiss, P.L.; Kizony, R.; Feintuch, U.; Katz, N. Virtual reality in neurorehabilitation. In Textbook of Neural Repair and Rehabilitation; Selzer, M., Clarke, S., Cohen, L., Duncan, P., Gage, F., Eds.; Cambridge University Press: Cambridge, UK, 2006; pp. 182–197. [Google Scholar]
  9. Tieri, G.; Morone, G.; Paolucci, S.; Iosa, M. Virtual reality in cognitive and motor rehabilitation: Facts, fiction and fallacies. Expert Rev. Med Devices 2018, 15, 107–117. [Google Scholar] [CrossRef] [PubMed]
  10. Proffitt, R.; Warren, J.; Lange, B.; Chang, C.-Y. Safety and Feasibility of a First-Person View, Full-Body Interaction Game for Telerehabilitation Post-Stroke. Int. J. Telerehab. 2018, 10, 29–36. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  11. Morel, M.; Bideau, B.; Lardy, J.; Kulpa, R. Advantages and limitations of virtual reality for balance assessment and rehabilitation. Neurophysiol. Clin./Clin. Neurophysiol. 2015, 45, 315–326. [Google Scholar] [CrossRef]
  12. Luo, H.; Wang, X.; Fan, M.; Deng, L.; Jian, C.; Wei, M.; Luo, J. The Effect of Visual Stimuli on Stability and Complexity of Postural Control. Front. Neurol. 2018, 9, 48. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Robert, M.T.; Ballaz, L.; Lemay, M. The effect of viewing a virtual environment through a head-mounted display on balance. Gait Posture 2016, 48, 261–266. [Google Scholar] [CrossRef] [PubMed]
  14. Chiarovano, E.; Wang, W.; Rogers, S.J.; MacDougall, H.G.; Curthoys, I.S.; de Waele, C. Balance in Virtual Reality: Effect of Age and Bilateral Vestibular Loss. Front. Neurol. 2017, 20, 8. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  15. D’Antonio, E.; Tieri, G.; Patané, F.; Morone, G.; Iosa, M. Stable or able? Effect of virtual reality stimulation on static balance of post-stroke patients and healthy subjects. Hum. Mov. Sci. 2020, 70, 102569. [Google Scholar] [CrossRef] [PubMed]
  16. Schuemie, M.J.; van der Straaten, P.; Krijn, M.; van der Mast, C.A. Research on presence in virtual reality: A survey. Cyberpsychol. Behav. 2001, 4, 183–201. [Google Scholar] [CrossRef] [PubMed]
  17. Lemieux, L.G.; Simoneau, M.; Tessier, J.-F.; Billot, M.; Blouin, J.; Teasdale, N. Balance control interferes with the tracing performance of a pattern with mirror-reversed vision in older persons. AGE 2014, 36, 823–837. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Billot, M.; Handrigan, G.A.; Simoneau, M.; Corbeil, P.; Teasdale, N. Short term alteration of balance control after a reduction of plantar mechanoreceptor sensation through cooling. Neurosci. Lett. 2013, 535, 40–44. [Google Scholar] [CrossRef] [PubMed]
  19. Billot, M.; Handrigan, G.A.; Simoneau, M.; Teasdale, N. Reduced plantar sole sensitivity induces balance control modifications to compensate ankle tendon vibration and vision deprivation. J. Electromyogr. Kinesiol. 2015, 25, 155–160. [Google Scholar] [CrossRef] [PubMed]
  20. Brown, L.A.; Sleik, R.J.; Winder, T.R. Attentional demands for static postural control after stroke. Arch. Phys. Med. Rehabil. 2002, 83, 1732–1735. [Google Scholar] [CrossRef] [PubMed]
  21. Hinderaker, M.; Sylcott, B.; Williams, K.; Lin, C.-C. Aging Affects the Ability to Process the Optic Flow Stimulations: A Functional Near-Infrared Spectrometry Study. J. Mot. Behav. 2020, 52, 466–473. [Google Scholar] [CrossRef] [PubMed]
Figure 1. General experimental setup (A), static virtual environment (B), and dynamic virtual environment in sparse crowd and wide space (C), dense crowd and wide space (D), sparse crowd and narrow space (E), and dense crowd and narrow space (F) conditions.
Figure 1. General experimental setup (A), static virtual environment (B), and dynamic virtual environment in sparse crowd and wide space (C), dense crowd and wide space (D), sparse crowd and narrow space (E), and dense crowd and narrow space (F) conditions.
Applsci 11 01510 g001
Figure 2. CoP velocity (A) and Vrcost (B) between HA (healthy adults) and SP (stroke patients). CoP: Center of Pressure; Vrcost: Virtual Reality cost; D: dense; N: Narrow; S: sparse; W: Wide.
Figure 2. CoP velocity (A) and Vrcost (B) between HA (healthy adults) and SP (stroke patients). CoP: Center of Pressure; Vrcost: Virtual Reality cost; D: dense; N: Narrow; S: sparse; W: Wide.
Applsci 11 01510 g002
Table 1. Clinical data and postural control data.
Table 1. Clinical data and postural control data.
ParametersStroke PatientsHealthy Adultsp-Value
Population (N)515-
Women (N (%)2 (40)9 (60)0.479
Age (years)61.2 ± 7.159.1 ± 2.80.334
BMI (Kg/cm2)25.9 ± 2.225.7 ± 3.91
Barthel87.0 [70:100]--
PASS33.6 [32:36]--
FES-I26.6 [16:39]--
NIHSS6.0 [6:9]--
CoP velocity (mm.s−1)
Open Eyes13.1 [11.0; 18.0]11.8 [9.2; 14.0]0.275
Closed Eyes19.7 [18.8; 21.3]17.6 [14.9; 22.0]0.266
Static virtual environment15.6 [13.6; 15.8]10.7 [9.7; 12.9]0.098
Dynamic virtual environment SW 15.1 [14.8; 15.9]10.7 [9.7; 14.0]0.025 *
Dynamic virtual environment DW16.9 [15.8; 18.2]12.2 [10.5; 14.7]0.015 *
Dynamic virtual environment SN16.0 [13.4; 18.9]11.7 [8.7; 13.8]0.025 *
Dynamic virtual environment DN 18.2 [18.1; 19.2]13.0 [11.3; 17.2]0.042 *
Vrcost (%)
Static virtual environment17.72.80.23
Dynamic virtual environment SW 28.84.50.197
Dynamic virtual environment DW31.213.10.081
Dynamic virtual environment SN38.12.90.081
Dynamic virtual environment DN51.920.90.033 *
VAS (0–10)
Open Eyes2.0 ± 2.11.1 ± 1.20.479
Closed Eyes3.2 ± 2.62.2 ± 1.90.403
Static virtual environment2.3 ± 1.71.3 ± 1.70.215
Dynamic virtual environment SW 2.4 ± 2.1 1.2 ± 1.20.215
Dynamic virtual environment DW2.6 ± 1.8 1.5 ± 1.60.25
Dynamic virtual environment SN2.4 ± 2.1 1.3 ± 1.30.271
Dynamic virtual environment DN 3.5 ± 2.61.9 ± 1.70.202
Age and BMI (Body Mass Index): mean ± SD (standard deviation); CoP displacement velocity (mean velocity): Med [Q1; Q3]. D: dense; N: Narrow; S: sparse; W: Wide; PASS: Postural Assessment Scale for Stroke; FES-I: Fall Efficacy Scale—International; NIHSS: National Institutes of Health Stroke Scale; Vrcost: Virtual Reality cost (Vrcost = (VE − OE)/OE × 100). VAS: Visual Analogic Scale from 0 no difficulty to 10 unable to perform (mean ± SD); *: p < 0.05.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Morizio, C.; Billot, M.; Daviet, J.-C.; Baudry, S.; Barbanchon, C.; Compagnat, M.; Perrochon, A. Postural Control Disturbances Induced by Virtual Reality in Stroke Patients. Appl. Sci. 2021, 11, 1510. https://doi.org/10.3390/app11041510

AMA Style

Morizio C, Billot M, Daviet J-C, Baudry S, Barbanchon C, Compagnat M, Perrochon A. Postural Control Disturbances Induced by Virtual Reality in Stroke Patients. Applied Sciences. 2021; 11(4):1510. https://doi.org/10.3390/app11041510

Chicago/Turabian Style

Morizio, Charles, Maxime Billot, Jean-Christophe Daviet, Stéphane Baudry, Christophe Barbanchon, Maxence Compagnat, and Anaick Perrochon. 2021. "Postural Control Disturbances Induced by Virtual Reality in Stroke Patients" Applied Sciences 11, no. 4: 1510. https://doi.org/10.3390/app11041510

APA Style

Morizio, C., Billot, M., Daviet, J. -C., Baudry, S., Barbanchon, C., Compagnat, M., & Perrochon, A. (2021). Postural Control Disturbances Induced by Virtual Reality in Stroke Patients. Applied Sciences, 11(4), 1510. https://doi.org/10.3390/app11041510

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop