Next Article in Journal
Cardiorespiratory Fitness, Blood Pressure and Ethnicity Are Related to Salivary Cortisol Responses after an Exercise Test in Children: The ExAMIN Youth SA Study
Next Article in Special Issue
Health Literacy and Frailty in Community-Dwelling Older Adults: Evidence from a Nationwide Cohort Study in South Korea
Previous Article in Journal
Human Rights and Empowerment in Aged Care: Restraint, Consent and Dying with Dignity
Previous Article in Special Issue
The Ecology of Unsheltered Homelessness: Environmental and Social-Network Predictors of Well-Being among an Unsheltered Homeless Population
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Access to Healthcare for Migrant Patients in Europe: Healthcare Discrimination and Translation Services

1
Polibienestar Research Institute, University of Valencia, 46010 Valencia, Spain
2
International Foundation for Integrated Care, Oxford OX2 6UD, UK
3
Institute of Preventive Medicine Environmental and Occupational Health Prolepsis, 15125 Marousi, Greece
4
Department of Hygiene Epidemiology, Medical Statistics Medical School National, Kapodistrian University of Athens, 11527 Athens, Greece
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2021, 18(15), 7901; https://doi.org/10.3390/ijerph18157901
Submission received: 21 April 2021 / Revised: 17 July 2021 / Accepted: 19 July 2021 / Published: 26 July 2021
(This article belongs to the Special Issue Health and Well-Being in Vulnerable Communities)

Abstract

:
Background: Discrimination based on ethnicity and the lack of translation services in healthcare have been identified as main barriers to healthcare access. However, the actual experiences of migrant patients in Europe are rarely present in the literature. Objectives: The aim of this study was to assess healthcare discrimination as perceived by migrants themselves and the availability of translation services in the healthcare systems of Europe. Methods: A total of 1407 migrants in 10 European Union countries (consortium members of the Mig-HealthCare project) were surveyed concerning healthcare discrimination, access to healthcare services, and need of translation services using an interviewer-administered questionnaire. Migrants in three countries were excluded from the analysis, due to small sample size, and the new sample consisted of N = 1294 migrants. Descriptive statistics and multivariable regression analyses were conducted to investigate the risk factors on perceived healthcare discrimination for migrants and refugees in the EU. Results: Mean age was 32 (±11) years and 816 (63.26%) participants were males. The majority came from Syria, Afghanistan, Iraq, Nigeria, and Iran. Older migrants reported better treatment experience. Migrants in Italy (0.191; 95% CI [0.029, 0.352]) and Austria (0.167; 95% CI [0.012, 0.323]) scored higher in the Discrimination Scale to Medical Settings (DMS) compared with Spain. Additionally, migrants with better mental health scored lower in the DMS scale (0.994; 95% CI [0.993, 0.996]), while those with no legal permission in Greece tended to perceive more healthcare discrimination compared with migrants with some kind of permission (1.384; 95% CI [1.189, 1.611]), as opposed to Austria (0.763; 95% CI [0.632, 0.922]). Female migrants had higher odds of needing healthcare assistance but not being able to access them compared with males (1.613; 95% CI [1.183, 2.199]). Finally, migrants with chronic problems had the highest odds of needing and not having access to healthcare services compared with migrants who had other health problems (3.292; 95% CI [1.585, 6.837]). Conclusions: Development of culturally sensitive and linguistically diverse healthcare services should be one of the main aims of relevant health policies and strategies at the European level in order to respond to the unmet needs of the migrant population.

1. Introduction

Given the continuous immigration flows to Europe and the increased numbers of migrants arriving to EU member states [1], issues related to the more equitable and effective delivery of healthcare become a high priority for EU member states. Increasing access to basic health services for the migrant population in host countries is of utmost importance. The scant evidence that exists on healthcare access for migrants in Europe makes it difficult to make comparisons between systems and countries or to support public policy decision making [2]. Although the lack of language services overall and the fear of being ethnically discriminated within the healthcare setting are highlighted in the literature as prevalent problems for migrants when accessing healthcare [3], less is known about the experiences of the migrant population once they actually access the healthcare services in Europe.
The rights of refugees, asylum seekers, and migrants to healthcare access vary across European countries in terms of regulation and laws [4]. Even when access to healthcare is granted by law, research suggests that migrant groups, in particular asylum seekers [5] and undocumented migrants [6,7], face several obstacles when trying to access healthcare. In this regard, the difficulty to communicate effectively in a host country’s language can impede access to healthcare services [8]. The provision of translation services can help to overcome these communication barriers; however, the availability of free and accessible interpretation services is highly variable across Europe [9]. Studies indicate that the presence of professional interpreters can improve both the quality of care [10] as well as patient satisfaction. Alternatively, inefficient communication between healthcare providers and patients increases the risk of misunderstanding and misdiagnosis [11]. Moreover, patients who face communication difficulties visit their healthcare provider less often [12] and are less compliant with medication and treatment advice [13]. The lack of professional interpreters has been associated with unnecessary, extensive, and potentially harmful medical exams, treatments, and hospitalisations [14,15].
Furthermore, a large-scale study (QUALICOPC) conducted in 31 European countries showed that most differences on perceived healthcare discrimination were found between the native inhabitants of a country and first-generation migrants, reporting more discrimination within healthcare settings for the migrant population [16]. In this regard, previous studies have identified that experiencing ethnic discrimination in the healthcare setting is associated with poor quality of care [17], highest levels of distrust in healthcare providers as well as distrust in the healthcare system overall [18,19]. Moreover, different studies have also found an association between experiencing healthcare discrimination and decreased healthcare utilization and participation in preventive services. Studies have also indicated that discrimination within healthcare settings is related to increased emergency department visits and hospital admissions [20]. Moreover, experiencing healthcare discrimination has a detrimental impact on health outcomes. As a result of racial discrimination, studies have shown that migrant patients with diabetes do not receive appropriate diabetes care [21] and experience higher risk of diabetes comorbidities [22]. Moreover, racial discrimination in healthcare is also associated with lower medication adherence in ethnic diverse patients taking hypertension medication [23,24].

Mig-HealthCare Background

The Mig-HealthCare project was a three-year project launched in May 2017 [25]. The project’s main scope was to provide evidence-based information and practical guidance to primary healthcare professionals, primarily in the EU member states, on how to best address the health issues of refugee and migrant populations.
In order to support public policy decision making to reduce obstacles in healthcare access, and thus reduce health disparities for migrant populations in Europe, it is important to study the migrant patient experience as a result of them accessing healthcare services. Given the worse health outcomes, healthcare misuse and receiving lower quality of care associated with the perception of ethnic discrimination in healthcare and the lack of interpretation services, generating a greater understanding of the patient experience of the migrant groups in Europe is important to plan future tailored healthcare systems and services [26]. This paper, has collected and analysed migrant patient experiences in 10 European countries (Austria, Bulgaria, Cyprus, France, Germany, Greece, Italy, Malta, Spain, and Sweden). The primary aim of this study is to assess the perception of ethnic discrimination within healthcare services in Europe while having a comparative framework to understand the commonalities and differences across European countries by country of origin and language proficiency of migrants. The secondary aim is to assess the need for translation services when accessing healthcare services and identify factors that predict the feeling of being ethnically discriminated when accessing healthcare in Europe.

2. Methodology

Design and Sample

This study was a cross sectional survey that included 1407 participants. A detailed description of the methodology is reported elsewhere [27]. Participants were recruited via the snowball sampling method in 10 European countries (Austria, Bulgaria, Cyprus, France, Germany, Greece, Italy, Malta, Spain, and Sweden). Participants were eligible if they were aged 18 years old or over, had resided in the country of interview for a period of 6 months to 5 years, and were able to both understand the study goals and give consent for participation in the survey. In the case where study participants could not communicate in the language of the host country of interview, the assistance of an accredited interpreter was sought and provided to complete the questionnaire. Demographics of the total sample (N = 1407) are described in Supplementary Table S4. We excluded migrants from France, Germany, and Malta from our analysis, due to a small sample size in these countries and the new sample consisted of N = 1294 migrants.
Participation in the study was purely voluntary, with the sample comprising refugees/migrants who were visiting health and social care delivery services in which Mig-HealthCare partners operated (such as health centres in refugee camps, primary healthcare centres, community centres, and NGO clinics). The specifically designed questionnaire was tested and translated into Arabic, Farsi, Dari, Pashto, Somali as well as into the languages of the partner countries (migrant and refugee host countries). All interviewers were trained prior to the initiation of the study. Data collection took place from April 2018 to September 2019, after being approved by the Ethical Committee of both the University of Valencia and the National and Kapodistrian University of Athens, Medical School. Additional ethical approvals were obtained as necessary by each participating partner organization. No identifiable personal data were collected, and a unique identity was assigned to each study participant which was only available to the main researchers. The purpose of the study and the data collection method was clearly explained to each study participant and informed consent was sought and received before participation.

3. Measures

3.1. Discrimination in Medical Settings (DMS)

Ethnic discrimination experienced in medical settings was measured through the Discrimination Scale in Medical Settings (DMS scale) [28]. Participants were asked the following questions: “When getting healthcare of any kind, have you ever had any of the following things happen to you because of your race or ethnicity?”, with seven adapted items: (1) you are treated with less courtesy than other people, (2) you are treated with less respect than other people, (3) you receive poorer service than others, (4) a doctor or nurse acts as if they think you are not smart, (5) a doctor or nurse acts as if they are afraid of you, (6) a doctor or nurse acts as if they are better than you, and (7) you feel like a doctor or nurse is not listening to what you were saying. Response categories were 1 = never, 2 = rarely, 3 = sometimes, 4 = most of the time, and 5 = always. Then, all seven questions were summed and a mean score on the entire scale was computed, with higher scores indicating more perceived discrimination (range from 1–5 units). The DMS scale was evaluated for reliability. Pearson correlations between DMS score and its seven component items were positive and larger than 0.3 and the diagonal Cronbach’s α results scored excellent (more than 0.9), therefore confirming the reliability of the DMS scale (Supplementary Tables S1 and S2). A mean score on the entire scale was computed, with higher scores indicating more perceived discrimination.

3.2. Mental and Physical Health Status

For this study, the relevant to mental health scale of the Short Form 36 (SF-36) [29] was used to provide an assessment of mental health status. The following five questions were used to assess psychological distress and well-being, scoring from 0 (low) to 100 (high): “Have you been a very nervous person? Have you felt so down in the dumps that nothing could cheer you up? Have you felt calm and peaceful? Have you felt downhearted and blue? Have you been a happy person?”. The total mental health score was evaluated for reliability. Pearson correlations between the mental health score and its five component items were positive and larger than 0.3 and total Cronbach’s α was acceptable (>0.7), therefore confirming the reliability of the mental health score (Supplementary Table S3).
To assess physical health status, participants were asked about chronic conditions. A list of 22 choices was provided (multiple options possible) following the question: “Do you suffer from any of the following chronic diseases or long-term conditions? (tick all that apply)”.

3.3. Accessibility to Healthcare Services and Translation Services

Accessibility to healthcare services was measured with the following question “Need to use healthcare services the last 6 months”, respondents were given the following response options: “Needed and did not have access”, “Needed and had access” and “Did not need”. Two more indicators of accessibility to healthcare were also asked. Participants were asked if they perceived having worse access to healthcare services compared with local people with the response categories being “yes” and “no”. Moreover, participants were asked about the need of translation services during their medical visits with the response scale being never, few times, most times, and always.

3.4. Sociodemographic Characteristics

The following variables were included to further elaborate on the study sample characteristics: age, sex, country of origin, country of interview, number of years in the education system, having children, speaking language of the country of interview, and legal situation in the country.

3.5. Statistical Analysis

A descriptive analysis was performed for all available data. We conducted linear regression analysis to investigate variations in DMS scale (dependent variable) by country of interview, country of origin, and socioeconomics. Confounding factors were examined by applying three different linear models, starting from the univariable model (forward procedure). To investigate relevant changes in DMS score by legal situation in the country and health status, we selected a negative binomial model due to poor fit of the linear (Poisson distribution of the residuals) and Poisson (overdispersion) models. Finally, a multivariable logistic model was used to compare the odds of having access between migrants with different health status, country of interview, and kind of permission to stay in the country. All variables in each model were initially conceptualized by the perspective of clinical interest. Following this, the final models with the best fit were selected using Collet’s method, based on Akaike information criterion (AIC). All variables in the models were tested for collinearity. The level of statistical significance was defined as alpha = 0.05.

4. Results

The general demographics of the study participants are presented in Table 1. In Table 2 we observed a high DMS score in Greece, Italy, Cyprus, and Austria. The lowest score was reported in Spain. Migrants from Afghanistan tended to score higher in the DMS scale.
In Table 3, perceived discrimination in migrant women is presented by country of interview, country of origin, and age. We observed a high DMS score in migrant women in Cyprus and Greece. The lowest score was reported in Spain (same as Table 2). Migrant women from Iran scored significantly higher in the DMS scale compared with other countries of origin. Higher age indicated lower DMS score among migrant women.
Migrants in Greece reported needing and not having access to healthcare services less frequently, compared with other countries (Table 4). Almost all migrants in Bulgaria and the majority in Italy and Greece required a translator, while the lowest percentage of requiring a translator was observed in Spain.
Linear regression models are presented in Table 5 to observe associations in DMS scale by country of interview, country of origin, and socioeconomic status. In Model 3, the natural logarithmic transformation of the DMS score as the dependent variable was used due to poor fit of the standard linear regression model. Migrants in Italy and Austria scored higher in the DMS scale compared with Spain (Models 1–3). The DMS score was lower for older participants and for those with more years of education (Models 2,3). Migrants from Nigeria, Syria, Iraq, and other countries scored lower in the DMS scale compared with migrants from Iran. Age was significantly associated with DMS scores, with younger migrants scoring higher DMS (−0.006; 95% CI [−0.012, −0.001]).
Speaking the language of the country of interview was negatively associated with the DMS score in the univariate model (−0.093; 95% CI [−0.164, −0.021]) (not shown in the Table) and it was not statistically significant in the multivariate model. This could be explained perhaps due to the potential confounding effect of country of interview and country of origin (Supplementary Table S5).
DMS was transformed into an integer scale (e.g., from 1.257 to 1257) due to poor fit in the linear model and Poisson distribution of the residuals. We used negative binomial regression due to increased dispersion in the Poisson model (Table 6). Older migrants reported better treatment experience (Models 1,2). Migrants with better self-perceived mental health scored lower in the DMS scale (0.994; 95% CI [0.993, 0.996]).
In Model 2, we added an interaction term for country of interview to include other kinds of legal permission. Migrants with no other kind of residence permission in Greece had higher DMS scores compared with migrants with some kind of permission (1.384; 95% CI [1.189, 1.611]). Migrants with no kind of permission in Austria had lower DMS score by 24% compared with migrants who had some kind of permission in Austria (0.763; 95% CI [0.632, 0.922]).
Logistic regression analysis was performed to investigate the likelihood of having access to healthcare services (Table 7). Migrants in Greece were more likely to need and not have access to healthcare services compared with Spain (0.293; 95% CI [0.166, 0.516]). Female migrants had 60% higher odds of needing and not having access to healthcare services compared with males (1.613; 95% CI [1.183, 2.199]). Migrants with health problems (chronic problems from injury/accidents, gastrointestinal disease, diabetes, skin disease, headaches/migraines, and diseases related to bone and muscle) were more likely to needing and not having had access to healthcare services compared with healthy migrants. Migrants with chronic problems had the highest odds of needing and not having access to healthcare services compared with other health problems (3.292; 95% CI [1.585, 6.837]). Migrants with gastrointestinal disease or diabetes had higher odds ratios compared with migrants with skin diseases, headaches, or migraines, and diseases related to bone and muscle.

5. Discussion

To our knowledge, this is one of the first studies to assess the patient experience of the migrant population across 10 countries in Europe. The main finding from this study is that better self-reported mental health outcomes as measured by the SF-36 relevant mental health scale were associated with lower perceived discrimination in medical settings. Moreover, migrant women were more likely to not be able to access healthcare services when needed. Likewise, the same findings were reported for migrants suffering from chronic illnesses. Finally, older migrants reported higher feelings of health discrimination.
To our knowledge, this is the first study using the DMS questionnaire among a multi-cultural group of migrants in Europe. Two previous studies validated the DMS in two ethnic minorities samples in the United States [21,30]. Similar to these studies, we evaluated the reliability of the scale by calculating Cronbach’s α and Pearson correlations between the DMS score and its seven component items (r > 0.3). Although there is a large number of migrants arriving to Europe, no studies have been conducted assessing ethnic discrimination in healthcare services. For this reason, future studies should explore psychometric properties of this scale among specific migrant groups, different healthcare settings, and across Europe. In this regard, the results of this study contribute to healthcare systems efforts to assess and address healthcare discrimination in the migrant population [31].
Moreover, findings indicate that the migrant population reported higher levels of healthcare discrimination in Greece, Italy, Cyprus, and Austria and lower levels in Spain (p < 0.001). Additionally, migrants from Afghanistan tended to score higher in the DMS scale (p < 0.05). The lower level of healthcare discrimination reported in Spain may be explained by some characteristics of the study population because 186 (92.08%) participants were from South America and were Spanish speakers. Therefore, our results demonstrate a high relationship between the need for translators and the feelings of being discriminated when accessing healthcare services, making the role of translation services highly relevant for the quality of care for migrant populations. Recent evidence has also suggested additional benefits at system and professional levels, such as cost-saving for the healthcare system, and reducing difficulties during the appointments [8].
The results of our study also show significant differences of perceived ethnic discrimination in the healthcare system especially regarding age. Here, younger migrants reported worse treatment experience in our sample from 7 EU countries. A recent systematic literature review by Robards and colleagues [32] found that for marginalized young people, the decision to access health services is affected by previous bad experiences during which they felt treated differently and with disrespect by healthcare professionals. In the same article, the study population highlighted different actions to be considered in the delivery of healthcare for migrant young groups such as culturally appropriate services, cultural sensitivity of staff, and the use of interpreters.
Our study also highlighted the vulnerability of migrant women regarding both the lack of access to healthcare services and the perception of higher discrimination within healthcare services. In addition to the common barriers of migrants in accessing healthcare [33], the results of this study could be explained by differences in care seeking behaviour of this group [34] or in health services-related factors [35]. For this reason, health interventions aiming to mitigate gender-driven inequalities in accessing quality healthcare have to be put in practice in the European context [36].
The management of chronic diseases in European migrants and refugees has been identified as a priority for health service provision [27]. However, the results of this study show that migrant chronic patients reported the highest odds for needing but not having access to healthcare services. This could be explained by the higher contact frequency that chronic patients have with the healthcare systems, which may increase their health literacy and empowerment, making them more aware of their needs, their rights, and their expectations from the system. Hence, this group could be more perceptive to discrimination during their medical visits. Moreover, more frequent health visits could increase the likelihood to be exposed to experiences of healthcare discrimination towards them. Another reason may be that frequent visits expose general inefficiencies of the healthcare system and provision (such as lack of personnel or equipment) that are more attenuated in these groups.
Additionally, in our study differences were found concerning healthcare discrimination towards undocumented migrants depending on the host country. In this regard, undocumented migrants in Greece (1.384; 95% CI [1.189, 1.611]) had higher DMS scores compared with migrants with legal status. However, the opposite trend was reported in Austria, where undocumented migrants had a lower DMS score by 24% compared with migrants with legal status (0.763; 95% CI [0.632, 0.922]). In this regard, Robertshaw and colleagues [37] found that immigration status and legislative policy are a challenge for the provision of healthcare by creating or reinforcing vulnerability of marginalized groups [38]. However, the results of our study could be interpreted in the light of the results of Dauvrin and colleagues [39] who also reported insufficiencies in the actual delivery of care for undocumented migrants despite the variations in healthcare entitlement related to the immigration status across Europe, suggesting that even in countries with “minimum rights”, health professionals may consider treating undocumented migrants more important than abiding by law (“pragmatic health professional”). For that reason, our results might be outlining a complex interplay of different factors that might be worsening the provision of healthcare for migrant patients in the host country beyond their legal status, as this is not a general result and differences appear between countries in our study.
Although specific migrant groups have reported experiencing discrimination in healthcare, ethnic discrimination, and translation services, these are still under-researched topics in Europe. In this regard, our results could be also interpreted from a structural and organizational point of view for healthcare delivery. Indeed, King and colleagues [40] argue how (1) compulsory assigned residency, (2) resources (including language skills), and (3) freedom of movement (related to documented/undocumented status) could be consolidating heavy and stable forms of devaluation, reification, and stigma, denying access to healthcare for certain groups with negative consequences on the health of migrants.
This study shows the experiences of migrants and refugees between 2018 and 2019. The inequalities and vulnerabilities shown in this study may have been further amplified by the COVID-19 pandemic, which was not captured in our study but should be briefly discussed. Even though healthcare services face an unprecedented demand, groups such as undocumented migrants and migrant women are still facing barriers to access appropriate quality care that contributes to poorer health outcomes [41,42]. Moreover, epidemiological data shows that COVID-19 disproportionately affects patients who have chronic conditions and underlying comorbidities [43]. This is in line with the report of Fiorini and colleagues [38], indicating that due to limited access to appropriate care and quality of care, migrant and refugee populations, and specifically some groups such as undocumented female or chronic patients, might experience higher morbidity and mortality during the COVID-19 pandemic,
To the best of our knowledge, this is the first study assessing the migrant patient experience with a multicultural sampling in seven EU Member states regarding healthcare discrimination and the need for translation services. Another strength includes the range of data and populations investigated allowing for comparative and intersectional analyses across many dimensions of discrimination in healthcare services. On the other hand, limitations include relying on self-reported information which may introduce reporting bias. Furthermore, study participants varied greatly in country of origin, duration of stay in country of interview, and integration phase. The use of interpreters may have introduced additional information bias, and cultural barriers in female representation in the survey for some countries (such as Afghanistan) may have biased their responses. Moreover, considering the convenience sampling method used, respondents may have given socially desirable answers, leading to underestimation of healthcare discrimination. Finally, as this is a cross-sectional study which relies on a non-random sample, causal relationships cannot be established.

6. Conclusions

In conclusion, migrant and refugee patients reported unequal access to healthcare and perceived discrimination when they did access services. We found that country of origin and not speaking the host country language were associated with increased discrimination in healthcare provision. Language communication support and cultural mediation in healthcare services will facilitate healthcare access. Moreover, younger migrants, migrant women, and chronic migrant patients reported experiencing higher discrimination within the healthcare services. The need for health interventions to address the unequal access of these groups to appropriate care and quality of care is now further stressed in the light of the COVID-19 pandemic that has amplified shortcomings in the provision of care to these groups. Finally, it is important to conduct cohort studies monitoring healthcare access and perceived discrimination towards different migrant groups as part of the quality control of healthcare provision.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/ijerph18157901/s1, Table S1. DMS questionnaire. Each question of the DMS scale scores from 1: Never true to 5: Always true. Table S2. Pearson’s correlation and Cronbach’s diagonal results for each question with total DMS scale. Migrants and refugees from Germany, France and Malta were excluded from the analysis. Table S3. Pearson’s correlation and Cronbach’s diagonal results for each question with total Mental Health Score. Migrants and refugees from Germany, France and Malta were excluded from the analysis. Table S4. Characteristics of the total sample of migrants and refugees (N = 1407). Table S5. Communication skills by country of origin and country of interview. Migrants without DMS score (missing values) were excluded from the analysis.

Author Contributions

Conceptualization, A.G.-S.; methodology, P.K., E.R., A.G.-S.; formal analysis, K.K.; resources, A.L.; data curation, K.K.; writing—original draft preparation, A.G.-S.; writing—review and editing, P.K., E.R., K.K.; supervision, A.L.; project administration, P.K., A.G.-S., E.R.; funding acquisition, A.L. All authors have read and agreed to the published version of the manuscript.

Funding

The survey forms a part of the tasks described in the EU project “Strengthen Community Based Care to minimize health inequalities and improve the integration of vulnerable migrants and refugees into local communities”. The project received partial funding from the European Union’s Health programme (2014–2020) Consumers, Health, Agriculture and Food Executive Agency (CHAFEA), Grant Agreement 738186 Acronym Mig-HealthCare. Alejandro Gil-Salmerón was funded by the Training programme for Academic Staff (FPU) fellowship, from the Ministry of Education and Vocational Training, Spanish Government (Ref.: FPU15/05251).

Institutional Review Board Statement

Approved in 2018 by the Ethical Committee of both the University of Valencia and the National and Kapodistrian University of Athens, Medical School.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Limited dataset used for this analysis is available upon reasonable request from the Mig-HealthCare consortium.

Acknowledgments

The authors would like to thank the Mig-HealthCare partner consortium (www.mighealthcare.eu, accessed on 9 July 2021) for the fruitful collaboration in the project implementation. Project partners are: Austria Verein Multikulturell, Bulgaria National Center of Infectious and Parasitic Diseases, Cyprus Centre for Advancement of Research and Development in Educational Technology Ltd. Cardet, Germany Ethno-Medizinisches Zentrum EV, Greece Institute of Preventive Medicine, Environmental and Occupational Health, Prolepsis, National and Kapodistrian University of Athens, Perifereia Stereas Elladas, Kentriki Enosi Dimon kai Koinotiton Ellados, Ministry of Health, France Ecole des Hautes Etudes en Sante Publique, Italy Oxfam Italia Onlus, Malta Koperazzjoni Internazzjonali Malta (Kopin) Association, Spain Universitat de Valencia, Sweden Uppsala Universitet.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Future Migration Trends; Migration Data Portal. Available online: http://migrationdataportal.org/themes/future-migration-trends (accessed on 23 November 2020).
  2. Lebano, A.; Hamed, S.; Bradby, H.; Gil-Salmerón, A.; Durá-Ferrandis, E.; Garcés-Ferrer, J.; Azzedine, F.; Riza, E.; Karnaki, P.; Zota, D.; et al. Migrants’ and refugees’ health status and healthcare in Europe: A scoping literature review. BMC Public Health 2020, 20, 1039. [Google Scholar] [CrossRef] [PubMed]
  3. Chiarenza, A.; Dauvrin, M.; Chiesa, V.; Baatout, S.; Verrept, H. Supporting access to healthcare for refugees and migrants in European countries under particular migratory pressure. BMC Health Serv. Res. 2019, 19, 513. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Bradby, H.; Humphris, R.; Newall, D.; Phillimore, J.; Health Evidence Network; World Health Organization; Regional Office for Europe. Public Health Aspects of Migrant Health: A Review of the Evidence on Health Status for Refugees and Asylum Seekers in the European Region. 2015. Available online: http://www.euro.who.int/__data/assets/pdf_file/0004/289246/WHO-HEN-Report-A5-2-Refugees_FINAL.pdf?ua=1 (accessed on 1 December 2020).
  5. Marques, T.V.G.A. Refugees and migrants struggle to obtain health care in Europe. CMAJ Can. Med Assoc. J. 2012, 184, E531–E532. [Google Scholar] [CrossRef] [Green Version]
  6. Björngren Cuadra, C. Policies on Health Care for Undocumented Migrants in EU27; Country Report. 2010. Available online: https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.460.3995&rep=rep1&type=pdf (accessed on 18 July 2021).
  7. Winters, M.; Rechel, B.; de Jong, L.; Pavlova, M. A systematic review on the use of healthcare services by undocumented migrants in Europe. BMC Health Serv. Res. 2018, 18, 30. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  8. Jaeger, F.N.; Pellaud, N.; Laville, B.; Klauser, P. The migration-related language barrier and professional interpreter use in primary health care in Switzerland. BMC Health Serv. Res. 2019, 19, 429. [Google Scholar] [CrossRef] [PubMed]
  9. Napier, A.D.; Ancarno, C.; Butler, B.; Calabrese, J.; Chater, A.; Chatterjee, H.; Guesnet, F.; Horne, R.; Jacyna, S.; Jadhav, S.; et al. Culture and health. Lancet 2014, 384, 1607–1639. [Google Scholar] [CrossRef]
  10. Flores, G. The impact of medical interpreter services on the quality of health care: A systematic review. Med. Care Res. Rev. MCRR 2005, 62, 255–299. [Google Scholar] [CrossRef] [PubMed]
  11. Flores, G.; Abreu, M.; Barone, C.P.; Bachur, R.; Lin, H. Errors of medical interpretation and their potential clinical consequences: A comparison of professional versus ad hoc versus no interpreters. Ann. Emerg. Med. 2012, 60, 545–553. [Google Scholar] [CrossRef]
  12. Derose, K.P.; Escarce, J.J.; Lurie, N. Immigrants and health care: Sources of vulnerability. Health Aff. 2007, 26, 1258–1268. [Google Scholar] [CrossRef] [PubMed]
  13. van Wieringen, J.C.M.; Harmsen, J.A.M.; Bruijnzeels, M.A. Intercultural communication in general practice. Eur. J. Public Health 2002, 12, 63–68. [Google Scholar] [CrossRef] [Green Version]
  14. Priebe, S.; Sandhu, S.; Dias, S.; Gaddini, A.; Greacen, T.; Ioannidis, E.; Kluge, U.; Krasnik, A.; Lamkaddem, M.; Lorant, V.; et al. Good practice in health care for migrants: Views and experiences of care professionals in 16 European countries. BMC Public Health 2011, 11, 187. [Google Scholar] [CrossRef]
  15. Lindholm, M.; Hargraves, J.L.; Ferguson, W.J.; Reed, G. Professional language interpretation and inpatient length of stay and readmission rates. J. Gen. Intern. Med. 2012, 27, 1294–1299. [Google Scholar] [CrossRef] [Green Version]
  16. Hanssens, L.G.M.; Detollenaere, J.D.J.; Pottelberge, A.V.; Baert, S.; Willems, S.J.T. Perceived discrimination In Primary Healthcare in Europe: Evidence from the cross-sectional QUALICOPC study. Health Soc. Care Community 2017, 25, 641–651. [Google Scholar] [CrossRef] [PubMed]
  17. Benjamins, M.R.; Middleton, M. Perceived discrimination in medical settings and perceived quality of care: A population-based study in Chicago. PLoS ONE 2019, 14, e0215976. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Armstrong, K.; Putt, M.; Halbert, C.H.; Grande, D.; Schwartz, J.S.; Liao, K.; Marcus, N.; Demeter, M.B.; Shea, J.A. Prior experiences of racial discrimination and racial differences in health care system distrust. Med. Care 2013, 51, 144–150. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  19. Hong, H.C.; Lee, H.; Collins, E.G.; Park, C.; Quinn, L.; Ferrans, C.E. Factors affecting trust in healthcare among middle-aged to older Korean American women. BMC Women’s Health 2018, 18, 109. [Google Scholar] [CrossRef]
  20. Bird, S.T.; Bogart, L.M. Perceived race-based and socioeconomic status (SES)-based discrimination in interactions with health care providers. Ethn. Dis. 2001, 11, 554–563. [Google Scholar] [PubMed]
  21. Peek, M.E.; Wagner, J.; Tang, H.; Baker, D.C.; Chin, M.H. Self-reported racial discrimination in health care and diabetes outcomes. Med. Care 2011, 49, 618–625. [Google Scholar] [CrossRef] [Green Version]
  22. Gonzales, K.L.; Lambert, W.E.; Fu, R.; Jacob, M.; Harding, A.K. Perceived racial discrimination in health care, completion of standard diabetes services, and diabetes control among a sample of American Indian women. Diabetes Educ. 2014, 40, 747–755. [Google Scholar] [CrossRef]
  23. Gonzales, K.L.; Harding, A.K.; Lambert, W.E.; Fu, R.; Henderson, W.G. Perceived Experiences of Discrimination in Health Care: A Barrier for Cancer Screening Among American Indian Women with Type 2 Diabetes. Women’s Health Issues Off. Publ. Jacobs Inst. Women’s Health 2013, 23, e61–e67. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  24. Cuffee, Y.L.; Hargraves, J.L.; Rosal, M.; Briesacher, B.A.; Schoenthaler, A.; Person, S.; Hullett, S.; Allison, J. Reported Racial Discrimination, Trust in Physicians, and Medication Adherence Among Inner-City African Americans with Hypertension. Am. J. Public Health 2013, 103, e55–e62. [Google Scholar] [CrossRef]
  25. Home—Mig Healthcare. Available online: https://www.mighealthcare.eu/index.php (accessed on 12 April 2021).
  26. Rechel, B.; Mladovsky, P.; Ingleby, D.; Mackenbach, J.P.; McKee, M. Migration and health in an increasingly diverse Europe. Lancet 2013, 381, 1235–1245. [Google Scholar] [CrossRef]
  27. Riza, E.; Karnaki, P.; Gil-Salmerón, A.; Zota, K.; Ho, M.; Petropoulou, M.; Katsas, K.; Garcés-Ferrer, J.; Linos, A. Determinants of Refugee and Migrant Health Status in 10 European Countries: The Mig-HealthCare Project. Int. J. Environ. Res. Public Health 2020, 17, 6353. [Google Scholar] [CrossRef]
  28. Peek, M.E.; Nunez-Smith, M.; Drum, M.; Lewis, T.T. Adapting the Everyday Discrimination Scale to Medical Settings: Reliability and Validity Testing in a Sample of African American Patients. Ethn. Dis. 2011, 21, 502–509. [Google Scholar] [PubMed]
  29. Ware, J.E.; Snow, K.K.; Kosinski, M.; Gandek, B. SF-36 Health Survey: Manual & Interpretation Guide; LINCOLN, QualityMetric Incorporated: Lincoln, RI, USA, 2000. [Google Scholar]
  30. López-Cevallos, D.F.; Harvey, S.M. Psychometric Properties of a Healthcare Discrimination Scale Among Young-Adult Latinos. J. Racial Ethn. Health Disparities 2019, 6, 618–624. [Google Scholar] [CrossRef] [PubMed]
  31. Shavers, V.L.; Fagan, P.; Jones, D.; Klein, W.M.P.; Boyington, J.; Moten, C.; Rorie, E. The state of research on racial/ethnic discrimination in the receipt of health care. Am. J. Public Health 2012, 102, 953–966. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  32. Robards, F.; Kang, M.; Usherwood, T.; Sanci, L. How marginalized young people access, engage with, and navigate health-care systems in the digital age: Systematic review. J. Adolesc. Health 2018, 62, 365–381. [Google Scholar] [CrossRef]
  33. Graetz, V.; Rechel, B.; Groot, W.; Norredam, M.; Pavlova, M. Utilization of health care services by migrants in Europe—A systematic literature review. Br. Med. Bull. 2017, 121, 5–18. [Google Scholar] [CrossRef] [PubMed]
  34. Redwood-Campbell, L.; Fowler, N.; Laryea, S.; Howard, M.; Kaczorowski, J. «Before you teach me, I cannot know»: Immigrant women’s barriers and enablers with regard to cervical cancer screening among different ethnolinguistic groups in Canada. Can. J. Public Health Rev. Can. Sante Publique 2011, 102, 230–234. [Google Scholar] [CrossRef]
  35. Marques, P.; Nunes, M.; da Luz Antunes, M.; Heleno, B.; Dias, S. Factors associated with cervical cancer screening participation among migrant women in Europe: A scoping review. Int. J. Equity Health 2020, 19, 160. [Google Scholar] [CrossRef]
  36. Gkiouleka, A.; Huijts, T. Intersectional migration-related health inequalities in Europe: Exploring the role of migrant generation, occupational status & gender. Soc. Sci. Med. 2020, 267, 113218. [Google Scholar] [CrossRef] [PubMed]
  37. Robertshaw, L.; Dhesi, S.; Jones, L.L. Challenges and facilitators for health professionals providing primary healthcare for refugees and asylum seekers in high-income countries: A systematic review and thematic synthesis of qualitative research. BMJ Open 2017, 7, e015981. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  38. Ahmad, A.; Chung, R.; Eckenwiler, L.; Ganguli-Mitra, A.; Hunt, M.; Richards, R.; Saghai, Y.; Schwartz, L.; Scully, J.L.; Wild, V. What does it mean to be made vulnerable in the era of COVID-19? Lancet 2020, 395, 1481–1482. [Google Scholar] [CrossRef]
  39. Dauvrin, M.; Lorant, V.; Sandhu, S.; Devillé, W.; Dia, H.; Dias, S.; Gaddini, A.; Ioannidis, E.; Jensen, N.K.; Kluge, U.; et al. Health care for irregular migrants: Pragmatism across Europe. A qualitative study. BMC Res. Notes 2012, 5, 99. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  40. King, D.; Le Galès, P.; Vitale, T. Assimilation, Security, and Borders in the Member States. In Reconfiguring European States in Crisis; King, D., le Galès, P., Eds.; Oxford University Press: Oxford, UK, 2017; pp. 428–450. ISBN 9780198793373. [Google Scholar]
  41. Fiorini, G.; Rigamonti, A.E.; Galanopoulos, C.; Adamoli, M.; Ciriaco, E.; Franchi, M.; Genovese, E.; Corrao, G.; Cella, S.G. Undocumented migrants during the Covid-19 pandemic: Social conditions, clinical features and pharmacological treatment. J. Public Health Res. 2020, 9, 4. [Google Scholar] [CrossRef]
  42. Germain, S.; Yong, A. COVID-19 Highlighting Inequalities in Access to Healthcare in England: A Case Study of Ethnic Minority and Migrant Women. Fem. Leg. Stud. 2020, 28, 301–310. [Google Scholar] [CrossRef]
  43. Gudbjartsson, D.F.; Helgason, A.; Jonsson, H.; Magnusson, O.T.; Melsted, P.; Norddahl, G.L.; Saemundsdottir, J.; Sigurdsson, A.; Sulem, P.; Agustsdottir, A.B.; et al. Spread of SARS-CoV-2 in the Icelandic Population. N. Engl. J. Med. 2020. [Google Scholar] [CrossRef]
Table 1. Characteristics of migrants and refugees, except those from Germany, France, and Malta (N = 1294).
Table 1. Characteristics of migrants and refugees, except those from Germany, France, and Malta (N = 1294).
N (%)
Country of interview
 Austria126 (9.74)
 Bulgaria226 (17.47)
 Cyprus110 (8.50)
 Greece255 (19.71)
 Italy271 (20.94)
 Spain202 (15.61)
 Sweden104 (8.04)
Gender (males)816 (63.26)
Country of origin
 Afghanistan187 (14.55)
 Iran48 (3.74)
 Iraq122 (9.49)
 Nigeria115 (8.95)
 Syria281 (21.87)
 Other532 (41.40)
Having at least one child535 (50.71)
Asylum (yes)320 (27.80)
Other kind of permission (yes)562 (45.03)
Speaking country of interview language (yes)428 (38.73)
Comorbidity
Not having a disease or chronic condition653 (50.46)
Having one disease or chronic condition311 (24.03)
Having at least two diseases or chronic conditions330 (25.5)
Mean ± standard deviation
Age (years)32 ± 11
Education (years)8.9 ± 5.1
Table 2. Discrimination Scale to Medical Settings by country of interview, country of origin, and communication skills. Migrants and refugees from Germany, France, and Malta were excluded from the analysis.
Table 2. Discrimination Scale to Medical Settings by country of interview, country of origin, and communication skills. Migrants and refugees from Germany, France, and Malta were excluded from the analysis.
NMedianInterquartile Range
Country of interview ***
 Austria911.28612.286
 Bulgaria93112
 Cyprus631.28612.286
 Greece1781.28612.571
 Italy1151.28612.286
 Spain160111.286
 Sweden481.07111.786
Country of origin **
 Afghanistan1101.57113
 Iran29213
 Iraq721.07112.071
 Nigeria541.14312
 Syria169112
 Other309111.857
Ability to communicate in host country language *
Not speaking country of interview language104112.143
Speaking country of interview language644111.714
*** p < 0.001, ** p < 0.05, * p < 0.1.
Table 3. Discrimination Scale to Medical Settings by country of interview, country of origin, and age in migrant women. Migrants and refugees from Germany, France, and Malta were excluded from the analysis.
Table 3. Discrimination Scale to Medical Settings by country of interview, country of origin, and age in migrant women. Migrants and refugees from Germany, France, and Malta were excluded from the analysis.
NMedianInterquartile Range
Country of interview 
 Austria451.28612
 Bulgaria47112
 Cyprus311.71412.429
 Greece1011.57112.857
 Italy941.28612
 Spain87111.286
 Sweden201.35712.5
Country of origin 
 Afghanistan751.57113
 Iran192.71413.143
 Iraq271.28612.429
 Nigeria30111.286
 Syria761.07112.429
 Other196111.929
Correlation coefficient
Age (years) 301−0.1937
All p < 0.001.
Table 4. Access to healthcare services and requirement of a translator by country of interview. Migrants and refugees from Germany, France, and Malta were excluded from the analysis.
Table 4. Access to healthcare services and requirement of a translator by country of interview. Migrants and refugees from Germany, France, and Malta were excluded from the analysis.
AustriaBulgariaCyprusGreeceItalySpainSweden
Need to use healthcare services the last 6 months (%)
Needed and did not have access59.2054.2641.4127.8348.2445.7739.42
Needed and had access11.200.019.1936.521.9627.3619.23
Did not need29.6045.7439.3935.6549.826.8741.35
Worse access to healthcare services compared with local people (% yes) 26.0932.5725.5642.5215.0519.920.19
Need for a translator
Never33.050.9132.6312.959.4774.7332.67
Few times38.141.8218.9511.1632.5816.6715.84
Most times16.1025.9124.2122.3224.622.6918.81
Always12.7171.3624.2153.5733.335.9132.67
All p < 0.001.
Table 5. Multivariable linear regression models investigating the impact of country of interview, country of origin, and socioeconomic characteristics on Discrimination Scale to Medical Settings (dependent variable) in migrants and refugees. Migrants and refugees from Germany, France, and Malta were excluded from the analysis.
Table 5. Multivariable linear regression models investigating the impact of country of interview, country of origin, and socioeconomic characteristics on Discrimination Scale to Medical Settings (dependent variable) in migrants and refugees. Migrants and refugees from Germany, France, and Malta were excluded from the analysis.
Model 1 (N = 748)Model 2 (N = 569)Model 3 (N = 497)
Estimate95% CIEstimate95% CIEstimate95% CI
Austria as country of interview0.318 **0.0800.5560.239 *−0.0340.5110.167 **0.0120.323
Bulgaria as country of interview0.261 *0.0250.4970.092−0.2220.4060.031−0.1730.234
Cyprus as country of interview0.384 **0.1150.6540.185−0.1560.5270.110−0.0900.309
Greece as country of interview0.588 ***0.3910.7860.276 *−0.0300.5810.140−0.0470.328
Italy as country of interview0.369 **0.1470.5900.347 **0.0440.6510.224 **0.0620.386
Sweden as country of interview0.183−0.1160.4810.211−0.1200.5420.136−0.0400.311
Age (years) −0.006 **−0.012−0.001−0.003 *−0.007−0.0004
Females 0.009−0.1380.156−0.013−0.0940.068
Education (years) −0.019 **−0.034−0.003−0.009 **−0.018−0.001
Not having children −0.002−0.1460.151−0.017−0.0960.063
Afghanistan as country of origin −0.176−0.5660.214−0.112−0.3840.160
Iraq as country of origin −0.418 **−0.836−0.0004−0.158−0.4040.089
Nigeria as country of origin −0.606 **−1.100−0.111−0.276 *−0.5540.003
Syria as country of origin −0.608 **−0.987−0.229−0.253 **−0.479−0.027
Other as country of origin −0.427 **−0.830−0.024−0.170−0.4080.068
Speaking country of interview language −0.006−0.1030.115
Compared with Spain (reference group for country of interview) and with Iran (reference group for country of origin). *** p < 0.001, ** p < 0.05, * p < 0.1. The natural logarithmic transformation for DMS score (dependent variable) was used in Model 3 due to poor fit of the standard linear regression model.
Table 6. Negative binomial regression investigating the impact of country of interview, permission to stay, and health status on Discrimination Scale to Medical Settings (dependent variable) in migrants and refugees. Estimates are presented as incidence rate ratio. Migrants and refugees from Germany, France, and Malta were excluded from the analysis.
Table 6. Negative binomial regression investigating the impact of country of interview, permission to stay, and health status on Discrimination Scale to Medical Settings (dependent variable) in migrants and refugees. Estimates are presented as incidence rate ratio. Migrants and refugees from Germany, France, and Malta were excluded from the analysis.
Model 1 (N = 614)Model 2 (N = 614)
IRR95% CIIRR95% CI
Not having asylum1.0520.9711.1391.0410.9571.132
Mental Health Score 0.994 ***0.9920.9950.994 ***0.9930.996
Age (years)0.995 **0.9920.9980.996 **0.9930.999
Females0.9570.8891.0300.9780.9101.052
Having one disease or chronic condition (morbidity)1.0000.8891.0300.9810.8931.078
Having at least two diseases or chronic conditions (comorbidity)1.093 *0.9981.1961.0410.9491.142
Not having other kind of permission in Austria 0.763 **0.6320.922
Not having other kind of permission in Bulgaria 1.0190.8861.173
Not having other kind of permission in Cyprus 1.1650.9061.498
Not having other kind of permission in Greece 1.384 ***1.1891.611
Not having other kind of permission in Italy 1.247 *0.9931.173
Not having other kind of permission in Spain 1.0190.8861.173
Not having other kind of permission in Sweden 0.8820.6831.138
Not having other kind of permission in total1.0470.9741.126
*** p < 0.001, ** p < 0.05, * p < 0.1. Model 1 is adjusted for country of interview. In Model 2, interaction terms for having another kind of asylum with country of interview were inserted. Not having another kind of permission in Bulgaria was omitted because all migrants in Bulgaria did not have other kinds of permission.
Table 7. Multivariable logistic regression investigating the impact of country of interview, need of a translator, permission to stay, and health status on access to healthcare services (dependent variable) in migrants and refugees. Migrants and refugees from Germany, France, and Malta were excluded from the analysis.
Table 7. Multivariable logistic regression investigating the impact of country of interview, need of a translator, permission to stay, and health status on access to healthcare services (dependent variable) in migrants and refugees. Migrants and refugees from Germany, France, and Malta were excluded from the analysis.
N = 899Odds Ratio95% CI
Austria as country of interview1.3780.8042.362
Bulgaria as country of interview1.3010.7072.393
Cyprus as country of interview0.537 *0.2781.039
Greece as country of interview0.293 ***0.1660.516
Italy as country of interview0.9760.5761.653
Sweden as country of interview0.597 *0.3401.047
Age (years)0.9980.9841.013
Females1.613 **1.1832.199
Need for a translator (yes)1.2090.8261.770
Not having asylum0.7980.5731.112
Not having other kind of permission0.8200.5921.137
Mental Health Score1.0000.9931.007
Chronic problems from injury/accidents3.292 ***1.5856.837
Gastrointestinal disease2.917 ***1.5565.468
Diabetes2.912 **1.3706.190
Skin disease1.912 **1.0523.475
Disease related to bone and muscle1.907 **1.0303.531
Headaches/migraines1.643 **1.0842.489
Compared with migrants in Spain as reference group for country of interview *** p < 0.001, ** p < 0.05, * p < 0.1.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Gil-Salmerón, A.; Katsas, K.; Riza, E.; Karnaki, P.; Linos, A. Access to Healthcare for Migrant Patients in Europe: Healthcare Discrimination and Translation Services. Int. J. Environ. Res. Public Health 2021, 18, 7901. https://doi.org/10.3390/ijerph18157901

AMA Style

Gil-Salmerón A, Katsas K, Riza E, Karnaki P, Linos A. Access to Healthcare for Migrant Patients in Europe: Healthcare Discrimination and Translation Services. International Journal of Environmental Research and Public Health. 2021; 18(15):7901. https://doi.org/10.3390/ijerph18157901

Chicago/Turabian Style

Gil-Salmerón, Alejandro, Konstantinos Katsas, Elena Riza, Pania Karnaki, and Athena Linos. 2021. "Access to Healthcare for Migrant Patients in Europe: Healthcare Discrimination and Translation Services" International Journal of Environmental Research and Public Health 18, no. 15: 7901. https://doi.org/10.3390/ijerph18157901

APA Style

Gil-Salmerón, A., Katsas, K., Riza, E., Karnaki, P., & Linos, A. (2021). Access to Healthcare for Migrant Patients in Europe: Healthcare Discrimination and Translation Services. International Journal of Environmental Research and Public Health, 18(15), 7901. https://doi.org/10.3390/ijerph18157901

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop