Intra-Rater Test-Retest Reliability of a Modified Child Functioning Module, Self-Report Version
Abstract
:1. Introduction
Difficulties in Measuring Disabilities
2. Materials and Methods
2.1. Procedures
2.2. Measures in This Study
2.3. Analyses
3. Results
3.1. Descriptive Results
3.2. Test-Retest Results
4. Discussion
4.1. Reliability as a Scale
4.2. Reliability as Two Types of Cut-Offs
4.3. Limitations
5. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- The Salamanca Statement and Framework for Action on Special Needs Education. In Proceedings of the World Conference on Special Needs Education: Access and Quality, Salamanca, Spain, 7–10 June 1994.
- Implementing Rules and Regulations of the Enhanced Basic Education Act of 2013. Available online: https://www.officialgazette.gov.ph/2013/09/04/irr-republic-act-no-10533/ (accessed on 22 September 2020).
- Official Statistics of Finland. Available online: https://www.stat.fi/til/erop/2018/erop_2018_2019-06-19_tie_001_en.html (accessed on 22 September 2020).
- Pesonen, H.; Itkonen, T.; Jahnukainen, M.; Kontu, E.; Kokko, T.; Ojala, T.; Pirttimaa, R. The Implementation of New Special Education Legislation in Finland. Educ. Policy 2015, 29, 162–178. [Google Scholar]
- Tardi, R.; Njelesani, J. Disability and the Post-2015 Development Agenda. Disabil. Rehabil. 2015, 37, 1496–1500. [Google Scholar] [PubMed]
- Rathmann, K.; Vockert, T.; Bilz, L.; Gebhardt, M.; Hurrelmann, K. Self-Rated Health and Wellbeing among School-Aged Children with and without Special Educational Needs: Differences between Mainstream and Special Schools. Res. Dev. Disabil. 2018, 81, 134–142. [Google Scholar] [PubMed]
- Cappa, C.; Petrowski, N.; Njelesani, J. Navigating the Landscape of Child Disability Measurement: A Review of Available Data Collection Instruments. ALTER 2015, 9, 317–330. [Google Scholar]
- World Report on Disability. Available online: https://datacatalog.worldbank.org/dataset/world-report-disability (accessed on 22 September 2020).
- WHO. International Classification of Functioning, Disability and Health (ICF); World Health Organization: Geneva, Switzerland, 2001. [Google Scholar]
- WHO. International Classification of Functioning, Disability and Health: Children and Youth Version: ICF-CY; World Health Organization: Geneva, Switzerland, 2007; p. 322. [Google Scholar]
- Mont, D. Measuring Health and Disability. Lancet 2007, 369, 1658–1663. [Google Scholar] [PubMed]
- Monitoring Child Disability in Developing Countries. Available online: https://asksource.info/resources/monitoring-child-disability-developing-countries-results-multiple-indicator-cluster (accessed on 22 September 2020).
- Cappa, C.; Mont, D.; Loeb, M.; Misunas, C.; Madans, J.; Comic, T.; de Castro, F. The Development and Testing of a Module on Child Functioning for Identifying Children with Disabilities on Surveys. III: Field Testing. Disabil. Health J. 2018, 11, 510–518. [Google Scholar]
- Loeb, M.; Mont, D.; Cappa, C.; De Palma, E.; Madans, J.; Crialesi, R. The Development and Testing of a Module on Child Functioning for Identifying Children with Disabilities on Surveys. I: Background. Disabil. Health J. 2018, 11, 495–501. [Google Scholar]
- Mactaggart, I.; Cappa, C.; Kuper, H.; Loeb, M.; Polack, S. Field Testing a Draft Version of the UNICEF/Washington Group Module on Child Functioning and Disability. Background, Methodology and Preliminary Findings from Cameroon and India. ALTER 2016, 10, 345–360. [Google Scholar]
- Massey, M. The Development and Testing of a Module on Child Functioning for Identifying Children with Disabilities on Surveys. II: Question Development and Pretesting. Disabil. Health J. 2018, 11, 502–509. [Google Scholar]
- Zia, N.; Loeb, M.; Kajungu, D.; Galiwango, E.; Diener-West, M.; Wegener, S.; Pariyo, G.; Hyder, A.A.; Bachani, A.M. Adaptation and Validation of UNICEF/Washington Group Child Functioning Module at the Iganga-Mayuge Health and Demographic Surveillance Site in Uganda. BMC Public Health 2020, 20, 1334. [Google Scholar]
- Andersen, C.S.; Dolva, A. Children’s Perspective on their Right to Participate in Decision-Making According to the United Nations Convention on the Rights of the Child Article 12. Phys. Occup. Ther. Pediatr. 2015, 35, 218–230. [Google Scholar] [CrossRef]
- Marker, A.M.; Steele, R.G.; Noser, A.E. Physical Activity and Health-Related Quality of Life in Children and Adolescents: A Systematic Review and Meta-Analysis. Health Psychol. 2018, 37, 893–903. [Google Scholar] [CrossRef] [PubMed]
- Simeoni, M.; Schmidt, S.; Muehlan, H.; Debensason, D.; Bullinger, M. Field Testing of a European Quality of Life Instrument for Children and Adolescents with Chronic Conditions: The 37-Item DISABKIDS Chronic Generic Module. Qual. Life Res. 2007, 16, 881–893. [Google Scholar] [CrossRef] [PubMed]
- Cohen, J.; Cohen, P. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences; Lawrence Erlbaum, Routledge: Abingdon, UK, 1975; ISBN 9780203774441. [Google Scholar]
- Lenzner, T.; Kaczmirek, L.; Lenzner, A. Cognitive Burden of Survey Questions and Response Times: A Psycholinguistic Experiment. Appl. Cogn. Psychol. 2010, 24, 1003–1020. [Google Scholar] [CrossRef] [Green Version]
- Sallis, J.F.; Saelens, B.E. Assessment of Physical Activity by Self-Report: Status, Limitations, and Future Directions. Res. Q. Exerc. Sport 2000, 71, 1–14. [Google Scholar] [CrossRef]
- Donner, A.; Eliasziw, M. Sample Size Requirements for Reliability Studies. Statist. Med. 1987, 6, 441–448. [Google Scholar] [CrossRef]
- Kokko, S.; Martin, L.; Villberg, J.; Ng, K.W.; Mehtälä, A. Itsearvioitu liikunta-aktiivisuus, ruutuaika ja sosiaalinen media sekä liikkumisen seurantalaiteet ja -sovellukset [Self-reported physical activity, screen-time, social media and physical activity tracking devices]. In Lasten Ja Nuorten Liikuntakäyttäytyminen Suomessa: LIITU-Tutkimuksen Tuloksia 2018 [Finnish Children and Adolescents Physical Activity Behaviours: Results of the F-SPA Study 2018]; Kokko, S., Martin, L., Eds.; Valtion Liikuntaneuvosto [Finnish Sports Council]: Helsinki, Finland, 2019; pp. 15–26. [Google Scholar]
- Inchley, J.C.; Stevens, G.W.J.M.; Samdal, O.; Currie, D.B. Enhancing Understanding of Adolescent Health and Well-being: The Health Behaviour in School-Aged Children Study. J. Adoles. Health 2020, 66, S3–S5. [Google Scholar] [CrossRef]
- Ravens-Sieberer, U.; Torsheim, T.; Hetland, J.; Vollebergh, W.; Cavallo, F.; Jericek, H.; Alikasifoglu, M.; Välimaa, R.S.; Ottova, V.; Erhart, M. Subjective Health, Symptom Load and Quality of Life of Children and Adolescents in Europe. Int. J. Public Health 2009, 54, 151–159. [Google Scholar] [CrossRef]
- Kämppi, K.; Välimaa, R.S.; Tynjälä, J.A.; Haapasalo, I.; Villberg, J.; Kannas, L.K. Peruskoulun 5., 7. Ja 9. Luokan Oppilaiden Koulukokemukset Ja Koettu Terveys; WHO-Koululaistutkimuksen Trendejä Vuosina 1994–2006 [Basic School 5th, 7th, and 9th Grade Pupils School Experience and Perceived Health: HBSC Study Trends 1994–2006]; Jyväskylän yliopisto, Opetushallitus [University of Jyväskylä, National Board of Education]: Jyväskylä, Helsinki, 2008; pp. 1–85. ISBN 9789521338243. [Google Scholar]
- Ng, K.W.; Sainio, P.; Sit, C.H. Physical Activity of Adolescents with and without Disabilities from a Complete Enumeration Study (N = 128,803): School Health Promotion Study 2017. Int. J. Environ. Res. Public Health 2019, 16, 3156. [Google Scholar] [CrossRef] [Green Version]
- Bergendorff, S. Contextual Translation – Concepts and Practices among the Mekeo of Papua New Guinea. Oceania 2009, 79, 225–237. [Google Scholar]
- Lee, S. Self-Rated Health in Health Surveys. In Health Survey Methods; Johnson, T.P., Ed.; John Wiley & Sons: Hoboken, NJ, USA, 2014; pp. 193–216. [Google Scholar]
- Boyce, W.F.; Davies, D.; Raman, S.R.; Tynjälä, J.A.; Välimaa, R.S.; King, M.; Gallupe, O.; Kannas, L.K. Emotional Health of Canadian and Finnish Students with Disabilities or Chronic Conditions. Int. J. Rehabil. Res. 2009, 32, 154–161. [Google Scholar] [CrossRef]
- Landis, J.R.; Koch, G.G. The Measurement of Observer Agreement for Categorical Data. Biometrics 1977, 33, 159–174. [Google Scholar] [CrossRef] [Green Version]
- Cohen, J. Statistical Power Analysis for the Behavioral Sciences, 2nd ed.; Lawrence Earlbaum Associates: Hillsdale, NJ, USA, 1988. [Google Scholar]
- Sprunt, B.; McPake, B.; Marella, M. The UNICEF/Washington Group Child Functioning Module-Accuracy, Inter-Rater Reliability and Cut-Off Level for Disability Disaggregation of Fiji’s Education Management Information System. Int. J. Environ. Res. Public Health 2019, 16, 6. [Google Scholar] [CrossRef] [Green Version]
- Kostanjsek, N. Use of the International Classification of Functioning, Disability and Health (ICF) as a Conceptual Framework and Common Language for Disability Statistics and Health Information Systems. BMC Public Health 2011, 11, S3. [Google Scholar] [CrossRef] [Green Version]
- Rosenbaum, P.; Gorter, J.W. The ‘F-Words’ in Childhood Disability: I Swear This Is How We Should Think! Child Care Health Dev. 2012, 38, 457–463. [Google Scholar] [CrossRef]
- Viner, R.M.; Ozer, E.M.; Denny, S.; Marmot, M.; Resnick, M.; Fatusi, A.O.; Currie, C. Adolescence and the Social Determinants of Health. Lancet 2012, 379, 1641–1652. [Google Scholar] [CrossRef]
- Madans, J.H.; Loeb, M.E.; Altman, B.M. Measuring Disability and Monitoring the UN Convention on the Rights of Persons with Disabilities: The Work of the Washington Group on Disability Statistics. BMC Public Health 2011, 11, S4. [Google Scholar] [CrossRef] [Green Version]
- Harter, S. The Perceived Competence Scale for Children. Child Dev. 1982, 53, 87–97. [Google Scholar]
- Hagborg, W.J. The Rosenberg Self-Esteem Scale and Harter’s Self-Perception Profile for Adolescents: A Concurrent Validity Study. Psychol. Sch. 1993, 30, 132–136. [Google Scholar] [CrossRef]
- Kanste, O.; Sainio, P.; Halme, N.; Nurmi-Koikkalainen, P. Toimintarajoitteisten Nuorten Hyvinvointi Ja Avun Saaminen—Toteutuuko Yhdenvertaisuus? Kouluterveyskyselyn Tuloksia [Well-being of Youth with Disabilities and Access to Help—Is Equality Going to Happen? The Results of the School Health Promotion Survey]; Terveyden ja hyvinvoinnin laitos [Department of Health and Welfare]: Helsinki, Finland, 2017; ISBN 978-952-302-838-8. [Google Scholar]
- Leppä, N.; Asunta, P.; Rintala, P. Tukea Tarvitsevat Lapset Vahvemmin Mukaan Lasten Ja Nuorten Liikuntakäyttäytymistutkimuksiin. [Children with Supportive Needs Have to Be Included Stronger in Physical Activity Behaviour Research]. Liikunta Tiede 2018, 55, 13–15. [Google Scholar]
- Ng, K.W.; Rintala, P.; Asunta, P. Toimintarajoitteiden yhteydet liikuntakäyttäytymiseen [Associations between physical activity behaviours and functional difficulties]. In Lasten Ja Nuorten Liikuntakäyttäytyminen Suomessa: LIITU-Tutkimuksen Tuloksia 2018 [Finnish Children and Adolescents Physical Activity Behaviours: Results of the F-SPA Study 2018]; Kokko, S., Martin, L., Eds.; Valtion Liikuntaneuvosto [Finnish Sports Council]: Helsinki, Finland, 2019; pp. 107–116. [Google Scholar]
- Sabariego, C.; Oberhauser, C.; Posarac, A.; Bickenbach, J.; Kostanjsek, N.; Chatterji, S.; Officer, A.; Coenen, M.; Chhan, L.; Cieza, A. Measuring Disability: Comparing the Impact of Two Data Collection Approaches on Disability Rates. Int. J. Environ. Res. Public Health 2015, 12, 10329–10351. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Normative Language | Easy to Read Version | |
---|---|---|
Compared to children of the same age, do you have difficulties in… | Compare yourself to other teenagers of the same age. Which things are easy or difficult for you? | |
1 | Seeing (even if you have to wear glasses or contact lens)? | Do you have difficulties in seeing (even if you have to wear glasses or contact lens)? |
2 | Hearing (even if you have a hearing aid)? | Do you have difficulties in hearing (even if you have a hearing aid)? |
3 | Walking 100m, for example a length of a football pitch (even if you use assistance)? | Do you have difficulties in walking 100m, for example the length of a football pitch (even if you use assistance)? |
4 | Self-care, for example eating or dressing up? | Do you have difficulties in self-care, for example eating or dressing up? |
5 | Being understood when speaking (outside of the home)? | Do you have difficulties in being understood when speaking to people outside of your home? |
6 | Learning things? | Do you have difficulties in learning things? |
7 | Remembering things? | Do you have difficulties in remembering things? |
8 | Concentrating on things you enjoy? | Do you have difficulties in concentrating on things you enjoy? |
9 | Making changes to your own routine? | Do you have difficulties in making changes to your own routine? |
10 | Controlling your own behaviors? | Do you have difficulties in controlling you own behaviors? |
11 | Getting friends? | Do you have difficulties in getting friends? |
Variable | M-er | M | L-er | L | Total |
---|---|---|---|---|---|
Total | 6 | 54 | 5 | 9 | 74 |
Boys | 5 | 32 | 3 | 7 | 47 |
Girls | 1 | 22 | 2 | 2 | 27 |
Disabilities (% some or more) | 4 (67%) | 31 (56%) | 5 (100%) | 5 (55%) | 57 (65%) |
Age in years (SD) | 14.8 (1.2) | 12.3 (1.5) | 16.1 (0.5) | 15.6 (0.6) | 13.2 (1.8) |
Difficulties in… | None | Some | A Lot | Cannot do | Missing | Some + % | A Lot + % |
---|---|---|---|---|---|---|---|
Seeing | 66 | 6 | 0 | 2 | 0 | 10.8 | 2.7 |
Hearing | 66 | 7 | 0 | 1 | 0 | 10.8 | 1.4 |
Walking | 68 | 4 | 0 | 2 | 0 | 8.1 | 2.7 |
Self-Care | 65 | 7 | 1 | 1 | 0 | 12.2 | 2.7 |
Speaking | 63 | 8 | 2 | 1 | 0 | 14.9 | 4.1 |
Learning | 50 | 19 | 3 | 2 | 0 | 32.4 | 6.8 |
Remembering | 51 | 20 | 2 | 1 | 0 | 31.1 | 4.1 |
Concentration | 58 | 14 | 0 | 1 | 1 | 20.5 | 1.4 |
Routine | 56 | 15 | 1 | 1 | 1 | 23.3 | 2.7 |
Behavior | 56 | 14 | 1 | 2 | 1 | 23.3 | 4.1 |
Friends | 58 | 10 | 2 | 3 | 1 | 20.5 | 6.8 |
Function | ICC | LCI | UCI | Landis and Koch | Kappa 1 | p | Cohen | Kappa 2 | p | Cohen |
---|---|---|---|---|---|---|---|---|---|---|
Seeing | 0.753 | 0.608 | 0.845 | Substantial | 0.801 | <0.001 | Large | 0.486 | <0.001 | moderate |
Hearing | 0.63 | 0.411 | 0.768 | Substantial | 0.457 | <0.001 | Moderate | n/a | ||
Walking | 0.645 | 0.449 | 0.782 | Substantial | 0.509 | <0.001 | Large | 0.486 | <0.001 | moderate |
Self-Care | 0.345 | −0.027 | 0.585 | Fair | 0.251 | 0.018 | Small | 0 | poor | |
Speaking | 0.76 | 0.619 | 0.849 | Substantial | 0.472 | <0.001 | Moderate | 0.490 | <0.001 | moderate |
Learn | 0.59 | 0.35 | 0.742 | Moderate | 0.410 | <0.001 | Moderate | 0.210 | 0.061 | small |
Remembering | 0.625 | 0.405 | 0.764 | Substantial | 0.563 | <0.001 | Large | −0.210 | 0.836 | small |
Concentration | 0.204 | −0.274 | 0.502 | Fair | 0.28 | 0.016 | Small | −0.019 | 0.866 | poor |
Routine | 0.277 | −0.145 | 0.544 | Fair | 0.341 | 0.003 | Moderate | −0.034 | 0.767 | poor |
Behavior | 0.732 | 0.573 | 0.832 | Substantial | 0.548 | <0.001 | Large | 0.652 | <0.001 | large |
Friends | 0.57 | 0.314 | 0.73 | Moderate | 0.531 | <0.001 | Large | 0.256 | 0.015 | small |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Ng, K.; Asunta, P.; Leppä, N.; Rintala, P. Intra-Rater Test-Retest Reliability of a Modified Child Functioning Module, Self-Report Version. Int. J. Environ. Res. Public Health 2020, 17, 6958. https://doi.org/10.3390/ijerph17196958
Ng K, Asunta P, Leppä N, Rintala P. Intra-Rater Test-Retest Reliability of a Modified Child Functioning Module, Self-Report Version. International Journal of Environmental Research and Public Health. 2020; 17(19):6958. https://doi.org/10.3390/ijerph17196958
Chicago/Turabian StyleNg, Kwok, Piritta Asunta, Niko Leppä, and Pauli Rintala. 2020. "Intra-Rater Test-Retest Reliability of a Modified Child Functioning Module, Self-Report Version" International Journal of Environmental Research and Public Health 17, no. 19: 6958. https://doi.org/10.3390/ijerph17196958
APA StyleNg, K., Asunta, P., Leppä, N., & Rintala, P. (2020). Intra-Rater Test-Retest Reliability of a Modified Child Functioning Module, Self-Report Version. International Journal of Environmental Research and Public Health, 17(19), 6958. https://doi.org/10.3390/ijerph17196958