Readability of Online Patient Education Materials Related to Liver Transplantation in the United States

Abstract

Background: Healthcare outcomes are influenced by both the effectiveness of healthcare delivery and the health literacy of patients. Patient education materials (PEMs) play a crucial role in disseminating health information to the patients. PEMs need to be at the level of grade six for optimal comprehension across diverse populations. However, that has not been the case in prior studies in healthcare-related fields. We aimed to evaluate the readability status of online PEMs of active adult and pediatric liver transplant institutions. Methods: We used standardized tools to calculate indices, namely Flesch Reading Ease (FRE), Flesch–Kincaid Grade Level (FKGL), Gunning–Fog Score (GFS), Coleman–Liau Index (CLI), and Simple Measure of Gobbledygook for readability assessment. These indices use various measures, like average sentence length, average syllable per word, polysyllable count, and/or average number of letters per 100 words, to determine grade level. Results: The mean reading level of the PEMs was grade 10.73 ± 3.07, corresponding to grade 7 to 14. One-way ANOVA showed no statistically significant difference between these indices (p-value < 0.05). Conclusions: The readability of liver transplant centers’ PEMs exceeded the recommended level, hindering their generalizability to the broader population. This highlights the importance of optimizing the readability of PEMs to improve outcomes for equitable healthcare services.

1. Introduction

The efficacy of healthcare outcomes is multifactorial. It is influenced not only by the quality of care rendered but also by the health literacy levels of patients [1,2]. Research indicates that lower health literacy correlates with heightened reliance on emergency services and diminished engagement in preventive care measures [3]. Disparities in health access are evident among patients with varying levels of health literacy [4]. The literacy level of the patient is the key factor that determines whether the patient truly understands the information being provided. The literacy level also is a determinant of comprehension of medical information, concurrently posing a significant communication barrier, particularly for patients with lower literacy levels [5].

While direct interaction with healthcare providers remains the gold standard for health information dissemination, high-quality written patient education materials (PEMs) are critical to complement those encounters [6]. A notable advantage of written PEMs lies in the reusability of such resources as references, which facilitates long-term knowledge retention [7]. Additionally, written PEMs enhance understanding of diseases and treatment modalities [8].

The Internet has revolutionized the access to information. It has emerged as a widely accessible platform for disseminating education to the general population [9,10]. It has been estimated that over 8 million Americans obtain information about health conditions and treatment options from the Internet [11,12,13,14,15]. However, multiple studies have revealed that PEMs hosted on medical websites often surpass appropriate communication levels, which potentially hinders comprehension [16,17,18,19]. Given the heightened reliance on the Internet for health, the available information must be understandable and accurate. Health institutions are also faced with the added responsibility of providing accurate and evidence-based healthcare information.

The American population has a wide linguistic diversity and varying literacy levels. Additionally, it has been determined that the average Medicaid enrollee reads at the fifth-grade level [20]. Lack of patient comprehension has been shown to compromise health care quality [21]. For wide applicability, the American Medical Association (AMA) has recommended that PEMs be written at the level of grade six [22].

In the field of liver transplantation, low literacy levels have been consistently linked to reduced access to transplantation as well as suboptimal post-transplant outcomes [23,24].

This study aims to evaluate the readability of PEMs disseminated through the websites of liver transplant institutions across the United States. In this novel approach, we aimed to ascertain if the communication levels of PEMs align with the recommended standards set forth by AMA.

2. Materials and Methods

2.1. Study Design

This is an observational study using the publicly available PEMs from liver transplantation institutions in the United States. Institutional review board (IRB) approval was not necessary, as the research involved analysis of publicly available PEMs. There was no direct human participation.

2.2. Article Search

We reviewed and obtained the list of active organ transplant centers that perform liver transplants in the U.S. from the website “Organ Procurement and Transplantation Network” (OPTN). We included all adult and pediatric liver transplant centers. The specific website of each transplantation center was accessed, and the written PEMs were obtained.

2.3. Inclusion and Exclusion Criteria

We obtained the PEMs specifically targeting the liver transplant recipient for analysis. For the transplant centers that performed multiple organ transplantation, we analyzed only the PEMs pertaining to liver transplantation. If the healthcare information was directed to healthcare providers, the information was excluded from analysis. PEMs were included irrespective of deceased donor liver transplant (DDLT) or living donor liver transplant (LDLT) information, as the majority of the transplant centers did not distinguish the PEMs based on the type of donor. Out of 140 active adults’ and children’s liver transplant institutions, 108 had online patient information pertaining to liver transplantation. For comparison, we analyzed the readability of the patient education packet for adult living donor liver transplantation and pediatric liver transplantation from the American Society of Transplantation. The OPTN website did not offer the option to search for simultaneous kidney liver transplant (SKLT) separately; therefore, the PEMs for these procedures could not be sub-analyzed or included separately. Any offered audio-visual, graphics, or pictures as PEMs were not analyzable elements and were excluded. Articles written in a language other than English were excluded from the analysis. Duplicate websites, patient forums, and academic research were excluded.

2.4. Readability Assessment

Readability is defined as the “ease of understanding or comprehension due to the style of writing” [25]. While the design of effective written educational materials relies upon myriad factors, such as content and organization, readability also contributes to patient comprehension.

Before readability was calculated, we copied the text from each centers’ website into a plain text format. This eliminated the underlying codes and non-textual elements including acronyms, paragraph breaks, various syntax, numbers, and decimals to avoid affecting the readability results. We then used a standardized online tool to assess the readability of the materials by using multiple indices. We copied the plain text from each of the PEMs into the readability calculator to eliminate biases arising from formatting. We used the readability tool at “readabilityformulas.com” accessed on 26 July 2024 for analysis. Then, we calculated the standardized readability indices Flesch Reading Ease (FRE), Flesch–Kincaid Grade Level (FKGL), Gunning–Fog Score (GFS), Coleman–Liau Index (CLI), and the Simple Measure of Gobbledygook (SMOG). All these tools are validated and have been used in multiple prior studies for the purpose of determining readability of medical texts.

The FRE and FKGL are determined using the average sentence length as well as average syllables per word [26,27]. For FRE, the higher the score, the easier it is to comprehend the text. An FRE score of 91 to 100 corresponds to grade level 5, 81 to 90 with grade level 6, 71 to 80 with grade level 7, 61 to 70 with grade level 8 to 9, 51 to 60 with grade 10 to 12, 31 to 50 with grade 13 to 16, and 0 to 30 with the level of a college graduate [28]. The GFS uses the average sentence length and the number of polysyllabic words to determine the grade level of the text [29]. The CLI uses the average number of letters per 100 words and the average sentence length to calculate the grade level [30]. The SMOG scale calculates the grade level associated with the text based on complex word density [31]. The details on interpretation of the results are described in the

Table 1. Readability indices and their description/interpretation.

Indices	Description	Interpretation
Flesch Reading Ease (FRE)	Determined using the average sentence length as well as average syllables per word. For FRE, the higher the score, the easier it is to comprehend the text.	91–100—Grade 5 81–90—Grade 6 71–80—Grade 7 61–70—Grade 8 to 9 51–60—Grade 10 to 12 31–50—Grade 13 to 16 0–30—College graduate
Flesch–Kincaid Grade Level (FKGL)	Determined using the average sentence length as well as average syllables per word.	Score corresponds to the grade level. Score of 9.92 ± 2.56 corresponds to grade 7 to 11.
Gunning–Fog Score (GFS)	Uses the average sentence length and the number of polysyllabic words to determine the grade level of the text.	Score corresponds to the grade level. Score of 12.10 ± 2.52 corresponds to grade 10 to 14.
Coleman–Liau Index (CLI)	Uses the average number of letters per 100 words and the average sentence length to calculate the grade level.	Score corresponds to the grade level. Score of 12.58 ± 2.37 corresponds to grade 10 to 14.
Simple Measure of Gobbledygook (SMOG) scale	Calculates the grade level associated with the text on the basis of complex word density.	Score corresponds to the grade level. Score of 8.88 ± 1.92 corresponds to grade 6 to 11.

Recommended level: Grade 6 for all indices per American Medical Association (AMA).

.

2.5. Statistical Analysis

Readability indices were summarized using descriptive statistics (mean and standard deviation). We considered PEMs with grade six and below to be at the recommended comprehension level. We also classified the PEMs into grade 6 to 10 and above grade 10. We compared the readability levels calculated from different indices using one-way repeated measures analysis of variance (ANOVA) using Statistical Package for Social Sciences (SPSS v27), as the same text was used to calculate the indices. Statistical significance was defined as p < 0.05.

3. Results

Out of 140 active liver transplant institutions in the United States, 108 programs had some form of patient information pertaining to liver transplantation. The list of liver transplant centers is available in Supplementary Table S1. Of the 108 PEMs analyzed, the mean reading level of the PEMs was grade 10.73 ± 3.07. The mean readability index for FRE was 48.96 ± 12.42. Similarly, the GFS, FKGL, CLI, and SMOG were 12.10 ± 2.52, 9.92 ± 2.56, 12.58 ± 2.37, and 8.88 ± 1.92, respectively, as demonstrated in Table 1.

As shown in

Table 2. Various Readability Indices of the Text.

	Flesch Reading Ease (FRE)	Gunning–Fog Score (GFS)	Flesch–Kincaid Grade Level (FKGL)	Coleman–Liau Index (CLI)	Smog Index Readability Score (SMOG)
Mean ± SD	48.96 ± 12.42	12.10 ± 2.52	9.92 ± 2.56	12.58 ± 2.37	8.88 ± 1.92

, none of the PEMs were in the “easy” category for FRE, while 19 PEMs were in the “average” and 89 in the “difficult” categories, respectively. Similarly, no PEMs were among the “below 6” grade level for GFS, 4 for FKGL, 0 for CLI, and 3 for SMOG. Moreover, 19 PEMs were in the category “6 to 10” grade for GFS and 59, 16, and 80 for FKGL, CLI, and SMOG, respectively. Additionally, for the category “above 10” grade level, there were 89, 45, 92, and 25 for GFS, FKGL, CLI, and SMOG, respectively.

Table 3. Different Readability Indices and Patient Education Materials counts as Compared to Recommended Grade Level.

Flesch Reading Ease	Easy (80–100)	0
	Average (60–79)	19
	Difficult (0–59)	89
Gunning–Fog Index	Below 6	0
	6 to 10	19
	Above 10	89
Flesch–Kincaid Grade Level	Below 6	4
	6 to 10	59
	Above 10	45
Coleman–Liau Index	Below 6	0
	6 to 10	16
	Above 10	92
Simple Measure of Gobbledygook Index	Below 6	3
	6 to 10	80
	Above 10	25

summarizes the categorization of different PEMs as per the ease of reading.

There was no significant difference in readability scales among the different indices used in this study using one-way repeated measures analysis of variance (ANOVA) (p > 0.05). The list of all institutions with the readability scores is provided in Supplementary Materials Table S1.

The patient education packet for living donor liver transplantation by the American Society of Transplantation scored 75 in FRE, 6.2 in GFS, 4.4 in FKGL, 6.25 in CLI, and 4.9 in SMOG. The pediatric liver transplant PEM scored 71 in FRE, 7 in GFS, 4.9 in FKGL, 7.62 in CLI, and 5.29 in SMOG.

4. Discussion

We found that PEMs oriented to patients were presented at readability levels significantly higher than the recommended sixth-grade level. None of the PEMs assessed fell within the “below 6” grade category. Overall, 17.59%, 54.62%, 14.81%, and 74.07% of the total PEMs belonged in the “6 to 10” grade category for GFS, FKGL, CLI, and SMOG, respectively. These findings are similar to those of various previously published articles that suggest that medical websites are at a higher comprehension level than recommended [16,17,18,19]. However, the PEM provided by the American Society of Transplantation was within the recommended level. Patients may be directed to websites with better comprehension for increased efficacy, but the individual institutions should also be accountable. The transplant centers are certified by the Centers for Medicare and Medicaid, and patient education provision is one of the components [32]. The results are alarming, especially when it is part of the informed consent process for the transplant centers.

Along with the readability of the education materials, the second equally important aspect of comprehension that requires attention is health literacy. The U.S. Department of Health and Human Services (HHS) has coined the terms organizational and individual health literacy, which relate to equitability in the dissemination of health information and the ease of receipt of the same, respectively. [33]. Poor health literacy has an inverse relationship with health outcomes [34].

A systematic review found that patients with low literacy levels had a longer time for transplantation referral and longer waitlist time [35]. These individuals face barriers to accessing transplantation services, which results in inequities in healthcare access and outcomes within the transplantation system. In a comprehensive multi-center study, it was discovered that patients who require non-English PEMs and those with lower literacy levels faced disadvantages in accessing liver transplantation services [23]. Additionally, limited health literacy was identified as a contributing factor to post-transplant medication non-adherence [36]. The limited readability of patient education material further worsens the situation.

Low health literacy levels were also associated with diminished healthcare task performance, like dosing and organizing medications, as well as healthcare problem solving [37]. These patients might encounter difficulties understanding medication instructions and adhering to complex medication regimens [38]. Addressing health literacy barriers is essential for optimizing medication adherence and improving long-term transplant success. The inability to comprehend PEMs places individuals at a significant disadvantage both before and after transplantation.

Healthcare providers must recognize and address this disparity by understanding the social determinants of health [39]. Tailoring communication methods to meet the literacy needs of patients is essential. This may involve employing plain language, incorporating visual aids, and engaging in collaborative efforts with other healthcare providers to ensure that all patients have access to comprehensible and actionable health information. Needless to say, the information should be communicated in the patient’s native language or the language of which they have a full grasp.

In recent years, some transplant centers have embraced positive changes, utilizing digital technology to develop medication and transplant regimen programs. These innovative initiatives not only offer patients longitudinal learning opportunities from the comfort of their homes but also pave the way for better outcomes [38]. For instance, New York-Presbyterian Hospital has spearheaded this approach, demonstrating its potential for wider adoption.

However, despite these advancements, there remains a significant gap in the literature regarding the accessibility and comprehensibility of online patient education materials for liver transplant recipients. Sadly, this trend seems to be persistent across different specialties and countries. Surprisingly, there have been no studies to date analyzing the readability of such materials for all the transplant centers across the United States. There have been no efforts to assess their comprehensibility. Research must be conducted in this area to evaluate how well patients can grasp the information provided, especially because of the extensive care involved in the process.

To address this need, these materials need to undergo regular assessment and review to ensure their content and organization are appropriate. This process should be ongoing and include quality control measures to maintain standards. Additionally, transplant centers must utilize audio-visual resources to enhance the accessibility of patient education materials across various patient demographics. With the increasing use of artificial intelligence (AI), we can use it to our advantage. AI can very easily bring changes to the existing PEMs to improve their readability to the sixth-grade level. A study analyzing the difference in PEMs generated by human-based EPIC software whose headquarters is located in Verona, Wisconsin in United States and another large language model (LLMs) showed that the FRE and FKGL were more comprehensible if generated by LLM (ChatGPT4). Ref. [40] By employing these strategies, centers can effectively reach and educate all patient sub-groups, ultimately improving patient outcomes and satisfaction.

Our study underscores that the current comprehension levels of PEMs related to liver transplantation could perpetuate this disparity among disadvantaged groups that leads to poorer post-transplantation outcomes.

Limitations

Readability formulas are based on sentence/word length as well as word complexity, but they do not assess the suitability of information or its content [41]. Additionally, the comprehension of the PEMs also depends on the prior literacy level and knowledge. Also, different formats of PEMs, like graphical format or video, could not be analyzed using this method. In addition, transplant centers had PEMs in various other language besides English, which is important for health equity. However, we were limited by our indices that analyzed only the English language. We also could not distinguish between PEMs related to deceased donor liver transplant (DDLT) or living donor liver transplant (LDLT) information, as the transplant center websites were not explicit about the differences.

5. Conclusions

In conclusion, the observed discrepancy in the readability levels of health education materials underscores the importance of equitable access to essential health information. Addressing this gap is critical to improving patient comprehension, adherence to medical recommendations, and overall healthcare outcomes. Healthcare providers must prioritize tailored communication methods and collaborative efforts to ensure that all patients, especially those from socially disadvantaged groups, have access to comprehensible and actionable health information.