Investigating Students’ Digital Literacy Levels during Online Education Due to COVID-19 Pandemic

Abstract

COVID-19 has brought serious challenges to education and many other sectors. Within the educational context, the main difficulties experienced have been reported as challenges related to not only mechanical impediments—such as technology or infrastructure—but also to methodological and personal features, such as lack of motivation or online learning/teaching experiences. The abrupt transition from face-to-face to online education has created the need for some specific abilities, such as digital literacy on the side of the learners at all educational levels. In this context, this mixed-method study aims to determine the digital literacy levels of learners belonging to different school levels and whether age, gender and school degree were significant variables. This study also investigates the technology-related challenges students experienced during COVID-19 pandemic. The sample consisted of 510 participants representing different school levels, age groups, and genders; in addition, a smaller representative group (n = 30) revealed their own evaluations related to their levels of digital literacy and the technology-related challenges they had about online learning. The findings suggest that there is a statistically significant relationship between students’ digital literacy and their gender and school degree, while age was not found to be a statistically significant variable. The qualitative self-reported data suggested that learners have sufficient levels of digital literacy, and that the major technology-related challenges were reported to be lack of the necessary technologies and difficulties in adapting to a new approach to learning.

1. Introduction

Since the World Health Organization (WHO) declared COVID-19 as a global pandemic, it has had a significant impact on many different sectors, the education sector being one of them. Different measures have been taken, e.g., the shutting down of schools and the use of social distancing precautions [1], and schools of all levels have utilized all of their available technical resources to continue their education online. Some countries tried to overcome this unexpected crisis through distance education whereas the others suffered from socioeconomic or digital infrastructure problems [2]. Due to the COVID-19 outbreak, more than 91% of students had to be away from their schools [3]. One hundred and eighty-five nations in the world have suspended face-to-face education, and have been in need of adopting alternative methods since 31 May 2020, which brought about the spread of ‘online education’. The use of computers and web-based tools in education has a long history and it has become prevalent thanks to the advancement of internet facilities, and these developments created the term ‘online learning’ [4]. However, the widespread use of online learning has revealed some challenges experienced by students [5,6,7] as well. A vast majority of students were blindsided in the face of such a drastic change in education [8], because basic qualifications were required for them without which their success in an online course would not be possible [9]. One of these necessary qualifications is digital literacy (DL), because online learning definitely requires effective use of technology and digital skills or computer skills. Moreover, in online learning contexts, it is unlikely for individuals with low levels of DL to go on benefiting from web-based learning [10].

In Turkey, there has been a good understanding of the necessity of digital skills, and the government’s efforts in relation to DL have contributed positively to this understanding [11], and this pandemic has proved the significance of DL for individuals, specifically for students, as schools of all levels had to go online due to the measures taken.

The term digital literacy first appeared in literature at the end of 90s with the work of Gilster ([12], p. 10) and he defined it as ‘the ability to understand and use the information coming from different resources with the help of computers. He suggested that it includes not only ‘literacy’ but also ‘evaluation of the information accessed’ and ‘the ability to solve the problems encountered during the process’. On the other hand, ref.[13,14] contended that DL has similarity to such concepts as ‘know-how’ and defined it as the functional use of technology. However, with the help of very recent studies, DL has turned into a multidimensional concept. For instance, it was contended that DL is not related to hardware or software-related literacy only, as it has cognitive, social and critical aspects [15]. In this sense, Bayrakci defines DL as ‘the combination of the competences required to use digital technologies effectively in social, cultural, educational and economy-related fields, to evaluate the information and the sources of information, to be aware of the risks related to digital world and to adapt oneself to the digital era’ ([15], p. 16).

It is possible to encounter different terms that are sometimes used synonymously with DL, such as ‘computer literacy’, ‘information literacy’, or ‘computer and information literacy’. In the related literature [16], and in some other studies, DL is used as an umbrella term embracing all of these terms [17]. DL is generally preferred when all kinds of digital devices and environments are the main focus [18]. As these two are the main areas of concern during online education, in this study DL was preferred. Having a high level of DL allows users to reach, produce and share reliable information and make use of technological tools in the process of education [19]. Digitally literate individuals are regarded as more productive, innovative, and cooperative with their better skills of critical thinking, problem-solving and decision-making concerning using technology [20]. Due to COVID-19, DL skills have become an important part of the classrooms and become a requirement for both students and the faculty. However, it was suggested that this pandemic has proved ‘the unreadiness of educational institutions’ and ‘the inadequacy of functional digital literacy training’ for all the stakeholders of education ([21], p. 276).

Even under usual conditions of face-to-face teaching, doing research online or communicating through email or instant messaging with teachers or peers is a part of every single course [22,23] and, as a result, their teachers consider them as ‘digital natives’ [24]. However, in reality, not all the students possess the same level of digital literacy [23]. For this reason, it is necessary to approach the term DL with caution and it should be analyzed in relation to different variables. It was claimed that it is never possible to fully understand what DL really includes without referring to students’ individual differences such as age, gender, etc. [18]. In this context, this study aims to investigate DL levels of students in Turkey, who take part in online education due to COVID-19 pandemic, within the scope of different variables, namely, age, gender and school level. The researchers also aim to identify the main challenges learners of different ages and school levels had during their online learning experience as a result of COVID-19 pandemic.

In the 21st century, DL is considered to be a necessity to successfully master the tasks of professional and personal life and this individual capacity, and this capacity becomes more important specifically in online education settings. In a distance learning context, it was revealed that one of the important predictors of learners’ overall success in distance learning was found to be digital literacy skills as well as self-efficacy and self-regulated learning skills [25]. Similarly, it was found out that out of 44% students who dropped out, 42% stated that they were dissatisfied with the online learning environment [26]. The basic reason why they dropped out was that the students did not feel competent enough in learning via the internet and they could not use the online communication software as a learning tool.

On the other hand, digital skills are not ‘evenly spread amongst the people’ ([27], p. 9). It was claimed that a ‘theoretically multidimensional concept like DL’ should be studied in accordance with these dimensions; thus, such dimensions as age, gender, race are quite significant ([18], p. 1). Previous research investigated the role of individual differences on students’ DL and age is one of the most commonly studied individual factors in different contexts. Age-related DL research has led to conflicting results [28,29] but the findings of the studies have proved that ‘The younger the better’ assumption may not be true all the time because it is not age but experience that really matters [30]. In another study, it was revealed that age is not the only determinant of digital literacy for this reason; it should not constitute an important part of decision-making even though it is necessary to take it into consideration during material design and development of educational policies [28].

Another individual factor leading to conflicting research findings is gender. Even though in some studies girls were found to be more digitally-literate [31,32,33], others found that males outperformed females [34]. In a third group of studies, no significant difference was found between DL and gender [35,36,37]. However, it is acknowledged that more research is needed [28]. In addition, previous research including another important factor, school degree, is also very rare [38] and the studies related to DL in the Turkish context are rather scarce [11,19]. To the best of our knowledge, no other study has investigated the digital literacy skills of students representing different school levels with such a large sample referring to the pandemic era. Based on these calls for further research and online education contexts gaining importance everywhere in the world due to COVID-19, this study sought to answer the following research questions:

(1)

What are the self-reported digital literacy levels of students of different school degrees?

(2)

Is there a statistically significant relationship between students’ self-reported digital literacy levels and

• their gender?
• their age?
• their school degree?

(3)

What are the major technology-related challenges learners experience related to online learning during COVID-19 pandemic?

2. Materials and Methods

2.1. The Context of the Study

This mixed-method study aimed to investigate the students’ digital literacy levels during online learning due to COVID-19 pandemic. It also aimed to identify the basic challenges experienced by students of different school levels during online education in Turkey. Just like the other nations suspending face-to-face education, Turkey was also in need of alternative methods for education and as of March 2020, Turkey started to use alternative distance education models in different school levels as an emergency plan [39]. The Ministry of National Education (MEB) launched EBA (Network of Educational Information Systems) through satellite TV broadcast and the internet for elementary, secondary and high school education levels [40] through which all the classes were taught both synchronously and asynchronously; on the other hand, universities continued their education with the help of their own online education platforms, again either synchronously or asynchronously.

2.2. The Method and Participants

This study adopted an explanatory sequential design including both quantitative data from 510 students through a scale and qualitative data gathered with the help of the interviews conducted with 30 volunteering students from the sample to be able to ‘extend, refine or explain’ [41] the quantitative findings.

A study sample consisting of 510 students representing different age groups was generated from different school degrees, namely, high school, university, masters and doctorate. Detailed information related to the participants is given in

Table 1. The distribution of the sample based on age, gender and degree.

	Gender		Age				Degree
	Female	Male	16–18	19–21	22–25	26+	High School	Undergraduate	Master	PhD
F	360	150	90	82	164	174	149	163	142	56
%	70.6	29.4	17.6	16.1	32.2	34.1	29.2	32	27.8	11

below:

As illustrated in Table 1 above, females constitute the majority of the participants in the study (70.6%). Within the scope of age, the percentages of age groups show a relatively close distribution even though the majority of the participant students belong to the age group <26. (34.1%). When it comes to the distribution of the participants based on their school degree, out of 510 participants, 149 of them are high school students (29.2%), 163 of them are university students (32%), 142 of them are MA students and 56 of them are PhD students.

2.3. The Procedure

In order to answer the first and second research questions, after the necessary ethical considerations were ensured, the scale developed by Bayrakci [15] was converted to Google Forms and the participants of this study were recruited via snowball sampling method thanks to Facebook and the colleagues taking part in online education. All of the 510 participants gave consent to take part in our study. Once the data collection procedure was over, the data were analyzed with the help of the SPSS 28.0 program.

In order to answer the third research question, 30 participants representing different age groups, school levels and genders were invited for online interviews. The main motive behind these interviews was in-depth exploration of the participants’ self-evaluations of their levels of digital literacy and also revealing their main challenges they had during online education supplied by either online platforms of their universities or the EBA. Each interview took approximately 30–35 min and they were recorded with the consent of the interviewees through Zoom. The questions were prepared by the researchers in line with the scope of the study. Professional opinion was gathered from two instructors holding a PhD in the field of educational research and applied linguistics related to the wording of the interview questions and their suitability for the research aims.

After the interviews were over, the researchers listened to the recordings multiple times, and they were transcribed verbatim. We adopted an open strategy for coding whereby we coded everything so that we were able to identify as many potential themes as possible from the dataset. This strategy was preferred as a part of the cross-sectional qualitative coding approach in which we first coded the themes and then we used some representative quotes to support all those themes [42,43,44].

To ensure the interrater reliability of the qualitative findings, a second coder checked the sample codings. We used simple percentages to be able to calculate the coding agreements. In accordance with [42], we calculated the percentage of interrater reliability in the following way: the number of coding agreements over the number of coding episodes multiplied by 100. Accordingly, we reached 87% agreement for the sample coding of the interview transcripts. This percentage is considered to be satisfactory as any percentage above 75% is generally considered ‘good’ in the related literature ([45], p. 244). The researchers discussed and resolved the disagreements afterwards.

2.4. The Instrument

The Digital Literacy Scale [15] was utilized in this study to be able to identify the participant students’ digital literacy levels. This scale consists of 29 items and 6 dimensions, namely, ethics and responsibility (7 items), general knowledge and functional skills (6 items), daily use (6 items), professional production (2 items), secrecy and security (4 items), social aspect (4 items). The Cronbach’s Alpha Level of the whole scale is 0.911 and that of its sub dimensions are as follows: ethics and responsibility (0.842), general knowledge and functional skills (0.875), daily use (0.782), professional production (0.719), secrecy and security (0.820), and social aspect (0.761). The rationale for including this scale as the measurement instrument is that these six dimensions included in this scale are in line with the competences that have significance within the context of online education. The digital competences included by each level of DL are given in

Table 2. The levels of DL defined by the scale.

DL Scale Score Range	Level
1.62–3.07	Low/Very Poor
3.08–3.62	Below moderate/Poor
3.63–4.17	Moderate
4.18–4.72	Above moderate

below:

The first part in the questionnaire includes questions of the participants’ demographic information, namely, age, gender, and school degree. The second part of the questionnaire including those six dimensions is represented by 29 items in a five-point Likert format including (1) ‘totally agree’, (2) ‘agree’, 3 ‘undecided’, (4) ‘disagree’, and (5) ‘totally disagree’.

The reliability of this scale was calculated once again with this specific group of participants to prove that it is also reliable for the sample of this study. The findings of the analysis are given in

Table 3. The results of the reliability analysis.

N of Items	Cronbach’s Alpha	Mean	Median	Std. Deviation	Variance
29	0.900	3.940	4	0.56075	0.474

below and they prove that this scale is also reliable for this study:

After the reliability analysis, we explored the distribution of the data using the Shapiro–Wilk test of normality. The results show that the data is not normally distributed (df = 3; p = 0.002 < 0.05); therefore, non-parametric tests were chosen for further analysis.

3. Results

In line with our first research question, we initially tried to identify students’ self-reported DL levels and we carried out a descriptive analysis of the survey and in

Table 4. Descriptive results for the dimensions of digital literacy.

Dimensions	Mean	Median	Std. Deviation
Ethics and Responsibility	4.44	5	0.91461
General Knowledge	3.68	4	1.16414
Daily use	4.38	5	0.98805
Professional Production	2.15	2	1.21982
Secrecy and Security	4.32	5	0.930045
Social Dimension	3.08	3	1.18193

below, descriptive results belonging to the six dimensions of the scale were given.

As shown in Table 4 above, overall, the participants reported very high levels for ethics and responsibility, daily use, and secrecy and security dimensions. While ‘general knowledge’ and ‘social dimension’ was also at moderate levels, ‘professional production’ was at a low level.

For the second research question of the study, we analyzed the relationship between students’ self-reported DL levels and their age, their gender and their school degree. First of all, to investigate whether age variable has any impact on self-reported DL levels of the participants, Kruskal–Wallis H test was conducted. The results for the whole test indicate that age is not a significant factor in the DL levels of the participants (x² =4.266; df = 3; p = 0.335). Second, whether gender variable displayed a statistically significant role in the participants’ reported DL levels was investigated and it was found to be a statistically significant variable (x² = 6.070; df = 1; p = 0.014). Furthermore, in order to find out the dimensions where the participants differed based on gender variable, Kruskal–Wallis H test was run for each dimension. The results of the test are given in

Table 5. The impact of the participants’ gender on their self-reported digital literacy.

Dimensions of the Scale	Groups	N	Mean Rank	x2	df	p
Ethics and responsibility	Female	360	258.04	0.369	1	0.544
	Male	150	249.40
General knowledge	Female	360	224.53	54.262	1	0.000
	Male	150	329.84
Daily use	Female	360	251.68	0.833	1	0.362
	Male	150	264.67
Professional production	Female	360	254.45	0.064	1	0.800
	Male	150	258.01
Secrecy and Security	Female	360	252.41	0.562	1	0.453
	Male	150	262.91
Social dimension	Female	360	255.83	0.006	1	0.937
	Male	150	254.71

below:

• Kruskal–Wallis Test
• Grouping variable: gender.

According to the results, the DL levels of the participants differed significantly regarding only general knowledge dimension (x² =54,262; df = 1; p = 0.000) while there were not any significant differences in terms of the other dimensions. The male participants in the study reported significantly higher levels of general knowledge than female participants. Third, the relationship between the participants’ school degree and their self-reported DL was investigated. The related findings are given in

Table 6. The relationship between the participants’ school degree and their self-reported digital literacy.

Dimensions of the Scale	Groups	N	Mean Rank	x2	df	p
Ethics and Responsibility	High School	149	248.52	5.286	3	0.152
	Undergraduate	163	264.55
	Graduate/Master	142	239.77
	Graduate/PhD	56	287.62
General Knowledge	High School	149	230.99	18.656	3	0.000
	Undergraduate	163	262.17
	Graduate/Master	142	245.11
	Graduate/PhD	56	327.66
Daily Use	High School	149	186.08	55.688	3	0.000
	Undergraduate	163	284.56
	Graduate/Master	142	264.89
	Graduate/PhD	56	331.81
Professional Production	High School	149	242.17	18.933	3	0.000
	Undergraduate	163	229.06
	Graduate/Master	142	297.18
	Graduate/PhD	56	255.38
Security and Safety	High School	149	252.23	4.985	3	0.173
	Undergraduate	163	258.70
	Graduate/Master	142	241.19
	Graduate/PhD	56	291.15
Social dimension	High School	149	240.67	3.536	3	0.316
	Undergraduate	163	262.68
	Graduate/Master	142	253.10
	Graduate/PhD	56	280.16

below:

• Kruskal–Wallis test.
• Grouping variable: degree

The results indicate that the educational level of the participants is a significant factor in the reported levels of general knowledge (x² = 18.656; df = 3; p = 0.000), daily use (x² = 55.688; df = 3; p = 0.000), and professional production dimensions (x² = 18.933; df = 3; p = 0.000). Accordingly, graduate PhD students reported higher levels of general knowledge in literacy and higher levels of daily use while graduate master students stated higher levels of professional production. High school students had the lowest levels of literacy for general knowledge, daily use, and social dimensions whereas the lowest rates for security and safety and ethics and responsibility dimensions belong to master students. Professional production dimension, on the other hand, was the lowest for graduate students. In the analysis of the qualitative data coming from the semi-structured online interviews, first of all, the participants’ general evaluation of their own DL levels based on the dimensions of the scale was investigated. Regardless of their school degree, nearly all of the participant students had very positive evaluations of their DL level. L14 said ‘I can carry out basic transactions or even more complicated ones without the guidance of another person and when I encounter a problem, I do my best to solve it and I generally succeed.’ Similarly, L16 mentioned his self-confidence in his digital skills, and he said ‘I don’t think I have major problems. Thanks to my experience, I also easily adapted myself to online education during pandemic’.

Secondly, we explored the major technology-related challenges students experienced during online education. The most frequently stated technology-related challenges were ‘lack of the necessary technologies’ and ‘difficulties in adapting to a new approach to learning’. For the first challenge, L3 mentioned the lack of a stable internet connection in the following way: ‘I experienced the same frustration every time I tried to attend the synchronous classes because of the unstable internet we use at home. The voices were garbled; there were cut-ins and cut-outs. For this reason, I had to watch the recordings of the online classes.’ Similarly, L2 mentioned the difficulties she had as she did not have a personal computer. She said ‘I had to wait for my elder brother until his classes were over; as a result, I missed many of my synchronous classes. The reason was that I do not have my own laptop.’ The second most frequently mentioned challenge was related to ‘the difficulties learners have had in adapting themselves to the new learning/teaching approaches’. L4 stated that he felt quite unprepared for such a new way of learning/teaching saying that ‘it started all of a sudden and I did not know anything about how to follow an online lesson synchronously. Sometimes voices were garbled or cut in and out’. Likewise, L12 revealed the difficulties he experienced by saying ‘it was a completely new thing for me. I had never participated in an online lesson. I was not used to watching a live lesson video with my teacher and looking at her face and trying not to lose my concentration for hours. It was a real challenge for me’.

4. Discussion and Conclusions

The present study aimed to explore the digital literacy levels of learners of different school levels and whether age, gender and school degree were significant variables. We also investigated the technology-related challenges students experienced during COVID-19 pandemic. With regard to the first part of the first research question, the self-reported data coming from the students revealed very high levels for ‘ethics and responsibility’; ‘daily use’; and ‘security and safety’ dimensions. While ‘general knowledge’ and ‘social dimension’ was also at moderate levels, ‘professional production’ was at a low level. The participants also reported moderate and high levels of digital literacy during the interviews. This finding is in line with that of the study by Bayrakci [15]. In this study, the majority of the participants also reported their DL levels are either moderate or high. This finding might be related to the fact that all the participants in this study belong to a world in which nearly everything is carried out through technology, and as it is also a part of their daily life, they easily define themselves as digitally-literate.

The other research question investigated whether age, gender and school degree were significant variables and different results were obtained for different variables. Age was not found to be a significant factor in the literacy levels of the participants. Previous literature obtained different results revealing that as people get older; their DL levels decrease [46,47,48]. Secondly, gender was found to be a statistically significant variable. The DL levels of the participants differed significantly with regards to only general knowledge dimension and no significant difference was found for other dimensions. The male students reported significantly higher levels of general knowledge than female ones. This finding supports those of [15,49,50,51], which also revealed higher scores for male participants. The possible reasons behind this distinction could be attributed to multiple factors such as the common interest of men in computer games, mobile apps, their preferences of computer-related faculties and occupations more than women [15].

Thirdly, we explored whether the students’ school degree was a significant variable. In all the dimensions indicating statistical significance (general knowledge, daily use and professional production) of the scale, graduate students had the highest mean ranks. Similarly, in the study by Horrigan [46] which aimed to compare and contrast the DL levels of students, master’s and doctorate level students got the highest scores in the survey. The possible reason for this might be the immediate shift of these students to business life, which requires a lot more engagement with digital activities. Moreover, Bayrakci [15] obtained a similar result of the comparison of the DL levels of students representing different school degrees suggesting that the main distinction between the scores of undergraduate and graduate students might stem from the amount of digital work they are involved in and also the increasing possibilities for in-service training for graduates.

When it comes to the findings of the online interviews, regardless of age, gender, and school degree the participants generally reported their DL levels as high, and they also stated that they generally do not need the guidance of another (usually a more experienced) person. The reason behind this ease was explained by the fact that they already use technology in their daily life for personal activities. For the challenges they had during online education, ‘the lack of the necessary technologies’ and ‘difficulties in adapting to a new approach to learning’ were reported as the major technology-related challenges during online education. Likewise, in a study by Almahasees [52], the most important challenges students had during online education due to COVID-19 were ‘adaptation problems to online education’, ‘technical and internet issues’ and ‘data privacy and security. On the other hand, in a study carried out in a Saudi Arabian context, Khalil et al. [53] grouped the challenges students had under three categories, namely, technical, methodological and behavioral challenges. Similar to our study, they also reported ‘internet connectivity’ and ‘poor utility of technological tools’ as the major technical challenges.

To conclude, it is an undeniable fact that we have been living in an era of digitalization and the COVID-19 pandemic has also proved how essential these skills are in all fields of life, but mainly in education in different settings all around the world. For this reason, training related to these skills could be a part of the school curricula regardless of the level of education, as terms such as ‘distance learning’, ‘online learning’ and ‘lifelong learning’ have already been a part of our lives and we are in fact in the middle of a digital transformation. Future work on this area of study could assist all the stakeholders of education and could contribute to prospective educational planning at all levels of education.