Assessing Water Literacy: Undergraduate Student Conceptions of Groundwater and Surface Water Flow

Abstract

Given the importance of fresh water, we investigated undergraduate students’ understanding of water flow and its consequences. We probed introductory geology students’ pre-instruction knowledge using a classroom management system at two large research-intensive universities. Open-ended clicker questions, where students click directly on diagrams using their smart device (e.g., cell phone, tablet) to respond, probed students’ predictions about: (1) groundwater movement and (2) velocity and erosion in a river channel. Approximately one-third of students correctly identified groundwater flow as having lateral and vertical components; however, the same number of students identified only vertical components to flow despite the diagram depicting enough topographic gradient for lateral flow. For rivers depicted as having a straight channel, students correctly identified zones of high velocity. However, for curved river channels, students incorrectly identified the inside of the bend as the location of greatest erosion and highest velocity. Systematic errors suggest that students have mental models of water flow that are not consistent with fluid dynamics. The use of students’ open-ended clicks to reveal common errors provided an efficient tool to identify conceptual challenges associated with the complex spatial and temporal processes that govern water movement in the Earth system.

1. Introduction

In his critical and thought-provoking piece on science literacy, Feinstein posits that science literacy is “not incidentally but fundamentally about identifying relevance: learning to see how science is or could be significant to the things you care about most” [1] (p. 180). Science literacy has moved beyond a collection of canonical facts, what Feinstein calls rhetorical science literacy, and to a socially meaningful construct in service to everyday life. The Next Generation Science Standards (NGSS) underscore this modern definition by prominently featuring human impacts in the Earth system, climate change, biogeology, water, natural resources, and natural hazards [2]. One significant barrier to Earth systems science literacy is conceptual understanding of the fundamental physical processes driving movement of energy and matter. Decades of research on the conceptual understanding of hydrological systems reveals that students struggle with issues of scale and the connectedness of system components in the natural world [3,4,5,6,7]. Similar to recent studies by Arthurs and Elwondger [3] and Lally and Forbes [8], the present study leverages visual models (e.g., diagrams) to explore students’ reasoning about surface and ground water processes. One rationale for this approach is to understand our students’ knowledge base and how that differs from the scientific knowledge that people need to make informed decisions about water-related issues (e.g., town planning boards, community referenda). Specifically, management of freshwater resources, such as the High Plains aquifer of North America and the Guarani Aquifer of South America, require understanding and cooperation that crosses political boarders [9,10,11]. The goal of this study was to assess undergraduate students’ conceptual understanding of fluid dynamics (e.g., velocity, flow direction) in rivers and groundwater using diagrams, a largely unexplored aspect of water literacy. These topics are of particular importance because groundwater flow determines the migration of pollutants and surface water flow patterns determine zones of high erosion. A well-informed public must understand these spatially and temporally dynamic processes to predict their impact on health and safety in their community (e.g., drinking water, riverbank erosion).

The following research questions guided this work:

1. What is the pattern of students’ predictions about groundwater flow?

2. What is the pattern of students’ predictions about velocity and erosion in straight and curved river channels?

1.1. Students’ Understanding of Water

In their recent contribution to this special issue, McCarroll and Hamann [6] conducted a systematic review of literature in pursuit of a better definition of water literacy. Within the context of their framework, the present study probes into students’ “science and systems knowledge” [6] (p. 8) of water’s physical properties and ability to transport dissolved and solid materials [6]. We pursue this line of research in response to prior work identifying gaps and disconnections in students’ mental models of surface and groundwater.

1.1.1. Groundwater

In a study of elementary students in Arizona, only 18% included groundwater in their drawings of the water cycle [12]. In a different study, two-thirds of high school students acknowledged an underground component of the water cycle, though they described it as a static subsurface lake [13] (p. 373). This is consistent with more recent findings that secondary and undergraduate students conceptualize water as an underground river or layer of water [3], despite the fact that karst systems provide less than 20% of the groundwater storage in the U.S. When probing secondary and undergraduate students, Dickerson and Dawkins [7,14] found that students’ terminology did not consistently reflect their understanding. The authors found as many as one-third used terminology that is inappropriate for the scale of the feature (e.g., pipes as a microscopic conduit for water transport). Some students could correctly elaborate on groundwater occupying small pore spaces despite using terms such as “underground stream” or “pools”, while others who utilized terms such as “porosity” and “permeability” did not have a conceptual understanding of those terms.

In an effort to develop a learning progression describing how student understanding transitions from naïve views to scientific models of socio-ecological systems, including groundwater systems, data were collected with students across the K-12 grade levels [5,15]. Students were asked to draw “what it looks like underground where there is water” [15] (p. 45) and whether a landfill could pollute a well. Sixty-five percent of middle and high school students drew underground rivers and lakes. Only 17% of high school students accurately represented groundwater as occurring in spaces within rocks or sediment. With respect to pollution, 59% of high school students recognize that pollution can occur underground but only 18% described how groundwater movement assists in the transport of pollution. Consequently, Gunckel et al. [5] developed a four-level learning progression to elaborate on the levels of understanding. The learning progression consists of four levels. In level 1, students describe only the visible the components of the water cycle (e.g., lakes, rivers) with no mechanisms except humans to move water from place to place. In level 2, students also only describe macroscopic, visible components of the water cycle, but they are beginning to identify some mechanisms for how water moves (e.g., downhill). In level 3, students are able to describe microscopic (e.g., substances in water) through macroscopic components and recognize the three-dimensional nature of water systems. For example, students possessing level 3 understanding will recognize the presence of groundwater stored underground and the mixing of materials with water. However, only level 4 students will recognize the driving mechanisms involved in systems. For example, level 4 students will recognize that topography, permeability, and other factors constrain the flow of water. Gunckel et al. [5] found that only 11% of their participants achieved level 4 water literacy. The present study investigates undergraduate students’ understanding at this fourth level, where groundwater movement is determined by a combination of gravity and topography.

1.1.2. Surface Water

Despite the focus of water literacy studies on the water cycle or watersheds, there are fewer studies focused on the mechanisms involved with rivers and streams than on groundwater. In a study of rural Indiana middle school students’ concepts of watersheds, none of the students’ drawings connected water on adjacent land or underground to the water in rivers [7]. Similar patterns are observed with adults, where personal experience plays a big role in one’s conceptual model. A study in Florida revealed that only one third of the survey sample could name the watershed and only 19% could name a body of water in the watershed [16]. There are no studies providing data on students’ understanding of water movement in a river.

Within the learning progression described by Gunckel et al. [5], a student with a level 3 understanding may be able to trace connections between bodies of water (i.e., structures) but only those with level 4 understanding would be able to explain or make predictions about the driving forces of gravity and the controlling variables, such as permeability and topography, that would determine where and how water will flow on the surface. Building on the learning progression, Sadler et al. [17] explored the physical dynamics of water flow with more precision in a framework called Understandings of Water Systems. Sadler’s [17] framework categorizes the dimensions of surface and groundwater in water systems into process mechanisms, energy transfer, scale, representations, and human agency. For the present study, we focused on students’ understanding of: (1) the movement of surface water (Sadler’s process mechanism for surface water) and its implications for erosion, and (2) the movement of groundwater (Sadler’s energy transfer for groundwater) and its implications for pollution transport. These foci map on to Sadler’s [17] learning goals for the dimensions of process mechanisms and representations (i.e., diagrams) elaborated in their

Table 1. Number of students consenting to participate in each course by semester, instructor and institution.

Institution	Instructor	Semester	N
Institution 1	Instructor 1	Spring 2017	56
		Fall 2017	53
		Spring 2018	26
		Fall 2018	37
		Fall 2019	50
	Instructor 2	Fall 2017	15
		Spring 2018	18
		Fall 2018	17
Institution 2	Instructor 3	Fall 2017	115
		Spring 2019	126

. Our efforts to probe these concepts are unique with respect to the empirical findings reported in published studies. Sadler’s [17] conceptual framework is yet untested and the present study extends investigation of students’ understanding of water to a finer grain than the learning progression on socio-ecological systems [5,15].

1.2. Learning with Diagrams

One often overlooked aspect of science literacy is the ability to draw meaning from a variety of visual representations. Citizens seeking information to better understand issues in their communities may encounter websites containing tables, graphs, and diagrams designed by experts to communicate with other experts. Additionally, the physical scale and temporal variability of Earth’s processes require science educators to rely on several types of models, including diagrams, when communicating about Earth [18]. Students can learn more from words and pictures than from words alone [19]. The content of diagrams may not include important elements that students need to learn. For example, in an analysis of Indian pre-college textbooks, Vinisha and Ramadas [20] found that subsurface structures and processes were largely absent from water cycle diagrams. A recent analysis of water cycle diagrams (N = 350) from textbooks, peer-reviewed articles, and government materials from 12 countries revealed that although groundwater and surface water are commonly depicted, the aspects of pollution transport, groundwater recharge and discharge, and intermittent streams are under-represented [21].

Despite the informational affordances of diagrams, there is a cognitive cost to processing diagrams [22,23], and diagrams can facilitate learning for some students disproportionately based on their spatial skills while leaving others behind [23]. Expert-designed visuals can be complicated and novices may struggle to extract their meaning [24]. In addition to the cognitive effort of processing diagrams, the individual elements in a diagram can be seductive, distracting from the learning process [23]. Clark et al. [25] found that aspects of a plate tectonic diagram, such as an orange mantle, reinforces erroneous conceptions of the Earth’s mantle as molten. Dolphin and Benoit [26] found that metaphors and visualizations, for example representing continents as distinct objects rather than components of tectonic plates, hindered students’ mental models of plate tectonics. Consequently, for this study, we designed diagrams that adhere to principles of good graphic design to maximize students’ processing of the diagrams [27] and minimize distracting details (e.g., signals [28]) for the present study. Specifically, we omitted features commonly observed in depictions of the water cycle including color, dimensional shading, realism (e.g., clouds, waves) and arrows [29].

Several studies on students’ conceptions of hydrologic systems utilize student sketches to elicit their understanding of water in the Earth system [3,4,5,6,7,12,13,14,30]. The approach used in the present study has enabled us to gather data with a broader sample of students because classroom response systems (CRS) can be employed with large numbers of students. Covitt et al. [15] probed students’ reasoning with diagrams by asking students to interpret cross-sectional diagrams and maps of water ways that depicted elevation gradients. However, they did not report on the students’ errors with the level of specificity we investigate in the present study since their goal was to describe a learning progression of water in environmental systems [15]. The present study is smaller in scope, focusing on fewer concepts associated with water movement in two discrete scenarios. This study is novel because we use diagrams and a CRS to investigate aspects of surface and groundwater movement that are missing in prior studies.

1.3. Current Study

The goal of the present study is to investigate undergraduate introductory students’ common errors associated with ground and surface water processes. Most studies on students’ conceptual understanding involves interviews [3,13] or analysis of open-ended text [3,5,13] data. These methods are time intensive and involve relatively small sample sizes. Alternatively, we leveraged a web-based CRS (i.e., Top Hat) to collect a large volume of data. The use of smart technology (i.e., cell phones and tablets) allows CRS to move beyond traditional multiple-choice clicker questions. Now, students can respond by clicking on a diagram to efficiently reveal conceptions of spatial phenomena [31]. In the present study, students were prompted to identify a feature or make a prediction. The predictions indicate students’ understanding of the dynamic and spatial processes associated with water flow in groundwater and rivers.

2. Materials and Methods

2.1. Setting and Population

Student data were gathered in introductory courses at two institutions: Northern Illinois University, a large, high research activity, public Midwestern university (Institution 1) in the United States and University of Calgary, one of the U15 Canadian public research universities (Institution 2). Data were collected from spring 2017 through Fall 2019 (Table 1). Individual student-level demographics were not collected. All sections at Institution 1 were the same introductory geology course, a general education elective for most and a required course for pre-service middle grades and geology majors. Fewer than 10% of the enrolled students were science and geology majors. Enrollment in the course reflects the demographics of the university, which at the time of data collection included approximately 51% White, 17% Hispanic/Latino/a/x, 14% Black with approximately equal gender enrollment. Data were collected with two introductory geology courses at Institution 2. One course, Principles of Geoscience (Fall 2017), served science majors, primarily biology, with approximately ten percent geology majors. The other course, Introduction to Geology (spring 2019), served non-science majors, predominantly humanities and social science majors, with fewer than 10% education majors. Racial demographics are not reported by this institution but the gender balance was not equal. Greater than 60% of enrollment in the science majors course comprised male and greater than 60% of enrollment in the non-science majors course comprised female. Three instructors, two at Institution 1 and one at Institution 2, administered the set of questions in the first two weeks of the semester as assessment of students’ initial knowledge prior to instruction. Students were given approximately 45 s to answer each question before the instructor advanced to the next question. They received points for class participation for taking the assessment regardless of whether they consented to have their responses included in this research study. Only data from consenting students were included. This study was approved by the Northern Illinois University Institutional Review Board (IRB) and Canadian Institution’s Research Ethics Board.

2.2. Diagrams and Prompts

2.2.1. Diagrams

Three diagrams were created to investigate student conceptions related to groundwater and rivers (Figure 1 and Figure 2). The diagrams were modeled after common representations used in textbooks and assessments, and designed to reduce complexity (see 1.2. Learning with Diagrams). The groundwater diagram (Figure 1) contained several layers and included a confined aquifer (e.g., sandstone), confining layer or aquitard (e.g., shale), and an unconfined aquifer (e.g., gravel and sand). The diagram includes a house with a septic tank and the water table is labeled. The same diagram was used for all of the groundwater prompts (Figure 1B). The river diagrams included a curved river channel (Figure 2A) and a straight (Figure 2B) river channel. The latter was added in spring 2019 to evaluate how students’ responses for a curved river compare to their general understanding of erosion and velocity in the river. Both river channel diagrams (Figure 2A,B) were used for each of the prompts (Figure 2C).

2.2.2. Prompts

The prompts used in conjunction with the diagrams asked students to click on the diagrams to identify a structure or feature, and to predict direction or velocity of water flow (Figure 1 and Figure 2). Based on the positions of their clicks, we identified whether they responded based on scientifically accurate reasoning or naïve understandings of water flow. For groundwater, we first probed their conception of a confined aquifer and then groundwater flow (Figure 1). The latter was probed using a prompt asking students to “Click where you expect to find the pollution in a month”. We revised the pollution prompt in spring 2019 to expand the time period to “several months” to further evaluate whether students understood local flow. Local flow is the three-dimensional movement of water based on the combined impact of gravity and topography. For example, if gravity were considered alone, water would flow down. In Figure 1, the house is located on a hill and therefore the water table is higher under the hill. This will cause the water to flow laterally, from left to right, under the hill. Extending the length of time in the prompt increased the likelihood of probing students’ conception of Level 4 understanding [15]—that groundwater is influenced by topography.

The river prompts probed students’ understanding of erosion and velocity (e.g., water flow) in rivers. Prior to 2019, only the river bend (Figure 2A) questions were implemented. In spring 2019, we added the straight channel (Figure 2B) to compare students’ responses along a river bend with a more straightforward example. In a straight river channel, the greatest rate of erosion occurs along the middle of channel bed, at the location of maximum velocity and greatest force from the weight over overlying water. In a river bend, or curved river channel, the greatest rate of erosion occurs on the outside of the curve (e.g., on the right of Figure 2A), where the momentum of the flowing water will exert greater force on the river bed. Prior to 2019, the prompt asked students to: “Click where in the river you expect to find erosion”. We modified this prompt in spring 2019 to direct students to focus on the rate of erosion and bed of the river “Click in the river where you expect to find the greatest rate of erosion in the river bed”. Throughout data collection, students were first asked where they expect to find erosion and subsequently asked about velocity to minimize priming them to connect the water flow (e.g., velocity) to the geologic process (e.g., erosion).

2.3. Data Analysis

Student click data were recorded using the Top Hat (Toronto, Ontario, CA, USA) classroom response system. Within Top Hat, the gradebook allows instructors to download the coordinates of each student click. These coordinate data were then plotted and counted using ArcGIS (version 10.7.1, ESRI, Redlands, CA, USA). We created polygons in ArcGIS to denote regions of interest. The boundaries of the polygons were selected based on correctness (i.e., where the correct answer should be located), common conceptual errors documented in the literature (e.g., water table label as a distracting signal), and surrounding emerging regions of high density of student clicks. The boundary lines between regions were positioned along natural breaks, when possible, and discussed among the authors until consensus was reached. For questions with the same diagram, we kept the polygons the same across the prompts. This decision enabled us to evaluate how students’ conceptions of related processes (e.g., erosion and velocity) compare and how modifications to prompts influenced the students’ responses. Subsequently, the amount of clicks in each polygon were tallied. We ran a chi-square test to compare the counts of clicks in polygons for prompts that were modified in 2019 to include more precise language (e.g., length of time for pollution, erosion in a river bed) and to compare students’ clicks for locations of high velocity and erosion. This test was selected because it is a non-parametric test for categorical data used to evaluate whether there is a significant difference between the response pattern (e.g., correct, incorrect) for the two types of questions we studied. For the revised prompts, we tested whether the revision significantly changed the response pattern. For the velocity and erosion question, we tested whether students’ responses were coupled, indicating that they selected the same region for high velocity and high erosion. The total number of consenting student responses for each prompt is available in Appendix A. For greater detail on the methods for analyzing the coordinates of students’ clicks to evaluate their conceptual understanding, refer to LaDue and Shipley [31].

2.3.1. Groundwater

The regions of interest for the groundwater diagram were constrained by the features depicted (e.g., layers of rock, water table) (Figure 3). In addition to the geologic layers, regions of groupings of clicks included the labels for “water table” and “tank”, as well as the tank itself, depicted below the house. One challenge was whether to group clicks in the zone of aeration (Figure 3, red) and in the zone of saturation (Figure 3, orange) since both of these layers are the unconfined aquifer. In order to use the same polygons across prompts with the same diagram, we decided to make these separate regions of interest (i.e., polygons). Analysis of the prompts relating to pollution transport utilized the same horizontal layers and water table polygon as the confined aquifer question, and added polygons to capture the most common regions of interest for the flow of pollution (Figure 4 and Figure 5).

2.3.2. Surface Water

The regions of interest for the straight and curved river channels were constrained first by the shape of the river and second by the density of clicks. The river channel was split into left bank, middle, and right bank polygons, where natural breaks in the click patterns occurred (Figure 6, Figure 7, Figure 8, Figure 9 and Figure 10). To maintain consistency between questions, the same regions of interest (i.e., polygons) were utilized for all of the prompts for a particular diagram.

3. Results

3.1. Research Question 1. What Is the Pattern of Students’ Predictions about Groundwater Flow?

3.1.1. Confined Aquifers

Students were asked to identify the confined aquifer, which is the porous layer of sandstone bounded below by a non-descript layer of “bedrock” and above by an impermeable confining layer of shale. When prompted to identify the location of the confined aquifer, only 15% (n = 60) of students clicked in the correct layer for this question; the sandstone layer is the confined aquifer (Figure 3). A similar percentage selected the confining shale layer (16%, n = 61). Most students click in the unconfined aquifer consisting of the zone of saturation, the sand and gravel layer labeled in orange in Figure 4 (26%, n = 100) and the zone of aeration, the unlabeled region above the water table labeled in red in Figure 4 (8%, n = 32). The other polygons received the rest of the students’ clicks: water table labels (8%, n = 30), tank (7%, n = 29), and bedrock layer (4%, n = 17). There were seven additional clicks (2%) that fell outside of all polygons that we attributed to accidental clicks or students’ not putting effort into answering the questions.

3.1.2. Pollution Transport

Students were asked to predict where pollution leaking from a septic tank will be located in the ground after a month. To answer this question, students must reason that both gravity and topography influence groundwater flow. Therefore, we constructed a polygon representing the down-sloped plume of pollution that would be transported by the groundwater (Figure 4, dark purple). When prompted to predict the location of pollution after “a month”, 29% (n = 96) of students clicked in the correct region labeled in dark purple in Figure 4 (e.g., down-sloping plume). Most students clicked in the area directly below the septic tank, labeled in light purple in Figure 5 (33%, n = 111). Additional popular regions of interest were located near the water table labels (11%, n = 37) and uphill of the tank (9%, n = 30). Regions with fewer clicks include the unconfined aquifer (4%, n = 13); the confined sandstone aquifer (4%, n = 14); the septic tank (3%, n = 11); the confining shale layer (2%, n = 7). Additionally, 5% (n = 17) of students clicked outside of any polygon.

The clustering of higher density regions of clicks (e.g., directly below the tank and below and slightly towards the right) led to discussions amongst the researchers about the length of time included in the prompt. With only a month for pollution transport and a lack of scale in the diagram, it is impossible to distinguish between students’ understanding of the differential effects of gravity versus topography on the migration of the pollutants. Therefore, as noted previously we adjusted the prompt (Figure 1) by increasing the length of time to better distinguish between students’ conceptions of groundwater movement.

When prompted to predict the location of pollution after “several months”, 32% (n = 40) of students clicked in the correct region in dark purple in Figure 5 (e.g., down-sloped plume). A similar number of students clicked directly below the tank (26%, n = 33). Other common areas students’ clicked were the down-slope label for the water table (10%, n = 13) and the unconfined aquifer (10%, n = 13). Few students clicked in the confining shale layer (7%, n = 9), the confined sandstone aquifer (6%, n = 7), the uphill plume (5%, n = 6), or the septic tank (2% n = 2). Additionally, 2% (n = 3) of students clicked outside of any polygon. A chi-square test was conducted to examine whether there was a relationship between number of correct (down-sloped plume) versus incorrect clicks (all other clicks) and the prompt given (“one month” or “several months”). The chi-square test showed no significant relationship between accuracy and prompt, Χ² (1, N = 447) = 0.28, p = 0.597. To evaluate whether more students selected the correct region (down-sloped plume) versus the most common alternative (directly below the tank), a chi-square test showed no significant relationship between accuracy and prompt, Χ² (1, N = 281) = 1.44, p = 0.230.

In summary, pre-instruction assessments of undergraduate students indicate that students do not have a strong conceptual understanding of groundwater storage or flow. Only 15% of students correctly identified the sandstone layer as the confined aquifer shown in the diagram. Approximately one third of students (29% for the “one month” prompt and 32% for the “several months” prompt) correctly identified a region requiring water flow driven by both gravity and local topography. A similar percentage of students attribute groundwater movement only to gravity and selected a region directly downward from the pollution source (33% for the “one month” prompt and 26% for the “several months” prompt).

3.2. Research Question 2. What Is the Pattern of Students’ Predictions about Velocity and Erosion in Straight and Curved River Channels?

The results for the river questions are presented in the order that they were administered to the students and implemented in this study. We presented the erosion questions before the velocity questions in all classes to probe the students’ understanding of erosion before priming them to consider water velocity, which drives erosion patterns. The straight channel questions were added in 2017 to provide a comparison to contextualize their responses to the curved river channel prompts; therefore, we present results from the curved river channel questions first in this section.

3.2.1. Erosion and Water Velocity in a Curved River Channel

The momentum of water moving in a curved river channel is in the direction of the outside of the curve, in this case the right bank. Consequently, water will be flowing faster on the outside of the curve causing greater erosion of sediment. When prompted to predict erosion in the curved river channel, most students clicked on the left bank (44%, n = 42), followed by the right bank—the correct answer (40%, n = 38) (Figure 6). The center of the channel received the fewest clicks (16%, n = 15).

To investigate whether the simplicity of the prompt led to assumptions about erosion only occurring on the riverbanks depicted in the diagram, we modified the prompt to include “in the riverbed” (Figure 7). For this revised prompt, most students clicked within the correct polygon on the right bank (46%, n = 80) (Figure 7). Fewer students selected the left bank (39%, n = 67) and the middle area (15%, n = 26). To test whether the modified prompt changed the proportion of students with correct clicks, we conducted a chi-square test comparing accuracy of responses for the prompts (e.g., “erosion” versus “erosion in the riverbed”). Despite having more clicks in the correct region for the revised prompt, the chi-square test showed no significant relationship between the regions clicked and the prompt, Χ² (1, N = 268) = 0.970, p = 0.325.

When prompted to predict the velocity of water in the curved river channel, most students clicked in the middle of the river channel (38%, n = 102) (Figure 8). However, the students were similarly inclined to select the left bank (33%, n = 91) and the right bank (29%, n = 79). Since water velocity determines the pattern of erosions in rivers, we conducted a chi-square test to examine whether there was a relationship between accuracy of clicks (e.g., correct clicks in the right bank region versus incorrect clicks elsewhere) and the prompt given (“curved channel erosion” or “curved channel velocity”). The chi-square test showed a significant relationship between accuracy and prompt, Χ² (1, N = 348) = 13.08, p < 0.001. There were proportionally more correct clicks about erosion compared to the water velocity in a curved river channel.

3.2.2. Erosion and Water Velocity in a Straight River Channel

When prompted to predict erosion in the river bed of a straight channel, the greatest number of student clicks were on the left bank (48%, n = 83) (Figure 9). Fewer students selected the right bank (28%, n = 48) and the middle of the river channel (24%, n = 41).

When prompted to predict the highest velocity in a straight river channel, most students clicked in the middle of the river channel (78%, n = 138) (Figure 10). Fewer students selected the left bank (15%, n = 26) and the right bank (7%, n = 12). Since the highest water velocity will occur in the middle of a straight river channel, and determines the location of greatest erosion, we conducted a chi-square test to examine whether there was a relationship between accuracy of clicks and the prompt given (“straight channel erosion” or “straight channel velocity”). The chi-square test showed a significant relationship between accuracy and prompt, Χ² (1, N = 348) = 103.71, p < 0.001. There were proportionally more correct clicks about water velocity compared to correct clicks about greatest erosion in a straight river channel.

In summary, for the curved channel prompts, students were more likely to select the outside and inside of the curve as the regions of greatest erosion. However, the student clicks were more equally divided between the three locations for the region of highest water velocity in the curved river channel. The chi-square test indicated that students were more likely to select the correct region identifying erosion in the river than the correct region of highest water velocity. For the straight river channel, students identified the left bank as the region of greatest erosion but overwhelmingly selected the middle of the river channels as the region of highest water velocity.

4. Discussion

The present study leverages a classroom response system and the open-ended click-on-diagram questions to gather students’ responses to groundwater and surface water questions using simple diagrams. Investigations of students’ understanding associated with underground water storage [3,4,5,12,13,14,15] and surface water systems [7,16,32,33] reveal that students struggle with connecting components in water systems. Therefore, in the present study we focus on the processes and consequences of water flowing in the ground and in a river channel. The prompts designed for this study probed the most robust level of scientific understanding identified in the Learning Progressions for Socioscientific Systems [5]—understanding of the variables controlling water movement. The prompts also align Sadler’s [17] process mechanisms for understanding water systems. Specifically, we probed introductory college-level students’ understanding of groundwater flow driven by gravity and topographic relief, combined, and water flow in straight and curved river channels and the correlated erosion patterns. This work fills a gap in the prior literature and targets concepts related to dynamic processes associated with water flow.

4.1. Groundwater

To investigate students’ understanding of groundwater, we probed their interpretation of the term “confined aquifer”. The majority of students clicked the unconfined aquifer (e.g., gravel and sand) and an equal number of students clicked the confined aquifer (e.g., sandstone) and confining layer (e.g., shale). We attribute this distribution of clicks among the main layers in the diagram to students’ lack of knowledge about the term “confined aquifer. The students responded prior to instruction to a question using academic terminology. Many students selected the line labeling the water table. It is possible that these students were unsure of the correct response and relied on signals [28] in the diagram to direct them to important information that they thought would have a higher chance of being correct. During analysis the research team observed that, in addition to the water table labels, this diagram has some confusing or misleading information. Specifically, shale and sandstone are types of bedrock and a more accurate label for the lowest layer would be gneiss, granite, or an equally crystalline basement rock type that is found beneath sedimentary features. Additionally, the presence of the water table, while relevant to the subsequent prompts probing pollution transport, implies that there are distinct layers: one composed of sand and gravel, and one above it that is unlabeled. One reason for this decision was to remove blue color, which could imply a continuous layer of water and perpetuate a common misconception about groundwater [3,13]. Nevertheless, the students’ poor understanding of the term “confined aquifer” does not impede our ability to interpret the results of the pollution transport question.

The pollution transport questions probed students’ understanding of the relative contributions of gravity, causing downward flow of groundwater, and topography, causing lateral flow of groundwater. In the diagram, the septic tank appears on the right slope of a hill, causing groundwater to flow to the right and downward. To keep the prompt and diagram simple, we eliminated details of scale, permeability, and flow rates. Consequently, there is not one specific point that is correct, but rather the ideal region encompassing vertical and horizontal movement of water (dark purple in Figure 4 and Figure 5). We first probed students’ understanding of the location of pollution a month after the septic tank leaked for a week. The students were approximately split, with the most common responses grouping directly below the tank, taking into account gravity only, and to the right and down from the tank, the correct response taking into account gravity and topography. The research team recognized that the short time span in the question, implied volume of leak, and the lack of scale in the diagram were challenges to discerning the correct location of pollution after one month. Therefore, we added that the pollution was transported over several months to increase the likelihood that students would predict the pollution was transported over a greater distance. For the revised prompt, we have fewer data points but the students’ responses group in the same two locations as with the first prompt, with a slightly greater number of students selecting the correct plume accounting for gravity and topography. Nevertheless, these differences were non-significant and only one-third of students answered correctly. This is likely to be a high estimate when accounting for those who clicked close to the tank but by chance selected the downhill side rather than the uphill side of the tank and those who clicked in the red region, which could be correct depending on scale, permeability, and flow rates (Figure 4 and Figure 5). Overall, the pattern of students’ clicks implies that few consider that water flows laterally underground and that flow is determined based on topography. These aspects of groundwater flow are not typically taught in K-12, but since they are a major contributor to pollution transport, which impacts students’ everyday lives, we recommend that upper-level secondary and undergraduate instructors engage students with groundwater models that demonstrate these factors controlling groundwater flow. The use of models can build students’ understanding by making visible what is currently invisible to them in the subsurface, enabling the students to interpret patterns they observe in water flow.

4.2. Surface Water

The surface water questions were designed to probe students’ understanding of surface water flow and the consequences for erosion in river channels. We observed that student responses tend to cluster on the inside and outside of the river bend regardless of whether they were prompted to respond to a general prompt about where erosion will occur (Figure 6) or a more specific prompt about where erosion will occur along the river bed (Figure 7). The momentum of water in the river will cause greater erosion on the outside of a curved river channel. Surprisingly, approximately 40% of students identified the inside of the river bend as the location of greatest erosion. One possible explanation for this is that students think the river will wear away the feature sticking out into the river (e.g., the inside of the curve). However, this interpretation warrants further investigation involving open-ended questions where students can explain their reasoning. The prompt investigating students’ understanding of erosion in the straight river channel was added in 2019 to provide a comparison to the curved river channel. In this instance, students overwhelmingly chose the left bank of the river channel. A straight channel should have maximum erosion along the middle of the channel where the water volume and velocity is the greatest, with no difference in erosion on the two banks. We conjecture that students selected the left bank of the channel for two possible reasons: (1) they completed the curved channel question first where they preferred the left bank for a specific reason, and therefore we are observing ordering effects in our question bank, and/or (2) they selected the bank that is vertically lower in the diagram because they misunderstood the perspective of the diagram is a plan view and not a cross-section. The latter reason underscores the importance of communicating the intent of diagrams to students and careful design for educational materials. In future studies, we suggest two possible adjustments to the visuals may improve how they communicate to students: changing the angle of the straight river channel to be aligned with the vertical axis to avoid this potential confound or adding a callout that reveals the cross-sectional view of the river channel.

The velocity prompt was ordered after the erosion prompts intentionally to avoid priming students to consider how water velocity may impact erosion patterns. Responses to the curved and straight channel velocity questions (Figure 8 and Figure 10, respectively) reflect that students understand that the region of greatest water flow is usually the middle of a river channel. However, in a curved river channel, the greatest velocity should be off-center and closer to the outside of the bend. In Figure 8, students reveal confusion about velocity in a curved river since the majority select the middle or inside of the curve. This suggests a weak understanding of water flow in a curved channel is one source of misunderstanding of the erosion patterns in a curved river. In the straight river channel (Figure 10), students correctly identify the middle of the river channel. However, they identify the left bank of the river as the location of greatest erosion (Figure 9). This disconnect further supports the previous claim that answers to the erosion prompt are an artifact of ordering effects or the diagram features. The curious pattern of responses in the surface water questions indicates that further inquiry into students’ reasoning is necessary. Additionally, we have found this to be a challenging topic to teach in the classroom. The use of stream table models is one approach that K-12 and undergraduate instructors may use to engage students in reasoning about the cause and effect of velocity and erosion in straight and curved river channels. Additionally, engaging students in sketching may reveal their reasoning about where water will move in a curved river channel and the resulting erosional patterns. In summary, the main findings that emerged from this study of incoming undergraduate students are:

• Students struggle to identify a confined aquifer in a groundwater diagram.
• Students attribute groundwater flow to gravity, but ignore lateral flow driven by topographic relief.
• Students do not associate high water velocity with locations of maximum erosion in river channels.
• Students incorrectly identify regions on the inside of curved river channels as sites of maximum erosion.

4.3. Limitations

The present study probes students’ understanding of the physical behavior of water and its role in transport of dissolved and solid material. This is but one component of water literacy as defined by McCarroll and Hamman [6]. Although not assessed in this study, students’ local knowledge of their water infrastructure likely impacted their answers on the assessment questions. Information about students’ individual context may yield richer insights into what we have categorized as errors than we can observe from click data alone. Most other studies utilize interviews [3,13] and open-ended text response [3,5,13] data. Schwartz et al. [12] found that students’ drawings of groundwater systems were a better representation of their content mastery than their answers to constructed questions. The use of click data with diagrams in the present study introduces the dimension of diagram comprehension, adding complexity to interpretation of the results. However, we feel this approach is ecologically valid for an educational context because students are presented with and assessed on their understanding of new concepts using diagrams frequently in school settings. It is also common for information presented by scientists to the public to include diagrams when representing information related to their everyday lives (e.g., flood risk, water safety). Additionally, a limitation of our approach is the use of terminology (e.g., confined aquifer, river bed) that may be unfamiliar to the students prior to instruction. This may lead to guessing, for which errors are not indicative of underlying misunderstandings about the water processes. Future work must further probe the topics we investigated and students’ personal context through qualitative investigation that includes student elaboration of their lived experiences and mental models to disentangle the role of diagrams, terminology, and context.

5. Conclusions

The diagrams and prompts used in this study probe undergraduate students’ understanding of pollution transport in groundwater and erosion in surface water (i.e., rivers) using click-on-diagram questions with a classroom response system (e.g., Top Hat) at two large research-active universities. The present study reveals that one-third of undergraduate introductory geology students account only for gravity in the movement of groundwater in a context where topography indicates there will be a lateral component of flow. However, we are encouraged to find that almost 30% of our undergraduate students identify that both gravity and topography contribute to groundwater flow prior to instruction. The present study also demonstrates that pre-instruction, undergraduate students have a strong conception of water velocity in a straight river channel, but struggle to identify velocity and erosion in a curved river channel. Overall, we attribute errors in students’ responses to questions involving straight rivers to misunderstandings of the diagram or item ordering effects. In the curved river channel questions, students’ tendency to select the inside of the curve as a location of greatest erosion is intriguing and merits further study. Similarly, the dissociation between the selected regions of greatest erosion and greatest velocity may indicate that they do not recognize the cause and effect nature of these processes.

This study also demonstrates the opportunities and limitations of assessing student understanding of spatial phenomena in the Earth system using classroom response systems and click-on-diagram questions. These findings can guide policy and education efforts as well as offer insights to maximize the communication validity of science diagrams.