Home > Running head: LEARNING PROGRESSION FOR CARBON CYCLING
In Press
Carbon Cycle Learning Progression
In Press
Running head:
LEARNING PROGRESSION FOR CARBON CYCLING
Developing a
Multi-year Learning Progression for Carbon Cycling in Socio-Ecological
Systems
Lindsey Mohan, Jing Chen, and Charles W. Anderson
Michigan State
University
Contact Author:
Lindsey Mohan
4391 Pompano Lane
Palmetto, FL 34221
941-803-8310
941-803-8142 (fax)
mohanlin@msu.edu
Jing Chen
Teacher Education (3rd Floor)
Michigan State University
East Lansing, MI 48824
jingchen@msu.edu
Andy Anderson
319A Erickson Hall
Michigan State University
East Lansing, MI 48824
517-432-4648
andya@msu.edu
Abstract
This study reports on our steps toward achieving a conceptually coherent and empirically validated learning progression for carbon cycling in socio-ecological systems. It describes an iterative process of designing and analyzing assessment and interview data from students in upper elementary through high school. The product of our development process—the learning progression itself—is a story about how learners from upper elementary grades through high school develop understanding in an important and complex domain: biogeochemical processes that transform carbon in socio-ecological systems at multiple scales. These processes: (a) generate organic carbon (photosynthesis), (b) transform organic carbon (biosynthesis, digestion, food webs, carbon sequestration), and (c) oxidize organic carbon (cellular respiration, combustion). The primary cause of global climate change is the current worldwide imbalance among these processes. We identified Levels of Achievement, which described patterns in the way students made progress toward more sophisticated reasoning about these processes. Younger learners perceived a world where events occurred at a macroscopic scale and carbon sources, such as foods and fuels, were treated as enablers of life processes and combustion rather than sources of matter transformed by those processes. Students at the transitional levels—levels 2 and 3—traced matter in terms of materials changed by hidden mechanisms (level 2) or changed by chemical processes (level 3). More advanced students (level 4) used chemical models to trace matter through hierarchically organized systems that connected organisms and inanimate matter. Although level 4 reasoning is consistent with current national standards, few high school students reasoned this way consistently. We discuss further plans for conceptual and empirical validation of the learning progression.
Developing a Multi-year Learning Progression for Carbon Cycling in Socio-Ecological Systems
Learning Progressions for Environmental Science Literacy
We report in this article on learning progression research focusing on how schools can prepare students to be environmental science literate—that is, students and adults should have the capacity to understand and participate in evidence-based discussions about complex socio-ecological systems. 1 More specifically, we describe our progress in developing a learning progression that focuses on an important and complex domain of knowledge and practice—learners’ accounts of processes that generate, transform, and oxidize organic carbon. We report on students’ reasoning in this domain from upper elementary to high school.
Learning progressions are “descriptions of the successively more sophisticated ways of thinking about a topic that can follow one another as children learn about and investigate a topic over a broad span of time (e.g., six to eight years)” (Duschl, Schweingruber, & Shouse, 2007). They are anchored on one end by what we know about student reasoning about specific concepts entering school (i.e., Lower Anchor). On the other end, learning progressions are anchored by societal expectations (e.g., science standards) about what we want high school and college students to understand about science when they graduate (i.e., Upper Anchor). We conceive of our Upper Anchor—environmental science literacy—as discourse, as practice, and as specific knowledge.
Environmental Science Literacy
as Discourse
Our broadest definition of environmental science literacy follows Gee’s definition of literacy: “Literacy is control of secondary uses of language (i.e., uses of language in secondary discourses)” (Gee, 1991, p. 8). Gee defines a discourse as “a socially accepted association among ways of using language, of thinking, and of acting that can be used to identify oneself as a member of a socially meaningful group” (Gee, 1991, p. 3), and he distinguishes between primary discourses that we acquire in our homes and secondary discourses that we learn in other social settings.
Following Gee’s definition we see environmental science literacy in part as an “association among ways of using language, of thinking, and of acting” that is apparent in policy debates and media reports about climate change and other environmental issues. For example, the 2007 Nobel Peace Prize was awarded to Al Gore and the Intergovernmental Panel on Climate Change (IPCC) for developing reports and presentations intended to promote public understanding of scientific research on global climate change (Gore, 2006; IPCC, 2007). We discuss below our evidence that for many students these reports are incomprehensible products of an unfamiliar secondary discourse. One goal of our carbon cycle learning progression is to investigate how students make progress toward control of the secondary discourse that produces reports like these.
Environmental Science Literacy as Practice
Like all forms of literacy, environmental science literacy is embedded in verbal and nonverbal practices. Responding to environmental challenges like climate change will require collective human action on an unprecedented scale. This leads to a core question that is the basis for our research: How well prepared are our citizens to understand and respond to research on global climate change? In particular, how will people respond to scientific evidence in their actions in a variety of citizenship roles, such as consumers, learners, voters, workers, and advocates?
In keeping with our definition of environmental science literacy as mastery of a secondary discourse, we focus specifically on the scientific practices that citizens will need to play these roles. Our framework includes three key practices (each of which is actually a complex domain of practice) that are essential for responsible citizenship and that students can engage in as learners: inquiry, accounts, and decision-making in citizenship roles.
While environmental decision-making is the domain of practice that ultimately justifies our research program (see Anderson, 2008; Tsurusaki, Tan, Covitt, & Anderson, 2008), we focus in this article on accounts (explanations and predictions) as a simpler domain of practice that is central to environmental science literacy. Responsible environmental decisions are based on deliberations in which citizens use their knowledge to construct accounts that explain the situation and predict likely outcomes of different courses of action. The practice that is the focus of the carbon cycling learning progression reported here is developing and using accounts of carbon cycling to explain and predict different situations and events.
Environmental Science Literacy as Specific Knowledge
We focus on students’ accounts of carbon transforming processes in part because those accounts reveal the knowledge students have available as a resource for their practices as citizens. The global climate is changing and with this change comes increasing awareness of how the actions of human populations are altering processes that occur in natural ecosystems. The “carbon cycle” is no longer a cycle, on either local or global scales; most socio-ecological systems—especially terrestrial systems2 altered by humans—are net producers or net consumers of organic carbon. Humans have altered the global system so that there is now a net flow of carbon from forests and fossil fuels to atmospheric carbon dioxide. These changes are caused by the individual and collective actions of humans. In a democratic society like the United States, human actions will change only through the consent and active participation of our citizens, which places a special burden on science educators. Responsible environmental decision-making requires citizens to reason about complex systems. In the case of carbon cycling, this reasoning involves accounting for matter transformations that change carbon between organic forms and greenhouse gases, especially carbon dioxide.
Figure 1: Loop diagram for carbon cycling in socio-ecological systems
Human Impact: Waste from human energy use (CO2)
Human Social and Economic
Systems
Human Actions in
Roles such as:
Consumers
Voters
Workers
Learners
CO2 emissions
Atmosphere (Physical Systems)
(composition of air; atmospheric CO2)
Oxidation of organic carbon & Energy dissipation (respiration, combustion)
Generation of organic carbon & Harnessing of energy (photosynthesis)
Biosphere (Biological Systems)
Transformation of organic carbon & Passing on of energy
(biosynthesis/growth, digestion,
food chains, sequestration)
combustion
Food & Fuels
Environmental system services: Foods and fuels as the sources for energy use
Figure 1 is a Loop Diagram3 that represents the scientific knowledge we see as necessary for citizens to know about carbon cycling. It also represents how we have conceptually organized our domain of study around carbon cycling in terrestrial ecosystems. The key elements of Figure 1 are two boxes—environmental systems and human social and economic systems—and two arrows connecting the boxes—human impact and environmental system services.
Our Loop Diagram specifies that scientifically literate citizens need to be able to interpret the boxes and arrows of Figure 1 in terms of chemical models. The right-hand Environmental Systems4 box includes the familiar ecological carbon cycle, which students need to understand at multiple scales—as atomic-molecular, cellular, organismal, and ecological processes. This understanding is included in the current national standards documents (AAAS Project 2061, 1993; NRC, 1996; NAGB, 2006). It highlights carbon-transforming processes in environmental systems, as well as the process of combustion that connects environmental systems to the needs and impact of human systems. We grouped the processes into those that generate organic carbon through photosynthesis, those that transform organic carbon through biosynthesis, digestion, and food chains, and those that oxidize organic carbon through cellular respiration and combustion. We have chosen to organize the Upper Anchor around these processes because they are the means by which living and human systems acquire energy and the means by which environmental systems regulate levels of atmospheric CO2.
Although the balance has never been exact (IPCC, 2007, page 14), in natural ecosystems the processes that generate and oxidize organic carbon are roughly in balance. However, humans are now extracting large amounts of organic carbon from environmental systems as biomass and fossil fuels (Environmental System Services arrow), oxidizing it to extract chemical potential energy to support our lifestyles (Human Systems box), and returning CO2 to the atmosphere (Human Impact arrow). Making sense of the location and flow of carbon in environmental and human systems requires the ability to trace matter within the systems and processes of Figure 1. We depend on carbon compounds in biomass and fossil fuels for most of our food, energy, transportation, and shelter. The primary product of our activities—carbon dioxide—regulates global temperatures, atmospheric circulation, and precipitation. Thus, an understanding of the many processes that transform carbon compounds, and the ability to trace those compounds through systems, is central to understanding environmental processes and the human systems that depend on them.
While we advocate that school science and social studies curricula should include both boxes and both arrows, in this report we focus primarily on the part of the loop that is included in the current science curriculum: the environmental systems box. The environmental systems box includes tracing matter and tracing energy through key processes, and thus our carbon cycle learning progression is built around both matter and energy. This report focuses on the learning progression for tracing matter; we have comparable reports on tracing energy (Jin & Anderson, 2007).
Challenges in Achieving Environmental Science Literacy
We have an abundance of evidence from research studies over the last 20 years that most high school and college students do not use scientific models and principles to explain and predict carbon-transforming processes (e.g., Anderson, Sheldon, & Dubay, 1993; Driver, Squires, Rushworth, & Wood-Robinson, 1994; Hesse & Anderson, 1992; Mohan, Sharma, Jin, Cho, & Anderson, 2006; Pozo & Gomez Crespo, 2005; Wilson et al., 2006). What makes Upper Anchor reasoning difficult? Our reading of the literature and our experience in conducting the research reported in this paper suggest Upper Anchor reasoning as described above embodies a worldview—discourse, practice, and knowledge—that is profoundly different from informal reasoning and communication about environmental processes.
Our understanding of those differences has been informed by literature on the semantics of grammar (Pinker, 2007), informal cultural models (Kempton, 1995; Gee, 1996), and embodied experience (Pozo & Gomez Crespo, 2005). In particular, our reading of this literature suggests several important differences between the primary discourse that we all use for everyday accounts and the secondary discourse described in our Upper Anchor, above. People relying on informal cultural models and embodied experience do NOT trace matter through the processes identified in Figure 1. In informal accounts of these processes, matter routinely appears (e.g., when people gain weight or when trees grow) and disappears (e.g., when people “burn off” weight with diet and exercise, when dead objects decay, or when fuels are consumed by flames). These informal accounts rely on force-dynamic causation (Pinker, 2007; Talmy, 1988) in which matter enables the natural tendencies of organisms and objects to change rather than being transformed and conserved during processes of chemical change.
Thus our learning progression must describe a transition in which learners must shift from informal accounts in which events are caused by actors or agents to scientific accounts in which matter is transformed at multiple scales. Our research and our reading of the research literature suggests four components, or elements, that are present in both informal and scientific accounts. These elements of accounts change profoundly as students progress from informal to scientific accounts; by the end of the learning progression they include:
Recognizing the chemical basis of life
At a very young age children develop the idea that living and nonliving systems are governed by different rules. They recognize that living organisms have “natural tendencies” to act differently from inanimate objects—for example, living things eat and grow while dead organisms and inanimate materials do not. Thus, students using informal cultural models explain changes in organisms using the notion of vitalistic causality (e.g., organisms eat to stay alive) (Inagaki & Hatano, 2002). Young learners often do not recognize that life-sustaining processes occur at the cellular level, although typically by the end of elementary school, most children have learned about some organs in the human body, and therefore can explain life-sustaining processes localized to these parts (e.g., lungs help us breathe, heart pumps blood) (Carey, 1985). Thus, they recognize that organs have specific functions in the body, although they do not associate functions with chemical changes in materials. By middle and high school, students learn about cellular work that supports organism function, but struggle to develop descriptions for materials and functions at a cellular level (Dreyfus & Jungworth, 1989; Flores, Tovar, & Gallegos, 2003). Even though students acquire some understanding of cell functioning by high school, their ability to make sense of matter transformations in cells during metabolic processes remains challenging for most students (Canal, 1999; Driver et al., 1994; Leach, Driver, Scott, Wood-Robinson, 1996a, 1996b; Hesse & Anderson, 1992).
Identifying materials involved in systems and processes
The idea that processes in living systems and combustion involve changes in matter is critical to scientific accounts, but absent from informal accounts. Thus learners who are relying on their primary discourse to generate accounts identify forms of matter (e.g., food, fuel, air) that enable processes, but do not consider it important to describe what happens to the matter during the process.
Numerous studies (e.g., Anderson et al., 1990; Songer & Mintzes, 1994; Wiser & Smith, 2008; Zoller, 1990) document troubling gaps in young students’ and adults’ understandings of chemical substances involved in matter transforming processes. For instance, students identify few materials chemically (Johnson, 2000, 2002; Liu & Lesniak, 2006) and draw incorrect analogies between chemical and physical changes (Hesse & Anderson, 1992). They struggle particularly with explaining chemical structures of organic materials, and may default to gas-gas cycles (e.g., oxygen becomes carbon dioxide in the body) because they cannot account for all the materials involved in chemical reactions. Gases are particularly difficult, especially when solid materials are transformed into gases during chemical processes (Benson, Wittrock, & Baur, 1993; Wiser & Smith, 2008).
Reasoning about systems and processes at multiple scales
A hierarchy of systems at different scales is largely absent from informal accounts but plays a critical role in scientific accounts. The observable manifestations of carbon-transforming processes occur at a macroscopic scale, in the form of organismal growth and weight loss, decay, and burning. These macroscopic processes have atomic-molecular mechanisms and contribute to longer-term processes in large-scale systems, such as global climate change. Complex systems, such those in as Figure 1, are hierarchically organized at multiples scales, most of what are too small or too large for us to perceive directly. Students tend to focus on visible aspects of systems and do not readily use atomic-molecular accounts to explain macroscopic or large-scale events (Ben-Zvi, Eylon, & Silberstein, 1987; Hesse & Anderson, 1992, Hmelo-Silver et al., 2007; Lin & Hu, 2003; Mohan et al., 2006; Nussbaum, 1998). They do not easily maneuver the complex hierarchy that exists, even when they have verbal knowledge of systems at different scales.
Connecting carbon-transforming processes using scientific models and principles
Pinker (2007) documents how metaphors are pervasive in all our accounts of processes, including both informal and scientific accounts. Scientific models develop from informal metaphors, but they include characteristics not present in informal accounts: formal metaphors and constraining principles. We discuss ideas from the literature on each of these characteristics.
Formal metaphors. In contrast to the metaphors or cultural models that connect informal accounts, scientific accounts are characterized by formal, elaborated metaphors that explain processes in specific detail (Kempton, 1995; Gee, 1996; Pinker, 2007, pages 256-9). For example, atomic-molecular theory describes atoms as “like” particles that bond to one another, but goes on to elaborate on characteristics of atoms and how they stick to one another in great detail—far more detail than the metaphors and analogies that we use in less formal accounts.
Constraining principles. Informal accounts incorporate force-dynamic ideas about causation (Pinker, 2007; Talmy, 1988) that characterize events as resulting from the interplay between natural tendencies or capacities of organisms and objects and forces or factors that enable or oppose the expression of those natural tendencies (enablers and antagonists). These informal accounts are not necessarily constrained by scientific principles such as conservation of matter. In contrast, scientific models are constrained by a “sense of necessity.” For example, mass and individual atoms must be conserved in physical and chemical changes. Few students demonstrate this sense that there are necessary constraints in their reasoning about processes (Driver et al., 1994; Leach et al., 1996a, 1996b).
Research Goals
We know from this body of prior work that the worldview of young children is profoundly different from what we have outlined in our Upper Anchor. Therefore, progress toward the Upper Anchor is a major intellectual accomplishment. The purpose of our work is to document a progression of how students make sense of what happens to “stuff” during carbon transforming processes, which involves aspects of each of the elements discussed above (i.e., life, materials, scale, models). Four research goals guide our work:
Methods
Design
Our long-term goal is to develop and validate a learning progression that begins with levels of knowledge and practice that are typical of students in upper elementary school, defines a series of transitional levels, and culminates in the Upper Anchor defined by Figure 1. The development of the learning progression is an iterative process, as is typical of design-based research. Design-based research is used to develop design artifacts using iterative cycles of implementation and evaluation (Collins, 1992; Kelly, 2004).
Design-based research in education has typically focused on the development of instructional strategies, principles, or models as design artifacts (e.g., Brown, 1992; Cobb, Confrey, diSessa, Lehrer, & Schauble, 2003). In contrast, our work focused on developing design artifacts in the form of connected assessments and a learning progression framework. We developed an initial hypothetical framework defining Upper and Lower Anchors and transitional levels, developed assessments based on that framework, used the results of the assessments to revise the framework, which led to new assessments, and so forth. Our learning progression also differs from some other work on learning progressions in that we relied on a cross-sectional design involving students at different grade levels rather than a longitudinal design focusing on changes in individual students due to instruction.
As part of the design-based research, we developed and used criteria to help us determine how to revise and adjust the assessments and framework. The following three qualities were used to guide our theoretical and empirical validation process (from Anderson, 2008):
The results reported here reflect our findings after initial pilot work and three full cycles of framework design and assessment. While we are currently engaged in further design of the framework and assessments using teaching experiments, our report here reflects three years of study on what students know about carbon cycling in “status-quo teaching”—that is, we report on a learning progression that emerges in grades 4-12 without special instructional interventions. We feel this report not only provides an important understanding of the baseline for our plans for teaching experiments, but it provides a necessary and critical piece toward validation of a learning progression. We argue that we have developed a learning progression that is conceptually coherent and compatible with current research, and one that also documents progress toward empirical validation of our frameworks and assessments. This framework documents what may be one of several possible sequences or learning trajectories (Duschl, Schweingruber, & Shouse, 2007, Chapter 7). We discuss other possible sequences in the discussion section, below.
We used a cross-sectional design to gather assessment data, sampling from students in grades 4, 6, 7, 8, and 9-12 (i.e., high school biology) across a three-year period. While each iteration, or assessment cycle, was important to the design process, we only use data from the final assessment cycle in this report. The cross-sectional design allowed us to sample large groups of students at different grade levels at a single point in time. In looking at patterns within a particular age level, as well as patterns between age levels, we were able to determine qualitatively different types of reasoning occurring across the age groups. Furthermore, completing three assessment cycles provided evidence that despite different groups of students participating in each cycle, similar patterns still emerged among the same age levels (e.g., fourth grade students tended to give similar responses to the fourth grade students the year before with respect to the items that did not change substantially between iterations).
Participants
Michigan location. Nine teachers and 280 students participated in the written assessments, including 2 teachers and 90 students from 4th grade (i.e., elementary), 3 teachers and 80 students from 6th, 7th, and 8th grades (i.e., middle school), and 4 teachers and 110 students from 9th-12th grades (i.e., high school). Eighteen high school students from two of the high school classrooms also participated in clinical interviews. The majority of participants were recruited from Michigan public school districts, except one high school teacher and 40 of her students were recruited from a math and science center for gifted high school students. Her students attended the center for their math and science classes, but returned to their public schools for their other courses. The Michigan teachers and students could be described as belonging to rural and suburban communities, and according to diversity indicators, they were in school districts serving largely Caucasian populations (i.e., 84-95% of students were Caucasian) and working and middle class families (i.e., 20-45% of students qualified for free or reduced lunches).
Korea location. We also had one grade-6 teacher and 20 of her American students located in a Korean-based Department of Defense school. The teacher was from Michigan and had worked closely with our project previously. While we know these students and their families were from various regions in the United States, we do not have demographic information about this group of participants.
California location. We conducted interviews with 14 middle school students from two classes in a California school district (i.e., 7 students from each classroom). The students who attended the school could be described as belonging to urban and suburban areas, with the school serving families of low, middle, and upper socioeconomic classes. According to diversity indicators the school district served an ethnically diverse population with the student body composed of roughly 29% Caucasian students, 25% African-American students, 20% Hispanic students, 7% Asian students, and the remainder of students being of multiple ethnic groups or other ethnic backgrounds. Thirty-three percent of students qualified for free or reduced lunch.
Assessments
Written assessments. Written assessments contained items developed during the three-year period (2004-2007). The written assessments varied in length depending on age level, but typically included 12 to 15 open-ended questions. The total item pool included 63 items, though no individual student responded to all of them. The written items focused on what happens to matter during carbon-transforming processes. For this report we selected five written items to exemplify how students accounted for five different types of macroscopic changes: tree growth, animal growth, animal digestion and weight loss, decomposition, and burning. We also selected three written items about large scale systems and processes including: a food chain, how cutting down trees might affect climate change, how burning gasoline in a car might affect climate change. These items are described in more detail in the results and Appendix A.
In some cases teachers administered pre- and post-assessments to their students. The teachers developed their own instruction between the pre-and post-assessments, although a few teachers incorporated activities from our research group in their regular instruction. Our activities focused on principles associated with matter, energy, and scale during carbon cycling processes, and teachers may have used these activities or the principles guiding the activities to inform their instruction. When given the option, we used both pre- and post-assessments since the goal of our study was to document the range of responses from students, hoping that post-assessments might provide more sophisticated explanations compared to pre-assessments. Thus, we sought to increase, as much as possible, the diversity of responses used in our analysis, hoping that this diversity would improve the development of the learning progression levels. Thus, it was not our intention to test the effectiveness of teaching materials, but rather to support gathering as much diversity in student responses within and across grade levels.
Clinical interview. In the clinical interviews we gave students a set of cards, with each card showing a color picture and written description of a macroscopic event. The events included physical and chemical changes, but we were particularly interested in carbon-transforming processes, so the events in our analysis included: corn plant growth, a cow eating corn, a child eating a hamburger, a child running, a car climbing a hill, and a tree decaying. These events used in interviews were similar to the macroscopic events that were the focus of our analysis of written assessments. Based on how students explained the underlying matter transformation and how they classified the macroscopic events, interviewers adjusted probing questions to elicit student ideas about atomic-molecular and large-scale connections. Researchers interviewed students individually for approximately 30 minutes. Interviews were either videotaped or audio taped and then transcribed for analysis. A description of the events and initial probes are included in Appendix A.
Data Analysis
Unit of analysis. For this learning progression our unit of analysis was accounts of processes in socio-ecological systems (e.g., accounts of tree growth or photosynthesis; accounts of combustion of gasoline). In focusing on accounts we decided to look at students’ language, particularly students’ stories about different macroscopic events and processes. This unit of analysis allowed us to make comparisons among accounts of the same or similar events (and processes) for students of different ages. The accounts could take the form of narrative or scientific model-based explanations. For example, students could respond to a question about where the mass of a tree comes from using a force-dynamic narrative form about plant needs (e.g., plants need water, sun, soil, and love to stay alive) or using a scientific model-based form (e.g., plant mass comes originally from carbon dioxide and water molecules that changed into sugars and starches during metabolic processes).
Development of Levels of Achievement. We defined Levels of Achievement as patterns in learners’ knowledge and practice that extended across processes. While these levels focused on the overall pattern of tracing matter, each level included the elements described above (i.e., life, materials, scale, and models). The development of levels was a multi-step process using the following procedures.
1. Development of exemplar workbooks. Before analyzing the entirety of our data, we initially focused on developing an exemplar workbook based on responses to the written assessments. The exemplar workbook was a tool we used to distinguish between qualitatively different patterns in student responses. A sample of thirty written responses was randomly selected for each assessment item, drawing ten responses from each age band (elementary, middle, and high) and from various classrooms at those age bands. These responses were transcribed onto spreadsheets and sorted in terms of common characteristics, such as how students described and identified materials, whether or not they attempted to conserve matter, and what scales they used in their responses. We grouped and then ordered the responses from least to most sophisticated, allowing us to identify initial patterns. One or two student responses were chosen as representative examples of similar-type responses. We used the patterns and exemplar responses to suggest initial Levels of Achievement.
2. Analysis of a larger sample of written responses. After the exemplar workbook was complete, we then transcribed additional responses, for a total of 60 responses to items from the written assessments. The 60 responses were randomly selected from the entire pool of data and reflected approximately 20 student accounts from each age band. The sample represented students from each classroom, and the sample included a mix of responses on pre- and post-tests. Two researchers scored the responses using the emerging levels and exemplar workbook. The responses were scored without using identifying information for the response (i.e., scorers did not know information about age bands, teachers, or whether the responses came from pre- or post-assessments).
3. Refinement of levels and reliability checks. Since multiple researchers scored responses to written items, the first round of reliability checks involved discussion about difference between initial scores. In the first round reliability reached between 70-100% agreement for the items. Some items had slightly lower disagreement than others, so the researchers reflected on how best to revise levels to capture the types of responses to those items. The two scorers discussed other differences, and with revisions to the levels and exemplar, reached complete agreement regarding their differences. The items then received an “agreed upon” score from the two researchers. A third researcher then coded a random sample of written responses (i.e., roughly 30%). Agreement during the second round of coding reached 90% or higher depending on the item. In a third round of reliability checking, two additional researchers scored a random sample of responses and also reached 90% or higher agreement depending on the item. During the second and third round, disagreements were discussed and revisions made to the exemplar workbook and levels.
4. Coding of interview data. Analyzing interview data was an important step toward empirically validating the learning progression. We wanted to see if students’ accounts were consistent across different modes of assessments. We selected interview responses from 20 transcripts out of the 32 students’ interviewed to reflect a sample of interviews from the various age bands and classrooms (i.e., 10 interview from middle and 10 from high school, and interviews from each classroom at that grade level). The analysis of interviews followed similar procedures as the written assessments. The accounts shared during interviews were transcribed and then coded according to the emerging levels. Three researchers scored the interviews using the emerging levels and the exemplars from the written assessments, and reached 95% agreement or higher depending the event/process discussed in the interview (i.e., about 95% of the time or more all researchers agreed on the level of student response to a particular event or process). The researchers discussed disagreements, making minor changes to the levels.
Results
We present our results in four sections. First, we provide a brief comparison between the Upper and Lower anchors of the learning progressions. Then, we present four Levels of Achievement and provide examples of student responses to written and interview assessments that demonstrate key characteristics of each level. In the third part of the Results, we examine responses to two additional written items about large-scale systems. Lastly, we explore trends in levels across age groups.
Upper and Lower Anchor Points
We begin the results section by summarizing what we found at the Upper and Lower Anchor points of our learning progression. We use this brief summary of the anchor points to consider the big picture of the learning progression (the forest) before getting into the details of each Level of Achievement (the trees). For the purposes of comparison, we focus on five macroscopic events that are familiar to students at all levels: plant growth, animal growth, animal movement/weight loss, decay, and burning. Upper Anchor and Lower Anchor students differ both in how they group the accounts and in the key elements of their accounts. We have tried to capture the differences between Upper and Lower Anchor accounts in Table 1.
Table 1. Contrasting ways of
grouping carbon-transforming processes
| Upper
Anchor |
Carbon-transforming process | Generating organic carbon | Transforming organic carbon | Oxidizing organic carbon | ||||
| Scientific accounts | Photosynthesis | Biosyn-thesis | Digest-ion | Biosyn-thesis | Cellular respiration | Combus-tion | ||
| Macroscopic events | Plant growth | Animal growth | Breathing, exercise
Weight loss |
Decay | Burning | |||
| Lower
Anchor:
Informal accounts |
Natural processes in plants and animals, enabled by food, water, sunlight, air, and/or other things | Natural process in dead things | Flame consuming fuel | |||||
In Table 1, the Lower Anchor is represented in the bottom row—the Informal Accounts. This row shows that Lower Anchor learners perceive a world where macroscopic events are the result of different natural tendencies. Living things grow and move, dead things decay or rot away, and flames consume their fuel. Thus, students at the Lower Anchor reason using models that invoke notions of natural tendency and vitalism. At the Upper Anchor, however, there is a focus on chemical processes that connect systems (generation, transformation, and oxidation of organic carbon). These processes occur as matter transformations at the atomic-molecular scale (e.g., photosynthesis, combustion) and are manifested at the macroscopic scale as growth, weight loss, decay, or burning. The transition from the Lower Anchor accounts (level 1) to Upper Anchor accounts (level 4) is complex and intellectually challenging. We now shift to the trees, presenting evidence about the nature of Lower Anchor and Upper Anchor accounts, as well as evidence about student reasoning at two intermediate levels (2 and 3).
Tracing Matter Levels
We organized this section of our results to focus on core characteristics of each level. We selected five written assessment items, and corresponding interview questions, that accounted for the five macroscopic events in Table 1. The following items were selected:
In the following sections we provide an overview of each level and detailed descriptions with respect to the elements of accounts (i.e., life, materials, scale, models) that were described in the introduction. These elements are present in the accounts of students at all levels, and by documenting changes in these elements we can see the ways students change as they gain in understanding of matter and how it transforms during processes.
Level 1: Separate Macroscopic Narratives of Plants, Animals, Dead Things, and Objects
Our youngest and least scientifically sophisticated students described the world in terms of objects and events rather than chemically connected processes. Events result from the interplay between the natural tendencies of agents (including plants, animals, dead organisms, and flames) and enablers or antagonists. They saw living things—plants and animals—as separate from inanimate objects and materials, and they interpreted events primarily in terms of patterns in visible appearances. Thus, they explained processes in the lives of plants and animals such as growth, death, and decay as expressions of natural tendencies of those actors. Similarly, they saw events such as burning matches as expressions of the natural tendency of flames to consume fuels. Table 2 provides examples of level 1 written and interview responses.
Life. Level 1 students expressed the vitalistic notion that plants and animals are governed by different rules from dead things and inanimate objects and that living things were made up of different “stuff” from non-living objects. They explained that plants and animals require materials (food, water, nutrients, air) in order to sustain life, but these materials were not distinguished from other enabling conditions or forms of energy (e.g., shelter, love, exercise, sunlight). Although they were aware that plants and animals had complex internal structures (Carey, 1985) they made little use of those structures in their accounts of life processes and enablers. Moreover, their accounts of plant and animal growth focused on changes in external features of organisms, for example, explaining that plant mass comes from “leaves” and that “the weight comes from when a plant grows, the weight also grows bigger” (see Table 2). Organisms, such as the plant, gained mass because of their natural capacity for growth, expressed with the help of enablers such as sunlight, water, soil, and air.
Materials. Materials used by organisms were viewed as enablers for natural tendencies to live, grow, decay or burn. At level 1 changes in materials were explained by focusing on visible changes in objects or agents, such as “the flame moving down the [match]” or observable materials, such as the “apple shrivels and loses all moisture”. Level 1 students had not yet developed a commitment to conservation, including conservation of solids and liquids, and therefore allowed matter to appear or disappear. Gases were treated more like conditions or forms of energy such as heat and light than like “real matter”—solids and liquids (Wiser & Smith, 2008).
Scale. Students at level 1 gave accounts limited to macroscopic scale about organisms and objects. They did not yet describe or invoke hidden parts and mechanisms, such as organs, cells or atomic-molecular processes; they focused on observable mechanisms that coincided with events (e.g., exercising explains weight loss; weather, such as rain, explains decomposition).
Models. Although level 1 responses were rich with metaphors and analogies, they tended to relate the processes to familiar events in the students’ own lives, rather than explaining the processes in terms of mechanisms and contexts. Level 1 accounts were not constrained by the principle of conservation of matter, and level 1 students used everyday language rather than technical vocabulary or other specifically scientific representations.
|
||||||||||||||||||||||||||||
Level 2: Causal Sequences of Events with Hidden Mechanisms
An important characteristic of level 2 reasoning was the emergence of “hidden mechanisms” to explain macroscopic events (see Table 3 for example responses). While level 1 accounts were based on macroscopic descriptions of organisms or objects, level 2 students explained that macroscopic changes resulted from internal or invisible parts and mechanisms, such as organs (e.g., lungs, stomach), decomposers, and gases (CO2, O2). Even though level 2 accounts continued to focus mostly on observable materials going in and out of systems, their acknowledgement and use of hidden mechanisms indicated a qualitatively different approach to accounting for the macroscopic events. Yet, while level 2 students recognized that processes such as growth, breathing, decay, and digestion required and used materials, they did not account for the processes in ways that used constraining principles—conservation of mass and atoms in chemical changes. Level 2 students were mostly committed to conservation of solids and liquids, and although they recognized gases as materials, they did not attempt to conserve gases and did not treat gases as having mass (e.g., gases were not used to explain weight gain or loss).
Life. Reasoning about living things at level 2 was characterized by the notion of hidden structures, especially organs, responsible for observable changes in organisms. Level 2 students viewed organs as carrying out separate functions (e.g., lungs are for breathing, intestines are for digesting) without describing organs or cells as transforming or transporting materials inside plant and animal bodies (Inagaki & Hatano, 2002). They described eating as the process of food moving through a system of organs (especially the stomach and intestines), without attention to chemical needs of organisms or chemical make-up of the materials being digested. They described breathing as the process of moving air into and out of the lungs in order to stay alive but did not connect the materials in air to processes inside organisms.
Another important step forward for level 2 students was the recognition that organisms grow because their bodies need and use materials (e.g., soil, water, food, air). Yet, these materials were still treated as enablers for macroscopic events, but level 2 accounts do not describe the materials in food as being transformed into the materials that plant and animal bodies are made of. Level 2 students also began to recognize that dead things decay because living organisms, such as bugs and bacteria, use those materials for food. For instance, they explained that decomposing trees become smaller because “bugs eat it and live in it” (see Table 4). It was unclear, however, whether level 2 students believed bugs and bacteria to be enablers of decomposition, or active agents themselves.
Materials. Level 2 students recognized the hidden structure of materials, showing awareness that many materials were actually mixtures of other materials. Level 2 students also recognized that matter can be broken down into smaller parts that are not visible to the human eye, and these smaller parts are consequential to the events they observe. Thus, they recognized that gases as materials, and sometimes used chemical names for the most familiar ones: oxygen and carbon dioxide. They recognized oxygen and carbon dioxide as enablers or products of processes in plants, animals, and burning materials. For example, they recognized oxygen must be present for breathing and burning, but treated it as a condition, or enabler, as opposed to a material that is combined with other materials.
Level 2 students appeared to rely on two key cycles involving changes in materials as shown in Figure 2—the solid-solid and gas-gas (CO2-O2) cycles. The solid-solid cycle followed food through food chains, which later became soil and nutrients through decay that were reabsorbed by plants. The gas-gas cycle followed carbon dioxide and oxygen between plants and animals (i.e., people take in oxygen and exhale carbon dioxide, while the opposite is true for plants). The cycles were an important achievement of level 2 reasoning because the students paid attention to the flow of materials through systems at the macroscopic and “hidden” scales. The cycles were also indicative of how level 2 students still treated gases as ontologically different from solids and liquids—that is, while gases were recognized as materials, level 2 students did not see gases as having mass like other solid and liquid materials.
Animals
Plants
Carbon dioxide
Oxygen
The oxygen-carbon dioxide cycle
Decay
Plants
Nutrients, soil
Food chain
The solid-solid cycle
Figure
2: Gas-gas and solid-solid cycles at level 2
While level 2 accounts recognized that processes such as digestion, breathing, growth, decay, and combustion had material enablers—food, fuel, oxygen, etc.—and material products—waste, soil, carbon dioxide, etc—they did not have a systematic way to explain how reactants could become products. As Figure 2 indicates, the matter transformations that they recognized mostly involved solids and liquids becoming other solids and liquids or gases becoming other gases (e.g., dead plants becoming soil, match burning into smoke, plants gaining weight due to soil or water, animals breathing in oxygen and breathing out carbon dioxide).
Level 2 students used two kinds of strategies to account for solid-gas or liquid-gas transformations. First, they invoked “evaporation” to account for the disappearance of some liquids. For example, some level 2 students explained that decomposition of an apple involves evaporation of moisture or water from the apple. Some students also explained that when a gasoline tank becomes empty, the gasoline has evaporated. At level 2 students also began to use energy as an expedient means of explaining the disappearance of solids and liquids (in lieu of gases). For instance, they explained that weight loss in animals happens when fat “turned into energy” and that cars are able to run because “gasoline is being used for the car for energy”.
Scale. An important accomplishment for Level 2 students was the recognition that visible changes can be explained by hidden mechanisms (e.g., linking decomposition to bacteria, linking digestion to stomach/intestines, linking breathing to gas exchange in lungs). However, these hidden mechanisms did not include scientific cellular or atomic-molecular models.
| Table 3: Level 2 Exemplar Responses | |||
| Item | Written Response | Interview Response | Core Characteristics |
| Acorn, Corn (What contributes to plant growth?) | I think their weight comes from the soil and fertilizer because as it grows it increases in weight and fertilizer and soil are the things that make a plant grow | Corn plant has to have sunlight and it has to have water and it has to have nutrients in the ground…well I know the nutrients go up the stem, and then sunlight goes in the leaves | Plants grow from materials taken inside them, focusing on materials/conditions, such as water, soil, or sunlight. |
| Eatapple, Eatburger (What happens to things when they are eaten?) | It gets digested and it stores energy for him. | [The child] needs the energy from the cow meat in order to run. | Explain changes in materials (i.e., food) as mysterious “digestion” process to obtain energy. |
| Jared, Running (What happens to matter during weight loss and exercise?) | It turned into energy & it got burnt and came out through sweat. | Well water is being sweated out…water it’s just evaporating out the body and so it’s coming out from him. | Materials, such as food or stored fat, are used for energy transformed into observable products (e.g., sweat). |
| Applerot, Treedecay (What happens to things when they decompose?) | Decomposers are breaking it down and it is put back into soil as nutrients | Bugs eat it and live in it, so it would slowly decay after time…It forms into dirt. | Identify hidden mechanism of decomposition and decomposers, and trace observable products, such as soil. |
| Match, Gasoline (What happens to objects when they are burned for energy purposes?) | The wood burns into ash and it loses weight because it's losing mass. | Because the gas is being burned off slowly. The gasoline is being used for the car for energy and the exhaust, the rest, the by products are going into the air. | Focus on products of burning, such as “ash” or “exhaust” |
Level 3: “School Science” Narratives about Processes
Level 3 is an understanding of matter transformation observed mostly among high school students. In contrast with level 2 students who showed little awareness of chemical processes, level 3 students tried to explain both life processes and combustion in chemical terms (i.e., the hidden mechanisms observed at level 2 were replaced by mechanisms for chemical change). Level 3 accounts recognized the transformation of matter as essential to carbon-transforming processes, but their accounts were limited by their lack of understanding of chemical substances and their continued use of energy as a “fudge factor”—to account for materials that seemed to mysteriously appear or disappear. They had a general commitment to tracing matter, recognizing that the materials in objects and organisms have to come from somewhere and go somewhere, but still resorted to matter-energy conversions during chemical processes rather than solid-gas conversions. Like level 2 students, level 3 students were reluctant to attribute mass gain and loss to gases.
Life. Level 3 students recognized cells as the basic unit of structure and function in living organisms. They differentiated between processes in living organisms, including decomposers, and combustion in terms of cellular work. They explained that cells do chemical work and named atomic-molecular processes (e.g., photosynthesis, cellular respiration) as this work. In plants they accounted for generation of mass through glucose production during photosynthesis, yet they still identified water and minerals as the primary contributors to plant mass. For example, one student explained that, “[plant] weight comes mostly from H2O it receives, which it uses in its light reactions to eventually produce glucose to provide itself with energy”. This type of response recognized the chemical work of plants, but relied mostly on enablers for photosynthesis as opposed to matter and energy conservation principles. In animals they recognized that materials (i.e., food) become part of the body, and therefore are transformed into body mass, but they were unable to provide accounts that identified specific chemical substances such as carbohydrates, lipids, and proteins.
Materials. Level 3 students named some materials by chemical identity, such as CO2, O2, and glucose, when cued to think specifically about a process, but they did not identify substances that made up common foods, plants, matter in animals (i.e. proteins, lipids, and carbohydrates), or fuels. Similar to level 2, they recognized that gases were matter, but at level 3 students were more successful at conserving these during chemical change. For instance, they were able to identify gas products of exercise, decay, and combustion (e.g., “His fat was burned as carbon dioxide”; “the tree’s matter is actually decomposing, so there’s carbon dioxide being let off”; “the match is getting smaller and the CO2 is leaving”). However, they still tended to account for weight changes primarily by pointing to minor products and reactants (e.g., water contributing to weight gain in plants) or incorrectly converting matter to energy (e.g., claiming fat is turned to energy through cellular respiration). They did not trace both matter and energy through processes without confusing the two.
Scale. At level 3, cells and molecules were the basic unit for explaining macroscopic changes in organisms and objects. In living things, materials were traced through organs to cells, but explanations of cellular work were inconsistent (containing the errors described above), and even though several materials were described at the molecular level, students’ limited knowledge of chemical principles and of substances involved in processes prevented them from effectively applying atomic-molecular models.
| Table 4: Level 3 Exemplar Responses | |||
| Item | Written Response | Interview Response | Core Characteristics |
| Acorn, Corn (What contributes to plant growth?) | The weight comes mostly from H2O it receives which it uses in its light reactions to eventually produce glucose to provide itself with energy. | Plants need carbon dioxide to grow…It goes into the plants metabolism and then back out and becomes oxygen with enzymes in the plant. | Explain changes in organisms as a cellular or metabolic process, but focuses on minor materials (H2O) or gives incomplete explanation. |
| Eatapple, Cowcorn, (What happens to things that are eaten?) | It is turned into glucose and used as energy. The apple substances that are separated from the glucose are than wasted through the waste process. | Then cows eating corn are taking in the glucose bonds…taking in oxygen and letting off carbon dioxide | Digestion is linked to cellular level focusing on materials, but not explained as a cellular process or cellular work. |
| Jared, Running (What happens to matter during weight loss and exercise?) | His fat was burned as carbon dioxide. Jared went through cellular respiration more rapidly and by doing this gained ATP. He had more energy which caused him to burn more CO2 | The child is breathing in oxygen and then exhaling carbon dioxide while he’s running…cellular respiration… the body’s using them for his cellular respiration, which goes to the molecules and they make glucose | Explain changes in organisms as a cellular or metabolic process and trace gas products, but also confuse matter and energy and make other errors. |
| Applerot, Treedecay (What happens to things when they decompose?) | The stuff in the apple usually decomposes becoming other nutrients and releasing CO2 into the air. | The [materials] that are changing would be the trees matter is actually decomposing, so there’s carbon dioxide being let off into the air. | Explain changes in objects, by identifying decomposition as mechanisms for change, and tracing some gas products. |
| Match, Gasoline (What happens to objects when they are burned for energy purposes?) | The match gets lighter because the match is getting smaller and the CO2 is leaving. | The car is giving off different gases like carbon monoxide and carbon dioxide and different things like that. The oxygen level doesn’t change because the car is not using it or giving off oxygen. | Explain changes in objects by identifying key products of CO2, but does not explain process of combustion or O2 role as reactant. |
Level 4: Qualitative Model-Based Accounts of Processes in Systems
Level 4 students traced matter systematically through all of the processes mentioned in the scientific accounts of Table 1, and as described in our account of the Upper Anchor of the learning progression. We labeled this level “qualitative model-based accounts” because the accounts given by students at this level were descriptions of chemical changes that were constrained by principles, such as conservation of matter and mass, as opposed to narratives about events and enablers. In our sample very few students provided this type of account. For this reason, we developed a table of exemplar responses that included responses from science teachers and some that we developed ourselves, in addition to the ones we received from students. Table 5 summarizes the exemplar responses.
Life. Similar to level 3, level 4 students identified the cell as the basic unit of structure and function in organisms. An accomplishment at this level, however, was that students accounted for changes in organisms as cellular work, with cellular functions following chemical rules. They identified CO2 as the primary contributor to plant mass, and knew that plant cells engage in several cellular processes to construct more complex molecules from the simple sugar made during photosynthesis (e.g., plants mass comes from “several polysaccharides used for support”). Level 4 students identified cellular respiration as the means for weight loss and decay, and identified key reactants and products, and did not convert matter to energy in their accounts (e.g., “the child is taking in starches or sugars … through cellular respiration, it’s getting rid of water, it breathes it out…they expel carbon dioxide as a waste product”). Students at level 4 recognized similarities and differences between processes that transform organisms and processes that transform objects (i.e., combustion).
Materials. Level 4 students consistently identified key materials going into and out of living systems, and easily made sense of matter transformations between organic and inorganic materials. Unlike level 3, level 4 students recognized the chemical substances that make up plants and animals included lipids, carbohydrates, and protein, and substances that make up fuels were chemically similar. Although they were not completely familiar with the atomic structure of these materials, they were aware that the materials were constructed primarily of C, O, and H atoms, and contained high-energy bonds. They also knew these materials could be oxidized to obtain energy, yielding water and carbon dioxide as products (e.g., “the carbon in glucose and the oxygen from air transform into CO2 and H2O”). Importantly, they consistently identified key gas reactants and products and distinguished matter from energy during atomic-molecular processes.
Scale. Like level 3 students, level 4 students explained macroscopic changes in organisms and objects at cellular and atomic-molecular levels. Materials were identified at a molecular level and traced through cellular transformations. Level 4 students consistently used their atomic-molecular ideas and chemical models to explain changes at different scales.
|
||||||||||||||||||||||||
Large-Scale Contexts
For our next analyses, we selected two additional items from the written assessments to explore how students applied chemical models to large-scale systems. We believe the large-scale system items provide particularly interesting information about student progress toward our goals for environmental science literacy. We asked all three age groups (i.e., elementary, middle, and high) to respond to items about how cutting down trees or burning gasoline in cars would influence global warming. Example responses to these two items are provided in Table 6.
On these two assessment items, we found that many students gave level 2 accounts (24 of 60 students), and were unable to identify atomic-molecular processes related to global warming, or name chemical identities of key materials involved. The students wrote that cars produced some kind of material that is bad for the environment, and that plants take in materials to help our environment. The students who gave level 2 accounts could not identify or name carbon dioxide as a key product of combustion and reactant in photosynthesis, or as a key substance related to global warming.
Level 4 accounts on these items would have recognized that processes such as photosynthesis and combustion are important because the balance among those processes affects the location of carbon atoms in the environment—either sequestered in biomass and fossil fuels or in the atmosphere as carbon dioxide. We observed, however, that 10 of 60 students connected photosynthesis and combustion to CO2 levels, and more generally to global climate change (i.e., level 3 accounts). Although these accounts were not considered level 4 for various reasons (e.g., still relying on gas-gas cycles, uncertainty about how CO2 relates to global warming, etc), they provided evidence that some high school students recognized the key role that carbon dioxide levels play in linking these processes to global warming.
| Table 6: Large-Scale Processes | ||
| Level | Trees and Global Warming | Gasoline and Global Warming |
| 4 | Authors: Trees convert CO2 and water into organic materials such as glucose and other carbohydrates. Cutting down trees would cause higher atmospheric carbon dioxide levels because fewer plants do photosynthesis and because carbon stored in the trees and soil is released into the atmosphere as CO2. . The greenhouse gases, such as carbon dioxide, trap heat from the sun, which causes global warming. | Authors: The organic material in gasoline, such as octane, reacted with oxygen to obtain energy from the octane bonds, which produced CO2 and water. The CO2 released into the atmosphere helps trap heat from the sun causing global warming. |
| 3 | When we cut down trees it leaves a lot of CO2 in the atmosphere because there are less trees to take CO2 and make O2 with more CO2 in the atmosphere it keeps more heat on earth which is what already is causing global warming. | It is being used by the engine then it goes out the tailpipe as a fume. Yes it can because when we use gas and start up our cars it gives off CO2 and that causes global warming |
| 2 | The decrease in trees leads to a decrease in the oxygen production from plants. It changes the oxygen levels in the atmosphere, which means there are fewer gases to shield the sun's harmful rays letting more heat in causing the temperatures in our climates to rise. | It is burnt up and extracted out the exhaust into the air. The matter turns into a gas. Yes, because when the car extracts the gas as a gas into the air the gas is polluting the air and tarring the ozone layer causing more heat to come through the atmosphere. |
| 1 | Animals need trees, they are food and shelter to most animals. | The gasoline gets all burned up from the engine using it. Yes, because it puts some kind of exhaust in the air that could be harmful. |
Trends Across Age Levels
Lastly, we considered trends in Levels of Achievement across age levels using the eight written assessment items previously discussed (see Figure 3).
Figure
3: Trends across age groups
Note: 4 % of the high school
students, 12 % of the middle school students and 14% of the elementary
students did not respond or gave unintelligible answers. Therefore,
the percentages add up to less than 100%.
As Figure 3 shows, most students in our sample provided levels 1, 2, and 3 accounts. Elementary students in grade-4 were concentrated around level 1 and 2 accounts, while middle school students in grades 6-8 gave predominantly level 2 accounts. Although high school students still gave many level 2 accounts, over 35% provided level 3 accounts, and 10% explained with level 4 reasoning, meaning that almost half of the high school students were attempting to use chemical processes (more or less consistently) to explain macroscopic and large-scale events.
Discussion
Our research indicates that scientific accounts of macroscopic carbon-transforming events, such as growth, weight loss, decay, and burning, in scientific terms are major intellectual accomplishments. Students at level 1, the Lower Anchor of our learning progression, explain these events using informal accounts—what Gee (1991) refers to as their primary discourse. These accounts explain using force-dynamic reasoning (Pinker, 2007; Talmy, 1988) in which actors (e.g., animals, plants, flames) use enablers (e.g., air, water, food, fuel, sunlight) to fulfill their capacities or natural tendencies. Likewise, lower anchor accounts predict outcomes based on the notion of balance of forces between enablers that support actors in fulfilling their capacities, and antagonists, which can prevent this from happening. Although level 1 accounts recognize the existence of materials in both actors and enablers, they do not attempt to trace matter through these processes.
In contrast, level 4 accounts—our Upper Anchor—trace chemical substances through hierarchically organized systems. Level 4 accounts explain using atomic-molecular models that are constrained by chemical principles such as conservation of matter (both mass and atoms). Level 4 accounts predict outcomes by applying these models and principles to systems. At large scales, carbon cycles between gaseous (carbon dioxide) and solid or liquid (organic carbon) forms, as described in Figure 1. Level 4 reasoning is important for citizens; it is incorporated into our national high school standards and is assumed as the basis scientific reports for the general public (e.g., Gore, 2006; IPCC, 2007). Yet only about 10% of the high school students in our sample could explain these processes with level 4 accounts (Figure 3). Even in our work with college level students and practicing science teachers, level 3 responses to questions about carbon-transforming processes were more common than level 4 responses (Merritt, Wilson, & Mohan, 2008; Wilson, et al., 2006).
So how can learners get from level 1 to level 4? The intermediate levels in our learning progression—levels 2 and 3—describe one possible sequence. Level 2 reasoning is itself a substantial intellectual accomplishment because students at this level begin to delve into the hidden mechanisms (including functions of organs) underlying visible life processes, and recognize that materials taken in by living organisms enable these processes. They begin to trace matter at the organism and object scale through solid-solid and gas-gas cycles (Figure 2). Students at level 3 move further down the hierarchy of systems, recognizing that organs function through cellular processes that are chemical in nature, though their limited understanding of atomic-molecular models and organic substances usually prevents them from successfully tracing matter through cellular processes or combustion.
While the transitional levels we have described constitute one possible pathway from Lower Anchor to Upper Anchor accounts, we have reasons to doubt whether this is the only or the best possible sequence. We are troubled by the low rate at which students seem to make the transition from level 3 to level 4 reasoning (Figure 3) and we are troubled by the nature of level 3 reasoning itself. Level 3 accounts often include a lot of details about chemical substances and cell structures without developing a sense of necessity about conservation of mass and atoms. The pervasive matter-energy conversions, for example, hinder progress toward principled level 4 reasoning.
We feel that what we have captured with respect to transitional levels (particularly level 3) may reflect, in part, status quo teaching and curricular incoherence. We are currently exploring possible alternate pathways through teaching experiments at the elementary, middle, and high school levels. In our teaching experiments we wish to test whether an early emphasis on conservation principles—especially the idea that chemical changes rearrange atoms but do not create or destroy them, and that gases have mass—and scaffolded application of these principles can help students achieve principled accounts sooner and more consistently.
We would also like to explore whether it appears students make progress sooner, or more readily, in regards to certain processes, while other processes seem particularly difficult. Thus far, we have not yet demonstrated empirically that students’ accounts for one process (such as photosynthesis) are predictive of students’ accounts of other processes (such as combustion and cellular respiration). The learning progression we have described in this paper allows us to propose hypotheses for such analyses. One of the uniquely important features of learning progressions is that progress from one level to the next is not inevitable, and in the case of our progression, it is likely that progress in accounting for one process enables progress in accounts of other processes, but does not make that progress inevitable. We are currently collecting data for a calibration study that will enable us to assess our Levels of Achievement across process.
Limitations
We have made progress toward defining conceptually coherent and empirically validated learning progression that describes how students’ construct and use accounts of carbon cycling. It is necessary, however, to point out limitations in this work and steps we are currently taking to address these issues. First, in the last three years we have been working with a sample of convenience, with most teachers located relatively near to our location and involved in professional development with the local ecological research center. While sampling classrooms near to us allowed us to build relationships with teachers and visit classrooms for observations and interviews, we recognize the limitations of using a sample such as this. Although we continue to collect data from these classrooms during the fourth round of assessments, we are also sampling from other regions across the United States and in other countries, including China.
A second limitation is the nature of the items we asked. During the first three years of our study, we mainly developed open-ended assessment items, and avoided multiple-choice items, or items that asked for very specific responses. This allowed us to capture diversity in students’ accounts, but it also introduced challenges for developing items that would probe both accounts from fourth graders, as well as accounts from high school students. During the fourth round of assessments we worked to develop items that are open-ended and use non-technical language, so that elementary students can understand the questions, yet we also developed additional probes to these items, so that level 4 reasoners are encouraged to respond with as much detail as possible.
Furthermore, our learning progression is essentially “snapshots” of student accounts at different grade levels. We feel confident after the third assessment cycle that these “snapshots” capture important patterns that make up our learning progression. They do not, however, capture the progress that occurs in individual students, which would be possible with longitudinal data. As described above, we are currently conducting teaching experiments that will enable us to trace changes in a sample of individual elementary, middle, and high school students.
Conclusions
In our work, we have realized that the K-12 science curriculum does a reasonable job of getting students from levels 1 and 2 to level 3 accounts of carbon-transforming processes. By level 3 students can give relatively coherent accounts of processes in single systems and name materials involved in those processes. For passing current standardized science assessments, this level of understanding is often sufficient.
However, our research shows that good performance on standardized assessments can conceal fundamental problems with tracing matter that will diminish their understanding of the global issues that our society faces. We are asking the American public to consider profound changes in their lifestyles on the basis of arguments from scientific evidence that, according to our data, they cannot understand. We believe that many of the arguments and counter-arguments around global climate change require at least level 4 reasoning to interpret. A notable limitation for level 3 students is that they cannot consistently follow carbon through key processes, nor can they fluidly move through the hierarchy of systems to explain large-scale change using atomic-molecular accounts, both of which are essential for making sense of environmental issues involving global carbon cycling. Level 4 understanding is essential for students to evaluate evidence-based arguments and participate knowledgeably in responsible citizenship. They will not achieve this understanding without sustained, well-organized support from schools and science teachers. A conceptually coherent and empirically validated learning progression can be a critical tool for developing standards, assessments, and curricula and teaching materials that enable this change to happen.
Author Note
The authors would like to thank several people for their invaluable contributions to the work presented in this paper. We would like to acknowledge the contributions made by Hui Jin, Hsin-Yuan Chen, Kennedy Onyancha, and Hamin Baek, from Michigan State University and Karen Draney, Mark Wilson, Yong-Sang Lee, and Jinnie Choi, at the University of California, Berkeley. We would also like to thank Alicia Alonzo, Alan Berkowitz, Angela Calabrese Barton, Joe Krajcik, JoEllen Roseman, Christina Schwarz, and Carol Smith for comments on earlier versions of this manuscript.
This research is supported in part by three grants from the National Science Foundation: Developing a research-based learning progression for the role of carbon in environmental systems (REC 0529636), the Center for Curriculum Materials in Science (ESI-0227557) and Long-term Ecological Research in Row-crop Agriculture (DEB 0423627. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
References
AAAS Project 2061(1993). Benchmarks for science literacy. Oxford: Oxford University Press.
Anderson, C. W. (2008, February). Conceptual and empirical validation of learning progressions. Presented at the Meeting on Advancing Research on Adaptive Instruction And Formative Assessment, sponsored by the Center on Continuous Instructional Improvement (CCII). Philadelphia, PA.
Anderson, C. W., Sheldon, T. H., & Dubay, J. (1990). The effects of instruction on college non-
majors' conceptions of respiration and photosynthesis. Journal of Research in Science
Teaching, 27 (8), 761-776.
Ben-Zvi, R., Eylon, B., & Silberstein, J. (1987). Students’ visualization of a chemical reaction. Education in Chemistry, 24(4), 117-120.
Benson, D.L., Wittrock, M.C., and Baur, M.E. (1993). Students’ preconceptions of the nature of
gases. Journal of Research in Science Teaching, 30, 587-597
Brown, A.L. (1992). Design experiments: Theoretical and methodological challenges to creating
complex interventions in classroom settings. The Journal of the Learning Sciences, 2(2),
141-178.
Canal, P. (1999). Photosynthesis and ‘inverse respiration’ in plants: An inevitable
misconception? International Journal of Science Education, 21(4), 363-371.
Carey, S. (1985). Conceptual change in childhood. Cambridge, MA: MIT Press.
Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9-13.
Collins, A. (1992). Toward a design science of education. In E. Scanlon & T. O’Shea (Eds.), New directions in educational technology (pp. 15-22). New York: Springer-Verlag.
Covitt, B. A., Gunckel, K. L., Abdel-Kareem, H., Dudek, R., and Anderson, C. W. (2008, April). A Learning Progression for Processes that alter water quality in Socio-Ecological Systems Presented at the Annual Meeting of the National Association for Research in Science Teaching, Baltimore, MD
Dreyfus, A., & Jungworth, E. (1989). The pupil and the living cell: A taxonomy of dysfunctional ideas about an abstract idea. Journal of Biological Education, 23, 49-55.
Driver, R., Squires, A., Rushworth, P., & Wood-Robison, V. (1994). Making sense of secondary science: Research into children’s ideas. London: Routledge.
Duschl, R.A., Schweingruber, H.A, & Shouse, A.W. (2007). Taking science to school: Learning
and Teaching science in grades K-8. Washington, DC: The National Academies Press
Flores, F., Tovar, M., & Gallegos, L. (2003). Representation of the cell and its processes in
high school students: An integrated view. International Journal of Science Education,
269-286.
Gee, J. P. (1991). What is literacy? In C. Mitchell & K. Weiler (Eds.), Rewriting literacy:
Culture and the discourse of the other (pp. 3-12). New York: Bergin & Garvey.
Gee, J. P. (1996). Social linguistics and literacies: Ideology in discourses (2nd ed.).
Bristol, PA: Taylor & Francis.
Gore, A. (2006). An Inconvenient Truth: The planetary emergency of global warming and what
we can do about it. New York: Rodale.
Gunckel, K. L., Covitt, B. A., Abdel-Kareem, H., Dudek, R., and Anderson, C. W. (2008, April). A Learning Progression for Processes that Move Water through Socio-Ecological Systems Presented at the Annual Meeting of the National Association for Research in Science Teaching, Baltimore, MD
Hesse, J.J., & Anderson, C.W. (1992). Students’ conceptions of chemical change. Journal of
Research in Science Teaching, 29(3), 277-299.
Hmelo-Silver, C.E., Marathe, S., & Liu, L. (2007). Fish swim, rocks sit, and lungs breathe:
Expert-novice understanding in complex systems. Journal of the Learning Sciences,
16(3), 307-331.
Inagaki, K, & Hatano, G. (2002). Young children’s naïve thinking about the biological world. New York: Psychology Press.
Intergovernmental Panel on Climate Change (2007). Climate change 2007: Synthesis report: Summary for policymakers. Retrieved on March 7, 2008 from http://www.ipcc.ch/ipccreports/ar4-syr.htm
Jin, H., and Anderson, C. W. (2007, July). Developing a Learning Progression for Energy in Environmental Systems. Paper presented at the Knowledge Sharing Institute of the Center for Curriculum Studies in Science. Washington, D. C.
Johnson, P. (2000). Children’s understanding of substances, part 1: Recognizing chemical
change. International Journal of Science Education, 22(7), 719-737.
Johnson, P. (2002). Children’s understanding of substances, part 2: Explaining chemical change.
International Journal of Science Education, 24(10), 1037-1054.
Kelly, A. (2004). Design research in education: Yes, but is it methodological? The Journal of the Learning Sciences, 13(1), 115-128.
Kempton, W., Boster, J. S., and Hartley, J. A. (1995). Environmental values and American culture. Cambridge, MA: MIT Press
Leach, J., Driver, R., Scott, P., & Wood-Robinson, C. (1996a). Children’s ideas about ecology 2: ideas found in children aged 5-16 about the cycling of matter. International Journal of Science Education, 18(1), 19-34.
Leach, J., Driver, R., Scott, P., & Wood-Robinson, C. (1996b). Children’s ideas about ecology 3: ideas found in children aged 5-16 about the interdependency or organisms. International Journal of Science Education, 18(2), 129-141.
Lin, C.Y., & Hu, R. (2003). Students' understanding of energy flow and matter cycling in the
context of the food chain, photosynthesis, and respiration. International Journal of
Science Education, 25(12), 1529-1544.
Long Term Ecological Research Network Research Initiatives Subcommittee (2007). Integrative
Science for Society and Environment: A Strategic Research Plan. Long Term Ecological
Rsearch Network.
Liu, X., & Lesniak, K. (2006). Progression in Children's Understanding of the Matter
Concept from Elementary to High School. Journal of Research in Science Teaching,
43(3), 320-347.
Merritt, B., Wilson, C.D., & Mohan, L. (2008). Principled reasoning about biological systems:
Patterns across secondary students, undergraduates, science teachers, and university
faculty. To be presented at the annual meeting of the National Association for Research
in Science Teaching, Baltimore, MD.
Mohan, L., Sharma, A., Jin, H., Cho, I., & Anderson, C.W. (2006). Developing a carbon cycling
learning progression for K-12. Paper presented at the annual meeting of the National
Association for Research on Science Teaching (San Francisco, CA).
National Assessment Governing Board (2006). Special analysis 2006: U.S. student and adult
performance on international assessments of educational achievement. Retrieved March
7, 2008 from http://nces.ed.gov/programs/coe/2006/analysis/index.asp
National Research Council. (1996). The national science education standards. Washington, DC:
National Academies Press.
Nussbaum, J. (1985). The particulate nature of matter in the gaseous phase. In R. Driver, E.
Guesne, and A. Tiberghien (Eds.), Children’s ideas in science. Philadelphia, PA: Open
University Press.
Pinker, S. (2007). The stuff of thought: Language as a window into human nature. New York:
Penguin Group.
Pozo, J.I., & Gomez Crespo, M.A. (2005). The embodied nature of implicit theories: the
consistency of ideas about the nature of matter. Cognition & Instruction, 23(3), 351-387.
Songer, C. J., and Mintzes, J. J. (1994). Understanding cellular respiration: An analysis of
conceptual change in college biology. Journal of Research in Science Teaching 31(6),
621-637.
Talmy, L. (1988). Force dynamics in language and cognition. Cognitive Science, 12(1), 49-100.
Tsurusaki, B.K., Tan, E., Covitt, B. A., & Anderson, C.W. (2008, March). Students’ use of
family, individual, and school-based resources for making socio-ecological decisions.
Paper presented at the annual meeting of the National Association of Research in Science
Teaching, Baltimore, MD.
Wilson, C.D., Anderson, C.W., Heidemann, M., Merrill, J.E., Merritt, B.W., Richmond, G.
Sibley, D.F., & Parker, J.M. (2006). Assessing students’ ability to trace matter in
dynamic systems in cell biology. CBE Life Science Education, 5, 323-331.
Wiser, M., & Smith, C.L. (2008). Learning and teaching about matter in grades K-8: When
should the atomic-molecular theory be introduced? In S. Vosniadou (Ed.) International Handbook of Research on Conceptual Change (pp. 205-239). London: Routledge.
Zesaguli, J., Wilson, C. W., Tsurusaki, B., Wilke, B., Tan, E., and Anderson, C. W. (2008, April). The Development of a K-12 Learning Progression for Biodiversity in Environmental Systems Presented at the Annual Meeting of the National Association for Research in Science Teaching, Baltimore, MD
Zoller, U. (1990). Students' misunderstandings and misconceptions in college freshman
chemistry (general and organic). Journal of Research in Science Teaching 27(10), 1053-
1065.
Appendix A: Items used in analysis
Written Assessment Items
1. Growth of an Acorn
A small acorn grows into a large oak tree.
(a) Which of the following
is FOOD for plants (circle ALL correct answers)?
Soil
Air
Sunlight
Fertilizer
Water
Minerals in soil
Sugar that plants make
(b) Where do you think the plant’s increase in weight comes from?
2. Eating an Apple
Explain what happens to an
apple after we eat it. Explain as much as you can about what happens
to the apple in your body.
3. The Decomposition of an Apple
When an apple is left outside
for a long time, it rots.
(a) What causes the
apple to rot?
(b) Explain what
happens to the weight of an apple as it rots.
4. Jared Lost Weight
Jared, the Subway® man, lost
a lot of weight eating a low calorie diet. Where did the mass of his
fat go (how was it lost)?
5. Burning Match
What happens to the wood
of a match as the match burns? Why does the match lose weight
as it burns?
6. Trees and Climate Change
Some people are worried that
cutting down forests will increase the rate of global climate change.
Can you explain their reasoning? How could cutting down trees
affect our climate?
7. The Burning of Gasoline and Global Warming
When you are riding in a car, the car burns gasoline to make it run. Eventually the gasoline tank becomes empty.
(a) What do you think happens
to the gas? What happens to the matter the gasoline is made of?
(c) Can using gasoline in car affect global warming? How?
Interview Assessment Items
Corn Plants Growing in the Sunlight
A child running
A car climbing a hill
A tree decaying
1 The term socio-ecological systems comes from the Strategic Research Plan of the Long Term Environmental Research Network (LTER Planning Committee, 2007). It reflects the understanding of these scientists that cutting-edge ecological research can no longer be conducted without considering the interactions between ecosystems and the human communities that occupy and manage them.
2 We recognize that the oceans and aquatic ecosystems also play an important role in the global carbon balance. We chose to focus on terrestrial ecosystems for two reasons. First the fundamental principles and processes we are studying are similar in terrestrial and aquatic systems. Second, anthropogenic changes in terrestrial systems are the primary drivers of climate change.
3 Figure 1 is based on a diagram from the LTER strategic plan (LTER Network, 2007, page 11) describing the structure and function of socio-ecological systems.
4 We define environmental systems to include both natural ecosystems and ecosystems that have been substantially altered by humans, such as farms.
All Rights Reserved Powered by Free Document Search and Download
Copyright © 2011