Using Gamestar Mechanic Within a Nodal Learning Ecology to Learn Systems Thinking: A Worked Example

Introduction

We are currently witnessing a foregrounding of complexity as one of the defining characteristics of our new century. Stephen Hawking (2000) has said that we are living in the era of complexity and that complexity itself will form the science of the 21st century. Similarly, Heinz Pagel (1988) has written that those who master this science will form the economic, political, and cultural superpowers of this new century (Rambihar and Rambihar 2009). That we are living in a global era of vastly complex economic, political, and technological change may be in part why “complex” or “systems thinking” has been identified in many a current list as a critical 21^st-century skill. Though research has shown that systems thinking is a seemingly difficult skill to attain (Sweeney and Sterman 2007), in recent years game scholars (Gee 2007; Salen 2007; Zimmerman 2007) and science and engineering organizations (Federation of American Scientists 2006) have claimed that video game play and game design may be useful means through which to develop this essential skill.

I present here a “worked example” taken from a game design research study conducted in the spring of 2008 using Gamestar Mechanic, an online game intended to help middle and high school students develop basic game design skills. Game design and systems thinking skills, in this study conceptualized as dialogic in nature, were regarded as having the potential to guide learners to understand the dynamic complexity of systems of various types. The overall study focused on testing the viability of Gamestar Mechanic and the learning ecology it instantiated to improve participants’ systems thinking skills. A principal research question that guided the study was: Does a learning ecology generated and mediated by the game design software Gamestar Mechanic improve participants' ability to engage in systems thinking? A second question concerned the question of how: How did participants come to develop systems thinking skills? The worked example is made up of a set of artifacts created by one participant, Tania. The artifacts are examples of work generated as a result of designing games within Gamestar Mechanic.

An Overview of Gamestar Mechanic

Created out of a research and design collaboration between the Games and Learning Group at the University of Wisconsin-Madison and Gamelab, a New York City-based game development company, Gamestar Mechanic is an educational tool whose design, as a core part of its development, was guided by current research and theory on situated learning. In a micro sense, Gamestar Mechanic is a video game—a Flash-based software program—designed to give middle to high school-aged players a set of experiences through which they may come to develop basic game design skills. These skills include (a) designing two-dimensional games using a set of “sprites”—creatures players select to define create an interactive play space; (b) iterating from a prototype to a more complete design through a recursive process of trial and error, and feedback from other players; (c) designing play, that is, the skill of engineering sociotechnical play worlds (Salen 2007) that account for hosts of complex variables from narratives, balance of difficulty, win and loss conditions, and replayability. Iteration is built into the software as a key feature by incorporating an edit/play switch that allows players to continuously test their designs by toggling back and forth between play and edit mode (see figure 1). Players “level-up” by completing a set of play and design “jobs” intended to give the developing game designer a sense of the game’s capabilities and overall plasticity. In an incrementally challenging fashion, dozens of design and play jobs serve as a training ground, rewarding game designers with “experience points” and additional new sprites as they “level-up.”

Figure 1. Move feature, wrench tool, delete, clone and edit/play features

That every single type of sprite in Gamestar Mechanic can be modified is of special significance as player-designers have to account for large numbers of dynamic variability; making one change in their game system can lead to necessary consideration of a series of other consequential design choices. For example, one sprite, called the Chronox Sniper, has nine different parameters to choose from (such as units of health, speed, movement style, spawn rate), each having three to six options per setting (such as one to five units of health, or movement styles of random, straight, or patrol), totaling 42 different behavioral adjustments a designer can make to that sprite alone (see figure 2).

Figure 2. Sample parameter window in Gamestar Mechanic

In a more complex, macro sense, Gamestar Mechanic is an online social network that facilitates a game design community space for multiple player-designers. In essence, Gamestar Mechanic is a small world through which participants travel. At the time of this study the site reported about 600 single user accounts with which the 16 participants in this study were able to interact. As participants’ main activity is to produce games, the site’s games are the central currency of this social network, as, for example, new research would be the central currency of a professional research conference, or displaying new fashions would be the central currency at a fall fashion show in Milan. In addition, the site’s various features facilitate status-building within the community. Through a rating and comment system, designers can play, rate, and review games created by others. A “Game Alley” page showcases the top ten rated games within the community, the newest games designed, and top rated mechanics. Yet another page called “Workshop” allows players to create a watch list of favorite Gamestar Mechanic designers. The design intent here was to instantiate a unique world―a semiotic learning system―propelled by a set of values, or as Gee (2007) has a called it, an “appreciative system” of values; a system of values, as Gee has explained, which in fact exists in knowledge domains of all sorts, be it engineering, the sport of hockey, or a family.

A Brief Overview of Systems Thinking

<1)iteration: constructing, testing, receiving timely and authentic feedback, and revising of something that works;

<2)purposefulness: makes clearer the utility of learning targeted facts, concepts, and skills;

<2)modeling: can serve as vehicles for promoting model design, model building, model running, and an understanding of modeling as an investigative method.

Potential Systems Thinking Skills Afforded Through a Game Design Approach

This study focused on a particular set of systems thinking sub-skills that may be afforded as a result of playing Gamestar Mechanic as part of a larger situated set of mediating curricular experiences. This schematic of sub-skills built a framework through which to consider students’ development of these particular sub-skills. Results from the study show that these sub-skills may in fact be affordances of Gamestar Mechanic and the corresponding curriculum used for this study, and make up the distinct skill set around which the study’s assessment and research program was framed. The four systems thinking sub-skills included the following, although only the first two are discussed as part of the worked example below.

1)Understanding of system dynamics: understanding that multiple (i.e., dynamic) relationships exist within a system.

2)Understanding of feedback dynamics (i.e., reinforcing and balancing feedback loops): understanding that reinforcing and balancing feedback loops inform and can continually modify the workings of a system.

3)Understanding of the quality of relationships within a system: understanding when a system is working or not working at optimal levels.

4)Homological understanding: understanding that similar system dynamics can exist in other systems that may appear to be entirely different.

Learning as Situated Within an Ecology

It is important to note that the worked example discussed here is taken from a study for which a learning ecology composed of six learning “nodes” was designed. This nodal ecology was defined by degrees of intentional predictability and redundancy across nodes that bidirectionally connected to create an ecological learning system. Redundancy and predictability were characterized by the content particular to game design and the kinds of identities participants repeatedly stepped into―game designer, systems thinker, community member―to participate in each node. Nodes included:

1)Gamestar Mechanic;

2)a workshop format with an explicit systems thinking curriculum;

3)interactions with game designers;

4)pre- and post-tests;

5)use of Gamestar out-of-school;

6)a final game design exposition (see Torres 2009 for full description of the nodal ecology).

Hence, within this ecology, Gamestar Mechanic served as one node that interacted within a system of nodes. Each node was characterized by a set of common and cross-cutting elements: social activity, framed by constant and continual informal and formal feedback; tools such as game review protocols, storyboarding structures, Gamestar itself; specialist language (e.g., “core mechanics,” “reinforcing feedback loops”); distinct physical spaces, such as the classroom and lunchroom; norms defined early on by participants; ways of being (e.g., taking on varied identities: game designer, critic, competitor, game player); specified time allocations for each node; and, perhaps most importantly, material production. That is, it was clear that the purpose of all activity within nodes was to produce games that adhered to a set of criteria defined by the community. These nodes composed the study’s learning ecology through which participants traveled to negotiate and produce meaning about games, game design, and systems thinking.

The research study from which this worked example comes examined the learning system activated and mediated by Gamestar Mechanic through the lens of situated cognition. The theory of situated cognition framed the study’s overall design and the enquiry from which claims about participant learning were derived. Situated cognition learning theory focuses on interactive systems of activity of which the individual is only one part (Gee 2004; Lave and Wenger 1991). In this model, cognition cannot be computed in the head, but rather it is realized as a result of the interactivity of a dynamic system. These systems construct paradigms in which meaning is produced as a result of the social nature of humans and their relationships with the material world of symbols, culture, and historical elements. The structures, then, that define situated inquiry and settings are concerned with the interactivity of these elements, not with elemental components in the individual mind, such as stages of memory, storage and retrieval of information, pattern recognition, encoding, and the like (Driscoll 2005).

This approach to research and cognitive assessment took as crucial the integral nature of learning systems; that is, the relations between persons, identities, production, symbols, tools, spaces. and the systems of meaning these interconnected relations instantiated (Derry and Steinkuehler 2003; Kirshner and Whitson 1997; Lave and Wenger 1991; Rogoff 1990; Vygotsky 1978; Walkerdine 1997; Wertsch 1998). This research program took the critical position that this approach to examining learning is not only appropriate when trying to understand the nature of meaning-making in context, but crucial in a global age where young people increasingly use a wide variety of social and technological platforms to mediate and make meaning of everything they do—from communicating, constructing on and offline communities, experimenting with identities (Turkle 1984), creating and uploading online content, and participating in an open-source culture (Jenkins et al. 2006).

Bronfenbrenner’s (Bronfenbrenner and Morris 1998) bioecological systems theory and White’s (2008) social network theory provided a framework to design a nodal ecology and to make claims about the causes of cognitive change. Bronfenbrenner holds an interactionist view of human development and proposes that development is best measured and understood when units of analysis carefully account for the interactive process between person and context over time. He also argues that assessment of cognitive development that does not account for context is of little value. Contexts for Bronfenbrenner are defined as a set of nested and overlapping structures that he called micro-, meso-, exo- and macro-systems, which range in order of complexity from microsystems (such as a classroom) to macrosystems accounting for greater social systems, such as an urban setting. Cognitive development is accounted for as the result of proximal processes (Vygotsky 1978), which are short-term developmental processes by which skills are developed in locally specific contexts. Skill development is a result of activity within and between microsystems (for this study conceptualized as “nodes”). The character and predictability between these various microsystems largely drive cognitive development. For example, studies of middle class families (Barron 2006) have shown that dinner conversations at home (a type of microsystem) and learning activities in middle-class school classrooms (another type of microsystem) mirror each other and draw a level of parallel redundancy necessary for cognitive development, in this case school-related types of skills.

Paramount to my study of kids playing Gamestar Mechanic is the notion that learning happens (and should be assessed) within the context of learning ecologies composed of a kind of constellation (Goldman-Segall 1998) of nodes. The idea of cognitive development resulting from social, physical, and mental activity through the passage across nodes offers a striking departure from the design of learning environments in schools today where the only learning node available for most urban youth is often the single 45-minute Carnegie period. If learning necessarily requires travel though a system of nodes, or rather, if redundant activity and predictability across nodes determine cognitive development, then how are we to expect academically oriented cognitive change from urban students, for example, whose social and cultural capital may relegate their learning opportunities to one node (the classroom)? This central concern (of cognitive development as a process of situated travel through microsystems, netdoms (White 2008), or portals) informed the design of this study’s nodal learning ecology. Further, the study sought to investigate the potential learning ecologies hold in facilitating learning, and made claims as to how learning happened in the context of a nodal design (Torres 2009). The emerging games and learning field has not yet defined the need to redesign classroom learning spaces in this way per se, but it has advocated for the design of learning spaces that are more “game-like.” Most modern electronic games are in fact systems defined by an ecology of nodes, through which players travel (most often via a recursive process of completing challenges and ascending levels) to solve complex problems. Games also, as is the case for Gamestar Mechanic, sometimes serve as central nodes, or “generators,” as Gee (2005) calls them, activating the creation of entire ecologies of nodes, from online forums, chat rooms, and informational web sites created by users.

Overview of Research Study: Site and Participants

Sixteen middle school students (15 sixth graders and one seventh grader) who received free or reduced cost lunch participated in this study. The study, framed as a “game design workshop,” was conducted in a charter school in the New York City area from February to June 2008. I served as the primary instructor along with two assisting technology teachers employed by the school. The workshop met for a period of 35 sessions each lasting 75 minutes in duration. Eight participants attended the workshop from beginning to end. Six of these were chosen as participants of focus based on a desire to maintain a heterogeneous balance in gender, ethnicity, race, and prior academic achievement levels as evidenced by reports from teachers, participant report cards, and standardized test scores; level of English language proficiency; and consistent attendance on the part of participants.

Research Design and Method

This study employed a largely naturalistic, design-based research program with quantitative elements to allow for tabulations of participants’ pre- and post-mean scores. Design-based research attempts to investigate an intervention through a particular theoretical lens with an intent to make further claims about theory (Barab 2006; Barab and Squire 2004). A design-based research approach customarily calls for an iterative research process, allowing for flexibility to adapt or redesign research procedures, with (in the case of this study) participants and/or game designers influencing the design of the research. Pioneered ( Collins, Joseph, and Bielaczyc 2004) by Brown (1992) and Collins (1992), design-based research treats as fundamental the problem of context (Hoadley 2004) and entails both "engineering" (Cobb et al. 2003) particular forms of learning and, in a systematic and iterative fashion, studying those forms of learning within the very context defined as the means for supporting them. In this way, design-based research ideally results in a greater understanding of a learning ecology and constitutes a means of addressing the complexity that is especially characteristic of educational settings.

The Systems-Based Inquiry (S-BI) protocol designed by Sweeney and Sterman (2007) largely framed both the pre- and post-test protocol created for this study and the analysis of results. The rubric developed for the S-BI was adapted and used to assess and score levels of participants’ systems intelligence, ranging five levels from 0 (no response) to 4 (integrated use of systemic reasoning). For the purposes of this study, scores ranging from 3 to 4 signified average or above average levels of systems intelligence. Scores ranging from 0 to 2 signified below average levels of systems intelligence. Independent inter-raters were employed to score participants’ responses to the pre- and post-test questions, film treatment assignments, and think-alouds. Inter-raters scored 68 items for which they established agreement 96% of the time.

Summary of Study Results

Post-test scores and workshop work samples indicated promising results. Post-test scores indicated that five out of six participants of focus showed gains in systemic reasoning. Four demonstrated gains of 0.5 points on a scale of 0 to 4, and two demonstrated gains of 1 point, with five of the six showing overall mean scores of 3.1 or higher. In-workshop work samples showed that five of the six participants achieved systemic reasoning levels of 3.8 points or higher. An average of post-test and in-workshop scores indicated that five of the six participants established overall standings of 3.57 points or higher on the 0 to 4 scale. Most significantly, three of the six participants moved from scoring at below average levels (0-2) of systemic reasoning skills to average and above average levels (3-4). This suggests that instantiating a learning ecology mediated by a video game designed to teach middle and high school-aged students game design skills may serve to facilitate the development of systems thinking skills.

A Worked Example

This worked example is composed of artifacts created by one participant, Tania (a pseudonym). It is taken from a larger study that sought to understand the viability of creating a learning environment in which Gamestar Mechanic mediated the development of systems thinking. The worked example includes a writing sample and a concept map think-aloud, and seeks to exemplify the kind of thinking this participant was able to demonstrate at the conclusion of the study. As stated earlier, the larger study was designed to facilitate the development of four distinct systems thinking skills. This worked example shows one participant’s work as it relates to her emerging facility with two of these: 1) identifying dynamics and 2) identifying reinforcing and balancing feedback loops.

The Participant

An African-American girl, Tania was 11 at the time of study. A questionnaire asking about favorite subjects and uses of technology was given to participants at the start of the study. Tania reported on her questionnaire that her favorite subjects were reading, math, art, music and theatre, science, technology, and wellness. She also reported her computer skills as “above average.” Tania likes to play a variety of games including board games, word and card games, and sports, though she prefers to play computer games on weekends, which she does for three hours or more with friends and family. She has been playing games for about four and half years. She has Internet access at home, which she uses to communicate with friends through email. Tania’s teachers reported that she was a “low to medium” achiever, and her report card grades indicate scores of 2s and 3s in literacy and math.

At the end of the study, each participant received a “special achievement” award in an area for which they had shown special talent. Tania received hers for “innovation.” Early in the workshop, Tania asserted that she was interested in creating “games that made you relax,” as opposed to games that made you anxious. Therefore, she created games where players traveled, explored, and collected coins, instead of shooting games that involved conflict. In this way, Tania offered the workshop participants—and myself as the lead instructor—some reframing as to what a game could be. Our early critiques were driven by an inquiry into where “challenge” and “conflict” rested within games; later in the workshop, however, as will be evident in a discussion of her work below, Tania did fuse aspects of conflict into one game—but conflict encountered by her players in search of a “relaxation center.”

Systems Thinking Sub-Skill: Identifying Dynamics

This worked example shows instances of Tania’s emerging development of two systems thinking sub-skills identified for this study: (1) identifying dynamics and (2) identifying reinforcing and balancing feedback loops. The understanding of system dynamics was defined as the ability to identify when multiple (i.e., dynamic) relationships exist within a system (Forrester 1994). Five core game design concepts (core mechanics, space, rules, goals, and game components—also referred to in this study as “specialist terms” or “game design elements”) were introduced in the first days of the workshop. Without necessarily waiting for participants to show an understanding of these concepts we were interested in creating socially situated circumstances or, rather, a social practice spaces where participants “practiced” using these terms correctly or incorrectly in various situations, such as describing their games to each other in pairs or in more public moments for all participants to see. The goal was to provide opportunities for participants to develop conceptual understandings intramentally (within the individual mind) after practicing them intermentally (between individuals; Vygotsky 1978), so as to legitimize the specialist terms within the discourse of the learning ecology. To achieve this, specialist terms (or design elements) were paired (core mechanics and space, for example); then, participants—working in small groups, though each with their own computer—were asked to design a game that showed how these two elements interacted and interrelated to make the game work. At the end of each workshop session, participants volunteered to share and elicit feedback on their work, with given feedback needing to incorporate specialist terms. In essence we were learning a new language of design in situ, while also beginning to test for and learn about dynamic interactions between elements.

Two work samples are included below: (1) a “film treatment” and (2) a concept map think-aloud. All participants completed treatments in which they were to “pitch” a narrative to a film executive based on one of their games. For concept maps, participants selected one of their games and designed a map depicting the relationships between design elements in the game with particular attention to reinforcing and balancing feedback dynamics.

Work Sample 1 shows Tania’s film treatment. Before describing it further, I will note here that various “narratives” ran through aspects of the learning ecology. One of these included the narrative, “we live in systems and most systems are homologous,” meaning that systems have elemental characteristics (core mechanics, rules, space) that appear across all systems: natural, social, technological. Hence, we discussed and revisited this notion repeatedly. For instance, the workshop itself was considered a “system.” There were various spaces through which we moved (various classrooms), we performed various core mechanics (we designed games, shared games for feedback, iterated), and adhered to a set of rules of behaviors participants agreed on early in the workshop. We considered the system of the workshop social and applied the same elements to other social systems, like churches, a movie theatre, and the school they attended. We held discussions about the interconnectedness of these elements and how their interactivity resulted in the dynamic the system emitted. In this spirit, the film treatment itself was discussed as a system. Like game designers, authors “design” narratives using design elements in particular ways to attain a particular result. For example, they use elements such point of view, settings, and characters to activate a dynamic, which in turn emits a particular meaning to the reader. The film treatment, then, was itself seen as a homology within the context of other systems. While the assessment focus of the assignment was to gauge participants’ ability to design a narrative that depicted the interconnectedness and interrelationship between elements, the overall context of the assignment existed within a narrative discourse of homologies.

Work Sample 1 scored at a Level 4 for identifying dynamics within systems. A level 4 for identifying dynamics, adapted from the Sweeney and Sterman (2007) rubric, was defined:

A Level 4 response includes fuller manifestation of understanding of discreet elements within a system, understanding of the behaviors and characteristics of elements within a system, including multiple dynamic interconnections at multiples levels affecting an outcome.

Film treatments asked participants to “pitch” one of their games to a film company executive in the form of a film narrative. Film treatments were not assessed for content, style, or grammatical conventions, but rather for how well participants could use design elements to represent dynamical interactions between elements in their narratives. While this study did not set out to track literacy improvement among participants, we were interested in seeing how game design could support literacy development.

To complete this assignment, participants moved back and forth between their games and treatments, making adjustments to both in a reflection-in-action (Schön 1983), iterative fashion. In this work sample, Tania has assembled a game and film treatment that tell the story of the plight of trying to fend off her sister and her sister’s friends as she moves through seven rooms in her house (represented as seven levels in her game) to get somewhere where she can finally get some rest: the relaxation center. Tania chose the narrative of the game/film treatment and the manner in which she chose to label and create a corresponding coding scheme in her treatment. System dynamics understanding is suggested in the correlations she makes between her color-coding scheme and the treatment. The color-coding scheme throughout the film treatment shows various elements as having multiple interactional relationships, which she shows by assigning them multiple color codes. She codes her first sentence (see example below) as having four distinct interacting elements: “home” is coded in purple to indicate space; “me” is coded in blue to indicate the avatar; “little sister” is coded in pink to indicate an enemy; and “my lil sister gives me an attitude” is coded in green to indicate the sister’s core mechanic (“to give attitude”). The entire film treatment indicates various levels of dynamic interactionism as the avatar strives to reach an outcome: a place of rest. Additionally, Tania’s work demonstrates appropriate use of the specialist terms that undergirded the workshop.

Work sample 1: Film treatment and coding scheme

Systems Thinking Sub-Skill: Identifying Reinforcing and Balancing Feedback Loops

Work sample 2 shows a concept map think-aloud in which Tania explains the way in which she designed reinforcing and balancing dynamics into her game. The understanding of feedback dynamics was defined for this study as the ability to identify reinforcing and balancing feedback loops, and to show how they can inform and continually modify the workings of a system (Senge, 2006). Participants were introduced to this conceptual skill midway through the workshop by which point most demonstrated a working understanding and use of the system dynamics. I held a significant degree of concern regarding the introduction of this conceptual skill as Senge (2006) and others (Sweeney and Sterman 2000) have written about the difficulty most adults have in acquiring facility with its application. As a basic introduction, participants were given an example from the children’s book A River Runs Wild, a text that has been used to teach children how to think systemically (Sweeney, 2001). In this real life story the Nashua River had endured centuries of pollution resulting in contamination and the end of wildlife ecologies until 30 years ago when Marrion Stoddart began a movement to rescue it. Industry pollutants were described as causing a reinforcing dynamic that eventually led to the river’s once natural ability (described as a balancing dynamic) to fend off the pollutants and sustain river life. Ms. Stoddart was also described as a balancing agent who helped bring life back to the river.

For a few sessions after the introduction of the book, participants were asked to come up with and dramatize examples from “real life” in which they knew of a reinforcing and balancing dynamic happening simultaneously. To our surprise, participants were able to identify various scenarios with relative ease. One included the reinforcing and balancing dynamic between the AIDS virus and CDT-4 cells, while another depicted the greenhouse warming dynamics between carbon dioxide and the earth’s atmosphere. Participants were then challenged to select a real life situation that demonstrated a reinforcing and balancing feedback dynamic and use it to frame the narrative of a game they were to design in Gamestar. Participants had to be able to show specifically where a balancing and/or reinforcing feedback dynamic was at work in their game. Once their game design was completed, participants were asked to make a concept map of one of their games depicting balancing and reinforcing feedback loops.

Before beginning their concept map designs, various visual models appearing in engineering or behavioral research journals depicting reinforcing and balancing feedback loops (see, for example, Lane 2008) were shown and discussed with participants. Critical to demonstrating competency in this skill was a participant’s ability to make particular observations within systems of such things as patterns, cycles, causality, and feedback dynamics. Tania’s concept map think-aloud was also scored a Level 4 skill level by inter-raters for identifying reinforcing and balancing feedback loops. Level 4 (the highest level possible) was defined:

A Level 4 response includes fuller manifestation of systemic reasoning including description of a reinforcing and balancing feedback loops and includes observations, such as time delays, patterns, cycles and causality..

For concept map think-alouds participants were asked to work independently to design a map that would show the relationships between elements in a game system. Tania chose to design her concept map using the same game for which she wrote her film treatment (above). The concept map think-aloud (see transcript below) demonstrated an understanding of dynamic loops of interconnectedness. For example, Tania shows and describes how core mechanics, rules, and goals are interrelated and share a connectedness, which she identifies as a “B” balancing feedback loop, a related systems thinking skill I will discuss further below.

Observable in Tania’s response are also a number of 21st century skills as identified by the Partnership for 21st Century Skills (P21 2006). First, Tania is able to use two technological tools (Gamestar Mechanic and OminGraffle Professional) to design, innovate (around a game idea), account for, and communicate about complex dynamics within her game. Notice the complexity in Tania’s account of designing “to make a balance” between a variety of elements (core mechanics, rules, goal) that she identified as existing within her game.

Levy and Murnane (2004) have named complex communication (e.g., the ability to synthesize large amounts of information) as one of two core skills that will be required of learners in the 21st century; it is a skill rarely required of students in schools today (Spires 2008; Spires, Lee, and Lester 2008). Tania’s careful design of her concept and her synthetic explanation of her game suggest that using Gamestar Mechanic and concept maps as an assessment tool are potentially effective strategies for helping learners develop a level of “information and communication technology literacy” (defined as the ability to use technology to learn content and skills; P21 2006), as well as 21^st-century skills such as accounting for complexity, complex communication, and innovation.

Work sample 2: Concept map think-aloud and transcript

Robert:Tell us what you were thinking about when you designated the elements to where they are and the connections and all that.

Tania:First, I started with the goal, which is to get to the relaxation center. It was connected to core mechanics because the goal is to reach the goal block, and by doing that you have to shoot and travel and make the enemies disappear. The core mechanics is connected to the rules because as soon as you enter the game, what you first have to do is start shooting. Then the rules are to make sure all the enemies have disappeared, which is connected to the goal and all of these make a balance: the goal, the core mechanics, the rules all make a balance. So it's all connected.

Robert:What do you mean by "a balance"?

Tania:That it's not too hard and it's not too easy at the same time to do what you have to do.

Robert:Okay.

Tania:So from the rules it's going to be connected to the space because after you tell the rules, you have to know where you have to make the enemies disappear. From the space you're connected to the avatar so you know who you are, where you're going to be, and what you have to do. And the avatar is connected to the enemies because they're both human or they're both little characters, so they're connected to each other.

Robert:What's the connection between the avatars and the enemies? What does the avatar have to do to the enemies? I guess the enemies are all the friends?

Tania: The avatar has to make sure the enemies disappear and the avatar has to get to the goal, so you can go home and get to the relaxation center.

Robert: It has to move through the enemies, I guess.

Tania: Yes. So from the avatar to the enemy to the goal. It's basically from the rules to the space to the avatar to the enemies to reach your goal.

Robert:I see you have things creating the reinforcement feedback loop. Can you talk about that? What are the connections there and how they play out?

Tania:The enemies are connected to reinforcing feedback because they're hard to get through, because there's many of them. They start little by little and then get higher and higher and higher. That's what makes it hard. Then the space is hard because you never know where you have to go. That would be into the reinforcing feedback.

Robert:So it actually reinforces the reinforcing feedback. That's interesting.

Tania:The core mechanics are also, because it's hard to travel, it's hard to make the enemies get away from you, and shooting could be in balance, but it could also be reinforcing. The shooting can be sometimes hard, and can be easy at times.

Robert:Terrific. Thank you very much. Anything else you want to tell us?

Tania:Yes. In the balancing feedback, the rules go to it because it's easy. All you have to do is follow the rules that they tell you, and the avatar is easy to control and it's easy to know what you have to do after reading the rules. The goal is easy: just to get to the relaxation center. The core mechanics, like I said, can also be in balancing because of the shooting.

Robert:Great. Thank you very much.

Conclusion

This worked example is an attempt to show the potential Gamestar Mechanic may hold in helping young people account for systemic complexity by narrowing in on a set of systems thinking sub-skills. Claims as to how participants came to develop systems thinking skills were attributed to an intentionally designed nodal ecology that took guidance from the principles of situated cognition, such that learning is a process marked by social, technological, and symbolic negotiations and productions across predictable and redundant spaces.

There are limitations to the worked example presented here. For example, we don’t know how well Tania might have designed a concept map depicting interrelational dynamics prior to the workshop. Unfortunately, the pre-test protocol did not ask participants to create concept maps. One probe in the pre- and post-test assessed for the understanding of dynamics, however, and though Tania scored a Level 1 on her pre-test and Level 4 on her post-test, the nature of the probe (which asked participants to select a non-game system, describe its parts and the relationship between parts) is not adequately comparable to the concept map think-aloud. Still, Tania showed significant gains of one point (from a 3 to a 4 on a 0 to 4 scale) on her pre- and post-tests, and, overall, five of six participants of focus showed gains of 0.5 points or more (Torres 2009).

But the focus here should not be placed on cognitive gains alone. Today education researchers and practitioners are working in a time when the advances made in fields like the learning sciences, and more recently, in the emerging games and learning space, are barely surfacing in mainstream classrooms, let alone in education policy. Instead, NCLB’s grip has driven the country deeper into behaviorist and cognitivist-like teaching and assessment methods that have done relatively little to inspire improved change in classroom practices. To be sure, to say that learning is a complex endeavor is a gross understatement. Yet explicitly designing and iterating, continually accounting for, and making concerted efforts to understand learning processes tends not to be the focus of our schools. Instead, we deliver information, cross our fingers, and hope kids will show gains on increasingly larger batteries of tests. These types of practices are not surprising if we consider that behaviorist and cognitivist notions of learning (as a process of storing information in single minds) pervade our greater domain of public education.

The purpose of this worked example was to theoretically explicate learning as an ecologically dependent phenomenon and in so doing make general claims about how participants may have come to develop systems thinking skills. Participants didn’t learn in one node, such as in a 45-minute period, but through an ecology composed of discreet but predictable and necessarily redundant experiences. Nodes housed a set of common and cross-cutting elements, including specialist terms, specialized tools (e.g., Gamestar, game review protocols), distinct in-school and out-of-school physical spaces, behavioral norms defined by participants, and specific kinds of identities—game designer, game critic, game player, community member—that they necessarily stepped into to participate in the domain of game design instantiated by the study’s ecological learning system.

The implications of this approach to the design of school learning environments are significant if we consider, for example, how a school algebra or history class would be changed following an ecological principle. Imagine, for example, an online platform with social networking capacities where students designed and uploaded games (with all their corresponding complexity) in which players had to solve for algebraic equations. These novice mathematicians would use a set of standards and norms with which to debate—online with players across site and in school with the fellow classmates—the effectiveness of the both the game and its ability to make a solution viable; they might share their designs with people outside of school; they might invite actual mathematicians to critique their work; and they might hold special convenings in their regions or nationally. They would, in essence, establish membership within a kind of epistemic domain (Shaffer 2006) of algebra defined by a network of nodes that share in content and purpose―to innovate in the ways algebra-based games can be designed.

The overall concern with systems thinking behind this study comes at a time when understanding systemic complexity appears to be a vital skill. Researchers have indicated that this may be a difficult skill to attain (at least in Western societies), yet game scholars have begun to claim that game play and design may be useful means through which to acquire systems thinking skills. While the small sample size of the study from which the worked example is taken makes the study and the worked example exploratory, overall results are promising. It may be that the human behaviors that are among the most innate—playing and designing—may indeed be useful means through which to develop deep understandings of complexity. Tania seems a good example.

References

Assaraf, O. B., and N. Orion. 2005. Development of system thinking skills in the context of earth system education. Journal of Research and Science Teaching 42 (5): 518-560.

Barab, S. 2006. Design-based research: A methodological toolkit for the learning scientist. In The Cambridge handbook of the learning sciences, edited by R. K. Sawyer. Cambridge, UK: Cambridge University Press.

Barab, S., and K. Squire. 2004. Design-based research: Putting a stake in the ground The Journal of the Learning Sciences 13 (1): 1-14.

Barron, B. 2006. Interest and self-sustained learning as catalysts of development: A learning ecology perspective. Human Development, 49: 193-224.

Bronfenbrenner, U., and P. Morris, P. 1998. The ecology of developmental processes. In Handbook of Child Psychology, edited by W. Damon and R. M. Lerner. New York: John Wiley & Sons.

Brown, A. L. 1992. Design experiments: Theoretical and methodological challenges increating complex interventions. Journal of the Learning Sciences 2: 141-178.

Cobb, P., J. Confrey,A. diSessa, R. Lehrer, and L. Schauble. 2003. Design experiments in educational research. Educational Researcher 32 (1): 9-13.

Collins, A. 1992. Toward a design science in education. Berlin: Springer-Verlag.

Collins, A., D. Joseph, D., and K. Bielaczyc. 2004. Design research: Theoretical and methodological issues. The Journal of the Learning Sciences 13 (1): 15-42.

Derry, S. J., and C. A. Steinkuehler. 2003. Cognitive and situative theories of learning instruction. In Encyclopedia of cognitive science, edited by L. Nadel, 800-805. London, UK: Nature Publishing Group.

Driscoll, M. P. 2005. Psychology of learning for instruction, 3rd ed. Boston: Pearson Education.

Federation of American Scientists. 2006. Harnessing the power of video games for learning. Paper presented at the Summit on Educational Games, Washington, DC.

Forrester, J. W. (1994). Learning through system dynamics as preparation for the 21st century. Paper presented at the Concord Academy, Concord, MA.

Forrester, J. W. 1996. System dynamics and K-12 teachers. Paper presented at the University of Virginia Curry School of Education. Charlottesville, VA

Gee, J. P. 2004. Situated language and learning: A critique of traditional schooling. New York: Routledge.

Gee, J. P. 2005. Semiotic social spaces and affinity spaces: From the age of mythology to today's schools. In Beyond communities of practice: Language, power and social context, edited by D. Barton and K. Tusting. New York: Cambridge University Press.

Gee, J. P. 2007. Good video games + good learning. New York: Peter Lang.

Goldman-Segall, R. 1998. Point of viewing children's thinking. Mahwah, NJ: Lawrence Erlbaum.

Hammond, D. 2003. The science of synthesis: Exploring the social implications of general systems theory. Boulder, CO: University Press of Colorado.

Hanson, B. G. 1995. General systems theory: Beginning with wholes. Washington, DC: Taylor & Francis.

Hawking, S. 2000. Science in the next millennium. White House Millenium Council Speech. http://clinton4.nara.gov/ Initiatives/Millennium/shawking.html, accessed December 20, 2008.

Hmelo, C. E., D. L. Holton, and J. L. Kolodner. 2000. Designing to learn about complex systems. The Journal of the Learning Sciences 9 (3): 247-298.

Hoadley, C. M. 2004. Methodological alignment in design-based research. Educational Psychologist 39 (4): 203-212.

Jenkins, H., K. Clinton, R. Purushotma, A. J. Robison, and M. Weigel. 2006. Confronting the challenges of participatory culture: Media education for the 21st century. The John D. and Catherine T. MacArthur Foundation. http://digitallearning. macfound.org/atf/cf/% 7B7E45C7E0-A3E0-4B89-AC9C- E807E1B0AE4E%7D/JENKINS_WHITE_ PAPER.PDF (accesssed April 16, 2009)

Kirshner, D., and J. A. Whitson, eds. 1997. Situated cognition: Social, semiotic, and psychological perspectives. Mahwah, NJ: Lawrence Erlbaum.

Lane, D. C. 2008. The emergence of use of diagramming in system dynamics: A critical account. Systems Research and Behavioral Science 25: 3-23.

Lave, J., and E. Wenger. 1991. Situated learning: Legitimate peripheral participation. New York: Cambridge University Press.

Levy, F., and R. J. Murnane. 2004. The new division of labor: How computer are creating the next job market. Princeton, NJ: Princeton University Press.

Pagel, H. 1988. The dreams of reason: The computer and the rise of the sciences of complexity. New York: Simon and Schuster.

Partnership for 21st Century Skills (P21). 2006. A state leader's action guide to 21st century skills. http://www.21stcenturyskills. org/index.php?option=com_ content&task=view&id=221& Itemid=116, accessed December 10, 2007.

Perkins, D. N. 1986. Knowledge as design. Hillsdale, NJ: Lawrence Erlbaum.

Rambihar, V. S., and V. Rambihar. 2009. Complexity science may help in defining health. BMJ. http://www.bmj.com/cgi/ content/extract/338/jan09_1/ b32, accessed February 2, 2009.

Rogoff, B. 1990. Apprenticeship in thinking: Cognitive development in social context. New York: Oxford University Press.

Salen, K. 2007. Gaming literacies: A game design study in action. The Journal of Educational Multimedia and Hypermedia 16 (3): 301-322.

Schön, D. A. 1983. The reflective practioner: How professionals think in action. New York: Basic Books.

Senge, P. 1990, 2006. The fifth discipline: The art and practice of the learning organization. New York: Doubleday.

Shaffer, D. W. 2006. How computer games help children learn. New York: Palgrave Macmillan.

Spires, H. A. 2008. 21st century skills in serious games: Preparing the N generation. In Serious games, edited by L. A. Annetta. Rotterdam, Netherlands: Sense Publishing.

Spires, H. A., J. K. Lee, and J. Lester. 2008. The twenty-first century learner and game-based learning. Meridian: A Middle School Computer Technologies Journal 11 (1). http://www.ncsu.edu/meridian/ win2008/21st/index.htm, accessed January 3, 2009.

Sweeney, L. B. 2001. When a butterfly sneezes: A guide for helping kids explore interconnections in our world through favorite stories. Waltham, MA: Pegasus Communications.

Sweeney, L. B., and J. D. Sterman. 2000. Bathtub dynamics: Initial results of a systems thinking inventory. Systems Dynamics Review 16 (4): 249-286.

Sweeney, L. B., and J. D. Sterman. 2007. Thinking about systems: Student and teacher conceptions of natural and social systems. System Dynamics Review 23 (2/3): 285-312.

Torres, R. J. 2009. Learning on a 21st century platform: Gamestar Mechanic as a means to game design and systems thinking within a nodal ecology. Unpublished dissertation, New York University, New York.

Turkle, S. 1984. The second self: Computers and the human spirit. New York: Simon & Schuster.

Vygotsky, L. 1978. Mind in society. Cambridge, MA: MIT Press.

Walkerdine, V. 1997. Redefining the subject in situated cognition theory. In Situated cognition: Social, semiotic, and psychological perspectives, edited by D. Kirshner and J. A. Whitson (pp. 57-70). Mahwah, NJ: Lawrence Erlbaum.

Wertsch, J. V. 1998. Mind as action. New York: Oxford University Press.

White, H. C. 2008. Identity and control: How social formations emerge. Princeton, NJ: Princeton University Press.

Zimmerman, E. 2007. Gaming literacy. Harvard Interactive Media Review 1 (1): 30-35.