Embodied Games, Next Gen Interfaces, and Assessment of High School Physics

Abstract

In this worked example we present ongoing research in the realization and evaluation of a new mixed-reality learning environment called SMALLab. Within SMALLab, students interact in real time with each other and with dynamic visual, textual, physical, and sonic media through full-body 3D movements and gestures. The environment fosters embodied and multimodal learning in a manner that brings together contemporary research in the learning sciences and human-computer interaction. The need for new approaches to science education and a recent study of SMALLab learning in a high school physics classroom are presented. We describe a game-based scenario for learning about constant velocity. We present an assessment framework that integrates a variety of measures to provide a broad view of SMALLab-facilitated learning in a formal school context. The primary focus of this study is to explore the impact of SMALLab learning on representational fluency. Results suggest that embodied activity in SMALLab scenarios with multiple representations (i.e., representing physics constructs graphically, algebraically, verbally, etc.) is strongly related to better performance on more traditional measures of representational fluency. The current study is one component in a longitudinal examination of the efficacy of embodied SMALLab learning.

What is SMALLab?

SMALLab is a mixed-reality environment developed by a collaborative team of researchers including those in education, psychology, interactive media, computer science, and the arts. By “mixed-reality,” we mean that there is an integration of physical manipulation objects, 3D physical gestures, and digitally-mediated components where the physical body functions as an expressive interface . Within SMALLab, students use a set of “glowballs” and wireless peripherals to interact in real time with each other and with dynamic visual, textual, physical, and sonic media through full-body 3D movements and gestures. For example, working on a spring physics scenario, students are immersed in a complex physics simulation that involves multiple sensory inputs to engage student attention. They can hear the sound of a spring picking up speed, see projected bodies moving across the floor, feel a physical ball in their own hands and integrate how the projected ball moves in accordance with their own body movements enabling them to construct a robust conceptual model of the motion system.

SMALLab is a highly collaborative space at all levels of design and implementation. It builds upon prior work in the domain of social computing interfaces in that participants can freely enter and exit the space without the need for wearing specialized display or sensing devices such as head-mounted displays or motion capture suits. As shown in Insert Figure 1 <figure1.jpg>, participants seated or standing around SMALLab can readily see and hear the dynamic media, and they can directly communicate with their peers in the active space. As such, SMALLab establishes a porous relationship between the mediated space and the larger physical learning environment.

Figure 1. Whole class interaction within and around SMALLab.

Physically, SMALLab is an open cube-shaped space with the following sensing and feedback equipment: a 3D-object tracking system, a top-mounted video projector providing real time visual feedback, four audio speakers for surround sound feedback, and an array of tracked physical objects (“glowballs”). A networked computing cluster with custom software drives the interactive system. In past work our team has deployed SMALLab in a series of pilot regional school and museum programs.

Current state of High School physics. In the domain of physics education, there can be a persistent gap between instructional goals and student achievement. For example, research by McDermott (1984) reveals that in many cases, students are able to generate computational solutions to difficult physics problems, but they are unable to demonstrate a basic conceptual understanding of the underlying laws. Similarly, a study by Hestenes and colleagues (1992) demonstrates that conventional approaches to physics education that rely heavily on textbook-based computational methods of instruction have strikingly little impact on students’ understanding of physics concepts . There is clear evidence of the positive impact of active and cooperative strategies for physics learning . However, pressures of time, space, and logistics often mean that these approaches are neglected in traditional classrooms.

In 2003, the American Association for Physics Teachers published a report that proposes a number of revisions to the teaching of physics . Importantly, the report includes references to a number of initiatives that rely upon mediated approaches that are implemented as desktop or browser-based applications. While these mediated scenarios do facilitate new modes of experimentation, given the nature of their interfaces, they greatly reduce the positive impacts of physically embodied scenarios, and do not encourage direct collaboration among students. We believe that SMALLab addresses these issues.

The National Science Education Standards have embraced “science as inquiry” as a central component of classroom learning. Inquiry is a practice that shapes learning through creativity, curiosity, reflection, and the scientific process. It has been demonstrated to advance students’ ability to problem-solve, communicate, construct and apply scientific models, and think critically . The Modeling Instruction in Physics program provides a well-defined and evidence-supported approach to inquiry-based teaching and learning. In addition, the program defines lab activities and instructional methods that are designed to foster student-centered, collaborative inquiry.

Representational fluency. One of the key tenets of Modeling Instruction (and many states’ curricula) is the notion that students work to construct knowledge by moving through a variety of representations of data. Modeling Instruction in Physics provides diagrammatic, graphical, and algebraic representations of each concept that students will encounter. An example of a diagrammatic representation of movement is a motion map: a one-dimensional plot of position marks coupled with arrows that indicate the direction and magnitude of motion at a series of moments during a movement sequence. Graphically, this same movement sequence can be represented by a two-dimensional graph of position vs. time. Algebraically, a movement sequence where velocity is constant can be accurately represented as an equation in slope-intercept form (i. e., y = mx + b where m is the slope and b is the y-intercept of the line).

Figure 2. As student moves physically in the real world, multiple abstract representations are displayed in SMALLab in real time.

In the example discussed here, we have partnered with a high school Modeling Instruction teacher to co-develop a set of new physics-based SMALLab scenarios that support, enrich, and extend the existing curriculum and materials. We endeavor to do so by leveraging two capabilities of the learning environment. First, we take advantage of SMALLab’s ability to generate multimodal representations of data including both visual and sonic data. Second, we take advantage of the embodied nature of interaction in the media environment whereby students use full-body 3D movements. Thus, students can rapidly shift from creating real world kinesthetic movements to a variety of representations of their movements, and back again. They work as a whole group to collectively build a robust conceptual understanding of how the abstract representations correspond to their actions in the real world. The working hypothesis is that in many instances embodied and multimodal learning leads to deeper understanding and better retention (e.g., reading comprehension math performance ]; multimodal learning ).

Figure 3. Velocity arrows are plotted on a Motion Map diagram as the student moves.

The constant velocity challenge. In SMALLab, students are introduced to motion maps and graphs one element at a time. (caption) Figure 3 shows a student using the physical glowball to create a Motion Map diagram. (Caption) Figure 4 is a detailed “screenshot” of what the floor in SMALLab might look like after a pair of students has completed the first stages of play. Along the top edge are two Motion Map (one dimensional) diagrams; along the left side is a two-dimensional graph of position vs. time. Toward the right is a series of one to four slope-intercept equations. Near the center is a simplified clock face that indicates the progression of time during the interaction.

Figure 4. Screenshot of Creation Stage

Three devices are used during the interaction. Two glowballs are tracked in real time and are used by the participants to create movement sequences. With the completion of each clock cycle (approximately one second), the position and velocity vector of each glowball along the one-dimensional Motion Map axis is stored and plotted. At the same time each position is plotted on the position vs. time graph, and a slope-intercept equation is computed from the linear regression of the points. A variety of buttons on a wireless gamepad provide control over starting, stopping, resetting, and advancing through the game stages.

There are four stages to the Constant Velocity Challenge Game: pregame, movement creation, movement matching, and movement comparison. . In the pre-game stage students divide into “movement creator” and “movement challenger” teams and prepare for the activity. The game can be played with either two or four players. The challenger students leave the classroom. During the movement creation stage, a single student (or pair of students) devises a movement sequence with constant velocity. These students physically move (walk at a constant rate) through the space with the glowballs for a total of ten clock cycles. Each time the cycle completes, a point is plotted for the green glowball; if students are working in pairs, a point is plotted separately for the orange glowball. Students can make multiple attempts at line creation and can reset and perform their movement sequences until they are satisfied with the result. In the movement matching stage the challenger students are invited back to the classroom. They are prompted to specify which of three data representations they would like to see: motion map, graph, or equation. Once specified, only this one representation is visible.

The challenger students analyze the representation, plan a movement sequence, and then attempt to match the representation of their choice. They can reattempt the matching process an indefinite number of times. The final stage is the movement comparison. Here, as shown in (Caption) Figure 5, the creator and challenger sequences are overlaid upon one another. The entire class, having observed the process, now participates in an analysis and discussion regarding the successes and failures of the matching game process. A team of students are assigned the task of identifying the winner by computing a weighted average of the resultant slope-intercept equations. If the challengers are able to match the original motion sequence within a certain tolerance, they win. If not, the creators win the round.

Figure 5. Floor display of Creator and Challenger movement sequences with two players.

Importantly, the students themselves have a great deal of control over the difficulty of the task. The creator students can create more difficult or tricky slopes depending on their starting positions, direction of movement, and rate of movement. The challenger students have the opportunity to choose the data representation they wish to match. This is usually the one that makes the most sense to them. The Motion Map is the most direct translation of movement into data, and thus easiest to analyze and reproduce. By contrast, the algebraic representation is the most abstract, and it appeared to be the most difficult for most students. It was the least chosen challenger representation.

Method

We have several questions about how SMALLab affects learning and have given this first cohort of students multiple measures over the semester. For the purposes of this worked example, we would like to focus on how prior knowledge, representational fluency, spatial skills, and performance in SMALLab interact.

Participants. The participants are 21 eleventh graders in an urban high school. On any day there are a number absent. The class is 62% male. There are seven Hispanic students, two Asian and one African-American, the remaining students are White.

Measures. We present five tests with relevance to the velocity matching scenario.

1) Readiness to Learn. This is one of six dimensions on the Views about Science Survey (VASS) developed by Halloun . It assesses whether students are low or high in their beliefs that, for example, a) science is learnable by anyone willing to make the effort and not just the purview of a few talented people, and b) “achievement depends more on personal effort and perseverance than on the influence of teacher, peers, or textbooks” (p. 21).

2) Unit Pretest. The Modeling Curriculum package contains several unit tests and we report the pretest score before students begin the unit on constant velocity.

3) The Card Rotation Task from the ETS factor kit . This subtest was given at the beginning of the school year to address the question of whether being in SMALLab helped facilitate spatial skills. It is a forced-choice, timed paradigm where students pick whether a second rotated card matches the first criterion card.

4) Representational Fluency. Students were given a text description of a moving car and were asked to describe the motion of a car with as many representations as possible (e.g., diagrammatically, verbally, graphically, with formulae, etc.).

5) Slope Match. There were two SMALLab sessions where students were able to play the Velocity Match Game. A linear regression line was computed for the position vs. time points generated by the student movement. Using this regression line, for each challenger, we computed the absolute distance the challenger was from the creator’s a) intercept and b) slope. We summed these differences and then averaged them over a play session if a challenger decided to try several times to match the representation. Error is reported here, thus, the larger the number the worse the match. To compute the error in the second game, we used a weighted average of the absolute difference between the creator and challenger slopes and y-intercepts (E = error; S_ch = challenger slope; S_cr = creator slope; Y_ch = challenger y-intercept; Y_cr = creator y-intercept).

E = |2.0(S_ch – S_cr) + 0.1(Y_ch – Y_cr)|

The slope component was given greater weight as this feature of the movement is most critical in assessing student understanding of constant velocity. The two types of scores are very similar.

Procedure. The Constant Velocity Challenge is composed of four stages: a) pregame, b) movement creation, c) movement matching, and d) movement comparison. We have prepared video segments that demonstrate the rich interactions during the final three stages. These are available via the videos below, and also at the SMALLab web site . Due to privacy issues these videos primarily show reenactments of the real students in the game.

Results

Table 1 presents the correlations between the five variables. It is important to note that on any day several students would be gone and/or not have been active in SMALLab, given the classroom time constraints. Thus, the number of participants in the tests/scenarios below ranges from nine to 18. To obtain significant inferential and correlational results, it is important to have statistical power and this depends (among other factors) on the number of participants. Clearly, this is a pilot study with one class of students and traditional significance levels should not be expected. We present these correlations with the caveat that they represent only this sample, and we do not make claims about generalizations and populations. Because one would not expect traditional significance with these n’s, we report the Pearson correlation and the variance accounted for in the bivariate correlations. The conventional medium effect size in a population would be an r of .30 . In Table 1 the correlations over .30 are highlighted.

Table 1. Correlations for variables of interest; variance accounted for in sample is in parentheses.

Note: The n ranges from nine - 18. Variance greater than 10% is highlighted.

One of the questions we had was whether being active in SMALLab affected the student’s ability to think in multiple representations. We coded the students as either being active in SMALLab over the two sessions or not. Recall that even though several of the students did not get the opportunity to be active inside SMALLab (walking with the glowballs to recreate the representations), they were able to observe and participate in discussions during the creation/challenge process. The 11 students active in SMALLab generated an average of 4.10 (SD=.94) representations on the hard copy test. The seven students who were not active generated an average of 3.14 (SD=1.07) representations. A t test revealed this difference to be a statistical trend, t ₍₁₆₎ = 1.97, p = .066, using a two-tailed alpha. The correlation was .44, and this approaches a large effect. Thus, we conclude that being active in SMALLab scenarios with multiple representations is strongly related to better performance on traditional measures of representational fluency.

We are also interested in issues surrounding graphical representation of individual performance and how a student’s profile can change over time. Because the study is ongoing, we present graphs for students at Time 1 only. Further comparisons will be made at the end of the school year. For the five variables listed in Table 1 we derived the z scores for each measure and mean-deviated those by three. This results in only positive scores. Figure 6 shows two students with somewhat different profiles.

Figure 6. Student A (top) and B (bottom) adjusted z scores on five measures (SMALLab error refers to performance on the velocity match tasks; Max. score = 6.0).

The constructs on the left are more causal and distal from the SMALLab task. The dark green circles represent the (perhaps) inherent individual differences that the students come to the class possessing. The lighter green-to-blue constructs are more immediate or proximal—these are gathered multiple times during the year with the variable measures. The major discrepancy between profiles is seen in the students’ representational fluency scores. We are conjecturing here, but perhaps Student A, with a higher Readiness to Learn score was more amenable to the SMALLab experience. That is, the student possessed an intellectual openness that facilitated learning fluencies in representations. This profile as a whole is compared to that of Student B, who actually performed somewhat better on the paper-and-pencil, distal tests measuring more “crystallized” skills (e.g., spatial card rotation and prior knowledge of physics concepts on the pretest), but Student B performed comparatively worse on the representational fluency task.

One of our goals is to create reports that are easy to parse and that teachers themselves will be able to immediately put to use. These highly visual profiles present a more multidimensional, nuanced understanding of a student’s capabilities. This method also holds promise for tracking learners as they (hopefully) move along a novice-to-expert continuum. In our current model an expert would approach the score of 6.00 on all variables. We are experimenting with using the square root of the z score to represent area in an effort to control for spatial artifacts. However, for our current purposes, we did not adjust the area as doing so would squash the differences and we wish to highlight them in a context where scores ceiling at 6.0.

Discussion and limitations. It is interesting to note the .43 correlation between velocity match (SMALLab error) and unit pretest. Because it is positive, it implies that the students who performed better on the physics unit pretest also had larger errors during the velocity match scenario. This may be because those students are more extroverted and higher achieving. We find that these students are more capable of getting the attention of the teacher and being called into SMALLab. This selection bias may lead to a situation where those students had more opportunities to perform poorly (or well) in the velocity challenge game. It might also imply that the embodied method of learning slopes is a separate, dissociable construct from the sort of knowledge that is measured by traditional paper-and-pencil physics tests that tap a more “crystallized” form of knowledge (defined as acquired knowledge or experience). Obviously, a larger sample than ten students and a controlled experimental design are needed to address these issues.

Control groups, either waitlist control groups or different experimental manipulation control groups, need to be included in future studies. Unfortunately, at the high school where SMALLab is currently installed, the only other physics class is an honors class with a different teacher. A comparison of pretest scores revealed the honors group to be significantly higher than the group on SMALLab. Given these two confounds, valid comparisons cannot be made. We hope to remedy this situation next year when SMALLab expands to another school. Even though teachers may differ, we should be able to match pretest scores and report inferential analyses.

In this present study we have demonstrated the promise that mixed-reality technologies offer for successful teaching and learning. We do believe that well-designed mixed-reality learning environments hold the potential for a transformative effect on K-12 education. The combination of collaboration within a digitally mediated environment facilitates free and authentic discussion, where all students are encouraged to “shout out hypotheses” at the same time that other students are immersed in the interactive space. Due to its unique interface, SMALLab is collaborative and engaging in a way that is difficult to deliver with traditional desktop computing paradigms. Despite this promise, to date, much of mixed-reality has remained relegated to specialized facilities due to the logistical and financial obstacles. Our work is focused on the realization of a low-cost, robust learning environment that can be replicated on a large scale in mainstream K-12 classrooms and informal learning environments. As costs continue to fall, we expect that SMALLab will be increasingly accessible, and we are currently extending the reach of SMALLab to a network of diverse school and informal learning sites. This will allow us to collect data from a wide range of students in different geographical and cultural contexts and we anticipate this will allow us to generalize these preliminary findings.

Conclusions. The work presented here builds upon three years of prior work to ready the mixed-reality learning environment, SMALLab, for deployment in a conventional classroom. We are now beginning a series of rigorous studies on its efficacy and value. SMALLab advances hands-on, student-centered learning through multimodal representation, physical embodiment, and whole class collaboration. In the design of new SMALLab learning scenarios, our research team collaborates with classroom teachers to co-design interactive learning scenarios, curricula, and assessment methodologies. In the realm of science learning we have sought to bring together best practices in inquiry-based learning with the affordances of interactive digital media. Specifically, we posit that the combination of multimodal representations and embodied action offers promising opportunities for new types of learning.

To explore this claim, we implemented a game-like SMALLab learning scenario for physics learning called the Constant Velocity Matching Game. The scenario empowers students to pace the difficulty level of their learning as they engage complex topics in a multimodal, embodied fashion. The current study is only one component of a longitudinal examination of the efficacy of SMALLab learning. There are two goals for our future work. First, we are working to collect larger samples that will lead to generalizations. Second, we hope to formalize the student learning profiles; those data should then feed back into the design of future learning scenarios. One of our long-term goals is to add system-level adaptivity to SMALLab to better address the individual needs of students in a dynamic manner.

Acknowledgements

We gratefully acknowledge that these materials document work supported by the MacArthur Foundation under the grant titled Gaming SMALLab: A game-like approach to embodied learning and by the National Science Foundation under CISE Infrastructure Grant No. 0403428 and IGERT Grant No. 0504647. We extend our gratitude to the students, teachers, and staff of Coronado High School.

References

American Association for
Physics Teachers. 2003. Strategic programs for innovations in undergraduate
physics: Project report. http://www.aapt.org/Projects/ntfup.cfm (accessed April 30, 2009).

Birchfield, D. 2006. K-12
embodied and mediated learning. http://ame2.asu.edu/projects/emlearning
(accessed April 30, 2009).

Birchfield, D., T. Ciufo,
G. Minyard, G. Qian, W. Savenye, H. Sundaram, H., Thornburg, H., Todd,
C.  2006. SMALLab: A mediated
platform for dducation. Paper presented at the ACM SIGGRAPH, Boston,
MA.

Birchfield, D., B. Mechtley,
S. Hatton, and H. Thornburg. 2008. Mixed-reality
learning in the art museum context. Paper presented at the ACM SIG
Multimedia, Vancouver, BC.

Cohen, J., and P. Cohen.
1983. Applied multiple regression/correlation
analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.

Cook, S., and S. Goldin-Meadow.
2006. The role of gesture in learning: Do children use their hands to
change their minds? Journal of Cognition and Development 7 (2):
211–232.

Cuthbertson, A., S. Hatton,
G. Minyard, H. Piver, C, Todd, and D. Birchfield. 2007. Mediated
education in a creative arts context:
Research and practice at Whittier Elementary School. Paper presented
at the 6th International Conference on Interaction Design and Children,
Aalborg, Denmark.

Dourish, P. 2001. Where
the action is: The foundations of embodied interaction. Cambridge,
MA, and London: MIT Press.

Ekstrom, R. B., J. W. French,
and H. H. Harman. 1979. Cognitive factors: Their identification and
replication. Multivariate Behavioral Research Monographs
79 (2): 3–84.

Glenberg, A. M., T. Gutierrez,
J. Levin, S. Japuntich, and M. P. Kaschak. 2004. Activity and imagined
activity can enhance young children's reading comprehension. Journal
of Educational Psychology 96 (3): 424–436.

Halloun, I. A. 2001. Students'
views about science: A comparative survey. http://www.halloun.net/images/stories/vass-2001monograph.pdf
(accessed November, 21, 2008).

Hatton, S., D. Birchfield,
and M. C. Megowan-Romanowicz. 2008. Learning
metaphor through mixed-reality game
design and game play. Paper presented at the AMC Sandbox, Los Angeles,
CA.

Heller, P., and M. Hollabaugh.
1992. Teaching problem solving through cooperative grouping, Part 2:
Designing problems and structuring groups. American Journal of Physics
60: 637.

Heller, P., R. Keith, and
S. Anderson. 1992. Teaching problem solving through cooperative grouping,
Part 1: Group versus individual problem solving. American Journal
of Physics 60: 627.

Hestenes, D. 1992. Modeling
games in the Newtonian world. American Journal of Physics
60: 732–748.

Hestenes, D. 1996. Modeling
methodology for physics teachers. Paper presented at the International
Conference on Undergraduate Physics, College Park, MD.

Hestenes, D., M. Wells,
and G. Swackhamer. 1992. Force concept inventory. Physics Teacher
30: 141.

Kress, G., C. Jewitt, J.
Ogborn, and C. Tsatsarelis. 2001. Multimodal
teaching and learning: The rhetorics of the
science classroom. London and New York: Continuum.

Llewellyn, D. 2005.
Teaching high school science through inquiry:
A case study approach. Rochester, NY: Corwin Press.

McDermott, L. C. 1984.
Research on conceptual understanding in mechanics. Physics Today

37 (7): 24–33.

National Research Council
(U.S.). 1996. National science education
standards. Washington, DC: National Academy Press.

Ramsier, R. D. 2001. A
hybrid approach to active learning. Physics Education
36 (2): 124–128.

Van Heuvelen, A. 1991.
Learning to think like a physicist: A review of research-based instructional
strategies. American Journal of Physics
59: 891.