http://www.wceruw.org/publications/workingPapers/Working_Paper_No_2010_05.php

In response to industry’s demands for better prepared engineering graduates, capstone design courses have been included in undergraduate engineering programs for over 3 decades. These practicum experiences, typically encountered during the final year or semester of the course sequence, provide valuable “real world” experiences for students by allowing them to engage in the engineering design process within an authentic context. While the success of these courses has been extensively studied, much less attention has been paid to investigating how specific components of the practicum affect the learning processes of the undergraduates. This paper presents an ethnographic study of an engineering practicum. The study followed a student design team consisting of four undergraduates as they worked with an actual client to design a biomedical device. The paper specifically examines how two activities—meeting regularly with a design advisor to discuss progress and maintaining a design notebook throughout the project—contributed to the students’ development of engineering skills, knowledge, and ways of thinking. Understanding how these activities facilitate student learning can influence the design of future capstone experiences as well as other, more general engineering learning contexts.

http://epistemicgames.org/eg/wp-content/uploads/svarovsky-dissertation-revision-v26.pdf

Today’s global economy requires our nation to continue developing highly trained engineering professionals. Recently, K-12 engineering education has received increased attention as a pathway to building stronger foundations in math and science and introducing young people to the profession. However, the National Academy of Engineering found that most K-12 engineering programs focus heavily engineering design and science and math learning while minimizing the development of engineering habits of mind. This narrowly focused engineering activity can leave young people – and in particular, girls – with a limited view of the profession.

This study describes Digital Zoo, an engineering learning environment that engaged girls in authentic engineering activity in order to link the development of engineering skills and knowledge to engineering ways of thinking. Digital Zoo was an educational design experiment based on a particular theory of learning, the Epistemic Frame Hypothesis. Specific activities from an engineering practicum were recreated in the learning environment, where ten middle school girls from diverse backgrounds role played as engineers designing solutions to a client-based project. Responses on pre, post, and follow up interviews suggest the participants were able to develop each of the five epistemic frame elements –engineering skills, knowledge, identity, values, and epistemology – as a result of Digital Zoo. In situ data from the intervention was analyzed with a sophisticated mixed methods approach that integrated qualitative methods with a new quantification technique, Epistemic Network Analysis. These techniques allowed for the exploration of complex thinking and learning throughout the different activities of Digital Zoo. The results of this analysis identified client-focused activity and notebook-based reflection as two activities within Digital Zoo that evoked girls’ reflection on engineering values and epistemology. These qualitative claims were further warranted by intra-sample statistical analysis that utilized fixed-effects logistic regression.

Thus, this work has potential implications for the engineering education community by highlighting specific activities that may possibly develop engineering ways of thinking within young people, and girls in particular. Moreover, this study has implications for the learning sciences community by presenting an example of an integrated mixed methods approach to exploring complex thinking and learning within a naturalistic setting.

Gina’s dissertation is one of the first studies to use Epistemic Network Analysis to look at the impact of specific parts of a game on the development of professional thinking. Her study of Digital Zoo found that interactions with clients and reflections in an engineering notebook play a key role in making the values and epistemology of engineering explicit for players.

As the first graduate student in the Epistemic Games Group, Gina was instrumental in helping to establish the research directions and practices of the group. We will miss her very much, wish her well in in future ventures, and hope that from time to time she will drop in with a post on news epistemic!

Epistemic games have been developed to help players develop domain-specific expertise that characterizes how professionals in a particular domain reason, communicate, and act (Shaffer, 2006; Shaffer & Bagley, 2009). Grounded in a sociocultural and sociocognitive approach to learning, epistemic games are designed to foster situated learning that leads to data structures with high levels of dependencies. As one might expect, traditional measurement models struggle to accommodate such contextual dependencies, especially when data are collected at smaller scales and epistemic network analysis (ENA) has been developed to provide a practically feasible modeling alternative (e.g., Rupp et al., 2009; Shaffer et al., in press). In this paper, we describe a research program that addresses key statistical considerations for modeling data from epistemic games using ENA with an eye toward representing different learning progressions of players within such games. Current approaches for representing learning progressions using ENA are juxtaposed with approaches for simulating such data using particular statistical constraints.

In this paper we look at educational assessment in the 21st Century. Digital learning environments emphasize learning in action. In such environments, assessments need to focus on performance in context rather than on tests of abstracted and isolated skills and knowledge. Digital learning environments also provide the potential to assess performance in context, because digital tools make it possible to record rich streams of data about learning in progress. But what assessment methods will use this data to measure mastery of complex problem solving, the kind of thinking in action that takes place in digital learning environments?

Here we argue that one way to address this challenge is through evidence-centered design: a framework for developing assessments by systematically linking models of understanding, observable actions, and evaluation rubrics to provide evidence of learning. We examine how evidence-centered design can address the challenge of assessment in new media learning environments by presenting one specific theory-based approach to digital learning, known as epistemic games, and describing a method, epistemic network analysis, to assess learner performance based on this theory. We use the theory and its related assessment method to illustrate the concept of a digital learning system: a system composed of a theory of learning and its accompanying method of assessment, linked into an evidence-based, digital intervention. And we argue that whatever tools of learning and assessment digital environments use, they need to be integrated into a coherent digital learning system linking learning and assessment through evidence-centered design.

This short video gives an overview of our work on Urban Science and other epistemic games as part of the Macarthur Digital Media and Learning Project and the National Science Foundation.

In these games, players have a chance to learn 21st century skills by playing as urban planners, engineers, journalists, and other professionals in the knowledge economy.

I suppose next we’ll need to make an epistemic game about making epistemic games….

If you haven’t already installed Real Player, get it here.

In an earlier post on the Macarthur blog, Edward Miller, a senior researcher at the Alliance for Childhood, was quoted as saying: ‘There is no evidence that video games are good at teaching problem-solving or ‘higher-order skills.’

Sadly–or perhaps I should say, happily–that’s simply not true.

In the game Digital Zoo, players become biomechanical engineers and design creatures of the kind you might see in an animated movie. And it turns out that, yes, by playing as engineers they learn to think about problems the way engineers do.

In one test, for example, we ask players before and after the game to draw a flowchart of how they would solve an engineering design problem. Their design process, as reflected in the flowcharts, becomes significantly more complex–and more like the real engineering design process–after playing the game than it was before. In other words, they learn to think more like engineers.

This was a carefully-designed study as part of my dissertation research. The design problems we asked them to solve before and after the game had nothing to do with the content of the game itself. The players of Digital Zoo don’t use flowcharts of this kind in the game, so they weren’t merely getting better at drawing flowcharts. We created problem isomorphs (meaning problems with the same structure but different details) so players would not be getting better at solving the problem because they had seen it before. And it was a month between the tests before and after the game.

To make sure this was not just an artifact of the test or the statistics, we conducted a controlled study. Just as we presented the design problems to players of Digital Zoo before and after the game, we gave the same problem to players of another game of the same duration–but that game was about being a journalist rather than being an engineer.

The results held up. In fact, before the games, players of the engineering game did significantly worse than players of the journalism game at thinking like an engineer. Afterwards, the players of the engineering game did better.

In other words, playing a well-designed game CAN help you learn creative, higher-order thinking.