http://epistemicgames.org/eg/wp-content/uploads/svarovsky-dissertation-revision-v26.pdf

Today’s global economy requires our nation to continue developing highly trained engineering professionals. Recently, K-12 engineering education has received increased attention as a pathway to building stronger foundations in math and science and introducing young people to the profession. However, the National Academy of Engineering found that most K-12 engineering programs focus heavily engineering design and science and math learning while minimizing the development of engineering habits of mind. This narrowly focused engineering activity can leave young people – and in particular, girls – with a limited view of the profession.

This study describes Digital Zoo, an engineering learning environment that engaged girls in authentic engineering activity in order to link the development of engineering skills and knowledge to engineering ways of thinking. Digital Zoo was an educational design experiment based on a particular theory of learning, the Epistemic Frame Hypothesis. Specific activities from an engineering practicum were recreated in the learning environment, where ten middle school girls from diverse backgrounds role played as engineers designing solutions to a client-based project. Responses on pre, post, and follow up interviews suggest the participants were able to develop each of the five epistemic frame elements –engineering skills, knowledge, identity, values, and epistemology – as a result of Digital Zoo. In situ data from the intervention was analyzed with a sophisticated mixed methods approach that integrated qualitative methods with a new quantification technique, Epistemic Network Analysis. These techniques allowed for the exploration of complex thinking and learning throughout the different activities of Digital Zoo. The results of this analysis identified client-focused activity and notebook-based reflection as two activities within Digital Zoo that evoked girls’ reflection on engineering values and epistemology. These qualitative claims were further warranted by intra-sample statistical analysis that utilized fixed-effects logistic regression.

Thus, this work has potential implications for the engineering education community by highlighting specific activities that may possibly develop engineering ways of thinking within young people, and girls in particular. Moreover, this study has implications for the learning sciences community by presenting an example of an integrated mixed methods approach to exploring complex thinking and learning within a naturalistic setting.

http://epistemicgames.org/eg/wp-content/uploads/hatfield-ICLS2010-review.pdf

As bloggers and mobile phone eye-witnesses increasingly supplement the ‘news,’ it is more important than ever to understand how professional journalists develop their expertise. In this paper, we examine an intermediate level reporting practicum course to explore the learning processes therein. Using a new method called Epistemic Network Analysis, we also explore emergent relationships within developing journalistic expertise. Understanding these relationships should be useful for journalism education as well as the design of research on learning environments.

ICLS2010-submission-173-revised-final

Preparing citizens to address the complex problems inherent in cities requires our changing society to embrace a new kind of education. One way to train people to think about complex problems is to identify and study how professionals who think in those ways develop their epistemic frame. In this paper, we examine one of the ways urban planners master and appropriate relevant expertise through an ethnographic study of an urban planning practicum. Specifically, we use a new method called epistemic network analysis to look at presentation feedback sessions during two weeks of the practicum to explore emergent relationships between the teacher’s planning expertise and the students’ expertise. The results of this study indicate that epistemic network analysis offers a technique for analyzing the kinds of situated understanding that result from sociocultural learning and for observing the translation of pedagogy into practice in various types of learning environments.

ICLS2010-submission-173-revised-final

http://epistemicgames.org/eg/wp-content/uploads/ICLS2010_pnash_submission_final.pdf

Players of epistemic games–computer games that simulate professional practica—have been shown to develop epistemic frames: a profession’s particular way of seeing and solving problems. This study examines the interactions between players and mentors in one epistemic game, Urban Science. Using a new method called epistemic network analysis, we explore how players develop epistemic frames through playing the game. Our results show that players imitate and internalize the professional way of thinking that the mentors model, suggesting that mentors can effectively model epistemic frames, and that epistemic network analysis is a useful way to chart the development of learning through mentoring relationships.

http://epistemicgames.org/eg/wp-content/uploads/ECD-for-Epistemic-Games-JTLA-Final-Version-with-Editorial-Edits.doc

We are currently at an exciting juncture in developing effective means for assessing so-called 21st-century skills in an innovative yet reliable fashion. One of these avenues leads through the world of epistemic games (Shaffer, 2006a), which are games designed to give learners the rich experience of professional practica within a discipline. They serve to develop domain-specific expertise based on principles of collaborative learning, distributed expertise, and complex problem-solving. In this paper, we describe a comprehensive research programme for investigating the methodological challenges that await rigorous inquiry within the epistemic games context. We specifically demonstrate how the evidence-centered design framework (Mislevy, Almond, & Steinberg, 2003) as well as current conceptualizations of reliability and validity theory can be used to structure the development of epistemic games as well as empirical research into their functioning. Using the epistemic game Urban Science (Bagley & Shaffer, 2009), we illustrate the numerous decisions that need to be made during game development and their implications for amassing qualitative and quantitative evidence about learners’ developing expertise within epistemic games.

Epistemic games have been developed to help players develop domain-specific expertise that characterizes how professionals in a particular domain reason, communicate, and act (Shaffer, 2006; Shaffer & Bagley, 2009). Grounded in a sociocultural and sociocognitive approach to learning, epistemic games are designed to foster situated learning that leads to data structures with high levels of dependencies. As one might expect, traditional measurement models struggle to accommodate such contextual dependencies, especially when data are collected at smaller scales and epistemic network analysis (ENA) has been developed to provide a practically feasible modeling alternative (e.g., Rupp et al., 2009; Shaffer et al., in press). In this paper, we describe a research program that addresses key statistical considerations for modeling data from epistemic games using ENA with an eye toward representing different learning progressions of players within such games. Current approaches for representing learning progressions using ENA are juxtaposed with approaches for simulating such data using particular statistical constraints.

http://epistemicgames.org/eg/wp-content/uploads/Multisubculturalismchapter-2.pdf

Multiculturalism is an essential tool for democratic citizenship in a world made ever more closely interconnected by information technologies. In this paper, I propose a model for progressive multicultural education in the computer age. I begin by describing the Pragmatic Progressive model of learning implicit in Dewey’s writing on education. I then discuss two revisions to the model in light of technological developments and theoretical work over the last few decades. Taken together, these revisions suggest that we might profitably revisit—and revise—Dewey’s ideas in the post-industrial era. I bring these ideas together to describe a theory of pedagogical praxis that offers an opportunity to move from multiculturalism to multisubculturalism: a view of education that focuses on diverse educational goals rather than diverse pathways to a single pedagogical end—and thus a view of learning more suited to the diverse ways of thinking and living that characterize our increasingly integrated world.

In this paper we look at educational assessment in the 21st Century. Digital learning environments emphasize learning in action. In such environments, assessments need to focus on performance in context rather than on tests of abstracted and isolated skills and knowledge. Digital learning environments also provide the potential to assess performance in context, because digital tools make it possible to record rich streams of data about learning in progress. But what assessment methods will use this data to measure mastery of complex problem solving, the kind of thinking in action that takes place in digital learning environments?

Here we argue that one way to address this challenge is through evidence-centered design: a framework for developing assessments by systematically linking models of understanding, observable actions, and evaluation rubrics to provide evidence of learning. We examine how evidence-centered design can address the challenge of assessment in new media learning environments by presenting one specific theory-based approach to digital learning, known as epistemic games, and describing a method, epistemic network analysis, to assess learner performance based on this theory. We use the theory and its related assessment method to illustrate the concept of a digital learning system: a system composed of a theory of learning and its accompanying method of assessment, linked into an evidence-based, digital intervention. And we argue that whatever tools of learning and assessment digital environments use, they need to be integrated into a coherent digital learning system linking learning and assessment through evidence-centered design.

http://epistemicgames.org/eg/wp-content/uploads/ijgcms-bagley-shaffer.pdf

http://epistemicgames.org/cv/papers/Nash_Shaffer_ICLS_08.pdf