Epistemic Network Analysis

Shaffer, DW, Hatfield, D, Svarovsky, GN, Nash, P, Nulty, A, Bagley, E, Franke, K, Rupp, AA, Mislevy, R (2009). Epistemic Network Analysis: A prototype for 21st Century assessment of learning. The International Journal of Learning and Media. 1(2), 33-53.
http://epistemicgames.org/eg/wp-content/uploads/IJLM0102_Shaffer.pdf

In this paper we look at educational assessment in the 21st Century. Digital learning environments emphasize learning in action. In such environments, assessments need to focus on performance in context rather than on tests of abstracted and isolated skills and knowledge. Digital learning environments also provide the potential to assess performance in context, because digital tools make it possible to record rich streams of data about learning in progress. But what assessment methods will use this data to measure mastery of complex problem solving, the kind of thinking in action that takes place in digital learning environments?

Here we argue that one way to address this challenge is through evidence-centered design: a framework for developing assessments by systematically linking models of understanding, observable actions, and evaluation rubrics to provide evidence of learning. We examine how evidence-centered design can address the challenge of assessment in new media learning environments by presenting one specific theory-based approach to digital learning, known as epistemic games, and describing a method, epistemic network analysis, to assess learner performance based on this theory. We use the theory and its related assessment method to illustrate the concept of a digital learning system: a system composed of a theory of learning and its accompanying method of assessment, linked into an evidence-based, digital intervention. And we argue that whatever tools of learning and assessment digital environments use, they need to be integrated into a coherent digital learning system linking learning and assessment through evidence-centered design.