Evidence-Centered Design

New Thread

Ctrl

Home

Discover

IntroductionIntroduction
Historical Evolution of Evidence-Centered DesignHistorical Evolution of Evidence-Centered Design
Integrating Learning Goals in ECDIntegrating Learning Goals in ECD
Task Model Variables in Assessment DesignTask Model Variables in Assessment Design

Evidence-Centered Design

Curated by

dianahughes

3 min read

389

Evidence-Centered Design (ECD) is a systematic framework for designing assessments that ensures the assessments provide valid evidence of the knowledge, skills, and abilities they are intended to measure. By carefully considering the evidentiary arguments and validity evidence needed from the onset of assessment design, ECD helps create more reliable and meaningful assessments.

Historical Evolution of Evidence-Centered Design

Evidence-Centered Design (ECD) emerged in the 1990s as a response to the limitations of traditional assessment practices in the face of rapid advancements in fields critical to assessment, such as cognitive psychology, technology, and statistical modeling.

Researchers at Educational Testing Service, led by Robert Mislevy, Linda Steinberg, and Russell Almond, developed ECD as a principled framework for designing assessments that could incorporate complex student models and interactive simulations, which went beyond the capabilities of existing assessment design approaches.

The origins of ECD can be traced back to Mislevy's 1994 paper "Evidence and inference in educational assessment", which laid the conceptual groundwork by emphasizing the importance of evidentiary arguments in assessment design and the need to explicitly link the desired inferences to the evidence needed to support them.

Over the following years, the ECD framework was further elaborated and refined through a series of technical reports and publications, detailing its layers, structures, and terminology.

Key milestones in the development of ECD include the introduction of the four-process architecture for assessment delivery systems in 2002

, the explication of the student, evidence, and task models as the three core models of ECD in 2003

, and the application of ECD to the design of simulation-based assessments in various domains.

The 2006 chapter "Concepts, terminology, and basic models of evidence-centered design" provided a comprehensive overview of the ECD framework and its components.

The evolution of ECD has been shaped by the need to leverage advances in learning sciences, technology, and psychometrics to create assessments that provide valid, reliable, and actionable evidence of student learning.

By reconceptualizing assessment design as an evidence-based and iterative process, ECD has provided a foundation for enhancing the validity and fairness of both traditional and innovative forms of assessment.

Today, ECD stands as a widely adopted and influential framework in the field of educational assessment, guiding the design and development of assessments that effectively bridge the gap between research and practice.

5 sources

Integrating Learning Goals in ECD

The expanded Evidence-Centered Design (e-ECD) framework integrates learning goals and processes within the assessment design, allowing for a more comprehensive approach that blends learning and assessment. This integration ensures that assessments not only measure student knowledge but also support and enhance the learning process.

Key aspects of integrating learning goals in e-ECD include:

Domain Model Expansion: The domain model in e-ECD is expanded to include not only the knowledge and skills to be assessed but also the learning objectives and processes relevant to the domain. This allows for a more holistic representation of what students should know and be able to do, as well as how they acquire this knowledge.
1
2
Learner Model Enhancement: The learner model in e-ECD incorporates aspects of learning progression and development over time. This enables tracking of student proficiency and understanding as they engage with learning activities and assessments.
1
3
Task Model Adaptation: Tasks are designed to provide evidence of both learning and assessment. This may involve creating activities that simultaneously support learning while generating evidence of student knowledge and skills.
1
2
Evidence Model Refinement: The evidence model in e-ECD is expanded to include indicators of learning processes and progress, not just final outcomes. This allows for a more nuanced understanding of student development and learning trajectories.
1
3
Learning-Assessment Relationship Specification: e-ECD provides a framework for explicitly defining the relationship between learning activities and assessment tasks. This ensures alignment between instructional design and assessment design.
1
2
Flexibility in Learning Theories: The e-ECD framework does not prescribe a specific learning theory but allows for the incorporation of various learning theories and goals as determined by the assessment developers or researchers.
1
2
Technology Integration: e-ECD supports the integration of technology-enhanced learning and assessment, allowing for more dynamic and interactive assessment experiences that can adapt to individual learner needs.
1
3

By integrating learning goals within the ECD framework, e-ECD enables the development of assessment systems that not only measure student knowledge but also actively support the learning process. This approach aligns with the growing recognition of the importance of formative assessment and the need for assessments that provide actionable insights to improve learning outcomes.

5 sources

Task Model Variables in Assessment Design

Task model variables play a crucial role in the Evidence-Centered Design (ECD) framework for assessment design. These variables are key components of the task model, which specifies the characteristics of tasks or situations that will elicit the desired evidence about students' knowledge, skills, and abilities.

In the context of ECD, task model variables serve several important functions:

Defining task characteristics: They specify the features and parameters of assessment tasks, such as the context, complexity, and format of the items or problems presented to students.
1
5
Guiding task generation: Task model variables provide a structured approach to creating multiple versions of tasks that maintain consistency in difficulty and measurement properties.
2
5
Linking to evidence models: They establish connections between task features and the types of evidence needed to support inferences about student proficiency.
1
2
Facilitating adaptivity: By manipulating task model variables, assessments can be tailored to individual students or specific assessment goals.
2
5
Supporting validity arguments: The explicit specification of task model variables helps strengthen the validity argument for the assessment by clarifying how tasks relate to the constructs being measured.
1
5

Task model variables can be categorized into different types, including:

Presentation variables: These define how the task is presented to the student, such as the format of questions or the medium of delivery.
2
Response variables: These specify the nature of the expected student response, such as multiple-choice selections or constructed responses.
2
Work product variables: These describe the characteristics of the work products that result from the student's engagement with the task.
1
5
Difficulty variables: These control aspects of the task that influence its complexity or challenge level.
2

By carefully designing and manipulating these variables, assessment developers can create tasks that effectively elicit the desired evidence about student proficiency while maintaining consistency and fairness across different versions of the assessment.

The use of task model variables in ECD represents a shift from traditional item writing approaches to a more systematic and theory-driven method of assessment design. This approach allows for greater flexibility in task creation while ensuring that each task aligns with the overall assessment goals and measurement model.