Designing High Quality Science Assessment Systems

National Research Council Workshop: May 6-7, 2004

View the committee’s charge to the design teams for the below models of science assessment systems.
View the workshop attendee list.
View the design team PowerPoint presentations by clicking on the links below (in PDF format).
Listen to Audio Proceedings of the Workshop

May 6 – Four Model Designs: High Quality Systems that Can be Implemented by States

		PRESENTER	TOPIC
7:30– 8:00	Continental Breakfast
8:00 – 8:30	Welcome & Overview of the Meeting	Martin Orland, Director, Center for Education Mark Wilson, Committee Chair	The Committee’s Charge
8:30 – 9:15	Designing and Building Science Assessments Through Partnerships	Richard Patz, Assessment and Accountability Consultant	This presentation focuses on the ways in which partnerships among test publishers, research organizations, and scientific industries may be effectively used by states in the design, development, and implementation of a successful K-12 science assessment system.
9:15 – 10:00	Comments and Questions	Open Discussion	Possible discussion points: •What are the educational, policy, and fiscal implications of this design? •Will the design lead to science assessments that support learning and teaching in science? Why or why not? •Can states practically implement this design? •What are the strengths and challenges inherent in the design?
10:00 – 10:15	BREAK

10:15 – 11:00	Designing and Building an Instructionally Supportive Science Assessment System	James Pellegrino, Member of the Commission on Instructionally Supportive Assessment and Member of the Committee on Test Design for K-12 Science Achievement	This presentation examines the applicability of the advice offered by the national Commission on Instructionally Supportive Assessment for the design and development of high quality science assessment systems.
11:00 – 11:45	Comments and Questions	Open Discussion	Possible discussion points as above
11:45 – 12:30	LUNCH
12:30 – 1:15	Cooperatively Building a Model Science Assessment System	Mark Moody, Assessment and Accountability Consultant Edys Quellmalz, Associate Director of the Assessment, Center for Technology in Learning, SRI International	This presentation focuses on the design and development of a model state science assessment system and item bank that is multi-level (classroom, district, state); is developed collaboratively by multiple states; and includes the use of technology for the management of the system and for the delivery of prompts that could not otherwise be included in large-scale assessment systems.
1:15 – 2:30	Comments and Questions	Open Discussion	As above
2:30 – 2:45	BREAK
2:45– 3:30	Building a Teacher-Led, Classroom-Based Science Assessment and Accountability System	Chad Buckendahl, Co-Director Buros Institute for Assessment, Consultation, and Outreach	This presentation focuses on strategies for building a coherent, instructionally useful, teacher-led assessment program that would meet criteria for technical rigor and provide adequate reporting at different levels and to different stakeholders.
3:30 – 4:15	Comments and Questions	Open Discussion	As above
4:15 – 5:00	Discussion Groups		In-depth discussion and compare/contrast of the four designs.
5:00 – 5:45	Reports from Groups
5:45	ADJOURN

May 7 – New Directions in Science Assessment: Laying the Groundwork for Better Assessment Systems

		PRESENTER	TOPIC
7:30 – 8:00	Continental Breakfast
8:00 – 8:15	Welcome and Overview of the Day	Mark Wilson
8:15 – 9:00	Building from the Research: Designing Science Assessments to Measure Conceptual Understanding of Big Ideas in Science	Richard Lehrer, Vanderbilt University Brian Reiser, Northwestern University Kefyn Catley, Vanderbilt University	The NRC report, How People Learn: Bridging Research and Practice recommends that teams of experts from science, cognitive psychology, education, and psychometrics develop science assessments. This presentation will describe the results of one such collaboration. It will focus on the implications of research on learning and cognition for the intentional and systemic design of assessments to measure students’ conceptual understanding of natural selection and evolution.
9:30 – 10:15	Building from the Research: Designing Science Assessments to Measure Conceptual Understanding of Big Ideas in Science	Andy Anderson, Michigan State University Joseph Krajcik, University of Michigan Audrey Champagne, SUNY Albany Brian Coppola, University of Michigan Marianne Wiser, Clark University	This presentation will focus on the implications of research on learning and cognition for the intentional and systemic design of assessments to measure students’ conceptual understanding of kinetic molecular theory.
10:15 – 10:30	BREAK
10:30 – 11:15	Comments and Questions
11:15 – 12:00	Tracking Student Achievement In Science Over Time	Mark Reckase, Michigan State University	This presentation discusses how advances in psychometric theory may be applied to improve the quality of information gathered and reported in K-12 science assessments.
12:00 – 12:30	Comments and Questions
12:30 -- 1:00	LUNCH
1:00 – 1:45	Breaking the Mold: Technology-Based Science Assessment for the 21st Century	Edys Quellmalz, Associate Director of the Assessment, Center for Technology in Learning, SRI International	This presentation will explore uses of technologies to support efficient assessment design, delivery, scoring, and reporting.
1:45 – 2:15	Comments and Questions
2:15 – 3:00	Assessing Complex Problem-Solving Performance	Randy Bennett, Educational Testing Service	This presentation will explore strategies for understanding the data generated from student performances on extended, integrated, complex tasks such as computer-based simulations to make inferences about proficiency.
3:00 – 3:30	Comments and Questions
3:30 – 3:45	Adjourn Open Session