Does the Test Matter? Evaluating teachers when tests differ in their sensitivity to instruction
Abstract
"Implicit in the main MET project reports is the assumption that the policy recommendations offered apply equally across states. However, there is evidence from previous work that state tests differ in the extent to which they reflect the content or quality of teachers' instruction (their instructional sensitivity). This chapter applies the methods of the main MET reports to data disaggregated by state to explore whether state tests vary in their sensitivity and whether variation in sensitivity affects the recommendations for weighting components in multiple-measures teacher evaluation systems. The chapter shows that state tests indeed vary considerably in their correlations with observational and student survey measures of effective teaching. Furthermore, some state tests correlate weakly or not at all with these measures, suggesting weak sensitivity to instructional quality. These state-to-state differences in correlations produce weighting recommendations that also differ across states. However, an equally weighted composite of measures may work well in most cases. The chapter concludes by recommending that states explore their own data to ensure their tests are adequately sensitive to high-quality instruction."
