Comparison of MSP and Non-MSP Schools in Six States
Abstract
This pilot study proposes a set of analytical steps for comparing schools that participate in the National Science Foundation's Math and Science Partnership (MSP) Program and their intrastate non-participating peers. This pilot is part of a larger effort to evaluate the MSP Program's role in student achievement, along with two companion analyses. While our pilot study uses a comparative approach, the paper by Dimiter Dimitrov (this volume) follows a within-group design. The third analysis by Robert K. Yin (this volume) covers the varied designs used by the MSPs themselves in their own evaluations. As this pilot study has progressed, there has emerged three distinct phases of analysis. Phase I focused on the participating schools within four Math and Science Partnerships (MSPs) located in three states. Phase II expanded Phase I to focus on three more MSPs in three additional states. The Phase III study was conducted using six cohort I MSPs in four states, and will eventually include the nine cohort I MSPs in all six states from Phase I and Phase II. For each phase, the MSP participating schools were carefully matched with the non-participating schools on eight demographic variables to form a comparison group. This paper offers detailed documentation on how we operationalize matching methods for comparative purposes. We conclude that carefully executed matching methods are promising for large-scale comparative analysis on the effects of the MSP Program across all involved states. The study draws on publicly accessible school-level standardized test data from six states and from data available at the National Center for Education Statistics' Common Core of Data (NCES CCD). In addition, the study uses documents available via MSPnet, and Web site information reported by the individual MSPs in the MSP Program accessible through the school year 2005-06.