A Continuation of Earlier Studies Comparing MSP and Non MSP Schools: Whether Distinguishing Different Levels of Participation Makes Any Difference
Abstract
This study is a follow-up to our previous pilot study comparing schools that participate in the National Science Foundation's Math and Science Partnership (MSP) Program with their intrastate non-participating peers in four states (Wong, Boben, Kim, & Socha, in press). The study is part of a larger effort to evaluate the MSP Program's role in student achievement, along with two companion studies. Although the pilot and this follow-up study use a comparative approach, the study by Dimiter Dimitrov (in press) follows a within-group design and the third study by Robert K. Yin (in press) covers the varied results reported by the MSPs themselves in their own "local" evaluations.At the end of the previous pilot study, the study team suggested items for future work, including adding more categories of what it means to be a participating school. There is only one criterion available that categorizes the level of participation of each school for a given year. The study team used this criterion to categorize the schools into four levels of participation: high, moderate, low, and non-participating. For each of three levels of MSP participating schools, each school has been carefully matched with the non-participating schools on seven demographic variables to form a comparison group. Research conducted by Wong, Boben, and Socha (Wong & Socha 2008; Wong, Boben, Kim, & Socha, in press) offered detailed documentation on how the team operationalized matching methods for comparative purposes. The present study summarizes the matching methodology and discusses the results from the three levels of participation. The analysis leads us to believe that carefully executed matching methods are promising for large- scale comparative analysis on the effects of the MSP Program across states.
The present study draws on publicly accessible school-level standardized test data for 2001-02 and 2007-08 from four states and from data available from the National Center for Education Statistics' Common Core of Data (NCES CCD). In addition, the study uses documents available via MSPnet, and Web site information reported by the individual MSPs accessible through the school year 2007-08.