Evaluating the sensitivity of a reading comprehension benchmark assessment as a predictor of performance on a high stakes academic assessment for middle school students

Evaluating the sensitivity of a reading comprehension benchmark assessment as a predictor of performance on a high stakes academic assessment for middle school students. (2014). Evaluating the sensitivity of a reading comprehension benchmark assessment as a predictor of performance on a high stakes academic assessment for middle school students.

Download PDF

Metadata

MODS Metadata


Title	Evaluating the sensitivity of a reading comprehension benchmark assessment as a predictor of performance on a high stakes academic assessment for middle school students
Abstract	This study examined the predictive validity and diagnostic accuracy of STAR Reading, a computer-adaptive benchmark assessment tool, with the Pennsylvania System of School Assessment (PSSA), the state summative test. Student data for 320 students in grades 7 and 8 over two years of assessment were examined. Indices of correlation and predictions were analyzed. Multiple and hierarchical regression analyses examined the relationships of fall and winter administrations of STAR Reading for patterns of relationships of proximity of administration to the summative test. Logistic regression and ROC Curve analyses provided indices of classification accuracy for prediction of proficient performance and local cut scores for fall and winter administrations of STAR Reading. These local cut scores were compared to the published national benchmark cut points for STAR Reading. An attempt was made to cross-validate the prediction of proficient performance from the first year of the study to the second year. Results showed moderate to strong correlations among STAR Reading and PSSA Reading scores. Despite significant prediction indices for combined fall and winter STAR Reading scores in both grades, predictive values were weaker in grade 8 data. No clear evidence of a stronger relationship of prediction as a function of proximity to the completion of the state test was found, but hierarchical regression showed that the winter score added predictive value in three of four analyses conducted. Logistic regression indicated good classification accuracy and prediction of proficient performance on PSSA Reading for grade 7 students, but produced inconsistent statistical results for grade 8. ROC curve analyses showed adequate area under the curve (AUC) indices for three of four analyses. Inconsistencies were demonstrated frequently in grade 8 data; the attempt at cross-validation of probability of prediction was not able to be completed for grade 8 due to a slope intercept of .000 for one of the groups of eighth grade data. While evidence was provided that the overall models of prediction of STAR Reading to PSSA Reading was statistically significant, the lower effect size for grade 8 provided limitations to the mathematical utility of prediction to PSSA Reading scores.
Date	2014

Details

Title: Evaluating the sensitivity of a reading comprehension benchmark assessment as a predictor of performance on a high stakes academic assessment for middle school students
Personal Name(s): Fetchkan, Rene
Kovaleski, Joseph F.
Corporate Name(s): Indiana University of Pennsylvania
Educational and School Psychology.
Note: Publisher info.: Dissertation/Thesis.
Advisors: Kovaleski, Joseph F. Committee members: Joseph Betts; Tammy Croce; Timothy Runge; Mark Staskiewicz.
Topical Subject(s): Educational tests & measurements
Educational evaluation
Educational psychology
Literacy
Reading instruction