Now the the website is updated with the 2014- 2015 data,(YES!) some questions have arisen about validity in light of the move to a new exam, the FSA. Short answer is that the change has not hurt the validity of the data, long answer can be found below.
Transition to FSA Does Not Adversely Impact 2014-15 Teacher Value-Added Model Scores
Florida moving to the Florida Standards Assessment (FSA) during the 2014-15 school year did not change the way value-added scores are computed by the state nor adversely impact teacher value-added scores. Because Florida uses a covariate adjustment model, a type of regression model, the variables used in the models do not have to be on the same scale.
Florida’s value-added model controls for student, classroom and school characteristics that impact student learning to determine if a teacher’s students, on average, are above or below the expected earning growth of similar students in the state. Since the value-added model’s inception there have been at least 10 different predictor variables, using many different scales, included in the model.
- The number of subject-relevant courses in which the student is enrolled
- Up to two prior years of achievement scores
- Students with Disabilities (SWD) status
- English Language Learner (ELL) status
- Gifted status
- Attendance
- Mobility (number of transitions)
- Difference from modal age in grade (as an indicator of retention)
- Class size
- Homogeneity of entering test scores in the class
These predictor variables estimate the relative impact of all of the variables, including prior year achievement scores, on the outcome of how well a student is expected to score on the FSA. Regardless of scale, the value-added model adjusts the student’s expected FSA score based on a unit change in the predictor variable, whether that is an assessment score, or in how many subject relevant courses the student is enrolled. The use of assessment data on different scales has been demonstrated in the Algebra 1 value-added model over the last few years. In the Algebra I model, FCAT 2.0 is used as a predictor of the expected student scores on Algebra I and these assessments are on different scales. This is also true when FSA is used as a predictor.
Value-added model scores are calculated differently than learning gains used for school grades.
Unlike the value-added model, learning gains for school grades are dependent on achievement level definitions to measure student learning within and between these achievement levels. As a result of the assessment transition, achievement levels and expectations for determining proficiency have been changed. This means that the percent of students’ proficient is not comparable between the FSA and FCAT 2.0 because the expectation for proficiency has changed. It also means measuring learning gains for students from achievement level 1 on FCAT 2.0 to achievement level 2 on FSA is not possible because the scores and ranges for achievement levels have change. Fortunately, value-added models are not dependent on achievement levels and proficiency expectation determinations.
Aggregation of Value-Added Scores Across Grades, Subjects, and Years
Florida Standards Assessment scores were linked to the FCAT 2.0 scale in order to calculate year-to-year growth for English/Language Arts (ELA) and Math only, which is required for standardization prior to aggregation. For the purpose of standardizing and aggregating the individual grade, subject, year value-added model estimates to be combined into an aggregate score, the FSA theta scores were linked to the FCAT 2.0 scale. However, this is not necessary to compute individual grade-level value-added models, since both the observed and predicted scores are based on the 14-15 FSA results. In the FSA value-added models the prior score and modeled score do not have to be on the same scale in order to calculate value-added model estimates. However, in order to standardize the scores for aggregation across grades, subjects and years, average growth within the grade and subject from one year to the next must be calculated, which did require linking of the FSA ELA and Mathematics scores to the FCAT 2.0 scale in order to compute.