Matching Items (3)
Filtering by

Clear all filters

151477-Thumbnail Image.png
Description
This study examined the intended and unintended consequences associated with the Education Value-Added Assessment System (EVAAS) as perceived and experienced by teachers in the Houston Independent School District (HISD). To evaluate teacher effectiveness, HISD is using EVAAS for high-stakes consequences more than any other district or state in the country.

This study examined the intended and unintended consequences associated with the Education Value-Added Assessment System (EVAAS) as perceived and experienced by teachers in the Houston Independent School District (HISD). To evaluate teacher effectiveness, HISD is using EVAAS for high-stakes consequences more than any other district or state in the country. A large-scale electronic survey was used to investigate the model's reliability and validity; to determine whether teachers used the EVAAS data in formative ways as intended; to gather teachers' opinions on EVAAS's claimed benefits and statements; and to understand the unintended consequences that occurred as a result of EVAAS use in HISD. Mixed methods data collection and analyses were used to present the findings in user-friendly ways, particularly when using the words and experiences of the teachers themselves. Results revealed that the reliability of the EVAAS model produced split and inconsistent results among teacher participants, and teachers indicated that students biased the EVAAS results. The majority of teachers did not report similar EVAAS and principal observation scores, reducing the criterion-related validity of both measures of teacher quality. Teachers revealed discrepancies in the distribution of EVAAS reports, the awareness of trainings offered, and among principals' understanding of EVAAS across the district. This resulted in an underwhelming number of teachers who reportedly used EVAAS data for formative purposes. Teachers disagreed with EVAAS marketing claims, implying the majority did not believe EVAAS worked as intended and promoted. Additionally, many unintended consequences associated with the high-stakes use of EVAAS emerged through teachers' responses, which revealed among others that teachers felt heightened pressure and competition, which reduced morale and collaboration, and encouraged cheating or teaching to the test in attempt to raise EVAAS scores. This study is one of the first to investigate how the EVAAS model works in practice and provides a glimpse of whether value-added models might produce desired outcomes and encourage best teacher practices. This is information of which policymakers, researchers, and districts should be aware and consider when implementing the EVAAS, or any value-added model for teacher evaluation, as many of the reported issues are not specific to the EVAAS model.
ContributorsCollins, Clarin (Author) / Amrein-Beardsley, Audrey (Thesis advisor) / Berliner, David C. (Committee member) / Fischman, Gustavo E (Committee member) / Arizona State University (Publisher)
Created2012
153205-Thumbnail Image.png
Description
Teacher evaluation policies have recently shifted in the United States. For the first time in history, many states, districts, and administrators are now required to evaluate teachers by methods that are up to 50% based on their "value-added," as demonstrated at the classroom-level by growth on student achievement data over

Teacher evaluation policies have recently shifted in the United States. For the first time in history, many states, districts, and administrators are now required to evaluate teachers by methods that are up to 50% based on their "value-added," as demonstrated at the classroom-level by growth on student achievement data over time. Other related instruments and methods, such as classroom observations and rubrics, have also become common practices in teacher evaluation systems. Such methods are consistent with the neoliberal discourse that has dominated the social and political sphere for the past three decades. Employing a discourse analytic approach that called upon a governmentality framework, the author used a complementary approach to understand how contemporary teacher evaluation polices, practices, and instruments work to discursively (re)define teachers and teacher quality in terms of their market value.

For the first part of the analysis, the author collected and analyzed documents and field notes related to the teacher evaluation system at one urban middle school. The analysis included official policy documents, official White House speeches and press releases, evaluation system promotional materials, evaluator training materials, and the like. For the second part of the analysis, she interviewed teachers and their evaluators at the local middle school in order to understand how the participants had embodied the market-based discourse to define themselves as teachers and qualify their practice, quality, and worth accordingly.

The findings of the study suggest that teacher evaluation policies, practices, and instruments make possible a variety of techniques, such as numericization, hierarchical surveillance, normalizing judgments, and audit, in order to first make teachers objects of knowledge and then act upon that knowledge to manage teachers' conduct. The author also found that teachers and their evaluators have taken up this discourse in order to think about and act upon themselves as responsibilized subjects. Ultimately, the author argues that while much of the attention related to teacher evaluations has focused on the instruments used to measure the construct of teacher quality, that teacher evaluation instruments work in a mutually constitutive ways to discursively shape the construct of teacher quality.
ContributorsHolloway-Libell, Jessica (Author) / Amrein-Beardsley, Audrey (Thesis advisor) / Anderson, Kate T. (Thesis advisor) / Berliner, David C. (Committee member) / Arizona State University (Publisher)
Created2014
158108-Thumbnail Image.png
Description
Over the past 20 years in the United States (U.S.), teachers have seen a marked

shift in how teacher evaluation policies govern the evaluation of their performance.

Spurred by federal mandates, teachers have been increasingly held accountable for their

students’ academic achievement, most notably through the use of value-added models

Over the past 20 years in the United States (U.S.), teachers have seen a marked

shift in how teacher evaluation policies govern the evaluation of their performance.

Spurred by federal mandates, teachers have been increasingly held accountable for their

students’ academic achievement, most notably through the use of value-added models

(VAMs)—a statistically complex tool that aims to isolate and then quantify the effect of

teachers on their students’ achievement. This increased focus on accountability ultimately

resulted in numerous lawsuits across the U.S. where teachers protested what they felt

were unfair evaluations informed by invalid, unreliable, and biased measures—most

notably VAMs.

While New Mexico’s teacher evaluation system was labeled as a “gold standard”

due to its purported ability to objectively and accurately differentiate between effective

and ineffective teachers, in 2015, teachers filed suit contesting the fairness and accuracy

of their evaluations. Amrein-Beardsley and Geiger’s (revise and resubmit) initial analyses

of the state’s teacher evaluation data revealed that the four individual measures

comprising teachers’ overall evaluation scores showed evidence of bias, and specifically,

teachers who taught in schools with different student body compositions (e.g., special

education students, poorer students, gifted students) had significantly different scores

than their peers. The purpose of this study was to expand upon these prior analyses by

investigating whether those conclusions still held true when controlling for a variety of

confounding factors at the school, class, and teacher levels, as such covariates were not

included in prior analyses.



Results from multiple linear regression analyses indicated that, overall, the

measures used to inform New Mexico teachers’ overall evaluation scores still showed

evidence of bias by school-level student demographic factors, with VAMs potentially

being the most susceptible and classroom observations being the least. This study is

especially unique given the juxtaposition of such a highly touted evaluation system also

being one where teachers contested its constitutionality. Study findings are important for

all education stakeholders to consider, especially as teacher evaluation systems and

related policies continue to be transformed.
ContributorsGeiger, Tray (Author) / Amrein-Beardsley, Audrey (Thesis advisor) / Anderson, Kate (Committee member) / McGuire, Keon (Committee member) / Holloway, Jessica (Committee member) / Arizona State University (Publisher)
Created2020