Matching Items (7)
Filtering by
- Status: Published
Description
This action research project engages questions about the relationship of teacher evaluation and teacher learning, joining the national conversation of accountability and teacher quality. It provides a solid philosophical foundation for changes in teacher evaluation and staff development, and analyzes past and current methods and trends in teacher evaluation. Set in the context of a suburban elementary charter school, the problems of traditional evaluation methods are confronted. The innovation proposed and implemented is Teacher Evaluation for Learning, Accountability, and Recognition (TELAR), a teacher evaluation system designed to support learning and accountability. TELAR includes multiple data points and perspectives, ongoing feedback and support, an evaluation instrument centered on collective values and a shared vision for professional work, and an emphasis on teacher reflection and self-assessment. This mixed-methods study employs both qualitative and quantitative measures to provide an enriched understanding of the current problem and the impact of the change effort. Results suggest that TELAR 1) helps teachers re-define their role as professionals in their own evaluation, positively increasing perceptions of value, 2) promotes a culture of learning through a focus on shared values for professional work, a spirit of support and teamwork, and continuous improvement; and 3) empowers teachers to assess their own practice, self-diagnose areas for growth, and generate goals through a continuous process of feedback, reflection, conversation, and support. Implications for practice and future studies are presented.
ContributorsMusser, Stephanie (Author) / Zambo, Ronald (Thesis advisor) / Jiménez, Rosa (Committee member) / Harrington, Timothy (Committee member) / Arizona State University (Publisher)
Created2013
Description
This study examined the intended and unintended consequences associated with the Education Value-Added Assessment System (EVAAS) as perceived and experienced by teachers in the Houston Independent School District (HISD). To evaluate teacher effectiveness, HISD is using EVAAS for high-stakes consequences more than any other district or state in the country. A large-scale electronic survey was used to investigate the model's reliability and validity; to determine whether teachers used the EVAAS data in formative ways as intended; to gather teachers' opinions on EVAAS's claimed benefits and statements; and to understand the unintended consequences that occurred as a result of EVAAS use in HISD. Mixed methods data collection and analyses were used to present the findings in user-friendly ways, particularly when using the words and experiences of the teachers themselves. Results revealed that the reliability of the EVAAS model produced split and inconsistent results among teacher participants, and teachers indicated that students biased the EVAAS results. The majority of teachers did not report similar EVAAS and principal observation scores, reducing the criterion-related validity of both measures of teacher quality. Teachers revealed discrepancies in the distribution of EVAAS reports, the awareness of trainings offered, and among principals' understanding of EVAAS across the district. This resulted in an underwhelming number of teachers who reportedly used EVAAS data for formative purposes. Teachers disagreed with EVAAS marketing claims, implying the majority did not believe EVAAS worked as intended and promoted. Additionally, many unintended consequences associated with the high-stakes use of EVAAS emerged through teachers' responses, which revealed among others that teachers felt heightened pressure and competition, which reduced morale and collaboration, and encouraged cheating or teaching to the test in attempt to raise EVAAS scores. This study is one of the first to investigate how the EVAAS model works in practice and provides a glimpse of whether value-added models might produce desired outcomes and encourage best teacher practices. This is information of which policymakers, researchers, and districts should be aware and consider when implementing the EVAAS, or any value-added model for teacher evaluation, as many of the reported issues are not specific to the EVAAS model.
ContributorsCollins, Clarin (Author) / Amrein-Beardsley, Audrey (Thesis advisor) / Berliner, David C. (Committee member) / Fischman, Gustavo E (Committee member) / Arizona State University (Publisher)
Created2012
DescriptionThis research was focused on determining the relationship between student achievement, teacher evaluation scores, and performance pay for an Arizona school district.
ContributorsRazo, Adrienne (Author) / Spencer, Dee Ann (Thesis advisor) / Appleton, Nicholas (Committee member) / Jasmer, Shelby (Committee member) / Arizona State University (Publisher)
Created2014
Description
Policymakers at the national level have recently initiated K-12 education reforms focused on teacher quality and teacher evaluation. Far-reaching legislation was subsequently enacted in the state of Arizona requiring schools to adopt standards-based teacher evaluation systems and link them to student outcomes. The end product is to result in annual summative measures of teacher effectiveness. Because of this, Arizona school administrators have become concerned about rapidly becoming experts in high-stakes teacher evaluation. Principals rarely have time to come together to talk about teacher evaluation, and consider the reliability of their evaluations and how to use teacher evaluation to help teachers improve their practice. This action research study focused on a group of nine administrators in a small urban district grappling with a more complex and high-stakes teacher evaluation system. An existing community of practice was engaged to help administrators become more effective, fair, and consistent evaluators. Activities were designed to engage the group in dynamic, contextualized learning. Participants interacted in small groups to interpret the meaning of newly adopted evaluation instruments and professional teaching standards, share practical knowledge, and compare teacher evaluation experiences in classrooms. Data were gathered with mixed methods. Prior to, and immediately after engaging in this 20-week innovation, principals and district administrators were given two surveys and interviewed about teacher evaluation. Additionally, a detailed record of this project was kept in the form of meeting records and a research journal. Quantitative and qualitative data were triangulated to validate findings. Results identified concerns and understandings of administrators as they attempted to come to a shared consensus regarding teacher evaluation, increase inter-rater reliability, and use teacher evaluation to improve professional practice. As a result of working and learning together administrators lowered their concerns about inter-rater reliability. Other concerns, however, remained and grew. Administrators found the process of gaining a common understanding of teacher evaluation to be complex and far more time-consuming than anticipated. Intense concerns about alignment of the evaluation system with other reforms led these administrators to consider modifications in their evaluation system. Implications from this study can be used to help other administrators grappling with common concerns.
ContributorsCanelake, Chris (Author) / Zambo, Debby (Thesis advisor) / Hansen, Cory (Thesis advisor) / Ashby, Tacy (Committee member) / Arizona State University (Publisher)
Created2012
Description
Teacher evaluation policies have recently shifted in the United States. For the first time in history, many states, districts, and administrators are now required to evaluate teachers by methods that are up to 50% based on their "value-added," as demonstrated at the classroom-level by growth on student achievement data over time. Other related instruments and methods, such as classroom observations and rubrics, have also become common practices in teacher evaluation systems. Such methods are consistent with the neoliberal discourse that has dominated the social and political sphere for the past three decades. Employing a discourse analytic approach that called upon a governmentality framework, the author used a complementary approach to understand how contemporary teacher evaluation polices, practices, and instruments work to discursively (re)define teachers and teacher quality in terms of their market value.
For the first part of the analysis, the author collected and analyzed documents and field notes related to the teacher evaluation system at one urban middle school. The analysis included official policy documents, official White House speeches and press releases, evaluation system promotional materials, evaluator training materials, and the like. For the second part of the analysis, she interviewed teachers and their evaluators at the local middle school in order to understand how the participants had embodied the market-based discourse to define themselves as teachers and qualify their practice, quality, and worth accordingly.
The findings of the study suggest that teacher evaluation policies, practices, and instruments make possible a variety of techniques, such as numericization, hierarchical surveillance, normalizing judgments, and audit, in order to first make teachers objects of knowledge and then act upon that knowledge to manage teachers' conduct. The author also found that teachers and their evaluators have taken up this discourse in order to think about and act upon themselves as responsibilized subjects. Ultimately, the author argues that while much of the attention related to teacher evaluations has focused on the instruments used to measure the construct of teacher quality, that teacher evaluation instruments work in a mutually constitutive ways to discursively shape the construct of teacher quality.
For the first part of the analysis, the author collected and analyzed documents and field notes related to the teacher evaluation system at one urban middle school. The analysis included official policy documents, official White House speeches and press releases, evaluation system promotional materials, evaluator training materials, and the like. For the second part of the analysis, she interviewed teachers and their evaluators at the local middle school in order to understand how the participants had embodied the market-based discourse to define themselves as teachers and qualify their practice, quality, and worth accordingly.
The findings of the study suggest that teacher evaluation policies, practices, and instruments make possible a variety of techniques, such as numericization, hierarchical surveillance, normalizing judgments, and audit, in order to first make teachers objects of knowledge and then act upon that knowledge to manage teachers' conduct. The author also found that teachers and their evaluators have taken up this discourse in order to think about and act upon themselves as responsibilized subjects. Ultimately, the author argues that while much of the attention related to teacher evaluations has focused on the instruments used to measure the construct of teacher quality, that teacher evaluation instruments work in a mutually constitutive ways to discursively shape the construct of teacher quality.
ContributorsHolloway-Libell, Jessica (Author) / Amrein-Beardsley, Audrey (Thesis advisor) / Anderson, Kate T. (Thesis advisor) / Berliner, David C. (Committee member) / Arizona State University (Publisher)
Created2014
Description
This study examined the shared understanding between principals and special education teachers (SETs) regarding the instructional practices and components essential for special education teacher evaluations. Shared mental models and Cultural Consensus Theory (CCT) guided the methodology. Cultural Domain Analysis (CDA), Cultural Consensus Analysis (CCA), and Discourse Analysis provided the structure for data collection and analysis. Findings suggest that principals and SETs share consensus about how evaluation items are grouped yet differ in their understanding of how items are grouped overall. Principals tended to group items into two main topic areas: evaluation items applicable to all teachers and special education-specific themes. SETs tended to group items into three topic areas including (a) working in classrooms; (b) working with other adults; and (c) compliance-related activities. An existing pre-conference observation checklist was enhanced based on these results and was shared with one former principal, one SET, and one special education director. The SET focused on the importance of student agency, the former principal focused on high expectations for teachers, and the special education director focused on high expectations for SETs and the field of special education in general.
ContributorsRice, Kimberly Lynn (Author) / Mathur, Sarup (Thesis advisor) / Puckett, Kathleen (Committee member) / Jones, Nathan (Committee member) / Arizona State University (Publisher)
Created2023
Description
Over the past 20 years in the United States (U.S.), teachers have seen a marked
shift in how teacher evaluation policies govern the evaluation of their performance.
Spurred by federal mandates, teachers have been increasingly held accountable for their
students’ academic achievement, most notably through the use of value-added models
(VAMs)—a statistically complex tool that aims to isolate and then quantify the effect of
teachers on their students’ achievement. This increased focus on accountability ultimately
resulted in numerous lawsuits across the U.S. where teachers protested what they felt
were unfair evaluations informed by invalid, unreliable, and biased measures—most
notably VAMs.
While New Mexico’s teacher evaluation system was labeled as a “gold standard”
due to its purported ability to objectively and accurately differentiate between effective
and ineffective teachers, in 2015, teachers filed suit contesting the fairness and accuracy
of their evaluations. Amrein-Beardsley and Geiger’s (revise and resubmit) initial analyses
of the state’s teacher evaluation data revealed that the four individual measures
comprising teachers’ overall evaluation scores showed evidence of bias, and specifically,
teachers who taught in schools with different student body compositions (e.g., special
education students, poorer students, gifted students) had significantly different scores
than their peers. The purpose of this study was to expand upon these prior analyses by
investigating whether those conclusions still held true when controlling for a variety of
confounding factors at the school, class, and teacher levels, as such covariates were not
included in prior analyses.
Results from multiple linear regression analyses indicated that, overall, the
measures used to inform New Mexico teachers’ overall evaluation scores still showed
evidence of bias by school-level student demographic factors, with VAMs potentially
being the most susceptible and classroom observations being the least. This study is
especially unique given the juxtaposition of such a highly touted evaluation system also
being one where teachers contested its constitutionality. Study findings are important for
all education stakeholders to consider, especially as teacher evaluation systems and
related policies continue to be transformed.
shift in how teacher evaluation policies govern the evaluation of their performance.
Spurred by federal mandates, teachers have been increasingly held accountable for their
students’ academic achievement, most notably through the use of value-added models
(VAMs)—a statistically complex tool that aims to isolate and then quantify the effect of
teachers on their students’ achievement. This increased focus on accountability ultimately
resulted in numerous lawsuits across the U.S. where teachers protested what they felt
were unfair evaluations informed by invalid, unreliable, and biased measures—most
notably VAMs.
While New Mexico’s teacher evaluation system was labeled as a “gold standard”
due to its purported ability to objectively and accurately differentiate between effective
and ineffective teachers, in 2015, teachers filed suit contesting the fairness and accuracy
of their evaluations. Amrein-Beardsley and Geiger’s (revise and resubmit) initial analyses
of the state’s teacher evaluation data revealed that the four individual measures
comprising teachers’ overall evaluation scores showed evidence of bias, and specifically,
teachers who taught in schools with different student body compositions (e.g., special
education students, poorer students, gifted students) had significantly different scores
than their peers. The purpose of this study was to expand upon these prior analyses by
investigating whether those conclusions still held true when controlling for a variety of
confounding factors at the school, class, and teacher levels, as such covariates were not
included in prior analyses.
Results from multiple linear regression analyses indicated that, overall, the
measures used to inform New Mexico teachers’ overall evaluation scores still showed
evidence of bias by school-level student demographic factors, with VAMs potentially
being the most susceptible and classroom observations being the least. This study is
especially unique given the juxtaposition of such a highly touted evaluation system also
being one where teachers contested its constitutionality. Study findings are important for
all education stakeholders to consider, especially as teacher evaluation systems and
related policies continue to be transformed.
ContributorsGeiger, Tray (Author) / Amrein-Beardsley, Audrey (Thesis advisor) / Anderson, Kate (Committee member) / McGuire, Keon (Committee member) / Holloway, Jessica (Committee member) / Arizona State University (Publisher)
Created2020