Matching Items (3)
Filtering by

Clear all filters

Description
ABSTRACT

This study examines validity evidence of a state policy-directed teacher evaluation system implemented in Arizona during school year 2012-2013. The purpose was to evaluate the warrant for making high stakes, consequential judgments of teacher competence based on value-added (VAM) estimates of instructional impact and observations of professional practice (PP).

ABSTRACT

This study examines validity evidence of a state policy-directed teacher evaluation system implemented in Arizona during school year 2012-2013. The purpose was to evaluate the warrant for making high stakes, consequential judgments of teacher competence based on value-added (VAM) estimates of instructional impact and observations of professional practice (PP). The research also explores educator influence (voice) in evaluation design and the role information brokers have in local decision making. Findings are situated in an evidentiary and policy context at both the LEA and state policy levels.

The study employs a single-phase, concurrent, mixed-methods research design triangulating multiple sources of qualitative and quantitative evidence onto a single (unified) validation construct: Teacher Instructional Quality. It focuses on assessing the characteristics of metrics used to construct quantitative ratings of instructional competence and the alignment of stakeholder perspectives to facets implicit in the evaluation framework. Validity examinations include assembly of criterion, content, reliability, consequential and construct articulation evidences. Perceptual perspectives were obtained from teachers, principals, district leadership, and state policy decision makers. Data for this study came from a large suburban public school district in metropolitan Phoenix, Arizona.

Study findings suggest that the evaluation framework is insufficient for supporting high stakes, consequential inferences of teacher instructional quality. This is based, in part on the following: (1) Weak associations between VAM and PP metrics; (2) Unstable VAM measures across time and between tested content areas; (3) Less than adequate scale reliabilities; (4) Lack of coherence between theorized and empirical PP factor structures; (5) Omission/underrepresentation of important instructional attributes/effects; (6) Stakeholder concerns over rater consistency, bias, and the inability of test scores to adequately represent instructional competence; (7) Negative sentiments regarding the system's ability to improve instructional competence and/or student learning; (8) Concerns regarding unintended consequences including increased stress, lower morale, harm to professional identity, and restricted learning opportunities; and (9) The general lack of empowerment and educator exclusion from the decision making process. Study findings also highlight the value of information brokers in policy decision making and the importance of having access to unbiased empirical information during the design and implementation phases of important change initiatives.
ContributorsSloat, Edward F. (Author) / Wetzel, Keith (Thesis advisor) / Amrein-Beardsley, Audrey (Thesis advisor) / Ewbank, Ann (Committee member) / Shough, Lori (Committee member) / Arizona State University (Publisher)
Created2015
136787-Thumbnail Image.png
Description
There is a serious need for early childhood intervention practices for children who are living at or below the poverty line. Since 1965 Head Start has provided a federally funded, free preschool program for children in this population. The City of Phoenix Head Start program consists of nine delegate agencies,

There is a serious need for early childhood intervention practices for children who are living at or below the poverty line. Since 1965 Head Start has provided a federally funded, free preschool program for children in this population. The City of Phoenix Head Start program consists of nine delegate agencies, seven of which reside in school districts. These agencies are currently not conducting local longitudinal evaluations of their preschool graduates. The purpose of this study was to recommend initial steps the City of Phoenix grantee and the delegate agencies can take to begin a longitudinal evaluation process of their Head Start programs. Seven City of Phoenix Head Start agency directors were interviewed. These interviews provided information about the attitudes of the directors when considering longitudinal evaluations and how Head Start already evaluates their programs through internal assessments. The researcher also took notes on the Third Grade Follow-Up to the Head Start Executive Summary in order to make recommendations to the City of Phoenix Head Start programs about the best practices for longitudinal student evaluations.
Created2014-05
158565-Thumbnail Image.png
Description
Making significant progress on the U.N. Sustainable Development Goals (SDGs) needs change agents equipped with key competencies in sustainability. While thousands of sustainability programs have emerged at various educational levels over the past decade, there is, as of yet, no reliable way to assess if these programs successfully convey key

Making significant progress on the U.N. Sustainable Development Goals (SDGs) needs change agents equipped with key competencies in sustainability. While thousands of sustainability programs have emerged at various educational levels over the past decade, there is, as of yet, no reliable way to assess if these programs successfully convey key competencies in sustainability. This dissertation contributes to addressing this gap in three ways. First, it reviews the body of work on key competencies in sustainability. Based on broad agreement around five key competencies as well as an emerging set of three, an extended framework is outlined that can be used as unified set of learning objectives across sustainability programs. The next chapter reviews the scholarly work on assessing sustainability competencies. Based on this review, a typology of assessment tools is proposed offering guidance to both educators and researchers. Finally, drawing on experience of the four-year “Educating Future Change Agents” project, the last chapter explores the results from a diverse set of competency assessments in numerous courses. The study appraises assessment practices and results to demonstrate opportunities and challenges in the current state of assessing key competencies in sustainability. The results of this doctoral thesis are expected to make a practical and scholarly contribution to the teaching and learning in sustainability programs, in particular with regards to reliably assessing key competencies in sustainability.
ContributorsRedman, Aaron (Author) / Wiek, Arnim (Thesis advisor) / Barth, Matthias (Committee member) / Basile, George (Committee member) / Fischer, Daniel (Committee member) / Mochizuki, Yoko (Committee member) / Arizona State University (Publisher)
Created2020