Matching Items (2)
Filtering by
- All Subjects: policy
- Genre: Academic theses
- Creators: Amrein-Beardsley, Audrey
Description
ABSTRACT
This study examines validity evidence of a state policy-directed teacher evaluation system implemented in Arizona during school year 2012-2013. The purpose was to evaluate the warrant for making high stakes, consequential judgments of teacher competence based on value-added (VAM) estimates of instructional impact and observations of professional practice (PP). The research also explores educator influence (voice) in evaluation design and the role information brokers have in local decision making. Findings are situated in an evidentiary and policy context at both the LEA and state policy levels.
The study employs a single-phase, concurrent, mixed-methods research design triangulating multiple sources of qualitative and quantitative evidence onto a single (unified) validation construct: Teacher Instructional Quality. It focuses on assessing the characteristics of metrics used to construct quantitative ratings of instructional competence and the alignment of stakeholder perspectives to facets implicit in the evaluation framework. Validity examinations include assembly of criterion, content, reliability, consequential and construct articulation evidences. Perceptual perspectives were obtained from teachers, principals, district leadership, and state policy decision makers. Data for this study came from a large suburban public school district in metropolitan Phoenix, Arizona.
Study findings suggest that the evaluation framework is insufficient for supporting high stakes, consequential inferences of teacher instructional quality. This is based, in part on the following: (1) Weak associations between VAM and PP metrics; (2) Unstable VAM measures across time and between tested content areas; (3) Less than adequate scale reliabilities; (4) Lack of coherence between theorized and empirical PP factor structures; (5) Omission/underrepresentation of important instructional attributes/effects; (6) Stakeholder concerns over rater consistency, bias, and the inability of test scores to adequately represent instructional competence; (7) Negative sentiments regarding the system's ability to improve instructional competence and/or student learning; (8) Concerns regarding unintended consequences including increased stress, lower morale, harm to professional identity, and restricted learning opportunities; and (9) The general lack of empowerment and educator exclusion from the decision making process. Study findings also highlight the value of information brokers in policy decision making and the importance of having access to unbiased empirical information during the design and implementation phases of important change initiatives.
This study examines validity evidence of a state policy-directed teacher evaluation system implemented in Arizona during school year 2012-2013. The purpose was to evaluate the warrant for making high stakes, consequential judgments of teacher competence based on value-added (VAM) estimates of instructional impact and observations of professional practice (PP). The research also explores educator influence (voice) in evaluation design and the role information brokers have in local decision making. Findings are situated in an evidentiary and policy context at both the LEA and state policy levels.
The study employs a single-phase, concurrent, mixed-methods research design triangulating multiple sources of qualitative and quantitative evidence onto a single (unified) validation construct: Teacher Instructional Quality. It focuses on assessing the characteristics of metrics used to construct quantitative ratings of instructional competence and the alignment of stakeholder perspectives to facets implicit in the evaluation framework. Validity examinations include assembly of criterion, content, reliability, consequential and construct articulation evidences. Perceptual perspectives were obtained from teachers, principals, district leadership, and state policy decision makers. Data for this study came from a large suburban public school district in metropolitan Phoenix, Arizona.
Study findings suggest that the evaluation framework is insufficient for supporting high stakes, consequential inferences of teacher instructional quality. This is based, in part on the following: (1) Weak associations between VAM and PP metrics; (2) Unstable VAM measures across time and between tested content areas; (3) Less than adequate scale reliabilities; (4) Lack of coherence between theorized and empirical PP factor structures; (5) Omission/underrepresentation of important instructional attributes/effects; (6) Stakeholder concerns over rater consistency, bias, and the inability of test scores to adequately represent instructional competence; (7) Negative sentiments regarding the system's ability to improve instructional competence and/or student learning; (8) Concerns regarding unintended consequences including increased stress, lower morale, harm to professional identity, and restricted learning opportunities; and (9) The general lack of empowerment and educator exclusion from the decision making process. Study findings also highlight the value of information brokers in policy decision making and the importance of having access to unbiased empirical information during the design and implementation phases of important change initiatives.
ContributorsSloat, Edward F. (Author) / Wetzel, Keith (Thesis advisor) / Amrein-Beardsley, Audrey (Thesis advisor) / Ewbank, Ann (Committee member) / Shough, Lori (Committee member) / Arizona State University (Publisher)
Created2015
Description
The Committee on Rare and Endangered Wildlife Species (CREWS) of the U.S. Fish and Wildlife Service (FWS) made important and lasting contributions to one of the most significant pieces of environmental legislation in U.S. history: the Endangered Species Act of 1973 (ESA). CREWS was a prominent science-advisory body within the U.S. Department of the Interior (DOI) in the 1960s and 1970s, responsible for advising on the development of federal endangered-wildlife policy. The Committee took full advantage of its scientific and political authority by identifying a particular object of conservation--used in the development of the first U.S. list of endangered species--and establishing captive breeding as a primary conservation practice, both of which were written into the ESA and are employed in endangered-species listing and recovery to this day. Despite these important contributions to federal endangered-species practice and policy, CREWS has received little attention from historians of science or policy scholars. This dissertation is an empirical history of CREWS that draws on primary sources from the Smithsonian Institution (SI) Archives and a detailed analysis of the U.S. congressional record. The SI sources (including the records of the Bird and Mammal Laboratory, an FWS staffed research group stationed at the Smithsonian Institution) reveal the technical and political details of CREWS's advisory work. The congressional record provides evidence showing significant contributions of CREWS and its advisors and supervisors to the legislative process that resulted in the inclusion of key CREWS-inspired concepts and practices in the ESA. The foundational concepts and practices of the CREWS's research program drew from a number of areas currently of interest to several sub-disciplines that investigate the complex relationship between science and society. Among them are migratory bird conservation, systematics inspired by the Evolutionary Synthesis, species-focused ecology, captive breeding, reintroduction, and species transplantation. The following pages describe the role played by CREWS in drawing these various threads together and codifying them as endangered-species policy in the ESA.
ContributorsWinston, Johnny (Author) / Hamilton, Andrew (Thesis advisor) / Maienschein, Jane (Committee member) / Henson, Pamela (Committee member) / Collins, James (Committee member) / Minteer, Ben (Committee member) / Arizona State University (Publisher)
Created2011