Matching Items (4)
Filtering by
- All Subjects: Evaluation
- Creators: Accetta, Danielle
- Creators: Amrein-Beardsley, Audrey
- Creators: Armfield, Jessica Ann
- Status: Published
Description
ABSTRACT
This study examines validity evidence of a state policy-directed teacher evaluation system implemented in Arizona during school year 2012-2013. The purpose was to evaluate the warrant for making high stakes, consequential judgments of teacher competence based on value-added (VAM) estimates of instructional impact and observations of professional practice (PP). The research also explores educator influence (voice) in evaluation design and the role information brokers have in local decision making. Findings are situated in an evidentiary and policy context at both the LEA and state policy levels.
The study employs a single-phase, concurrent, mixed-methods research design triangulating multiple sources of qualitative and quantitative evidence onto a single (unified) validation construct: Teacher Instructional Quality. It focuses on assessing the characteristics of metrics used to construct quantitative ratings of instructional competence and the alignment of stakeholder perspectives to facets implicit in the evaluation framework. Validity examinations include assembly of criterion, content, reliability, consequential and construct articulation evidences. Perceptual perspectives were obtained from teachers, principals, district leadership, and state policy decision makers. Data for this study came from a large suburban public school district in metropolitan Phoenix, Arizona.
Study findings suggest that the evaluation framework is insufficient for supporting high stakes, consequential inferences of teacher instructional quality. This is based, in part on the following: (1) Weak associations between VAM and PP metrics; (2) Unstable VAM measures across time and between tested content areas; (3) Less than adequate scale reliabilities; (4) Lack of coherence between theorized and empirical PP factor structures; (5) Omission/underrepresentation of important instructional attributes/effects; (6) Stakeholder concerns over rater consistency, bias, and the inability of test scores to adequately represent instructional competence; (7) Negative sentiments regarding the system's ability to improve instructional competence and/or student learning; (8) Concerns regarding unintended consequences including increased stress, lower morale, harm to professional identity, and restricted learning opportunities; and (9) The general lack of empowerment and educator exclusion from the decision making process. Study findings also highlight the value of information brokers in policy decision making and the importance of having access to unbiased empirical information during the design and implementation phases of important change initiatives.
This study examines validity evidence of a state policy-directed teacher evaluation system implemented in Arizona during school year 2012-2013. The purpose was to evaluate the warrant for making high stakes, consequential judgments of teacher competence based on value-added (VAM) estimates of instructional impact and observations of professional practice (PP). The research also explores educator influence (voice) in evaluation design and the role information brokers have in local decision making. Findings are situated in an evidentiary and policy context at both the LEA and state policy levels.
The study employs a single-phase, concurrent, mixed-methods research design triangulating multiple sources of qualitative and quantitative evidence onto a single (unified) validation construct: Teacher Instructional Quality. It focuses on assessing the characteristics of metrics used to construct quantitative ratings of instructional competence and the alignment of stakeholder perspectives to facets implicit in the evaluation framework. Validity examinations include assembly of criterion, content, reliability, consequential and construct articulation evidences. Perceptual perspectives were obtained from teachers, principals, district leadership, and state policy decision makers. Data for this study came from a large suburban public school district in metropolitan Phoenix, Arizona.
Study findings suggest that the evaluation framework is insufficient for supporting high stakes, consequential inferences of teacher instructional quality. This is based, in part on the following: (1) Weak associations between VAM and PP metrics; (2) Unstable VAM measures across time and between tested content areas; (3) Less than adequate scale reliabilities; (4) Lack of coherence between theorized and empirical PP factor structures; (5) Omission/underrepresentation of important instructional attributes/effects; (6) Stakeholder concerns over rater consistency, bias, and the inability of test scores to adequately represent instructional competence; (7) Negative sentiments regarding the system's ability to improve instructional competence and/or student learning; (8) Concerns regarding unintended consequences including increased stress, lower morale, harm to professional identity, and restricted learning opportunities; and (9) The general lack of empowerment and educator exclusion from the decision making process. Study findings also highlight the value of information brokers in policy decision making and the importance of having access to unbiased empirical information during the design and implementation phases of important change initiatives.
ContributorsSloat, Edward F. (Author) / Wetzel, Keith (Thesis advisor) / Amrein-Beardsley, Audrey (Thesis advisor) / Ewbank, Ann (Committee member) / Shough, Lori (Committee member) / Arizona State University (Publisher)
Created2015
Description
There is a serious need for early childhood intervention practices for children who are living at or below the poverty line. Since 1965 Head Start has provided a federally funded, free preschool program for children in this population. The City of Phoenix Head Start program consists of nine delegate agencies, seven of which reside in school districts. These agencies are currently not conducting local longitudinal evaluations of their preschool graduates. The purpose of this study was to recommend initial steps the City of Phoenix grantee and the delegate agencies can take to begin a longitudinal evaluation process of their Head Start programs. Seven City of Phoenix Head Start agency directors were interviewed. These interviews provided information about the attitudes of the directors when considering longitudinal evaluations and how Head Start already evaluates their programs through internal assessments. The researcher also took notes on the Third Grade Follow-Up to the Head Start Executive Summary in order to make recommendations to the City of Phoenix Head Start programs about the best practices for longitudinal student evaluations.
ContributorsArmfield, Jessica Ann (Author) / Duggan, Mary Anne (Thesis director) / Dumka, Larry (Committee member) / Barrett, The Honors College (Contributor) / T. Denny Sanford School of Social and Family Dynamics (Contributor) / School of International Letters and Cultures (Contributor)
Created2014-05
Description
Preventing heat-associated morbidity and mortality is a public health priority in Maricopa County, Arizona (United States). The objective of this project was to evaluate Maricopa County cooling centers and gain insight into their capacity to provide relief for the public during extreme heat events. During the summer of 2014, 53 cooling centers were evaluated to assess facility and visitor characteristics. Maricopa County staff collected data by directly observing daily operations and by surveying managers and visitors. The cooling centers in Maricopa County were often housed within community, senior, or religious centers, which offered various services for at least 1500 individuals daily. Many visitors were unemployed and/or homeless. Many learned about a cooling center by word of mouth or by having seen the cooling center’s location. The cooling centers provide a valuable service and reach some of the region’s most vulnerable populations. This project is among the first to systematically evaluate cooling centers from a public health perspective and provides helpful insight to community leaders who are implementing or improving their own network of cooling centers.
ContributorsBerisha, Vjollca (Author) / Hondula, David M. (Author) / Roach, Matthew (Author) / White, Jessica R. (Author) / McKinney, Benita (Author) / Bentz, Darcie (Author) / Mohamed, Ahmed (Author) / Uebelherr, Joshua (Author) / Goodin, Kate (Author)
Created2016-09-23
Description
The goal of the first study was to characterize the Miocene arkosic conglomerate in the Goldfield Mountains to determine the paleocurrent direction and source of the cobbles. This conglomerate is tilted to the northeast and unconformably overlies Proterozoic basement. Imbrication measurements are scattered but suggest the direction of paleoflow was toward the northwest and northeast, which suggests the cobbles were sourced from the southeast and southwest. The abundance of Dripping Spring Quartzite and the presence of Barnes Conglomerate in the cobbles, suggests an Apache Group source. In addition, south-southeast of the map area, there are several rock units composed of the same material as cobbles within the arkosic conglomerate. The arkosic conglomerate was likely deposited during onset of mid-Cenozoic extension, where the resulting highlands could provide a nearby source for the cobbles. This nearby source is interpreted to be south-southeast of the study area. A second study examined the effectiveness of course reform conducted on an introductory undergraduate course sequence. Questions of this study included: (1) How does the curriculum cater to a student population with diverse goals? (2) How do reformed courses add educational value as perceived by the student? (3) How does the redesigned curriculum and the instructional strategies, as implemented, address the goals of the reform?
The curriculum addressed the goals of the reform by (1) creating more opportunities for students to gain the skills relevant to their future goals, (2) having students utilize big data to make observations, interpretations, and predictions, (3) engaging students in scientific collaboration through group work and discussion, (4) giving students the opportunity to utilize computer programs that apply across various subjects and fields (i.e. Excel, MS Word, ArcGIS), and (5) requiring students to conduct original research to solve a problem and present their results orally and in written form. These redesign efforts were successful in meeting the objectives, and majority of the student participants reported one or more of the reformed experiences were valuable to their education and future goals. An understanding of teaching methods and educational values held by undergraduate students within the School of Earth and Space Exploration can be adapted and applied across subjects.
ContributorsAccetta, Danielle (Author) / Reynolds, Stephen J (Thesis advisor) / Johnson, Julia K (Committee member) / Semken, Steven (Committee member) / Arizona State University (Publisher)
Created2021