Matching Items (4)
Filtering by
- All Subjects: Measurement
- Genre: Academic theses
- Creators: Corbin, William R.
- Creators: Amrein-Beardsley, Audrey
- Resource Type: Text
Description
ABSTRACT
This study examines validity evidence of a state policy-directed teacher evaluation system implemented in Arizona during school year 2012-2013. The purpose was to evaluate the warrant for making high stakes, consequential judgments of teacher competence based on value-added (VAM) estimates of instructional impact and observations of professional practice (PP). The research also explores educator influence (voice) in evaluation design and the role information brokers have in local decision making. Findings are situated in an evidentiary and policy context at both the LEA and state policy levels.
The study employs a single-phase, concurrent, mixed-methods research design triangulating multiple sources of qualitative and quantitative evidence onto a single (unified) validation construct: Teacher Instructional Quality. It focuses on assessing the characteristics of metrics used to construct quantitative ratings of instructional competence and the alignment of stakeholder perspectives to facets implicit in the evaluation framework. Validity examinations include assembly of criterion, content, reliability, consequential and construct articulation evidences. Perceptual perspectives were obtained from teachers, principals, district leadership, and state policy decision makers. Data for this study came from a large suburban public school district in metropolitan Phoenix, Arizona.
Study findings suggest that the evaluation framework is insufficient for supporting high stakes, consequential inferences of teacher instructional quality. This is based, in part on the following: (1) Weak associations between VAM and PP metrics; (2) Unstable VAM measures across time and between tested content areas; (3) Less than adequate scale reliabilities; (4) Lack of coherence between theorized and empirical PP factor structures; (5) Omission/underrepresentation of important instructional attributes/effects; (6) Stakeholder concerns over rater consistency, bias, and the inability of test scores to adequately represent instructional competence; (7) Negative sentiments regarding the system's ability to improve instructional competence and/or student learning; (8) Concerns regarding unintended consequences including increased stress, lower morale, harm to professional identity, and restricted learning opportunities; and (9) The general lack of empowerment and educator exclusion from the decision making process. Study findings also highlight the value of information brokers in policy decision making and the importance of having access to unbiased empirical information during the design and implementation phases of important change initiatives.
This study examines validity evidence of a state policy-directed teacher evaluation system implemented in Arizona during school year 2012-2013. The purpose was to evaluate the warrant for making high stakes, consequential judgments of teacher competence based on value-added (VAM) estimates of instructional impact and observations of professional practice (PP). The research also explores educator influence (voice) in evaluation design and the role information brokers have in local decision making. Findings are situated in an evidentiary and policy context at both the LEA and state policy levels.
The study employs a single-phase, concurrent, mixed-methods research design triangulating multiple sources of qualitative and quantitative evidence onto a single (unified) validation construct: Teacher Instructional Quality. It focuses on assessing the characteristics of metrics used to construct quantitative ratings of instructional competence and the alignment of stakeholder perspectives to facets implicit in the evaluation framework. Validity examinations include assembly of criterion, content, reliability, consequential and construct articulation evidences. Perceptual perspectives were obtained from teachers, principals, district leadership, and state policy decision makers. Data for this study came from a large suburban public school district in metropolitan Phoenix, Arizona.
Study findings suggest that the evaluation framework is insufficient for supporting high stakes, consequential inferences of teacher instructional quality. This is based, in part on the following: (1) Weak associations between VAM and PP metrics; (2) Unstable VAM measures across time and between tested content areas; (3) Less than adequate scale reliabilities; (4) Lack of coherence between theorized and empirical PP factor structures; (5) Omission/underrepresentation of important instructional attributes/effects; (6) Stakeholder concerns over rater consistency, bias, and the inability of test scores to adequately represent instructional competence; (7) Negative sentiments regarding the system's ability to improve instructional competence and/or student learning; (8) Concerns regarding unintended consequences including increased stress, lower morale, harm to professional identity, and restricted learning opportunities; and (9) The general lack of empowerment and educator exclusion from the decision making process. Study findings also highlight the value of information brokers in policy decision making and the importance of having access to unbiased empirical information during the design and implementation phases of important change initiatives.
ContributorsSloat, Edward F. (Author) / Wetzel, Keith (Thesis advisor) / Amrein-Beardsley, Audrey (Thesis advisor) / Ewbank, Ann (Committee member) / Shough, Lori (Committee member) / Arizona State University (Publisher)
Created2015
Description
Dimensional Metrology is the branch of science that determines length, angular, and geometric relationships within manufactured parts and compares them with required tolerances. The measurements can be made using either manual methods or sampled coordinate metrology (Coordinate measuring machines). Manual measurement methods have been in practice for a long time and are well accepted in the industry, but are slow for the present day manufacturing. On the other hand CMMs are relatively fast, but these methods are not well established yet. The major problem that needs to be addressed is the type of feature fitting algorithm used for evaluating tolerances. In a CMM the use of different feature fitting algorithms on a feature gives different values, and there is no standard that describes the type of feature fitting algorithm to be used for a specific tolerance. Our research is focused on identifying the feature fitting algorithm that is best used for each type of tolerance. Each algorithm is identified as the one to best represent the interpretation of geometric control as defined by the ASME Y14.5 standard and on the manual methods used for the measurement of a specific tolerance type. Using these algorithms normative procedures for CMMs are proposed for verifying tolerances. The proposed normative procedures are implemented as software. Then the procedures are verified by comparing the results from software with that of manual measurements.
To aid this research a library of feature fitting algorithms is developed in parallel. The library consists of least squares, Chebyshev and one sided fits applied on the features of line, plane, circle and cylinder. The proposed normative procedures are useful for evaluating tolerances in CMMs. The results evaluated will be in accordance to the standard. The ambiguity in choosing the algorithms is prevented. The software developed can be used in quality control for inspection purposes.
To aid this research a library of feature fitting algorithms is developed in parallel. The library consists of least squares, Chebyshev and one sided fits applied on the features of line, plane, circle and cylinder. The proposed normative procedures are useful for evaluating tolerances in CMMs. The results evaluated will be in accordance to the standard. The ambiguity in choosing the algorithms is prevented. The software developed can be used in quality control for inspection purposes.
ContributorsVemulapalli, Prabath (Author) / Shah, Jami J. (Thesis advisor) / Davidson, Joseph K. (Committee member) / Takahashi, Timothy (Committee member) / Arizona State University (Publisher)
Created2014
Description
Self-control has been shown to predict both health risk and health protective outcomes. Although top-down or “good” self-control is typically examined as a unidimensional construct, research on “poor” self-control suggests that multiple dimensions may be necessary to capture aspects of self-control. The current study sought to create a new brief survey measure of top-down self-control that differentiates between self-control capacity, internal motivation, and external motivation. Items were adapted from the Brief Self-Control Scale (BSCS; Tangney, Baumeister, & Boone, 2004) and were administered through two online surveys to 347 undergraduate students enrolled in introductory psychology courses at Arizona State University. The Self-Control Motivation and Capacity Survey (SCMCS) showed strong evidence of validity and reliability. Exploratory and confirmatory factor analyses supported a 3-factor structure of the scale consistent with the underlying theoretical model. The final 15-item measure demonstrated excellent model fit, chi-square = 89.722 p=.077, CFI = .989, RMSEA = .032, SRMR = .045. Despite several limitations including the cross-sectional nature of most analyses, self-control capacity, internal motivation, and external motivation uniquely related to various self-reported behavioral outcomes, and accounted for additional variance beyond that accounted for by the BSCS. Future studies are needed to establish the stability of multiple dimensions of self-control, and to develop state-like and domain-specific measures of self-control. While more research in this area is needed, the current study demonstrates the importance of studying multiple aspects of top-down self-control, and may ultimately facilitate the tailoring of interventions to the needs of individuals based on unique profiles of self-control capacity and motivation.
ContributorsPapova, Anna (Author) / Corbin, William R. (Thesis advisor) / Karoly, Paul (Committee member) / Brewer, Gene (Committee member) / Arizona State University (Publisher)
Created2016
Description
Self-control has been shown to be an important influence behind a variety of risk and protective behaviors, such as substance abuse. Although prior research points to the existence of multiple dimensions of self-control, this concept is not consistently defined and frequently only studied as a conglomerate in clinical research. The current study sought to examine how two experimental manipulations of subcomponents of self-control (motivation and self-efficacy) affect real-world consumptive behavior after accounting for executive function. Additionally, the validity and reliability of a brief state survey measure of perceived self-control capacity, internal motivation, and external motivation was tested. The goal was to examine how basic scientific principles involved in self-control translate into clinically relevant behaviors, which may inform understanding of momentary lapses in self-control behavior, potentially leading to novel prevention and intervention efforts. 94 college students completed a 1-2 hour laboratory protocol during which they completed survey and laboratory-based tasks of self-control and related behaviors, executive function, and ad libitum alcohol consumption. Results showed that the self-efficacy manipulation successfully increased perceived self-control capacity, although this did not lead to a significant reduction in consumption. The motivation manipulation neither increased motivation nor reduced consumption in this sample. However, the brief state survey measure of self-control subcomponents demonstrated strong test-retest reliability and distinction from trait self-control, demonstrating its viability for use in future research. By elucidating the relationships between specific mechanisms of self-control, laboratory-based tasks and manipulations, and real-world consumptive behaviors, prevention and intervention efforts for problems such as alcohol abuse may be tailored to the needs of the individual and made more impactful and cost-effective.
ContributorsPapova, Anna (Author) / Corbin, William R. (Thesis advisor) / Brewer, Gene (Committee member) / Karoly, Paul (Committee member) / McClure, Samuel (Committee member) / Arizona State University (Publisher)
Created2020