Filtering by
- All Subjects: Education Policy
- All Subjects: Evaluation
- Creators: Amrein-Beardsley, Audrey
- Status: Published
Choropleth maps are a common form of online cartographic visualization. They reveal patterns in spatial distributions of a variable by associating colors with data values measured at areal units. Although this capability of pattern revelation has popularized the use of choropleth maps, existing methods for their online delivery are limited in supporting dynamic map generation from large areal data. This limitation has become increasingly problematic in online choropleth mapping as access to small area statistics, such as high-resolution census data and real-time aggregates of geospatial data streams, has never been easier due to advances in geospatial web technologies. The current literature shows that the challenge of large areal data can be mitigated through tiled maps where pre-processed map data are hierarchically partitioned into tiny rectangular images or map chunks for efficient data transmission. Various approaches have emerged lately to enable this tile-based choropleth mapping, yet little empirical evidence exists on their ability to handle spatial data with large numbers of areal units, thus complicating technical decision making in the development of online choropleth mapping applications. To fill this knowledge gap, this dissertation study conducts a scalability evaluation of three tile-based methods discussed in the literature: raster, scalable vector graphics (SVG), and HTML5 Canvas. For the evaluation, the study develops two test applications, generates map tiles from five different boundaries of the United States, and measures the response times of the applications under multiple test operations. While specific to the experimental setups of the study, the evaluation results show that the raster method scales better across various types of user interaction than the other methods. Empirical evidence also points to the superior scalability of Canvas to SVG in dynamic rendering of vector tiles, but not necessarily for partial updates of the tiles. These findings indicate that the raster method is better suited for dynamic choropleth rendering from large areal data, while Canvas would be more suitable than SVG when such rendering frequently involves complete updates of vector shapes.
This study examines validity evidence of a state policy-directed teacher evaluation system implemented in Arizona during school year 2012-2013. The purpose was to evaluate the warrant for making high stakes, consequential judgments of teacher competence based on value-added (VAM) estimates of instructional impact and observations of professional practice (PP). The research also explores educator influence (voice) in evaluation design and the role information brokers have in local decision making. Findings are situated in an evidentiary and policy context at both the LEA and state policy levels.
The study employs a single-phase, concurrent, mixed-methods research design triangulating multiple sources of qualitative and quantitative evidence onto a single (unified) validation construct: Teacher Instructional Quality. It focuses on assessing the characteristics of metrics used to construct quantitative ratings of instructional competence and the alignment of stakeholder perspectives to facets implicit in the evaluation framework. Validity examinations include assembly of criterion, content, reliability, consequential and construct articulation evidences. Perceptual perspectives were obtained from teachers, principals, district leadership, and state policy decision makers. Data for this study came from a large suburban public school district in metropolitan Phoenix, Arizona.
Study findings suggest that the evaluation framework is insufficient for supporting high stakes, consequential inferences of teacher instructional quality. This is based, in part on the following: (1) Weak associations between VAM and PP metrics; (2) Unstable VAM measures across time and between tested content areas; (3) Less than adequate scale reliabilities; (4) Lack of coherence between theorized and empirical PP factor structures; (5) Omission/underrepresentation of important instructional attributes/effects; (6) Stakeholder concerns over rater consistency, bias, and the inability of test scores to adequately represent instructional competence; (7) Negative sentiments regarding the system's ability to improve instructional competence and/or student learning; (8) Concerns regarding unintended consequences including increased stress, lower morale, harm to professional identity, and restricted learning opportunities; and (9) The general lack of empowerment and educator exclusion from the decision making process. Study findings also highlight the value of information brokers in policy decision making and the importance of having access to unbiased empirical information during the design and implementation phases of important change initiatives.
This study sought to reveal how students’ perceptions of the policy and schooling in general affect their understanding of the concept of college readiness and the college readiness binary and to identify factors that help formulate those perceptions. This interpretivist, qualitative study relied on analysis of multiple face-to-face interviews with students to better understand how they think and act within the context of Move On When Ready, paying particular attention to students from historically vulnerable minority subgroups (e.g., the Latina (a)/Hispanic sub-population) enrolled in two schools deploying the MOWR strategy.
Findings suggest that interviewed students understand little about MOWR's design, intent or implications for their future educational trajectories. Moreover, what they believe is generally misinformed, regardless of aspiration, socio-cultural background, or academic standing. School-based sources of messaging (e.g., teachers and administrators) supply the bulk of information to students about MOWR. However, in these two schools, the flow of information is constricted. In addition, the information conveyed is either distorted by message mediators or misinterpreted by the students. The data reveal that formal and informal mediators of policy messages influence students’ engagement with the policy and affect students’ capacity to play an active role in determining the policy’s effect on their educational outcomes.
For the first part of the analysis, the author examined bilingual education and DLPs policies, access, and practices impacting Latina/o communities by utilizing a case study methodology framework to present the phenomenon of DLPs in a state that by law only supports English only education. The author discussed the case study research design to answer the research questions: (1) Which public k-12 schools are implementing Dual Language Programs (DLPs) in the state of AZ? (2) What are the DLPs’ characteristics? (3) Where are the schools located? (4) What are the stakeholder participants’ perceptions of DLPs and the context in which these DLPs navigate? The author also describe the context of the study, the participants, data, and the data collection process, as well as the analytical techniques she used to make sense of the data and draw findings.
The findings suggest that bilingual education programs in the form of DLPs are being implemented in the state of Arizona despite the English only law of Proposition 203, English for the Children. The growing demand for DLPs is increasing the implementation of such programs, however, language minority students that are classified as ELL are excluded from being part of such programs. Moreover, the findings of the study suggest that although bilingual education is being implemented in Arizona through DLPs, language minority education policy is being negatively influenced by Interest Convergence tenets and Racist Nativist ideology in which the interest of the dominant culture are further advanced to the detriment of minority groups’ interest.
For the first part of the analysis, the author collected and analyzed documents and field notes related to the teacher evaluation system at one urban middle school. The analysis included official policy documents, official White House speeches and press releases, evaluation system promotional materials, evaluator training materials, and the like. For the second part of the analysis, she interviewed teachers and their evaluators at the local middle school in order to understand how the participants had embodied the market-based discourse to define themselves as teachers and qualify their practice, quality, and worth accordingly.
The findings of the study suggest that teacher evaluation policies, practices, and instruments make possible a variety of techniques, such as numericization, hierarchical surveillance, normalizing judgments, and audit, in order to first make teachers objects of knowledge and then act upon that knowledge to manage teachers' conduct. The author also found that teachers and their evaluators have taken up this discourse in order to think about and act upon themselves as responsibilized subjects. Ultimately, the author argues that while much of the attention related to teacher evaluations has focused on the instruments used to measure the construct of teacher quality, that teacher evaluation instruments work in a mutually constitutive ways to discursively shape the construct of teacher quality.
shift in how teacher evaluation policies govern the evaluation of their performance.
Spurred by federal mandates, teachers have been increasingly held accountable for their
students’ academic achievement, most notably through the use of value-added models
(VAMs)—a statistically complex tool that aims to isolate and then quantify the effect of
teachers on their students’ achievement. This increased focus on accountability ultimately
resulted in numerous lawsuits across the U.S. where teachers protested what they felt
were unfair evaluations informed by invalid, unreliable, and biased measures—most
notably VAMs.
While New Mexico’s teacher evaluation system was labeled as a “gold standard”
due to its purported ability to objectively and accurately differentiate between effective
and ineffective teachers, in 2015, teachers filed suit contesting the fairness and accuracy
of their evaluations. Amrein-Beardsley and Geiger’s (revise and resubmit) initial analyses
of the state’s teacher evaluation data revealed that the four individual measures
comprising teachers’ overall evaluation scores showed evidence of bias, and specifically,
teachers who taught in schools with different student body compositions (e.g., special
education students, poorer students, gifted students) had significantly different scores
than their peers. The purpose of this study was to expand upon these prior analyses by
investigating whether those conclusions still held true when controlling for a variety of
confounding factors at the school, class, and teacher levels, as such covariates were not
included in prior analyses.
Results from multiple linear regression analyses indicated that, overall, the
measures used to inform New Mexico teachers’ overall evaluation scores still showed
evidence of bias by school-level student demographic factors, with VAMs potentially
being the most susceptible and classroom observations being the least. This study is
especially unique given the juxtaposition of such a highly touted evaluation system also
being one where teachers contested its constitutionality. Study findings are important for
all education stakeholders to consider, especially as teacher evaluation systems and
related policies continue to be transformed.