Matching Items (18)
Description
This study examined the intended and unintended consequences associated with the Education Value-Added Assessment System (EVAAS) as perceived and experienced by teachers in the Houston Independent School District (HISD). To evaluate teacher effectiveness, HISD is using EVAAS for high-stakes consequences more than any other district or state in the country. A large-scale electronic survey was used to investigate the model's reliability and validity; to determine whether teachers used the EVAAS data in formative ways as intended; to gather teachers' opinions on EVAAS's claimed benefits and statements; and to understand the unintended consequences that occurred as a result of EVAAS use in HISD. Mixed methods data collection and analyses were used to present the findings in user-friendly ways, particularly when using the words and experiences of the teachers themselves. Results revealed that the reliability of the EVAAS model produced split and inconsistent results among teacher participants, and teachers indicated that students biased the EVAAS results. The majority of teachers did not report similar EVAAS and principal observation scores, reducing the criterion-related validity of both measures of teacher quality. Teachers revealed discrepancies in the distribution of EVAAS reports, the awareness of trainings offered, and among principals' understanding of EVAAS across the district. This resulted in an underwhelming number of teachers who reportedly used EVAAS data for formative purposes. Teachers disagreed with EVAAS marketing claims, implying the majority did not believe EVAAS worked as intended and promoted. Additionally, many unintended consequences associated with the high-stakes use of EVAAS emerged through teachers' responses, which revealed among others that teachers felt heightened pressure and competition, which reduced morale and collaboration, and encouraged cheating or teaching to the test in attempt to raise EVAAS scores. This study is one of the first to investigate how the EVAAS model works in practice and provides a glimpse of whether value-added models might produce desired outcomes and encourage best teacher practices. This is information of which policymakers, researchers, and districts should be aware and consider when implementing the EVAAS, or any value-added model for teacher evaluation, as many of the reported issues are not specific to the EVAAS model.
ContributorsCollins, Clarin (Author) / Amrein-Beardsley, Audrey (Thesis advisor) / Berliner, David C. (Committee member) / Fischman, Gustavo E (Committee member) / Arizona State University (Publisher)
Created2012
Description
ABSTRACT
This study examines validity evidence of a state policy-directed teacher evaluation system implemented in Arizona during school year 2012-2013. The purpose was to evaluate the warrant for making high stakes, consequential judgments of teacher competence based on value-added (VAM) estimates of instructional impact and observations of professional practice (PP). The research also explores educator influence (voice) in evaluation design and the role information brokers have in local decision making. Findings are situated in an evidentiary and policy context at both the LEA and state policy levels.
The study employs a single-phase, concurrent, mixed-methods research design triangulating multiple sources of qualitative and quantitative evidence onto a single (unified) validation construct: Teacher Instructional Quality. It focuses on assessing the characteristics of metrics used to construct quantitative ratings of instructional competence and the alignment of stakeholder perspectives to facets implicit in the evaluation framework. Validity examinations include assembly of criterion, content, reliability, consequential and construct articulation evidences. Perceptual perspectives were obtained from teachers, principals, district leadership, and state policy decision makers. Data for this study came from a large suburban public school district in metropolitan Phoenix, Arizona.
Study findings suggest that the evaluation framework is insufficient for supporting high stakes, consequential inferences of teacher instructional quality. This is based, in part on the following: (1) Weak associations between VAM and PP metrics; (2) Unstable VAM measures across time and between tested content areas; (3) Less than adequate scale reliabilities; (4) Lack of coherence between theorized and empirical PP factor structures; (5) Omission/underrepresentation of important instructional attributes/effects; (6) Stakeholder concerns over rater consistency, bias, and the inability of test scores to adequately represent instructional competence; (7) Negative sentiments regarding the system's ability to improve instructional competence and/or student learning; (8) Concerns regarding unintended consequences including increased stress, lower morale, harm to professional identity, and restricted learning opportunities; and (9) The general lack of empowerment and educator exclusion from the decision making process. Study findings also highlight the value of information brokers in policy decision making and the importance of having access to unbiased empirical information during the design and implementation phases of important change initiatives.
This study examines validity evidence of a state policy-directed teacher evaluation system implemented in Arizona during school year 2012-2013. The purpose was to evaluate the warrant for making high stakes, consequential judgments of teacher competence based on value-added (VAM) estimates of instructional impact and observations of professional practice (PP). The research also explores educator influence (voice) in evaluation design and the role information brokers have in local decision making. Findings are situated in an evidentiary and policy context at both the LEA and state policy levels.
The study employs a single-phase, concurrent, mixed-methods research design triangulating multiple sources of qualitative and quantitative evidence onto a single (unified) validation construct: Teacher Instructional Quality. It focuses on assessing the characteristics of metrics used to construct quantitative ratings of instructional competence and the alignment of stakeholder perspectives to facets implicit in the evaluation framework. Validity examinations include assembly of criterion, content, reliability, consequential and construct articulation evidences. Perceptual perspectives were obtained from teachers, principals, district leadership, and state policy decision makers. Data for this study came from a large suburban public school district in metropolitan Phoenix, Arizona.
Study findings suggest that the evaluation framework is insufficient for supporting high stakes, consequential inferences of teacher instructional quality. This is based, in part on the following: (1) Weak associations between VAM and PP metrics; (2) Unstable VAM measures across time and between tested content areas; (3) Less than adequate scale reliabilities; (4) Lack of coherence between theorized and empirical PP factor structures; (5) Omission/underrepresentation of important instructional attributes/effects; (6) Stakeholder concerns over rater consistency, bias, and the inability of test scores to adequately represent instructional competence; (7) Negative sentiments regarding the system's ability to improve instructional competence and/or student learning; (8) Concerns regarding unintended consequences including increased stress, lower morale, harm to professional identity, and restricted learning opportunities; and (9) The general lack of empowerment and educator exclusion from the decision making process. Study findings also highlight the value of information brokers in policy decision making and the importance of having access to unbiased empirical information during the design and implementation phases of important change initiatives.
ContributorsSloat, Edward F. (Author) / Wetzel, Keith (Thesis advisor) / Amrein-Beardsley, Audrey (Thesis advisor) / Ewbank, Ann (Committee member) / Shough, Lori (Committee member) / Arizona State University (Publisher)
Created2015
Description
This mixed-methods study explored perceptions of new adjuncts on various trainings with regards to satisfying their professional and aspirational needs. Three trainings were offered in fall 2018 quarter as optional professional development: workshop, and two roundtable sessions. These trainings assisted adjuncts with their teaching skills, educational technology and pedagogy. Guidance was provided from experienced adjuncts and staff.
Surveys and interviews with adjuncts, along with a focus group with staff were the sources of data for this study. A repeated measures Analysis of Covariance (ANCOVA) model was utilized. Analysis of data showed that there was a positive and statistical significance of change in perceptions of adjuncts who participated in all trainings towards fulfilling their needs, as compared to those who did not participate in any trainings. Adjuncts perceived an improvement in their professional growth based on Herzberg’s motivation-hygiene theory and the trainings also fulfilled their higher-level growth needs based on Maslow’s hierarchical needs theory. A large practical significance was also found which measures the practical impact of such trainings at local communities of practice.
Surveys and interviews with adjuncts, along with a focus group with staff were the sources of data for this study. A repeated measures Analysis of Covariance (ANCOVA) model was utilized. Analysis of data showed that there was a positive and statistical significance of change in perceptions of adjuncts who participated in all trainings towards fulfilling their needs, as compared to those who did not participate in any trainings. Adjuncts perceived an improvement in their professional growth based on Herzberg’s motivation-hygiene theory and the trainings also fulfilled their higher-level growth needs based on Maslow’s hierarchical needs theory. A large practical significance was also found which measures the practical impact of such trainings at local communities of practice.
ContributorsSreekaram, Siddhartha (Author) / Marsh, Josephine (Thesis advisor) / Amrein-Beardsley, Audrey (Committee member) / Kim, Jeongeun (Committee member) / Arizona State University (Publisher)
Created2019
Description
An explanatory sequence mixed methods design was used to examine the effects of choice on the writing performance and motivation of college students (n = 242). The randomized control trial was followed by semi-structured interviews to determine the perceptions students (n = 20) held on the experiment as well the importance of choosing writing topics in college writing assignments. The effects of choice were tested as part of a real writing assignment that was included in nine sections of an introductory special education course. Results from hierarchical linear modeling (HLM) analyses found choice had a statistically significant negative effect on holistic writing quality, number of words written, and intrinsic writing motivation. Findings from the semi-structured interviews provided context for understanding the unexpected quantitative results.
ContributorsAitken, April Angelique (Author) / Graham, Steve (Thesis advisor) / Anderson, Kate (Committee member) / Marley, Scott (Committee member) / Patall, Erika (Committee member) / Puckett, Kathleen (Committee member) / Arizona State University (Publisher)
Created2018
Description
In 2010, the Arizona Legislature established a performance-based diploma initiative known as Move On When Ready (MOWR). The policy relies on an education model designed to evaluate students' college and career readiness by measuring their academic ability to succeed in the first credit-bearing course in community college. Move On When Ready is a structurally oriented, qualification system that attempts to attain a relatively narrow goal: increase the number of students able to successfully perform at a college-level academic standard. By relying on a set of benchmarked assessments to measure success and failure, MOWR propagates a categorical binary. The binary establishes explicit performance criteria on a set of examinations students are required to meet in order to earn a high school qualification that, by design, certifies whether students are ready or not ready for college.
This study sought to reveal how students’ perceptions of the policy and schooling in general affect their understanding of the concept of college readiness and the college readiness binary and to identify factors that help formulate those perceptions. This interpretivist, qualitative study relied on analysis of multiple face-to-face interviews with students to better understand how they think and act within the context of Move On When Ready, paying particular attention to students from historically vulnerable minority subgroups (e.g., the Latina (a)/Hispanic sub-population) enrolled in two schools deploying the MOWR strategy.
Findings suggest that interviewed students understand little about MOWR's design, intent or implications for their future educational trajectories. Moreover, what they believe is generally misinformed, regardless of aspiration, socio-cultural background, or academic standing. School-based sources of messaging (e.g., teachers and administrators) supply the bulk of information to students about MOWR. However, in these two schools, the flow of information is constricted. In addition, the information conveyed is either distorted by message mediators or misinterpreted by the students. The data reveal that formal and informal mediators of policy messages influence students’ engagement with the policy and affect students’ capacity to play an active role in determining the policy’s effect on their educational outcomes.
This study sought to reveal how students’ perceptions of the policy and schooling in general affect their understanding of the concept of college readiness and the college readiness binary and to identify factors that help formulate those perceptions. This interpretivist, qualitative study relied on analysis of multiple face-to-face interviews with students to better understand how they think and act within the context of Move On When Ready, paying particular attention to students from historically vulnerable minority subgroups (e.g., the Latina (a)/Hispanic sub-population) enrolled in two schools deploying the MOWR strategy.
Findings suggest that interviewed students understand little about MOWR's design, intent or implications for their future educational trajectories. Moreover, what they believe is generally misinformed, regardless of aspiration, socio-cultural background, or academic standing. School-based sources of messaging (e.g., teachers and administrators) supply the bulk of information to students about MOWR. However, in these two schools, the flow of information is constricted. In addition, the information conveyed is either distorted by message mediators or misinterpreted by the students. The data reveal that formal and informal mediators of policy messages influence students’ engagement with the policy and affect students’ capacity to play an active role in determining the policy’s effect on their educational outcomes.
ContributorsSilver, Michael Greg (Author) / Berliner, David C. (Thesis advisor) / Fischman, Gustavo (Committee member) / Amrein-Beardsley, Audrey (Committee member) / Arizona State University (Publisher)
Created2015
Description
Employing Queer Intersectionality, this study explored how undocuqueer activists made sense of, interacted and worked within the intersection of their LGBTQ and undocumented experience. Participants ascribed three overarching self-meanings: Vulnerability, Complexity, and Resilience. These self-meanings describe the ways participants perceived the interplay of their gender, sexuality and immigration status within the current sociopolitical context of the U.S. Recognizing their vulnerability within a state of illegibility, participants described a sense of exclusion within spaces of belonging, and wariness managing relationships with others; opting for more complex self-definitions, they resisted simplistic conceptions of identity that rendered their social locations invisible (e.g., homonormativity, heteronormativity, DREAMer); and describing themselves as resilient, they described surviving societal as well as familial rejection even when surviving seemed impossible to do so. Interacting and working within the intersection of gender, sexuality and immigration status, participants described identity negotiation and coming out as a form of resistance to institutionalized oppression, and resilience amidst simultaneous anti-immigrant, xenophobic and heterosexist power structures. Participants learned to live in multiple worlds at the same time, and embrace the multiplicity of their undocuqueer identity while seeking to bridge their communities through stories, activism and peer education. This study has implications for further understanding the way that queer politics and identity interact/ relate with various axes of inequality.
ContributorsCisneros, Jesus (Author) / Ott, Molly (Thesis advisor) / Fischman, Gustavo (Thesis advisor) / Anderson, Kate (Committee member) / Arizona State University (Publisher)
Created2015
Description
This study examines ninth graders’ negotiation of meaning with one canonical work, Romeo and Juliet. The study’s sample was 88% Latino at a Title I high school. The study adopts a sociocultural view of literacy and learning. I employed ethnographic methods (participant observation, data collection, interviews, and focus groups) to investigate the teacher’s instructional approaches and the literacy practices used while teaching the canonical work. With a focus on students’ interpretations, I examined what they said and wrote about Romeo and Juliet. One finding was that the teacher employed instructional approaches that facilitated literacy practices that allowed students to draw on their cultural backgrounds, personal lived experiences, and values as they engaged with Romeo and Juliet. As instructional approaches and literacy practices became routine, students formed a community of learners. Because the teacher allowed students to discuss their ideas before, during, and after reading, students were provided with multiple perspectives to think about as they read and negotiated meaning. A second finding was that students drew on their personal lived experiences, backgrounds, and values as they made sense and negotiated the meaning of Romeo and Juliet’s plot and characters. Although the text’s meaning was not always obvious to students, in their work they showed their growing awareness that multiple interpretations were welcomed and important in the teacher’s classroom. Through the unit, students came to recognize that their own and their peers’ understandings, negotiations, and interpretations of the canonical work were informed by a variety of complex factors. Students came to find relevance in the text’s themes and characters to their experiences as adolescents. The study’s findings point to the importance of allowing students to draw from their cultural backgrounds and experiences as they negotiate meaning with texts, specifically canonical ones, and to welcome and encourage multiple meanings in the English classroom.
ContributorsBaez, Felipe J., Jr (Author) / Warriner, Doris (Thesis advisor) / Anderson, Kate (Committee member) / Early, Jessica (Committee member) / Gee, Elisabeth (Committee member) / Arizona State University (Publisher)
Created2016
Description
In this study, I investigate how secondary reclassified ELLs use the Learning Management System Schoology in three secondary English classrooms. Particularly, I focus on the digital literacy practices reclassified ELLs use as they navigate Schoology to complete a multi-page research paper. In examining the digital literacy practices of secondary reclassified ELLs who have recently exited the language development program, I add to research in the fields of New Literacies and Multiliteracies, sociocultural approaches to learning, and identity studies.
In this qualitative study, I employed ethnographic techniques (i.e., data collection, participant observation, interviewing, and collection of archived material and digital artifacts stored in Schoology). I drew from communities of practice and identity frameworks to examine focal participants' literacy practices when participating in the online space of Schoology and provided screenshots to showcase this participation. I examined email exchanges that were co-created by teacher and student that demonstrated their reliance on a digital tool to continue the teaching and learning processes. I exhibit screenshots of focal participants' engagement with the revision process as they used Schoology’s and Microsoft Word's digital editing tools. Finally, I examined focal participants' participation in Schoology's online discussion forum to highlight how they revealed aspects of their identities and performed these identities in a mainstream-learning environment as well.
My analysis establishes that focal participants' access to an LMS like Schoology and other digital spaces (e.g., email) supports the language learning and literacy practices of reclassified ELLs. In addition, my analysis of focal participants' digital and communication practices shows that they contributed to their agency, positioned themselves as empowered and knowledgeable learners, and performed the role of "peer as mentor" when providing feedback to their peers. Finally, in my analysis of focal participants' inventories of digital literacy practices, I discovered that their engagement in Schoology for the purposes of learning and communication reinforced their language learning, both traditional and digital literacies, and overall academic achievement. Findings of this study emphasizes the importance of technology integration at the secondary level so that all students have equal access to digital and multimodal ways of learning in today's digital age.
In this qualitative study, I employed ethnographic techniques (i.e., data collection, participant observation, interviewing, and collection of archived material and digital artifacts stored in Schoology). I drew from communities of practice and identity frameworks to examine focal participants' literacy practices when participating in the online space of Schoology and provided screenshots to showcase this participation. I examined email exchanges that were co-created by teacher and student that demonstrated their reliance on a digital tool to continue the teaching and learning processes. I exhibit screenshots of focal participants' engagement with the revision process as they used Schoology’s and Microsoft Word's digital editing tools. Finally, I examined focal participants' participation in Schoology's online discussion forum to highlight how they revealed aspects of their identities and performed these identities in a mainstream-learning environment as well.
My analysis establishes that focal participants' access to an LMS like Schoology and other digital spaces (e.g., email) supports the language learning and literacy practices of reclassified ELLs. In addition, my analysis of focal participants' digital and communication practices shows that they contributed to their agency, positioned themselves as empowered and knowledgeable learners, and performed the role of "peer as mentor" when providing feedback to their peers. Finally, in my analysis of focal participants' inventories of digital literacy practices, I discovered that their engagement in Schoology for the purposes of learning and communication reinforced their language learning, both traditional and digital literacies, and overall academic achievement. Findings of this study emphasizes the importance of technology integration at the secondary level so that all students have equal access to digital and multimodal ways of learning in today's digital age.
ContributorsHurtado, Fernando (Author) / Warriner, Doris (Thesis advisor) / Gee, Elisabeth (Committee member) / Anderson, Kate (Committee member) / Arizona State University (Publisher)
Created2017
Description
This study explores the relationships and implications of Foucault's genealogical analytic, his most recently published course, The Punitive Society and its connections to Discipline and Punish through an analysis of productive power, and the potential offerings for educational research. The purpose of this study is to clarify Foucault's genealogical approach in making it more accessible to educational researchers, to investigate the applications and significance of Foucault's most recently available lectures to education, and to analyze Foucault's reimagining of the notion of power as it is developed throughout the lectures and fully realized in Discipline and Punish to better develop an analytic lens from which to interrogate relations of power in pedagogical practices.
ContributorsSusser, Erica (Author) / Carlson, David L. (Thesis advisor) / Anderson, Kate (Committee member) / Koro-Ljungberg, Mirka (Committee member) / Arizona State University (Publisher)
Created2016
Description
Public schools across the country are increasingly dealing with children who enter schools speaking a language other than English and Arizona is not the exception. As a result, schools across the country have to adequately ensure this populations’ academic achievement, which is directly impacted by English proficiency and ELLs (English Language Learners) program placement. However, restrictive language policies such as Proposition 203, the four-hour English Language Development (ELD) block, and the exclusion of ELLs from Dual Language Programs (DLPs) in Arizona are not effectively preparing linguistic minority and ethnic student populations for academic achievement and competitiveness in a global economy.
For the first part of the analysis, the author examined bilingual education and DLPs policies, access, and practices impacting Latina/o communities by utilizing a case study methodology framework to present the phenomenon of DLPs in a state that by law only supports English only education. The author discussed the case study research design to answer the research questions: (1) Which public k-12 schools are implementing Dual Language Programs (DLPs) in the state of AZ? (2) What are the DLPs’ characteristics? (3) Where are the schools located? (4) What are the stakeholder participants’ perceptions of DLPs and the context in which these DLPs navigate? The author also describe the context of the study, the participants, data, and the data collection process, as well as the analytical techniques she used to make sense of the data and draw findings.
The findings suggest that bilingual education programs in the form of DLPs are being implemented in the state of Arizona despite the English only law of Proposition 203, English for the Children. The growing demand for DLPs is increasing the implementation of such programs, however, language minority students that are classified as ELL are excluded from being part of such programs. Moreover, the findings of the study suggest that although bilingual education is being implemented in Arizona through DLPs, language minority education policy is being negatively influenced by Interest Convergence tenets and Racist Nativist ideology in which the interest of the dominant culture are further advanced to the detriment of minority groups’ interest.
For the first part of the analysis, the author examined bilingual education and DLPs policies, access, and practices impacting Latina/o communities by utilizing a case study methodology framework to present the phenomenon of DLPs in a state that by law only supports English only education. The author discussed the case study research design to answer the research questions: (1) Which public k-12 schools are implementing Dual Language Programs (DLPs) in the state of AZ? (2) What are the DLPs’ characteristics? (3) Where are the schools located? (4) What are the stakeholder participants’ perceptions of DLPs and the context in which these DLPs navigate? The author also describe the context of the study, the participants, data, and the data collection process, as well as the analytical techniques she used to make sense of the data and draw findings.
The findings suggest that bilingual education programs in the form of DLPs are being implemented in the state of Arizona despite the English only law of Proposition 203, English for the Children. The growing demand for DLPs is increasing the implementation of such programs, however, language minority students that are classified as ELL are excluded from being part of such programs. Moreover, the findings of the study suggest that although bilingual education is being implemented in Arizona through DLPs, language minority education policy is being negatively influenced by Interest Convergence tenets and Racist Nativist ideology in which the interest of the dominant culture are further advanced to the detriment of minority groups’ interest.
ContributorsGomez Gonzalez, Laura M (Author) / Jimenez-Silva, Margarita (Thesis advisor) / Amrein-Beardsley, Audrey (Thesis advisor) / Combs, Mary C (Committee member) / Arizona State University (Publisher)
Created2016