Matching Items (8)
Filtering by
- All Subjects: Professional Development
- All Subjects: Educational evaluation
- Creators: Fischman, Gustavo E
- Creators: Beardsley, Audrey
Description
This study examined the intended and unintended consequences associated with the Education Value-Added Assessment System (EVAAS) as perceived and experienced by teachers in the Houston Independent School District (HISD). To evaluate teacher effectiveness, HISD is using EVAAS for high-stakes consequences more than any other district or state in the country. A large-scale electronic survey was used to investigate the model's reliability and validity; to determine whether teachers used the EVAAS data in formative ways as intended; to gather teachers' opinions on EVAAS's claimed benefits and statements; and to understand the unintended consequences that occurred as a result of EVAAS use in HISD. Mixed methods data collection and analyses were used to present the findings in user-friendly ways, particularly when using the words and experiences of the teachers themselves. Results revealed that the reliability of the EVAAS model produced split and inconsistent results among teacher participants, and teachers indicated that students biased the EVAAS results. The majority of teachers did not report similar EVAAS and principal observation scores, reducing the criterion-related validity of both measures of teacher quality. Teachers revealed discrepancies in the distribution of EVAAS reports, the awareness of trainings offered, and among principals' understanding of EVAAS across the district. This resulted in an underwhelming number of teachers who reportedly used EVAAS data for formative purposes. Teachers disagreed with EVAAS marketing claims, implying the majority did not believe EVAAS worked as intended and promoted. Additionally, many unintended consequences associated with the high-stakes use of EVAAS emerged through teachers' responses, which revealed among others that teachers felt heightened pressure and competition, which reduced morale and collaboration, and encouraged cheating or teaching to the test in attempt to raise EVAAS scores. This study is one of the first to investigate how the EVAAS model works in practice and provides a glimpse of whether value-added models might produce desired outcomes and encourage best teacher practices. This is information of which policymakers, researchers, and districts should be aware and consider when implementing the EVAAS, or any value-added model for teacher evaluation, as many of the reported issues are not specific to the EVAAS model.
ContributorsCollins, Clarin (Author) / Amrein-Beardsley, Audrey (Thesis advisor) / Berliner, David C. (Committee member) / Fischman, Gustavo E (Committee member) / Arizona State University (Publisher)
Created2012
Description
Increasing public criticism of traditional teacher evaluation systems based largely on classroom observations has spurred an unprecedented shift in the debate surrounding educational accountability policies, specifically about the purposes for and measures used to evaluate teachers. In response to growing public demand and associated federal mandates, states have been prompted to design and implement teacher evaluation systems that use increasingly available, statistically complex models (i.e., value-added) intended to isolate and measure the effects of individual teachers on student academic growth over time. The purpose of this study was to examine the perceptions of school administrators and teachers within one of the largest school districts in the state of Arizona with regards to the design and implementation of a federally-supported, state policy-directed teacher evaluation system based on professional practice and value-added measures. While much research has been conducted on teacher evaluation, few studies have examined teacher evaluation systems in context to better understand the standards of effectiveness used by school administrators and teachers to measure system effectiveness. The perceptions of school administrators and teachers, considering their lived experiences as the subjects of the nation's new and improved teacher evaluation systems in context, must be better understood if state and federal policymakers are to also better recognize and understand the consequences (intended and unintended) associated with the design and implementation of these systems in practice.
ContributorsPaufler, Noelle A (Author) / Amrein-Beardsley, Audrey L (Thesis advisor) / Berliner, David C. (Committee member) / Fischman, Gustavo E (Committee member) / Arizona State University (Publisher)
Created2014
Description
ABSTRACT
The purpose of this descriptive study was to gain an understanding of the confidence level held by third, fourth, and fifth grade teachers as to their preparedness for teaching the cognitive demands of the Common Core State Standards (Arizona's College and Career Ready Standards) to all students, in particular Hispanic students living in poverty, who occupy close to a third of all classroom seats in Arizona. The achievement gap between Hispanic students living in poverty and non-Hispanic students of non-poverty status is one of the largest achievement gaps in Arizona, which has existed with minimal change for more than 12 years. By gaining an understanding of the teachers' confidence in teaching critical thinking skills, further support and professional development is suggested to link a teacher's knowledge to instructional practice that in turn increases the academic achievement of Arizona's poor Hispanic students.
The process of gaining this understanding was by using a multi-dimensional survey with 500 third through fifth grade teachers in two uniquely different, but representative, Arizona school districts. Approximately one-third of those teachers responded to the multi-dimensional survey about teaching the critical thinking (CT) skills of Arizona's College and Career Ready Standards for English Language Arts. The survey asked teachers to rate their levels of preparedness for teaching CT to several types of students, to choose a CT definition, describe the relationship of CT and reading, explain how they teach CT to students who are reading below grade level, express the support they need to teach CT to those students, and rate the effectiveness of several CT classroom vignettes for different types of students. Although the questions involved several types of students, the primary focus was on exploring the teachers' position with teaching CT to Low SES Hispanic students.
A disconnect was revealed between the teachers' perception that they had the ability and knowledge necessary to teach critical thinking skills and their ability to identify ineffective critical thinking instructional practices. This disconnect may be interfering with the link between the professional development teachers are currently receiving to implement Common Core State Standards and teachers actively engaging in learning what is needed to effectively teach critical thinking skills to their students.
The purpose of this descriptive study was to gain an understanding of the confidence level held by third, fourth, and fifth grade teachers as to their preparedness for teaching the cognitive demands of the Common Core State Standards (Arizona's College and Career Ready Standards) to all students, in particular Hispanic students living in poverty, who occupy close to a third of all classroom seats in Arizona. The achievement gap between Hispanic students living in poverty and non-Hispanic students of non-poverty status is one of the largest achievement gaps in Arizona, which has existed with minimal change for more than 12 years. By gaining an understanding of the teachers' confidence in teaching critical thinking skills, further support and professional development is suggested to link a teacher's knowledge to instructional practice that in turn increases the academic achievement of Arizona's poor Hispanic students.
The process of gaining this understanding was by using a multi-dimensional survey with 500 third through fifth grade teachers in two uniquely different, but representative, Arizona school districts. Approximately one-third of those teachers responded to the multi-dimensional survey about teaching the critical thinking (CT) skills of Arizona's College and Career Ready Standards for English Language Arts. The survey asked teachers to rate their levels of preparedness for teaching CT to several types of students, to choose a CT definition, describe the relationship of CT and reading, explain how they teach CT to students who are reading below grade level, express the support they need to teach CT to those students, and rate the effectiveness of several CT classroom vignettes for different types of students. Although the questions involved several types of students, the primary focus was on exploring the teachers' position with teaching CT to Low SES Hispanic students.
A disconnect was revealed between the teachers' perception that they had the ability and knowledge necessary to teach critical thinking skills and their ability to identify ineffective critical thinking instructional practices. This disconnect may be interfering with the link between the professional development teachers are currently receiving to implement Common Core State Standards and teachers actively engaging in learning what is needed to effectively teach critical thinking skills to their students.
ContributorsFast, Deborah (Author) / Fischman, Gustavo E (Thesis advisor) / Harris, Connie (Committee member) / Bader, Beth (Committee member) / Arizona State University (Publisher)
Created2014
Description
As scrutiny of teacher effectiveness increases, there is a greater call for multiple instruments to measure teacher effectiveness and provide robust feedback to support teacher growth and development. Student perception surveys, questionnaires completed by K-12 students about their teachers, have increasingly been used to evaluate teachers and provide feedback. Situated in the Mary Lou Fulton Teachers College (MLFTC) at Arizona State University, this action research study used Attribution Theory, Sensemaking Theory, and research on teacher emotion to 1) document the experiences of pre-service teachers as they related to the administration and subsequent results from a student perception survey (SPS), and 2) examine the influence of two online professional development modules created to prepare teachers for the SPS process and make sense of the results. Teacher candidates participated in the SPS process in their final, year-long residency. Results from the mixed-methods study provided evidence that pre-service teachers had both positive and negative experiences that were influenced by the SPS results they received from their students. Also, depending on the results they received, teacher candidates either attributed the cause of the results to themselves or to characteristics of their students. Results from the study also indicate that teacher candidates use few strategies to make sense of the results and used those strategies to varying degrees. Pre-service teachers indicated that they regarded the modules as helpful in the sense-making process. Furthermore, evidence indicates that pre-service teachers value their students’ feedback.
ContributorsVilla, Lessita Ann Lorin (Author) / Jordan, Michelle E (Thesis advisor) / Beardsley, Audrey (Committee member) / Ribbens, Teressa (Committee member) / Arizona State University (Publisher)
Created2017
Description
ABSTRACT This study details the pilot of a collaborative peer-coaching model as a form of job embedded professional development, to guide teacher collaboration and planning based on benchmark assessments. The collaborative peer-coaching framework used (including reflection and collaboration about student data, and classroom instruction) was informed by the five propositions outlined by the National Board of Professional Teacher Standards (NBPTS). This intervention included teacher training, discussion (pre and post instruction), collaboration about student benchmark data, and classroom observations with further data collected through surveys and interviews. Using a mixed methods approach to data collection and analysis, I focused on how participants engaged in a collaborative peer-coaching model to guide their instruction based on the use of student data they collected from common benchmark assessments.
ContributorsWright, Kerri M (Author) / Beardsley, Audrey (Thesis advisor) / Serafini, Frank (Committee member) / Reyes, Elba (Committee member) / Arizona State University (Publisher)
Created2011
Description
The purpose of the project was to explore the extent to which an asynchronous online professional development (PD) model focusing on instructor presence would improve feedback and interactions with students. The study is grounded in Community of Inquiry theory, which situates learning at the intersection of teaching presence, social presence and cognitive presence. The study aimed to improve student success by empowering instructors to integrate engaging strategies and technology tools into fully online courses. The participants were 4 higher education instructors teaching in fully online degree programs delivered to 160-200 undergraduate students. For eight weeks the 4 instructors participated in the PD. The goals of the PD were to learn strategies for improving instructor presence and integrating student engagement opportunities in a collaborative online format. Data was collected from pre- and post-intervention offerings of the instructors’ courses to determine the impact of participation in the PD. Results suggest that the PD model was an effective intervention to increase presence and engagement. Presence and engagement were found to have increased in participants’ courses. Interactive video was found to serve multiple purposes including increasing instructor presence and student engagement, facilitating feedback between instructors and students, and elevating the level of cognitive presence of students. As a result, instructors and students both indicated a perception of improved interactions and feedback.
ContributorsRomanoski, Matthew Peter (Author) / Fischman, Gustavo E (Thesis advisor) / Wolf, Leigh (Committee member) / Farmakis, Heather (Committee member) / Arizona State University (Publisher)
Created2020
Description
Data use in higher education continues to increase as college and university leaders aim to meet accreditor and governmental expectations to use data to improve student outcomes. However, the steady increase in data use over the past decades has not been accompanied by an increase in employee data literacy in order for employees to use the data effectively. Further, inequitable student outcomes continue to persist in higher education, and more specifically at two-year community colleges, as potentially exacerbated by a lack of employee equity-mindedness. These concurrent problems—inadequate employee data literacy and persistent inequitable student outcomes—provide an opportunity to address both with one intervention. In this critical race, mixed-methods, action research study, I piloted an online professional development course, aimed at community college employees with the purpose to build data literacy and equity-mindedness. I used Bandura’s (1989) Social Cognitive Theory as a guiding theoretical framework paired with a quasi-experimental, delayed-start research design to study the effectiveness of the course in building employee data literacy and equity-mindedness, in addition to better understanding the impacts of environmental factors (i.e., organizational culture) on the implementation of the course. Using pre- and post-intervention surveys, pre- and post-intervention knowledge assessments, and post-intervention participant interviews, I determined that the professional development course contributed to improvements in employee data literacy and equity-mindedness. In particular, the course helped increase employee self-efficacy for data use, increased employee knowledge of data use and equity-mindedness, and increased employee intent to use data in the future. I also found that the organization’s culture related to data and equity to be complex and evolving, both hindering and facilitating data use, in general, and data use specifically, to address inequitable student outcomes.
ContributorsMitchell, Dennis Shane (Author) / Beardsley, Audrey (Thesis advisor) / Ott, Molly (Committee member) / Jacobsen, Craig (Committee member) / Arizona State University (Publisher)
Created2022
Description
Although an integral part of the pedagogical process is the evaluation of students, questions remain about the purpose and characteristics of effective assessments. Assessments should benefit both the instructor and the student, but this could be a challenge in large classes, such as the English service courses offered at the University of Guyana (UG), which are compulsory courses offered to over 2,000 first year students annually. However, the transition to online delivery of these courses because of the Coronavirus (COVID-19) pandemic has offered new opportunities for innovation in relation to course assessments. Consequently, this Action Research study was undertaken with the intention of improving the methods of assessment in the course, Introduction to the Use of English (ENG 1105), one of the three English service courses currently offered at UG.Multiple methods of data collection, including surveys, and semi-structured interviews, observations and analyses were used to determine how the assessment strategies used in the course helped develop academic self-efficacy in students and prepare them for other courses in their programs of study. The findings from the first two cycles of this study suggest that while the current assessment methods used in the course are beneficial to both lecturers and students, there is a need to adjust aspects of the assessments so students benefit from assessments that better align with other courses in their programs, as well as sharpen their English language skills. The third cycle captures the impact that the use of an innovation-an ungraded portfolio-had on student learning and suggests it should become a regular feature in the English service courses.
ContributorsMc Gowan, Mark Alastair (Author) / Thompson, Nicole L (Thesis advisor) / Fischman, Gustavo E (Committee member) / Wolf, Leigh G (Committee member) / Arizona State University (Publisher)
Created2023