Matching Items (2)
Filtering by
- All Subjects: Educational tests and measurements--Study and teaching (Higher)
- All Subjects: refugee
Norming at scale: faculty perceptions of assessment culture and student learning outcomes assessment
Description
To foster both external and internal accountability, universities seek more effective models for student learning outcomes assessment (SLOA). Meaningful and authentic measurement of program-level student learning outcomes requires engagement with an institution’s faculty members, especially to gather student performance assessment data using common scoring instruments, or rubrics, across a university’s many colleges and programs. Too often, however, institutions rely on faculty engagement for SLOA initiatives like this without providing necessary support, communication, and training. The resulting data may lack sufficient reliability and reflect deficiencies in an institution’s culture of assessment.
This mixed methods action research study gauged how well one form of SLOA training – a rubric-norming workshop – could affect both inter-rater reliability for faculty scorers and faculty perceptions of SLOA while exploring the nature of faculty collaboration toward a shared understanding of student learning outcomes. The study participants, ten part-time faculty members at the institution, each held primary careers in the health care industry, apart from their secondary role teaching university courses. Accordingly, each contributed expertise and experience to the rubric-norming discussions, surveys of assessment-related perceptions, and individual scoring of student performance with a common rubric. Drawing on sociocultural learning principles and the specific lens of activity theory, influences on faculty SLOA were arranged and analyzed within the heuristic framework of an activity system to discern effects of collaboration and perceptions toward SLOA on consistent rubric-scoring by faculty participants.
Findings suggest participation in the study did not correlate to increased inter-rater reliability for faculty scorers when using the common rubric. Constraints found within assessment tools and unclear institutional leadership prevented more reliable use of common rubrics. Instead, faculty participants resorted to individual assessment approaches to meaningfully guide students to classroom achievement and preparation for careers in the health care field. Despite this, faculty participants valued SLOA, collaborated readily with colleagues for shared assessment goals, and worked hard to teach and assess students meaningfully.
This mixed methods action research study gauged how well one form of SLOA training – a rubric-norming workshop – could affect both inter-rater reliability for faculty scorers and faculty perceptions of SLOA while exploring the nature of faculty collaboration toward a shared understanding of student learning outcomes. The study participants, ten part-time faculty members at the institution, each held primary careers in the health care industry, apart from their secondary role teaching university courses. Accordingly, each contributed expertise and experience to the rubric-norming discussions, surveys of assessment-related perceptions, and individual scoring of student performance with a common rubric. Drawing on sociocultural learning principles and the specific lens of activity theory, influences on faculty SLOA were arranged and analyzed within the heuristic framework of an activity system to discern effects of collaboration and perceptions toward SLOA on consistent rubric-scoring by faculty participants.
Findings suggest participation in the study did not correlate to increased inter-rater reliability for faculty scorers when using the common rubric. Constraints found within assessment tools and unclear institutional leadership prevented more reliable use of common rubrics. Instead, faculty participants resorted to individual assessment approaches to meaningfully guide students to classroom achievement and preparation for careers in the health care field. Despite this, faculty participants valued SLOA, collaborated readily with colleagues for shared assessment goals, and worked hard to teach and assess students meaningfully.
ContributorsWilliams, Nicholas (Author) / Liou, Daniel D (Thesis advisor) / Rotheram-Fuller, Erin (Committee member) / Turbow, David (Committee member) / Arizona State University (Publisher)
Created2018
Description
In this three-article dissertation study, I examine the educational experiences of students from refugee backgrounds (SRBs) and the teachers that serve them in an urban high school in Arizona. Through a year-long ethnographic study, I attempt to contribute to the existing literature by exploring three salient issues that mediate experiences for participating teachers (n = 3) and SRBs (n = 32) in three classrooms. The participating SRBs came from a wide variety of home countries and spoke a combined 15 different home languages. In the United States, where the instruction of SRBs is generally framed by language policies, English as a second language (ESL) teachers play an crucial role in SRBs’ schooling. In the first article, I examined how teachers’ language ideologies shaped their implementation of structured English immersion (SEI), the authorized language policy in the state of Arizona. Findings describe how the teachers enacted agency to appropriate authorized language policy and create new, unauthorized policies that met the perceived needs of SRBs in their classrooms. I also examined the identity construction of SRBs in figured worlds of resettlement. Once resettled, SRBs are legally tied to their status as refugees, which may operate as a mechanism of oppression in the host country. These individuals are often stripped of all identities but one—that of being a refugee—which essentializes their vulnerability and perpetuates deficit-oriented perspectives that may limit learning opportunities for SRBs. Findings describe how participating teachers constructed SRBs’ identities and how SRBs constructed refugee-ness for themselves, highlighting the strength and resiliency of this student population. Finally, I used phenomenology as a methodological frame from which to interpret SRBs’ experiences with SEI and the policy-related barriers they described as negatively impacting their education. As refugees flee their countries of origin, educational systems in their countries of resettlement have struggled to provide quality education to their children. Themes summarizing participants’ collective experiences highlighted specific challenges related to SEI policy mandates, including SRBs’ isolation, limited opportunities to interact with English-speaking peers, and low graduation rates.
ContributorsAmbroso, Eric Patrick (Author) / Anderson, Katherine T (Thesis advisor) / Fischman, Gustavo (Committee member) / Liou, Daniel D (Committee member) / Warriner, Doris (Committee member) / Arizona State University (Publisher)
Created2021