Matching Items (4)
Filtering by
- All Subjects: Higher Education Administration
- Creators: Johnson, Melissa
- Creators: Rotheram-Fuller, Erin
Norming at scale: faculty perceptions of assessment culture and student learning outcomes assessment
Description
To foster both external and internal accountability, universities seek more effective models for student learning outcomes assessment (SLOA). Meaningful and authentic measurement of program-level student learning outcomes requires engagement with an institution’s faculty members, especially to gather student performance assessment data using common scoring instruments, or rubrics, across a university’s many colleges and programs. Too often, however, institutions rely on faculty engagement for SLOA initiatives like this without providing necessary support, communication, and training. The resulting data may lack sufficient reliability and reflect deficiencies in an institution’s culture of assessment.
This mixed methods action research study gauged how well one form of SLOA training – a rubric-norming workshop – could affect both inter-rater reliability for faculty scorers and faculty perceptions of SLOA while exploring the nature of faculty collaboration toward a shared understanding of student learning outcomes. The study participants, ten part-time faculty members at the institution, each held primary careers in the health care industry, apart from their secondary role teaching university courses. Accordingly, each contributed expertise and experience to the rubric-norming discussions, surveys of assessment-related perceptions, and individual scoring of student performance with a common rubric. Drawing on sociocultural learning principles and the specific lens of activity theory, influences on faculty SLOA were arranged and analyzed within the heuristic framework of an activity system to discern effects of collaboration and perceptions toward SLOA on consistent rubric-scoring by faculty participants.
Findings suggest participation in the study did not correlate to increased inter-rater reliability for faculty scorers when using the common rubric. Constraints found within assessment tools and unclear institutional leadership prevented more reliable use of common rubrics. Instead, faculty participants resorted to individual assessment approaches to meaningfully guide students to classroom achievement and preparation for careers in the health care field. Despite this, faculty participants valued SLOA, collaborated readily with colleagues for shared assessment goals, and worked hard to teach and assess students meaningfully.
This mixed methods action research study gauged how well one form of SLOA training – a rubric-norming workshop – could affect both inter-rater reliability for faculty scorers and faculty perceptions of SLOA while exploring the nature of faculty collaboration toward a shared understanding of student learning outcomes. The study participants, ten part-time faculty members at the institution, each held primary careers in the health care industry, apart from their secondary role teaching university courses. Accordingly, each contributed expertise and experience to the rubric-norming discussions, surveys of assessment-related perceptions, and individual scoring of student performance with a common rubric. Drawing on sociocultural learning principles and the specific lens of activity theory, influences on faculty SLOA were arranged and analyzed within the heuristic framework of an activity system to discern effects of collaboration and perceptions toward SLOA on consistent rubric-scoring by faculty participants.
Findings suggest participation in the study did not correlate to increased inter-rater reliability for faculty scorers when using the common rubric. Constraints found within assessment tools and unclear institutional leadership prevented more reliable use of common rubrics. Instead, faculty participants resorted to individual assessment approaches to meaningfully guide students to classroom achievement and preparation for careers in the health care field. Despite this, faculty participants valued SLOA, collaborated readily with colleagues for shared assessment goals, and worked hard to teach and assess students meaningfully.
ContributorsWilliams, Nicholas (Author) / Liou, Daniel D (Thesis advisor) / Rotheram-Fuller, Erin (Committee member) / Turbow, David (Committee member) / Arizona State University (Publisher)
Created2018
Description
Honors colleges have offered an academically rigorous option for growing numbers of diverse students. This study took place at a large, public university that required undergraduate students to complete a thesis to graduate from the honors college. In 2017, 97% of students who began the honors thesis prior to senior year completed it. Thus, the aim of this study was to help more students begin the honors thesis process early.
Thesis Launch was a six-week intervention that was designed to provide support for students in the critical early steps of thesis work such as brainstorming topics, examining professors’ research interests, reaching out to professors, preparing for meetings with potential thesis committee members, and writing a thesis prospectus. Thesis Launch offered web-based resources, weekly emails and text message reminders, and was supplemented by in-person advising options.
A mixed methods action research study was conducted to examine: (a) students’ perceptions of barriers that prevented beginning thesis work; (b) self-efficacy towards thesis work; (c) how to scale the intervention using technology; and (d) whether participants began the thesis early. Quantitative data was collected via pre- and post-intervention surveys, journals, and prospectus submissions. Qualitative data came from student interviews, journals, and open-ended questions on the surveys.
Quantitative data showed that after students participated in Thesis Launch, they had higher self-efficacy to work with professors, perceived fewer barriers to thesis work, and greater proportions of students began thesis work early. The qualitative data were complementary and showed that participants overcame barriers to thesis initiation, built self-efficacy, preferred an online intervention, and began thesis work early. Findings also showed that a primarily technology-based intervention was preferred by students and showed promise for scaling to a larger audience.
Thesis Launch provided a framework for students to begin work on the honors thesis and have mastery experiences to build self-efficacy. Strategies that fostered “small wins” and reflective efforts also assisted in this aim. Participants accomplished tasks tied to thesis work and customized their personal thesis timelines based on work begun during Thesis Launch. Finally, a discussion of limitations, implications for practice and research, and personal reflection was included.
Thesis Launch was a six-week intervention that was designed to provide support for students in the critical early steps of thesis work such as brainstorming topics, examining professors’ research interests, reaching out to professors, preparing for meetings with potential thesis committee members, and writing a thesis prospectus. Thesis Launch offered web-based resources, weekly emails and text message reminders, and was supplemented by in-person advising options.
A mixed methods action research study was conducted to examine: (a) students’ perceptions of barriers that prevented beginning thesis work; (b) self-efficacy towards thesis work; (c) how to scale the intervention using technology; and (d) whether participants began the thesis early. Quantitative data was collected via pre- and post-intervention surveys, journals, and prospectus submissions. Qualitative data came from student interviews, journals, and open-ended questions on the surveys.
Quantitative data showed that after students participated in Thesis Launch, they had higher self-efficacy to work with professors, perceived fewer barriers to thesis work, and greater proportions of students began thesis work early. The qualitative data were complementary and showed that participants overcame barriers to thesis initiation, built self-efficacy, preferred an online intervention, and began thesis work early. Findings also showed that a primarily technology-based intervention was preferred by students and showed promise for scaling to a larger audience.
Thesis Launch provided a framework for students to begin work on the honors thesis and have mastery experiences to build self-efficacy. Strategies that fostered “small wins” and reflective efforts also assisted in this aim. Participants accomplished tasks tied to thesis work and customized their personal thesis timelines based on work begun during Thesis Launch. Finally, a discussion of limitations, implications for practice and research, and personal reflection was included.
ContributorsEardley, Trisha Lynn (Author) / Buss, Ray (Thesis advisor) / Henderson, Mark (Committee member) / Johnson, Melissa (Committee member) / Arizona State University (Publisher)
Created2019
Description
College completion has become a national priority in the United States. Before students can graduate from a college or university, however, they must survive their first year in higher education. The retention of out-of-state freshmen is a major piece of the larger college student retention puzzle due to recent national enrollment trends and the financial implications of out-of-state student enrollment. With public universities nationwide receiving less financial support from state governments, many of these institutions have used a strategy of aggressively recruiting and increasingly enrolling out-of-state students because the higher tuition these students pay can help offset the loss of state funding. Despite the importance of out-of-state students to the national higher education landscape, little research has been conducted on out-of-state student retention.
This study examined the relation between a resource website and the engagement, sense of belonging, homesickness, and retention of out-of-state freshmen at Arizona State University (ASU). Mixed methods of inquiry were utilized; data sources included a pre- and post-intervention student survey, student interviews, student essay artifacts, website utilization records, and university retention reports.
This study demonstrated that freshmen coming to ASU from another state experienced four main challenges related to being an out-of-state student. Those challenges were homesickness, adjusting to living in Arizona, managing finances, and making friends at ASU. Out-of-state students therefore needed extra support for their transition. The study found that an out-of-state student resource website had a positive association with co-curricular engagement and homesickness frequency reduction. Moreover, the site provided useful information on the challenges experienced by out-of-state freshmen. Discussion includes possible explanations for the findings and implications for practice and research.
This study examined the relation between a resource website and the engagement, sense of belonging, homesickness, and retention of out-of-state freshmen at Arizona State University (ASU). Mixed methods of inquiry were utilized; data sources included a pre- and post-intervention student survey, student interviews, student essay artifacts, website utilization records, and university retention reports.
This study demonstrated that freshmen coming to ASU from another state experienced four main challenges related to being an out-of-state student. Those challenges were homesickness, adjusting to living in Arizona, managing finances, and making friends at ASU. Out-of-state students therefore needed extra support for their transition. The study found that an out-of-state student resource website had a positive association with co-curricular engagement and homesickness frequency reduction. Moreover, the site provided useful information on the challenges experienced by out-of-state freshmen. Discussion includes possible explanations for the findings and implications for practice and research.
ContributorsCorrea, Kevin (Author) / Rotheram-Fuller, Erin (Thesis advisor) / Ott, Molly (Committee member) / Blakemore, Arthur (Committee member) / Arizona State University (Publisher)
Created2017
Description
Most redesigns involve exhaustive plans, dense reports, and thorough documentation of site requirements. When the website is perfected, this brand-new site is rolled out to users. This is often a lengthy process. The principals of Agile development champion interactions over processes, working software over comprehensive documentation and responding to change over following a plan. These methods encourage small, iterative changes on a rapid development and delivery cycle. This allows developers to respond to needs more quickly and decreases the cognitive load on users while allowing them to more easily adapt to change. Get tips for making a case for library websites to adopt Agile practices —small improvements through iterative changes allow users to adapt and increase user satisfaction.
ContributorsAllgood, Tammy (Author) / Johnson, Melissa (Author)
Created2014-10-27