Matching Items (13)
Filtering by
- Creators: Arizona State University
- Creators: Glenberg, Arthur
Description
Lots of previous studies have analyzed human tutoring at great depths and have shown expert human tutors to produce effect sizes, which is twice of that produced by an intelligent tutoring system (ITS). However, there has been no consensus on which factor makes them so effective. It is important to know this, so that same phenomena can be replicated in an ITS in order to achieve the same level of proficiency as expert human tutors. Also, to the best of my knowledge no one has looked at student reactions when they are working with a computer based tutor. The answers to both these questions are needed in order to build a highly effective computer-based tutor. My research focuses on the second question. In the first phase of my thesis, I analyzed the behavior of students when they were working with a step-based tutor Andes, using verbal-protocol analysis. The accomplishment of doing this was that I got to know of some ways in which students use a step-based tutor which can pave way for the creation of more effective computer-based tutors. I found from the first phase of the research that students often keep trying to fix errors by guessing repeatedly instead of asking for help by clicking the hint button. This phenomenon is known as hint refusal. Surprisingly, a large portion of the student's foundering was due to hint refusal. The hypothesis tested in the second phase of the research is that hint refusal can be significantly reduced and learning can be significantly increased if Andes uses more unsolicited hints and meta hints. An unsolicited hint is a hint that is given without the student asking for one. A meta-hint is like an unsolicited hint in that it is given without the student asking for it, but it just prompts the student to click on the hint button. Two versions of Andes were compared: the original version and a new version that gave more unsolicited and meta-hints. During a two-hour experiment, there were large, statistically reliable differences in several performance measures suggesting that the new policy was more effective.
ContributorsRanganathan, Rajagopalan (Author) / VanLehn, Kurt (Thesis advisor) / Atkinson, Robert (Committee member) / Burleson, Winslow (Committee member) / Arizona State University (Publisher)
Created2011
Description
This study empirically evaluated the effectiveness of the instructional design, learning tools, and role of the teacher in three versions of a semester-long, high-school remedial Algebra I course to determine what impact self-regulated learning skills and learning pattern training have on students' self-regulation, math achievement, and motivation. The 1st version was a business-as-usual traditional classroom teaching mathematics with direct instruction. The 2rd version of the course provided students with self-paced, individualized Algebra instruction with a web-based, intelligent tutor. The 3rd version of the course coupled self-paced, individualized instruction on the web-based, intelligent Algebra tutor coupled with a series of e-learning modules on self-regulated learning knowledge and skills that were distributed throughout the semester. A quasi-experimental, mixed methods evaluation design was used by assigning pre-registered, high-school remedial Algebra I class periods made up of an approximately equal number of students to one of the three study conditions or course versions: (a) the control course design, (b) web-based, intelligent tutor only course design, and (c) web-based, intelligent tutor + SRL e-learning modules course design. While no statistically significant differences on SRL skills, math achievement or motivation were found between the three conditions, effect-size estimates provide suggestive evidence that using the SRL e-learning modules based on ARCS motivation model (Keller, 2010) and Let Me Learn learning pattern instruction (Dawkins, Kottkamp, & Johnston, 2010) may help students regulate their learning and improve their study skills while using a web-based, intelligent Algebra tutor as evidenced by positive impacts on math achievement, motivation, and self-regulated learning skills. The study also explored predictive analyses using multiple regression and found that predictive models based on independent variables aligned to student demographics, learning mastery skills, and ARCS motivational factors are helpful in defining how to further refine course design and design learning evaluations that measure achievement, motivation, and self-regulated learning in web-based learning environments, including intelligent tutoring systems.
ContributorsBarrus, Angela (Author) / Atkinson, Robert K (Thesis advisor) / Van de Sande, Carla (Committee member) / Savenye, Wilhelmina (Committee member) / Arizona State University (Publisher)
Created2013
Description
The purposes of the study are to: 1) investigate how students' motivation towards reading is related to their reading comprehension skills, and 2) assess the impact of using an Intelligent Tutoring System to improve comprehension. Interactive Strategy Training for Active Reading and Thinking-3 (iSTART-3) is a game-based tutoring system designed to improve students' reading comprehension skills. The current study was conducted in South Africa with 8th and 9th graders between the ages of 14 and 18. These students are multilingual and they learn English as a First Additional Language (English-FAL). Firstly, we predict that students who are highly motivated to read will have high comprehension scores than those who are slightly or not at all motivated to read. Secondly, we predict that the use of iSTART-3 will improve students' reading comprehension, regardless of their level of reading motivation, with better results for those who are more motivated to read. Counter to our predictions, the results did not reveal a relation between reading motivation and reading comprehension. Furthermore, an effect of iSTART-3 on reading comprehension was not found. These results were likely influenced by the small sample size and the length of the intervention.
ContributorsSeerane, Thato (Author) / McNamara, Danielle (Thesis director) / Glenberg, Arthur (Committee member) / Lynch, Christa (Committee member) / Department of Psychology (Contributor) / School of Social and Behavioral Sciences (Contributor) / Barrett, The Honors College (Contributor)
Created2018-12
Description
The mathematics test is the most difficult test in the GED (General Education Development) Test battery, largely due to the presence of story problems. Raising performance levels of story problem-solving would have a significant effect on GED Test passage rates. The subject of this formative research study is Ms. Stephens’ Categorization Practice Utility (MS-CPU), an example-tracing intelligent tutoring system that serves as practice for the first step (problem categorization) in a larger comprehensive story problem-solving pedagogy that purports to raise the level of story problem-solving performance. During the analysis phase of this project, knowledge components and particular competencies that enable learning (schema building) were identified. During the development phase, a tutoring system was designed and implemented that algorithmically teaches these competencies to the student with graphical, interactive, and animated utilities. Because the tutoring system provides a much more concrete rather than conceptual, learning environment, it should foster a much greater apprehension of a story problem-solving process. With this experience, the student should begin to recognize the generalizability of concrete operations that accomplish particular story problem-solving goals and begin to build conceptual knowledge and a more conceptual approach to the task. During the formative evaluation phase, qualitative methods were used to identify obstacles in the MS-CPU user interface and disconnections in the pedagogy that impede learning story problem categorization and solution preparation. The study was conducted over two iterations where identification of obstacles and change plans (mitigations) produced a qualitative data table used to modify the first version systems (MS-CPU 1.1). Mitigation corrections produced the second version of the MS-CPU 1.2, and the next iteration of the study was conducted producing a second set of obstacle/mitigation tables. Pre-posttests were conducted in each iteration to provide corroboration for the effectiveness of the mitigations that were performed. The study resulted in the identification of a number of learning obstacles in the first version of the MS-CPU 1.1. Their mitigation produced a second version of the MS-CPU 1.2 whose identified obstacles were much less than the first version. It was determined that an additional iteration is needed before more quantitative research is conducted.
ContributorsRitchey, ChristiAnne (Author) / VanLehn, Kurt (Thesis advisor) / Savenye, Wilhelmina (Committee member) / Hong, Yi-Chun (Committee member) / Arizona State University (Publisher)
Created2018
Description
Feedback represents a vital component of the learning process and is especially important for Computer Science students. With class sizes that are often large, it can be challenging to provide individualized feedback to students. Consistent, constructive, supportive feedback through a tutoring companion can scaffold the learning process for students.
This work contributes to the construction of a tutoring companion designed to provide this feedback to students. It aims to bridge the gap between the messages the compiler delivers, and the support required for a novice student to understand the problem and fix their code. Particularly, it provides support for students learning about recursion in a beginning university Java programming course. Besides also providing affective support, a tutoring companion could be more effective when it is embedded into the environment that the student is already using, instead of an additional tool for the student to learn. The proposed Tutoring Companion is embedded into the Eclipse Integrated Development Environment (IDE).
This thesis focuses on the reasoning model for the Tutoring Companion and is developed using the techniques of a neural network. While a student uses the IDE, the Tutoring Companion collects 16 data points, including the presence of certain key words, cyclomatic complexity, and error messages from the compiler, every time it detects an event, such as a run attempt, debug attempt, or a request for help, in the IDE. This data is used as inputs to the neural network. The neural network produces a correlating single output code for the feedback to be provided to the student, which is displayed in the IDE.
The effectiveness of the approach is examined among 38 Computer Science students who solve a programming assignment while the Tutoring Companion assists them. Data is collected from these interactions, including all inputs and outputs for the neural network, and students are surveyed regarding their experience. Results suggest that students feel supported while working with the Companion and promising potential for using a neural network with an embedded companion in the future. Challenges in developing an embedded companion are discussed, as well as opportunities for future work.
This work contributes to the construction of a tutoring companion designed to provide this feedback to students. It aims to bridge the gap between the messages the compiler delivers, and the support required for a novice student to understand the problem and fix their code. Particularly, it provides support for students learning about recursion in a beginning university Java programming course. Besides also providing affective support, a tutoring companion could be more effective when it is embedded into the environment that the student is already using, instead of an additional tool for the student to learn. The proposed Tutoring Companion is embedded into the Eclipse Integrated Development Environment (IDE).
This thesis focuses on the reasoning model for the Tutoring Companion and is developed using the techniques of a neural network. While a student uses the IDE, the Tutoring Companion collects 16 data points, including the presence of certain key words, cyclomatic complexity, and error messages from the compiler, every time it detects an event, such as a run attempt, debug attempt, or a request for help, in the IDE. This data is used as inputs to the neural network. The neural network produces a correlating single output code for the feedback to be provided to the student, which is displayed in the IDE.
The effectiveness of the approach is examined among 38 Computer Science students who solve a programming assignment while the Tutoring Companion assists them. Data is collected from these interactions, including all inputs and outputs for the neural network, and students are surveyed regarding their experience. Results suggest that students feel supported while working with the Companion and promising potential for using a neural network with an embedded companion in the future. Challenges in developing an embedded companion are discussed, as well as opportunities for future work.
ContributorsDay, Melissa (Author) / Gonzalez-Sanchez, Javier (Thesis advisor) / Bansal, Ajay (Committee member) / Mehlhase, Alexandra (Committee member) / Arizona State University (Publisher)
Created2019
Description
Embedded assessment constantly updates a model of the student as the student works on instructional tasks. Accurate embedded assessment allows students, instructors and instructional systems to make informed decisions without requiring the student to stop instruction and take a test. This thesis describes the development and comparison of several student models for Dragoon, an intelligent tutoring system. All the models were instances of Bayesian Knowledge Tracing, a standard method. Several methods of parameterization and calibration were explored using two recently developed toolkits, FAST and BNT-SM that replaces constant-valued parameters with logistic regressions. The evaluation was done by calculating the fit of the models to data from human subjects and by assessing the accuracy of their assessment of simulated students. The student models created using node properties as subskills were superior to coarse-grained, skill-only models. Adding this extra level of representation to emission parameters was superior to adding it to transmission parameters. Adding difficulty parameters did not improve fit, contrary to standard practice in psychometrics.
ContributorsGrover, Sachin (Author) / VanLehn, Kurt (Thesis advisor) / Walker, Erin (Committee member) / Shiao, Ihan (Committee member) / Arizona State University (Publisher)
Created2015
Description
Science instructors need questions for use in exams, homework assignments, class discussions, reviews, and other instructional activities. Textbooks never have enough questions, so instructors must find them from other sources or generate their own questions. In order to supply instructors with biology questions, a semantic network approach was developed for generating open response biology questions. The generated questions were compared to professional authorized questions.
To boost students’ learning experience, adaptive selection was built on the generated questions. Bayesian Knowledge Tracing was used as embedded assessment of the student’s current competence so that a suitable question could be selected based on the student’s previous performance. A between-subjects experiment with 42 participants was performed, where half of the participants studied with adaptive selected questions and the rest studied with mal-adaptive order of questions. Both groups significantly improved their test scores, and the participants in adaptive group registered larger learning gains than participants in the control group.
To explore the possibility of generating rich instructional feedback for machine-generated questions, a question-paragraph mapping task was identified. Given a set of questions and a list of paragraphs for a textbook, the goal of the task was to map the related paragraphs to each question. An algorithm was developed whose performance was comparable to human annotators.
A multiple-choice question with high quality distractors (incorrect answers) can be pedagogically valuable as well as being much easier to grade than open-response questions. Thus, an algorithm was developed to generate good distractors for multiple-choice questions. The machine-generated multiple-choice questions were compared to human-generated questions in terms of three measures: question difficulty, question discrimination and distractor usefulness. By recruiting 200 participants from Amazon Mechanical Turk, it turned out that the two types of questions performed very closely on all the three measures.
To boost students’ learning experience, adaptive selection was built on the generated questions. Bayesian Knowledge Tracing was used as embedded assessment of the student’s current competence so that a suitable question could be selected based on the student’s previous performance. A between-subjects experiment with 42 participants was performed, where half of the participants studied with adaptive selected questions and the rest studied with mal-adaptive order of questions. Both groups significantly improved their test scores, and the participants in adaptive group registered larger learning gains than participants in the control group.
To explore the possibility of generating rich instructional feedback for machine-generated questions, a question-paragraph mapping task was identified. Given a set of questions and a list of paragraphs for a textbook, the goal of the task was to map the related paragraphs to each question. An algorithm was developed whose performance was comparable to human annotators.
A multiple-choice question with high quality distractors (incorrect answers) can be pedagogically valuable as well as being much easier to grade than open-response questions. Thus, an algorithm was developed to generate good distractors for multiple-choice questions. The machine-generated multiple-choice questions were compared to human-generated questions in terms of three measures: question difficulty, question discrimination and distractor usefulness. By recruiting 200 participants from Amazon Mechanical Turk, it turned out that the two types of questions performed very closely on all the three measures.
ContributorsZhang, Lishang (Author) / VanLehn, Kurt (Thesis advisor) / Baral, Chitta (Committee member) / Hsiao, Ihan (Committee member) / Wright, Christian (Committee member) / Arizona State University (Publisher)
Created2015
Description
EMBRACE (Enhanced Moved By Reading to Accelerate Comprehension in English) is an IPad application that uses the Moved By Reading strategy to help improve the reading comprehension skills of bilingual (Spanish speaking) English Language Learners (ELLs). In EMBRACE, students read the text of a story and then move images corresponding to the text that they read. According to the embodied cognition theory, this grounds reading comprehension in physical experiences and thus is more engaging.
In this thesis, I used the log data from 20 students in grades 2-5 to design a skill model for a student using EMBRACE. A skill model is the set of knowledge components that a student needs to master in order to comprehend the text in EMBRACE. A good skill model will improve understanding of the mistakes students make and thus aid in the design of useful feedback for the student.. In this context, the skill model consists of vocabulary and syntax associated with the steps that students performed. I mapped each step in EMBRACE to one or more skills (vocabulary and syntax) from the model. After every step, the skill level is updated in the model. Thus, if a student answered the previous step incorrectly, the corresponding skills are decremented and if the student answered the previous question correctly, the corresponding skills are incremented, through the Bayesian Knowledge Tracing algorithm.
I then correlated the students’ predicted scores (computed from their skill levels) to their posttest scores. I evaluated the students’ predicted scores (computed from their skill levels) by comparing them to their posttest scores. The two sets of scores were not highly correlated, but the results gave insights into potential improvements that could be made to the system with respect to user interaction, posttest scores and modeling algorithm.
In this thesis, I used the log data from 20 students in grades 2-5 to design a skill model for a student using EMBRACE. A skill model is the set of knowledge components that a student needs to master in order to comprehend the text in EMBRACE. A good skill model will improve understanding of the mistakes students make and thus aid in the design of useful feedback for the student.. In this context, the skill model consists of vocabulary and syntax associated with the steps that students performed. I mapped each step in EMBRACE to one or more skills (vocabulary and syntax) from the model. After every step, the skill level is updated in the model. Thus, if a student answered the previous step incorrectly, the corresponding skills are decremented and if the student answered the previous question correctly, the corresponding skills are incremented, through the Bayesian Knowledge Tracing algorithm.
I then correlated the students’ predicted scores (computed from their skill levels) to their posttest scores. I evaluated the students’ predicted scores (computed from their skill levels) by comparing them to their posttest scores. The two sets of scores were not highly correlated, but the results gave insights into potential improvements that could be made to the system with respect to user interaction, posttest scores and modeling algorithm.
ContributorsFurtado, Nicolette Dolores (Author) / Walker, Erin (Thesis advisor) / Hsiao, Ihan (Committee member) / Restrepo, M. Adelaida (Committee member) / Arizona State University (Publisher)
Created2016
Description
Students' ability to regulate and control their behaviors during learning has been shown to be a critical skill for academic success. However, researchers often struggle with ways to capture the nuances of this ability, often solely relying on self-report measures. This thesis proposal employs a novel approach to investigating variations in students' ability to self-regulate by using process data from the game-based Intelligent Tutoring System (ITS) iSTART-ME. This approach affords a nuanced examination of how students' regulate their interactions with game-based features at both a coarse-grained and fine-grain levels and the ultimate impact that those behaviors have on in-system performance and learning outcomes (i.e., self-explanation quality). This thesis is comprised of two submitted manuscripts that examined how a group of 40 high school students chose to engage with game-based features and how those interactions influenced their target skill performance. Findings suggest that in-system log data has the potential to provide stealth assessments of students' self-regulation while learning.
ContributorsSnow, Erica L (Author) / McNamara, Danielle S. (Thesis advisor) / Glenburg, Arthur M (Committee member) / Duran, Nicholas (Committee member) / Arizona State University (Publisher)
Created2014
Description
Many English Language Learner (ELL) children struggle with knowledge of vocabulary and syntax. Enhanced Moved by Reading to Accelerate Comprehension in English (EMBRACE) is an interactive storybook application that teaches children to read by moving pictures on the screen to act out the sentences in the text. However, EMBRACE presents the same level of text to all users, and it is limited in its ability to provide error feedback, as it can only determine whether a user action is right or wrong. EMBRACE could help readers learn more effectively if it personalized its instruction with texts that fit their current reading level and feedback that addresses ways to correct their mistakes. Improvements were made to the system by applying design principles of intelligent tutoring systems (ITSs). The new system added features to track the student’s reading comprehension skills, including vocabulary, syntax, and usability, based on various user actions, as well as features to adapt text complexity and provide more specific error feedback using the skills. A pilot study was conducted with 7 non-ELL students to evaluate the functionality and effectiveness of these features. The results revealed both strengths and weaknesses of the ITS. While skill updates appeared most accurate when users made particular kinds of vocabulary and syntax errors, it was not able to correctly identify other kinds of syntax errors or provide feedback when skill values became too high. Additionally, vocabulary error feedback and adapting the complexity of syntax were helpful, but syntax error feedback and adapting the complexity of vocabulary were not as helpful. Overall, children enjoy using EMBRACE, and building an intelligent tutoring system into the application presents a promising approach to make reading a both fun and effective experience.
ContributorsWong, Audrey (Author) / Walker, Erin (Thesis advisor) / Nelson, Brian (Committee member) / Glenberg, Arthur (Committee member) / Arizona State University (Publisher)
Created2017