Matching Items (9)
Filtering by
- All Subjects: Assessment
- All Subjects: Multivariate analysis
- Creators: Levy, Roy
- Creators: Gray, Shelley
- Creators: Redman, Aaron
Description
In the current context of fiscal austerity as well as neo-colonial criticisms, the discipline of religious studies has been challenged to critically assess its teaching methods as well as articulate its relevance in the modern university setting. Responding to these needs, this dissertation explores the educational outcomes on undergraduate students as a result of religious studies curriculum. This research employs a robust quantitative methodology designed to assess the impact of the courses while controlling for a number of covariates. Based on data collected from pre- and post-course surveys of a combined 1,116 students enrolled at Arizona State University (ASU) and two area community colleges, the research examines student change across five outcomes: attributional complexity, multi-religious awareness, commitment to social justice, individual religiosity, and the first to be developed, neo-colonial measures. The sample was taken in the Fall of 2009 from courses including Religions of the World, introductory Islamic studies courses, and a control group consisting of engineering and political science students. The findings were mixed. From the "virtues of the humanities" standpoint, select within group changes showed a statistically significant positive shift, but when compared across groups and the control group, there were no statistically significant findings after controlling for key variables. The students' pre-course survey score was the best predictor of their post-course survey score. In response to the neo-colonial critiques, the non-findings suggest the critiques have been overstated in terms of their impact pedagogically or in the classroom.
ContributorsLewis, Bret (Author) / Gereboff, Joel (Thesis advisor) / Foard, James (Committee member) / Levy, Roy (Committee member) / Woodward, Mark (Committee member) / Arizona State University (Publisher)
Created2011
Description
The purpose of this study was to investigate the effect of complex structure on dimensionality assessment in compensatory and noncompensatory multidimensional item response models (MIRT) of assessment data using dimensionality assessment procedures based on conditional covariances (i.e., DETECT) and a factor analytical approach (i.e., NOHARM). The DETECT-based methods typically outperformed the NOHARM-based methods in both two- (2D) and three-dimensional (3D) compensatory MIRT conditions. The DETECT-based methods yielded high proportion correct, especially when correlations were .60 or smaller, data exhibited 30% or less complexity, and larger sample size. As the complexity increased and the sample size decreased, the performance typically diminished. As the complexity increased, it also became more difficult to label the resulting sets of items from DETECT in terms of the dimensions. DETECT was consistent in classification of simple items, but less consistent in classification of complex items. Out of the three NOHARM-based methods, χ2G/D and ALR generally outperformed RMSR. χ2G/D was more accurate when N = 500 and complexity levels were 30% or lower. As the number of items increased, ALR performance improved at correlation of .60 and 30% or less complexity. When the data followed a noncompensatory MIRT model, the NOHARM-based methods, specifically χ2G/D and ALR, were the most accurate of all five methods. The marginal proportions for labeling sets of items as dimension-like were typically low, suggesting that the methods generally failed to label two (three) sets of items as dimension-like in 2D (3D) noncompensatory situations. The DETECT-based methods were more consistent in classifying simple items across complexity levels, sample sizes, and correlations. However, as complexity and correlation levels increased the classification rates for all methods decreased. In most conditions, the DETECT-based methods classified complex items equally or more consistent than the NOHARM-based methods. In particular, as complexity, the number of items, and the true dimensionality increased, the DETECT-based methods were notably more consistent than any NOHARM-based method. Despite DETECT's consistency, when data follow a noncompensatory MIRT model, the NOHARM-based method should be preferred over the DETECT-based methods to assess dimensionality due to poor performance of DETECT in identifying the true dimensionality.
ContributorsSvetina, Dubravka (Author) / Levy, Roy (Thesis advisor) / Gorin, Joanna S. (Committee member) / Millsap, Roger (Committee member) / Arizona State University (Publisher)
Created2011
Description
The Culture-Language Interpretive Matrix (C-LIM) is a new tool hypothesized to help practitioners accurately determine whether students who are administered an IQ test are culturally and linguistically different from the normative comparison group (i.e., different) or culturally and linguistically similar to the normative comparison group and possibly have Specific Learning Disabilities (SLD) or other neurocognitive disabilities (i.e., disordered). Diagnostic utility statistics were used to test the ability of the Wechsler Intelligence Scales for Children-Fourth Edition (WISC-IV) C-LIM to accurately identify students from a referred sample of English language learners (Ells) (n = 86) for whom Spanish was the primary language spoken at home and a sample of students from the WISC-IV normative sample (n = 2,033) as either culturally and linguistically different from the WISC-IV normative sample or culturally and linguistically similar to the WISC-IV normative sample. WISC-IV scores from three paired comparison groups were analyzed using the Receiver Operating Characteristic (ROC) curve: (a) Ells with SLD and the WISC-IV normative sample, (b) Ells without SLD and the WISC-IV normative sample, and (c) Ells with SLD and Ells without SLD. Results of the ROC yielded Area Under the Curve (AUC) values that ranged between 0.51 and 0.53 for the comparison between Ells with SLD and the WISC-IV normative sample, AUC values that ranged between 0.48 and 0.53 for the comparison between Ells without SLD and the WISC-IV normative sample, and AUC values that ranged between 0.49 and 0.55 for the comparison between Ells with SLD and Ells without SLD. These values indicate that the C-LIM has low diagnostic accuracy in terms of differentiating between a sample of Ells and the WISC-IV normative sample. Current available evidence does not support use of the C-LIM in applied practice at this time.
ContributorsStyck, Kara M (Author) / Watkins, Marley W. (Thesis advisor) / Levy, Roy (Thesis advisor) / Balles, John (Committee member) / Arizona State University (Publisher)
Created2012
Description
Identification of primary language impairment (PLI) in sequential bilingual children is challenging because of the interaction between PLI and second language (L2) proficiency. An important step in improving the accurate diagnosis of PLI in bilingual children is to investigate how differences in L2 performance are affected by a length of L2 exposure and how L2 assessment contributes to differentiation between children with and without PLI at different L2 proficiency levels. Sixty one children with typical language development (TD) ages 5;3-8 years and 12 children with PLI ages 5;5-7;8 years participated. Results revealed that bilingual children with and without PLI, who had between 1 and 3 years of L2 exposure, did not differ in mean length of utterance (MLU), number of different words, percent of maze words, and performance on expressive and receptive grammatical tasks in L2. Performance on a grammaticality judgment task by children with and without PLI demonstrated the largest effect size, indicating that it may potentially contribute to identification of PLI in bilingual populations. In addition, children with PLI did not demonstrate any association between the length of exposure and L2 proficiency, suggesting that they do not develop their L2 proficiency in relation to length of exposure in the same manner as children with TD. Results also indicated that comprehension of grammatical structures and expressive grammatical task in L2 may contribute to differentiation between the language ability groups at the low and intermediate-high proficiency levels. The discriminant analysis with the entire sample of bilingual children with and without PLI revealed that among L2 measures, only MLU contributed to the discrimination between the language ability groups. However, poor classification accuracy suggested that MLU alone is not a sufficient predictor of PLI. There were significant differences among L2 proficiency levels in children with TD in MLU, number of different words, and performance on the expressive and receptive grammatical tasks in L2, indicating that L2 proficiency level may potentially impact the differentiation between language difficulties due to typical L2 acquisition processes and PLI.
ContributorsSmyk, Ekaterina (Author) / Restrepo, Maria Adelaida (Thesis advisor) / Gorin, Joanna (Committee member) / Gray, Shelley (Committee member) / Arizona State University (Publisher)
Created2012
Description
Currently, there are few quality tools available to screen for developmental language disorder (DLD) in Spanish-speaking children despite the continued increase of this population in the United States. The lack of valid and reliable screening tools may be a factor leading to difficulties with the identification of and delivery of services to these children. This study plans to improve upon the screening of Spanish-English bilingual children.The Spanish Screener for Language Impairment in Children (SSLIC) tests Spanish oral language skills in Spanish-speaking children. It measures language skills through morphology elicitation of Spanish clitics, prepositions, derivational morphemes, subjunctive verb tenses, and articles and repetition of nonwords and sentences, which have all been shown to be affected in Spanish-speaking children with DLD.
The purpose of the study is to provide preliminary validity evidence of the SSLIC. Children's results on the SSLIC were compared to other validated measures. Fourteen Spanish-English bilingual students were recruited: 11 children with typical language development (TD) and 3 with DLD. The Bilingual English-Spanish Assessment and the Dynamic Measure of Oral Narrative Discourse were used to establish preliminary validity evidence. Pearson correlations were run to determine if SSLIC scores correlated with other validated measures. Significant correlations were found between the SSLIC’s scores and scores on the BESA. One-way analysis of variance (ANOVA) was used to determine mean differences between groups. No significant mean differences for SSLIC scores were found between children with typical and atypical language. Yet, effect sizes suggested group differences. Point to point analysis revealed that the SSLIC has excellent inter-rater reliability.
Despite a small sample size, this study serves as preliminary evidence that the SSLIC is both valid and reliable and supports that the SSLIC has the potential to be used as a screening tool for DLD for Spanish-speaking kindergarten and 1st grade students with further validation, which should continue.
ContributorsSmith, Brandon Earl (Author) / Restrepo, María A (Thesis advisor) / Gray, Shelley (Committee member) / Brown, Jean C (Committee member) / Moen, Theresa C (Committee member) / Arizona State University (Publisher)
Created2023
Description
Investigation of measurement invariance (MI) commonly assumes correct specification of dimensionality across multiple groups. Although research shows that violation of the dimensionality assumption can cause bias in model parameter estimation for single-group analyses, little research on this issue has been conducted for multiple-group analyses. This study explored the effects of mismatch in dimensionality between data and analysis models with multiple-group analyses at the population and sample levels. Datasets were generated using a bifactor model with different factor structures and were analyzed with bifactor and single-factor models to assess misspecification effects on assessments of MI and latent mean differences. As baseline models, the bifactor models fit data well and had minimal bias in latent mean estimation. However, the low convergence rates of fitting bifactor models to data with complex structures and small sample sizes caused concern. On the other hand, effects of fitting the misspecified single-factor models on the assessments of MI and latent means differed by the bifactor structures underlying data. For data following one general factor and one group factor affecting a small set of indicators, the effects of ignoring the group factor in analysis models on the tests of MI and latent mean differences were mild. In contrast, for data following one general factor and several group factors, oversimplifications of analysis models can lead to inaccurate conclusions regarding MI assessment and latent mean estimation.
ContributorsXu, Yuning (Author) / Green, Samuel (Thesis advisor) / Levy, Roy (Committee member) / Thompson, Marilyn (Committee member) / Arizona State University (Publisher)
Created2018
Description
A simulation study was conducted to explore the robustness of general factor mean difference estimation in bifactor ordered-categorical data. In the No Differential Item Functioning (DIF) conditions, the data generation conditions varied were sample size, the number of categories per item, effect size of the general factor mean difference, and the size of specific factor loadings; in data analysis, misspecification conditions were introduced in which the generated bifactor data were fit using a unidimensional model, and/or ordered-categorical data were treated as continuous data. In the DIF conditions, the data generation conditions varied were sample size, the number of categories per item, effect size of latent mean difference for the general factor, the type of item parameters that had DIF, and the magnitude of DIF; the data analysis conditions varied in whether or not setting equality constraints on the noninvariant item parameters.
Results showed that falsely fitting bifactor data using unidimensional models or failing to account for DIF in item parameters resulted in estimation bias in the general factor mean difference, while treating ordinal data as continuous had little influence on the estimation bias as long as there was no severe model misspecification. The extent of estimation bias produced by misspecification of bifactor datasets with unidimensional models was mainly determined by the degree of unidimensionality (i.e., size of specific factor loadings) and the general factor mean difference size. When the DIF was present, the estimation accuracy of the general factor mean difference was completely robust to ignoring noninvariance in specific factor loadings while it was very sensitive to failing to account for DIF in threshold parameters. With respect to ignoring the DIF in general factor loadings, the estimation bias of the general factor mean difference was substantial when the DIF was -0.15, and it can be negligible for smaller sizes of DIF. Despite the impact of model misspecification on estimation accuracy, the power to detect the general factor mean difference was mainly influenced by the sample size and effect size. Serious Type I error rate inflation only occurred when the DIF was present in threshold parameters.
Results showed that falsely fitting bifactor data using unidimensional models or failing to account for DIF in item parameters resulted in estimation bias in the general factor mean difference, while treating ordinal data as continuous had little influence on the estimation bias as long as there was no severe model misspecification. The extent of estimation bias produced by misspecification of bifactor datasets with unidimensional models was mainly determined by the degree of unidimensionality (i.e., size of specific factor loadings) and the general factor mean difference size. When the DIF was present, the estimation accuracy of the general factor mean difference was completely robust to ignoring noninvariance in specific factor loadings while it was very sensitive to failing to account for DIF in threshold parameters. With respect to ignoring the DIF in general factor loadings, the estimation bias of the general factor mean difference was substantial when the DIF was -0.15, and it can be negligible for smaller sizes of DIF. Despite the impact of model misspecification on estimation accuracy, the power to detect the general factor mean difference was mainly influenced by the sample size and effect size. Serious Type I error rate inflation only occurred when the DIF was present in threshold parameters.
ContributorsLiu, Yixing (Author) / Thompson, Marilyn (Thesis advisor) / Levy, Roy (Committee member) / O’Rourke, Holly (Committee member) / Arizona State University (Publisher)
Created2019
Description
Making significant progress on the U.N. Sustainable Development Goals (SDGs) needs change agents equipped with key competencies in sustainability. While thousands of sustainability programs have emerged at various educational levels over the past decade, there is, as of yet, no reliable way to assess if these programs successfully convey key competencies in sustainability. This dissertation contributes to addressing this gap in three ways. First, it reviews the body of work on key competencies in sustainability. Based on broad agreement around five key competencies as well as an emerging set of three, an extended framework is outlined that can be used as unified set of learning objectives across sustainability programs. The next chapter reviews the scholarly work on assessing sustainability competencies. Based on this review, a typology of assessment tools is proposed offering guidance to both educators and researchers. Finally, drawing on experience of the four-year “Educating Future Change Agents” project, the last chapter explores the results from a diverse set of competency assessments in numerous courses. The study appraises assessment practices and results to demonstrate opportunities and challenges in the current state of assessing key competencies in sustainability. The results of this doctoral thesis are expected to make a practical and scholarly contribution to the teaching and learning in sustainability programs, in particular with regards to reliably assessing key competencies in sustainability.
ContributorsRedman, Aaron (Author) / Wiek, Arnim (Thesis advisor) / Barth, Matthias (Committee member) / Basile, George (Committee member) / Fischer, Daniel (Committee member) / Mochizuki, Yoko (Committee member) / Arizona State University (Publisher)
Created2020
Description
Businesses, as with other sectors in society, are not yet taking sufficient action towards achieving sustainability. The United Nations recently agreed upon a set of Sustainable Development Goals (SDGs), which if properly harnessed, provide a framework (so far lacking) for businesses to meaningfully drive transformations to sustainability. This paper proposes to operationalize the SDGs for businesses through a progressive framework for action with three discrete levels: communication, tactical, and strategic. Within the tactical and strategic levels, several innovative approaches are discussed and illustrated. The challenges of design and measurement as well as opportunities for accountability and the social side of Sustainability, together call for transdisciplinary, collective action. This paper demonstrates feasible pathways and approaches for businesses to take corporate social responsibility to the next level and utilize the SDG framework informed by sustainability science to support transformations towards the achievement of sustainability.
ContributorsRedman, Aaron (Author)
Created2018-06-30