Matching Items (3)
Filtering by
- All Subjects: psychometrics
- Creators: Green, Samuel
Description
The purpose of this study was to investigate the effect of complex structure on dimensionality assessment in compensatory and noncompensatory multidimensional item response models (MIRT) of assessment data using dimensionality assessment procedures based on conditional covariances (i.e., DETECT) and a factor analytical approach (i.e., NOHARM). The DETECT-based methods typically outperformed the NOHARM-based methods in both two- (2D) and three-dimensional (3D) compensatory MIRT conditions. The DETECT-based methods yielded high proportion correct, especially when correlations were .60 or smaller, data exhibited 30% or less complexity, and larger sample size. As the complexity increased and the sample size decreased, the performance typically diminished. As the complexity increased, it also became more difficult to label the resulting sets of items from DETECT in terms of the dimensions. DETECT was consistent in classification of simple items, but less consistent in classification of complex items. Out of the three NOHARM-based methods, χ2G/D and ALR generally outperformed RMSR. χ2G/D was more accurate when N = 500 and complexity levels were 30% or lower. As the number of items increased, ALR performance improved at correlation of .60 and 30% or less complexity. When the data followed a noncompensatory MIRT model, the NOHARM-based methods, specifically χ2G/D and ALR, were the most accurate of all five methods. The marginal proportions for labeling sets of items as dimension-like were typically low, suggesting that the methods generally failed to label two (three) sets of items as dimension-like in 2D (3D) noncompensatory situations. The DETECT-based methods were more consistent in classifying simple items across complexity levels, sample sizes, and correlations. However, as complexity and correlation levels increased the classification rates for all methods decreased. In most conditions, the DETECT-based methods classified complex items equally or more consistent than the NOHARM-based methods. In particular, as complexity, the number of items, and the true dimensionality increased, the DETECT-based methods were notably more consistent than any NOHARM-based method. Despite DETECT's consistency, when data follow a noncompensatory MIRT model, the NOHARM-based method should be preferred over the DETECT-based methods to assess dimensionality due to poor performance of DETECT in identifying the true dimensionality.
ContributorsSvetina, Dubravka (Author) / Levy, Roy (Thesis advisor) / Gorin, Joanna S. (Committee member) / Millsap, Roger (Committee member) / Arizona State University (Publisher)
Created2011
Description
The use of exams for classification purposes has become prevalent across many fields including professional assessment for employment screening and standards based testing in educational settings. Classification exams assign individuals to performance groups based on the comparison of their observed test scores to a pre-selected criterion (e.g. masters vs. nonmasters in dichotomous classification scenarios). The successful use of exams for classification purposes assumes at least minimal levels of accuracy of these classifications. Classification accuracy is an index that reflects the rate of correct classification of individuals into the same category which contains their true ability score. Traditional methods estimate classification accuracy via methods which assume that true scores follow a four-parameter beta-binomial distribution. Recent research suggests that Item Response Theory may be a preferable alternative framework for estimating examinees' true scores and may return more accurate classifications based on these scores. Researchers hypothesized that test length, the location of the cut score, the distribution of items, and the distribution of examinee ability would impact the recovery of accurate estimates of classification accuracy. The current simulation study manipulated these factors to assess their potential influence on classification accuracy. Observed classification as masters vs. nonmasters, true classification accuracy, estimated classification accuracy, BIAS, and RMSE were analyzed. In addition, Analysis of Variance tests were conducted to determine whether an interrelationship existed between levels of the four manipulated factors. Results showed small values of estimated classification accuracy and increased BIAS in accuracy estimates with few items, mismatched distributions of item difficulty and examinee ability, and extreme cut scores. A significant four-way interaction between manipulated variables was observed. In additional to interpretations of these findings and explanation of potential causes for the recovered values, recommendations that inform practice and avenues of future research are provided.
ContributorsKunze, Katie (Author) / Gorin, Joanna (Thesis advisor) / Levy, Roy (Thesis advisor) / Green, Samuel (Committee member) / Arizona State University (Publisher)
Created2013
Modeling motivation: examining the structural validity of the Sport Motivation Scale-6 among runners
Description
Two models of motivation are prevalent in the literature on sport and exercise participation (Deci & Ryan, 1991; Vallerand, 1997, 2000). Both models are grounded in self-determination theory (Deci & Ryan, 1985; Ryan & Deci, 2000) and consider the relationship between intrinsic, extrinsic, and amotivation in explaining behavior choice and outcomes. Both models articulate the relationship between need satisfaction (i.e., autonomy, competence, relatedness; Deci & Ryan, 1985, 2000; Ryan & Deci, 2000) and various cognitive, affective, and behavioral outcomes as a function of self-determined motivation. Despite these comprehensive models, inconsistencies remain between the theories and their practical applications. The purpose of my study was to examine alternative theoretical models of intrinsic, extrinsic, and amotivation using the Sport Motivation Scale-6 (SMS-6; Mallett et al., 2007) to more thoroughly study the structure of motivation and the practical utility of using such a scale to measure motivation among runners. Confirmatory factor analysis was used to evaluate eight alternative models. After finding unsatisfactory fit of these models, exploratory factor analysis was conducted post hoc to further examine the measurement structure of motivation. A three-factor structure of general motivation, external accolades, and isolation/solitude explained motivation best, although high cross-loadings of items suggest the structure of this construct still lacks clarity. Future directions to modify item content and re-examine structure as well as limitations of this study are discussed.
ContributorsKube, Erin (Author) / Thompson, Marilyn (Thesis advisor) / Tracey, Terence (Thesis advisor) / Green, Samuel (Committee member) / Arizona State University (Publisher)
Created2012