Matching Items (14)
Filtering by
- All Subjects: psychology
- All Subjects: psychometrics
Description
This project studied a four-variable single mediator model, a single mediator model: X (independent variable) to M (mediator) to Y (dependent variable), and a confounder (U) that influences M and Y. Confounding represents a threat to the causal interpretation in mediation analysis. For instance, if X represents random assignment to control and treatment conditions, the effect of X on M and the effect of X on Y have a causal interpretation under certain reasonable assumptions. However, the randomization of X does not allow for a causal interpretation of the M to Y effect unless certain confounding assumptions are satisfied. The aim of this project was to develop a significance test and an effect size comparison for two sensitivity to confounding analyses methods: Left Out Variables Error (L.O.V.E.) and the correlated residuals method. Further, the project assessed the accuracy of the methods for identifying confounding bias by simulating data with and without confounding bias.
ContributorsAlvarez Bartolo, Diana (Author) / Mackinnon, David P. (Thesis advisor) / Grimm, Kevin J. (Committee member) / McNeish, Daniel (Committee member) / Arizona State University (Publisher)
Created2022
Description
The goal of diagnostic assessment is to discriminate between groups. In many cases, a binary decision is made conditional on a cut score from a continuous scale. Psychometric methods can improve assessment by modeling a latent variable using item response theory (IRT), and IRT scores can subsequently be used to determine a cut score using receiver operating characteristic (ROC) curves. Psychometric methods provide reliable and interpretable scores, but the prediction of the diagnosis is not the primary product of the measurement process. In contrast, machine learning methods, such as regularization or binary recursive partitioning, can build a model from the assessment items to predict the probability of diagnosis. Machine learning predicts the diagnosis directly, but does not provide an inferential framework to explain why item responses are related to the diagnosis. It remains unclear whether psychometric and machine learning methods have comparable accuracy or if one method is preferable in some situations. In this study, Monte Carlo simulation methods were used to compare psychometric and machine learning methods on diagnostic classification accuracy. Results suggest that classification accuracy of psychometric models depends on the diagnostic-test correlation and prevalence of diagnosis. Also, machine learning methods that reduce prediction error have inflated specificity and very low sensitivity compared to the data-generating model, especially when prevalence is low. Finally, machine learning methods that use ROC curves to determine probability thresholds have comparable classification accuracy to the psychometric models as sample size, number of items, and number of item categories increase. Therefore, results suggest that machine learning models could provide a viable alternative for classification in diagnostic assessments. Strengths and limitations for each of the methods are discussed, and future directions are considered.
ContributorsGonzález, Oscar (Author) / Mackinnon, David P (Thesis advisor) / Edwards, Michael C (Thesis advisor) / Grimm, Kevin J. (Committee member) / Zheng, Yi (Committee member) / Arizona State University (Publisher)
Created2018
Description
A record number of Latino students are enrolling in higher education in the U.S., but as a group Latinos are the least likely to complete a bachelor’s degree. Cultural factors theoretically contribute to Latino students’ success, including orientation toward ethnic heritage and mainstream cultures (i.e., dual cultural adaptation), feeling comfortable navigating two cultural contexts (i.e., biculturalism), and the degree of fit between students’ cultural backgrounds and the cultural landscapes of educational institutions (i.e., cultural congruity). In a two-part study, these cultural factors were examined in relation to the hypothalamic-pituitary-adrenal (HPA) axis stress response (indexed by salivary cortisol), a physiological mechanism that may underlie how psychosocial stress influences academic achievement and health. First, Latino students’ cortisol responses to stress were estimated in their daily lives prior to college using ecological momentary assessment (N = 206; 64.6% female; Mage = 18.10). Results from three-level growth models indicated that cortisol levels were lower following greater perceived stress than usual for students endorsing greater Latino cultural values (e.g., familism), compared to students endorsing average or below-average levels of these values. Second, cortisol and subjective responses to a standard public speaking stress task were examined in a subsample of these same students in their first semester of college (N = 84; 63.1% female). In an experimental design, viewing a brief video prior to the stress task conveying the university’s commitment to cultural diversity and inclusion (compared to a generic campus tour) reduced cortisol reactivity and negative affect for students with greater Latino cultural values, and also reduced post-task cortisol levels for students with greater mainstream U.S. cultural values (e.g., competition). These findings join the growing science of culture and biology interplay, while also informing initiatives to support first-year Latino students and the universities that serve them.
ContributorsSladek, Michael R. (Author) / Doane, Leah D (Thesis advisor) / Gonzales, Nancy A. (Committee member) / Grimm, Kevin J. (Committee member) / Luecken, Linda J. (Committee member) / Arizona State University (Publisher)
Created2018
Description
To make meaningful comparisons on a construct of interest across groups or over time, measurement invariance needs to exist for at least a subset of the observed variables that define the construct. Often, chi-square difference tests are used to test for measurement invariance. However, these statistics are affected by sample size such that larger sample sizes are associated with a greater prevalence of significant tests. Thus, using other measures of non-invariance to aid in the decision process would be beneficial. For this dissertation project, I proposed four new effect size measures of measurement non-invariance and analyzed a Monte Carlo simulation study to evaluate their properties and behavior in addition to the properties and behavior of an already existing effect size measure of non-invariance. The effect size measures were evaluated based on bias, variability, and consistency. Additionally, the factors that affected the value of the effect size measures were analyzed. All studied effect sizes were consistent, but three were biased under certain conditions. Further work is needed to establish benchmarks for the unbiased effect sizes.
ContributorsGunn, Heather J (Author) / Grimm, Kevin J. (Thesis advisor) / Edwards, Michael C (Thesis advisor) / Tein, Jenn-Yun (Committee member) / Anderson, Samantha F. (Committee member) / Arizona State University (Publisher)
Created2019
Description
Statistical mediation analysis allows researchers to identify the most important the mediating constructs in the causal process studied. Information about the mediating processes can be used to make interventions more powerful by enhancing successful program components and by not implementing components that did not significantly change the outcome. Identifying mediators is especially relevant when the hypothesized mediating construct consists of multiple related facets. The general definition of the construct and its facets might relate differently to external criteria. However, current methods do not allow researchers to study the relationships between general and specific aspects of a construct to an external criterion simultaneously. This study proposes a bifactor measurement model for the mediating construct as a way to represent the general aspect and specific facets of a construct simultaneously. Monte Carlo simulation results are presented to help to determine under what conditions researchers can detect the mediated effect when one of the facets of the mediating construct is the true mediator, but the mediator is treated as unidimensional. Results indicate that parameter bias and detection of the mediated effect depends on the facet variance represented in the mediation model. This study contributes to the largely unexplored area of measurement issues in statistical mediation analysis.
ContributorsGonzález, Oscar (Author) / Mackinnon, David P (Thesis advisor) / Grimm, Kevin J. (Committee member) / Zheng, Yi (Committee member) / Arizona State University (Publisher)
Created2016
Description
Researchers who conduct longitudinal studies are inherently interested in studying individual and population changes over time (e.g., mathematics achievement, subjective well-being). To answer such research questions, models of change (e.g., growth models) make the assumption of longitudinal measurement invariance. In many applied situations, key constructs are measured by a collection of ordered-categorical indicators (e.g., Likert scale items). To evaluate longitudinal measurement invariance with ordered-categorical indicators, a set of hierarchical models can be sequentially tested and compared. If the statistical tests of measurement invariance fail to be supported for one of the models, it is useful to have a method with which to gauge the practical significance of the differences in measurement model parameters over time. Drawing on studies of latent growth models and second-order latent growth models with continuous indicators (e.g., Kim & Willson, 2014a; 2014b; Leite, 2007; Wirth, 2008), this study examined the performance of a potential sensitivity analysis to gauge the practical significance of violations of longitudinal measurement invariance for ordered-categorical indicators using second-order latent growth models. The change in the estimate of the second-order growth parameters following the addition of an incorrect level of measurement invariance constraints at the first-order level was used as an effect size for measurement non-invariance. This study investigated how sensitive the proposed sensitivity analysis was to different locations of non-invariance (i.e., non-invariance in the factor loadings, the thresholds, and the unique factor variances) given a sufficient sample size. This study also examined whether the sensitivity of the proposed sensitivity analysis depended on a number of other factors including the magnitude of non-invariance, the number of non-invariant indicators, the number of non-invariant occasions, and the number of response categories in the indicators.
ContributorsLiu, Yu, Ph.D (Author) / West, Stephen G. (Thesis advisor) / Tein, Jenn-Yun (Thesis advisor) / Green, Samuel (Committee member) / Grimm, Kevin J. (Committee member) / Arizona State University (Publisher)
Created2016
Description
Currently, there is a clear gap in the missing data literature for three-level models.
To date, the literature has only focused on the theoretical and algorithmic work
required to implement three-level imputation using the joint model (JM) method of
imputation, leaving relatively no work done on fully conditional specication (FCS)
method. Moreover, the literature lacks any methodological evaluation of three-level
imputation. Thus, this thesis serves two purposes: (1) to develop an algorithm in
order to implement FCS in the context of a three-level model and (2) to evaluate
both imputation methods. The simulation investigated a random intercept model
under both 20% and 40% missing data rates. The ndings of this thesis suggest
that the estimates for both JM and FCS were largely unbiased, gave good coverage,
and produced similar results. The sole exception for both methods was the slope for
the level-3 variable, which was modestly biased. The bias exhibited by the methods
could be due to the small number of clusters used. This nding suggests that future
research ought to investigate and establish clear recommendations for the number of
clusters required by these imputation methods. To conclude, this thesis serves as a
preliminary start in tackling a much larger issue and gap in the current missing data
literature.
To date, the literature has only focused on the theoretical and algorithmic work
required to implement three-level imputation using the joint model (JM) method of
imputation, leaving relatively no work done on fully conditional specication (FCS)
method. Moreover, the literature lacks any methodological evaluation of three-level
imputation. Thus, this thesis serves two purposes: (1) to develop an algorithm in
order to implement FCS in the context of a three-level model and (2) to evaluate
both imputation methods. The simulation investigated a random intercept model
under both 20% and 40% missing data rates. The ndings of this thesis suggest
that the estimates for both JM and FCS were largely unbiased, gave good coverage,
and produced similar results. The sole exception for both methods was the slope for
the level-3 variable, which was modestly biased. The bias exhibited by the methods
could be due to the small number of clusters used. This nding suggests that future
research ought to investigate and establish clear recommendations for the number of
clusters required by these imputation methods. To conclude, this thesis serves as a
preliminary start in tackling a much larger issue and gap in the current missing data
literature.
ContributorsKeller, Brian Tinnell (Author) / Enders, Craig K. (Thesis advisor) / Grimm, Kevin J. (Committee member) / Levy, Roy (Committee member) / Arizona State University (Publisher)
Created2015
Description
Time metric is an important consideration for all longitudinal models because it can influence the interpretation of estimates, parameter estimate accuracy, and model convergence in longitudinal models with latent variables. Currently, the literature on latent difference score (LDS) models does not discuss the importance of time metric. Furthermore, there is little research using simulations to investigate LDS models. This study examined the influence of time metric on model estimation, interpretation, parameter estimate accuracy, and convergence in LDS models using empirical simulations. Results indicated that for a time structure with a true time metric where participants had different starting points and unequally spaced intervals, LDS models fit with a restructured and less informative time metric resulted in biased parameter estimates. However, models examined using the true time metric were less likely to converge than models using the restructured time metric, likely due to missing data. Where participants had different starting points but equally spaced intervals, LDS models fit with a restructured time metric resulted in biased estimates of intercept means, but all other parameter estimates were unbiased, and models examined using the true time metric had less convergence than the restructured time metric as well due to missing data. The findings of this study support prior research on time metric in longitudinal models, and further research should examine these findings under alternative conditions. The importance of these findings for substantive researchers is discussed.
ContributorsO'Rourke, Holly P (Author) / Grimm, Kevin J. (Thesis advisor) / Mackinnon, David P (Thesis advisor) / Chassin, Laurie (Committee member) / Aiken, Leona S. (Committee member) / Arizona State University (Publisher)
Created2016
Description
The comparison of between- versus within-person relations addresses a central issue in psychological research regarding whether group-level relations among variables generalize to individual group members. Between- and within-person effects may differ in magnitude as well as direction, and contextual multilevel models can accommodate this difference. Contextual multilevel models have been explicated mostly for cross-sectional data, but they can also be applied to longitudinal data where level-1 effects represent within-person relations and level-2 effects represent between-person relations. With longitudinal data, estimating the contextual effect allows direct evaluation of whether between-person and within-person effects differ. Furthermore, these models, unlike single-level models, permit individual differences by allowing within-person slopes to vary across individuals. This study examined the statistical performance of the contextual model with a random slope for longitudinal within-person fluctuation data.
A Monte Carlo simulation was used to generate data based on the contextual multilevel model, where sample size, effect size, and intraclass correlation (ICC) of the predictor variable were varied. The effects of simulation factors on parameter bias, parameter variability, and standard error accuracy were assessed. Parameter estimates were in general unbiased. Power to detect the slope variance and contextual effect was over 80% for most conditions, except some of the smaller sample size conditions. Type I error rates for the contextual effect were also high for some of the smaller sample size conditions. Conclusions and future directions are discussed.
A Monte Carlo simulation was used to generate data based on the contextual multilevel model, where sample size, effect size, and intraclass correlation (ICC) of the predictor variable were varied. The effects of simulation factors on parameter bias, parameter variability, and standard error accuracy were assessed. Parameter estimates were in general unbiased. Power to detect the slope variance and contextual effect was over 80% for most conditions, except some of the smaller sample size conditions. Type I error rates for the contextual effect were also high for some of the smaller sample size conditions. Conclusions and future directions are discussed.
ContributorsWurpts, Ingrid Carlson (Author) / Mackinnon, David P (Thesis advisor) / West, Stephen G. (Committee member) / Grimm, Kevin J. (Committee member) / Suk, Hye Won (Committee member) / Arizona State University (Publisher)
Created2016
Description
Research has consistently shown that gay/lesbian/bisexual (GLB) or sexual minority youth are at an increased risk for adverse outcomes resulting from the stress caused by continual exposure to negative events (e.g., victimization, discrimination). The present study used a nationally representative sample of adolescents to test mechanisms that may be responsible for the differences in offending behaviors among sexual minority and heterosexual adolescents. Specifically, this study tested whether bisexual adolescents received less maternal support than did heterosexual adolescents because of their sexual orientation, thus increasing the likelihood that they run away from home. This study then examined whether the greater likelihood that bisexual adolescents running away would lead to them committing a significantly higher variety of income-based offenses, but not a significantly higher variety of aggression-based offenses. This study tested the hypothesized mediation model using two separate indicators of sexual orientation measured at two different time points, modeled outcomes in two ways, as well as estimated the models separately for boys and girls. Structural equation modeling was used to test the hypothesized direct and indirect relations. Results showed support for maternal support and running away mediating the relations between sexual orientation and offending behaviors for the model predicting the likelihood of committing either an aggressive or an income offense, but only for girls who identified as bisexual in early adulthood. Results did not support these relations for the other models, suggesting that bisexual females have unique needs when it comes to prevention and intervention. Results also highlight the need for a greater understanding of sexual orientation measurement methodology.
ContributorsMansion, Andre (Author) / Chassin, Laurie (Thesis advisor) / Barrera, Manuel (Committee member) / Grimm, Kevin J. (Committee member) / Toomey, Russell B (Committee member) / Arizona State University (Publisher)
Created2018