Matching Items (12)
Description
The goal of diagnostic assessment is to discriminate between groups. In many cases, a binary decision is made conditional on a cut score from a continuous scale. Psychometric methods can improve assessment by modeling a latent variable using item response theory (IRT), and IRT scores can subsequently be used to determine a cut score using receiver operating characteristic (ROC) curves. Psychometric methods provide reliable and interpretable scores, but the prediction of the diagnosis is not the primary product of the measurement process. In contrast, machine learning methods, such as regularization or binary recursive partitioning, can build a model from the assessment items to predict the probability of diagnosis. Machine learning predicts the diagnosis directly, but does not provide an inferential framework to explain why item responses are related to the diagnosis. It remains unclear whether psychometric and machine learning methods have comparable accuracy or if one method is preferable in some situations. In this study, Monte Carlo simulation methods were used to compare psychometric and machine learning methods on diagnostic classification accuracy. Results suggest that classification accuracy of psychometric models depends on the diagnostic-test correlation and prevalence of diagnosis. Also, machine learning methods that reduce prediction error have inflated specificity and very low sensitivity compared to the data-generating model, especially when prevalence is low. Finally, machine learning methods that use ROC curves to determine probability thresholds have comparable classification accuracy to the psychometric models as sample size, number of items, and number of item categories increase. Therefore, results suggest that machine learning models could provide a viable alternative for classification in diagnostic assessments. Strengths and limitations for each of the methods are discussed, and future directions are considered.
ContributorsGonzález, Oscar (Author) / Mackinnon, David P (Thesis advisor) / Edwards, Michael C (Thesis advisor) / Grimm, Kevin J. (Committee member) / Zheng, Yi (Committee member) / Arizona State University (Publisher)
Created2018
Description
Scant research examines the associations between parenting behaviors and the psychological health of Middle Eastern and North African (MENA) American youth. Developmental research consistently demonstrates that an authoritarian parenting style (often characterized by rejecting and controlling behaviors, and a common style among MENA parents) is maladaptive for offspring health; however, no study has empirically tested the associations of these behaviors from mothers and fathers with the health of MENA American youth. Using survey data from 314 MENA American young adults (Mage = 20 years, range 18 – 25 years, 56% female), the current study tested the associations between commonly studied parenting behaviors - acceptance, rejection, harsh parenting, and control - with the mental (stress, depression, and anxiety) and physical health (general health perceptions, pain, and somatization) of MENA American youth. Confirmatory factor analysis tested new items informed by preliminary focus groups with original items from the Child Report Parenting Behavior Inventory (CRPBI) to create culturally-informed parenting factors. Results indicated that youth-reported higher maternal acceptance was associated with fewer mental health symptoms, higher maternal harsh parenting with higher mental health symptoms, and higher maternal rejection with worse physical health; father rejection was associated with higher mental health symptoms and worse physical health. Further, the associations between parenting and physical health were moderated by youth Arabic orientation, such that those with higher Arabic orientation showed the best physical health at higher levels of acceptance, and the worst physical health at higher levels of rejection, harsh parenting, and control. Associations between parenting and health did not differ by youth gender. The current findings suggest cross-cultural similarities in the beneficial functions of parental acceptance, and detrimental functions of parental rejection and harsh parenting, with MENA American youth. The associations between parenting and health were exacerbated, for better or for worse, for more Arabic-oriented youth, suggesting these youth may be more greatly impacted by perceptions of their parents’ behaviors. Findings have implications for family interventions working with MENA populations.
ContributorsIbrahim, Mariam Hanna (Author) / Luecken, Linda J. (Thesis advisor) / Gonzales, Nancy A. (Committee member) / Edwards, Michael C (Committee member) / Doane, Leah D (Committee member) / Arizona State University (Publisher)
Created2019
Description
To make meaningful comparisons on a construct of interest across groups or over time, measurement invariance needs to exist for at least a subset of the observed variables that define the construct. Often, chi-square difference tests are used to test for measurement invariance. However, these statistics are affected by sample size such that larger sample sizes are associated with a greater prevalence of significant tests. Thus, using other measures of non-invariance to aid in the decision process would be beneficial. For this dissertation project, I proposed four new effect size measures of measurement non-invariance and analyzed a Monte Carlo simulation study to evaluate their properties and behavior in addition to the properties and behavior of an already existing effect size measure of non-invariance. The effect size measures were evaluated based on bias, variability, and consistency. Additionally, the factors that affected the value of the effect size measures were analyzed. All studied effect sizes were consistent, but three were biased under certain conditions. Further work is needed to establish benchmarks for the unbiased effect sizes.
ContributorsGunn, Heather J (Author) / Grimm, Kevin J. (Thesis advisor) / Edwards, Michael C (Thesis advisor) / Tein, Jenn-Yun (Committee member) / Anderson, Samantha F. (Committee member) / Arizona State University (Publisher)
Created2019
Description
Collider effects pose a major problem in psychological research. Colliders are third variables that bias the relationship between an independent and dependent variable when (1) the composition of a research sample is restricted by the scores on a collider variable or (2) researchers adjust for a collider variable in their statistical analyses. Both cases interfere with the accuracy and generalizability of statistical results. Despite their importance, collider effects remain relatively unknown in the social sciences. This research introduces both the conceptual and the mathematical foundation for collider effects and demonstrates how to calculate a collider effect and test it for statistical significance. Simulation studies examined the efficiency and accuracy of the collider estimation methods and tested the viability of Thorndike’s Case III equation as a potential solution to correcting for collider bias in cases of biased sample selection.
ContributorsLamp, Sophia Josephine (Author) / Mackinnon, David P (Thesis advisor) / Anderson, Samantha F (Committee member) / Edwards, Michael C (Committee member) / Arizona State University (Publisher)
Created2021
Description
Latent profile analysis (LPA), a type of finite mixture model, has grown in popularity due to its ability to detect latent classes or unobserved subgroups within a sample. Though numerous methods exist to determine the correct number of classes, past research has repeatedly demonstrated that no one method is consistently the best as each tends to struggle under specific conditions. Recently, the likelihood incremental percentage per parameter (LI3P), a method using a new approach, was proposed and tested which yielded promising initial results. To evaluate this new method more thoroughly, this study simulated 50,000 datasets, manipulating factors such as sample size, class distance, number of items, and number of classes. After evaluating the performance of the LI3P on simulated data, the LI3P is applied to LPA models fit to an empirical dataset to illustrate the method’s application. Results indicate the LI3P performs in line with standard class enumeration techniques, and primarily reflects class separation and the number of classes.
ContributorsHoupt, Russell Paul (Author) / Grimm, Kevin J (Thesis advisor) / McNeish, Daniel (Committee member) / Edwards, Michael C (Committee member) / Arizona State University (Publisher)
Created2022
Description
Decision trees is a machine learning technique that searches the predictor space for the variable and observed value that leads to the best prediction when the data are split into two nodes based on the variable and splitting value. Conditional Inference Trees (CTREEs) is a non-parametric class of decision trees that uses statistical theory in order to select variables for splitting. Missing data can be problematic in decision trees because of an inability to place an observation with a missing value into a node based on the chosen splitting variable. Moreover, missing data can alter the selection process because of its inability to place observations with missing values. Simple missing data approaches (e.g., deletion, majority rule, and surrogate split) have been implemented in decision tree algorithms; however, more sophisticated missing data techniques have not been thoroughly examined. In addition to these approaches, this dissertation proposed a modified multiple imputation approach to handling missing data in CTREEs. A simulation was conducted to compare this approach with simple missing data approaches as well as single imputation and a multiple imputation with prediction averaging. Results revealed that simple approaches (i.e., majority rule, treat missing as its own category, and listwise deletion) were effective in handling missing data in CTREEs. The modified multiple imputation approach did not perform very well against simple approaches in most conditions, but this approach did seem best suited for small sample sizes and extreme missingness situations.
ContributorsManapat, Danielle Marie (Author) / Grimm, Kevin J (Thesis advisor) / Edwards, Michael C (Thesis advisor) / McNeish, Daniel (Committee member) / Anderson, Samantha F (Committee member) / Arizona State University (Publisher)
Created2023
Description
This research explores tests for statistical suppression. Suppression is a statistical phenomenon whereby the magnitude of an effect becomes larger when another variable is added to the regression equation. From a causal perspective, suppression occurs when there is inconsistent mediation or negative confounding. Several different estimators for suppression are evaluated conceptually and in a statistical simulation study where we impose suppression and non-suppression conditions. For each estimator without an existing standard error formula, one was derived in order to conduct significance tests and build confidence intervals. Overall, two of the estimators were biased and had poor coverage, one worked well but had inflated type-I error rates when the population model was complete mediation. As a result of analyzing these three tests, a fourth was considered in the late stages of the project and showed promising results that address concerns of the other tests. When the tests were applied to real data, they gave similar results and were consistent.
ContributorsMuniz, Felix (Author) / Mackinnon, David P (Thesis advisor) / Anderson, Samantha F. (Committee member) / McNeish, Daniel M (Committee member) / Arizona State University (Publisher)
Created2020
Description
The last two decades have seen growing awareness of and emphasis on the replication of empirical findings. While this is a large literature, very little of it has focused on or considered the interaction of replication and psychometrics. This is unfortunate given that sound measurement is crucial when considering the complex constructs studied in psychological research. If the psychometric properties of a scale fail to replicate, then inferences made using scores from that scale are questionable at best. In this dissertation, I begin to address replication issues in factor analysis – a widely used psychometric method in psychology. After noticing inconsistencies across results for studies that factor analyzed the same scale, I sought to gain a better understanding of what replication means in factor analysis as well as address issues that affect the replicability of factor analytic models. With this work, I take steps toward integrating factor analysis into the broader replication discussion. Ultimately, the goal of this dissertation was to highlight the importance of psychometric replication and bring attention to its role in fostering a more replicable scientific literature.
ContributorsManapat, Patrick D. (Author) / Edwards, Michael C. (Thesis advisor) / Anderson, Samantha F. (Thesis advisor) / Grimm, Kevin J. (Committee member) / Levy, Roy (Committee member) / Arizona State University (Publisher)
Created2022
Description
Statistical inference from mediation analysis applies to populations, however, researchers and clinicians may be interested in making inference to individual clients or small, localized groups of people. Person-oriented approaches focus on the differences between people, or latent groups of people, to ask how individuals differ across variables, and can help researchers avoid ecological fallacies when making inferences about individuals. Traditional variable-oriented mediation assumes the population undergoes a homogenous reaction to the mediating process. However, mediation is also described as an intra-individual process where each person passes from a predictor, through a mediator, to an outcome (Collins, Graham, & Flaherty, 1998). Configural frequency mediation is a person-oriented analysis of contingency tables that has not been well-studied or implemented since its introduction in the literature (von Eye, Mair, & Mun, 2010; von Eye, Mun, & Mair, 2009). The purpose of this study is to describe CFM and investigate its statistical properties while comparing it to traditional and casual inference mediation methods. The results of this study show that joint significance mediation tests results in better Type I error rates but limit the person-oriented interpretations of CFM. Although the estimator for logistic regression and causal mediation are different, they both perform well in terms of Type I error and power, although the causal estimator had higher bias than expected, which is discussed in the limitations section.
ContributorsSmyth, Heather Lynn (Author) / Mackinnon, David P (Thesis advisor) / Grimm, Kevin J. (Committee member) / Edwards, Michael C (Committee member) / Arizona State University (Publisher)
Created2019
Description
Past research suggests that both Alcohol Expectancies and Subjective Response are strong predictors of drinking. However, most studies do not account for the shared variance or relations between the two. Social cognitive and expectancy theories suggest that cognitions may distort reality, creating a discrepancy between expected and subjective effects. Only one study has tested the effects of such discrepancies (Morean et al., 2015), but that study was cross-sectional, making it impossible to determine the direction of effects. As such, the present study sought to test prospective associations between expectancy-subjective response interactions and future drinking behavior. Participants (N=448) were randomly assigned to receive alcohol (target blood alcohol alcohol =.08 g%) or placebo, with 270 in the alcohol condition. Alcohol expectancies and subjective response were assessed across the full range of affective space of valence by arousal. Hierarchical regression tested whether expectancies, subjective response, and their interaction predicted follow-up drinking in 258 participants who reached a blood alcohol curve of >.06 (to differentiate blood alcohol curve limbs). Covariates included gender, age, drinking context, and baseline drinking. High arousal subjective response was tested on the ascending limb and low arousal subjective response on the descending limb. High arousal positive expectancies and subjective response interacted to predict future drinking, such that mean and low levels of high arousal positive subjective response were associated with more drinking when expectancies were higher. High arousal negative expectancies and subjective response also interacted to predict future drinking, such that high levels of high arousal negative subjective response marginally predicted more drinking when expectancies were lower. There were no interactions between low arousal positive or low arousal negative expectancies and subjective response. Results suggest that those who expected high arousal positive subjective response but did not receive many of these effects drank more, and those who did not expect to feel high arousal negative subjective response but did in fact feel these effects also drank more. The results suggest that challenging inaccurate positive expectancies and increasing awareness of true negative subjective response may be efficacious ways to reduce drinking.
ContributorsWaddell, Jack Thomas (Author) / Corbin, William R. (Thesis advisor) / Chassin, Laurie (Thesis advisor) / Anderson, Samantha F. (Committee member) / Arizona State University (Publisher)
Created2020