Matching Items (15)
Filtering by
- All Subjects: Machine Learning
- All Subjects: Game Theory
- Creators: Xue, Guoliang
Description
Sparsity has become an important modeling tool in areas such as genetics, signal and audio processing, medical image processing, etc. Via the penalization of l-1 norm based regularization, the structured sparse learning algorithms can produce highly accurate models while imposing various predefined structures on the data, such as feature groups or graphs. In this thesis, I first propose to solve a sparse learning model with a general group structure, where the predefined groups may overlap with each other. Then, I present three real world applications which can benefit from the group structured sparse learning technique. In the first application, I study the Alzheimer's Disease diagnosis problem using multi-modality neuroimaging data. In this dataset, not every subject has all data sources available, exhibiting an unique and challenging block-wise missing pattern. In the second application, I study the automatic annotation and retrieval of fruit-fly gene expression pattern images. Combined with the spatial information, sparse learning techniques can be used to construct effective representation of the expression images. In the third application, I present a new computational approach to annotate developmental stage for Drosophila embryos in the gene expression images. In addition, it provides a stage score that enables one to more finely annotate each embryo so that they are divided into early and late periods of development within standard stage demarcations. Stage scores help us to illuminate global gene activities and changes much better, and more refined stage annotations improve our ability to better interpret results when expression pattern matches are discovered between genes.
ContributorsYuan, Lei (Author) / Ye, Jieping (Thesis advisor) / Wang, Yalin (Committee member) / Xue, Guoliang (Committee member) / Kumar, Sudhir (Committee member) / Arizona State University (Publisher)
Created2013
Description
Multi-task learning (MTL) aims to improve the generalization performance (of the resulting classifiers) by learning multiple related tasks simultaneously. Specifically, MTL exploits the intrinsic task relatedness, based on which the informative domain knowledge from each task can be shared across multiple tasks and thus facilitate the individual task learning. It is particularly desirable to share the domain knowledge (among the tasks) when there are a number of related tasks but only limited training data is available for each task. Modeling the relationship of multiple tasks is critical to the generalization performance of the MTL algorithms. In this dissertation, I propose a series of MTL approaches which assume that multiple tasks are intrinsically related via a shared low-dimensional feature space. The proposed MTL approaches are developed to deal with different scenarios and settings; they are respectively formulated as mathematical optimization problems of minimizing the empirical loss regularized by different structures. For all proposed MTL formulations, I develop the associated optimization algorithms to find their globally optimal solution efficiently. I also conduct theoretical analysis for certain MTL approaches by deriving the globally optimal solution recovery condition and the performance bound. To demonstrate the practical performance, I apply the proposed MTL approaches on different real-world applications: (1) Automated annotation of the Drosophila gene expression pattern images; (2) Categorization of the Yahoo web pages. Our experimental results demonstrate the efficiency and effectiveness of the proposed algorithms.
ContributorsChen, Jianhui (Author) / Ye, Jieping (Thesis advisor) / Kumar, Sudhir (Committee member) / Liu, Huan (Committee member) / Xue, Guoliang (Committee member) / Arizona State University (Publisher)
Created2011
Description
Sparse learning is a technique in machine learning for feature selection and dimensionality reduction, to find a sparse set of the most relevant features. In any machine learning problem, there is a considerable amount of irrelevant information, and separating relevant information from the irrelevant information has been a topic of focus. In supervised learning like regression, the data consists of many features and only a subset of the features may be responsible for the result. Also, the features might require special structural requirements, which introduces additional complexity for feature selection. The sparse learning package, provides a set of algorithms for learning a sparse set of the most relevant features for both regression and classification problems. Structural dependencies among features which introduce additional requirements are also provided as part of the package. The features may be grouped together, and there may exist hierarchies and over- lapping groups among these, and there may be requirements for selecting the most relevant groups among them. In spite of getting sparse solutions, the solutions are not guaranteed to be robust. For the selection to be robust, there are certain techniques which provide theoretical justification of why certain features are selected. The stability selection, is a method for feature selection which allows the use of existing sparse learning methods to select the stable set of features for a given training sample. This is done by assigning probabilities for the features: by sub-sampling the training data and using a specific sparse learning technique to learn the relevant features, and repeating this a large number of times, and counting the probability as the number of times a feature is selected. Cross-validation which is used to determine the best parameter value over a range of values, further allows to select the best parameter value. This is done by selecting the parameter value which gives the maximum accuracy score. With such a combination of algorithms, with good convergence guarantees, stable feature selection properties and the inclusion of various structural dependencies among features, the sparse learning package will be a powerful tool for machine learning research. Modular structure, C implementation, ATLAS integration for fast linear algebraic subroutines, make it one of the best tool for a large sparse setting. The varied collection of algorithms, support for group sparsity, batch algorithms, are a few of the notable functionality of the SLEP package, and these features can be used in a variety of fields to infer relevant elements. The Alzheimer Disease(AD) is a neurodegenerative disease, which gradually leads to dementia. The SLEP package is used for feature selection for getting the most relevant biomarkers from the available AD dataset, and the results show that, indeed, only a subset of the features are required to gain valuable insights.
ContributorsThulasiram, Ramesh (Author) / Ye, Jieping (Thesis advisor) / Xue, Guoliang (Committee member) / Sen, Arunabha (Committee member) / Arizona State University (Publisher)
Created2011
Description
Discriminative learning when training and test data belong to different distributions is a challenging and complex task. Often times we have very few or no labeled data from the test or target distribution, but we may have plenty of labeled data from one or multiple related sources with different distributions. Due to its capability of migrating knowledge from related domains, transfer learning has shown to be effective for cross-domain learning problems. In this dissertation, I carry out research along this direction with a particular focus on designing efficient and effective algorithms for BioImaging and Bilingual applications. Specifically, I propose deep transfer learning algorithms which combine transfer learning and deep learning to improve image annotation performance. Firstly, I propose to generate the deep features for the Drosophila embryo images via pretrained deep models and build linear classifiers on top of the deep features. Secondly, I propose to fine-tune the pretrained model with a small amount of labeled images. The time complexity and performance of deep transfer learning methodologies are investigated. Promising results have demonstrated the knowledge transfer ability of proposed deep transfer algorithms. Moreover, I propose a novel Robust Principal Component Analysis (RPCA) approach to process the noisy images in advance. In addition, I also present a two-stage re-weighting framework for general domain adaptation problems. The distribution of source domain is mapped towards the target domain in the first stage, and an adaptive learning model is proposed in the second stage to incorporate label information from the target domain if it is available. Then the proposed model is applied to tackle cross lingual spam detection problem at LinkedIn’s website. Our experimental results on real data demonstrate the efficiency and effectiveness of the proposed algorithms.
ContributorsSun, Qian (Author) / Ye, Jieping (Committee member) / Xue, Guoliang (Committee member) / Liu, Huan (Committee member) / Li, Jing (Committee member) / Arizona State University (Publisher)
Created2015
Description
Need-based transfers (NBTs) are a form of risk-pooling in which binary welfare exchanges
occur to preserve the viable participation of individuals in an economy, e.g. reciprocal gifting
of cattle among East African herders or food sharing among vampire bats. With the
broad goal of better understanding the mathematics of such binary welfare and risk pooling,
agent-based simulations are conducted to explore socially optimal transfer policies
and sharing network structures, kinetic exchange models that utilize tools from the kinetic
theory of gas dynamics are utilized to characterize the wealth distribution of an NBT economy,
and a variant of repeated prisoner’s dilemma is analyzed to determine whether and
why individuals would participate in such a system of reciprocal altruism.
From agent-based simulation and kinetic exchange models, it is found that regressive
NBT wealth redistribution acts as a cutting stock optimization heuristic that most efficiently
matches deficits to surpluses to improve short-term survival; however, progressive
redistribution leads to a wealth distribution that is more stable in volatile environments and
therefore is optimal for long-term survival. Homogeneous sharing networks with low variance
in degree are found to be ideal for maintaining community viability as the burden and
benefit of NBTs is equally shared. Also, phrasing NBTs as a survivor’s dilemma reveals
parameter regions where the repeated game becomes equivalent to a stag hunt or harmony
game, and thus where cooperation is evolutionarily stable.
occur to preserve the viable participation of individuals in an economy, e.g. reciprocal gifting
of cattle among East African herders or food sharing among vampire bats. With the
broad goal of better understanding the mathematics of such binary welfare and risk pooling,
agent-based simulations are conducted to explore socially optimal transfer policies
and sharing network structures, kinetic exchange models that utilize tools from the kinetic
theory of gas dynamics are utilized to characterize the wealth distribution of an NBT economy,
and a variant of repeated prisoner’s dilemma is analyzed to determine whether and
why individuals would participate in such a system of reciprocal altruism.
From agent-based simulation and kinetic exchange models, it is found that regressive
NBT wealth redistribution acts as a cutting stock optimization heuristic that most efficiently
matches deficits to surpluses to improve short-term survival; however, progressive
redistribution leads to a wealth distribution that is more stable in volatile environments and
therefore is optimal for long-term survival. Homogeneous sharing networks with low variance
in degree are found to be ideal for maintaining community viability as the burden and
benefit of NBTs is equally shared. Also, phrasing NBTs as a survivor’s dilemma reveals
parameter regions where the repeated game becomes equivalent to a stag hunt or harmony
game, and thus where cooperation is evolutionarily stable.
ContributorsKayser, Kirk (Author) / Armbruster, Dieter (Thesis advisor) / Lampert, Adam (Committee member) / Ringhofer, Christian (Committee member) / Motsch, Sebastien (Committee member) / Gardner, Carl (Committee member) / Arizona State University (Publisher)
Created2018
Description
One of the salient challenges of sustainability is the Tragedy of the Commons, where individuals acting independently and rationally deplete a common resource despite their understanding that it is not in the group's long term best interest to do so. Hardin presents this dilemma as nearly intractable and solvable only by drastic, government-mandated social reforms, while Ostrom's empirical work demonstrates that community-scale collaboration can circumvent tragedy without any elaborate outside intervention. Though more optimistic, Ostrom's work provides scant insight into larger-scale dilemmas such as climate change. Consequently, it remains unclear if the sustainable management of global resources is possible without significant government mediation. To investigate, we conducted two game theoretic experiments that challenged students in different countries to collaborate digitally and manage a hypothetical common resource. One experiment involved students attending Arizona State University and the Rochester Institute of Technology in the US and Mountains of the Moon University in Uganda, while the other included students at Arizona State and the Management Development Institute in India. In both experiments, students were randomly assigned to one of three production roles: Luxury, Intermediate, and Subsistence. Students then made individual decisions about how many units of goods they wished to produce up to a set maximum per production class. Luxury players gain the most profit (i.e. grade points) per unit produced, but they also emit the most externalities, or social costs, which directly subtract from the profit of everybody else in the game; Intermediate players produce a medium amount of profit and externalities per unit, and Subsistence players produce a low amount of profit and externalities per unit. Variables influencing and/or inhibiting collaboration were studied using pre- and post-game surveys. This research sought to answer three questions: 1) Are international groups capable of self-organizing in a way that promotes sustainable resource management?, 2) What are the key factors that inhibit or foster collective action among international groups?, and 3) How well do Hardin's theories and Ostrom's empirical models predict the observed behavior of students in the game? The results of gameplay suggest that international cooperation is possible, though likely sub-optimal. Statistical analysis of survey data revealed that heterogeneity and levels of trust significantly influenced game behavior. Specific traits of heterogeneity among students found to be significant were income, education, assigned production role, number of people in one's household, college class, college major, and military service. Additionally, it was found that Ostrom's collective action framework was a better predictor of game outcome than Hardin's theories. Overall, this research lends credence to the plausibility of international cooperation in tragedy of the commons scenarios such as climate change, though much work remains to be done.
ContributorsStanton, Albert Grayson (Author) / Clark, Susan Spierre (Thesis director) / Seager, Thomas (Committee member) / Civil, Environmental and Sustainable Engineering Programs (Contributor) / Barrett, The Honors College (Contributor)
Created2014-12
Description
This dissertation investigates the dynamics of evolutionary games based on the framework of interacting particle systems in which individuals are discrete, space is explicit, and dynamics are stochastic. Its focus is on 2-strategy games played on a d-dimensional integer lattice with a range of interaction M. An overview of related past work is given along with a summary of the dynamics in the mean-field model, which is described by the replicator equation. Then the dynamics of the interacting particle system is considered, first when individuals are updated according to the best-response update process and then the death-birth update process. Several interesting results are derived, and the differences between the interacting particle system model and the replicator dynamics are emphasized. The terms selfish and altruistic are defined according to a certain ordering of payoff parameters. In these terms, the replicator dynamics are simple: coexistence occurs if both strategies are altruistic; the selfish strategy wins if one strategy is selfish and the other is altruistic; and there is bistability if both strategies are selfish. Under the best-response update process, it is shown that there is no bistability region. Instead, in the presence of at least one selfish strategy, the most selfish strategy wins, while there is still coexistence if both strategies are altruistic. Under the death-birth update process, it is shown that regardless of the range of interactions and the dimension, regions of coexistence and bistability are both reduced. Additionally, coexistence occurs in some parameter region for large enough interaction ranges. Finally, in contrast with the replicator equation and the best-response update process, cooperators can win in the prisoner's dilemma for the death-birth process in one-dimensional nearest-neighbor interactions.
ContributorsEvilsizor, Stephen (Author) / Lanchier, Nicolas (Thesis advisor) / Kang, Yun (Committee member) / Motsch, Sebastien (Committee member) / Smith, Hal (Committee member) / Thieme, Horst (Committee member) / Arizona State University (Publisher)
Created2016
Description
Passwords are ubiquitous and are poised to stay that way due to their relative usability, security and deployability when compared with alternative authentication schemes. Unfortunately, humans struggle with some of the assumptions or requirements that are necessary for truly strong passwords. As administrators try to push users towards password complexity and diversity, users still end up using predictable mangling patterns on old passwords and reusing the same passwords across services; users even inadvertently converge on the same patterns to a surprising degree, making an attacker’s job easier. This work explores using machine learning techniques to pick out strong passwords from weak ones, from a dataset of 10 million passwords, based on how structurally similar they were to the rest of the set.
ContributorsTodd, Margaret Nicole (Author) / Xue, Guoliang (Thesis advisor) / Ahn, Gail-Joon (Committee member) / Huang, Dijiang (Committee member) / Arizona State University (Publisher)
Created2016
Description
Large-scale $\ell_1$-regularized loss minimization problems arise in high-dimensional applications such as compressed sensing and high-dimensional supervised learning, including classification and regression problems. In many applications, it remains challenging to apply the sparse learning model to large-scale problems that have massive data samples with high-dimensional features. One popular and promising strategy is to scaling up the optimization problem in parallel. Parallel solvers run multiple cores on a shared memory system or a distributed environment to speed up the computation, while the practical usage is limited by the huge dimension in the feature space and synchronization problems.
In this dissertation, I carry out the research along the direction with particular focuses on scaling up the optimization of sparse learning for supervised and unsupervised learning problems. For the supervised learning, I firstly propose an asynchronous parallel solver to optimize the large-scale sparse learning model in a multithreading environment. Moreover, I propose a distributed framework to conduct the learning process when the dataset is distributed stored among different machines. Then the proposed model is further extended to the studies of risk genetic factors for Alzheimer's Disease (AD) among different research institutions, integrating a group feature selection framework to rank the top risk SNPs for AD. For the unsupervised learning problem, I propose a highly efficient solver, termed Stochastic Coordinate Coding (SCC), scaling up the optimization of dictionary learning and sparse coding problems. The common issue for the medical imaging research is that the longitudinal features of patients among different time points are beneficial to study together. To further improve the dictionary learning model, I propose a multi-task dictionary learning method, learning the different task simultaneously and utilizing shared and individual dictionary to encode both consistent and changing imaging features.
In this dissertation, I carry out the research along the direction with particular focuses on scaling up the optimization of sparse learning for supervised and unsupervised learning problems. For the supervised learning, I firstly propose an asynchronous parallel solver to optimize the large-scale sparse learning model in a multithreading environment. Moreover, I propose a distributed framework to conduct the learning process when the dataset is distributed stored among different machines. Then the proposed model is further extended to the studies of risk genetic factors for Alzheimer's Disease (AD) among different research institutions, integrating a group feature selection framework to rank the top risk SNPs for AD. For the unsupervised learning problem, I propose a highly efficient solver, termed Stochastic Coordinate Coding (SCC), scaling up the optimization of dictionary learning and sparse coding problems. The common issue for the medical imaging research is that the longitudinal features of patients among different time points are beneficial to study together. To further improve the dictionary learning model, I propose a multi-task dictionary learning method, learning the different task simultaneously and utilizing shared and individual dictionary to encode both consistent and changing imaging features.
ContributorsLi, Qingyang (Author) / Ye, Jieping (Thesis advisor) / Xue, Guoliang (Thesis advisor) / He, Jingrui (Committee member) / Wang, Yalin (Committee member) / Li, Jing (Committee member) / Arizona State University (Publisher)
Created2017
Description
Imaging genetics is an emerging and promising technique that investigates how genetic variations affect brain development, structure, and function. By exploiting disorder-related neuroimaging phenotypes, this class of studies provides a novel direction to reveal and understand the complex genetic mechanisms. Oftentimes, imaging genetics studies are challenging due to the relatively small number of subjects but extremely high-dimensionality of both imaging data and genomic data. In this dissertation, I carry on my research on imaging genetics with particular focuses on two tasks---building predictive models between neuroimaging data and genomic data, and identifying disorder-related genetic risk factors through image-based biomarkers. To this end, I consider a suite of structured sparse methods---that can produce interpretable models and are robust to overfitting---for imaging genetics. With carefully-designed sparse-inducing regularizers, different biological priors are incorporated into learning models. More specifically, in the Allen brain image--gene expression study, I adopt an advanced sparse coding approach for image feature extraction and employ a multi-task learning approach for multi-class annotation. Moreover, I propose a label structured-based two-stage learning framework, which utilizes the hierarchical structure among labels, for multi-label annotation. In the Alzheimer's disease neuroimaging initiative (ADNI) imaging genetics study, I employ Lasso together with EDPP (enhanced dual polytope projections) screening rules to fast identify Alzheimer's disease risk SNPs. I also adopt the tree-structured group Lasso with MLFre (multi-layer feature reduction) screening rules to incorporate linkage disequilibrium information into modeling. Moreover, I propose a novel absolute fused Lasso model for ADNI imaging genetics. This method utilizes SNP spatial structure and is robust to the choice of reference alleles of genotype coding. In addition, I propose a two-level structured sparse model that incorporates gene-level networks through a graph penalty into SNP-level model construction. Lastly, I explore a convolutional neural network approach for accurate predicting Alzheimer's disease related imaging phenotypes. Experimental results on real-world imaging genetics applications demonstrate the efficiency and effectiveness of the proposed structured sparse methods.
ContributorsYang, Tao (Author) / Ye, Jieping (Thesis advisor) / Xue, Guoliang (Thesis advisor) / He, Jingrui (Committee member) / Li, Baoxin (Committee member) / Li, Jing (Committee member) / Arizona State University (Publisher)
Created2017