Matching Items (43)
Filtering by
- Genre: Doctoral Dissertation
Description
The presence of strategic agents can pose unique challenges to data collection and distributed learning. This dissertation first explores the social network dimension of data collection markets, and then focuses on how the strategic agents can be efficiently and effectively incentivized to cooperate in distributed machine learning frameworks.
The first problem explores the impact of social learning in collecting and trading unverifiable information where a data collector purchases data from users through a payment mechanism. Each user starts with a personal signal which represents the knowledge about the underlying state the data collector desires to learn. Through social interactions, each user also acquires additional information from his neighbors in the social network. It is revealed that both the data collector and the users can benefit from social learning which drives down the privacy costs and helps to improve the state estimation for a given total payment budget. In the second half, a federated learning scheme to train a global learning model with strategic agents, who are not bound to contribute their resources unconditionally, is considered. Since the agents are not obliged to provide their true stochastic gradient updates and the server is not capable of directly validating the authenticity of reported updates, the learning process may reach a noncooperative equilibrium. First, the actions of the agents are assumed to be binary: cooperative or defective. If the cooperative action is taken, the agent sends a privacy-preserved version of stochastic gradient signal. If the defective action is taken, the agent sends an arbitrary uninformative noise signal. Furthermore, this setup is extended into the scenarios with more general actions spaces where the quality of the stochastic gradient updates have a range of discrete levels. The proposed methodology evaluates each agent's stochastic gradient according to a reference gradient estimate which is constructed from the gradients provided by other agents, and rewards the agent based on that evaluation.
ContributorsAkbay, Abdullah Basar (Author) / Tepedelenlioğlu, Cihan (Thesis advisor) / Spanias, Andreas (Committee member) / Kosut, Oliver (Committee member) / Ewaisha, Ahmed (Committee member) / Arizona State University (Publisher)
Created2023
Description
The past decade witnessed the success of deep learning models in various applications of computer vision and natural language processing. This success can be predominantly attributed to the (i) availability of large amounts of training data; (ii) access of domain aware knowledge; (iii) i.i.d assumption between the train and target distributions and (iv) belief on existing metrics as reliable indicators of performance. When any of these assumptions are violated, the models exhibit brittleness producing adversely varied behavior. This dissertation focuses on methods for accurate model design and characterization that enhance process reliability when certain assumptions are not met. With the need to safely adopt artificial intelligence tools in practice, it is vital to build reliable failure detectors that indicate regimes where the model must not be invoked. To that end, an error predictor trained with a self-calibration objective is developed to estimate loss consistent with the underlying model. The properties of the error predictor are described and their utility in supporting introspection via feature importances and counterfactual explanations is elucidated. While such an approach can signal data regime changes, it is critical to calibrate models using regimes of inlier (training) and outlier data to prevent under- and over-generalization in models i.e., incorrectly identifying inliers as outliers and vice-versa. By identifying the space for specifying inliers and outliers, an anomaly detector that can effectively flag data of varying semantic complexities in medical imaging is next developed. Uncertainty quantification in deep learning models involves identifying sources of failure and characterizing model confidence to enable actionability. A training strategy is developed that allows the accurate estimation of model uncertainties and its benefits are demonstrated for active learning and generalization gap prediction. This helps identify insufficiently sampled regimes and representation insufficiency in models. In addition, the task of deep inversion under data scarce scenarios is considered, which in practice requires a prior to control the optimization. By identifying limitations in existing work, data priors powered by generative models and deep model priors are designed for audio restoration. With relevant empirical studies on a variety of benchmarks, the need for such design strategies is demonstrated.
ContributorsNarayanaswamy, Vivek Sivaraman (Author) / Spanias, Andreas (Thesis advisor) / J. Thiagarajan, Jayaraman (Committee member) / Berisha, Visar (Committee member) / Tepedelenlioğlu, Cihan (Committee member) / Arizona State University (Publisher)
Created2023
Description
Modern machine learning systems leverage data and features from multiple modalities to gain more predictive power. In most scenarios, the modalities are vastly different and the acquired data are heterogeneous in nature. Consequently, building highly effective fusion algorithms is at the core to achieve improved model robustness and inferencing performance. This dissertation focuses on the representation learning approaches as the fusion strategy. Specifically, the objective is to learn the shared latent representation which jointly exploit the structural information encoded in all modalities, such that a straightforward learning model can be adopted to obtain the prediction.
We first consider sensor fusion, a typical multimodal fusion problem critical to building a pervasive computing platform. A systematic fusion technique is described to support both multiple sensors and descriptors for activity recognition. Targeted to learn the optimal combination of kernels, Multiple Kernel Learning (MKL) algorithms have been successfully applied to numerous fusion problems in computer vision etc. Utilizing the MKL formulation, next we describe an auto-context algorithm for learning image context via the fusion with low-level descriptors. Furthermore, a principled fusion algorithm using deep learning to optimize kernel machines is developed. By bridging deep architectures with kernel optimization, this approach leverages the benefits of both paradigms and is applied to a wide variety of fusion problems.
In many real-world applications, the modalities exhibit highly specific data structures, such as time sequences and graphs, and consequently, special design of the learning architecture is needed. In order to improve the temporal modeling for multivariate sequences, we developed two architectures centered around attention models. A novel clinical time series analysis model is proposed for several critical problems in healthcare. Another model coupled with triplet ranking loss as metric learning framework is described to better solve speaker diarization. Compared to state-of-the-art recurrent networks, these attention-based multivariate analysis tools achieve improved performance while having a lower computational complexity. Finally, in order to perform community detection on multilayer graphs, a fusion algorithm is described to derive node embedding from word embedding techniques and also exploit the complementary relational information contained in each layer of the graph.
We first consider sensor fusion, a typical multimodal fusion problem critical to building a pervasive computing platform. A systematic fusion technique is described to support both multiple sensors and descriptors for activity recognition. Targeted to learn the optimal combination of kernels, Multiple Kernel Learning (MKL) algorithms have been successfully applied to numerous fusion problems in computer vision etc. Utilizing the MKL formulation, next we describe an auto-context algorithm for learning image context via the fusion with low-level descriptors. Furthermore, a principled fusion algorithm using deep learning to optimize kernel machines is developed. By bridging deep architectures with kernel optimization, this approach leverages the benefits of both paradigms and is applied to a wide variety of fusion problems.
In many real-world applications, the modalities exhibit highly specific data structures, such as time sequences and graphs, and consequently, special design of the learning architecture is needed. In order to improve the temporal modeling for multivariate sequences, we developed two architectures centered around attention models. A novel clinical time series analysis model is proposed for several critical problems in healthcare. Another model coupled with triplet ranking loss as metric learning framework is described to better solve speaker diarization. Compared to state-of-the-art recurrent networks, these attention-based multivariate analysis tools achieve improved performance while having a lower computational complexity. Finally, in order to perform community detection on multilayer graphs, a fusion algorithm is described to derive node embedding from word embedding techniques and also exploit the complementary relational information contained in each layer of the graph.
ContributorsSong, Huan (Author) / Spanias, Andreas (Thesis advisor) / Thiagarajan, Jayaraman (Committee member) / Berisha, Visar (Committee member) / Tepedelenlioğlu, Cihan (Committee member) / Arizona State University (Publisher)
Created2018
Description
Human movement is a complex process influenced by physiological and psychological factors. The execution of movement is varied from person to person, and the number of possible strategies for completing a specific movement task is almost infinite. Different choices of strategies can be perceived by humans as having different degrees of quality, and the quality can be defined with regard to aesthetic, athletic, or health-related ratings. It is useful to measure and track the quality of a person's movements, for various applications, especially with the prevalence of low-cost and portable cameras and sensors today. Furthermore, based on such measurements, feedback systems can be designed for people to practice their movements towards certain goals. In this dissertation, I introduce symmetry as a family of measures for movement quality, and utilize recent advances in computer vision and differential geometry to model and analyze different types of symmetry in human movements. Movements are modeled as trajectories on different types of manifolds, according to the representations of movements from sensor data. The benefit of such a universal framework is that it can accommodate different existing and future features that describe human movements. The theory and tools developed in this dissertation will also be useful in other scientific areas to analyze symmetry from high-dimensional signals.
ContributorsWang, Qiao (Author) / Turaga, Pavan (Thesis advisor) / Spanias, Andreas (Committee member) / Srivastava, Anuj (Committee member) / Sha, Xin Wei (Committee member) / Arizona State University (Publisher)
Created2018
Description
The research on the topology and dynamics of complex networks is one of the most focused area in complex system science. The goals are to structure our understanding of the real-world social, economical, technological, and biological systems in the aspect of networks consisting a large number of interacting units and to develop corresponding detection, prediction, and control strategies. In this highly interdisciplinary field, my research mainly concentrates on universal estimation schemes, physical controllability, as well as mechanisms behind extreme events and cascading failure for complex networked systems.
Revealing the underlying structure and dynamics of complex networked systems from observed data without of any specific prior information is of fundamental importance to science, engineering, and society. We articulate a Markov network based model, the sparse dynamical Boltzmann machine (SDBM), as a universal network structural estimator and dynamics approximator based on techniques including compressive sensing and K-means algorithm. It recovers the network structure of the original system and predicts its short-term or even long-term dynamical behavior for a large variety of representative dynamical processes on model and real-world complex networks.
One of the most challenging problems in complex dynamical systems is to control complex networks.
Upon finding that the energy required to approach a target state with reasonable precision
is often unbearably large, and the energy of controlling a set of networks with similar structural properties follows a fat-tail distribution, we identify fundamental structural ``short boards'' that play a dominant role in the enormous energy and offer a theoretical interpretation for the fat-tail distribution and simple strategies to significantly reduce the energy.
Extreme events and cascading failure, a type of collective behavior in complex networked systems, often have catastrophic consequences. Utilizing transportation and evolutionary game dynamics as prototypical
settings, we investigate the emergence of extreme events in simplex complex networks, mobile ad-hoc networks and multi-layer interdependent networks. A striking resonance-like phenomenon and the emergence of global-scale cascading breakdown are discovered. We derive analytic theories to understand the mechanism of
control at a quantitative level and articulate cost-effective control schemes to significantly suppress extreme events and the cascading process.
Revealing the underlying structure and dynamics of complex networked systems from observed data without of any specific prior information is of fundamental importance to science, engineering, and society. We articulate a Markov network based model, the sparse dynamical Boltzmann machine (SDBM), as a universal network structural estimator and dynamics approximator based on techniques including compressive sensing and K-means algorithm. It recovers the network structure of the original system and predicts its short-term or even long-term dynamical behavior for a large variety of representative dynamical processes on model and real-world complex networks.
One of the most challenging problems in complex dynamical systems is to control complex networks.
Upon finding that the energy required to approach a target state with reasonable precision
is often unbearably large, and the energy of controlling a set of networks with similar structural properties follows a fat-tail distribution, we identify fundamental structural ``short boards'' that play a dominant role in the enormous energy and offer a theoretical interpretation for the fat-tail distribution and simple strategies to significantly reduce the energy.
Extreme events and cascading failure, a type of collective behavior in complex networked systems, often have catastrophic consequences. Utilizing transportation and evolutionary game dynamics as prototypical
settings, we investigate the emergence of extreme events in simplex complex networks, mobile ad-hoc networks and multi-layer interdependent networks. A striking resonance-like phenomenon and the emergence of global-scale cascading breakdown are discovered. We derive analytic theories to understand the mechanism of
control at a quantitative level and articulate cost-effective control schemes to significantly suppress extreme events and the cascading process.
ContributorsChen, Yuzhong (Author) / Lai, Ying-Cheng (Thesis advisor) / Spanias, Andreas (Committee member) / Tepedelenlioğlu, Cihan (Committee member) / Ying, Lei (Committee member) / Arizona State University (Publisher)
Created2016
Description
Buck converters are electronic devices that changes a voltage from one level to a lower one and are present in many everyday applications. However, due to factors like aging, degradation or failures, these devices require a system identification process to track and diagnose their parameters. The system identification process should be performed on-line to not affect the normal operation of the device. Identifying the parameters of the system is essential to design and tune an adaptive proportional-integral-derivative (PID) controller.
Three techniques were used to design the PID controller. Phase and gain margin still prevails as one of the easiest methods to design controllers. Pole-zero cancellation is another technique which is based on pole-placement. However, although these controllers can be easily designed, they did not provide the best response compared to the Frequency Loop Shaping (FLS) technique. Therefore, since FLS showed to have a better frequency and time responses compared to the other two controllers, it was selected to perform the adaptation of the system.
An on-line system identification process was performed for the buck converter using indirect adaptation and the least square algorithm. The estimation error and the parameter error were computed to determine the rate of convergence of the system. The indirect adaptation required about 2000 points to converge to the true parameters prior designing the controller. These results were compared to the adaptation executed using robust stability condition (RSC) and a switching controller. Two different scenarios were studied consisting of five plants that defined the percentage of deterioration of the capacitor and inductor within the buck converter. The switching logic did not always select the optimal controller for the first scenario because the frequency response of the different plants was not significantly different. However, the second scenario consisted of plants with more noticeable different frequency responses and the switching logic selected the optimal controller all the time in about 500 points. Additionally, a disturbance was introduced at the plant input to observe its effect in the switching controller. However, for reasonable low disturbances no change was detected in the proper selection of controllers.
Three techniques were used to design the PID controller. Phase and gain margin still prevails as one of the easiest methods to design controllers. Pole-zero cancellation is another technique which is based on pole-placement. However, although these controllers can be easily designed, they did not provide the best response compared to the Frequency Loop Shaping (FLS) technique. Therefore, since FLS showed to have a better frequency and time responses compared to the other two controllers, it was selected to perform the adaptation of the system.
An on-line system identification process was performed for the buck converter using indirect adaptation and the least square algorithm. The estimation error and the parameter error were computed to determine the rate of convergence of the system. The indirect adaptation required about 2000 points to converge to the true parameters prior designing the controller. These results were compared to the adaptation executed using robust stability condition (RSC) and a switching controller. Two different scenarios were studied consisting of five plants that defined the percentage of deterioration of the capacitor and inductor within the buck converter. The switching logic did not always select the optimal controller for the first scenario because the frequency response of the different plants was not significantly different. However, the second scenario consisted of plants with more noticeable different frequency responses and the switching logic selected the optimal controller all the time in about 500 points. Additionally, a disturbance was introduced at the plant input to observe its effect in the switching controller. However, for reasonable low disturbances no change was detected in the proper selection of controllers.
ContributorsSerrano Rodriguez, Victoria Melissa (Author) / Tsakalis, Konstantinos (Thesis advisor) / Bakkaloglu, Bertan (Thesis advisor) / Rodriguez, Armando (Committee member) / Spanias, Andreas (Committee member) / Arizona State University (Publisher)
Created2016
Description
In many applications, measured sensor data is meaningful only when the location of sensors is accurately known. Therefore, the localization accuracy is crucial. In this dissertation, both location estimation and location detection problems are considered.
In location estimation problems, sensor nodes at known locations, called anchors, transmit signals to sensor nodes at unknown locations, called nodes, and use these transmissions to estimate the location of the nodes. Specifically, the location estimation in the presence of fading channels using time of arrival (TOA) measurements with narrowband communication signals is considered. Meanwhile, the Cramer-Rao lower bound (CRLB) for localization error under different assumptions is derived. Also, maximum likelihood estimators (MLEs) under these assumptions are derived.
In large WSNs, distributed location estimation algorithms are more efficient than centralized algorithms. A sequential localization scheme, which is one of distributed location estimation algorithms, is considered. Also, different localization methods, such as TOA, received signal strength (RSS), time difference of arrival (TDOA), direction of arrival (DOA), and large aperture array (LAA) are compared under different signal-to-noise ratio (SNR) conditions. Simulation results show that DOA is the preferred scheme at the low SNR regime and the LAA localization algorithm provides better performance for network discovery at high SNRs. Meanwhile, the CRLB for the localization error using the TOA method is also derived.
A distributed location detection scheme, which allows each anchor to make a decision as to whether a node is active or not is proposed. Once an anchor makes a decision, a bit is transmitted to a fusion center (FC). The fusion center combines all the decisions and uses a design parameter $K$ to make the final decision. Three scenarios are considered in this dissertation. Firstly, location detection at a known location is considered. Secondly, detecting a node in a known region is considered. Thirdly, location detection in the presence of fading is considered. The optimal thresholds are derived and the total probability of false alarm and detection under different scenarios are derived.
In location estimation problems, sensor nodes at known locations, called anchors, transmit signals to sensor nodes at unknown locations, called nodes, and use these transmissions to estimate the location of the nodes. Specifically, the location estimation in the presence of fading channels using time of arrival (TOA) measurements with narrowband communication signals is considered. Meanwhile, the Cramer-Rao lower bound (CRLB) for localization error under different assumptions is derived. Also, maximum likelihood estimators (MLEs) under these assumptions are derived.
In large WSNs, distributed location estimation algorithms are more efficient than centralized algorithms. A sequential localization scheme, which is one of distributed location estimation algorithms, is considered. Also, different localization methods, such as TOA, received signal strength (RSS), time difference of arrival (TDOA), direction of arrival (DOA), and large aperture array (LAA) are compared under different signal-to-noise ratio (SNR) conditions. Simulation results show that DOA is the preferred scheme at the low SNR regime and the LAA localization algorithm provides better performance for network discovery at high SNRs. Meanwhile, the CRLB for the localization error using the TOA method is also derived.
A distributed location detection scheme, which allows each anchor to make a decision as to whether a node is active or not is proposed. Once an anchor makes a decision, a bit is transmitted to a fusion center (FC). The fusion center combines all the decisions and uses a design parameter $K$ to make the final decision. Three scenarios are considered in this dissertation. Firstly, location detection at a known location is considered. Secondly, detecting a node in a known region is considered. Thirdly, location detection in the presence of fading is considered. The optimal thresholds are derived and the total probability of false alarm and detection under different scenarios are derived.
ContributorsZhang, Xue (Author) / Tepedelenlioğlu, Cihan (Thesis advisor) / Spanias, Andreas (Thesis advisor) / Tsakalis, Konstantinos (Committee member) / Berisha, Visar (Committee member) / Arizona State University (Publisher)
Created2016
Description
From time immemorial, epilepsy has persisted to be one of the greatest impediments to human life for those stricken by it. As the fourth most common neurological disorder, epilepsy causes paroxysmal electrical discharges in the brain that manifest as seizures. Seizures have the effect of debilitating patients on a physical and psychological level. Although not lethal by themselves, they can bring about total disruption in consciousness which can, in hazardous conditions, lead to fatality. Roughly 1\% of the world population suffer from epilepsy and another 30 to 50 new cases per 100,000 increase the number of affected annually. Controlling seizures in epileptic patients has therefore become a great medical and, in recent years, engineering challenge.
In this study, the conditions of human seizures are recreated in an animal model of temporal lobe epilepsy. The rodents used in this study are chemically induced to become chronically epileptic. Their Electroencephalogram (EEG) data is then recorded and analyzed to detect and predict seizures; with the ultimate goal being the control and complete suppression of seizures.
Two methods, the maximum Lyapunov exponent and the Generalized Partial Directed Coherence (GPDC), are applied on EEG data to extract meaningful information. Their effectiveness have been reported in the literature for the purpose of prediction of seizures and seizure focus localization. This study integrates these measures, through some modifications, to robustly detect seizures and separately find precursors to them and in consequence provide stimulation to the epileptic brain of rats in order to suppress seizures. Additionally open-loop stimulation with biphasic currents of various pairs of sites in differing lengths of time have helped us create control efficacy maps. While GPDC tells us about the possible location of the focus, control efficacy maps tells us how effective stimulating a certain pair of sites will be.
The results from computations performed on the data are presented and the feasibility of the control problem is discussed. The results show a new reliable means of seizure detection even in the presence of artifacts in the data. The seizure precursors provide a means of prediction, in the order of tens of minutes, prior to seizures. Closed loop stimulation experiments based on these precursors and control efficacy maps on the epileptic animals show a maximum reduction of seizure frequency by 24.26\% in one animal and reduction of length of seizures by 51.77\% in another. Thus, through this study it was shown that the implementation of the methods can ameliorate seizures in an epileptic patient. It is expected that the new knowledge and experimental techniques will provide a guide for future research in an effort to ultimately eliminate seizures in epileptic patients.
In this study, the conditions of human seizures are recreated in an animal model of temporal lobe epilepsy. The rodents used in this study are chemically induced to become chronically epileptic. Their Electroencephalogram (EEG) data is then recorded and analyzed to detect and predict seizures; with the ultimate goal being the control and complete suppression of seizures.
Two methods, the maximum Lyapunov exponent and the Generalized Partial Directed Coherence (GPDC), are applied on EEG data to extract meaningful information. Their effectiveness have been reported in the literature for the purpose of prediction of seizures and seizure focus localization. This study integrates these measures, through some modifications, to robustly detect seizures and separately find precursors to them and in consequence provide stimulation to the epileptic brain of rats in order to suppress seizures. Additionally open-loop stimulation with biphasic currents of various pairs of sites in differing lengths of time have helped us create control efficacy maps. While GPDC tells us about the possible location of the focus, control efficacy maps tells us how effective stimulating a certain pair of sites will be.
The results from computations performed on the data are presented and the feasibility of the control problem is discussed. The results show a new reliable means of seizure detection even in the presence of artifacts in the data. The seizure precursors provide a means of prediction, in the order of tens of minutes, prior to seizures. Closed loop stimulation experiments based on these precursors and control efficacy maps on the epileptic animals show a maximum reduction of seizure frequency by 24.26\% in one animal and reduction of length of seizures by 51.77\% in another. Thus, through this study it was shown that the implementation of the methods can ameliorate seizures in an epileptic patient. It is expected that the new knowledge and experimental techniques will provide a guide for future research in an effort to ultimately eliminate seizures in epileptic patients.
ContributorsShafique, Md Ashfaque Bin (Author) / Tsakalis, Konstantinos (Thesis advisor) / Rodriguez, Armando (Committee member) / Muthuswamy, Jitendran (Committee member) / Spanias, Andreas (Committee member) / Arizona State University (Publisher)
Created2016
Description
Recently, the location of the nodes in wireless networks has been modeled as point processes. In this dissertation, various scenarios of wireless communications in large-scale networks modeled as point processes are considered. The first part of the dissertation considers signal reception and detection problems with symmetric alpha stable noise which is from an interfering network modeled as a Poisson point process. For the signal reception problem, the performance of space-time coding (STC) over fading channels with alpha stable noise is studied. We derive pairwise error probability (PEP) of orthogonal STCs. For general STCs, we propose a maximum-likelihood (ML) receiver, and its approximation. The resulting asymptotically optimal receiver (AOR) does not depend on noise parameters and is computationally simple, and close to the ML performance. Then, signal detection in coexisting wireless sensor networks (WSNs) is considered. We define a binary hypothesis testing problem for the signal detection in coexisting WSNs. For the problem, we introduce the ML detector and simpler alternatives. The proposed mixed-fractional lower order moment (FLOM) detector is computationally simple and close to the ML performance. Stochastic orders are binary relations defined on probability. The second part of the dissertation introduces stochastic ordering of interferences in large-scale networks modeled as point processes. Since closed-form results for the interference distributions for such networks are only available in limited cases, it is of interest to compare network interferences using stochastic. In this dissertation, conditions on the fading distribution and path-loss model are given to establish stochastic ordering between interferences. Moreover, Laplace functional (LF) ordering is defined between point processes and applied for comparing interference. Then, the LF orderings of general classes of point processes are introduced. It is also shown that the LF ordering is preserved when independent operations such as marking, thinning, random translation, and superposition are applied. The LF ordering of point processes is a useful tool for comparing spatial deployments of wireless networks and can be used to establish comparisons of several performance metrics such as coverage probability, achievable rate, and resource allocation even when closed form expressions for such metrics are unavailable.
ContributorsLee, Junghoon (Author) / Tepedelenlioğlu, Cihan (Thesis advisor) / Spanias, Andreas (Committee member) / Reisslein, Martin (Committee member) / Kosut, Oliver (Committee member) / Arizona State University (Publisher)
Created2014
Description
Audio signals, such as speech and ambient sounds convey rich information pertaining to a user’s activity, mood or intent. Enabling machines to understand this contextual information is necessary to bridge the gap in human-machine interaction. This is challenging due to its subjective nature, hence, requiring sophisticated techniques. This dissertation presents a set of computational methods, that generalize well across different conditions, for speech-based applications involving emotion recognition and keyword detection, and ambient sounds-based applications such as lifelogging.
The expression and perception of emotions varies across speakers and cultures, thus, determining features and classification methods that generalize well to different conditions is strongly desired. A latent topic models-based method is proposed to learn supra-segmental features from low-level acoustic descriptors. The derived features outperform state-of-the-art approaches over multiple databases. Cross-corpus studies are conducted to determine the ability of these features to generalize well across different databases. The proposed method is also applied to derive features from facial expressions; a multi-modal fusion overcomes the deficiencies of a speech only approach and further improves the recognition performance.
Besides affecting the acoustic properties of speech, emotions have a strong influence over speech articulation kinematics. A learning approach, which constrains a classifier trained over acoustic descriptors, to also model articulatory data is proposed here. This method requires articulatory information only during the training stage, thus overcoming the challenges inherent to large-scale data collection, while simultaneously exploiting the correlations between articulation kinematics and acoustic descriptors to improve the accuracy of emotion recognition systems.
Identifying context from ambient sounds in a lifelogging scenario requires feature extraction, segmentation and annotation techniques capable of efficiently handling long duration audio recordings; a complete framework for such applications is presented. The performance is evaluated on real world data and accompanied by a prototypical Android-based user interface.
The proposed methods are also assessed in terms of computation and implementation complexity. Software and field programmable gate array based implementations are considered for emotion recognition, while virtual platforms are used to model the complexities of lifelogging. The derived metrics are used to determine the feasibility of these methods for applications requiring real-time capabilities and low power consumption.
The expression and perception of emotions varies across speakers and cultures, thus, determining features and classification methods that generalize well to different conditions is strongly desired. A latent topic models-based method is proposed to learn supra-segmental features from low-level acoustic descriptors. The derived features outperform state-of-the-art approaches over multiple databases. Cross-corpus studies are conducted to determine the ability of these features to generalize well across different databases. The proposed method is also applied to derive features from facial expressions; a multi-modal fusion overcomes the deficiencies of a speech only approach and further improves the recognition performance.
Besides affecting the acoustic properties of speech, emotions have a strong influence over speech articulation kinematics. A learning approach, which constrains a classifier trained over acoustic descriptors, to also model articulatory data is proposed here. This method requires articulatory information only during the training stage, thus overcoming the challenges inherent to large-scale data collection, while simultaneously exploiting the correlations between articulation kinematics and acoustic descriptors to improve the accuracy of emotion recognition systems.
Identifying context from ambient sounds in a lifelogging scenario requires feature extraction, segmentation and annotation techniques capable of efficiently handling long duration audio recordings; a complete framework for such applications is presented. The performance is evaluated on real world data and accompanied by a prototypical Android-based user interface.
The proposed methods are also assessed in terms of computation and implementation complexity. Software and field programmable gate array based implementations are considered for emotion recognition, while virtual platforms are used to model the complexities of lifelogging. The derived metrics are used to determine the feasibility of these methods for applications requiring real-time capabilities and low power consumption.
ContributorsShah, Mohit (Author) / Spanias, Andreas (Thesis advisor) / Chakrabarti, Chaitali (Thesis advisor) / Berisha, Visar (Committee member) / Turaga, Pavan (Committee member) / Arizona State University (Publisher)
Created2015