Theses and Dissertations
This collection includes both ASU Theses and Dissertations, submitted by graduate students, and the Barrett, Honors College theses submitted by undergraduate students.
Displaying 1 - 2 of 2
Filtering by
- All Subjects: speech analysis
- All Subjects: vernacular
- Creators: Turaga, Pavan
Description
Audio signals, such as speech and ambient sounds convey rich information pertaining to a user’s activity, mood or intent. Enabling machines to understand this contextual information is necessary to bridge the gap in human-machine interaction. This is challenging due to its subjective nature, hence, requiring sophisticated techniques. This dissertation presents a set of computational methods, that generalize well across different conditions, for speech-based applications involving emotion recognition and keyword detection, and ambient sounds-based applications such as lifelogging.
The expression and perception of emotions varies across speakers and cultures, thus, determining features and classification methods that generalize well to different conditions is strongly desired. A latent topic models-based method is proposed to learn supra-segmental features from low-level acoustic descriptors. The derived features outperform state-of-the-art approaches over multiple databases. Cross-corpus studies are conducted to determine the ability of these features to generalize well across different databases. The proposed method is also applied to derive features from facial expressions; a multi-modal fusion overcomes the deficiencies of a speech only approach and further improves the recognition performance.
Besides affecting the acoustic properties of speech, emotions have a strong influence over speech articulation kinematics. A learning approach, which constrains a classifier trained over acoustic descriptors, to also model articulatory data is proposed here. This method requires articulatory information only during the training stage, thus overcoming the challenges inherent to large-scale data collection, while simultaneously exploiting the correlations between articulation kinematics and acoustic descriptors to improve the accuracy of emotion recognition systems.
Identifying context from ambient sounds in a lifelogging scenario requires feature extraction, segmentation and annotation techniques capable of efficiently handling long duration audio recordings; a complete framework for such applications is presented. The performance is evaluated on real world data and accompanied by a prototypical Android-based user interface.
The proposed methods are also assessed in terms of computation and implementation complexity. Software and field programmable gate array based implementations are considered for emotion recognition, while virtual platforms are used to model the complexities of lifelogging. The derived metrics are used to determine the feasibility of these methods for applications requiring real-time capabilities and low power consumption.
The expression and perception of emotions varies across speakers and cultures, thus, determining features and classification methods that generalize well to different conditions is strongly desired. A latent topic models-based method is proposed to learn supra-segmental features from low-level acoustic descriptors. The derived features outperform state-of-the-art approaches over multiple databases. Cross-corpus studies are conducted to determine the ability of these features to generalize well across different databases. The proposed method is also applied to derive features from facial expressions; a multi-modal fusion overcomes the deficiencies of a speech only approach and further improves the recognition performance.
Besides affecting the acoustic properties of speech, emotions have a strong influence over speech articulation kinematics. A learning approach, which constrains a classifier trained over acoustic descriptors, to also model articulatory data is proposed here. This method requires articulatory information only during the training stage, thus overcoming the challenges inherent to large-scale data collection, while simultaneously exploiting the correlations between articulation kinematics and acoustic descriptors to improve the accuracy of emotion recognition systems.
Identifying context from ambient sounds in a lifelogging scenario requires feature extraction, segmentation and annotation techniques capable of efficiently handling long duration audio recordings; a complete framework for such applications is presented. The performance is evaluated on real world data and accompanied by a prototypical Android-based user interface.
The proposed methods are also assessed in terms of computation and implementation complexity. Software and field programmable gate array based implementations are considered for emotion recognition, while virtual platforms are used to model the complexities of lifelogging. The derived metrics are used to determine the feasibility of these methods for applications requiring real-time capabilities and low power consumption.
ContributorsShah, Mohit (Author) / Spanias, Andreas (Thesis advisor) / Chakrabarti, Chaitali (Thesis advisor) / Berisha, Visar (Committee member) / Turaga, Pavan (Committee member) / Arizona State University (Publisher)
Created2015
Description
This thesis aims to explore the language of different bodies in the field of dance by analyzing
the habitual patterns of dancers from different backgrounds and vernaculars. Contextually,
the term habitual patterns is defined as the postures or poses that tend to re-appear,
often unintentionally, as the dancer performs improvisational dance. The focus lies in exposing
the movement vocabulary of a dancer to reveal his/her unique fingerprint.
The proposed approach for uncovering these movement patterns is to use a clustering
technique; mainly k-means. In addition to a static method of analysis, this paper uses
an online method of clustering using a streaming variant of k-means that integrates into
the flow of components that can be used in a real-time interactive dance performance. The
computational system is trained by the dancer to discover identifying patterns and therefore
it enables a feedback loop resulting in a rich exchange between dancer and machine. This
can help break a dancer’s tendency to create similar postures, explore larger kinespheric
space and invent movement beyond their current capabilities.
This paper describes a project that distinguishes itself in that it uses a custom database
that is curated for the purpose of highlighting the similarities and differences between various
movement forms. It puts particular emphasis on the process of choosing source movement
qualitatively, before the technological capture process begins.
the habitual patterns of dancers from different backgrounds and vernaculars. Contextually,
the term habitual patterns is defined as the postures or poses that tend to re-appear,
often unintentionally, as the dancer performs improvisational dance. The focus lies in exposing
the movement vocabulary of a dancer to reveal his/her unique fingerprint.
The proposed approach for uncovering these movement patterns is to use a clustering
technique; mainly k-means. In addition to a static method of analysis, this paper uses
an online method of clustering using a streaming variant of k-means that integrates into
the flow of components that can be used in a real-time interactive dance performance. The
computational system is trained by the dancer to discover identifying patterns and therefore
it enables a feedback loop resulting in a rich exchange between dancer and machine. This
can help break a dancer’s tendency to create similar postures, explore larger kinespheric
space and invent movement beyond their current capabilities.
This paper describes a project that distinguishes itself in that it uses a custom database
that is curated for the purpose of highlighting the similarities and differences between various
movement forms. It puts particular emphasis on the process of choosing source movement
qualitatively, before the technological capture process begins.
ContributorsIyengar, Varsha (Author) / Xin Wei, Sha (Thesis advisor) / Turaga, Pavan (Committee member) / Coleman, Grisha (Committee member) / Arizona State University (Publisher)
Created2016