Matching Items (28)
Filtering by
- All Subjects: deep learning
- All Subjects: Virtual Reality
- Creators: Computer Science and Engineering Program
- Resource Type: Text
Description
This paper presents work that was done to create a system capable of facial expression recognition (FER) using deep convolutional neural networks (CNNs) and test multiple configurations and methods. CNNs are able to extract powerful information about an image using multiple layers of generic feature detectors. The extracted information can be used to understand the image better through recognizing different features present within the image. Deep CNNs, however, require training sets that can be larger than a million pictures in order to fine tune their feature detectors. For the case of facial expression datasets, none of these large datasets are available. Due to this limited availability of data required to train a new CNN, the idea of using naïve domain adaptation is explored. Instead of creating and using a new CNN trained specifically to extract features related to FER, a previously trained CNN originally trained for another computer vision task is used. Work for this research involved creating a system that can run a CNN, can extract feature vectors from the CNN, and can classify these extracted features. Once this system was built, different aspects of the system were tested and tuned. These aspects include the pre-trained CNN that was used, the layer from which features were extracted, normalization used on input images, and training data for the classifier. Once properly tuned, the created system returned results more accurate than previous attempts on facial expression recognition. Based on these positive results, naïve domain adaptation is shown to successfully leverage advantages of deep CNNs for facial expression recognition.
ContributorsEusebio, Jose Miguel Ang (Author) / Panchanathan, Sethuraman (Thesis director) / McDaniel, Troy (Committee member) / Venkateswara, Hemanth (Committee member) / Computer Science and Engineering Program (Contributor) / Barrett, The Honors College (Contributor)
Created2016-05
Description
In this experiment, a haptic glove with vibratory motors on the fingertips was tested against the standard HTC Vive controller to see if the additional vibrations provided by the glove increased immersion in common gaming scenarios where haptic feedback is provided. Specifically, two scenarios were developed: an explosion scene containing a small and large explosion and a box interaction scene that allowed the participants to touch the box virtually with their hand. At the start of this project, it was hypothesized that the haptic glove would have a significant positive impact in at least one of these scenarios. Nine participants took place in the study and immersion was measured through a post-experiment questionnaire. Statistical analysis on the results showed that the haptic glove did have a significant impact on immersion in the box interaction scene, but not in the explosion scene. In the end, I conclude that since this haptic glove does not significantly increase immersion across all scenarios when compared to the standard Vive controller, it should not be used at a replacement in its current state.
ContributorsGriffieth, Alan P (Author) / McDaniel, Troy (Thesis director) / Selgrad, Justin (Committee member) / Computing and Informatics Program (Contributor) / School of Mathematical and Statistical Sciences (Contributor) / Economics Program in CLAS (Contributor) / Computer Science and Engineering Program (Contributor) / Barrett, The Honors College (Contributor)
Created2021-05
Description
This thesis is based on bringing together three different components: non-Euclidean geometric worlds, virtual reality, and environmental puzzles in video games. While all three exist in their own right in the world of video games, as well as combined in pairs, there are virtually no examples of all three together. Non-Euclidean environmental puzzle games have existed for around 10 years in various forms, short environmental puzzle games in virtual reality have come into existence in around the past five years, and non-Euclidean virtual reality exists mainly as non-video game short demos from the past few years. This project seeks to be able to bring these components together to create a proof of concept for how a game like this should function, particularly the integration of non-Euclidean virtual reality in the context of a video game. To do this, a Unity package which uses a custom system for creating worlds in a non-Euclidean way rather than Unity’s built-in components such as for transforms, collisions, and rendering was used. This was used in conjunction with the SteamVR implementation with Unity to create a cohesive and immersive player experience.
ContributorsVerhagen, Daniel William (Author) / Kobayashi, Yoshihiro (Thesis director) / Nelson, Brian (Committee member) / Computer Science and Engineering Program (Contributor, Contributor) / Barrett, The Honors College (Contributor)
Created2021-05
Description
This project produced a dual-medium (traditional screen & virtual reality) virtual environment of Barnhardt Canyon, in Payson, Arizona. The project showcases two different approaches to developing a virtual environment with both being centered by 360 degree content. The virtual environment allows a user to explore the area in a much more immersive way than offered by traditional media. Future uses of the project could include research on the educational efficacy of virtual reality content, or the project could be used as a teaching tool in geoscience classes.
ContributorsRuberto, James Richard (Author) / Semken, Steven (Thesis director) / Reynolds, Stephen (Committee member) / Proctor, Sian (Committee member) / Computer Science and Engineering Program (Contributor, Contributor) / Barrett, The Honors College (Contributor)
Created2020-12
Description
Emotion recognition in conversation has applications within numerous domains such as affective computing and medicine. Recent methods for emotion recognition jointly utilize conversational data over several modalities including audio, video, and text. However, state-of-the-art frameworks for this task do not focus on the feature extraction and feature fusion steps of this process. This thesis aims to improve the state-of-the-art method by incorporating two components to better accomplish these steps. By doing so, we are able to produce improved representations for the text modality and better model the relationships between all modalities. This paper proposes two methods which focus on these concepts and provide improved accuracy over the state-of-the-art framework for multimodal emotion recognition in dialogue.
ContributorsRawal, Siddharth (Author) / Baral, Chitta (Thesis director) / Shah, Shrikant (Committee member) / Computer Science and Engineering Program (Contributor) / Barrett, The Honors College (Contributor)
Created2020-05
Description
Accurate pose initialization and pose estimation are crucial requirements in on-orbit space assembly and various other autonomous on-orbit tasks. However, pose initialization and pose estimation are much more difficult to do accurately and consistently in space. This is primarily due to not only the variable lighting conditions present in space, but also the power requirements mandated by space-flyable hardware. This thesis investigates leveraging a deep learning approach for monocular one-shot pose initialization and pose estimation. A convolutional neural network was used to estimate the 6D pose of an assembly truss object. This network was trained by utilizing synthetic imagery generated from a simulation testbed. Furthermore, techniques to quantify model uncertainty of the deep learning model were investigated and applied in the task of in-space pose estimation and pose initialization. The feasibility of this approach on low-power computational platforms was also tested. The results demonstrate that accurate pose initialization and pose estimation can be conducted using a convolutional neural network. In addition, the results show that the model uncertainty can be obtained from the network. Lastly, the use of deep learning for pose initialization and pose estimation in addition with uncertainty quantification was demonstrated to be feasible on low-power compute platforms.
ContributorsKailas, Siva Maneparambil (Author) / Ben Amor, Heni (Thesis director) / Detry, Renaud (Committee member) / Economics Program in CLAS (Contributor) / School of Mathematical and Statistical Sciences (Contributor) / Computer Science and Engineering Program (Contributor) / Barrett, The Honors College (Contributor)
Created2020-05
Description
Spotify, one of the most popular music streaming services, has many
algorithms for recommending new music to users. However, at the
core of their recommendations is the collaborative filtering algorithm,
which recommends music based on what other people with similar
tastes have listened to [1]. While this can produce highly relevant
content recommendations, it tends to promote only popular content
[2]. The popularity bias inherent in collaborative-filtering based
systems can overlook music that fits a user’s taste, simply because
nobody else is listening to it. One possible solution to this problem is
to recommend music based on features of the music itself, and
recommend songs which have similar features. Here, a method for
extracting high-level features representing the mood of a song is
presented, with the aim of tailoring music recommendations to an
individual's mood, and providing music recommendations with
diversity in popularity.
algorithms for recommending new music to users. However, at the
core of their recommendations is the collaborative filtering algorithm,
which recommends music based on what other people with similar
tastes have listened to [1]. While this can produce highly relevant
content recommendations, it tends to promote only popular content
[2]. The popularity bias inherent in collaborative-filtering based
systems can overlook music that fits a user’s taste, simply because
nobody else is listening to it. One possible solution to this problem is
to recommend music based on features of the music itself, and
recommend songs which have similar features. Here, a method for
extracting high-level features representing the mood of a song is
presented, with the aim of tailoring music recommendations to an
individual's mood, and providing music recommendations with
diversity in popularity.
ContributorsGomez, Luis Angel (Author) / Kevin, Burger (Thesis director) / Alberto, Hernández (Committee member) / Arts, Media and Engineering Sch T (Contributor) / Computer Science and Engineering Program (Contributor) / Barrett, The Honors College (Contributor)
Created2020-05
Description
In this project, I investigated the impact of virtual reality on memory retention. The investigative approach to see the impact of virtual reality on memory retention, I utilized the memorization technique called the memory palace in a virtual reality environment. For the experiment, due to Covid-19, I was forced to be the only subject. To get effective data, I tested myself within randomly generated environments with a completely unique set of objects, both outside of a virtual reality environment and within one. First I conducted a set of 10 tests on myself by going through a virtual environment on my laptop and recalling as many objects I could within that environment. I recorded the accuracy of my own recollection as well as how long it took me to get through the data. Next I conducted a set of 10 tests on myself by going through the same virtual environment, but this time with an immersive virtual reality(VR) headset and a completely new set of objects. At the start of the project it was hypothesized that virtual reality would result in a higher memory retention rate versus simply going through the environment in a non-immersive environment. In the end, the results, albeit with a low test rate, leaned more toward showing the hypothesis to be true rather than not.
ContributorsDu, Michael Shan (Author) / Kobayashi, Yoshihiro (Thesis director) / McDaniel, Troy (Committee member) / Computer Science and Engineering Program (Contributor) / Barrett, The Honors College (Contributor)
Created2020-05
Description
In recent years, the development of new Machine Learning models has allowed for new technological advancements to be introduced for practical use across the world. Multiple studies and experiments have been conducted to create new variations of Machine Learning models with different algorithms to determine if potential systems would prove to be successful. Even today, there are still many research initiatives that are continuing to develop new models in the hopes to discover potential solutions for problems such as autonomous driving or determining the emotional value from a single sentence. One of the current popular research topics for Machine Learning is the development of Facial Expression Recognition systems. These Machine Learning models focus on classifying images of human faces that are expressing different emotions through facial expressions. In order to develop effective models to perform Facial Expression Recognition, researchers have gone on to utilize Deep Learning models, which are a more advanced implementation of Machine Learning models, known as Neural Networks. More specifically, the use of Convolutional Neural Networks has proven to be the most effective models for achieving highly accurate results at classifying images of various facial expressions. Convolutional Neural Networks are Deep Learning models that are capable of processing visual data, such as images and videos, and can be used to identify various facial expressions. The purpose of this project, I focused on learning about the important concepts of Machine Learning, Deep Learning, and Convolutional Neural Networks to implement a Convolutional Neural Network that was previously developed by a recommended research paper.
ContributorsFrace, Douglas R (Author) / Demakethepalli Venkateswara, Hemanth Kumar (Thesis director) / McDaniel, Troy (Committee member) / Computer Science and Engineering Program (Contributor) / Barrett, The Honors College (Contributor)
Created2020-05
Description
Immunotherapy is an effective treatment for cancer which enables the patient's immune system to recognize tumor cells as pathogens. In order to design an individualized treatment, the t cell receptors (TCR) which bind to a tumor's unique antigens need to be determined. We created a convolutional neural network to predict the binding affinity between a given TCR and antigen to enable this.
ContributorsCai, Michael Ray (Author) / Lee, Heewook (Thesis director) / Meuth, Ryan (Committee member) / Computer Science and Engineering Program (Contributor, Contributor) / Barrett, The Honors College (Contributor)
Created2020-12