Matching Items (119)
Filtering by
Description
In recent years, brain signals have gained attention as a potential trait for biometric-based security systems and laboratory systems have been designed. A real-world brain-based security system requires to be usable, accurate, and robust. While there have been developments in these aspects, there are still challenges to be met. With regard to usability, users need to provide lengthy amount of data compared to other traits such as fingerprint and face to get authenticated. Furthermore, in the majority of works, medical sensors are used which are more accurate compared to commercial ones but have a tedious setup process and are not mobile. Performance wise, the current state-of-art can provide acceptable accuracy on a small pool of users data collected in few sessions close to each other but still falls behind on a large pool of subjects over a longer time period. Finally, a brain security system should be robust against presentation attacks to prevent adversaries from gaining access to the system.
This dissertation proposes E-BIAS (EEG-based Identification and Authentication System), a brain-mobile security system that makes contributions in three directions. First, it provides high performance on signals with shorter lengths collected by commercial sensors and processed with lightweight models to meet the computation/energy capacity of mobile devices. Second, to evaluate the system's robustness a novel presentation attack was designed which challenged the literature's presumption of intrinsic liveness property for brain signals. Third, to bridge the gap, I formulated and studied the brain liveness problem and proposed two solution approaches (model-aware & model agnostic) to ensure liveness and enhance robustness against presentation attacks. Under each of the two solution approaches, several methods were suggested and evaluated against both synthetic and manipulative classes of attacks (a total of 43 different attack vectors). Methods in both model-aware and model-agnostic approaches were successful in achieving an error rate of zero (0%). More importantly, such error rates were reached in face of unseen attacks which provides evidence of the generalization potentials of the proposed solution approaches and methods. I suggested an adversarial workflow to facilitate attack and defense cycles to allow for enhanced generalization capacity for domains in which the decision-making process is non-deterministic such as cyber-physical systems (e.g. biometric/medical monitoring, autonomous machines, etc.). I utilized this workflow for the brain liveness problem and was able to iteratively improve the performance of both the designed attacks and the proposed liveness detection methods.
ContributorsSohankar Esfahani, Mohammad Javad (Author) / Gupta, Sandeep K.S. (Thesis advisor) / Santello, Marco (Committee member) / Dasgupta, Partha (Committee member) / Banerjee, Ayan (Committee member) / Arizona State University (Publisher)
Created2021
Description
Repetitive practice of functional movement patterns during motor rehabilitation are known to drive learning (or relearning) of novel motor skills, but the learning process is highly variable between individuals such that responsiveness to task-specific training is often patient-specific. A number of neuroimaging and neurophysiological methods have been proposed to better predict a patient’s responsiveness to a given type or dose of motor therapy. However, these methods are often time- and resource-intensive, and yield results that are not readily interpretable by clinicians. In contrast, standardized visuospatial tests may offer a more feasible solution. The work presented in this dissertation demonstrate that a clinical paper-and-pencil test of visuospatial function may improve predictive models of motor skill learning in older adults and individuals with stroke pathology. To further our understanding of the neuroanatomical correlates underlying this behavioral relationship, I collected diffusion-weighted magnetic resonance images from 19 nondemented older adults to determine if diffusion characteristics of white matter tracts explain shared variance in delayed visuospatial memory test scores and motor skill learning. Consistent with previous work, results indicated that the structural integrity of regions with the bilateral anterior thalamic radiations, corticospinal tracts, and superior longitudinal fasciculi are related to delayed visuospatial memory performance and one-week skill retention. Overall, results of this dissertation suggest that incorporating a clinical paper-and-pencil test of delayed visuospatial memory may prognose motor rehabilitation outcomes and support that personalized variables should be considered in standards of care. Moreover, regions within specific white matter tracts may underlie this behavioral relationship and future work should investigate these regions as potential targets for therapeutic intervention.
ContributorsLingo VanGilder, Jennapher (Author) / Schaefer, Sydney Y (Thesis advisor) / Santello, Marco (Committee member) / Buneo, Christopher (Committee member) / Rogalsky, Corianne (Committee member) / Duff, Kevin (Committee member) / Arizona State University (Publisher)
Created2021
Description
The importance of nonverbal communication has been well established through several theories including Albert Mehrabian's 7-38-55 rule that proposes the respective importance of semantics, tonality and facial expressions in communication. Although several studies have examined how emotions are expressed and preceived in communication, there is limited research investigating the relationship between how emotions are expressed through semantics and facial expressions. Using a facial expression analysis software to deconstruct facial expressions into features and a K-Nearest-Neighbor (KNN) machine learning classifier, we explored if facial expressions can be clustered based on semantics. Our findings indicate that facial expressions can be clustered based on semantics and that there is an inherent congruence between facial expressions and semantics. These results are novel and significant in the context of nonverbal communication and are applicable to several areas of research including the vast field of emotion AI and machine emotional communication.
ContributorsEverett, Lauren (Author) / Coza, Aurel (Thesis director) / Santello, Marco (Committee member) / Barrett, The Honors College (Contributor) / Harrington Bioengineering Program (Contributor) / Dean, W.P. Carey School of Business (Contributor)
Created2022-05
Description
This dissertation focuses on reinforcement learning (RL) controller design aiming for real-life applications in continuous state and control problems. It involves three major research investigations in the aspect of design, analysis, implementation, and evaluation. The application case addresses automatically configuring robotic prosthesis impedance parameters. Major contributions of the dissertation include the following. 1) An “echo control” using the intact knee profile as target is designed to overcome the limitation of a designer prescribed robotic knee profile. 2) Collaborative multiagent reinforcement learning (cMARL) is proposed to directly take into account human influence in the robot control design. 3) A phased actor in actor-critic (PAAC) reinforcement learning method is developed to reduce learning variance in RL. The design of an “echo control” is based on a new formulation of direct heuristic dynamic programming (dHDP) for tracking control of a robotic knee prosthesis to mimic the intact knee profile. A systematic simulation of the proposed control is provided using a human-robot system simulation in OpenSim. The tracking controller is then tested on able-bodied and amputee subjects. This is the first real-time human testing of RL tracking control of a robotic knee to mirror the profile of an intact knee. The cMARL is a new solution framework for the human-prosthesis collaboration (HPC) problem. This is the first attempt at considering human influence on human-robot walking with the presence of a reinforcement learning controlled lower limb prosthesis. Results show that treating the human and robot as coupled and collaborating agents and using an estimated human adaptation in robot control design help improve human walking performance. The above studies have demonstrated great potential of RL control in solving continuous problems. To solve more complex real-life tasks with multiple control inputs and high dimensional state space, high variance, low data efficiency, slow learning or even instability are major roadblocks to be addressed. A novel PAAC method is proposed to improve learning performance in policy gradient RL by accounting for both Q value and TD error in actor updates. Systematical and comprehensive demonstrations show its effectiveness by qualitative analysis and quantitative evaluation in DeepMind Control Suite.
ContributorsWu, Ruofan (Author) / Si, Jennie (Thesis advisor) / Huang, He (Committee member) / Santello, Marco (Committee member) / Papandreou- Suppappola, Antonia (Committee member) / Arizona State University (Publisher)
Created2023
Description
How humans coordinate digit forces to perform dexterous manipulation is not well understood. This gap is due to the use of tasks devoid of dexterity requirements and/or the use of analytical techniques that cannot isolate the roles that digit forces play in preventing object slip and controlling object position and orientation (pose). In our recent work, we used a dexterous manipulation task and decomposed digit forces into FG, the internal force that prevents object slip, and FM, the force responsible for object pose control. Unlike FG, FM was modulated from object lift onset to hold, suggesting their different sensitivity to sensory feedback acquired during object lift. However, the extent to which FG and FM can be controlled independently remains to be determined. To address this gap, we systematically changed either object mass or external torque. The FM normal component responsible for object orientation control was modulated to changes in object torque but not mass. In contrast, FG was distinctly modulated to changes in object mass and torque. These findings point to a differential sensitivity of FG and FM to task requirements and provide novel insights into the neural control of dexterous manipulation. Importantly, our results indicate that the proposed digit force decomposition has the potential to capture important differences in how sensory inputs are processed and integrated to simultaneously ensure grasp stability and dexterous object pose control.
ContributorsNoll, William (Author) / Santello, Marco (Thesis director) / Wu, Yen-Hsun (Committee member) / Barrett, The Honors College (Contributor) / Harrington Bioengineering Program (Contributor)
Created2024-05
Description
The current program of work explores the potential efficacy of textured insoles for improving biomechanical performance and cognitive acuity during static and dynamic performance. Despite the vast conceptual framework supporting the versatile benefits of textured insoles, the current literature has primarily focused on incorporating this treatment during low-phase movements within the diseased and elderly subset populations. The current study expands this research application by administering textured insole treatments to a healthy population during a physically demanding dynamic assessment and correlating the results to subjects' sensory perception. A convenience sample of 10 subjects was evaluated for their ability to maintain bilateral standing balance in a static condition and adapt to confined lane perturbations during standard track running. These evaluations were conducted under both control and textured insole conditions. Subjects also completed a visual analog scale test, rating the insole treatments based on surface roughness to establish a statistical relationship between individual perception and biomechanical performance. Results showed that textured insole treatments given intermediate ratings of perceived surface roughness significantly enhanced performance during bilateral standing balance and standard track running perturbation adaptation.
ContributorsBoll, Christopher Marly (Author) / Coza, Aurel (Thesis advisor) / Santello, Marco (Committee member) / Lockhart, Thurmon (Committee member) / Arizona State University (Publisher)
Created2024
Description
This project investigates the gleam-glum effect, a well-replicated phonetic emotion association in which words with the [i] vowel-sound (as in “gleam”) are judged more emotionally positive than words with the [Ʌ] vowel-sound (as in “glum”). The effect is observed across different modalities and languages and is moderated by mouth movements relevant to word production. This research presents and tests an articulatory explanation for this association in three experiments. Experiment 1 supported the articulatory explanation by comparing recordings of 71 participants completing an emotional recall task and a word read-aloud task, showing that oral movements were more similar between positive emotional expressions and [i] articulation, and negative emotional expressions and [Ʌ] articulation. Experiment 2 partially supported the explanation with 98 YouTube recordings of natural speech. In Experiment 3, 149 participants judged emotions expressed by a speaker during [i] and [Ʌ] articulation. Contradicting the robust phonetic emotion association, participants judged more frequently that the speaker’s [Ʌ] articulatory movements were positive emotional expressions and [i] articulatory movements were negative emotional expressions. This is likely due to other visual emotional cues not related to oral movements and the order of word lists read by the speaker. Findings from the current project overall support an articulatory explanation for the gleam-glum effect, which has major implications for language and communication.
ContributorsYu, Shin-Phing (Author) / Mcbeath, Michael K (Thesis advisor) / Glenberg, Arthur M (Committee member) / Stone, Greg O (Committee member) / Coza, Aurel (Committee member) / Santello, Marco (Committee member) / Arizona State University (Publisher)
Created2023
Description
Properly deciding to engage in or to withhold an action is a critical ability for goal-oriented movement control. Such decision may be driven by expected value from the choice of action but associating physical effort may discount such value. A novel anticipatory stopping task was developed to investigate effort discounted decision process potentially present in proactive inhibitory control. Subjects performed or abstained from target reach if they believed it was a Go or Stop trial respectively. Reward was awarded to a reach, correctly timed to hit a target at the same time as the moving bar in Go trials. During the Stop trials, correctly judging to not engage in a reach from the color of the moving bar that linked to the bar’s probability of stopping before the target resulted in gaining a reward. Resistive force field incurred additional physical effort for choosing to reach. Introducing effort expectedly decreased the tendency to respond at trials with higher stop probability. Surprisingly, tendency to respond increased and corresponding reaction time decreased in the trials with lower stop probability. Such asymmetric effect suggests that the value of context ineffective response is discounted, and the value of context effective response is flexibly enhanced by its associated effort cost to drive decision-process in goal-oriented manner. Medial frontal event related potential (ERP) locked to the onset of moving bar appearance reflected such effort discounted decision process. Theta band observed in Stop trials accounted for evaluation of effort and context possibly reinforcing such decision-making.
ContributorsTsuchiya, Toshiki (Author) / Santello, Marco (Thesis advisor) / Fine, Justin (Committee member) / McClure, Samuel (Committee member) / Arizona State University (Publisher)
Created2018
Description
Existing theories suggest that evidence is accumulated before making a decision with competing goals. In motor tasks, reward and motor costs have been shown to influence the decision, but the interaction between these two variables has not been studied in depth. A novel reward-based sensorimotor decision-making task was developed to investigate how reward and motor costs interact to influence decisions. In human subjects, two targets of varying size and reward were presented. After a series of three tones, subjects initiated a movement as one of the targets disappeared. Reward was awarded when participants reached through the remaining target within a specific amount of time. Subjects had to initiate a movement before they knew which target remained. Reward was found to be the only factor that influenced the initial reach. When reward was increased, there was a lower probability of intermediate movements. Both target size and reward lowered reaction times individually and jointly. This interaction can be interpreted as the effect of the expected value, which suggests that reward and target size are not evaluated independently during motor planning. Curvature, or the changing of motor plans, was driven primarily by the target size. After an initial decision was made, the motor costs to switch plans and hit the target had the largest impact on the curvature. An interaction between the reward and target size was also found for curvature, suggesting that the expected value of the target influences the changing of motor plans. Reward, target size, and the interaction between the two were all significant factors for different parts of the decision-making process.
ContributorsBoege, Scott (Author) / Santello, Marco (Thesis advisor) / Fine, Justin (Committee member) / McClure, Samuel (Committee member) / Buneo, Christopher (Committee member) / Arizona State University (Publisher)
Created2019
Description
Neural interfacing applications have advanced in complexity, with needs for increasingly high degrees of freedom in prosthetic device control, sharper discrimination in sensory percepts in bidirectional interfaces, and more precise localization of functional connectivity in the brain. As such, there is a growing need for reliable neurophysiological recordings at a fine spatial scale matching that of cortical columnar processing. Penetrating microelectrodes provide localization sufficient to isolate action potential (AP) waveforms, but often suffer from recorded signal deterioration linked to foreign body response. Micro-Electrocorticography (μECoG) surface electrodes elicit lower foreign body response and show greater chronic stability of recorded signals, though they typically lack the signal localization necessary to isolate individual APs. This dissertation validates the recording capacity of a novel, flexible, large area μECoG array with bilayer routing in a feline implant, and explores the ability of conventional μECoG arrays to detect features of neuronal activity in a very high frequency band associated with AP waveforms.
Recordings from both layers of the flexible μECoG array showed frequency features typical of cortical local field potentials (LFP) and were shown to be stable in amplitude over time. Recordings from both layers also showed consistent, frequency-dependent modulation after induction of general anesthesia, with large increases in beta and gamma band and decreases in theta band observed over three experiments. Recordings from conventional μECoG arrays over human cortex showed robust modulation in a high frequency (250-2000 Hz) band upon production of spoken words. Modulation in this band was used to predict spoken words with over 90% accuracy. Basal Ganglia neuronal AP firing was also shown to significantly correlate with various cortical μECoG recordings in this frequency band. Results indicate that μECoG surface electrodes may detect high frequency neuronal activity potentially associated with AP firing, a source of information previously unutilized by these devices.
Recordings from both layers of the flexible μECoG array showed frequency features typical of cortical local field potentials (LFP) and were shown to be stable in amplitude over time. Recordings from both layers also showed consistent, frequency-dependent modulation after induction of general anesthesia, with large increases in beta and gamma band and decreases in theta band observed over three experiments. Recordings from conventional μECoG arrays over human cortex showed robust modulation in a high frequency (250-2000 Hz) band upon production of spoken words. Modulation in this band was used to predict spoken words with over 90% accuracy. Basal Ganglia neuronal AP firing was also shown to significantly correlate with various cortical μECoG recordings in this frequency band. Results indicate that μECoG surface electrodes may detect high frequency neuronal activity potentially associated with AP firing, a source of information previously unutilized by these devices.
ContributorsBarton, Cody David (Author) / Greger, Bradley (Thesis advisor, Committee member) / Santello, Marco (Committee member) / Buneo, Christopher (Committee member) / Graudejus, Oliver (Committee member) / Artemiadis, Panagiotis (Committee member) / Arizona State University (Publisher)
Created2018