Matching Items (12)
Filtering by
- All Subjects: robotics
- All Subjects: Electric power systems
- Member of: Theses and Dissertations
Description
Under the framework of intelligent management of power grids by leveraging advanced information, communication and control technologies, a primary objective of this study is to develop novel data mining and data processing schemes for several critical applications that can enhance the reliability of power systems. Specifically, this study is broadly organized into the following two parts: I) spatio-temporal wind power analysis for wind generation forecast and integration, and II) data mining and information fusion of synchrophasor measurements toward secure power grids. Part I is centered around wind power generation forecast and integration. First, a spatio-temporal analysis approach for short-term wind farm generation forecasting is proposed. Specifically, using extensive measurement data from an actual wind farm, the probability distribution and the level crossing rate of wind farm generation are characterized using tools from graphical learning and time-series analysis. Built on these spatial and temporal characterizations, finite state Markov chain models are developed, and a point forecast of wind farm generation is derived using the Markov chains. Then, multi-timescale scheduling and dispatch with stochastic wind generation and opportunistic demand response is investigated. Part II focuses on incorporating the emerging synchrophasor technology into the security assessment and the post-disturbance fault diagnosis of power systems. First, a data-mining framework is developed for on-line dynamic security assessment by using adaptive ensemble decision tree learning of real-time synchrophasor measurements. Under this framework, novel on-line dynamic security assessment schemes are devised, aiming to handle various factors (including variations of operating conditions, forced system topology change, and loss of critical synchrophasor measurements) that can have significant impact on the performance of conventional data-mining based on-line DSA schemes. Then, in the context of post-disturbance analysis, fault detection and localization of line outage is investigated using a dependency graph approach. It is shown that a dependency graph for voltage phase angles can be built according to the interconnection structure of power system, and line outage events can be detected and localized through networked data fusion of the synchrophasor measurements collected from multiple locations of power grids. Along a more practical avenue, a decentralized networked data fusion scheme is proposed for efficient fault detection and localization.
ContributorsHe, Miao (Author) / Zhang, Junshan (Thesis advisor) / Vittal, Vijay (Thesis advisor) / Hedman, Kory (Committee member) / Si, Jennie (Committee member) / Ye, Jieping (Committee member) / Arizona State University (Publisher)
Created2013
Description
Contemporary methods for dynamic security assessment (DSA) mainly re-ly on time domain simulations to explore the influence of large disturbances in a power system. These methods are computationally intensive especially when the system operating point changes continually. The trajectory sensitivity method, when implemented and utilized as a complement to the existing DSA time domain simulation routine, can provide valuable insights into the system variation in re-sponse to system parameter changes. The implementation of the trajectory sensitivity analysis is based on an open source power system analysis toolbox called PSAT. Eight categories of sen-sitivity elements have been implemented and tested. The accuracy assessment of the implementation demonstrates the validity of both the theory and the imple-mentation. The computational burden introduced by the additional sensitivity equa-tions is relieved by two innovative methods: one is by employing a cluster to per-form the sensitivity calculations in parallel; the other one is by developing a mod-ified very dishonest Newton method in conjunction with the latest sparse matrix processing technology. The relation between the linear approximation accuracy and the perturba-tion size is also studied numerically. It is found that there is a fixed connection between the linear approximation accuracy and the perturbation size. Therefore this finding can serve as a general application guide to evaluate the accuracy of the linear approximation. The applicability of the trajectory sensitivity approach to a large realistic network has been demonstrated in detail. This research work applies the trajectory sensitivity analysis method to the Western Electricity Coordinating Council (WECC) system. Several typical power system dynamic security problems, in-cluding the transient angle stability problem, the voltage stability problem consid-ering load modeling uncertainty and the transient stability constrained interface real power flow limit calculation, have been addressed. Besides, a method based on the trajectory sensitivity approach and the model predictive control has been developed for determination of under frequency load shedding strategy for real time stability assessment. These applications have shown the great efficacy and accuracy of the trajectory sensitivity method in handling these traditional power system stability problems.
ContributorsHou, Guanji (Author) / Vittal, Vijay (Thesis advisor) / Heydt, Gerald (Committee member) / Tylavsky, Daniel (Committee member) / Si, Jennie (Committee member) / Arizona State University (Publisher)
Created2012
Description
Photovoltaic (PV) power generation has the potential to cause a significant impact on power system reliability since its total installed capacity is projected to increase at a significant rate. PV generation can be described as an intermittent and variable resource because its production is influenced by ever-changing environmental conditions. The study in this dissertation focuses on the influence of PV generation on trans-mission system reliability. This is a concern because PV generation output is integrated into present power systems at various voltage levels and may significantly affect the power flow patterns. This dissertation applies a probabilistic power flow (PPF) algorithm to evaluate the influence of PV generation uncertainty on transmission system perfor-mance. A cumulant-based PPF algorithm suitable for large systems is used. Correlation among adjacent PV resources is considered. Three types of approximation expansions based on cumulants namely Gram-Charlier expansion, Edgeworth expansion and Cor-nish-Fisher expansion are compared, and their properties, advantages and deficiencies are discussed. Additionally, a novel probabilistic model of PV generation is developed to obtain the probability density function (PDF) of the PV generation production based on environmental conditions. Besides, this dissertation proposes a novel PPF algorithm considering the conven-tional generation dispatching operation to balance PV generation uncertainties. It is pru-dent to include generation dispatch in the PPF algorithm since the dispatching strategy compensates for PV generation injections and influences the uncertainty results. Fur-thermore, this dissertation also proposes a probabilistic optimal power dispatching strat-egy which considers uncertainty problems in the economic dispatch and optimizes the expected value of the total cost with the overload probability as a constraint. The proposed PPF algorithm with the three expansions is compared with Monte Carlo simulations (MCS) with results for a 2497-bus representation of the Arizona area of the Western Electricity Coordinating Council (WECC) system. The PDFs of the bus voltages, line flows and slack bus production are computed, and are used to identify the confidence interval, the over limit probability and the expected over limit time of the ob-jective variables. The proposed algorithm is of significant relevance to the operating and planning studies of the transmission systems with PV generation installed.
ContributorsFan, Miao (Author) / Vittal, Vijay (Thesis advisor) / Heydt, Gerald Thomas (Committee member) / Ayyanar, Raja (Committee member) / Si, Jennie (Committee member) / Arizona State University (Publisher)
Created2012
Description
In electric power systems, phasor measurement units (PMUs) are capable of providing synchronized voltage and current phasor measurements which are superior to conventional measurements collected by the supervisory control and data acquisition (SCADA) system in terms of resolution and accuracy. These measurements are known as synchrophasor measurements. Considerable research work has been done on the applications of PMU measurements based on the as-sumption that a high level of accuracy is obtained in the field. The study in this dissertation is conducted to address the basic issue concerning the accuracy of actual PMU measurements in the field. Synchronization is one of the important features of PMU measurements. However, the study presented in this dissertation reveals that the problem of faulty synchronization between measurements with the same time stamps from different PMUs exists. A Kalman filter model is proposed to analyze and calcu-late the time skew error caused by faulty synchronization. In order to achieve a high level of accuracy of PMU measurements, inno-vative methods are proposed to detect and identify system state changes or bad data which are reflected by changes in the measurements. This procedure is ap-plied as a key step in adaptive Kalman filtering of PMU measurements to over-come the insensitivity of a conventional Kalman filter. Calibration of PMU measurements is implemented in specific PMU instal-lation scenarios using transmission line (TL) parameters from operation planning data. The voltage and current correction factors calculated from the calibration procedure indicate the possible errors in PMU measurements. Correction factors can be applied in on-line calibration of PMU measurements. A study is conducted to address an important issue when integrating PMU measurements into state estimation. The reporting rate of PMU measurements is much higher than that of the measurements collected by the SCADA. The ques-tion of how to buffer PMU measurements is raised. The impact of PMU meas-urement buffer length on state estimation is discussed. A method based on hy-pothesis testing is proposed to determine the optimal buffer length of PMU meas-urements considering the two conflicting features of PMU measurements, i. e. un-certainty and variability. Results are presented for actual PMU synchrophasor measurements.
ContributorsZhang, Qing (Author) / Heydt, Gerald (Thesis advisor) / Vittal, Vijay (Thesis advisor) / Ayyanar, Raja (Committee member) / Si, Jennie (Committee member) / Arizona State University (Publisher)
Created2012
Description
Soft continuum robots with the ability to bend, twist, elongate, and shorten, similar to octopus arms, have many potential applications, such as dexterous manipulation and navigation through unstructured, dynamic environments. Novel soft materials such as smart hydrogels, which change volume and other properties in response to stimuli such as temperature, pH, and chemicals, can potentially be used to construct soft robots that achieve self-regulated adaptive reconfiguration through on-demand dynamic control of local properties. However, the design of controllers for soft continuum robots is challenging due to their high-dimensional configuration space and the complexity of modeling soft actuator dynamics. To address these challenges, this dissertation presents two different model-based control approaches for robots with distributed soft actuators and sensors and validates the approaches in simulations and physical experiments. It is demonstrated that by choosing an appropriate dynamical model and designing a decentralized controller based on this model, such robots can be controlled to achieve diverse types of complex configurations. The first approach consists of approximating the dynamics of the system, including its actuators, as a linear state-space model in order to apply optimal robust control techniques such as H∞ state-feedback and H∞ output-feedback methods. These techniques are designed to utilize the decentralized control structure of the robot and its distributed sensing and actuation to achieve vibration control and trajectory tracking. The approach is validated in simulation on an Euler-Bernoulli dynamic model of a hydrogel based cantilevered robotic arm and in experiments with a hydrogel-actuated miniature 2-DOF manipulator. The second approach is developed for soft continuum robots with dynamics that can be modeled using Cosserat rod theory. An inverse dynamics control approach is implemented on the Cosserat model of the robot for tracking configurations that include bending, torsion, shear, and extension deformations. The decentralized controller structure facilitates its implementation on robot arms composed of independently-controllable segments that have local sensing and actuation. This approach is validated on simulated 3D robot arms and on an actual silicone robot arm with distributed pneumatic actuation, for which the inverse dynamics problem is solved in simulation and the computed control outputs are applied to the robot in real-time.
ContributorsDoroudchi, Azadeh (Author) / Berman, Spring (Thesis advisor) / Tsakalis, Konstantinos (Committee member) / Si, Jennie (Committee member) / Marvi, Hamid (Committee member) / Arizona State University (Publisher)
Created2022
Description
This dissertation focuses on reinforcement learning (RL) controller design aiming for real-life applications in continuous state and control problems. It involves three major research investigations in the aspect of design, analysis, implementation, and evaluation. The application case addresses automatically configuring robotic prosthesis impedance parameters. Major contributions of the dissertation include the following. 1) An “echo control” using the intact knee profile as target is designed to overcome the limitation of a designer prescribed robotic knee profile. 2) Collaborative multiagent reinforcement learning (cMARL) is proposed to directly take into account human influence in the robot control design. 3) A phased actor in actor-critic (PAAC) reinforcement learning method is developed to reduce learning variance in RL. The design of an “echo control” is based on a new formulation of direct heuristic dynamic programming (dHDP) for tracking control of a robotic knee prosthesis to mimic the intact knee profile. A systematic simulation of the proposed control is provided using a human-robot system simulation in OpenSim. The tracking controller is then tested on able-bodied and amputee subjects. This is the first real-time human testing of RL tracking control of a robotic knee to mirror the profile of an intact knee. The cMARL is a new solution framework for the human-prosthesis collaboration (HPC) problem. This is the first attempt at considering human influence on human-robot walking with the presence of a reinforcement learning controlled lower limb prosthesis. Results show that treating the human and robot as coupled and collaborating agents and using an estimated human adaptation in robot control design help improve human walking performance. The above studies have demonstrated great potential of RL control in solving continuous problems. To solve more complex real-life tasks with multiple control inputs and high dimensional state space, high variance, low data efficiency, slow learning or even instability are major roadblocks to be addressed. A novel PAAC method is proposed to improve learning performance in policy gradient RL by accounting for both Q value and TD error in actor updates. Systematical and comprehensive demonstrations show its effectiveness by qualitative analysis and quantitative evaluation in DeepMind Control Suite.
ContributorsWu, Ruofan (Author) / Si, Jennie (Thesis advisor) / Huang, He (Committee member) / Santello, Marco (Committee member) / Papandreou- Suppappola, Antonia (Committee member) / Arizona State University (Publisher)
Created2023
Description
Human-robot interactions can often be formulated as general-sum differential games where the equilibrial policies are governed by Hamilton-Jacobi-Isaacs (HJI) equations. Solving HJI PDEs faces the curse of dimensionality (CoD). While physics-informed neural networks (PINNs) alleviate CoD in solving PDEs with smooth solutions, they fall short in learning discontinuous solutions due to their sampling nature. This causes PINNs to have poor safety performance when they are applied to approximate values that are discontinuous due to state constraints. This dissertation aims to improve the safety performance of PINN-based value and policy models. The first contribution of the dissertation is to develop learning methods to approximate discontinuous values. Specifically, three solutions are developed: (1) hybrid learning uses both supervisory and PDE losses, (2) value-hardening solves HJIs with increasing Lipschitz constant on the constraint violation penalty, and (3) the epigraphical technique lifts the value to a higher-dimensional state space where it becomes continuous. Evaluations through 5D and 9D vehicle and 13D drone simulations reveal that the hybrid method outperforms others in terms of generalization and safety performance. The second contribution is a learning-theoretical analysis of PINN for value and policy approximation. Specifically, by extending the neural tangent kernel (NTK) framework, this dissertation explores why the choice of activation function significantly affects the PINN generalization performance, and why the inclusion of supervisory costate data improves the safety performance. The last contribution is a series of extensions of the hybrid PINN method to address real-time parameter estimation problems in incomplete-information games. Specifically, a Pontryagin-mode PINN is developed to avoid costly computation for supervisory data. The key idea is the introduction of a costate loss, which is cheap to compute yet effectively enables the learning of important value changes and policies in space-time. Building upon this, a Pontryagin-mode neural operator is developed to achieve state-of-the-art (SOTA) safety performance across a set of differential games with parametric state constraints. This dissertation demonstrates the utility of the resultant neural operator in estimating player constraint parameters during incomplete-information games.
ContributorsZhang, Lei (Author) / Ren, Yi (Thesis advisor) / Si, Jennie (Committee member) / Berman, Spring (Committee member) / Zhang, Wenlong (Committee member) / Xu, Zhe (Committee member) / Arizona State University (Publisher)
Created2024
Description
Unmanned aerial vehicles (UAVs) are widely used in many applications because of their small size, great mobility and hover performance. This has been a consequence of the fast development of electronics, cheap lightweight flight controllers for accurate positioning and cameras. This thesis describes modeling, control and design of an oblique-cross-quadcopter platform for indoor-environments.
One contribution of the work was the design of a new printed-circuit-board (PCB) flight controller (called MARK3). Key features/capabilities are as follows:
(1) a Teensy 3.2 microcontroller with 168MHz overclock –used for communications, full-state estimation and inner-outer loop hierarchical rate-angle-speed-position control,
(2) an on-board MEMS inertial-measurement-unit (IMU) which includes an LSM303D (3DOF-accelerometer and magnetometer), an L3GD20 (3DOF-gyroscope) and a BMP180 (barometer) for attitude estimation (barometer/magnetometer not used),
(3) 6 pulse-width-modulator (PWM) output pins supports up to 6 rotors
(4) 8 PWM input pins support up to 8-channel 2.4 GHz transmitter/receiver for manual control,
(5) 2 5V servo extension outputs for other requirements (e.g. gimbals),
(6) 2 universal-asynchronous-receiver-transmitter (UART) serial ports - used by flight controller to process data from Xbee; can be used for accepting outer-loop position commands from NVIDIA TX2 (future work),
(7) 1 I2C-serial-protocol two-wire port for additional modules (used to read data from IMU at 400 Hz),
(8) a 20-pin port for Xbee telemetry module connection; permits Xbee transceiver on desktop PC to send position/attitude commands to Xbee transceiver on quadcopter.
The quadcopter platform consists of the new MARK3 PCB Flight Controller, an ATG-250 carbon-fiber frame (250 mm), a DJI Snail propulsion-system (brushless-three-phase-motor, electronic-speed-controller (ESC) and propeller), an HTC VIVE Tracker and RadioLink R9DS 9-Channel 2.4GHz Receiver. This platform is completely compatible with the HTC VIVE Tracking System (HVTS) which has 7ms latency, submillimeter accuracy and a much lower price compared to other millimeter-level tracking systems.
The thesis describes nonlinear and linear modeling of the quadcopter’s 6DOF rigid-body dynamics and brushless-motor-actuator dynamics. These are used for hierarchical-classical-control-law development near hover. The HVTS was used to demonstrate precision hover-control and path-following. Simulation and measured flight-data are shown to be similar. This work provides a foundation for future precision multi-quadcopter formation-flight-control.
One contribution of the work was the design of a new printed-circuit-board (PCB) flight controller (called MARK3). Key features/capabilities are as follows:
(1) a Teensy 3.2 microcontroller with 168MHz overclock –used for communications, full-state estimation and inner-outer loop hierarchical rate-angle-speed-position control,
(2) an on-board MEMS inertial-measurement-unit (IMU) which includes an LSM303D (3DOF-accelerometer and magnetometer), an L3GD20 (3DOF-gyroscope) and a BMP180 (barometer) for attitude estimation (barometer/magnetometer not used),
(3) 6 pulse-width-modulator (PWM) output pins supports up to 6 rotors
(4) 8 PWM input pins support up to 8-channel 2.4 GHz transmitter/receiver for manual control,
(5) 2 5V servo extension outputs for other requirements (e.g. gimbals),
(6) 2 universal-asynchronous-receiver-transmitter (UART) serial ports - used by flight controller to process data from Xbee; can be used for accepting outer-loop position commands from NVIDIA TX2 (future work),
(7) 1 I2C-serial-protocol two-wire port for additional modules (used to read data from IMU at 400 Hz),
(8) a 20-pin port for Xbee telemetry module connection; permits Xbee transceiver on desktop PC to send position/attitude commands to Xbee transceiver on quadcopter.
The quadcopter platform consists of the new MARK3 PCB Flight Controller, an ATG-250 carbon-fiber frame (250 mm), a DJI Snail propulsion-system (brushless-three-phase-motor, electronic-speed-controller (ESC) and propeller), an HTC VIVE Tracker and RadioLink R9DS 9-Channel 2.4GHz Receiver. This platform is completely compatible with the HTC VIVE Tracking System (HVTS) which has 7ms latency, submillimeter accuracy and a much lower price compared to other millimeter-level tracking systems.
The thesis describes nonlinear and linear modeling of the quadcopter’s 6DOF rigid-body dynamics and brushless-motor-actuator dynamics. These are used for hierarchical-classical-control-law development near hover. The HVTS was used to demonstrate precision hover-control and path-following. Simulation and measured flight-data are shown to be similar. This work provides a foundation for future precision multi-quadcopter formation-flight-control.
ContributorsLu, Shi (Author) / Rodriguez, Armando A. (Thesis advisor) / Tsakalis, Konstantinos (Committee member) / Si, Jennie (Committee member) / Arizona State University (Publisher)
Created2018
Description
Toward the ambitious long-term goal of a fleet of cooperating Flexible Autonomous Machines operating in an uncertain Environment (FAME), this thesis addresses several
critical modeling, design and control objectives for ground vehicles. One central objective was to show how off-the-shelf (low-cost) remote-control (RC) “toy” vehicles can be converted into intelligent multi-capability robotic-platforms for conducting FAME research. This is shown for two vehicle classes: (1) six differential-drive (DD) RC vehicles called Thunder Tumbler (DDTT) and (2) one rear-wheel drive (RWD) RC car called Ford F-150 (1:14 scale). Each DDTT-vehicle was augmented to provide a substantive suite of capabilities as summarized below (It should be noted, however, that only one DDTT-vehicle was augmented with an inertial measurement unit (IMU) and 2.4 GHz RC capability): (1) magnetic wheel-encoders/IMU for(dead-reckoning-based) inner-loop speed-control and outer-loop position-directional-control, (2) Arduino Uno microcontroller-board for encoder-based inner-loop speed-control and encoder-IMU-ultrasound-based outer-loop cruise-position-directional-separation-control, (3) Arduino motor-shield for inner-loop motor-speed-control, (4)Raspberry Pi II computer-board for demanding outer-loop vision-based cruise- position-directional-control, (5) Raspberry Pi 5MP camera for outer-loop cruise-position-directional-control (exploiting WiFi to send video back to laptop), (6) forward-pointing ultrasonic distance/rangefinder sensor for outer-loop separation-control, and (7) 2.4 GHz spread-spectrum RC capability to replace original 27/49 MHz RC. Each “enhanced”/ augmented DDTT-vehicle costs less than 175 but offers the capability of commercially available vehicles costing over 500. Both the Arduino and Raspberry are low-cost, well-supported (software wise) and easy-to-use. For the vehicle classes considered (i.e. DD, RWD), both kinematic and dynamical (planar xy) models are examined. Suitable nonlinear/linear-models are used to develop inner/outer-loopcontrol laws.
All demonstrations presented involve enhanced DDTT-vehicles; one the F-150; one a quadrotor. The following summarizes key hardware demonstrations: (1) cruise-control along line, (2) position-control along line (3) position-control along curve (4) planar (xy) Cartesian stabilization, (5) cruise-control along jagged line/curve, (6) vehicle-target spacing-control, (7) multi-robot spacing-control along line/curve, (8) tracking slowly-moving remote-controlled quadrotor, (9) avoiding obstacle while moving toward target, (10) RC F-150 followed by DDTT-vehicle. Hardware data/video is compared with, and corroborated by, model-based simulations. In short, many capabilities that are critical for reaching the longer-term FAME goal are demonstrated.
critical modeling, design and control objectives for ground vehicles. One central objective was to show how off-the-shelf (low-cost) remote-control (RC) “toy” vehicles can be converted into intelligent multi-capability robotic-platforms for conducting FAME research. This is shown for two vehicle classes: (1) six differential-drive (DD) RC vehicles called Thunder Tumbler (DDTT) and (2) one rear-wheel drive (RWD) RC car called Ford F-150 (1:14 scale). Each DDTT-vehicle was augmented to provide a substantive suite of capabilities as summarized below (It should be noted, however, that only one DDTT-vehicle was augmented with an inertial measurement unit (IMU) and 2.4 GHz RC capability): (1) magnetic wheel-encoders/IMU for(dead-reckoning-based) inner-loop speed-control and outer-loop position-directional-control, (2) Arduino Uno microcontroller-board for encoder-based inner-loop speed-control and encoder-IMU-ultrasound-based outer-loop cruise-position-directional-separation-control, (3) Arduino motor-shield for inner-loop motor-speed-control, (4)Raspberry Pi II computer-board for demanding outer-loop vision-based cruise- position-directional-control, (5) Raspberry Pi 5MP camera for outer-loop cruise-position-directional-control (exploiting WiFi to send video back to laptop), (6) forward-pointing ultrasonic distance/rangefinder sensor for outer-loop separation-control, and (7) 2.4 GHz spread-spectrum RC capability to replace original 27/49 MHz RC. Each “enhanced”/ augmented DDTT-vehicle costs less than 175 but offers the capability of commercially available vehicles costing over 500. Both the Arduino and Raspberry are low-cost, well-supported (software wise) and easy-to-use. For the vehicle classes considered (i.e. DD, RWD), both kinematic and dynamical (planar xy) models are examined. Suitable nonlinear/linear-models are used to develop inner/outer-loopcontrol laws.
All demonstrations presented involve enhanced DDTT-vehicles; one the F-150; one a quadrotor. The following summarizes key hardware demonstrations: (1) cruise-control along line, (2) position-control along line (3) position-control along curve (4) planar (xy) Cartesian stabilization, (5) cruise-control along jagged line/curve, (6) vehicle-target spacing-control, (7) multi-robot spacing-control along line/curve, (8) tracking slowly-moving remote-controlled quadrotor, (9) avoiding obstacle while moving toward target, (10) RC F-150 followed by DDTT-vehicle. Hardware data/video is compared with, and corroborated by, model-based simulations. In short, many capabilities that are critical for reaching the longer-term FAME goal are demonstrated.
ContributorsLin, Zhenyu (Author) / Rodriguez, Armando Antonio (Committee member) / Si, Jennie (Committee member) / Berman, Spring Melody (Committee member) / Arizona State University (Publisher)
Created2015
Description
Robotic lower limb prostheses provide new opportunities to help transfemoral amputees regain mobility. However, their application is impeded by that the impedance control parameters need to be tuned and optimized manually by prosthetists for each individual user in different task environments. Reinforcement learning (RL) is capable of automatically learning from interacting with the environment. It becomes a natural candidate to replace human prosthetists to customize the control parameters. However, neither traditional RL approaches nor the popular deep RL approaches are readily suitable for learning with limited number of samples and samples with large variations. This dissertation aims to explore new RL based adaptive solutions that are data-efficient for controlling robotic prostheses.
This dissertation begins by proposing a new flexible policy iteration (FPI) framework. To improve sample efficiency, FPI can utilize either on-policy or off-policy learning strategy, can learn from either online or offline data, and can even adopt exiting knowledge of an external critic. Approximate convergence to Bellman optimal solutions are guaranteed under mild conditions. Simulation studies validated that FPI was data efficient compared to several established RL methods. Furthermore, a simplified version of FPI was implemented to learn from offline data, and then the learned policy was successfully tested for tuning the control parameters online on a human subject.
Next, the dissertation discusses RL control with information transfer (RL-IT), or knowledge-guided RL (KG-RL), which is motivated to benefit from transferring knowledge acquired from one subject to another. To explore its feasibility, knowledge was extracted from data measurements of able-bodied (AB) subjects, and transferred to guide Q-learning control for an amputee in OpenSim simulations. This result again demonstrated that data and time efficiency were improved using previous knowledge.
While the present study is new and promising, there are still many open questions to be addressed in future research. To account for human adaption, the learning control objective function may be designed to incorporate human-prosthesis performance feedback such as symmetry, user comfort level and satisfaction, and user energy consumption. To make the RL based control parameter tuning practical in real life, it should be further developed and tested in different use environments, such as from level ground walking to stair ascending or descending, and from walking to running.
This dissertation begins by proposing a new flexible policy iteration (FPI) framework. To improve sample efficiency, FPI can utilize either on-policy or off-policy learning strategy, can learn from either online or offline data, and can even adopt exiting knowledge of an external critic. Approximate convergence to Bellman optimal solutions are guaranteed under mild conditions. Simulation studies validated that FPI was data efficient compared to several established RL methods. Furthermore, a simplified version of FPI was implemented to learn from offline data, and then the learned policy was successfully tested for tuning the control parameters online on a human subject.
Next, the dissertation discusses RL control with information transfer (RL-IT), or knowledge-guided RL (KG-RL), which is motivated to benefit from transferring knowledge acquired from one subject to another. To explore its feasibility, knowledge was extracted from data measurements of able-bodied (AB) subjects, and transferred to guide Q-learning control for an amputee in OpenSim simulations. This result again demonstrated that data and time efficiency were improved using previous knowledge.
While the present study is new and promising, there are still many open questions to be addressed in future research. To account for human adaption, the learning control objective function may be designed to incorporate human-prosthesis performance feedback such as symmetry, user comfort level and satisfaction, and user energy consumption. To make the RL based control parameter tuning practical in real life, it should be further developed and tested in different use environments, such as from level ground walking to stair ascending or descending, and from walking to running.
ContributorsGao, Xiang (Author) / Si, Jennie (Thesis advisor) / Huang, He Helen (Committee member) / Santello, Marco (Committee member) / Papandreou-Suppappola, Antonia (Committee member) / Arizona State University (Publisher)
Created2020