Matching Items (5)
Filtering by
- All Subjects: Reinforcement Learning
- Creators: Berman, Spring
- Creators: Rodriguez, Armando Antonio
Description
To achieve the ambitious long-term goal of a feet of cooperating Flexible Autonomous
Machines operating in an uncertain Environment (FAME), this thesis addresses several
critical modeling, design, control objectives for rear-wheel drive ground vehicles.
Toward this ambitious goal, several critical objectives are addressed. One central objective of the thesis was to show how to build low-cost multi-capability robot platform
that can be used for conducting FAME research.
A TFC-KIT car chassis was augmented to provide a suite of substantive capabilities.
The augmented vehicle (FreeSLAM Robot) costs less than $500 but offers the capability
of commercially available vehicles costing over $2000.
All demonstrations presented involve rear-wheel drive FreeSLAM robot. The following
summarizes the key hardware demonstrations presented and analyzed:
(1)Cruise (v, ) control along a line,
(2) Cruise (v, ) control along a curve,
(3) Planar (x, y) Cartesian Stabilization for rear wheel drive vehicle,
(4) Finish the track with camera pan tilt structure in minimum time,
(5) Finish the track without camera pan tilt structure in minimum time,
(6) Vision based tracking performance with different cruise speed vx,
(7) Vision based tracking performance with different camera fixed look-ahead distance L,
(8) Vision based tracking performance with different delay Td from vision subsystem,
(9) Manually remote controlled robot to perform indoor SLAM,
(10) Autonomously line guided robot to perform indoor SLAM.
For most cases, hardware data is compared with, and corroborated by, model based
simulation data. In short, the thesis uses low-cost self-designed rear-wheel
drive robot to demonstrate many capabilities that are critical in order to reach the
longer-term FAME goal.
Machines operating in an uncertain Environment (FAME), this thesis addresses several
critical modeling, design, control objectives for rear-wheel drive ground vehicles.
Toward this ambitious goal, several critical objectives are addressed. One central objective of the thesis was to show how to build low-cost multi-capability robot platform
that can be used for conducting FAME research.
A TFC-KIT car chassis was augmented to provide a suite of substantive capabilities.
The augmented vehicle (FreeSLAM Robot) costs less than $500 but offers the capability
of commercially available vehicles costing over $2000.
All demonstrations presented involve rear-wheel drive FreeSLAM robot. The following
summarizes the key hardware demonstrations presented and analyzed:
(1)Cruise (v, ) control along a line,
(2) Cruise (v, ) control along a curve,
(3) Planar (x, y) Cartesian Stabilization for rear wheel drive vehicle,
(4) Finish the track with camera pan tilt structure in minimum time,
(5) Finish the track without camera pan tilt structure in minimum time,
(6) Vision based tracking performance with different cruise speed vx,
(7) Vision based tracking performance with different camera fixed look-ahead distance L,
(8) Vision based tracking performance with different delay Td from vision subsystem,
(9) Manually remote controlled robot to perform indoor SLAM,
(10) Autonomously line guided robot to perform indoor SLAM.
For most cases, hardware data is compared with, and corroborated by, model based
simulation data. In short, the thesis uses low-cost self-designed rear-wheel
drive robot to demonstrate many capabilities that are critical in order to reach the
longer-term FAME goal.
ContributorsLu, Xianglong (Author) / Rodriguez, Armando Antonio (Thesis advisor) / Berman, Spring (Committee member) / Artemiadis, Panagiotis (Committee member) / Arizona State University (Publisher)
Created2016
Description
Robotic systems are outmatched by the abilities of the human hand to perceive and manipulate the world. Human hands are able to physically interact with the world to perceive, learn, and act to accomplish tasks. Limitations of robotic systems to interact with and manipulate the world diminish their usefulness. In order to advance robot end effectors, specifically artificial hands, rich multimodal tactile sensing is needed. In this work, a multi-articulating, anthropomorphic robot testbed was developed for investigating tactile sensory stimuli during finger-object interactions. The artificial finger is controlled by a tendon-driven remote actuation system that allows for modular control of any tendon-driven end effector and capabilities for both speed and strength. The artificial proprioception system enables direct measurement of joint angles and tendon tensions while temperature, vibration, and skin deformation are provided by a multimodal tactile sensor. Next, attention was focused on real-time artificial perception for decision-making. A robotic system needs to perceive its environment in order to make decisions. Specific actions such as “exploratory procedures” can be employed to classify and characterize object features. Prior work on offline perception was extended to develop an anytime predictive model that returns the probability of having touched a specific feature of an object based on minimally processed sensor data. Developing models for anytime classification of features facilitates real-time action-perception loops. Finally, by combining real-time action-perception with reinforcement learning, a policy was learned to complete a functional contour-following task: closing a deformable ziplock bag. The approach relies only on proprioceptive and localized tactile data. A Contextual Multi-Armed Bandit (C-MAB) reinforcement learning algorithm was implemented to maximize cumulative rewards within a finite time period by balancing exploration versus exploitation of the action space. Performance of the C-MAB learner was compared to a benchmark Q-learner that eventually returns the optimal policy. To assess robustness and generalizability, the learned policy was tested on variations of the original contour-following task. The work presented contributes to the full range of tools necessary to advance the abilities of artificial hands with respect to dexterity, perception, decision-making, and learning.
ContributorsHellman, Randall Blake (Author) / Santos, Veronica J (Thesis advisor) / Artemiadis, Panagiotis K (Committee member) / Berman, Spring (Committee member) / Helms Tillery, Stephen I (Committee member) / Fainekos, Georgios (Committee member) / Arizona State University (Publisher)
Created2016
Description
A Graph Neural Network (GNN) is a type of neural network architecture that operates on data consisting of objects and their relationships, which are represented by a graph. Within the graph, nodes represent objects and edges represent associations between those objects. The representation of relationships and correlations between data is unique to graph structures. GNNs exploit this feature of graphs by augmenting both forms of data, individual and relational, and have been designed to allow for communication and sharing of data within each neural network layer. These benefits allow each node to have an enriched perspective, or a better understanding, of its neighbouring nodes and its connections to those nodes. The ability of GNNs to efficiently process high-dimensional node data and multi-faceted relationships among nodes gives them advantages over neural network architectures such as Convolutional Neural Networks (CNNs) that do not implicitly handle relational data. These quintessential characteristics of GNN models make them suitable for solving problems in which the correspondences among input data are needed to produce an accurate and precise representation of these data. GNN frameworks may significantly improve existing communication and control techniques for multi-agent tasks by implicitly representing not only information associated with the individual agents, such as agent position, velocity, and camera data, but also their relationships with one another, such as distances between the agents and their ability to communicate with one another. One such task is a multi-agent navigation problem in which the agents must coordinate with one another in a decentralized manner, using proximity sensors only, to navigate safely to their intended goal positions in the environment without collisions or deadlocks. The contribution of this thesis is the design of an end-to-end decentralized control scheme for multi-agent navigation that utilizes GNNs to prevent inter-agent collisions and deadlocks. The contributions consist of the development, simulation and evaluation of the performance of an advantage actor-critic (A2C) reinforcement learning algorithm that employs actor and critic networks for training that simultaneously approximate the policy function and value function, respectively. These networks are implemented using GNN frameworks for navigation by groups of 3, 5, 10 and 15 agents in simulated two-dimensional environments. It is observed that in $40\%$ to $50\%$ of the simulation trials, between 70$\%$ to 80$\%$ of the agents reach their goal positions without colliding with other agents or becoming trapped in deadlocks. The model is also compared to a random run simulation, where actions are chosen randomly for the agents and observe that the model performs notably well for smaller groups of agents.
ContributorsAyalasomayajula, Manaswini (Author) / Berman, Spring (Thesis advisor) / Mian, Sami (Committee member) / Pavlic, Theodore (Committee member) / Arizona State University (Publisher)
Created2022
Description
This work has improved the quality of the solution to the sparse rewards problemby combining reinforcement learning (RL) with knowledge-rich planning. Classical
methods for coping with sparse rewards during reinforcement learning modify the
reward landscape so as to better guide the learner. In contrast, this work combines
RL with a planner in order to utilize other information about the environment. As
the scope for representing environmental information is limited in RL, this work has
conflated a model-free learning algorithm – temporal difference (TD) learning – with
a Hierarchical Task Network (HTN) planner to accommodate rich environmental
information in the algorithm. In the perpetual sparse rewards problem, rewards
reemerge after being collected within a fixed interval of time, culminating in a lack of a
well-defined goal state as an exit condition to the problem. Incorporating planning in
the learning algorithm not only improves the quality of the solution, but the algorithm
also avoids the ambiguity of incorporating a goal of maximizing profit while using
only a planning algorithm to solve this problem. Upon occasionally using the HTN
planner, this algorithm provides the necessary tweak toward the optimal solution. In
this work, I have demonstrated an on-policy algorithm that has improved the quality
of the solution over vanilla reinforcement learning. The objective of this work has
been to observe the capacity of the synthesized algorithm in finding optimal policies to
maximize rewards, awareness of the environment, and the awareness of the presence
of other agents in the vicinity.
ContributorsNandan, Swastik (Author) / Pavlic, Theodore (Thesis advisor) / Das, Jnaneshwar (Thesis advisor) / Berman, Spring (Committee member) / Arizona State University (Publisher)
Created2022
Description
Toward the ambitious long-term goal of a fleet of cooperating Flexible Autonomous Machines operating in an uncertain Environment (FAME), this thesis addresses several
critical modeling, design and control objectives for ground vehicles. One central objective was to show how off-the-shelf (low-cost) remote-control (RC) “toy” vehicles can be converted into intelligent multi-capability robotic-platforms for conducting FAME research. This is shown for two vehicle classes: (1) six differential-drive (DD) RC vehicles called Thunder Tumbler (DDTT) and (2) one rear-wheel drive (RWD) RC car called Ford F-150 (1:14 scale). Each DDTT-vehicle was augmented to provide a substantive suite of capabilities as summarized below (It should be noted, however, that only one DDTT-vehicle was augmented with an inertial measurement unit (IMU) and 2.4 GHz RC capability): (1) magnetic wheel-encoders/IMU for(dead-reckoning-based) inner-loop speed-control and outer-loop position-directional-control, (2) Arduino Uno microcontroller-board for encoder-based inner-loop speed-control and encoder-IMU-ultrasound-based outer-loop cruise-position-directional-separation-control, (3) Arduino motor-shield for inner-loop motor-speed-control, (4)Raspberry Pi II computer-board for demanding outer-loop vision-based cruise- position-directional-control, (5) Raspberry Pi 5MP camera for outer-loop cruise-position-directional-control (exploiting WiFi to send video back to laptop), (6) forward-pointing ultrasonic distance/rangefinder sensor for outer-loop separation-control, and (7) 2.4 GHz spread-spectrum RC capability to replace original 27/49 MHz RC. Each “enhanced”/ augmented DDTT-vehicle costs less than 175 but offers the capability of commercially available vehicles costing over 500. Both the Arduino and Raspberry are low-cost, well-supported (software wise) and easy-to-use. For the vehicle classes considered (i.e. DD, RWD), both kinematic and dynamical (planar xy) models are examined. Suitable nonlinear/linear-models are used to develop inner/outer-loopcontrol laws.
All demonstrations presented involve enhanced DDTT-vehicles; one the F-150; one a quadrotor. The following summarizes key hardware demonstrations: (1) cruise-control along line, (2) position-control along line (3) position-control along curve (4) planar (xy) Cartesian stabilization, (5) cruise-control along jagged line/curve, (6) vehicle-target spacing-control, (7) multi-robot spacing-control along line/curve, (8) tracking slowly-moving remote-controlled quadrotor, (9) avoiding obstacle while moving toward target, (10) RC F-150 followed by DDTT-vehicle. Hardware data/video is compared with, and corroborated by, model-based simulations. In short, many capabilities that are critical for reaching the longer-term FAME goal are demonstrated.
critical modeling, design and control objectives for ground vehicles. One central objective was to show how off-the-shelf (low-cost) remote-control (RC) “toy” vehicles can be converted into intelligent multi-capability robotic-platforms for conducting FAME research. This is shown for two vehicle classes: (1) six differential-drive (DD) RC vehicles called Thunder Tumbler (DDTT) and (2) one rear-wheel drive (RWD) RC car called Ford F-150 (1:14 scale). Each DDTT-vehicle was augmented to provide a substantive suite of capabilities as summarized below (It should be noted, however, that only one DDTT-vehicle was augmented with an inertial measurement unit (IMU) and 2.4 GHz RC capability): (1) magnetic wheel-encoders/IMU for(dead-reckoning-based) inner-loop speed-control and outer-loop position-directional-control, (2) Arduino Uno microcontroller-board for encoder-based inner-loop speed-control and encoder-IMU-ultrasound-based outer-loop cruise-position-directional-separation-control, (3) Arduino motor-shield for inner-loop motor-speed-control, (4)Raspberry Pi II computer-board for demanding outer-loop vision-based cruise- position-directional-control, (5) Raspberry Pi 5MP camera for outer-loop cruise-position-directional-control (exploiting WiFi to send video back to laptop), (6) forward-pointing ultrasonic distance/rangefinder sensor for outer-loop separation-control, and (7) 2.4 GHz spread-spectrum RC capability to replace original 27/49 MHz RC. Each “enhanced”/ augmented DDTT-vehicle costs less than 175 but offers the capability of commercially available vehicles costing over 500. Both the Arduino and Raspberry are low-cost, well-supported (software wise) and easy-to-use. For the vehicle classes considered (i.e. DD, RWD), both kinematic and dynamical (planar xy) models are examined. Suitable nonlinear/linear-models are used to develop inner/outer-loopcontrol laws.
All demonstrations presented involve enhanced DDTT-vehicles; one the F-150; one a quadrotor. The following summarizes key hardware demonstrations: (1) cruise-control along line, (2) position-control along line (3) position-control along curve (4) planar (xy) Cartesian stabilization, (5) cruise-control along jagged line/curve, (6) vehicle-target spacing-control, (7) multi-robot spacing-control along line/curve, (8) tracking slowly-moving remote-controlled quadrotor, (9) avoiding obstacle while moving toward target, (10) RC F-150 followed by DDTT-vehicle. Hardware data/video is compared with, and corroborated by, model-based simulations. In short, many capabilities that are critical for reaching the longer-term FAME goal are demonstrated.
ContributorsLin, Zhenyu (Author) / Rodriguez, Armando Antonio (Committee member) / Si, Jennie (Committee member) / Berman, Spring Melody (Committee member) / Arizona State University (Publisher)
Created2015