Matching Items (18)
Filtering by
- All Subjects: engineering
- All Subjects: Reinforcement Learning
- Creators: Berman, Spring
Description
The purpose of this project is to determine the feasibility of a water tunnel designed to meet certain constraints. The project goals are to tailor a design for a given location, and to produce a repeatable design sizing and shape process for specified constraints. The primary design goals include a 1 m/s flow velocity in a 30cm x 30cm test section for 300 seconds. Secondary parameters, such as system height, tank height, area contraction ratio, and roof loading limits, may change depending on preference, location, or environment. The final chosen configuration is a gravity fed design with six major components: the reservoir tank, the initial duct, the contraction nozzle, the test section, the exit duct, and the variable control exit nozzle. Important sizing results include a minimum water weight of 60,000 pounds, a system height of 7.65 meters, a system length of 6 meters (not including the reservoir tank), a large shallow reservoir tank width of 12.2 meters, and height of 0.22 meters, and a control nozzle exit radius range of 5.25 cm to 5.3 cm. Computational fluid dynamic simulation further supports adherence to the design constraints but points out some potential areas for improvement in dealing with flow irregularities. These areas include the bends in the ducts, and the contraction nozzle. Despite those areas recommended for improvement, it is reasonable to conclude that the design and process fulfill the project goals.
ContributorsZykan, Brandt Davis Healy (Author) / Wells, Valana (Thesis director) / Middleton, James (Committee member) / Barrett, The Honors College (Contributor) / Mechanical and Aerospace Engineering Program (Contributor)
Created2014-05
Description
This thesis presents a process by which a controller used for collective transport tasks is qualitatively studied and probed for presence of undesirable equilibrium states that could entrap the system and prevent it from converging to a target state. Fields of study relevant to this project include dynamic system modeling, modern control theory, script-based system simulation, and autonomous systems design. Simulation and computational software MATLAB and Simulink® were used in this thesis.
To achieve this goal, a model of a swarm performing a collective transport task in a bounded domain featuring convex obstacles was simulated in MATLAB/ Simulink®. The closed-loop dynamic equations of this model were linearized about an equilibrium state with angular acceleration and linear acceleration set to zero. The simulation was run over 30 times to confirm system ability to successfully transport the payload to a goal point without colliding with obstacles and determine ideal operating conditions by testing various orientations of objects in the bounded domain. An additional purely MATLAB simulation was run to identify local minima of the Hessian of the navigation-like potential function. By calculating this Hessian periodically throughout the system’s progress and determining the signs of its eigenvalues, a system could check whether it is trapped in a local minimum, and potentially dislodge itself through implementation of a stochastic term in the robot controllers. The eigenvalues of the Hessian calculated in this research suggested the model local minima were degenerate, indicating an error in the mathematical model for this system, which likely incurred during linearization of this highly nonlinear system.
To achieve this goal, a model of a swarm performing a collective transport task in a bounded domain featuring convex obstacles was simulated in MATLAB/ Simulink®. The closed-loop dynamic equations of this model were linearized about an equilibrium state with angular acceleration and linear acceleration set to zero. The simulation was run over 30 times to confirm system ability to successfully transport the payload to a goal point without colliding with obstacles and determine ideal operating conditions by testing various orientations of objects in the bounded domain. An additional purely MATLAB simulation was run to identify local minima of the Hessian of the navigation-like potential function. By calculating this Hessian periodically throughout the system’s progress and determining the signs of its eigenvalues, a system could check whether it is trapped in a local minimum, and potentially dislodge itself through implementation of a stochastic term in the robot controllers. The eigenvalues of the Hessian calculated in this research suggested the model local minima were degenerate, indicating an error in the mathematical model for this system, which likely incurred during linearization of this highly nonlinear system.
ContributorsBaye-Wallace, Lily Catherine (Author) / Berman, Spring (Thesis director) / Marvi, Hamidreza (Committee member) / Mechanical and Aerospace Engineering Program (Contributor) / School of Music, Dance and Theatre (Contributor) / School for Engineering of Matter,Transport & Enrgy (Contributor) / Barrett, The Honors College (Contributor)
Created2020-12
Description
In the next decade or so, there will be a shift in the industry of transportation across the world. Already today we have autonomous vehicles (AVs) tested in the Greater Phoenix area showing that the technology has improved to a level available to the public eye. Although this technology is not yet released commercially (for the most part), it is being used and will continue to be used to develop a safer future. With a high incidence of human error causing accidents, many expect that autonomous vehicles will be safer than human drivers. They do still require driver attention and sometimes intervention to ensure safety, but for the most part are much safer. In just the United States alone, there were 40,000 deaths due to car accidents last year [1]. If traffic fatalities were considered a disease, this would be an epidemic. The technology behind autonomous vehicles will allow for a much safer environment and increased mobility and independence for people who cannot drive and struggle with public transport. There are many opportunities for autonomous vehicles in the transportation industry. Companies can save a lot more money on shipping by cutting the costs of human drivers and trucks on the road, even allowing for simpler drop shipments should the necessary AI be developed.Research is even being done by several labs at Arizona State University. For example, Dr. Spring Berman’s Autonomous Collective Systems Lab has been collaborating with Dr. Nancy Cooke of Human Systems Engineering to develop a traffic testbed, CHARTopolis, to study the risks of driver-AV interactions and the psychological effects of AVs on human drivers on a small scale. This testbed will be used by researchers from their labs and others to develop testing on reaction, trust, and user experience with AVs in a safe environment that simulates conditions similar to those experienced by full-size AVs. Using a new type of small robot that emulates an AV, developed in Dr. Berman’s lab, participants will be able to remotely drive around a model city environment and interact with other AV-like robots using the cameras and LiDAR sensors on the remotely driven robot to guide them.
Although these commercial and research systems are still in testing, it is important to understand how AVs are being marketed to the general public and how they are perceived, so that one day they may be effectively adopted into everyday life. People do not want to see a car they do not trust on the same roads as them, so the questions are: why don’t people trust them, and how can companies and researchers improve the trustworthiness of the vehicles?
Although these commercial and research systems are still in testing, it is important to understand how AVs are being marketed to the general public and how they are perceived, so that one day they may be effectively adopted into everyday life. People do not want to see a car they do not trust on the same roads as them, so the questions are: why don’t people trust them, and how can companies and researchers improve the trustworthiness of the vehicles?
ContributorsShuster, Daniel Nadav (Author) / Berman, Spring (Thesis director) / Cooke, Nancy (Committee member) / Mechanical and Aerospace Engineering Program (Contributor) / Barrett, The Honors College (Contributor)
Created2019-05
Description
This thesis details the design and construction of a torque-controlled robotic gripper for use with the Pheeno swarm robotics platform. This project required expertise from several fields of study including: robotic design, programming, rapid prototyping, and control theory. An electronic Inertial Measurement Unit and a DC Motor were both used along with 3D printed plastic components and an electronic motor control board to develop a functional open-loop controlled gripper for use in collective transportation experiments. Code was developed that effectively acquired and filtered rate of rotation data alongside other code that allows for straightforward control of the DC motor through experimentally derived relationships between the voltage applied to the DC motor and the torque output of the DC motor. Additionally, several versions of the physical components are described through their development.
ContributorsMohr, Brennan (Author) / Berman, Spring (Thesis director) / Ren, Yi (Committee member) / Mechanical and Aerospace Engineering Program (Contributor) / School for Engineering of Matter,Transport & Enrgy (Contributor) / Barrett, The Honors College (Contributor)
Created2019-05
Description
In this paper, we propose an autonomous throwing and catching system to be developed as a preliminary step towards the refinement of a robotic arm capable of improving strength and motor function in the limb. This will be accomplished by first autonomizing simpler movements, such as throwing a ball. In this system, an autonomous thrower will detect a desired target through the use of image processing. The launch angle and direction necessary to hit the target will then be calculated, followed by the launching of the ball. The smart catcher will then detect the ball as it is in the air, calculate its expected landing location based on its initial trajectory, and adjust its position so that the ball lands in the center of the target. The thrower will then proceed to compare the actual landing position with the position where it expected the ball to land, and adjust its calculations accordingly for the next throw. By utilizing this method of feedback, the throwing arm will be able to automatically correct itself. This means that the thrower will ideally be able to hit the target exactly in the center within a few throws, regardless of any additional uncertainty in the system. This project will focus of the controller and image processing components necessary for the autonomous throwing arm to be able to detect the position of the target at which it will be aiming, and for the smart catcher to be able to detect the position of the projectile and estimate its final landing position by tracking its current trajectory.
ContributorsLundberg, Kathie Joy (Co-author) / Thart, Amanda (Co-author) / Rodriguez, Armando (Thesis director) / Berman, Spring (Committee member) / Electrical Engineering Program (Contributor) / Barrett, The Honors College (Contributor)
Created2018-05
Description
A Graph Neural Network (GNN) is a type of neural network architecture that operates on data consisting of objects and their relationships, which are represented by a graph. Within the graph, nodes represent objects and edges represent associations between those objects. The representation of relationships and correlations between data is unique to graph structures. GNNs exploit this feature of graphs by augmenting both forms of data, individual and relational, and have been designed to allow for communication and sharing of data within each neural network layer. These benefits allow each node to have an enriched perspective, or a better understanding, of its neighbouring nodes and its connections to those nodes. The ability of GNNs to efficiently process high-dimensional node data and multi-faceted relationships among nodes gives them advantages over neural network architectures such as Convolutional Neural Networks (CNNs) that do not implicitly handle relational data. These quintessential characteristics of GNN models make them suitable for solving problems in which the correspondences among input data are needed to produce an accurate and precise representation of these data. GNN frameworks may significantly improve existing communication and control techniques for multi-agent tasks by implicitly representing not only information associated with the individual agents, such as agent position, velocity, and camera data, but also their relationships with one another, such as distances between the agents and their ability to communicate with one another. One such task is a multi-agent navigation problem in which the agents must coordinate with one another in a decentralized manner, using proximity sensors only, to navigate safely to their intended goal positions in the environment without collisions or deadlocks. The contribution of this thesis is the design of an end-to-end decentralized control scheme for multi-agent navigation that utilizes GNNs to prevent inter-agent collisions and deadlocks. The contributions consist of the development, simulation and evaluation of the performance of an advantage actor-critic (A2C) reinforcement learning algorithm that employs actor and critic networks for training that simultaneously approximate the policy function and value function, respectively. These networks are implemented using GNN frameworks for navigation by groups of 3, 5, 10 and 15 agents in simulated two-dimensional environments. It is observed that in $40\%$ to $50\%$ of the simulation trials, between 70$\%$ to 80$\%$ of the agents reach their goal positions without colliding with other agents or becoming trapped in deadlocks. The model is also compared to a random run simulation, where actions are chosen randomly for the agents and observe that the model performs notably well for smaller groups of agents.
ContributorsAyalasomayajula, Manaswini (Author) / Berman, Spring (Thesis advisor) / Mian, Sami (Committee member) / Pavlic, Theodore (Committee member) / Arizona State University (Publisher)
Created2022
Description
Navigation and mapping in GPS-denied environments, such as coal mines ordilapidated buildings filled with smog or particulate matter, pose a significant challenge
due to the limitations of conventional LiDAR or vision systems. Therefore there
exists a need for a navigation algorithm and mapping strategy which do not use vision
systems but are still able to explore and map the environment. The map can further
be used by first responders and cave explorers to access the environments.
This thesis presents the design of a collision-resilient Unmanned Aerial Vehicle
(UAV), XPLORER that utilizes a novel navigation algorithm for exploration and
simultaneous mapping of the environment. The real-time navigation algorithm uses
the onboard Inertial Measurement Units (IMUs) and arm bending angles for contact
estimation and employs an Explore and Exploit strategy. Additionally, the quadrotor
design is discussed, highlighting its improved stability over the previous design.
The generated map of the environment can be utilized by autonomous vehicles to
navigate the environment. The navigation algorithm is validated in multiple real-time
experiments in different scenarios consisting of concave and convex corners and circular
objects. Furthermore, the developed mapping framework can serve as an auxiliary
input for map generation along with conventional LiDAR or vision-based mapping
algorithms.
Both the navigation and mapping algorithms are designed to be modular, making
them compatible with conventional UAVs also. This research contributes to the
development of navigation and mapping techniques for GPS-denied environments,
enabling safer and more efficient exploration of challenging territories.
ContributorsPandian Saravanakumaran, Aravind Adhith (Author) / Zhang, Wenlong (Thesis advisor) / Das, Jnaneshwar (Committee member) / Berman, Spring (Committee member) / Arizona State University (Publisher)
Created2023
Description
Multibody Dynamic (MBD) models are important tools in motion analysis and are used to represent and accurately predict the behavior of systems in the real-world. These models have a range of applications, including the stowage and deployment of flexible deployables on spacecraft, the dynamic response of vehicles in automotive design and crash testing, and mapping interactions of the human body. An accurate model can aid in the design of a system to ensure the system is effective and meets specified performance criteria when built. A model may have many design parameters, such as geometrical constraints and component mechanical properties, or controller parameters if the system uses an external controller. Varying these parameters and rerunning analyses by hand to find an ideal design can be time consuming for models that take hours or days to run. To reduce the amount of time required to find a set of parameters that produces a desired performance, optimization is necessary. Many papers have discussed methods for optimizing rigid and flexible MBD models, and separately their controllers, using both gradient-based and gradient-free algorithms. However, these optimization methods have not been used to optimize full-scale MBD models and their controllers simultaneously. This thesis presents a method for co-optimizing an MBD model and controller that allows for the flexibility to find model and controller-based solutions for systems with tightly coupled parameters. Specifically, the optimization is performed on a quadrotor drone MBD model undergoing disturbance from a slung load and its position controller to meet specified position error performance criteria. A gradient-free optimization algorithm and multiple objective approach is used due to the many local optima from the tradeoffs between the model and controller parameters. The thesis uses nine different quadrotor cases with three different position error formulations. The results are used to determine the effectiveness of the optimization and the ability to converge on a single optimal design. After reviewing the results, the optimization limitations are discussed as well as the ability to transition the optimization to work with different MBD models and their controllers.
ContributorsGambatese, Marcus (Author) / Zhang, Wenlong (Thesis advisor) / Berman, Spring (Committee member) / Inoyama, Daisaku (Committee member) / Arizona State University (Publisher)
Created2022
Description
This work has improved the quality of the solution to the sparse rewards problemby combining reinforcement learning (RL) with knowledge-rich planning. Classical
methods for coping with sparse rewards during reinforcement learning modify the
reward landscape so as to better guide the learner. In contrast, this work combines
RL with a planner in order to utilize other information about the environment. As
the scope for representing environmental information is limited in RL, this work has
conflated a model-free learning algorithm – temporal difference (TD) learning – with
a Hierarchical Task Network (HTN) planner to accommodate rich environmental
information in the algorithm. In the perpetual sparse rewards problem, rewards
reemerge after being collected within a fixed interval of time, culminating in a lack of a
well-defined goal state as an exit condition to the problem. Incorporating planning in
the learning algorithm not only improves the quality of the solution, but the algorithm
also avoids the ambiguity of incorporating a goal of maximizing profit while using
only a planning algorithm to solve this problem. Upon occasionally using the HTN
planner, this algorithm provides the necessary tweak toward the optimal solution. In
this work, I have demonstrated an on-policy algorithm that has improved the quality
of the solution over vanilla reinforcement learning. The objective of this work has
been to observe the capacity of the synthesized algorithm in finding optimal policies to
maximize rewards, awareness of the environment, and the awareness of the presence
of other agents in the vicinity.
ContributorsNandan, Swastik (Author) / Pavlic, Theodore (Thesis advisor) / Das, Jnaneshwar (Thesis advisor) / Berman, Spring (Committee member) / Arizona State University (Publisher)
Created2022
Incorporating Causal Information using Temporal-Logic-Based Causal Diagram in Reinforcement Learning
Description
In this thesis, I investigate a subset of reinforcement learning (RL) tasks where the objective for the agent is to achieve temporally extended goals. A common approach, in this setting, is to represent the tasks using deterministic finite automata (DFA) and integrate them in the state space of the RL algorithms, yet such representations often disregard causal knowledge pertinent to the environment. To address this limitation, I introduce the Temporal-Logic-based Causal Diagram (TL-CD) in RL.TL-CD encapsulates temporal causal relationships among diverse environmental properties. We leverage the TL-CD to devise an RL algorithm that significantly reduces environment exploration requirements. By synergizing TL-CD with task-specific DFAs, I identify scenarios wherein the agent can efficiently determine expected rewards early during the exploration phases. Through a series of case studies, I empirically demonstrate the advantages of TL-CDs, particularly highlighting the accelerated convergence of the algorithm towards an optimal policy facilitated by diminished exploration of the environment.
ContributorsPaliwal, Yash (Author) / Xu, Zhe (Thesis advisor) / Marvi, Hamidreza (Committee member) / Berman, Spring (Committee member) / Arizona State University (Publisher)
Created2024