Matching Items (2)
Filtering by

Clear all filters

171816-Thumbnail Image.png

Combining learning with knowledge-rich planning allows for efficient multi-agent solutions to the problem of perpetual sparse rewards

Description
This work has improved the quality of the solution to the sparse rewards problemby combining reinforcement learning (RL) with knowledge-rich planning. Classical methods for coping with sparse rewards during reinforcement learning modify the reward landscape so as to better guide the learner. In contrast, this work combines RL with a planner in order…

This work has improved the quality of the solution to the sparse rewards problemby combining reinforcement learning (RL) with knowledge-rich planning. Classical methods for coping with sparse rewards during reinforcement learning modify the reward landscape so as to better guide the learner. In contrast, this work combines RL with a planner in order to utilize other information about the environment. As the scope for representing environmental information is limited in RL, this work has conflated a model-free learning algorithm – temporal difference (TD) learning – with a Hierarchical Task Network (HTN) planner to accommodate rich environmental information in the algorithm. In the perpetual sparse rewards problem, rewards reemerge after being collected within a fixed interval of time, culminating in a lack of a well-defined goal state as an exit condition to the problem. Incorporating planning in the learning algorithm not only improves the quality of the solution, but the algorithm also avoids the ambiguity of incorporating a goal of maximizing profit while using only a planning algorithm to solve this problem. Upon occasionally using the HTN planner, this algorithm provides the necessary tweak toward the optimal solution. In this work, I have demonstrated an on-policy algorithm that has improved the quality of the solution over vanilla reinforcement learning. The objective of this work has been to observe the capacity of the synthesized algorithm in finding optimal policies to maximize rewards, awareness of the environment, and the awareness of the presence of other agents in the vicinity.
ContributorsNandan, Swastik (Author) / Pavlic, Theodore (Thesis advisor) / Das, Jnaneshwar (Thesis advisor) / Berman, Spring (Committee member) / Arizona State University (Publisher)
Created2022
158901-Thumbnail Image.png

Design, Development, and Modeling, of a Novel Underwater Vehicle for Autonomous Reef Monitoring

Description
A novel underwater, open source, and configurable vehicle that mimics and leverages advances in quad-copter controls and dynamics, called the uDrone, was designed, built and tested. This vehicle was developed to aid coral reef researchers in collecting underwater spectroscopic data for the purpose of monitoring coral reef health. It is…

A novel underwater, open source, and configurable vehicle that mimics and leverages advances in quad-copter controls and dynamics, called the uDrone, was designed, built and tested. This vehicle was developed to aid coral reef researchers in collecting underwater spectroscopic data for the purpose of monitoring coral reef health. It is designed with an on-board integrated sensor system to support both automated navigation in close proximity to reefs and environmental observation. Additionally, the vehicle can serve as a testbed for future research in the realm of programming for autonomous underwater navigation and data collection, given the open-source simulation and software environment in which it was developed. This thesis presents the motivation for and design components of the new vehicle, a model governing vehicle dynamics, and the results of two proof-of-concept simulation for automated control.
ContributorsGoldman, Alex (Author) / Das, Jnaneshwar (Thesis advisor) / Asner, Greg (Committee member) / Marvi, Hamidreza (Committee member) / Arizona State University (Publisher)
Created2020