Matching Items (2)
Filtering by
- Member of: ASU Electronic Theses and Dissertations
- Status: Published
Description
Traditional Reinforcement Learning (RL) assumes to learn policies with respect to reward available from the environment but sometimes learning in a complex domain requires wisdom which comes from a wide range of experience. In behavior based robotics, it is observed that a complex behavior can be described by a combination of simpler behaviors. It is tempting to apply similar idea such that simpler behaviors can be combined in a meaningful way to tailor the complex combination. Such an approach would enable faster learning and modular design of behaviors. Complex behaviors can be combined with other behaviors to create even more advanced behaviors resulting in a rich set of possibilities. Similar to RL, combined behavior can keep evolving by interacting with the environment. The requirement of this method is to specify a reasonable set of simple behaviors. In this research, I present an algorithm that aims at combining behavior such that the resulting behavior has characteristics of each individual behavior. This approach has been inspired by behavior based robotics, such as the subsumption architecture and motor schema-based design. The combination algorithm outputs n weights to combine behaviors linearly. The weights are state dependent and change dynamically at every step in an episode. This idea is tested on discrete and continuous environments like OpenAI’s “Lunar Lander” and “Biped Walker”. Results are compared with related domains like Multi-objective RL, Hierarchical RL, Transfer learning, and basic RL. It is observed that the combination of behaviors is a novel way of learning which helps the agent achieve required characteristics. A combination is learned for a given state and so the agent is able to learn faster in an efficient manner compared to other similar approaches. Agent beautifully demonstrates characteristics of multiple behaviors which helps the agent to learn and adapt to the environment. Future directions are also suggested as possible extensions to this research.
ContributorsVora, Kevin Jatin (Author) / Zhang, Yu (Thesis advisor) / Yang, Yezhou (Committee member) / Praharaj, Sarbeswar (Committee member) / Arizona State University (Publisher)
Created2021
Description
Obesity impairs skeletal muscle maintenance and regeneration, a condition that can progressively lead to muscle loss, but the mechanisms behind it are unknown. Muscle is primarily composed of multinucleated cells called myotubes which are derived by the fusion of mononucleated myocytes. A key mediator in this process is the cellular fusion protein syncytin-1. This led to the hypothesis that syncytin-1 could be decreased in the muscle of obese/insulin resistant individuals. In contrast, it was found that obese/insulin resistant subjects had higher syncytin-1 expression in the muscle compared to that of the lean subjects. Across the subjects, syncytin-1 correlated significantly with body mass index, percent body fat, blood glucose and HbA1c levels, insulin sensitivity and muscle protein fractional synthesis rate. The concentrations of specific plasma fatty acids, such as the saturated fatty acid (palmitate) and monounsaturated fatty acid (oleate) are known to be altered in obese/insulin resistant humans, and also to influence the protein synthesis in muscle. Therefore, it was evaluated that the effects of palmitate and oleate on syncytin-1 expression, as well as 4E-BP1 phosphorylation, a key mechanism regulating muscle protein synthesis in insulin stimulated C2C12 myotubes. The results showed that treatment with 20 nM insulin, 300 µM oleate, 300 µM oleate +20 nM insulin and 300 µM palmitate + 300 µM oleate elevated 4E-BP1 phosphorylation. At the same time, 20 nM insulin, 300 µM palmitate, 300 µM oleate + 20 nM insulin and 300 µM palmitate + 300 µM oleate elevated syncytin-1 expression. Insulin stimulated muscle syncytin-1 expression and 4E-BP1 phosphorylation, and this effect was comparable to that observed in the presence of oleate alone. However, the presence of palmitate + oleate diminished the stimulatory effect of insulin on muscle syncytin-1 expression and 4E-BP1 phosphorylation. These findings indicate oleate but not palmitate increased total 4E-BP1 phosphorylation regardless of insulin and the presence of palmitate in insulin mediated C2C12 cells. The presence of palmitate inhibited the upregulation of total 4EB-P1 phosphorylation. Palmitate but not oleate increased syncytin-1 expression in insulin mediated C2C12 myotubes. It is possible that chronic hyperinsulinemia in obesity and/or elevated levels of fatty acids such as palmitate in plasma could have contributed to syncytin-1 overexpression and decreased muscle protein fractional synthesis rate in obese/insulin resistant human muscle.
ContributorsRavichandran, Jayachandran (Author) / Katsanos, Christos (Thesis advisor) / Coletta, Dawn (Committee member) / Dickinson, Jared (Committee member) / Arizona State University (Publisher)
Created2017