Matching Items (3)
Filtering by
- All Subjects: Control Theory
- Creators: Nedich, Angelia
Description
Coordination and control of Intelligent Agents as a team is considered in this thesis.
Intelligent agents learn from experiences, and in times of uncertainty use the knowl-
edge acquired to make decisions and accomplish their individual or team objectives.
Agent objectives are defined using cost functions designed uniquely for the collective
task being performed. Individual agent costs are coupled in such a way that group ob-
jective is attained while minimizing individual costs. Information Asymmetry refers
to situations where interacting agents have no knowledge or partial knowledge of cost
functions of other agents. By virtue of their intelligence, i.e., by learning from past
experiences agents learn cost functions of other agents, predict their responses and
act adaptively to accomplish the team’s goal.
Algorithms that agents use for learning others’ cost functions are called Learn-
ing Algorithms, and algorithms agents use for computing actuation (control) which
drives them towards their goal and minimize their cost functions are called Control
Algorithms. Typically knowledge acquired using learning algorithms is used in con-
trol algorithms for computing control signals. Learning and control algorithms are
designed in such a way that the multi-agent system as a whole remains stable during
learning and later at an equilibrium. An equilibrium is defined as the event/point
where cost functions of all agents are optimized simultaneously. Cost functions are
designed so that the equilibrium coincides with the goal state multi-agent system as
a whole is trying to reach.
In collective load transport, two or more agents (robots) carry a load from point
A to point B in space. Robots could have different control preferences, for example,
different actuation abilities, however, are still required to coordinate and perform
load transport. Control preferences for each robot are characterized using a scalar
parameter θ i unique to the robot being considered and unknown to other robots.
With the aid of state and control input observations, agents learn control preferences
of other agents, optimize individual costs and drive the multi-agent system to a goal
state.
Two learning and Control algorithms are presented. In the first algorithm(LCA-
1), an existing work, each agent optimizes a cost function similar to 1-step receding
horizon optimal control problem for control. LCA-1 uses recursive least squares as
the learning algorithm and guarantees complete learning in two time steps. LCA-1 is
experimentally verified as part of this thesis.
A novel learning and control algorithm (LCA-2) is proposed and verified in sim-
ulations and on hardware. In LCA-2, each agent solves an infinite horizon linear
quadratic regulator (LQR) problem for computing control. LCA-2 uses a learning al-
gorithm similar to line search methods, and guarantees learning convergence to true
values asymptotically.
Simulations and hardware implementation show that the LCA-2 is stable for a
variety of systems. Load transport is demonstrated using both the algorithms. Ex-
periments running algorithm LCA-2 are able to resist disturbances and balance the
assumed load better compared to LCA-1.
Intelligent agents learn from experiences, and in times of uncertainty use the knowl-
edge acquired to make decisions and accomplish their individual or team objectives.
Agent objectives are defined using cost functions designed uniquely for the collective
task being performed. Individual agent costs are coupled in such a way that group ob-
jective is attained while minimizing individual costs. Information Asymmetry refers
to situations where interacting agents have no knowledge or partial knowledge of cost
functions of other agents. By virtue of their intelligence, i.e., by learning from past
experiences agents learn cost functions of other agents, predict their responses and
act adaptively to accomplish the team’s goal.
Algorithms that agents use for learning others’ cost functions are called Learn-
ing Algorithms, and algorithms agents use for computing actuation (control) which
drives them towards their goal and minimize their cost functions are called Control
Algorithms. Typically knowledge acquired using learning algorithms is used in con-
trol algorithms for computing control signals. Learning and control algorithms are
designed in such a way that the multi-agent system as a whole remains stable during
learning and later at an equilibrium. An equilibrium is defined as the event/point
where cost functions of all agents are optimized simultaneously. Cost functions are
designed so that the equilibrium coincides with the goal state multi-agent system as
a whole is trying to reach.
In collective load transport, two or more agents (robots) carry a load from point
A to point B in space. Robots could have different control preferences, for example,
different actuation abilities, however, are still required to coordinate and perform
load transport. Control preferences for each robot are characterized using a scalar
parameter θ i unique to the robot being considered and unknown to other robots.
With the aid of state and control input observations, agents learn control preferences
of other agents, optimize individual costs and drive the multi-agent system to a goal
state.
Two learning and Control algorithms are presented. In the first algorithm(LCA-
1), an existing work, each agent optimizes a cost function similar to 1-step receding
horizon optimal control problem for control. LCA-1 uses recursive least squares as
the learning algorithm and guarantees complete learning in two time steps. LCA-1 is
experimentally verified as part of this thesis.
A novel learning and control algorithm (LCA-2) is proposed and verified in sim-
ulations and on hardware. In LCA-2, each agent solves an infinite horizon linear
quadratic regulator (LQR) problem for computing control. LCA-2 uses a learning al-
gorithm similar to line search methods, and guarantees learning convergence to true
values asymptotically.
Simulations and hardware implementation show that the LCA-2 is stable for a
variety of systems. Load transport is demonstrated using both the algorithms. Ex-
periments running algorithm LCA-2 are able to resist disturbances and balance the
assumed load better compared to LCA-1.
ContributorsKAMBAM, KARTHIK (Author) / Zhang, Wenlong (Thesis advisor) / Nedich, Angelia (Thesis advisor) / Ren, Yi (Committee member) / Arizona State University (Publisher)
Created2018
Description
In this thesis, we focus on some of the NP-hard problems in control theory. Thanks to the converse Lyapunov theory, these problems can often be modeled as optimization over polynomials. To avoid the problem of intractability, we establish a trade off between accuracy and complexity. In particular, we develop a sequence of tractable optimization problems - in the form of Linear Programs (LPs) and/or Semi-Definite Programs (SDPs) - whose solutions converge to the exact solution of the NP-hard problem. However, the computational and memory complexity of these LPs and SDPs grow exponentially with the progress of the sequence - meaning that improving the accuracy of the solutions requires solving SDPs with tens of thousands of decision variables and constraints. Setting up and solving such problems is a significant challenge. The existing optimization algorithms and software are only designed to use desktop computers or small cluster computers - machines which do not have sufficient memory for solving such large SDPs. Moreover, the speed-up of these algorithms does not scale beyond dozens of processors. This in fact is the reason we seek parallel algorithms for setting-up and solving large SDPs on large cluster- and/or super-computers.
We propose parallel algorithms for stability analysis of two classes of systems: 1) Linear systems with a large number of uncertain parameters; 2) Nonlinear systems defined by polynomial vector fields. First, we develop a distributed parallel algorithm which applies Polya's and/or Handelman's theorems to some variants of parameter-dependent Lyapunov inequalities with parameters defined over the standard simplex. The result is a sequence of SDPs which possess a block-diagonal structure. We then develop a parallel SDP solver which exploits this structure in order to map the computation, memory and communication to a distributed parallel environment. Numerical tests on a supercomputer demonstrate the ability of the algorithm to efficiently utilize hundreds and potentially thousands of processors, and analyze systems with 100+ dimensional state-space. Furthermore, we extend our algorithms to analyze robust stability over more complicated geometries such as hypercubes and arbitrary convex polytopes. Our algorithms can be readily extended to address a wide variety of problems in control such as Hinfinity synthesis for systems with parametric uncertainty and computing control Lyapunov functions.
We propose parallel algorithms for stability analysis of two classes of systems: 1) Linear systems with a large number of uncertain parameters; 2) Nonlinear systems defined by polynomial vector fields. First, we develop a distributed parallel algorithm which applies Polya's and/or Handelman's theorems to some variants of parameter-dependent Lyapunov inequalities with parameters defined over the standard simplex. The result is a sequence of SDPs which possess a block-diagonal structure. We then develop a parallel SDP solver which exploits this structure in order to map the computation, memory and communication to a distributed parallel environment. Numerical tests on a supercomputer demonstrate the ability of the algorithm to efficiently utilize hundreds and potentially thousands of processors, and analyze systems with 100+ dimensional state-space. Furthermore, we extend our algorithms to analyze robust stability over more complicated geometries such as hypercubes and arbitrary convex polytopes. Our algorithms can be readily extended to address a wide variety of problems in control such as Hinfinity synthesis for systems with parametric uncertainty and computing control Lyapunov functions.
ContributorsKamyar, Reza (Author) / Peet, Matthew (Thesis advisor) / Berman, Spring (Committee member) / Rivera, Daniel (Committee member) / Artemiadis, Panagiotis (Committee member) / Fainekos, Georgios (Committee member) / Arizona State University (Publisher)
Created2016
Description
Modern life is full of challenging optimization problems that we unknowingly attempt to solve. For instance, a common dilemma often encountered is the decision of picking a parking spot while trying to minimize both the distance to the goal destination and time spent searching for parking; one strategy is to drive as close as possible to the goal destination but risk a penalty cost if no parking spaces can be found. Optimization problems of this class all have underlying time-varying processes that can be altered by a decision/input to minimize some cost. Such optimization problems are commonly solved by a class of methods called Dynamic Programming (DP) that breaks down a complex optimization problem into a simpler family of sub-problems. In the 1950s Richard Bellman introduced a class of DP methods that broke down Multi-Stage Optimization Problems (MSOP) into a nested sequence of ``tail problems”. Bellman showed that for any MSOP with a cost function that satisfies a condition called additive separability, the solution to the tail problem of the MSOP initialized at time-stage k>0 can be used to solve the tail problem initialized at time-stage k-1. Therefore, by recursively solving each tail problem of the MSOP, a solution to the original MSOP can be found. This dissertation extends Bellman`s theory to a broader class of MSOPs involving non-additively separable costs by introducing a new state augmentation solution method and generalizing the Bellman Equation. This dissertation also considers the analogous continuous-time counterpart to discrete-time MSOPs, called Optimal Control Problems (OCPs). OCPs can be solved by solving a nonlinear Partial Differential Equation (PDE) called the Hamilton-Jacobi-Bellman (HJB) PDE. Unfortunately, it is rarely possible to obtain an analytical solution to the HJB PDE. This dissertation proposes a method for approximately solving the HJB PDE based on Sum-Of-Squares (SOS) programming. This SOS algorithm can be used to synthesize controllers, hence solving the OCP, and also compute outer bounds of reachable sets of dynamical systems. This methodology is then extended to infinite time horizons, by proposing SOS algorithms that yield Lyapunov functions that can approximate regions of attraction and attractor sets of nonlinear dynamical systems arbitrarily well.
ContributorsJones, Morgan (Author) / Peet, Matthew M (Thesis advisor) / Nedich, Angelia (Committee member) / Kawski, Matthias (Committee member) / Mignolet, Marc (Committee member) / Berman, Spring (Committee member) / Arizona State University (Publisher)
Created2021