Source author record

Matthew T. Hale

Matthew T. Hale appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record

math.OC

Catalog footprint

What is connected

5works

1topics

4close collaborators

Actions

Connect this record

Open graph Browse works

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

preprint2022arXiv

Faster Asynchronous Nonconvex Block Coordinate Descent with Locally Chosen Stepsizes

Distributed nonconvex optimization problems underlie many applications in learning and autonomy, and such problems commonly face asynchrony in agents' computations and communications. When delays in these operations are bounded, they are called partially asynchronous. In this paper, we present an uncoordinated stepsize selection rule for partially asynchronous block coordinate descent that only requires local information to implement, and it leads to faster convergence for a class of nonconvex problems than existing stepsize rules, which require global information in some form. The problems we consider satisfy the error bound condition, and the stepsize rule we present only requires each agent to know (i) a certain type of Lipschitz constant of its block of the gradient of the objective and (ii) the communication delays experienced between it and its neighbors. This formulation requires less information to be available to each agent than existing approaches, typically allows for agents to use much larger stepsizes, and alleviates the impact of stragglers while still guaranteeing convergence to a stationary point. Simulation results provide comparisons and validate the faster convergence attained by the stepsize rule we develop.

preprint2022arXiv

Linear Regularizers Enforce the Strict Saddle Property

Satisfaction of the strict saddle property has become a standard assumption in non-convex optimization, and it ensures that many first-order optimization algorithms will almost always escape saddle points. However, functions exist in machine learning that do not satisfy this property, such as the loss function of a neural network with at least two hidden layers. First-order methods such as gradient descent may converge to non-strict saddle points of such functions, and there do not currently exist any first-order methods that reliably escape non-strict saddle points. To address this need, we demonstrate that regularizing a function with a linear term enforces the strict saddle property, and we provide justification for only regularizing locally, i.e., when the norm of the gradient falls below a certain threshold. We analyze bifurcations that may result from this form of regularization, and then we provide a selection rule for regularizers that depends only on the gradient of an objective function. This rule is shown to guarantee that gradient descent will escape the neighborhoods around a broad class of non-strict saddle points, and this behavior is demonstrated on numerical examples of non-strict saddle points common in the optimization literature.

preprint2020arXiv

Totally Asynchronous Large-Scale Quadratic Programming: Regularization, Convergence Rates, and Parameter Selection

Quadratic programs arise in robotics, communications, smart grids, and many other applications. As these problems grow in size, finding solutions becomes more computationally demanding, and new algorithms are needed to efficiently solve them at massive scales. Targeting large-scale problems, we develop a multi-agent quadratic programming framework in which each agent updates only a small number of the total decision variables in a problem. Agents communicate their updated values to each other, though we do not impose any restrictions on the timing with which they do so, nor on the delays in these transmissions. Furthermore, we allow weak parametric coupling among agents, in the sense that they are free to independently choose their step sizes, subject to mild restrictions. We further provide the means for agents to independently regularize the problems they solve, thereby improving convergence properties while preserving agents' independence in selecting parameters and ensuring a global bound on regularization error is satisfied. Larger regularizations accelerate convergence but increase error in the solution obtained, and we quantify the trade off between convergence rates and quality of solutions. Simulation results are presented to illustrate these developments.

preprint2016arXiv

Asynchronous Multi-Agent Primal-Dual Optimization

We present a framework for asynchronously solving convex optimization problems over networks of agents which are augmented by the presence of a centralized cloud computer. This framework uses a Tikhonov-regularized primal-dual approach in which the agents update the system's primal variables and the cloud updates its dual variables. To minimize coordination requirements placed upon the system, the times of communications and computations among the agents are allowed to be arbitrary, provided they satisfy mild conditions. Communications from the agents to the cloud are likewise carried out without any coordination in their timing. However, we require that the cloud keep the dual variable's value synchronized across the agents, and a counterexample is provided that demonstrates that this level of synchrony is indeed necessary for convergence. Convergence rate estimates are provided in both the primal and dual spaces, and simulation results are presented that demonstrate the operation and convergence of the proposed algorithm.

preprint2015arXiv

Cloud-Based Centralized/Decentralized Multi-Agent Optimization with Communication Delays

We present and analyze a computational hybrid architecture for performing multi-agent optimization. The optimization problems under consideration have convex objective and constraint functions with mild smoothness conditions imposed on them. For such problems, we provide a primal-dual algorithm implemented in the hybrid architecture, which consists of a decentralized network of agents into which centralized information is occasionally injected, and we establish its convergence properties. To accomplish this, a central cloud computer aggregates global information, carries out computations of the dual variables based on this information, and then distributes the updated dual variables to the agents. The agents update their (primal) state variables and also communicate among themselves with each agent sharing and receiving state information with some number of its neighbors. Throughout, communications with the cloud are not assumed to be synchronous or instantaneous, and communication delays are explicitly accounted for in the modeling and analysis of the system. Experimental results are presented to support the theoretical developments made.