math.OC

State-Space Representation of Hysteresis Systems Exhibiting the Return Point Memory

Application of the minimal state-space realization to hysteresis systems is studied. The method allows to construct the space of states and establish the state transition rules using the input equivalence, which can be obtained for hysteresis systems basing on rate independence and the return point memory.

Kalman-based Stochastic Gradient Method with Stop Condition and Insensitivity to Conditioning

Modern proximal and stochastic gradient descent (SGD) methods are believed to efficiently minimize large composite objective functions, but such methods have two algorithmic challenges: (1) a lack of fast or justified stop conditions, and (2) sensitivity to the objective function's conditioning. In response to the first challenge, modern proximal and SGD methods guarantee convergence only after multiple epochs, but such a guarantee renders proximal and SGD methods infeasible when the number of component functions is very large or infinite. In response to the second challenge, second order SGD methods have been developed, but they are marred by the complexity of their analysis. In this work, we address these challenges on the limited, but important, linear regression problem by introducing and analyzing a second order proximal/SGD method based on Kalman Filtering (kSGD). Through our analysis, we show kSGD is asymptotically optimal, develop a fast algorithm for very large, infinite or streaming data sources with a justified stop condition, prove that kSGD is insensitive to the problem's conditioning, and develop a unique approach for analyzing the complex second order dynamic

The Monge problem with vanishing gradient penalization: Vortices and asymptotic profile

We investigate the approximation of the Monge problem (minimizing \int\_ $Ω$ |T (x) -- x| d $μ$ (x) among the vector-valued maps T with prescribed image measure T \# $μ$ ) by adding a vanishing Dirichlet energy, namely $ε$ \int\_ $Ω$ |DT |^2. We study the $Γ$ -convergence as $ε$ $\rightarrow$ 0, proving a density result for Sobolev (or Lipschitz) transport maps in the class of transport plans. In a certain two-dimensional framework that we analyze in details, when no optimal plan is induced by an H ^1 map, we study the selected limit map, which is a new "special" Monge transport, possibly different from the monotone one, and we find the precise asymptotics of the optimal cost depending on $ε$ , where the leading term is of order $ε$ | log $ε$ |.

Robust Mean Field Linear-Quadratic-Gaussian Games with Unknown $L^2$ -Disturbance

This paper considers a class of mean field linear-quadratic-Gaussian (LQG) games with model uncertainty. The drift term in the dynamics of the agents contains a common unknown function. We take a robust optimization approach where a representative agent in the limiting model views the drift uncertainty as an adversarial player. By including the mean field dynamics in an augmented state space, we solve two optimal control problems sequentially, which combined with consistent mean field approximations provides a solution to the robust game. A set of decentralized control strategies is derived by use of forward-backward stochastic differential equations (FBSDE) and shown to be a robust epsilon-Nash equilibrium.

preprint2007arXiv

Pareto Optima of Multicriteria Integer Linear Programs

We settle the computational complexity of fundamental questions related to multicriteria integer linear programs, when the dimensions of the strategy space and of the outcome space are considered fixed constants. In particular we construct: 1. polynomial-time algorithms to exactly determine the number of Pareto optima and Pareto strategies; 2. a polynomial-space polynomial-delay prescribed-order enumeration algorithm for arbitrary projections of the Pareto set; 3. an algorithm to minimize the distance of a Pareto optimum from a prescribed comparison point with respect to arbitrary polyhedral norms; 4. a fully polynomial-time approximation scheme for the problem of minimizing the distance of a Pareto optimum from a prescribed comparison point with respect to the Euclidean norm.

preprint2009arXiv

Nonlinear Integer Programming

Research efforts of the past fifty years have led to a development of linear integer programming as a mature discipline of mathematical optimization. Such a level of maturity has not been reached when one considers nonlinear systems subject to integrality requirements for the variables. This chapter is dedicated to this topic. The primary goal is a study of a simple version of general nonlinear integer problems, where all constraints are still linear. Our focus is on the computational complexity of the problem, which varies significantly with the type of nonlinear objective function in combination with the underlying combinatorial structure. Numerous boundary cases of complexity emerge, which sometimes surprisingly lead even to polynomial time algorithms. We also cover recent successful approaches for more general classes of problems. Though no positive theoretical efficiency results are available, nor are they likely to ever be available, these seem to be the currently most successful and interesting approaches for solving practical problems. It is our belief that the study of algorithms motivated by theoretical considerations and those motivated by our desire to solve practical i

Toward computer-assisted discovery and automated proofs of cutting plane theorems

Using a metaprogramming technique and semialgebraic computations, we provide computer-based proofs for old and new cutting-plane theorems in Gomory--Johnson's model of cut generating functions.

preprint2014arXiv

A convex solution to Psiaki's first joint attitude and spin-rate estimation problem

We consider the problem of jointly estimating the attitude and spin-rate of a spinning spacecraft. Psiaki (J. Astronautical Sci., 57(1-2):73--92, 2009) has formulated a family of optimization problems that generalize the classical least-squares attitude estimation problem, known as Wahba's problem, to the case of a spinning spacecraft. If the rotation axis is fixed and known, but the spin-rate is unknown (such as for nutation-damped spin-stabilized spacecraft) we show that Psiaki's problem can be reformulated exactly as a type of tractable convex optimization problem called a semidefinite optimization problem. This reformulation allows us to globally solve the problem using standard numerical routines for semidefinite optimization. It also provides a natural semidefinite relaxation-based approach to more complicated variations on the problem.

Nonlinear Flows for Displacement Correction and Applications in Tomography

In this paper we derive nonlinear evolution equations associated with a class of non-convex energy functionals which can be used for correcting displacement errors in imaging data. We study properties of these filtering flows and provide experiments for correcting angular perturbations in tomographical data.

Fooling Sets and the Spanning Tree Polytope

In the study of extensions of polytopes of combinatorial optimization problems, a notorious open question is that for the size of the smallest extended formulation of the Minimum Spanning Tree problem on a complete graph with $n$ nodes. The best known lower bound is $Ω(n^2)$ , the best known upper bound is $O(n^3)$ . In this note we show that the venerable fooling set method cannot be used to improve the lower bound: every fooling set for the Spanning Tree polytope has size $O(n^2)$ .

Inverse Protein Folding Problem via Quadratic Programming

This paper presents a method of reconstruction a primary structure of a protein that folds into a given geometrical shape. This method predicts the primary structure of a protein and restores its linear sequence of amino acids in the polypeptide chain using the tertiary structure of a molecule. Unknown amino acids are determined according to the principle of energy minimization. This study represents inverse folding problem as a quadratic optimization problem and uses different relaxation techniques to reduce it to the problem of convex optimizations. Computational experiment compares the quality of these approaches on real protein structures.

preprint2009arXiv

A parametric integer programming algorithm for bilevel mixed integer programs

We consider discrete bilevel optimization problems where the follower solves an integer program with a fixed number of variables. Using recent results in parametric integer programming, we present polynomial time algorithms for pure and mixed integer bilevel problems. For the mixed integer case where the leader's variables are continuous, our algorithm also detects whether the infimum cost fails to be attained, a difficulty that has been identified but not directly addressed in the literature. In this case it yields a ``better than fully polynomial time'' approximation scheme with running time polynomial in the logarithm of the relative precision. For the pure integer case where the leader's variables are integer, and hence optimal solutions are guaranteed to exist, we present two algorithms which run in polynomial time when the total number of variables is fixed.

Learning from Conditional Distributions via Dual Embeddings

Many machine learning tasks, such as learning with invariance and policy evaluation in reinforcement learning, can be characterized as problems of learning from conditional distributions. In such problems, each sample $x$ itself is associated with a conditional distribution $p(z|x)$ represented by samples $\{z_i\}_{i=1}^M$ , and the goal is to learn a function $f$ that links these conditional distributions to target values $y$ . These learning problems become very challenging when we only have limited samples or in the extreme case only one sample from each conditional distribution. Commonly used approaches either assume that $z$ is independent of $x$ , or require an overwhelmingly large samples from each conditional distribution. To address these challenges, we propose a novel approach which employs a new min-max reformulation of the learning from conditional distribution problem. With such new reformulation, we only need to deal with the joint distribution $p(z,x)$ . We also design an efficient learning algorithm, Embedding-SGD, and establish theoretical sample complexity for such problems. Finally, our numerical experiments on both synthetic and real-world datasets show that the pro

preprint2012arXiv

Graver basis and proximity techniques for block-structured separable convex integer minimization problems

We consider N-fold 4-block decomposable integer programs, which simultaneously generalize N-fold integer programs and two-stage stochastic integer programs with N scenarios. In previous work [R. Hemmecke, M. Koeppe, R. Weismantel, A polynomial-time algorithm for optimizing over N-fold 4-block decomposable integer programs, Proc. IPCO 2010, Lecture Notes in Computer Science, vol. 6080, Springer, 2010, pp. 219--229], it was proved that for fixed blocks but variable N, these integer programs are polynomial-time solvable for any linear objective. We extend this result to the minimization of separable convex objective functions. Our algorithm combines Graver basis techniques with a proximity result [D.S. Hochbaum and J.G. Shanthikumar, Convex separable optimization is not much harder than linear optimization, J. ACM 37 (1990), 843--862], which allows us to use convex continuous optimization as a subroutine.

Stability analysis of delay differential equations via Semidefinite programming

This paper studies the problem of stability of a parameterized delay differential equations (DDE see equation (0.1)). After discretizing the DDE (0.1), we show that the problem can be equivalently casted into a semi-definite programming (SDP) see (3.2), which can be solved efficiently through some popular algorithm, e.g., the interior point method [1].

preprint2014arXiv

Classical and strong convexity of sublevel sets and application to attainable sets of nonlinear systems

Necessary and sufficient conditions for convexity and strong convexity, respectively, of sublevel sets that are defined by finitely many real-valued $C^{1,1}$ -maps are presented. A novel characterization of strongly convex sets in terms of the so-called local quadratic support is proved. The results concerning strong convexity are used to derive sufficient conditions for attainable sets of continuous-time nonlinear systems to be strongly convex. An application of these conditions is a novel method to over-approximate attainable sets when strong convexity is present.

Equivariant Perturbation in Gomory and Johnson's Infinite Group Problem. III. Foundations for the k-Dimensional Case with Applications to k=2

We develop foundational tools for classifying the extreme valid functions for the k-dimensional infinite group problem. In particular, (1) we present the general regular solution to Cauchy's additive functional equation on bounded convex domains. This provides a k-dimensional generalization of the so-called interval lemma, allowing us to deduce affine properties of the function from certain additivity relations. (2) We study the discrete geometry of additivity domains of piecewise linear functions, providing a framework for finite tests of minimality and extremality. (3) We give a theory of non-extremality certificates in the form of perturbation functions. We apply these tools in the context of minimal valid functions for the two-dimensional infinite group problem that are piecewise linear on a standard triangulation of the plane, under the assumption of a regularity condition called diagonal constrainedness. We show that the extremality of a minimal valid function is equivalent to the extremality of its restriction to a certain finite two-dimensional group problem. This gives an algorithm for testing the extremality of a given minimal valid function.

Constructing numerically stable Kalman filter-based algorithms for gradient-based adaptive filtering

This paper addresses the numerical aspects of adaptive filtering (AF) techniques for simultaneous state and parameters estimation arising in the design of dynamic positioning systems in many areas of research. The AF schemes consist of a recursive optimization procedure to identify the uncertain system parameters by minimizing an appropriate defined performance index and the application of the Kalman filter (KF) for dynamic positioning purpose. The use of gradient-based optimization methods in the AF computational schemes yields to a set of the filter sensitivity equations and a set of matrix Riccati-type sensitivity equations. The filter sensitivities evaluation is usually done by the conventional KF, which is known to be numerically unstable, and its derivatives with respect to unknown system parameters. Recently, a novel square-root approach for the gradient-based AF by the method of the maximum likelihood has been proposed. In this paper, we show that various square-root AF schemes can be derived from only two main theoretical results. This elegant and simple computational technique replaces the standard methodology based on direct differentiation of the conventional KF equatio

preprint2005arXiv

FPTAS for mixed-integer polynomial optimization with a fixed number of variables

We show the existence of an FPTAS for the problem of maximizing a non-negative polynomial over mixed-integer sets in convex polytopes, when the number of variables is fixed.

preprint2006arXiv

Intermediate integer programming representations using value disjunctions

We introduce a general technique to create an extended formulation of a mixed-integer program. We classify the integer variables into blocks, each of which generates a finite set of vector values. The extended formulation is constructed by creating a new binary variable for each generated value. Initial experiments show that the extended formulation can have a more compact complete description than the original formulation. We prove that, using this reformulation technique, the facet description decomposes into one ``linking polyhedron'' per block and the ``aggregated polyhedron''. Each of these polyhedra can be analyzed separately. For the case of identical coefficients in a block, we provide a complete description of the linking polyhedron and a polynomial-time separation algorithm. Applied to the knapsack with a fixed number of distinct coefficients, this theorem provides a complete description in an extended space with a polynomial number of variables.

Simulated Tornado Optimization

We propose a swarm-based optimization algorithm inspired by air currents of a tornado. Two main air currents - spiral and updraft - are mimicked. Spiral motion is designed for exploration of new search areas and updraft movements is deployed for exploitation of a promising candidate solution. Assignment of just one search direction to each particle at each iteration, leads to low computational complexity of the proposed algorithm respect to the conventional algorithms. Regardless of the step size parameters, the only parameter of the proposed algorithm, called tornado diameter, can be efficiently adjusted by randomization. Numerical results over six different benchmark cost functions indicate comparable and, in some cases, better performance of the proposed algorithm respect to some other metaheuristics.

preprint2014arXiv

Adaptive Augmented Lagrangian Methods: Algorithms and Practical Numerical Experience

In this paper, we consider augmented Lagrangian (AL) algorithms for solving large-scale nonlinear optimization problems that execute adaptive strategies for updating the penalty parameter. Our work is motivated by the recently proposed adaptive AL trust region method by Curtis, Jiang, and Robinson [Math. Prog., DOI: 10.1007/s10107-014-0784-y, 2013]. The first focal point of this paper is a new variant of the approach that employs a line search rather than a trust region strategy, where a critical algorithmic feature for the line search strategy is the use of convexified piecewise quadratic models of the AL function for computing the search directions. We prove global convergence guarantees for our line search algorithm that are on par with those for the previously proposed trust region method. A second focal point of this paper is the practical performance of the line search and trust region algorithm variants in Matlab software, as well as that of an adaptive penalty parameter updating strategy incorporated into the Lancelot software. We test these methods on problems from the CUTEst and COPS collections, as well as on challenging test problems related to optimal power flow. Our n

An Extension of Chubanov's Polynomial-Time Linear Programming Algorithm to Second-Order Cone Programming

Recently, Chubanov proposed an interesting new polynomial-time algorithm for linear program. In this paper, we extend his algorithm to second-order cone programming.

Distributed Linearized Alternating Direction Method of Multipliers for Composite Convex Consensus Optimization

Given an undirected graph $\mathcal{G}=(\mathcal{N},\mathcal{E})$ of agents $\mathcal{N}=\{1,\ldots,N\}$ connected with edges in $\mathcal{E}$ , we study how to compute an optimal decision on which there is consensus among agents and that minimizes the sum of agent-specific private convex composite functions $\{Φ_i\}_{i\in\mathcal{N}}$ while respecting privacy requirements, where $Φ_i\triangleq ξ_i + f_i$ belongs to agent- $i$ . Assuming only agents connected by an edge can communicate, we propose a distributed proximal gradient method DPGA for consensus optimization over both unweighted and weighted static (undirected) communication networks. In one iteration, each agent- $i$ computes the prox map of $ξ_i$ and gradient of $f_i$ , and this is followed by local communication with neighboring agents. We also study its stochastic gradient variant, SDPGA, which can only access to noisy estimates of $\nabla f_i$ at each agent- $i$ . This computational model abstracts a number of applications in distributed sensing, machine learning and statistical inference. We show ergodic convergence in both sub-optimality error and consensus violation for DPGA and SDPGA with rates $\mathcal{O}(1/t)$ and $\m

Inspect nearby papers, researchers, institutions and communities without opening a separate graph page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

575 works

Y. Wang

Researcher

Y. Wang contributes to research discovery and scholarly infrastructure.

464 works

Wei Zhang

Researcher

Wei Zhang contributes to research discovery and scholarly infrastructure.

390 works

Wei Chen

Researcher

Wei Chen contributes to research discovery and scholarly infrastructure.

374 works

H. Vincent Poor

Researcher

H. Vincent Poor contributes to research discovery and scholarly infrastructure.

360 works

Yu Zhang

Researcher

Yu Zhang contributes to research discovery and scholarly infrastructure.

356 works

Jun Zhang

Researcher

Jun Zhang contributes to research discovery and scholarly infrastructure.

339 works

Xi Chen

Researcher

Xi Chen contributes to research discovery and scholarly infrastructure.

335 works

Feng Liu

Researcher

Feng Liu contributes to research discovery and scholarly infrastructure.

321 works

Lei Wang

Researcher

Lei Wang contributes to research discovery and scholarly infrastructure.

319 works

Xin Wang

Researcher

Xin Wang contributes to research discovery and scholarly infrastructure.

300 works

C. Zhang

Researcher

C. Zhang contributes to research discovery and scholarly infrastructure.

293 works

Wei Liu

Researcher

Wei Liu contributes to research discovery and scholarly infrastructure.