BZPEERPapers, people, evidence and research pathsDiscover
Research tools
Network
Opportunities
Account
Source author record
Heinz H. Bauschke appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.
Catalog footprint
Research graph
Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.
BZPEER is loading the nearby papers, people, topics and institutions for this page.
Published work
In 1996, Meshulam proved that every sequence generated by applying projections onto affine subspaces, drawn from a finite collection in Euclidean space, must be bounded. In this paper, we extend his result not only from affine subspaces to convex polyhedral subsets, but also from Euclidean to general Hilbert space. Various examples are provided to illustrate the sharpness of the results.
In 2023, Güntürk and Thao proved that the sequence $(x^{(n)})_{n\in\mathbb{N}}$ generated by random (relaxed) projections drawn from a finite collection of innately regular closed subspaces in a real Hilbert space satisfies $\sum_{n\in\mathbb{N}} \|x^{(n)}-x^{(n+1)}\|^γ<+\infty$ for all $γ>0$. We extend their result to a finite collection of polyhedral cones. Moreover, we construct examples showing the tightness of our extension: indeed, the result fails for a line and a convex set in $\mathbb{R}^2$, and for a plane and a non-polyhedral cone in $\mathbb{R}^3$.
Let $A$ be a closed affine subspace and let $B$ be a hyperplane in a Hilbert space. Suppose we are given their associated nearest point mappings $P_A$ and $P_B$, respectively. We present a formula for the projection onto their intersection $A\cap B$. As a special case, we derive a formula for the projection onto the intersection of two hyperplanes. These formulas provides useful information even if $A\cap B$ is empty. Examples and numerical experiments are also provided.
Recently, Simons provided a lemma for a support function of a closed convex set in a general Hilbert space and used it to prove the geometry conjecture on cycles of projections. In this paper, we extend Simons's lemma to closed convex functions, show its connections to Attouch-Thera duality, and use it to characterize (classical and phantom) cycles and gap vectors of proximal mappings.
We provide a proximal average with repect to a $1$-coercive Legendre function. In the sense of Bregman distance, the Bregman envelope of the proximal average is a convex combination of Bregman envelopes of individual functions. The Bregman proximal mapping of the average is a convex combination of convexified proximal mappings of individual functions. Techniques from variational analysis provide the keys for the Bregman proximal average.
Let $C$ be a closed convex subset of a real Hilbert space containing the origin, and assume that $K$ is the homogenization cone of $C$, i.e., the smallest closed convex cone containing $C \times \{1\}$. Homogenization cones play an important role in optimization as they include, for instance, the second-order/Lorentz/"ice cream" cone. In this note, we discuss the polar cone of $K$ as well as an algorithm for finding the projection onto $K$ provided that the projection onto $C$ is available. Various examples illustrate our results.
We consider the set of pairs of orthogonal vectors in Hilbert space, which is also called the cross because it is the union of the horizontal and vertical axes in the Euclidean plane when the underlying space is the real line. Crosses, which are nonconvex sets, play a significant role in various branches of nonsmooth analysis such as feasibility problems and optimization problems. In this work, we study crosses and show that in infinite-dimensional settings, they are never weakly (sequentially) closed. Nonetheless, crosses do turn out to be proximinal (i.e., they always admit projections) and we provide explicit formulas for the projection onto the cross in all cases.
The notion of best approximation mapping (BAM) with respect to a closed affine subspace in finite-dimensional space was introduced by Behling, Bello Cruz and Santos to show the linear convergence of the block-wise circumcentered-reflection method. The best approximation mapping possesses two critical properties of the circumcenter mapping for linear convergence. Because the iteration sequence of BAM linearly converges, the BAM is interesting in its own right. In this paper, we naturally extend the definition of BAM from closed affine subspace to nonempty closed convex set and from $\mathbb{R}^{n}$ to general Hilbert space. We discover that the convex set associated with the BAM must be the fixed point set of the BAM. Hence, the iteration sequence generated by a BAM linearly converges to the nearest fixed point of the BAM. Connections between BAMs and other mappings generating convergent iteration sequences are considered. Behling et al.\ proved that the finite composition of BAMs associated with closed affine subspaces is still a BAM in $\mathbb{R}^{n}$. We generalize their result from $\mathbb{R}^{n}$ to general Hilbert space and also construct a new constant associated with the composition of BAMs. This provides a new proof of the linear convergence of the method of alternating projections. Moreover, compositions of BAMs associated with general convex sets are investigated. In addition, we show that convex combinations of BAMs associated with affine subspaces are BAMs. Last but not least, we connect BAM with circumcenter mapping in Hilbert spaces.
Motivated by the circumcentered Douglas--Rachford method recently introduced by Behling, Bello Cruz and Santos to accelerate the Douglas--Rachford method, we study the properness of the circumcenter mapping and the circumcenter method induced by isometries. Applying the demiclosedness principle for circumcenter mappings, we present weak convergence results for circumcentered isometry methods, which include the Douglas--Rachford method (DRM) and circumcentered reflection methods as special instances. We provide sufficient conditions for the linear convergence of circumcentered isometry/reflection methods. We explore the convergence rate of circumcentered reflection methods by considering the required number of iterations and as well as run time as our performance measures. Performance profiles on circumcentered reflection methods, DRM and method of alternating projections for finding the best approximation to the intersection of linear subspaces are presented.
In 1964, Michael Edelstein presented an amazing affine isometry acting on the space of square-summable sequences. This operator has no fixed points, but a suborbit that converges to 0 while another escapes in norm to infinity! We revisit, extend and sharpen his construction. Moreover, we sketch a connection to modern optimization and monotone operator theory.
Nonexpansive mappings play a central role in modern optimization and monotone operator theory because their fixed points can describe solutions to optimization or critical point problems. It is known that when the mappings are sufficiently "nice", then the fixed point set of the composition coincides with the intersection of the individual fixed point sets. In this paper, we explore the situation for compositions of linear reflectors. We provide positive results, upper bounds, and limiting examples. We also discuss classical reflectors in the Euclidean plane.
A popular method for finding the projection onto the intersection of two closed convex subsets in Hilbert space is Dykstra's algorithm. In this paper, we provide sufficient conditions for Dykstra's algorithm to converge rapidly, in finitely many steps. We also analyze the behaviour of Dykstra's algorithm applied to a line and a square. This case study reveals stark similarities to the method of alternating projections. Moreover, we show that Dykstra's algorithm may stall for an arbitrarily long time. Finally, we present some open problems.
The Douglas-Rachford algorithm (DRA) is a powerful optimization method for minimizing the sum of two convex (not necessarily smooth) functions. The vast majority of previous research dealt with the case when the sum has at least one minimizer. In the absence of minimizers, it was recently shown that for the case of two indicator functions, the DRA converges to a best approximation solution. In this paper, we present a new convergence result on the the DRA applied to the problem of minimizing a convex function subject to a linear constraint. Indeed, a normal solution may be found even when the domain of the objective function and the linear subspace constraint have no point in common. As an important application, a new parallel splitting result is provided. We also illustrate our results through various examples.
The circumcentered Douglas--Rachford method (C--DRM), introduced by Behling, Bello Cruz and Santos, is an acceleration of the well-known Douglas-Rachford method (DRM) for finding the best approximation onto the intersection of finitely many affine subspaces. Inspired by the C--DRM, we introduced the more flexible circumcentered reflection method (CRM) and circumcentered isometry method (CIM). The CIM essentially chooses the closest point to the solution among all of the points in an associated affine hull as its iterate and is a generalization of the CRM. The circumcentered--reflection method introduced by Behling et al. to generalize the C--DRM is a special class of our CRM. We consider the CIM induced by a set of finitely many isometries for finding the best approximation onto the intersection of fixed point sets of the isometries which turns out to be an intersection of finitely many affine subspaces. We extend our previous linear convergence results on CRMs in finite-dimensional spaces from reflections to isometries. In order to better accelerate the symmetric method of alternating projections (MAP), the accelerated symmetric MAP first applies another operator to the initial point. (Similarly, to accelerate the DRM, the C--DRM first applies another operator to the initial point as well.) Motivated by these facts, we show results on the linear convergence of CIMs in Hilbert spaces with first applying another operator to the initial point. In particular, under some restrictions, our results imply that some CRMs attain the known linear convergence rate of the accelerated symmetric MAP in Hilbert spaces. We also exhibit a class of CRMs converging to the best approximation in Hilbert spaces with a convergence rate no worse than the sharp convergence rate of MAP. The fact that some CRMs attain the linear convergence rate of MAP or accelerated symmetric MAP is entirely new.
Moreau's seminal paper, introducing what is now called the Moreau envelope and the proximity operator (also known as the proximal mapping), appeared in 1965. The Moreau envelope of a given convex function provides a regularized version which has additional desirable properties such as differentiability and full domain. Fifty years ago, Attouch proposed using the Moreau envelope for regularization. Since then, this branch of convex analysis has developed in many fruitful directions. In 1967, Bregman introduced what is nowadays known as the Bregman distance as a measure of discrepancy between two points generalizing the square of the Euclidean distance. Proximity operators based on the Bregman distance have become a topic of significant research as they are useful in the algorithmic solution of optimization problems. More recently, in 2012, Kan and Song studied regularization aspects of the left Bregman-Moreau envelope even for nonconvex functions. In this paper, we complement previous works by analyzing the left and right Bregman-Moreau envelopes and by providing additional asymptotic results. Several examples are provided.
The Douglas--Rachford algorithm is a popular algorithm for solving both convex and nonconvex feasibility problems. While its behaviour is settled in the convex inconsistent case, the general nonconvex inconsistent case is far from being fully understood. In this paper, we focus on the most simple nonconvex inconsistent case: when one set is a hyperplane and the other a doubleton (i.e., a two-point set). We present a characterization of cycling in this case which --- somewhat surprisingly --- depends on whether the ratio of the distance of the points to the hyperplane is rational or not. Furthermore, we provide closed-form expressions as well as several concrete examples which illustrate the dynamical richness of this algorithm.
The Douglas-Rachford splitting algorithm was originally proposed in 1956 to solve a system of linear equations arising from the discretization of a partial differential equation. In 1979, Lions and Mercier brought forward a very powerful extension of this method suitable to solve optimization problems. In this paper, we revisit the original affine setting. We provide a powerful convergence result for finding a zero of the sum of two maximally monotone affine relations. As a by product of our analysis, we obtain results concerning the convergence of iterates of affine nonexpansive mappings as well as Attouch-Théra duality. Numerous examples are presented.
The problem of finding a zero of the sum of two maximally monotone operators is of central importance in optimization. One successful method to find such a zero is the Douglas-Rachford algorithm which iterates a firmly nonexpansive operator constructed from the resolvents of the given monotone operators. In the context of finding minimizers of convex functions, the resolvents are actually proximal mappings. Interestingly, as pointed out by Eckstein in 1989, the Douglas-Rachford operator itself may fail to be a proximal mapping. We consider the class of symmetric linear relations that are maximally monotone and prove the striking result that the Douglas-Rachford operator is generically not a proximal mapping.
The Douglas-Rachford algorithm is a very popular splitting technique for finding a zero of the sum of two maximally monotone operators. However, the behaviour of the algorithm remains mysterious in the general inconsistent case, i.e., when the sum problem has no zeros. More than a decade ago, however, it was shown that in the (possibly inconsistent) convex feasibility setting, the shadow sequence remains bounded and it is weak cluster points solve a best approximation problem. In this paper, we advance the understanding of the inconsistent case significantly by providing a complete proof of the full weak convergence in the convex feasibility setting. In fact, a more general sufficient condition for the weak convergence in the general case is presented. Several examples illustrate the results.
Solving feasibility problems is a central task in mathematics and the applied sciences. One particularly successful method is the Douglas-Rachford algorithm. In this paper, we provide many new conditions sufficient for finite convergence. Numerous examples illustrate our results.
We introduce and investigate the resolvent order, which is a binary relation on the set of firmly nonexpansive mappings. It unifies well-known orders introduced by Loewner (for positive semidefinite matrices) and by Zarantonello (for projectors onto convex cones). A connection with Moreau's order of convex functions is also presented. We also construct partial orders on (quotient sets of) proximal mappings and convex functions. Various examples illustrate our results.
In 1971, Pazy presented a beautiful trichotomy result concerning the asymptotic behaviour of the iterates of a nonexpansive mapping. In this note, we analyze the fixed-point free case in more detail. Our results and examples give credence to the conjecture that the iterates always converge cosmically.
The Douglas-Rachford algorithm is a classical and very successful method for solving optimization and feasibility problems. In this paper, we provide novel conditions sufficient for finite convergence in the context of convex feasibility problems. Our analysis builds upon, and considerably extends, pioneering work by Spingarn. Specifically, we obtain finite convergence in the presence of Slater's condition in the affine-polyhedral and in a hyperplanar-epigraphical case. Various examples illustrate our results. Numerical experiments demonstrate the competitiveness of the Douglas-Rachford algorithm for solving linear equations with a positivity constraint when compared to the method of alternating projections and the method of reflection-projection.
The Douglas-Rachford algorithm is a popular method for finding zeros of sums of monotone operators. By its definition, the Douglas-Rachford operator is not symmetric with respect to the order of the two operators. In this paper we provide a systematic study of the two possible Douglas-Rachford operators. We show that the reflectors of the underlying operators act as bijections between the fixed points sets of the two Douglas-Rachford operators. Some elegant formulae arise under additional assumptions. Various examples illustrate our results.
Many iterative methods for solving optimization or feasibility problems have been invented, and often convergence of the iterates to some solution is proven. Under favourable conditions, one might have additional bounds on the distance of the iterate to the solution leading thus to worst case estimates, i.e., how fast the algorithm must converge. Exact convergence estimates are typically hard to come by. In this paper, we consider the complementary problem of finding best case estimates, i.e., how slow the algorithm has to converge, and we also study exact asymptotic rates of convergence. Our investigation focuses on convex feasibility in the Euclidean plane, where one set is the real axis while the other is the epigraph of a convex function. This case study allows us to obtain various convergence rate results. We focus on the popular method of alternating projections and the Douglas-Rachford algorithm. These methods are connected to the proximal point algorithm which is also discussed. Our findings suggest that the Douglas-Rachford algorithm outperforms the method of alternating projections in the absence of constraint qualifications. Various examples illustrate the theory.
The Douglas--Rachford algorithm is a classical and very successful splitting method for finding the zeros of the sums of monotone operators. When the underlying operators are normal cone operators, the algorithm solves a convex feasibility problem. In this paper, we provide a detailed study of the Douglas--Rachford iterates and the corresponding {shadow sequence} when the sets are affine subspaces that do not necessarily intersect. We prove strong convergence of the shadows to the nearest generalized solution. Our results extend recent work from the consistent to the inconsistent case. Various examples are provided to illustrates the results.
The Douglas-Rachford algorithm is a simple yet effective method for solving convex feasibility problems. However, if the underlying constraints are inconsistent, then the convergence theory is incomplete. We provide convergence results when one constraint is an affine subspace. As a consequence, we extend a result by Spingarn from halfspaces to general closed convex sets admitting least-squares solutions.
Within convex analysis, a rich theory with various applications has been evolving since the proximal average of convex functions was first introduced over a decade ago. When one considers the subdifferential of the proximal average, a natural averaging operation of the subdifferentials of the averaged functions emerges. In the present paper we extend the reach of this averaging operation to the framework of monotone operator theory in Hilbert spaces, transforming it into the resolvent average. The theory of resolvent averages contains many desirable properties. In particular, we study a detailed list of properties of monotone operators and classify them as dominant or recessive with respect to the resolvent average. As a consequence, we recover a significant part of the theory of proximal averages. Furthermore, we shed new light on the proximal average and present novel results and desirable properties the proximal average possesses which have not been previously available.
We introduce regularity notions for averaged nonexpansive operators. Combined with regularity notions of their fixed point sets, we obtain linear and strong convergence results for quasicyclic, cyclic, and random iterations. New convergence results on the Borwein-Tam method (BTM) and on the cylically anchored Douglas-Rachford algorithm (CADRA) are also presented. Finally, we provide a numerical comparison of BTM, CADRA and the classical method of cyclic projections for solving convex feasibility problems.
The subgradient projector is of considerable importance in convex optimization because it plays the key role in Polyak's seminal work - and the many papers it spawned - on subgradient projection algorithms for solving convex feasibility problems. In this paper, we offer a systematic study of the subgradient projector. Fundamental properties such as continuity, nonexpansiveness, and monotonicity are investigated. We also discuss the Yamagishi-Yamada operator. Numerous examples illustrate our results.
Functions that are piecewise defined are a common sight in mathematics while convexity is a property especially desired in optimization. Suppose now a piecewise-defined function is convex on each of its defining components - when can we conclude that the entire function is convex? In this paper we provide several convenient, verifiable conditions guaranteeing convexity (or the lack thereof). Several examples are presented to illustrate our results.
The subgradient projection iteration is a classical method for solving a convex inequality. Motivated by works of Polyak and of Crombez, we present and analyze a more general method for finding a fixed point of a cutter, provided that the fixed point set has nonempty interior. Our assumptions on the parameters are more general than existing ones. Various limiting examples and comparisons are provided.
The problem of finding a minimizer of the sum of two convex functions - or, more generally, that of finding a zero of the sum of two maximally monotone operators - is of central importance in variational analysis. Perhaps the most popular method of solving this problem is the Douglas-Rachford splitting method. Surprisingly, little is known about the range of the Douglas-Rachford operator. In this paper, we set out to study this range systematically. We prove that for 3* monotone operators a very pleasing formula can be found that reveals the range to be nearly equal to a simple set involving the domains and ranges of the underlying operators. A similar formula holds for the range of the corresponding displacement mapping. We discuss applications to subdifferential operators, to the infimal displacement vector, and to firmly nonexpansive mappings. Various examples and counter-examples are presented, including some concerning the celebrated Brezis-Haraux theorem.
We present a systematic study on the linear convergence rates of the powers of (real or complex) matrices. We derive a characterization when the optimal convergence rate is attained. This characterization is given in terms of semi-simpleness of all eigenvalues having the second-largest modulus after 1. We also provide applications of our general results to analyze the optimal convergence rates for several relaxed alternating projection methods and the generalized Douglas-Rachford splitting methods for finding the projection on the intersection of two subspaces. Numerical experiments confirm our convergence analysis.
The basic optimization problem of road design is quite challenging due to a objective function that is the sum of nonsmooth functions and the presence of set constraints. In this paper, we model and solve this problem by employing the Douglas-Rachford splitting algorithm. This requires a careful study of new proximity operators related to minimizing area and to the stadium norm. We compare our algorithm to a state-of-the-art projection algorithm. Our numerical results illustrate the potential of this algorithm to significantly reduce cost in road design.
We introduce an abstract algorithm that aims to find the Bregman projection onto a closed convex set. As an application, the asymptotic behaviour of an iterative method for finding a fixed point of a quasi Bregman nonexpansive mapping with the fixed-point closedness property is analyzed. We also show that our result is applicable to Bregman subgradient projectors.
A common theme in mathematics is to define generalized solutions to deal with problems that potentially do not have solutions. A classical example is the introduction of least squares solutions via the normal equations associated with a possibly infeasible system of linear equations. In this paper, we introduce a "normal problem" associated with finding a zero of the sum of two maximally monotone operators. If the original problem admits solutions, then the normal problem returns this same set of solutions. The normal problem may yield solutions when the original problem does not admit any; furthermore, it has attractive variational and duality properties. Several examples illustrate our theory.
Suppose that $A$ and $B$ are closed subsets of a Euclidean space such that $A\cap B\neq\varnothing$, and we aim to find a point in this intersection with the help of the sequences $(a_n)_\nnn$ and $(b_n)_\nnn$ generated by the \emph{method of alternating projections}. It is well known that if $A$ and $B$ are convex, then $(a_n)_\nnn$ and $(b_n)_\nnn$ converge to some point in $A\cap B$. The situation in the nonconvex case is much more delicate. In 1990, Combettes and Trussell presented a dichotomy result that guarantees either convergence to a point in the intersection or a nondegenerate compact continuum as the set of cluster points. In this note, we construct two sets in the Euclidean plane illustrating the continuum case. The sets $A$ and $B$ can be chosen as countably infinite unions of closed convex sets. In contrast, we also show that such behaviour is impossible for finite unions.
We model a problem motivated by road design as a feasibility problem. Projections onto the constraint sets are obtained, and projection methods for solving the feasibility problem are studied. We present results of numerical experiments which demonstrate the efficacy of projection methods even for challenging nonconvex problems.
The Method of Alternating Projections (MAP), a classical algorithm for solving feasibility prob- lems, has recently been intensely studied for nonconvex sets. However, intrinsically available are only local convergence results: convergence occurs if the starting point is not too far away from solutions to avoid getting trapped in certain regions. Instead of taking full projection steps, it can be advantageous to underrelax, i.e., to move only part way towards the constraint set, in order to enlarge the regions of convergence. In this paper, we thus systematically study the Method of Alternating Relaxed Projections (MARP) for two (possibly nonconvex) sets. Complementing our recent work on MAP, we es- tablish local linear convergence results for the MARP. Several examples illustrate our analysis.
The convex feasibility problem asks to find a point in the intersection of a collection of nonempty closed convex sets. This problem is of basic importance in mathematics and the physical sciences, and projection (or splitting) methods solve it by employing the projection operators associated with the individual sets to generate a sequence which converges to a solution. Motivated by an application in road design, we present the method of cyclic intrepid projections (CycIP) and provide a rigorous convergence analysis. We also report on very promising numerical experiments in which CycIP is compared to a commercial state-of-the-art optimization solver.
The Douglas-Rachford splitting algorithm is a classical optimization method that has found many applications. When specialized to two normal cone operators, it yields an algorithm for finding a point in the intersection of two convex sets. This method for solving feasibility problems has attracted a lot of attention due to its good performance even in nonconvex settings. In this paper, we consider the Douglas-Rachford algorithm for finding a point in the intersection of two subspaces. We prove that the method converges strongly to the projection of the starting point onto the intersection. Moreover, if the sum of the two subspaces is closed, then the convergence is linear with the rate being the cosine of the Friedrichs angle between the subspaces. Our results improve upon existing results in three ways: First, we identify the location of the limit and thus reveal the method as a best approximation algorithm; second, we quantify the rate of convergence, and third, we carry out our analysis in general (possibly infinite-dimensional) Hilbert space. We also provide various examples as well as a comparison with the classical method of alternating projections.
We consider $\min\{f(x):g(x) \le 0, ~x\in X\},$ where $X$ is a compact convex subset of $\RR^m$, and $f$ and $g$ are continuous convex functions defined on an open neighbourhood of $X$. We work in the setting of derivative-free optimization, assuming that $f$ and $g$ are available through a black-box that provides only function values for a lower-$\mathcal{C}^2$ representation of the functions. We present a derivative-free optimization variant of the $\eps$-comirror algorithm \cite{BBTGBT2010}. Algorithmic convergence hinges on the ability to accurately approximate subgradients of lower-$\mathcal{C}^2$ functions, which we prove is possible through linear interpolation. We provide convergence analysis that quantifies the difference between the function values of the iterates and the optimal function value. We find that the DFO algorithm we develop has the same convergence result as the original gradient-based algorithm. We present some numerical testing that demonstrate the practical feasibility of the algorithm, and conclude with some directions for further research.
We introduce and analyze an abstract algorithm that aims to find the projection onto a closed convex subset of a Hilbert space. When specialized to the fixed point set of a quasi nonexpansive mapping, the required sufficient condition (termed "fixed-point closed") is less restrictive than the usual conditions based on the demiclosedness principle. A concrete example of a subgradient projector is presented which illustrates the applicability of this generalization.
We show that the moving arithmetic average is closely connected to a Gauss-Seidel type fixed point method studied by Bauschke, Wang and Wylie, and which was observed to converge only numerically. Our analysis establishes a rigorous proof of convergence of their algorithm in a special case; moreover, limit is explicitly identified. Moving averages in Banach spaces and Kolmogorov means are also studied. Furthermore, we consider moving proximal averages and epi-averages of convex functions.
Maximally monotone operators play a key role in modern optimization and variational analysis. Two useful subclasses are rectangular (also known as star monotone) and paramonotone operators, which were introduced by Brezis and Haraux, and by Censor, Iusem and Zenios, respectively. The former class has useful range properties while the latter class is of importance for interior point methods and duality theory. Both notions are automatic for subdifferential operators and known to coincide for certain matrices; however, more precise relationships between rectangularity and paramonotonicity were not known. Our aim is to provide new results and examples concerning these notions. It is shown that rectangularity and paramonotonicity are actually independent. Moreover, for linear relations, rectangularity implies paramonotonicity but the converse implication requires additional assumptions. We also consider continuous linear monotone operators, and we point out that in Hilbert space both notions are automatic for certain displacement mappings.
The problem of finding a vector with the fewest nonzero elements that satisfies an underdetermined system of linear equations is an NP-complete problem that is typically solved numerically via convex heuristics or nicely-behaved non convex relaxations. In this paper we consider the elementary method of alternating projections (MAP) for solving the sparsity optimization problem without employing convex heuristics. In a parallel paper we recently introduced the restricted normal cone which generalizes the classical Mordukhovich normal cone and reconciles some fundamental gaps in the theory of sufficient conditions for local linear convergence of the MAP algorithm. We use the restricted normal cone together with the notion of superregularity, which is naturally satisfied for the affine sparse optimization problem, to obtain local linear convergence results with estimates for the radius of convergence of the MAP algorithm applied to sparsity optimization with an affine constraint.
The method of alternating projections (MAP) is a common method for solving feasibility problems. While employed traditionally to subspaces or to convex sets, little was known about the behavior of the MAP in the nonconvex case until 2009, when Lewis, Luke, and Malick derived local linear convergence results provided that a condition involving normal cones holds and at least one of the sets is superregular (a property less restrictive than convexity). However, their results failed to capture very simple classical convex instances such as two lines in three-dimensional space. In this paper, we extend and develop the Lewis-Luke-Malick framework so that not only any two linear subspaces but also any two closed convex sets whose relative interiors meet are covered. We also allow for sets that are more structured such as unions of convex sets. The key tool required is the restricted normal cone, which is a generalization of the classical Mordukhovich normal cone. We thoroughly study restricted normal cones from the viewpoint of constraint qualifications and regularity. Numerous examples are provided to illustrate the theory.
The problem of finding the zeros of the sum of two maximally monotone operators is of fundamental importance in optimization and variational analysis. In this paper, we systematically study Attouch-Théra duality for this problem. We provide new results related to Passty's parallel sum, to Eckstein and Svaiter's extended solution set, and to Combettes' fixed point description of the set of primal solutions. Furthermore, paramonotonicity is revealed to be a key property because it allows for the recovery of all primal solutions given just one arbitrary dual solution. As an application, we generalize the best approximation results by Bauschke, Combettes and Luke [J. Approx. Theory 141 (2006), 63-69] from normal cone operators to paramonotone operators. Our results are illustrated through numerous examples.
Because of Minty's classical correspondence between firmly nonexpansive mappings and maximally monotone operators, the notion of a firmly nonexpansive mapping has proven to be of basic importance in fixed point theory, monotone operator theory, and convex optimization. In this note, we show that if finitely many firmly nonexpansive mappings defined on a real Hilbert space are given and each of these mappings is asymptotically regular, which is equivalent to saying that they have or "almost have" fixed points, then the same is true for their composition. This significantly generalizes the result by Bauschke from 2003 for the case of projectors (nearest point mappings). The proof resides in a Hilbert product space and it relies upon the Brezis-Haraux range approximation result. By working in a suitably scaled Hilbert product space, we also establish the asymptotic regularity of convex combinations.
In this paper, we construct maximally monotone operators that are not of Gossez's dense-type (D) in many nonreflexive spaces. Many of these operators also fail to possess the Brønsted-Rockafellar (BR) property. Using these operators, we show that the partial inf-convolution of two BC--functions will not always be a BC--function. This provides a negative answer to a challenging question posed by Stephen Simons. Among other consequences, we deduce that every Banach space which contains an isomorphic copy of the James space $\mathbf{J}$ or its dual $\mathbf{J}^*$, or $c_0$ or its dual $\ell^1$, admits a non type (D) operator.
We show that every maximally monotone operator of Fitzpatrick-Phelps type defined on a real Banach space must be of dense type. This provides an affirmative answer to a question posed by Stephen Simons in 2001 and implies that various important notions of monotonicity coincide.
The notion of a firmly nonexpansive mapping is central in fixed point theory because of attractive convergence properties for iterates and the correspondence with maximal monotone operators due to Minty. In this paper, we systematically analyze the relationship between properties of firmly nonexpansive mappings and associated maximal monotone operators. Dual and self-dual properties are also identified. The results are illustrated through several examples.
To provide generalized solutions if a given problem admits no actual solution is an important task in mathematics and the natural sciences. It has a rich history dating back to the early 19th century when Carl Friedrich Gauss developed the method of least squares of a system of linear equations - its solutions can be viewed as fixed points of averaged projections onto hyperplanes. A powerful generalization of this problem is to find fixed points of averaged resolvents (i.e., firmly nonexpansive mappings). This paper concerns the relationship between the set of fixed points of averaged resolvents and certain fixed point sets of compositions of resolvents. It partially extends recent work for two mappings on a question of C. Byrne. The analysis suggests a reformulation in a product space. Furthermore, two new algorithms are presented. A complete convergence proof that is based on averaged mappings is provided for the first algorithm. The second algorithm, which currently has no convergence proof, iterates a mapping that is not even nonexpansive. Numerical experiments indicate the potential of these algorithms when compared to iterating the average of the resolvents.
It is shown that, for maximally monotone linear relations defined on a general Banach space, the monotonicities of dense type, of negative-infimum type, and of Fitzpatrick-Phelps type are the same and equivalent to monotonicity of the adjoint. This result also provides affirmative answers to two problems: one posed by Phelps and Simons, and the other by Simons.
We study a question posed by Stephen Simons in his 2008 monograph involving "bigger conjugate" (BC) functions and the partial infimal convolution. As Simons demonstrated in his monograph, these function have been crucial to the understanding and advancement of the state-of-the-art of harder problems in monotone operator theory, especially the sum problem. In this paper, we provide some tools for further analysis of BC--functions which allow us to answer Simons' problem in the negative. We are also able to refute a similar but much harder conjecture which would have generalized a classical result of Brézis, Crandall and Pazy. Our work also reinforces the importance of understanding unbounded skew linear relations to construct monotone operators with unexpected properties.
We study nearly equal and nearly convex sets, ranges of maximally monotone operators, and ranges and fixed points of convex combinations of firmly nonexpansive mappings. The main result states that the range of an average of firmly nonexpansive mappings is nearly equal to the average of the ranges of the mappings. A striking application of this result yields that the average of asymptotically regular firmly nonexpansive mappings is also asymptotically regular. Throughout, examples are provided to illustrate the theory. We also obtain detailed information on the domain and range of the resolvent average.
The demiclosedness principle is one of the key tools in nonlinear analysis and fixed point theory. In this note, this principle is extended and made more flexible by two mutually orthogonal affine subspaces. Versions for finitely many (firmly) nonexpansive operators are presented. As an application, a simple proof of the weak convergence of the Douglas-Rachford splitting algorithm is provided.
During the 1970s Brézis and Browder presented a now classical characterization of maximal monotonicity of monotone linear relations in reflexive spaces. In this paper, we extend and refine their result to a general Banach space.
In Euclidean spaces, the geometric notions of nearest-points map, farthest-points map, Chebyshev set, Klee set, and Chebyshev center are well known and well understood. Since early works going back to the 1930s, tremendous theoretical progress has been made, mostly by extending classical results from Euclidean space to Banach space settings. In all these results, the distance between points is induced by some underlying norm. Recently, these notions have been revisited from a different viewpoint in which the discrepancy between points is measured by Bregman distances induced by Legendre functions. The associated framework covers the well known Kullback-Leibler divergence and the Itakura-Saito distance. In this survey, we review known results and we present new results on Klee sets and Chebyshev centers with respect to Bregman distances. Examples are provided and connections to recent work on Chebyshev functions are made. We also identify several intriguing open problems.
We show that the set of fixed points of the average of two resolvents can be found from the set of fixed points for compositions of two resolvents associated with scaled monotone operators. Recently, the proximal average has attracted considerable attention in convex analysis. Our results imply that the minimizers of proximal-average functions can be found from the set of fixed points for compositions of two proximal mappings associated with scaled convex functions. When both convex functions in the proximal average are indicator functions of convex sets, least squares solutions can be completely recovered from the limiting cycles given by compositions of two projection mappings. This provides a partial answer to a question posed by C. Byrne. A novelty of our approach is to use the notion of resolvent average and proximal average.
The most important open problem in Monotone Operator Theory concerns the maximal monotonicity of the sum of two maximal monotone operators provided that Rockafellar's constraint qualification holds. In this note, we provide a new maximal monotonicity result for the sum of a maximal monotone relation and the subdifferential operator of a proper, lower semicontinuous, sublinear function. The proof relies on Rockafellar's formula for the Fenchel conjugate of the sum as well as some results on the Fitzpatrick function.
The proximal average of two convex functions has proven to be a useful tool in convex analysis. In this note, we express Goebel's self-dual smoothing operator in terms of the proximal average, which allows us to give a simple proof of self duality. We also provide a novel self-dual smoothing operator. Both operators are illustrated by smoothing the norm.