A Convex Polynomial that is not SOSConvex Amir

A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of Technology FRG: Semidefinite Optimization and Convex Algebraic Geometry May 2009 - MIT 1

Deciding Convexity Given a multivariate polynomial p(x): =p(x 1, …, xn ) of even degree, how to decide if it is convex? A concrete example: Most direct application: global optimization § Global minimization of polynomials is NP-hard even when the degree is 4 § But in presence of convexity, no local minima exist, and simple gradient methods can find a global min 2

Other Applications In many problems, we would like to parameterize a family of convex polynomials that perhaps: § serve as a convex envelope to a non-convex function § approximate a more complicated function § fit data samples with “small” error To address these questions, we need an understanding of the algebraic structure of the set of convex polynomials [Magnani, Lall, Boyd] 3

Convexity and the Second Derivative Fact: a polynomial p(x) is convex if and only if its Hessian H(x) is positive semidefinite (PSD) Equivalently, H(x) is PSD if and only if the scalar polynomial y. TH(x)y in 2 n variables [x; y] is positive semidefinite (psd) Back to our example: But can we efficiently check if H(x) is PSD for all x? 4

Complexity of Deciding Convexity Checking polynomial nonnegativity is NP-hard for degree 4 or larger However, there is additional structure in the polynomial y. TH(x)y: § Quadratic in y (a “biform”) § H(x) is a matrix of second derivatives partial derivatives commute Pardalos and Vavasis (’ 92) included the following question proposed by Shor on a list of the seven most important open problems in complexity theory for numerical optimization: “What is the complexity of deciding convexity of a multivariate polynomial of degree four? ” To the best of our knowledge: still open 5

SOS-Convexity Defn. ([Helton, Nie]): a polynomial p(x) is sos-convex if its Hessian factors as for a possibly nonsquare polynomial matrix M(x). p(x) sos-convex p(x) convex As we will see, checking sos-convexity can be cast as the feasibility of a semidefinite program (SDP) , which can be solved in polynomial time using interior-point methods. 6

SOS-convexity ( Ctnd. ) sos-convexity in the literature: § Semidefinite representability of semialgebraic sets [Helton, Nie] § Generalization of Jensen’s inequality [Lasserre] § Polynomial fitting, minimum volume convex sets [Magnani, Lall, Boyd] Question that has been raised: Q: must every convex polynomial be sos-convex? Our main contribution (via an explicit counterexample) No 7

Agenda Nonnegativity and sum of squares § A bit of history § Connection to semidefinite programming § SOS-matrices Other (equivalent? ) notions for sos-convexity Our counterexample (convex but not sos-convex) § Ideas behind the proof § Several remarks § How did we find it? Conclusions 8

Nonnegative and Sum of Squares Polynomials Defn. A polynomial p(x) is nonnegative or positive semidefinite (psd) if Defn. A polynomial p(x) is a sum of squares (sos) if there exist some other polynomials q 1(x), …, qm(x) such that § p(x) sos p(x) psd (obvious) § When is the converse true? 9

Hilbert’s 1888 Paper In 1888, Hilbert proved that a nonnegative polynomial p(x) of degree d in n variables must be sos only in the following cases: § n=1 (univariate polynomials of any degree) § d=2 (quadratic polynomials in any number of variables) § n=2 and d=4 (bivariate quartics) In all other cases, there are polynomials that are psd but not sos 10

The Celebrated Example of Motzkin The first concrete counterexample was found about 80 years later! This polynomial is psd but not sos 11

Sum of Squares and Semidefinite Programming Unlike nonnegativity, checking whether a polynomial is SOS is a tractable problem Thm: A polynomial p(x) of degree 2 d is SOS if and only if there exists a PSD matrix Q such that where z is the vector of monomials of degree up to d Feasible set is the intersection of an affine subspace and the PSD cone, and thus is a semidefinite program. 12

SOS matrices Defn. ([Kojima], [Gatermann-Parrilo]): A symmetric polynomial matrix P(x) is an sos-matrix if for a possibly nonsquare polynomial matrix M(x). Lemma: P(x) is an sos-matrix if and only if the scalar polynomial y. TP(x)y in [x; y] is sos. Therefore, can solve an SDP to check if P(x) is an sos-matrix. 13

PSD matrices may not be SOS Explicit “biform” examples of Choi, Reznick (and others), yield PSD matrices that are not SOS. For instance, the biquadratic Choi form can be rewritten as: However this example (and all others we’ve found), is not a valid Hessian: 14

Equivalent notions for convexity §Basic definition: §First order condition: §Second order condition: 15

Each condition can be SOS- ified §Basic definition: A A’ §First order condition: B §Second order condition: C Thm: A A’ B C Proof: mimics the “standard” proof closely and uses closedness of the SOS cone 16

A convex polynomial that is not sos-convex Need a polynomial p(x) such that all the following polynomials A B C are psd but not sos. Without further ado. . . 17

Our Counterexample A homogeneous polynomial in three variables, of degree 8. Claim: § p(x) is convex: H(x) is PSD § p(x) is not sos-convex: H(x) ≠ MT(x)M(x) 18

Proof: H(x) is PSD Claim: Or equivalently the scalar polynomial is sos. Proof: Exact sos decomposition , with rational coefficients. Exploiting symmetries of this polynomial, we solve SDPs of significantly reduced size 19

Rational SOS Decomposition 20

Rational SOS Decomposition 21

Proof: H(x)≠MT(x)M(x) Lemma: if H(x) is an sos-matrix, then all its 2 n-1 principal minors are sos polynomials. In particular, all diagonal elements are sos. Proof: follows from the Cauchy-Binet formula. Therefore, it suffices to show that is not sos. We do this by a duality argument. 22

Separating Hyperplane PSD H(1, 1 ) µ SOS 23

A few remarks Our counterexample is robust to small perturbations § Follows from inequalities being strict A dehomogenized version is still convex but not sos-convex § Minimal in the number of variables § “Almost” minimal in the degree 24

How did we find this polynomial? H(1, 1 ) PSD M parameterize H(x) add Hessian constraints (partial derivatives must commute) µ SOS solve this sos-program 25

Messages to take home… SOS-relaxation is a tractable technique for certifying positive semidefiniteness of scalar or matrix polynomials We specialized to convexity and sos-convexity § Three natural notions for sos-convexity are equivalent § Not always exact § But very powerful (at least for low degrees and dimensions) Proposed a convex relaxation to search over a restricted family of psd polynomials that are not sos Open: what’s the complexity of deciding convexity? Our result further supports the hypothesis that it must be a hard problem 26

• Want to know more? Preprint at http: //arxiv. org/abs/0903. 1287 Questions? 27