In yet another excursion in to applications of discrete harmonic analysis, we will now see an application of the invariance principle, to hardness of approximation. We will complement this discussion with the proof of the invariance principle in the next lecture.

1. Invariance Principle

In its simplest form, the central limit theorem asserts the following: “As increases, the sum of independent bernoulli random variables ( random variables) has approximately the same distribution as the sum of independent normal (Gaussian with mean and variance ) random variables”

Alternatively, as increases, the value of the polynomial has approximately the same distribution whether the random variables are i.i.d Bernoulli random variables or i.i.d normal random variables. More generally, the distribution of is the approximately the same as long as the random variables are independent with mean , variance and satisfy certain mild regularity assumptions.

A phenomenon of this nature where the distribution of a function of random variables, depends solely on a small number of their moments is referred to as invariance.

A natural approach to generalize of the above stated central limit theorem, is to replace the “sum” () by other multivariate polynomials.   As we will see in the next lecture, the invariance principle indeed holds for low degree multilinear polynomials that are not influenced heavily by any single coordinate. Formally, define the influence of a coordinate on a polynomial as follows:

Definition 1 For a multilinear polynomial define the influence of the coordinate as follows:

Here denotes the variance of over random choice of .

The invariance principle for low degree polynomials was first shown by Rotar in 1979. More recently, invariance principles for low degree polynomials were shown in different settings in the work of Mossel-O’Donnell-Olekszciewicz and Chatterjee. The former of the two works also showed the Majority is Stablest conjecture, and has been influential in introducing the powerful tool of invariance to hardness of approximation.

Here we state a special case of the invariance principle tailored to the application at hand.  To this end, let us first define a rounding function as follows:

Theorem 2 (Invariance Principle [Mossel-ODonnell-Oleszkiewicz]) Let and be sets of Bernoulli and Gaussian random variables respectively. Furthermore, let

and . Let denote independent copies of the random variables and .

Let be a multilinear polynomial given by , and let . If for all , then the following statements hold:

1. For every function which is thrice differentiable with all its partial derivatives up to order bounded uniformly by ,
   

   

2. Define the function as Then, we have

2. Dictatorship Testing

A boolean function on bits is a dictator or a long code if for some . Given the truth table of a function , a dictatorship test is a randomized procedure that queries a few locations (say ) in the truth table of , and tests a predicate on the values it queried. If the queried values satisfy the predicate , the test outputs ACCEPT else it outputs REJECT. The goal of the test is to determine if is a dictator or is far from being a dictator. The main parameters of interest in a dictatorship test are :

• Completeness Every dictator function is accepted with probability at least .
• Soundness Any function which is far from every dictator is accepted with probability at most .
• Number of queries made, and the predicate used by the test.

2.1. Motivation

The motivation for the problem of dictatorship testing arises from hardness of approximation and PCP constructions. To show that an optimization problem is -hard to approximate, one constructs a reduction from a well-known -hard problem such as Label Cover to . Given an instance of the Label Cover problem, a hardness reduction produces an instance of the problem . The instance has a large optimum value if and only if has a high optimum. Dictatorship tests serve as “gadgets” that encode solutions to the Label Cover, as solutions to the problem . In fact, constructing an appropriate dictatorship test almost always translates in to a corresponding hardness result based on the Unique Games Conjecture.

Dictatorship tests or long code tests as they are also referred to, were originally conceived purely from the insight of error correcting codes. Let us suppose we are to encode a message that could take one of different values . The long code encoding of the message is a bit string of length , consisting of the truth table of the function . This encoding is maximally redundant in that any binary encoding with more than bits would contain bits that are identical for all messages. Intuitively, greater the redundancy in the encoding, the easier it is to perform the reduction.

While long code tests/dictatorship tests were originally conceived from a coding theoretic perspective, somewhat surprisingly these objects are intimately connected to semidefinite programs. This connection between semidefinite programs and dictatorship tests is the subject of today’s lecture. In particular, we will see a black-box reduction from SDP integrality gaps for Max Cut to dictatorship tests that can be used to show hardness of approximating Max Cut.

2.2. The case of Max Cut

The nature of dictatorship test needed for a hardness reduction varies with the specific problem one is trying to show is hard. To keep things concrete and simple, we will restrict our attention to the Max Cut problem.

A dictatorship test for the Max Cut problem consists of a graph on the set of vertices . By convention, the graph is a weighted graph where the edge weights form a probability distribution (sum up to ). We will write to denote an edge sampled from the graph (here ).

A cut of the graph can be thought of as a boolean function . For a boolean function , let denote the value of the cut. The value of a cut is given by

It is useful to define , for non-boolean functions that take values in the interval . To this end, we will interpret a value as a random variable that takes values. Specifically, we think of a number as the following random variable

$ latex a = -1 $ with probability and with probability  $\frac{1+a}{2}$.

With this interpretation, the natural definition of is as follows:

Indeed, the above expression is equal to the expected value of the cut obtained by randomly rounding the values of the function to as described in Equation 2.

A Dictator Cut

The dictator cuts are given by the functions for some . The of the test is the minimum value of a dictator cut, i.e.,

The soundness of the dictatorship test is the value of cuts that are far from every dictator. We will formalize the notion of being far from every dictator is formalized using influences as follows:

Cut Far From Every Dictator

Definition 3 (-quasirandom) A function is said to be -quasirandom if for all , .

Definition 4 () For a dictatorship test over and , define the soundness of as

2.3. SDP Relaxation

Let be an an instance of the Max Cut problem. Specifically, is a weighted graph over a set of vertices , whose edge weights sum up to (by convention). Thus, the set of edges will also be thought of as a probability distribution over edges. Let us the Goemans-Williamson semidefinite programming relaxation for Max Cut. The variables of the GW SDP consist of a set of vectors , one vector for each vertex in the graph .

GW SDP Relaxation
Maximize (Average Squared Length of Edges)

Subject to  (all vectors  $\latex \mathbf v_i\ $ are unit vectors)

3. Intuition

Let be an an arbitrary instance of the Max Cut problem and let denote a feasible solution to the GW SDP relaxation. We begin by presenting the intuition behind the black box reduction.

3.1. Dimension Reduction

Without loss of generality, the SDP solution can be assumed to lie in a large constant dimensional space. Specifically, given an arbitrary SDP solution in -dimensional space, project it in to a random subspace of dimension – a large constant. Random projections approximately preserve the lengths of vectors and distances between them. Hence, roughly speaking, the vectors produced after random projection yield a low-dimensional SDP solution to the same graph .

Formally, sample random Gaussian vectors of the same dimension as the SDP vectors . Here is to be thought of as a large constant independent of the size of the graph . Define a solution to the GW SDP relaxation as follows:

      for all vertices     in graph $latex  G$

The vector is just the projection of the vector along directions , normalized to unit length. Clearly, the vectors form a feasible SDP solution to GW SDP.

For every , by choosing to be a sufficiently large constant, the following can be ensured: the distance between any two vectors and is preserved up to -multiplicative factor with probability at least . Therefore, there exists some choice of such that the vectors form a low dimensional SDP solution with roughly the same value as , i.e., . Henceforth, without loss of generality, let us assume that the SDP solution consist of -dimensional vectors for a large enough constant .

3.2. Sphere Graph

A graph on the unit sphere, will consist of a set of unit vectors, and weigthed edges between them. As usual, the weights of the graph form a probability distribution, in that they sum up to .

The SDP solution for a graph , yields a graph on the -dimensional unit sphere, which is isomorphic to . Recall that the objective value of the GW SDP is the average squared length of the edges. Hence, the SDP value remains unchanged under the following transformations:

• Rotation Any rotation of the SDP vectors about the origin, preserves the lengths of edges, and the distances between them. Thus, rotating the SDP solution yields another feasible solution with the same objective value.
• Union of Rotations Let and be two solutions obtained by applying rotations to the SDP vectors . Let be the associated graphs on the unit sphere. Let denote the union of the two graphs, i.e., . The set of all distinct vectors in are the vertices of . The edge distribution of is the average of the edge distributions of and .The average squared lengths of edges in both and are equal to . Hence, the average squared edge length in is also equal to . Thus, taking the union of two rotations of a graph preserves the SDP value.

Define the sphere graph as follows:

Constructing the Sphere Graph

Sphere Graph : Union of all possible rotations of the graph (on the set of vectors ) on the -dimensional unit sphere.

Clearly the sphere graph is an infinite graph. The sphere graph is solely a conceptual tool, and an explicit representation is never needed in the reduction. Nevertheless, due to its symmetry, indeed the sphere graph can be represented succinctly. By construction, the SDP value of the sphere graph is the same as that of the original graph . However, we will argue that is a harder instance of Max Cut than the original graph . In fact, given a cut for the sphere graph , it is possible to retrieve a cut for the original graph with the same objective value. Let us suppose is a cut of the sphere graph that cuts a -fraction of the edges. Notice that consists of a union of infinitely many copies (or rotations) of the graph . Therefore, on at least one of the copies of , the cut must cut a -fraction of the edges. Indeed, if we have oracle access to the cut function , we can efficiently construct a cut of the graph with the same value as using the following rounding procedure:

• Sample a rotation of the unit sphere, uniformly at random.
• Output the cut induced by on the copy of the graph .

The expected value of the cut output by the procedure is equal to the value of the cut on the sphere graph . An immediate consequence is that,

The sphere graph inherits the same SDP value as , while the optimum value is at most that of the graph . In this light, the sphere graph is a harder instance of Max Cut than the original graph .

It is easy to see that the following is an equivalent definition for the sphere graph .

Definition 5 (Sphere Graph ) The set of vertices of is the set of all points on the -dimensional unit sphere. To sample an edge of use the following procedure,

• Sample an edge in the graph ,
• Sample two points on the sphere at a squared distance uniformly at random.
• Output the edge between .

3.3. Hypercube Graph

Finally, we are ready to describe the construction of the graph on the -dimensional hypercube. Here we refer to the hypercube suitably normalized to make all its points lie on the unit sphere.

The set of vertices of are points in . An edge of can be sampled as follows:

• Sample an edge in the graph .
• Sample two points in , at squared distance uniformly at random.
• Output the edge between .

It is likely that there are no pair of points on the hypercube at a distance exactly equal to . For the sake of exposition, let us ignore this issue for now. To remedy this issue, in the final construction, for each edge just introduce a probability distribution over edges such that the expected length of an edge is indeed .

Completeness:

Consider the th dictator cut given . This corresponds to the axis-parallel cut of the graph along the th axis of the hypercube. Let us estimate the value of the cut . An edge is cut by the th dictator cut if and only if . Therefore, the value of the th dictator cut is given by:

Notice that two points at a squared distance differ on exactly coordinates. Hence, two random points at distance at a squared distance differ on a given coordinate with probability . Therefore, let us rewrite the expression for as follows:

Hence,   is at least .

Soundness:

Consider a cut that is far from every dictator. Intuitively, the cut is not parallel to any of the axis of the hypercube. Note the strong similarity in the construction of the sphere graph and the hypercube graph . In both cases, we sampled two random points at a distance equal to the edge length. In fact, the hypercube graph is a subgraph of the sphere graph . The existence of special directions (the axis of the hypercube) is what distinguishes the hypercube graph from the sphere graph . Thus, roughly speaking, a cut which is not paralel to any axis must be unable to distinguish between the sphere graph and the hypercube graph . If we visualize the cut of as a geometric surface not parallel to any axis, then the same geometric surface viewed as a cut

Extending the cut from Hypercube to Sphere graph

of the sphere graph must separate roughly the same fraction of edges.

Indeed, the above intuition can be made precise if the cut is sufficiently smooth (low degree). The cut can be expressed as a multilinear polynomial (by Fourier expansion), thus extending the cut function from to . The function is smooth if the corresponding polynomial polynomial is low degree. If is smooth and far from every dictator, then one can show that,

By Equation 1, the maximum value of a cut of the sphere graph is at most . Therefore, for any cut that is smooth and far from every dictator, we get .

Ignoring the smoothness condition for now, the above argument shows that the soundness of the dictatorship test is at most . Summarizing the above discussion, starting from a SDP solution for a graph , we constructed hypercube graph (dictatorship test) such that , and .

By suitably modifying the construction of , the smoothness requirement for the cut can be dropped. The basic idea is fairly simple yet powerful. In the definition of , while introducing an edge between , perturb each coordinate of and with a tiny probability to obtain and respectively, then introduce the edge instead of . The introduction of noise to the vertices and has an averaging effect on the cut function, thus making it smooth.

4. Formal Proof

Let be an arbitrary instance of Max Cut. Let be a feasible solution to the SDP relaxation.

Locally, for every edge in , there exists a distribution over assignments that match the SDP inner products. In other words, there exists valued random variables such that

For each edge , let denote the local integral distribution over assignments.

The details of the construction of dictatorship test are as follows:

The set of vertices of consist of the -dimensional hypercube . The distribution of edges in is the one induced by the following sampling procedure:

• Sample an edge in the graph .
• Sample times independently from the distribution to obtain and , both in .
• Perturb each coordinate of and independently with probability to obtain respectively. Formally, for each ,

  

• Output the edge .

Theorem 6 There exists absolute constants such that for all , for any graph and an SDP solution for the GW-SDP relaxation of ,

• 
• 

Let be a cut of the graph. The fraction of edges cut by is given by

4.1. Completeness

Consider the th dictator cut given by . With probability , the perturbation does not affect the th coordinate of and . In other words, with probability , the following hold: and . Hence,

Observe that if the edge in is sampled, then the distribution is used to generate each coordinates of and . Specifically, this means that the coordinates and satisfy,

Therefore, .

4.2. Soundness

For the sake of analysis, let us construct a graph , roughly similar to the sphere graph described earlier. Gaussian Graph

The vertices of are points in . The graph is the union of all random projections of the SDP solution in to dimensions. Formally, an edge of can be sampled as follows:

• Sample random Gaussian vectors of the same dimension as the SDP solution .
• Project the SDP vectors along directions to obtain a copy of in a . Formally define,

  

• Sample an edge in , and output the corresponding edge in

As lengths of vectors are approximately preserved under random projections, most of the vectors are are roughly unit vectors. Hence, the Gaussian graph is a slightly fudged version of the sphere graph described earlier.

As the graph consists of a union of several isomorphic copies of , the following claim is an immediate consequence.

Claim 1

Let us suppose be a -quasirandom function. For the sake of succinctness, let us denote . Essentially, is the expected value returned by on querying a perturbation of the input . Thus the function is a smooth version of , obtained by averaging the values of .

Now we will extend the cut from the hypercube graph to the Gaussian graph . To this end, write the functions as written as a multilinear polynomials in the coordinates of . In particular, the Fourier expansion of and yields the intended multilinear polynomials.

The polynomials and yield natural extension of the cut functions and from to . However, unlike the original cut function , the range of its extension need not be restricted to . To ensure that the extension defines a cut of the graph , let us round the extension in the most natural fashion using .   Specifically, the extension of the cut to is given by

  where 

Let denote the value of the cut of the graph . Now we can show the following claim.

Claim 2 For a -quasirandom function ,

By definition of , we have . Along with Claim 1, this implies that , completing the proof of Theorem 6.

Proof: [Proof of Claim 2] Returning to the definition of , notice that the random variable depends only on . Thus, the value of a cut can be rewritten as,

By the definition of the noise operator ,. Hence can be rewritten as

By definition of the Gaussian graph , we have

Firstly, let us denote by the function given by . Let us restrict our attention to a particular edge . For this edge, we will show that

By averaging the above equality over all edges in the graph , the claim follows. We will use the invariance principle to show the above claim.

By design, for each edge the pairs of random variables and satisfy,

The predicate/payoff is currently defined as in the domain . Extend the payoff to a smooth function over the entire space , with all its partial derivatives up to order bounded uniformly throughout . Notice that the function by itself does not have uniformly bounded derivatives in . Further, it is easy to ensure that the extension satisfies the following Lipschitz condition for some large enough constant ,

for all

We will prove Equation 4 in two steps.

Step I : Apply the invariance principle with the ensembles and , for the vector of multilinear polynomials and the smooth function . This yields,

Step II : In this step, let us bound the effect of the rounding operation used in extending the cut from to .

As is a cut of , its range is . Hence, the corresponding polynomial takes values on inputs from . As is an average of the values of , the values and are always in the range .

By the invariance principle, the random variable has approximately the same behaviour as . Roughly speaking, this implies that the values are also nearly always in the range . Hence, intuitively the rounding operation must have little effect on the value of the cut.

This intuition is formalized by the second claim in the invariance principle. The function measures the squared deviation from the range . For random variables , clearly we have . By the invariance principle applied to polynomial we get,

Using the Lipschitz condition satisfied by the payoff,

             (By Cauchy Schwartz Inequality)

  (by definition of  )

           (Equation 2)

Along with Equation 4, the above inequality implies Equation 4. This finishes the proof of Claim 2.  

5. Dictatorship Tests and Rounding Schemes

The soundness analysis can be translated in to an efficient rounding scheme. Specifically, given a cut of the graph , let denote the extension of the cut to the Gaussian graph . The idea is to sample a random copy of the graph inside the Gaussian graph , and output the cut induced by on the copy. The details of the rounding scheme so obtained are as follows:

Input: SDP solution for the GW SDP relaxation of the graph .

• Sample random Gaussian vectors of the same dimension as the SDP solution .
• Project the SDP vectors along directions . Let

  

• Compute for all as    where
• Assign vertex , the value with probability and with the remaining probability.

Let denote the expected value of the cut returned by the above rounding scheme on an SDP solution . Then,

The following is an immediate consequence of Claim 2,

Theorem 7 For a -quasirandom function ,

The above theorem exposes an interesting duality between rounding schemes and dictatorship tests.

Simplices

It’s most intuitive to start with the geometric viewpoint, in which case an -simplex is defined to be the convex hull of affinely independent points in .  These points are called the vertices of the simplex.  Here are examples for

A simplicial complex is then a collection of simplices glued together along lower-dimensional simplices.  More formally, if is a (geometric) simplex, then a face of is a subset formed by taking the convex hull of a subset of the vertices of .

Finally, a (geometric) simplicial complex is a collection of simplices such that

1. If and is a face of , then , and
2. If and , then is a face of both and .

Property (1) gives us downward closure, and property (2) specifies how simplices can be glued together (only along faces).  For instance, the first picture depicts a simplicial complex.  The second does not.

There is also an abstract way to define a simplicial complex.  An abstract simplicial complex is a ground set of vertices, together with a collection of subsets of satisfying the axioms

1. 
2. If and , then .

For instance, a standard (undirected) graph can be thought of as a 1-dimensional abstract simplicial complex.  We can always realize an abstract complex as a geometric complex by taking affinely independent points in and adding in the appropriate convex hulls.  (Exercise:  Every 1-dimensional complex can be realized in , but not .)  In general, we will switch freely between the two notions.  (Here is an interesting paper of Matousek, Tancer, and Wagner on the computational complexity of realizability.)

Contractibility and collapsibility

Say that a set is contractible if there exists a continuous mapping such that

1. is the identity on .
2. for some fixed .

In other words, can be contracted to a point in place. For instance, the closed unit ball in is contractible while the unit sphere is not (think about it!)  As another example, consider that a geometric realization of a graph (a 1-dimensional simplicial complex) is contractible if and only if the graph is a connected tree.

We now define two basic operations on simplicial complexes.  The first involves removing a vertex

The second, called the link of in is

For example, consider the following complex , in which a distinguished simplex is marked.

Then we form the two simplices and (respectively) as follows.

The link of separates from .  The following lemma generalizes the natural strategy for showing that a connected tree is contractible.

Lemma: If is a geometric simplicial complex and both and are contractible, then so is .

Instead of proving this lemma, we will work with the more combinatorial notion of collapsibility.

Let be an abstract simplicial complex.  A free face of is a non-maximal face which is contained in a unique maximal face.  We can collapse to a subcomplex by removing a free face , along with all faces containing .

After collapsing the free face A.

This yeilds an inductive notion of collapsibility: is collapsible if there exists a sequence of collapses that leads from to the empty set.  It is straightforward to verify that a geometric simplicial complex is contractible whenever it is collapsible, but the reverse direction does not hold.

Now we state a version of the preceding lemma for collapsibility.

Collapsibility Lemma: If and are collapsible, then so is .

Proof: The key observation is that the sequence of moves used to collapse can be used to collapse to (verify using the definition of ), at which point we can collapse to the empty set by assumption.

Collapsibility and Evasiveness

Before ending the lecture, we should say why collapsibility is relevant to the evasiveness conjecture.  The connection is relatively straightforward:  Let be a non-trivial, monotone boolean function.  For a subset , let represent the characteristic vector of .  Then associated to , we have the simplicial complex

Let and be the two functions on bits corresponding to fixing the value of the th bit.  Then we have:

A simple induction using the Collapsibility Lemma (which we will do in the next lecture) now yields our main topological connection:

If is non-evasive, then is collapsible (hence also contractible).

[Credits: Some pictures taken from Wikipedia.]

Decision tree complexity and evasiveness

Consider a boolean function on n bits.  We define the decision tree complexity of f as follows.  Given an unknown input , you are allowed to ask about the values of various bits of x, e.g. .  Your goal is to compute using as few questions as possible, and your questions can be adaptive, depending on answers to previous questions.  The complexity of such a strategy is the maximum number of questions asked for any .  The decision tree complexity, written , is the minimum complexity of any strategy that computes f.  (There are many other interesting models of decision complexity, see e.g. this survey.)

Clearly , because we can trivially query all the bits of x, and then output f(x).  A function f is called evasive if this upper bound is met, i.e. .  As an example, consider the parity function , where is addition modulo 2.  Clearly is evasive because after bits of x are asked about, the setting of the final bit determines the value of f.

For a more general example, consider any such that is odd.  In this case, for every , exactly one of or has the same property that the number of inputs resulting in a 1 is odd.  (These two functions are the natural restriction of f to functions on n-1 bits, which results from fixing the value of the ith bit.)  Thus an adversary could keep answering questions “?” so that the restricted function retains this property.  Since the number of inputs yielding a 1 is always odd, the restricted function always takes both possible values, implying that f is evasive–the advesary ensures that the value cannot be determined until all n possible questions are asked.

For an example of a non-evasive property, think of a point as speciying a directed graph on vertices, where there is exactly one directed edges connecting every pair of vertices, and x specifies the direction of this edge (this is called a tournament).  Thinking of the vertices as players, a directed edge from u to v means that u defeats v.  Now if the digraph specified by x has one vertex that defeats everyone else.  What is ?

Well, first we can conduct a single elimination tournament, where vertex 1 plays vertex 2, and the winner players vertex 3, and the winner of that players vertex 4, etc.  At the end, there is only one remaining vertex that remains undefeated.  Now asking more questions, we can determine whether indeeds defeats everyone else.  The total number of questions was , hence , implying that f is not evasive.

Montone graph properties and the evasiveness conjecture

Let .  In general, we can encode an arbitrary undirected N-vertex graph as an element .  A function is called a graph property if relabeling the vertices of doesn’t affect the value of .  The function f is monotone if the value of the function can never change from 1 to 0 when flipping one of the input bits from 0 to 1.  In the setting of graph properties, this corresponds to those which are maintained under addition of edges to the graph, e.g. “is G connected?” or “does G have a k-clique?”

Evasiveness Conjecture (Aanderaa-Karp-Rosenberg): Every non-trivial monotone graph property is evasive.

Here, non-trivial means that , where and denote the all-zeros and all-ones strings, respectively.

For example, consider the example “is G connected?”  The adversary is simple:  When asked about a possible edge , she answers NO unless this answer would imply that the graph is disconnected.  In other words, she answers NO unless she has answered NO already for all edges across a cut except for , in which case she has to answer YES.

Now, suppose there is a strategy which figures out the connectivity of without asking a question about some edge .  In this case, the conclusion must be that because the adversary always maintains that by answering everything in the future YES, she could force the graph to be connected.  In this case, the edges answered YES have to form a spanning tree of G (otherwise by answering all unasked questions NO, the graph would become disconnected).  Consider a path P from i to j in this YES spanning tree.  Let be the edge of P which was asked about last.  Clearly the adversary answered YES for , but this contradicts the advesary’s strategy.  Since has not been asked yet, the adversary is safe to answer NO for , and still later by answering YES on , she could force the graph to be connected.  Thus no such strategy exists, and connectivity is evasive.

A natural generalization of the evasiveness conjecture is to general monotone functions which are invariant under a group acting transitively on the coordinates.  For a permutation , let act on via .  We say that is invariant under if for every .  Let be the group of all permutations under which f is invariant.  We will say that f is weakly symmetric if acts transitively on .

Generalized Evasiveness Conjecture: If f is a non-trivial, monotone, weakly symmetric function, then f is evasive.

Clearly this generalizes the previous conjecture because every graph property is invariant under permutations of the vertices (which induces a transitive action on the edges).

A proof when is prime

We’ll end this lecture with a proof of the generalized conjecture which holds when n is prime.  The proof will hint at the topological connections coming up in the next lecture.

First, we generalize the proof that f is evasive when is odd.  For , let denote the hamming weight of x, i.e. the number of 1′s in x, and for , define

Topologically-minded readers will recognize this as some kind of Euler characteristic, and this connection will bear out in the next lecture.  We claim that if , then f is evasive.  To see this, note that .  Hence if , then one of or is non-zero, so an adversary can keep answering so that the restriction has .  Finally, notice that if , then must take both values 0 and 1, so the adversary can maintain this until all n questions are asked.

Theorem: Suppose n is prime. If is monotone, non-trivial, and weakly symmetric, then f is evasive.

Proof: We’ll show that .  First, we argue that contains a cyclic permutation.  Write , where .  By transitivity, we have , hence divides .  Now since n is prime, by Cauchy’s theorem there is an element or order (i.e. an n-cycle).

Since f is non-trivial, we know that and .  For any with , it is clear that all n elements are distinct, since n is prime.  But f is invariant under , thus the set partitions into equivalence classes , where for every i.  But this immediately implies that , which gives .

The Borsuk-Ulam Theorem

We begin with a statement of the theorem.  We will think of the n-dimensional sphere as the subset of given by

Borsuk-Ulam Theorem: For every continuous mapping , there exists a point with .

Pairs of points are called antipodal.

There are a couple of common illustrative examples for the case .  The theorem says that if you take the air out of a basketball, crumple it (no tearing), and flatten it out, then there are two points directly on top of each other which were antipodal before.  Another common example states that at every point in time, there must be two points on the earth which both have exactly the same temperature and barometric pressure (assuming, of course, that these two parameters vary continuously over the surface of the eath).

The n=1 case is completely elementary.  For the rest of the lecture, let’s use and to denote the north and south poles (the dimension will be obvious from context).  To prove the n=1 case, simply trace out the path in starting at and going clockwise around .  Simultaneously, trace out the path starting at and going counter-clockwise at the same speed.  It is easy to see that eventually these two paths have to collide:  At the point of collision, .

We will give the sketch of a proof for   Let , and note that our goal is to prove that for some .  Note that is antipodal in the sense that for all .  Now, if for every , then by compactness there exists an such that for all .  Because of this, we may approximate arbitrarily well by a smooth map, and prove that the approximation has a 0.  So we will assume that itself is smooth.

Now, define by , i.e. the north/south projection map.  Let be a hollow cylinder, and let be given by so that linearly interpolates between and .

Also, let’s define an antipodality on by .  Note that is antipodal with respect to , i.e. , because both and are antipodal.  For the sake of contradiction, assume that for all .

Now let’s consider the structure of the zero set .  Certainly since , and these are h’s only zeros.  Here comes the sketchy part:  Since is smooth, is also smooth, and thus locally can be approximated by an affine mapping .  It follows that if is not empty, then it should be a subspace of dimension at least one.  By an arbitrarily small perturbation of the initial , ensuring that is sufficiently generic, we can ensure that is either empty or a subspace of dimension one.  Thus locally, should look like a two-sided curve, except at the boundaries and , where (if non-empty) would look like a one-sided curve.  But, for instance, cannot contain any Y-shaped branches.

It follows that is a union of closed cycles and paths whose endpoints must lie at the boundaries and .  ( is represented by red lines in the cylinder above.)  But since there are only two zeros on the sphere, and none on the sphere, must contain a path from to   Since is antipodal with respect to , must also satisfy this symmetry, making it impossible for the segment initiating at N to ever meet up with the segment initiating at S.  Thus we arrive at a contradiction, implying that must take the value 0.

Notice that the only important property we used about (other than its smoothness) is that is has a number of zeros which is twice an odd number.  If had, e.g. four zeros, then we could have two -symmetric paths emanating from and returning to the bottom.  But if has six zeros, then we would again reach a contradiction.

The Kneser conjecture

While the Borsuk-Ulam theorem seems initially far removed from any combinatorial applications, the following result of Lyusternik and Shnirel’man starts to hint at the combinatorial significance.

LS Lemma: Let be a cover of by closed sets.  Then some contains an antipodal pair.

Proof: Consider the continuous map given by , where denotes e.g. the geodesic distance on .  By the Borsuk-Ulam theorem, there exists a with .  Now if for some , then .  Otherwise, if this holds for no , then .

In fact, it is not difficult to show that the LS Lemma is equivalent to Borsuk-Ulam.  For our application to the Kneser conjecture, we will need to consider both open and closed sets.

Corollary 1: Let be a cover of by open sets.  Then some contains an antipodal pair.

Proof: This follows from the LS Lemma by choosing a family of closed sets such that forms a cover of .  The existence of such sets follows from applying compactness to the following open cover of :  For every , choose an open set whose closure lies completely within some .  By compactness, there is a finite subcover.  By piecing together elements of its closure, we can form the sets .

Finally, we address the case of both open and closed sets.

Corollary 2: Let be a cover of with each open or closed.  Then some contains an antipodal pair.

Proof: Suppose that are closed and are open.  Consider the open cover , where denotes the -neighborhood of .  Taking and applying Corollary 1 to each cover results in a sequence .  If for some , we are done.  Thus we may pass to a subsequence for which for some fixed .  Choose a convergent subsequence , and observe that since is closed, we have .  We conclude that there exists an with .

The Kneser graphs.

Now, we introduce the Kneser graphs .  The vertex set is , i.e. the vertices are k-element subsets of .  There is an edge between two sets if and only if and are disjoint.  In 1955, Kneser posed the following conjecture.

Conjecture: For every and , , where denotes the chromatic number.

First, we note that the upper bound is easy:  Construct a coloring by defining

Clearly if , then and have the same minimum element, and thus they are not disjoint.  On the other hand, if , then , and the latter set contains only elements.  Thus again .

The conjecture stood open until Lovász resolved it, using the Borsuk-Ulam theorem.  Recently, Greene (an undergraduate at the time) gave a particularly simple proof which we reproduce here.

Proof: Consider and let .  Let be a set of points in general position, so that no -dimensional hyperplane through the origin contains more than points of .  Clearly we may assume that actually has vertex set .  For every , let

denote the open hemisphere centered at .

Now, suppose we have a proper coloring of , and define sets as follows:  is the set of all points such that contains a k-set colored .  Finally, .

By Corollary 2 above, there must exist an and a such that .  If , then since and are disjoint, there must exist two disjoint k-sets colored , which means we did not start with a proper coloring.

If, instead , it implies that both .  But then the -dimensional hyperplane contains points, contradicting the fact that was chosen in general position.

Appearances in TCS

One of the most useful properties of the Kneser graphs is that there is a large gap between their chromatic number and their fractional chromatic number .  The fractional chromatic number of graph is the minimum value such that can be covered by independent sets, where each vertex occurs in at least of them.

It is easy to see that by choosing the n independent sets

The Kneser graphs are a particularly natural family to have a large gap between and because we can define for an arbtirary G as the minimum ratio such that G admits a homomorphism into .  One can also see that using the Erdős–Ko–Rado theorem.

See e.g. this FOCS’08 paper of Alon, et. al. for a recent application which uses this gap between fractional and actual chromatic numbers.  The Kneser graphs also arise very naturally in PCP constructions, as one can see in this paper of Dinur, Regev, and Smyth about the computational hardness of coloring hypergraphs.

Bounded genus graphs

For graphs of bounded genus, there is hope to use an approach based on conformal mappings.  In 1980, Yang and Yau proved that

for any compact Riemannian surface of genus .  (Note that for the Laplace-Beltrami operator, one usually writes as the first non-zero eigenvalue, rather than .)  In analog with Hersch’s proof of the genus 0 case, they use Riemann-Roch to obtain a degree- conformal mapping to the Riemann sphere, then try to pull back a second eigenfunction.  A factor of the degree is lost in the Rayleigh quotient (hence the factor in the preceding bound), and Hersch’s Möbius trick is still required.

genus 0

genus 1

genus 2

genus 3

An analogous proof for graphs of bounded genus would proceed by constructing a circle packing of on the sphere , but instead of the circles having disjoint interiors, we would be assured that every point of is contained in at most circles.  Unfortunately, such a result is impossible (this has to do with the handling of branch points in the discrete setting).  Kelner has to take a different approach in his proof that for graphs of genus at most .

He starts with a circle packing of on a compact surface of genus (whose existence follows from results of Beardon and Stephenon and He and Schramm).  Then Kelner randomly subdivides repeatedly, and these subdivisions give progressively better approximations to some sequence of surfaces .  Once the approximation is of high enough quality, one applies Riemann-Roch to , and infers something about a subdivision of .  The final element is to track how the second eigenvalue of changes (in expectation) under random subdivision.

Needless to say, this approach is already quite delicate, and for graphs that can’t be equipped with some kind of conformal structure, we seem to have reached a dead end.  In this lecture, we’ll see how to use intrinsic deformations of the geometry of in order to bound its eigenvalues.  Eventually, this will reduce to the study of certain kinds of multi-commodity flows.

Metrics on graphs

Let be an arbitrary n-vertex graph with maximum degree .  Recall that we can write

where .  (Also recall that we can replace by any Hilbert space, and the same formula holds.)  The first step is to prepare this equality for “non-linearization” by getting rid of the linear condition and the sum .  (This is a popular sort of passage in the non-linear geometry of Banach spaces, which also plays a rather important role in applications to the theoretical CS.)  The goal is to get only terms that look like .  Fortunately, there is a well-known way to do this:

which follows easily from the equality when .

Thus if we want to bound , we need to find an for which the latter ratio (without the ) is .  Now, for someone who works a lot with linear programming relaxations, it’s very natural to consider a “relaxation”

where the minimization is over all pseudo-metrics d, i.e. symmetric non-negative functions which satisfy the triangle inequality, but might have even for .  Certainly , but Bourgain’s embedding theorem (which states that every n-point metric space embeds into a Hilbert space with distortion at most ) also assures us that .  Since we are trying to show that , this term is morally negligible.  One can see the paper for a more advanced embedding argument that doesn’t lose this factor, but for now we concentrate on proving that .  The embedding theorems allow us to concentrate on finding an intrinsic metric on the graph with small “Rayleigh quotient,” without having to worry about an eventual geometric representation.

As a brief preview… we are going to find a good metric by taking a certain kind of all-pairs multi-commodity flow at optimality, and weighting the edges by their congestion in the optimal flow.  Thus as the flow spreads out on the graph, it has the effect of “uniformizing” its geometry.

Discrete Riemannian metrics, convexification, and duality

Let’s now assume that is planar.  We want to show that .  First, let’s restrict ourselves to vertex weighted metrics on .  Given any non-negative weight function , we can define the length of a path in by summing the weights of vertices along it:  .  Then we can define a vertex-weighted shortest-path pseudo-metric on in the natural way

where is the set of all u-v paths in .  We also have the nice relationship

since .

So if we define

then by (1), we have .

Examples. Let’s try to exhibit weights for two well-known examples:  the grid, and the complete binary tree.

For the grid, we can simply take for all .  Clearly .  On the other hand, a random pair of points in the grid is apart, hence .  It follows that , as desired.

For the complete binary tree with root , we can simply put and for .  (Astute readers will guess the geometrically decreasing weights are actually the optimal choice.)  In this case, , while all the pairs on opposite sides of the root have .  It again follows that .  Our goal is to provide such a weight for any planar graph.

A natural way to study the optimal weight is via duality; unfortunately, cannot be written as a convex program.  Thus we will pass to a “convexified” objetive:

The key here is that we have replaced the denominator by a linear form, which allows us to write as a convex program.  To relate it to , note that by Cauchy-Schwarz, for every

so our task reduces to finding an for which .  To see why we are not going lose much in the Cauchy-Schwarz bound, note that in the two “extremal” examples above, was concentrated near its maximum value (the tight case for Cauchy-Schwarz).

Now that we have a convex objective, we can find its dual.  If you are used to dealing with programs where the triangle inequality occurs in the primal, you already know that some kind of flow is going to rear its head in the dual.

Multi-flows and -congestion.

Let be the set of all paths in .  A flow in G will simply be a non-negative mapping which assigns values to paths.  We will define the flow from u to v as the quantity .  A complete flow in G is a flow F such that for all .  Finally, we define the congestion of the flow at a vertex v by , i.e. the amount of flow going through .  The -congestion is now the quantity

Working out the Lagrangian multipliers and using strong duality (Slater’s condition) yields the following.

Theorem (Duality): For any graph , we have

where the minimums are over all vertex weights and over all complete flows .

Thus in order to show the existence of a weight with , we need to show that for every complete flow F in G, .

Congestion, drawings, and crossings

First, let’s revisit our examples.  In the grid, a complete flow has total “length” about since about pairs need to send flow down a path of length at least .  To minimize the -congestion, we would spread this out evenly, putting congestion at every node, yelding for any complete flow .  In the complete binary tree, the argument is even easier since clearly flow has to go through the root.

Randomized rounding.

First, let’s round to an integral flow, i.e. such that the u-v flow is sent along a single u-v path for every pair of vertices.  Let be the random integral flow that arises in the following way:  For every pair of vertices u and v, let choose a path with probability .  It is easy to check that

Notice that is the amount of -congestion incurred by any complete flow congesting against itself (every vertex sends out flow n, so even that incurs congestion).

Thus we can assume that our flow is integral.  But now since our graph is planar, it has a drawing in the plane.  Now, I claim that every complete integral flow induces a drawing of the complete graph on n vertices in the plane, via the drawing of .  Furthermore, edges of cross only at vertices of .

In the picture, the graph is represented by black edges, while some of the edges of are drawn in colors.

How many edge crossings occur in the drawing of ?  Well, we have no idea how many crossings there are “inside” a vertex of , but if edges go in, we can make sure there are at most crossings (in the worst case, every edge crosses every other).  But this immediately implies that we have a drawing of in the plane with at most crossings.  Now, by the well-known crossing number inequality (found independently by Leighton, and also Ajtai, Chvatal, Newborn, and Szemeredi), the number of such crossings is always at least , implying that for any complete flow in .  This completes our alternate argument that for planar graphs.

In fact, it is trivial to prove the crossing inequality for the complete graph:  Every copy of must induce one crossing, now double count (it is easy to check that every crossing involving less than four points can be removed, so every crossing involves four points, hence we overcount by a factor of n, representing the one free vertex).  This kind of argument extends immediately to bounded genus graphs via similar crossing arguments.  For more advanced arguments, including the excluded-minor case conjectured by Spielman and Teng, one has to use not only complete graphs, but dense graphs, in which case the full power of the crossing number inequality is needed.  The beautiful probabilistic proof of Chazelle, Sharir, and Welzl (see the exposition on Terry Tao’s blog) can be appropriately generalized to excluded-minor graphs, and even to families of higher-dimensional graphs.

]]> http://tcsmath.wordpress.com/2008/10/08/lecture-4-conformal-mappings-circle-packings-and-spectral-geometry/ Wed, 08 Oct 2008 10:40:52 +0000 James Lee http://tcsmath.wordpress.com/2008/10/08/lecture-4-conformal-mappings-circle-packings-and-spectral-geometry/ In Lecture 2, we used spectral partitioning to rule out the existence of a strong parallel repetition theorem for unique games.  In practice, spectral methods are a very successful heuristic for graph partitioning, and in the present lecture we’ll see how to analyze these partitioning algorithms for some common families of graphs.

Balanced separators, eigenvalues, and Cheeger’s inequality

Lipton and Tarjan proved that every planar graph has a negligibly small set of nodes whose remval splits the graph into two roughly equal pieces.  More specifically, every n-node planar graph can be partitioned into three disjoint sets such that there are no edges from to , the separator has at most nodes, and .  This allows one to do all sorts of things, e.g. a simple divide-and-conquer algorithm gives a linear time -approximation for the maximum independent set problem in such graphs, for any .

So there is a natural question of how well spectral methods do, for example, on planar graphs.  Spielman and Teng showed that for bounded-degree planar graphs, a simple recursive spectral algorithm recovers a partition of the vertex set so that .  In other words, for bounded-degree planar graphs, spectral methods recover the Lipton-Tarjan separator theorem!  This is proved by combining Cheeger’s inequality with their main theorem.

Theorem [Spielman-Teng]: Every n-node planar graph with maximum degree has , where is the second eigenvalue of the combinatorial Laplacian on .

Recall that we introduced the combinatorial Laplacian in Lecture 2.  If is an arbitrary finite graph, in this lecture it will make more sense to think about the Laplacian as an operator on functions given by

If we define the standard inner product , then one can easily check that for any such , we have .  In particular, this implies that is a positive semi-definite operator.  If we denote its eigenvalues by , then it is also easy to check that , with corresponding eigenfunction for every .

Thus by standard variational principles, we have

Let us also define the Cheeger constant .  For an arbitrary subset , let

,

note that this definition varies from the we defined in Lecture 2, because we will be discussing eigenfunctions without boundary conditions.  Now one defines .

Finally, we have the version of Cheeger’s inequality (proved by Alon and Milman in the discrete setting) for graphs without boundary.

Cheeger’s inequality: If is any graph with maximum degree , then

This follows fairly easy from the Dirichlet version of Cheeger’s inequality presented in Lecture 2.  Here’s a sketch:  Let satisfy , and suppose, without loss of generality, that has .  Define for and otherwise.  Then for , so we can plug into the Dirichlet version of Cheeger’s inequality with boundary conditions on .  For the full analysis, see this note which essentially follows this approach.  By examining the proof, note that one can find a subset with by a simple “sweep” algorithm:  Arrange the vertices so that , and output the best of the cuts .

So using the eigenvalue theorem of Spielman and Teng, along with Cheeger’s inequality, we can find a set with .  While this cut has the right Cheeger constant, it is not necessarily balanced (i.e. could be very small).  But one can apply this algorithm recursively, perhaps continually cutting small chunks off of the graph until a balanced cut is collected.  Refer to the Spielman-Teng paper for details.  A great open question is how one might use spectral information about to recover a balanced cut immediately, without the need for recursion.

Conformal mappings and circle packings

Now we focus on proving the bound for any planar graph .  A natural analog is to look at what happens for the Laplace-Beltrami operator for a Riemannian metric on the 2-sphere.  In fact, Hersch considered this problem almost 40 years ago and proved that , for any such Riemannian manifold .  His approach was to first use the uniformization theorem to get a conformal mapping from onto , and then try to pull-back the standard second eigenfunctions on (which are just the three coordinate projections).  Since the Dirichlet energy is conformally invariant in dimension 2, this almost works, except that the pulled-back map might not be orthogonal to the constant function.  To fix this, he has to post-process the initial conformal mapping with an appropriate Möbius transformation.

Unaware of Hersch’s work, Spielman and Teng derived eigenvalue bounds for planar graphs using the discrete analog of this approach:  Circle packings replace conformal mappings, and one still has to show the existence of an appropriate post-processing Möbius transformation.

Circle packings and eigenvalue bounds

First, let’s consider mappings and write

where is the Euclidean norm on   To see that this holds, note that the minimum will always be achieve for some coordinate projection , , or of , by the elementary inequality for any non-negative numbers and .

So how do we get a geometric representation of our graph?  A very natural representation comes from the circle packing theorem of Koebe (rediscovered later by Thurston, following from work of Andreev).  Look here for a historical account, and the relationship with conformal mappings.

Circle-packing theorem: For any planar graph , there exists a set of interior-disjoint circles in such that and are tangent if and only if .

Dan Spielman’s notes contain a nice proof of the theorem using convex programming.

Now, suppose we take such a circle packing, and compose it with a stereographic projection (which takes circles to circles) so that we get a circle-packing on the unit sphere.  Let be the mapping which takes a vertex to the center of its circle on the unit sphere .  If we are somehow lucky, and it happens that , then we could use , along with (1) to get an eigenvalue bound as follows.

stereographic projection

Clearly we have since for every , so it suffices to bound the numerator in (1).  For each vertex , there is a cap on the sphere associated to it.  Let denote the Euclidean length from to the boundary of .  In that case, for any edge , we have

Thus the numerator in (1) is at most .  On the other hand, the area enclosed by is at least , and the caps are disjoint, hence

where is the surface area of .  It follows that the numerator is at most , and therefore .

Of course, this all depended on a statement of the following form, which would guarantee that we can take with .

Given any collection of disjoint caps on the sphere, there exists a circle-preserving map such that in the image, the center of mass of the centers of the caps lies at the origin.

Möbius transformation and moving the center of mass

In fact, we prove the following related, but slightly easier statement, and the interested reader can see Spielman and Teng for the full details.

Transform lemma: Let be points on the unit sphere.  Then there exists a circle-preserving map such that .

This lemma doesn’t manage to prove quite what we need, since the centers of circles do not have to map to the centers of circles in the image, but it captures the essence.

To prove the transform lemma, let’s first introduce some circle-preserving maps.

Let be the stereographic projection with as the pole.

Furthermore, let be the scaling map , with .  Finally, define .  Observe that as , we have for all .  In other words, this map pushes everything on the sphere toward .

Now consider the map given by

where is the open unit ball in .  For , maps .  Also, as approaches , we have for every .

Define by .  Now, as , we have and , as long as we do not have for any .  But this can be handled by spreading the point mass at each out uniformly in a cap of radius . Let be the center of mass of all the -caps around each .

Supposing we do this, we will have as .  This implies that can be extended to a continuous function , and furthermore its restriction to is the identity.  But now basic topology shows that there must exist an for which .  Finally, take to conclude the existence of an with .

The basic topological fact we needed is the following.

Lemma: If is a continuous map and is the identity, then there exists an with .

Proof: Let be defined by .  Note that is continuous.  Now, suppose that 0 is not in the image of , and define a map from to itself by .  By assumption, is continuous, however it has no fixed point, contradicting Brouwer’s theorem.

[Some picture credits go to Wikipedia, Oded Schramm, and Dan Spielman's lecture notes.]

Spines, Foams, and Isoperimetry

Consider again the -dimensional torus , which one can think of as with opposite sides identified.  Say that a nice set (e.g. a compact, surface) is a spine if it intersects every non-contractible loop in .  This is the continuous analog of a spine in .  We will try to find such a spine with surface area, i.e. , as small as possible.

Let’s consider some easy bounds.  First, it is clear that the set

is a spine with .  (A moment’s thought shows that this is “equivalent” to the provers playing independent games in each coordinate of .)

To get a good lower bound, it helps to relate spines to foams which tile according to , as follows.  Take two potential spines.

To determine which curve is actually a spine, we can repeatedly tile them side-by-side.

The first tiling contains the blue bi-infinite curve, which obviously gives a non-trivial cycle in , while the second yields a tiling of by bodies of volume 1.  It is easy to deduce the following claim.

Claim: A surface is a spine if and only if it induces a tiling of the plane by bodies of volume 1 which is invariant under shifts by .

By the isoperimetric inequality in , this immediately yields the bound

where is the unit -dimensional sphere.

So the the surface area of an optimal spine lies somewhere between and .  On the one hand, cubes tile very nicely but have large surface area.  On the other hand, we have sphere-like objects which have small surface area, but don’t seem (at least intuitively) to tile very well at all.  As first evidence that this isn’t quite right, note that it is known how to cover by disjoint bodies of volume at most 1 so that the surface area/volume ratio grows like .  See Lemma 3.16 in this paper, which is based on Chekuri, et. al.  It’s just that these covers are not invariant under shifts.

Before we reveal the answer, let’s see what consequences the corresponding discrete bounds would have for parallel repetition of the m-cycle game.  If the “cube bound” were tight, we would have , which doesn’t rule out a strong parallel repetition theorem ( in the previous lecture).  If the “sphere bound” were tight, we would have , which shows that .   In the latter case, the approach to proving equivalence of the UGC and MAX-CUT conjectures doesn’t even get off the ground.

As the astute reader might have guessed, recently Ran Raz proved that for some constant , showing that a strong parallel repetition theorem—even for unique games—is impossible.   Subsequently, Kindler, O’Donnell, Rao, and Wigderson showed that there exists a spine with .  While it is not difficult to show that the continuous result implies Raz’s discrete result, we will take a direct approach found recently by Alon and Klartag.

Spectral partitioning, Cheeger’s inequality, and Dirichlet boundary conditions

First, we will show that it suffices to find a subset so that the induced graph on contains no non-trivial cycles, and is small, where we use to denote the set of edges from to in .  A variant of this reduction appears in KORW.  For a subset , let .

Random Partitioning Lemma: For any subset which contains no non-trivial cycles, there exists a spine with .

Proof:

Let be i.i.d. uniformly random vectors, and put , with addition done over .  Clearly (with probability 1), we have for some finite time .  Let us define , so that is a partition into disjoint sets.  Since each is isomorphic to , and , no set contains a non-trivial cycle.  So if we define , then we certainly get a spine.

To calculate , we will use a “charging” argument common to many random clustering analyses.  If , then we put .  Clearly we can write , so it suffices to estimate for a fixed .

To this end, note that after conditioning on , is a uniformly random vertex of , and thus

.

By linearity of expectation, it follows that .

So our task is reduced to finding a subset with small and which contains no non-trivial cycles.  A natural method for finding such a set would be to prove a bound on the smallest non-zero eigenvalue of the Laplacian on , and then use Cheeger’s inequality to conclude the existence of a good cut.  But here we have to handle the special condition that should contain no non-trivial cycles.

There is a very natural way to ensure this:  If we impose that our eigenfunction vanishes on the boundary of (of course any other fundamental domain would do), then the standard “sweep” algorithm which chooses a level set of the eigenfunction will always find a set that doesn’t wrap around (and therefore doesn’t contain any non-trivial cycles).  The orange lines are examples of possible level sets.

Dirichlet boundary conditions for the Laplacian.

Fix a graph with .  We define its combinatorial Laplacian as the matrix , where is the diagonal degree matrix with , and is the adjacency matrix of .  Given a subset , we can consider the first Dirichlet eigenvalue with boundary conditions on :

.

By standard variational principles, there always exists a non-negative vector with for and for .  If we take , then this is the lowest-energy norm-1 function on , subject to the boundary conditions.

Next, we need a version of Cheeger’s inequality for .  As usual, we won’t actually need an eigenvector—any vector that has small Rayleigh quotient will do.

Discrete Cheeger inequality: If for all , then

where is the maximum degree in .

For the proof, I’ll refer to Theorem 4 in Alon-Klartag.  Note that the proof for the Dirichlet version is even simpler than for the “standard” (Neumann) eigenvalues—it’s just a straightforward combination of Cauchy-Schwarz and the coarea formula.

Choosing a good eigenvector.

So to finish cheating at the odd-cycle game, we need only produce a low-energy vector on .  We are aided greatly by the fact that this graph is a product of m-cycles, and thus we will only need to know about eigenvectors on the m-cycle .

Let be the adjacency matrix of the m-cycle, and let be the matrix obtained from it by replacing the last row and column by the zero vector (thus is the adjacency matrix of the -path, with an isolated vertex).  Now, the adjacency matrix of is simply , where denotes the k-fold tensor product of a matrix with itself.  Note that if satisfies , then

(1)

It’s easy to see that the vector with is an eigenvector of with eigenvalue and . Therefore is an eigenvector of with eigenvalue .  Using (1), this implies that

Finally, using the fact that the Laplacian of is , we see that

for .

Applying the Discrete Cheeger Inequality above, with implies that there exists a , with

Since , contains no non-trivial cycles.  Applying the Random Partitioning Lemma, and noting that the number of edges in is , we conclude that there exists a spine which cuts at most a

fraction of edges.

As discussed above, this finishes the proof that is the best possible exponent in the parallel repetition theorem (even for unique games), and thus this kind of gap amplification fails to mount an attack on the Unique Games Conjecture.  In the next lecture, we’ll see a more benevolent use of eigenvectors.

]]> http://tcsmath.wordpress.com/2008/09/21/lecture-1-cheating-with-foams/ Sun, 21 Sep 2008 20:35:06 +0000 James Lee http://tcsmath.wordpress.com/2008/09/21/lecture-1-cheating-with-foams/

This the first lecture for CSE 599S:  Analytical and geometric methods in the theory of computation.  Today we’ll consider the gap amplification problem for 2-prover games, and see how it’s intimately related to some high-dimensional isoperimetric problems about foams.  In the next lecture, we’ll use spectral techniques to find approximately optimal foams (which will then let us cheat at repeated games).

The PCP Theorem, 2-prover games, and parallel repetition

For a 3-CNF formula , let denote the maximum fraction of clauses in which are simultaneously satisfiable. For instance, is satisfiable if and only if . One equivalent formulation of the PCP Theorem is that the following problem is NP-complete:

Formulation 1.

Given a 3-CNF formula , answer YES if and NO if (any answer is acceptable if neither condition holds).

We can restate this result in the language of 2-prover games. A 2-prover game consists of four finite sets , where and are sets of questions, while and are sets of answers to the questions in and , respectively. There is also a verifier which checks the validity of answers. For a pair of questions and answers , the verifier is satisfied if and only if . The final component of is a probability distribution on .

Now a strategy for the game consists of two provers and who map questions to answers. The score of the two provers is precisely

where is drawn from . This is just the probability that the verifier is happy with the answers provided by the two provers. The value of the game is now defined as

i.e. the best-possible score achievable by any two provers.

Now we can again restate the PCP Theorem as saying that the following problem is NP-complete:

Formulation 2.

Given a 2-prover game with with , answer YES if and answer NO if .

To see that Formulation 1 implies Formulation 2, consider, for any 3-CNF formula , the game defined as follows. is the set of clauses in , is the set of variables in , while and . Here represents the set of eight possible truth assignments to a three-variable clause, and represents the set of possible truth assignments to a variable.

The distribution is defined as follows: Choose first a uniformly random clause , and then uniformly at random one of the three variables which appears in . An answer is valid if the assignment makes true, and if and are consistent in the sense that they give the same truth value to the variable . The following statement is an easy exercise:

For every 3-CNF formula , we have .

The best strategy is to choose an assignment to the variables in .  plays according to , while plays according to unless he is about to answer with an assignment that doesn’t satisfy the clause .   At that point, he flips one of the literals to make his assignment satisfying (in this case, the chance of catching cheating is only , the probability that is sent the variable that flipped).

This completes our argument that Formulation 1 implies Formulation 2.

Parallel repetition
A very natural question is whether the constant in Formulation 2 can be replaced by (or an arbitrarily small constant). A natural way of gap amplification is by “parallel repetition” of a given game. Starting with a game , we can consider the game , where is just the product distribution on . Here, we choose pairs of questions i.i.d. from and the two provers then respond with answers and . The verifier is satisfied if and only if for every .

Clearly because given a strategy for , we can play the same strategy in every coordinate, and then our probability to win is just the probability that we simultaneously win independent games. But is there a more clever strategy that can do better? Famously, early papers in this arena assumed it was obvious that .

In fact, there are easy examples of games where .  (Exercise: Show that this is true for the following game devised by Uri Feige. The verifier chooses two independent random bits , and sends to and to . The answers of the two provers are from the set . The verifier accepts if both provers answer or both provers answer .)

Nevertheless, in a seminal work, Ran Raz proved the Parallel Repetition Theorem, which states that the value of the repeated game does, in fact, drop exponentially.

Theorem 1.1: For every 2-prover game , there exists a constant such that if , then for every ,

The exponent 3 above is actually due to an improvement of Holenstein (Raz’s original paper can be mined for an exponent of 32).

Special games and the unique games conjecture

There are two special times of games which should be emphasized, depending on the structure of the mapping . A projection game is one in which, for every and , there is at most one value for which . In other words, after fixing an answer to the first question, there is at most a single answer to the second question which the verifier accepts. The 3-CNF game above is an example (given an assignment to the variables in the clause , the second prover has to give the consistent assignment to the variable ). Recently, Anup Rao gave an improved parallel repetition theorem for this special case.

Theorem 1.2: For every 2-prover projection game , with , and for every ,

Notice that in addition to the improved exponent of , there is no dependence on . (This dependence is known to be necessary for general games.)

Why do we care about the exponent?
Our main focus in this lecture will actually be on the exponent of in the preceding theorems, so it seems like a good time to discuss its importance. For that, we need to introduce unique games.  This is a game where , , and the verifier behaves as follows. Given a pair of questions , there exist a bijection such that if and only if . In other words, after fixing a pair of questions, for every answer of one prover there exists a unique answer of the other prover that satisfies the verifier.  (This is almost the same as satisfying the projection property from both sides, except that here we have enforced that after fixing and , there exists exactly one satisfying instead
of at most one.)

Now we state Khot’s unique games conjecture (UGC), which has far-reaching consequences in hardness of approximation (and is the central open question in the field).

Conjecture (UGC): For every , there exists a such that the following problem is NP-complete: Given a unique game with , answer YES if and answer NO if .

At present, there are very few good ideas on how to attack this conjecture, and there are differing beliefs about its truth. Observe that, unlike Formulations 1 and 2 above, in the YES instance, we now only require .  Of course, this problem is harder than distinguishing from . In fact, this is fundamental: It is easy to design an efficient algorithm which checks whether for a unique game.  (Exercise: Verify this.)

We now outline one possible approach to proving the UGC, in which the exponent of becomes crucial. For an undirected graph , define

where denotes the set of edges from to its complement. Consider the following conjecture.

Conjecture (MAX-CUT conjecture): There exists a constant such that for every , the following problem is NP-complete: Given a graph , distinguish between the two cases

Note that this conjectured hardness ( vs. ) is tight, as shown by the Goemans-Williamson algorithm.  See also a recent spectral algorithm of Luca Trevisan that achieves the same bound.  From work of Khot, Kindler, Mossel, and O’Donnell (and the subsequent Majority is Stablest theorem, which we’ll get to later in the course), we know that the UGC implies the MAX-CUT conjecture. Might we prove that the two conjectures are actually equivalent?

To address this, let’s first observe that we can turn the MAX-CUT problem into a 2-prover unique game in a straightforward way. Given a graph , consider the game where and .

The distribution on questions is as follows. With probability each: (1) Choose a uniformly random vertex and ask the questions , or (2) choose a uniformly random edge and ask the questions . The verifier is satisfied in case (1) only if the two provers give the same answer for . The verifier is satisfied in case (2) only if the two provers give different answers. It only takes a moment’s though to see that the two provers should fix a maximum cut in the graph, and play according to it, yielding

Now we are ready to see that the two conjectures are equivalent if we can obtain a good-enough exponent in parallel repetition of unique games. To this end, let be the infimal value of such that there exists a constant so that for every unique game , we have

for every . From Theorem 1.2, we know that . A little better would show the equivalence of these two conjectures.

Claim 1.3: If , then the the MAX-CUT conjecture implies the UGC.

Proof: The proof is straightforward: If , then , which implies that . On the other hand, if , then , and for some ,

for . Taking , we get in the first case, and in the other. Since is a unique game whenever is unique, this completes the proof.

MAX-CUT on a cycle

To this end, a number of authors have asked whether a strong parallel repetition theorem holds, i.e. whether (even for general games).

The rest of the lecture follows Feige, Kindler, and O’Donnell.  We’ll consider a very simple unique game: MAX-CUT on an odd cycle, though we’ll slightly change the probabilities of our earlier verifier for the sake of convenience. We define the game as follows: , , and is specified by first choosing uniformly at random, and then choosing where is chosen uniformly at random. As before, when , accepts only if the answers satisfy , and otherwise accepts only if the answers satisfy .

It is easy to check that for odd, we have

since any cut in an odd cycle must have some some edge which doesn’t cross it (e.g. the edge with two green endpoints above). The point now is to understand the behavior of .

Toward this end, consider the graph whose vertex set is and which has an edge whenever for (this is also known as the -fold AND-product of an -cycle with itself).  For instance, here is , where the dashed edges are meant to wrap around.

There is a natural embedding of into the -dimensional torus , which endows oriented cycles in with a homotopy class from .

Given such a cycle , we use to denote the corresponding element of the fundamental group. if wraps around the torus times in the th direction.

A cycle is said to be topologically non-trivial if .  This is the same as saying that cannot be continuously contracted to a point in .  The cycle is topologically odd if is odd for some .  Here are two topologically odd cycles on the 2-torus, corresponding to the classes and .

In the following pictures, the red cycle is topologically trivial, the blue cycle is simply a shift of the red cycle, and the purple cycle is topologically non-trivial (indeed, it is topologically odd).

Finally, consider the double cover , which is a bipartite graph with vertex set , and with an edge whenever is an edge in or . Oriented cycles in inherit the notions of topological non-triviality and oddness from the projection .  A curve in is said to be non-trivial or odd if its projection is.  Now consider the following problem.

Odd-cycle elimination problem in :

Remove the minimum-possible fraction of edges from so as to delete all topologically odd cycles.

For example, here is a natural set of blocking edges (projected from onto ).

It turns out that this is just in disguise.  For example, the above set of blocked edges corresponds to a strategy where and play each coordinate independently.  More generally, we have the following:

Claim 1.4: .

Proof:

Consider two provers for the repeated game .  Edges in correspond to pairs of questions .  Let be the set of edges corresponding to questions on which answer incorrectly, i.e. edges for which .  Clearly

where we recall that is the number of edges in .

Say that an arbitrary set of edges is blocking if removing leaves no topologically odd cycles.  We need to show that every set of the form is blocking, and that from every blocking set , we can recover two provers with .

First, assume that is not blocking.  Then there exists a cycle in and a coordinate such that is odd.  But if we consider the projection of onto coordinate , then we get an odd cycle on which play perfectly, which is clearly impossible.

On the other hand, consider an arbitrary blocking solution .  From any connected component of , choose an arbitrary vertex, say , and put .  Once this is set, since we cannot violate any edges in , there is a unique greedy extension of to the rest of .  For instance, if , then we must put as well.  If and and differ by 1 in the th coordinate, then , where the 1 occurs in the th coordinate.  It is easy to see that this greedy assignment cannot fail precisely because there are no topologically odd cycles remaining in (every bipartite graph has a cut which contains all edges).  Note that topologically trivial cycles are inconsequential since the coordinate projection of such a cycle results in a path.  This completes the proof.

Now define to be the minimum be number of edges which need to be removed from so as to eliminate all topologically non-trivial cycles. It is easy to see that because we can remove all topologically non-trivial cycles in (and, in particular, all topologically odd cycles) by taking a set of edges which do this for and using the edges in .

Thus in order to “cheat” at the odd-cycle game, we only need to find small sets of edges whose removal blocks all non-trivial cycles.  In this next lecture, we will see how this is intimately related to foams which tile , and how Dirichlet eigenfunctions help us find good foams.

[Credits:  Some pictures taken from this talk of Ryan O'Donnell.]