One of the primary goals of complexity theory is separating complexity classes, a.k.a proving lower bounds. Embarrassingly we have only a handful of unconditional separation results. Separating P from NP is of course the mother of all such goals. Anybody who understands the philosophical underpinnings of the P vs NP problem would love to LIVE to see its resolution. Towards resolving this, we made some (“anti”)-progress (Eg : Relativization, Natural proofs, Algebrization) and have a new geometric complexity theory approach which relies on an Extended-Extended-Extended-Extended-Riemann-Hypothesis !! For more information about the history and status of P vs NP problem read Sipser’s paper [Sipser'92], Allender’s status report [Allender'09] or Fortnow’s article [Fortnow'09].

Today’s post is about the Logspace (L) vs Polynomial time (P) problem, which (in my opinion) is right next to the P vs NP problem in its theoretical importance. I guess many researchers believe that . Did we make any progress/anti-progress towards resolving the conjecture ?  Here are two attempts both based on branching programs and appeared in MFCS with a gap of 20 years !!

1) A conjecture by Barrington and McKenzie (BM’89): The problem is defined as follows :

: Given an table filled with entries from , which we interpret as the multiplication table of an -element groupoid, and a subset of which includes element 1, determine whether the subgroupoid , defined as the closure of under the groupoid product, includes element .

Barrington-McKenzie Conjecture : For each , a branching program in which each node can only evaluate a binary product within an -element groupoid, branching ways according to the possible outcomes, must have at least nodes to solve all instances with singleton starting set .

The problem is known to be P-complete [JL'76]. Barrington-McKenzie Conjecture would imply that for any . In particular, it would imply that . I don’t know if there is any partial progress towards resolving this conjecture.

2) Thrifty Hypothesis : This is a recent approach by Braverman et. al [BCMSW'09] towards proving a stronger theorem . Stephen Cook presented this approach at Valiant’s 60th birthday celebration and Barriers Workshop. He also announced a $100 prize for solving an intermediate open problem mentioned in his slides.

Tree Evaluation Problem (TEP): The input to the problem is a rooted, balanced -ary tree of height , whose internal nodes are labeled with -ary functions on , and whose leaves are labeled with elements of . Each node obtains a value in equal to its -ary function applied to the values of its children. The output is the value of the root.

In their paper they show that and conjecture that . They introduce Thrifty Branching Programs and prove that TEP can be solved by a thrifty branching program. A proof of the following conjecture implies that . For more details, read this paper.

Thrifty Hypothesis : Thrifty Branching Programs are optimal among deterministic branching programs solving TEP.

Open Problems :

• My knowledge about the history of L vs P problem is limited.  Are there other approaches/attempts in the last four decades to separate L from P ?
• An intermediate open problem is mentioned in the last slide of these slides. The authors announced $100 prize for the first correct proof. Read their paper for more open problems.

References :

Read here the entire article on DDJ.com: http://www.ddj.com/architect/221600058

I love this remark:
“NP-completeness “tells you something very specific:It tells you that if you’re going to look for an algorithm that’s going to work in every case and give you the best solution, you’re doomed: don’t even try. That’s useful information.”

I am back in Atlanta after attending an awesome Barriers Workshop. This workshop is mainly about the barriers in resolving P vs NP problem and possible techniques to overcome these barriers. It is a five day workshop covering circuits lower bounds, arithmetic circuits, proof complexity, learning theory and pseudo-random generators. I enjoyed every talk and panel discussions. Everybody seemed very positive that .

Todays post is a quick introduction to Relativization Barrier. I will write separate posts about natural proofs and algebrization barriers soon. Relativization Barrier is one of the first barriers we learn in an introductory graduate level complexity course.

In relativized world we grant the Turing machine M the access to an oracle that could compute some function A in a single time step. Such a machine, called oracle TM, computes relative to A, is represented by . Oracle TMs are used in Turing-reducibility, a generalization of mapping-reducibility.

The method of diagonalization relativizes i.e.,  any separation result in the real world, proved using diagonalization, extends to the relativized world. Baker, Gill and Solovay [BGS'75] showed that there exists oracles A and B for which and are provably different and and are provably equal. That is P vs NP has different answers in different worlds !! Hence techniques such as diagonalization, cannot be used to resolve P versus NP.

Theorem : An oracle A exists such that . An oracle B exists such that .

Let B be any PSPACE-complete problem. We have . The first and third containments are trivial and the second containment follows from Savitch’s theorem. Hence . Constructing A is a non-trivial task. Look at Sipser’s book (page 350) for details.

Therefore any solution to the P versus NP problem will require non-relativizing techniques i.e., techniques that exploit properties of computation that are specific to the real world i.e., techniques that analyze the computations not just simulate them.

References :

• [BGS'75] T. Baker, J. Gill, and R. Solovay : Relativizations of the P=?NP question. SIAM J. Comput., 4:431–442, 1975.

Approaches to AI

The introductory chapter starts with a four-fold categorization of the approaches that have been taken in AI:

1. Thinking like Humans
Cognitive science pretty much sums up this category. The idea is to understand how humans make intelligent decisions (using neuroscience, psychology, etc) and model it on an artificial system.

2. Acting like Humans
Here the focus is on the results produced by humans. Any approach which has as it’s aim passing the Turing test can be considered in this category. AI approaches that can be seen as attempts to completely build an ‘artificial human’ are considered to be in this group: natural language processing, knowledge representation, automated reasoning, machine learning, vision, robotics, etc.

3. Thinking Rationally
The logicist tradition occupies this category. Focus: represent knowledge, make logical inferences on it to make decisions.

4. Acting Rationally
The main approach considered in the book. Any approach that attempts to build an agent whose actions can be considered rational can be considered to be in this category.

For the first two categories of approaches, success would be measured with respect to a human – which is quite a difficult task. With the latter two approaches, it would be measured to a more well defined rationality.

In my opinion, it is important to notice that the categorization is not absolute and there is considerable overlap; particularly, the last two categories do not have a clear boundary. From one perspective, to “think rationally” is to decide how to infer the most rational action to take, and hence, category 3 is included in category 4. On the other hand, if it can be considered that any agent that makes a rational action has implicitly made a rational decision, then category 4 is included in category 3.

The book cites the following example: “For example, recoiling from a hot stove is a reflex action that is usually more successful than a slower action taken after careful deliberation”. I disagree with this example, since the decision to recoil is a rational decision, taken after inferring that there is not enough time to engage in more complex decision making and that an action must be taken as quickly as possible. i.e. The example is a case of making logical inferences with temporal logic.

Question: To which category do neural network approaches fall?

Philosophy

Since Aristotle’s syllogisms there have been attempts to create formal reasoning systems. Philosophy discusses the limits and possiblities of logic, and also of AI.

Some important terms introduced are:
1. Dualism: the idea that there is a immaterial part to the universe, that the mind is not a manifestation of the physical brain. (Was this Descartes’ big mistake?)

2. Materialism: the physical universe is all that exists.

3. Empiricism: knowledge is distilled from one’s experiences.

4. Hume’s principle of induction: general rules are aquuired by exposure to repeated association between their elements. (Is this the same as pattern recognition?)

5. Logical positivism: all knowledge can be characterised by logical theories, and all meaningful statements of that logic system can be verified or falsified. Developed by the infamous Vienna Circle.

6. Confirmation Theory: attempts to understand how to do induction; i.e. how knowledge can be aquired from experience.

One good example of how Philosphy has lended to concrete AI systems is Newell and Simons’ General Problem Solver (GPS). The basic regression planning algorithm used in GPS is none other than the one proposed by Aristotle: consider what the outcome is, and plan backwards from it – see what actions lead to the outcome, and then what actions lead to those actions, and so on.

Goal based analysis is discussed in Philosophy with respect to the question of understanding the relation between thinking and acting.

Mathematics

The contribution from mathematics can roughly be categorised into three areas:

1. Logic
While philosophy gave birth to logic, the mathematical development of logic into a formal system was what enabled it to become a strong tool that can be used for AI.

2. Computation
Decidability
This is linked to logic and number theory. Of particular interest is Hilbert’s decision problem, which questions whether it was possible to find an algorithm to decide the truth value of any logical proposition involving the natural numbers – i.e. whether there were limits to the power of effective proof procedures.

Computability
Gödel’s findings tell that
1) any statement expressed in first order logic that is true can be proved of it’s truth
2) first order logic is not powerful enough to capture the principle of mathematical induction needed to characterise the natural numbers
3) incompleteness theorem: any language expressive enough to capture the natural numbers fully (i.e. describe all the properties of the natural numbers) has, in that language, statements that are true, but their truth cannot be established by an algorithm. This means that some functions on the natural numbers cannot be computed by an algorithm.

Turing brought the idea of computability – that is, to find which functions can be computed and which cannot. The Church-Turing thesis presents the idea of a Turing machine that is capable of computing any computable function.

Tractability
From a practical point of view, tractability is a much more useful concept to study, because even if a function is computable theoretically, it can be practically uncomputable because the resources required increase exponentially with respect to input size. Such problems are said to be intractable.

Although it has not been proven (it is one of the remaining Millenium problems) it is generally assumed that NP-complete problems are intractable.

3. Probability
The most important contribution from probability is Bayesian analysis, borne out of Baye’s theorem which shows how probabilities change when new conditions are added to an event. Bayesian analysis forms the basis of most approaches to dealing with uncertainity in AI.

Economics

1. Decision Theory
Utility theory investiages how decisions can be made leading to a peferred outcome; utility theory combines with probability results in decision theory, where decisions made under uncertain conditions can be investigated. Here probability captures the decision maker’s environment, and it is assumed that the decision maker’s world is not affected by others.

2. Game Theory
If the decision maker also needs to consider what the other decision makers are doing, then such problems are studied in game theory.

3. Operations Research
One important contribution from OR to AI is Markovian decision processes, where a number of sequencially made decisions are studied.

Note that decision theory, game theory and operations research are fields that are difficult to classify only as either mathematics or economics, as they are a hybrid of both.

Neuroscience

Neuroscience can help in discovering how our brains manage to function as intelligent agents. Particularly the invention of fMRI has been very helpful in monitoring brain activity.

Psychology

1. Behaviourism
Behaviourism rejects mental introspection and only considers objective results of a psychological experiment. While behaviourism has helped to understand animal (non-human) behaviour, it has been less successful at understanding human behaviour.

When you look at it, this should be expected from behaviourism. Behaviourism treats the intelligent-process of the agent as a black box and looks at only the inputs (percepts from the environment) and outputs (actions of the animal). It is reasonable to assume that the function which takes place inside the black box is simple and animals and much more complex in humans. Thus it would be easy the guess the function for animals, but not for humans.

2. Cognitive Psychology
Cognitive psychology, in contrast, attempts to understand exactly the functionality that is taking place inside the brain, and does not exclude mental introspection (e.g. examining a persons beliefs and goals). Such mental introspection can be considered to be parts of a virtual representation of the world created in the brain.

Cognitive science, where cognitive psychological models are developed as computed models, thus pay a lot of attention to how the external world can be represented in an intelligent agent.

Control Theory and Cybernetics

Control theory explores how an automated system can monitor and regulate itself. Cybernetics, from an AI perspective is mainly the application of control theory to computational models of cognition to produce AI.

Due to the different areas of mathematics used in control theory and AI, there is some gap between the two fields. Control theory is built using calculus and matrix algebra, whereas AI (at least traditionally) used logic and computation. The problems of language, vision and planning, that were considered from AI perspectives, fell outside the domain of control theory due to the different mathematical tools used.

Linguistics

Chomsky introduced the idea of syntactic structures, which could capture the potential of a language so that it could be modeled computationally. This introduced linguistics to AI, resulting in the field of natural language processing. The main obstacle in NLP is understanding the context and subjec matter of the language (which is required due to the ambigious nature of human language), and thus knowledge representation is the field devoted to studying how information about the world can be captured into a computational structure so the information can be utilized by an intelligent agent.

Kali ini saya mau menuliskan salah satu masalah dari seven melinium problem yaitu P vs NP. Ini adalah masalah termuda dari seven melinium problem, dicetuskan oleh Stephen Cook pada tahun 1971 dan ini adalah masalah didalam Algoritma.

P

Misalkan kita punya masalah menghitung nilai  determinan matriks bujur sangkar berukuran n maka dengan program komputer/algoritma   dengan mudah kita mengetahui solusinya. Nah..masalah menghitung nilai determinan ini disebut decision problem (masalah yang dapat diputuskan) yaitu masalah yang dapat ditemukan solusinya aloh suatu algoritma . Contoh-contoh decision problem yang lain adalah: mengkonnversi nilai celcius ke fahrenhait, menghitung perkalian dua bua matriks, dll pokoknya banyak lah masalah-masalah yang termasuk decision problem, cari aja sendiri contoh yang lainnya 

Nah..sekarang balik lagi ke masalah penghitungan matriks, kita tau bahwa semakin besar ukuran matriksnya maka semakin lama komputer memproses jawabannya, dengan rumusan waktu proses   untuk suatu a bilangan bulat positif, berapa a? nah..itu ada rumusan sendiri untuk mencarinya, saya tidak akan membahasnya. Nah rumusan waktu tersebut dikatakan polynomial time (waktu polynomial)

Suatu masalah berukuran n byte  dikatakan termuat di kelas P, jika algoritma membutuhkan polynomial time untuk memproses solusnya.atau dengan kata lain suatu masalah berukuran n byte termuat di P jika ada algorima A yang akan memproses solusi paling lama

Pada umumnya masalah-masalah di P mempunyai nilai a paling besar 4, tapi pada kasus ekstrim seperti malah tes keprimaan nilai a sebesar 6

NP

Nah..jika masalah di P, algoritma  memproses solusinya tetapi masalah  di NP, algoritma hanya mengecek/memverifikasi solusi dengan masalah, apakah cocok/tepat atau tidak.  Misakan kita punya suatu masalah NP, jika kita masukan ke algoritma maka algoritma akan mgecek maslah NP tersebut dengan dugaan-dugaan solusi satu persatu, apakah cocok atau tidak.

Contoh masalah NP adalah Traveling  salesman Problem (TSP)

Bayangkan kalian akan mengunjungi lima kota untuk keperluan dagang dan kalian semua jarak masing-masing kota, Pertanyaannya jalur mana yamg terpendek untuk mengunjungi semua kota dan kembali lagi ke kota semula?  Apakah A-B-C-E-D-A?apakah A-D-E-C-B-A?

Tentu saja untuk mencari solusinya adalah mencoba semua kemungkinan.

Semakin banyak kotanya maka semakin banyak pula kombinasinya.  Jadi metode ini memeerlukan “waktu faktorial” t=n!

Misalkan suatu algoritma pada suatu komputer dapat menyelesaikan TSP dengan 20 kota dalam waktu 1 detik maka 21 kota akan diselesaikan dalam waktu 21 detik dan 22 kota dalam waktu 462 detik (sekitar 7menit) kalau 30 kota akan membutuhkan waktu sebanyak  3 milyar tahun, wow..

Dari contoh TSP diatas diketahui bahwa untuk memecahkan suatu masalah NP, paertama-tama algoritma akan memproses dugaan solusi kemudian mencocokannya dengan masalah apakah cocok atau tidak kalau cocok maka berhenti kalau tidak cocok maka kembali memproses dugaan solsusi lain dan mencocokannya kembali begitu seterusnya sampai ditemukan solusi yang benar-benar cocok. Jadi masalah NP mememerlukan waktu yang jaug lebih lama untuk memecahkannya dibanding masalah P dan jelas bahwa

Nah..sekarang yang menjadi pertanyaan

Apakah P=NP?

Atau dengan kata lain apakah kita bisa membuat algoritma ber”polynomial time” untuk memecahkan masalah NP? Apakah sebenernya kita bisa membuat algoritma yang memecahkan masalah NP sama cepatnya dengan algoritma untuk memecahkan masalah P?

Banyak orang yang berkeyakinan nahwa P≠NP, tetapi sampai saat ini belum ada yang mampu memebuktikan secara matematis apakah P=NP atau kah P≠NP

Andaikan P=NP

Jika suatu saat terbukti secara matematis bahwa P=NP, maka berakibat semua masalah computable yaitu masalah yang bisa diproses melalaui komputer dengan mudah ditemukan solusinya. Banyak masalah dalam ilmu Riset Operasi yang termasuk NP, jika P=NP maka ilmu riset operasi akan melompat jauh kedepan ini berakibat semua proses Industri di segala bidang akan bekerja jauh lebih efisien. Tapi P=NP merupakan mimpi buruk bagi ilmu kriptografi karna itu artinya akan ada metode lain yang jauh lebih cepat, lebih efisien untuk memecahkan suatu enkripsi selain brute force

]]> http://janpich.wordpress.com/2009/03/20/filozoficko-historicky-pohlad-na-p-vs-np/ Fri, 20 Mar 2009 11:45:36 +0000 Nanyk http://janpich.wordpress.com/2009/03/20/filozoficko-historicky-pohlad-na-p-vs-np/ http://janpich.wordpress.com/2009/02/23/diagonalizacia-a-relativizacia-a-jedna-uloha-a-jej-riesenie-a/ Mon, 23 Feb 2009 15:50:05 +0000 Nanyk http://janpich.wordpress.com/2009/02/23/diagonalizacia-a-relativizacia-a-jedna-uloha-a-jej-riesenie-a/ http://openprogram.wordpress.com/2009/02/15/is-the-pnp-question-valid-or-a-product-of-sloppy-mathematics/ Sun, 15 Feb 2009 01:07:23 +0000 Petar http://openprogram.wordpress.com/2009/02/15/is-the-pnp-question-valid-or-a-product-of-sloppy-mathematics/

Sloppiness Level I

NP-hard problems were discovered in situations like this one:

A bunch of engineers designed a processor circuit and then, for whatever hardware-related reasons, they needed to know where was the SPARSEST CUT on their circuit, so they could lay it out on the multi-layeed silicon wafers more efficiently.

This is an over-simplification of the history, but the spirit is right. The circuit is a graph, the sparsest cut is what the engineers (thought they) were looking for, so the question was posed:

(1) Can we find the sparsest cut of a graph?

Behold the first layer of “sloppiness”. The question should have been:

(2) Can we find the sparest cut of graphs that we could possibly produce as circuits (or any other class of graphs that arises in a natural context)?

Now, I am going to dwell on this layer of “sloppiness” for a few paragraphs.

Mathematically it is more-or-less OK to start from the simplest-to-state and most obvious question. But having come to the realization that this is a problem we still know little about (we don’t even know how inapproximable it is for sure), perhaps it is time to rethink. Consider where problems come from before they are laid out to the Computer Scientist to solve. The instances of any combinatorial problem, arguably, come from some natural process or phenomenon (which is computational in nature — an assumption we are bound to work with). So it would appear that the question to ask should be:

(3) If a problem instance was generated by a randomized polynomial-time algorithm, then can we solve the problem (a combinatorial question about the instance) with a randomized polynomial-time algorithm?

In other words, any asymptotic family of instances that is not the product of a poly-time algorithm might be perverted beyond reason (i.e. beyond poly-time intelligent reasoning), which would explain why the P=NP question has never been solved. This isn’t so hard to imagine … there are levels and levels of complexity classes that are way beyong the PSPACE hierarchy, known to Mathematicians (was it Recursion Theory?) before Computer Science even existed. Why should we believe that we can solve a problem on an instance that comes from one of these higher complexity classes?

If one-way functions exist, then the answer to question (3) would be negative (but wait until the end of the article), however their non-existance sheds no light on question (3). Therefore, morally it seems more appropriate to address question (3) directly.

Sloppiness Level II

I am now going to focus on the SPARSEST CUT problem in particular. In my opinion, it is hard to even justify this problem as natural or useful in practice, even if it were solvable — and for a simple reason: it’s solution is unstable and lacking basic properties.

Let’s define the problem first, so we have a grip on the subject. Given an unweighted (also bounded-degree, if you wish) graph , the goal is to find a bi-partition of the vertices which minimizes the quantity

,

where the nominator counts the number of edges crossing the cut. The intuition here is that the smaller side of the cut, call it , has many more edges inside than the edges that connect it to the remainder of the graph. This intuition leads most to believe that is “well-connected” on the inside, but “weakly-connected” to the rest. Not true. A little tinkering will convince you that could, in fact, be completely disconnected — hardly a property that those engineers (from the first paragraph above) would have wanted. It is difficult to think of any application where this is OK.

On to stability now. Convince yourself that the following is true. There are plenty of examples where by deleting a single edge, you can change the size of the smaller side of the cut by an arbitrarily large amount, think a factor of . Once again, quite disconcerting from an application stand-point.

Note that the above concerns are not remedied by the BALANCED SPARSEST CUT problem, which has similar troubles of its own.

My conclusions

I don’t believe that the P=NP question is justified (even as a matter of Mathematical pride), since there is a sense that it could be humanly unreachable. If anything, we should be focusing on question (3). And if question (3) still appears too hard, we should be aiming at constraining the combinatorial problems in consideration so that they don’t fail the basic sanity checks illustrated in “Sloppiness Level II”. This latter observation also suggests that even if one-way functions exist, it may still be the case that all “stable” combinatorial problems are solvable. This thinking is encouraging because it divorces the search for efficient algorithms from the domain of cryptography.

This being said, the issues of “Sloppiness Level II” are essentially addressed by Inapproximability Theory (in Computer Science). Saying that a problem can be C-approximated is asserting that doing away with some of the rigidity in the initial problem statement leads to a “natural” problem that succumbs to a princpled solution. It is curious to verify this intuition by examining existing approximation algorithms and showing that the approximations they produce are in fact stable in the sense alluded to above.

Still I believe it is worth combinging question (3) with inapproximability efforts, and independently devising notions of natural combinatorial problems that by their very definition avoid “Sloppiness Level II”.

I found this post from 0xDE. The abstract of the paper referred to in the post contains a very funny sentence-“it is demonstrated that a deterministic polynomial time solution to any NP-Complete problem does not necessarily produce a deterministic polynomial time solution to all NP-Complete problems“. How can you prove a result which is the exact opposite of the definition!.

Check this out to find various “proofs” of the P vs NP problem. As they say “every difficult problem has a simple, easy to understand wrong solution”.

Cómo la semana pasada terminé de dar mis exámenes finales de julio, tuve un poco más de tiempo para dedicarme a los temas que me interesan. Hace rato que tenía ganas de empezar un curso de OpenCourseWare, el sitio de cursos del MIT (Massachusetts Institute of Technology). El elegido fue “Computability, and Complexity”.

El curso es bastante completo, incluye lecturas, ejercicios y clases de repaso. Al terminarlo, te encuentras preparado para encarar uno de los problemas matemáticos por los que el Clay Mathematics Institute. ofrece 1.000.000 de dólares:el problema P vs NP.

Obviamente no persigo resolver el problema, pero la complejidad computacional es un campo de estudio que aún no está muy desarrollado y realmente me parece muy interesante.

Por último dejo, para los que quieran iniciarlo, el link de la mula para el libro en el cual se basa todo el curso: Sipser, Michael. Introduction to the Theory of Computation. 2nd ed. Boston, MA: Course Technology, 2005. ISBN: 0534950973.

Came across a poetic version of the well-known problem in theoretical computer science.

And a quotable quote by Thomas Carruthers:

a teacher is one who makes himself progressively unnecessary.

A few more quotes are topically categorized here.