Ool la la!

Basic statistics is more useful than advanced machine learning.

I can’t tell you how many interviews I’ve had where someone has a really cool project on their resume. Support vector machines, topic analysis on CiteSeer, or whatever… But what it boils down to is someone took toy data set A and plugged it in to machine learning library B and took the output and was like, “sweet.”

People with “machine learning” on their resume fall from the sky these days, it seems to be a very sexy discipline. The problem is if I ask them explain a t-test, those same people can’t tell me what that is.

If I had a MacGyver of data analysis and all he had was a t-test and regression, he would probably be able to do 99.9% of the analyses that we do that are actually useful. (Jeff Hammerbacher)

“Công thức của vận may” là một trong những cuốn sách hấp dẫn nhất của William Poundstone (tác giả cuốn sách bán chạy tại Việt Nam “Làm thế nào dịch chuyển núi Phú Sĩ?”). Tác giả tiết lộ bí mật về một công thức toán học – được gọi là “Công thức Kelly” giúp bạn nắm bắt được vận may tại sòng bạc và sàn chứng khoán. Một cuốn sách kết hợp giữa cờ bạc, cá ngựa, đầu tư chứng khoán và sự chính xác của toán học, một cuốn cẩm nang cho những người muốn áp dụng công thức Kelly để làm giàu.
Dự án X

NÓ ĐƯỢC GỌI LÀ DỰ ÁN X. Mãi đến năm 1976, dự án này mới được tiết lộ và đây là một nỗ lực chung của cả Bell Labs và Trường Mật mã và Kí hiệu Anh quốc đặt tại Bletchley Park, miền bắc Luân Đôn. Nội dung của dự án X mang tính cạnh tranh với nội dung của dự án Manhattan, do một nhóm các nhà khoa học Anh và Mỹ đảm nhận, không chỉ có Shannon mà cả Alan Turing tham gia. Họ đang xây dựng một hệ thống gọi là SIGSALY. Đây không phải là chữ viết tắt của một cái tên nào mà chỉ là sự kết hợp ngẫu nhiên của một nhóm chữ cái nhằm làm cho người Đức lúng túng nếu như họ có nghiên cứu về nó.

SIGSALY là chiếc điện thoại vô tuyến và có thể đổi tần số kỹ thuật số đầu tiên. Mỗi chiếc SIGSALY là một cỗ máy tính có kích cỡ bằng một căn phòng, nặng 55 tấn với một phòng riêng biệt dành cho người sử dụng và một hệ thống điều hòa không khí để ngăn không cho các đèn điện tử chân không của máy bị nóng chảy. Chiếc máy này là giải pháp giúp các nhà lãnh đạo phe đồng minh nói chuyện với nhau một cách thoải mái mà không lo bị kẻ thù nghe trộm. Phe Đồng minh đặt một chiếc SIGSALY ở Lầu Năm Góc cho Roosevelt và một chiếc khác ở tầng hầm của một cửa hàng Selfridges cho Churchill. Hai chiếc nữa được đặt ở Bắc Mỹ cho Field Marshal Montgomery và ở Guam cho tướng MacArthur.

SIGSALY sử dụng một hệ thống mật mã duy nhất được coi là không thể bẻ gãy – mật mã “mã hóa một lần”. “Từ khóa” dùng cho một tin nhắn được xáo trộn và mã hóa một cách ngẫu nhiên. Chìa khóa để giải mã bao gồm một dãy những chữ cái hay chữ số được sắp xếp không theo một quy luật nào, do đó thông tin của chìa khóa cũng là ngẫu nhiên, không chứa đựng bất kì một quy luật nào có thể dựa vào đó mà giải mã. Vấn đề của mật mã “mã hóa một lần” này là chìa khóa phải được người đưa tin chuyển đến tất cả những người đang sử dụng hệ thống, một thách thức thực sự trong thời chiến.

SIGSALY mã hóa dữ liệu âm thanh tốt hơn dữ liệu văn bản. Chìa khóa của nó là một đĩa nhựa ghi những “tiếng ồn trắng” ngẫu nhiên. Khi thêm những “tiếng ồn trắng” này vào giọng nói củaRoosevelt sẽ khiến giọng nói rít lên như tiếng huýt gió, không thể nào hiểu được. Cách duy nhất để xác định Roosevelt nói gì là đem so sánh những tiếng ồn với một đĩa nhựa và “loại bỏ” những đoạn giống nhau. Sau khi gõ đúng con số mà chìa khóa yêu cầu, đoạn băng gốc bị phá hủy, những bản sao trên đĩa LP sẽ được những người đưa tin đáng tin cậy chuyển đến các nơi đặt máy SIGSALY. Điều tối quan trọng là những chiếc máy đọc đĩa của SIGSALY phải chạy ở cùng một tốc độ với sự chính xác tuyệt đối. Khi một chiếc bị sai lệch nhẹ, lập tức tiếng động đưa ra sẽ bị thay bằng tiếng ồn.

Alan Turing đã bẻ gãy được bộ mã “Enigm” của người Đức, giúp phe Đồng minh có thể nghe trộm được những mật lệnh của người Đức. Mục đích của SIGSALY là đảm bảo không cho người Đức làm được điều ngược lại với phe Đồng minh. Một phần công việc của Shannon là chứng minh rằng hệ thống này thực sự bất khả xâm phạm đối với bất cứ người nào không có chìa khóa giải mã trong tay. Nếu thiếu cơ sở đảm bảo về mặt toán học này, những tướng lĩnh quân Đồng minh sẽ không thể nào liên lạc với nhau một cách thoải mái. SIGSALY đã lần đầu tiên áp dụng một vài ý tưởng của Shannon vào thực tế, trong số đó có những ý tưởng liên quan đến phương pháp điều biến mã xung (pulse code modulation – một phương pháp được sử dụng để biến đổi tín hiệu tương tự ở lối vào thành tín hiệu số tương ứng, không bị nhiễu). AT&T đã cấp bằng sáng chế và thương mại hóa nhiều ý tưởng của Shannon trong thời kỳ hậu chiến.

Sau đó, Shannon nói rằng việc nghiên cứu về cách thức che giấu thông tin bằng tiếng động ngẫu nhiên đã thúc đẩy tiến trình xây dựng lý thuyết thông tin. Ông nói: “Một hệ thống bảo mật cũng gần như giống hệt một hệ thống liên lạc bằng tiếng động”.  Hai hướng nghiên cứu này “có mối liên hệ mật thiết đến nỗi bạn không thể tách rời chúng được”.

Năm 1943, Alan Turing đến thăm phòng nghiên cứu của Bell Labs ở New York. Hằng ngày, Turing và Shannon đều có những cuộc trò chuyện trong quán cà phê ở nơi làm việc. Shannon thông báo với Turing rằng ông đang theo đuổi cách thức để đo lường được thông tin. Ông sử dụng một đơn vị đo lường gọi là “bit” và nói đây là ý tưởng đặt tên của John Tukey, một nhà toán học khác ở Bell Labs. “Bit” là chữ viết tắt của “binary digit” – “số nhị phân”. Theo Shannon định nghĩa, bit là một tổng lượng thông tin cần thiết để phân biệt giữa hai kết quả cho ra ngang nhau.

Turning nói với Shannon rằng anh vừa nảy ra ý tưởng về một đơn vị gọi là “ban”, là tổng lượng dữ liệu làm tăng khả năng chính xác của một dự đoán lên gấp 10 lần. Nhà mật mã học người Anh lấy ý tưởng này một phần từ việc giải mã hệ thống mật mã Enigma của người Đức. “Ban” xuất phát từ “Banbury”, tên thị trấn đã sản xuất ra những tờ giấy mà đội mật mã sử dụng.

Chính “bit” chứ không phải “ban” đã làm thay đổi thế giới, chính xác là từ năm 1948. Sau chiến tranh Shannon vẫn tiếp tục làm việc cho Bell Labs. Một hôm ông đặt một bản kế hoạch kì lạ lên bàn làm việc của đồng nghiệp và hỏi đây là cái gì. William Shockley – tên của nhà nghiên cứu trả lời:

“Nó là một chiếc máy tăng âm chỉ dùng bán dẫn”.

Đó là thiết bị bán dẫn (transistor) đầu tiên trên thế giới. Shockley nói với Shannon chiếc máy khuếch đại này  có thể làm bất kì điều gì mà đèn điện tử chân không có thể làm được.

Nó rất nhỏ. Shannon nhận thấy thiết bị mới này hoạt động bằng cách cho các chất khác nhau tiếp xúc với nhau. Nó có thể nhỏ như mong muốn, miễn là trong phạm vi các chất còn tiếp xúc được.

Bóng bán dẫn là một công cụ rất hữu ích có thể biến nhiểu ứng dụng trong lý thuyết của Shannon vào thực tế. Sự việc tình cờ này xảy ra vào cuối năm 1948 hoặc đầu năm 1948, trước khi Bell Labs công bố phát minh về bóng bán dẫn vào ngày 30/6 và chỉ cách thời điểm lý thuyết thông tin kinh điển của Shannon xuất hiện.

Có một vụ xì căng đan nhỏ liên quan đến những tài liệu này. Shannon cho đăng bài báo “Một lý thuyết toán về truyền thông” trên tạp chí Bell System Technical năm 1948. Khi đó ông 32 tuổi. Phần lớn công việc liên quan đến lý thuyết thông tin đều được hoàn thành từ nhiều năm trước đó, trong khoảng từ 1939 đến 1943. Shannon chỉ kể cho một số người về công việc mà ông đang tiến hành. Theo thói quen, ông làm việc một mình trong văn phòng lúc nào cũng đóng kín cửa. Khi biết công trình này, những người ở Bell Labs lấy làm ngạc nhiên vì Shannon đã đạt được một kết quả quan trọng như vậy và họ muốn tham gia vào. Điều đó chẳng khác gì một phát minh khoa học, và họ thúc giục Shannon công bố lý thuyết này. Shannon nhớ lại quá trình hoàn thành công trình như một cơn ác mộng. Ông khẳng định việc mình xây dựng lý thuyết này nằm ngoài sự tò mò thuần túy, đó là khát vọng vươn tới những công nghệ tiên tiến hay hoàn thiện sự nghiệp của mình.

Năm 1948 cũng đánh dấu một bước ngoặt trong đời sống riêng của Shannon. Shannon thường lui tới văn phòng của John Pierce để trò chuyện. Pierce đang nghiên cứu về ra-đa và được biết đến như một người hâm mộ cuồng nhiệt thể loại tiểu thuyết khoa học. Trong những lần tới văn phòng này, Shannon đã gặp trợ lý củaPierce , Mary Elizabeth Moore. “Betty” Moore đang tham gia trực máy tính cho nhóm toán học, các thao tác được thực hiện trên một cái máy tính bàn kiểu cũ. Moore rất hoạt bát, có khả năng làm mọi thứ theo cách “Rosie-the-Riveter”, có thể dùng máy  khoan, máy tiện trong xưởng máy của phòng thí nghiệm. Cô có sức hấp dẫn và là một trong ba phụ nữ duy nhất làm việc ở đây. (“Một người đã có chồng còn người kia đã 50 tuổi”, Betty nhớ như vậy.) Cô và Claude hẹn hò lần đầu tiên vào tháng 12 năm 1948. Ngày 27-3-1949 họ cưới nhau.

Shannon bắt đầu dạy ở MIT vào học kỳ mùa xuân năm 1956. Lúc đầu công việc này chỉ mang tính tạm thời, và có ít nhất một người bạn ở Bell Labs (John Riordan) hiểu rằng có một lý do không được nói ra đằng sau việc này. Người ta đoán Shannon chuyển sang dạy học ở MIT là để có nhiều thời gian rảnh hơn cho việc viết sách về lý thuyết thông tin.

Trong một bức thư gửi cho Hendrik Bode – sếp của mình tại Bell Labs, Shannon viết: “Tôi đang có một khoảng thời gian hết sức vui vẻ tại MIT. Đề tài đang tiến triển rất tốt nhưng khối lượng công việc rất lớn. Ban đầu tôi chỉ mong có một nhóm nhỏ khoảng 8 đến 10 sinh viên giỏi, thế nhưng ngay ngày đầu đã có tới 40 người đến đăng ký, trong đó có cả các giảng viên của MIT, Harvard…”

Chỉ sau vài tháng ở MIT, Shannon gửi Bode đơn xin thôi làm việc tại Bell Labs và chuyển sang giảng dạy tại MIT. Ông thấy mình và Betty thích cuộc sống trí thức và nền văn hóa của Cambridge. “Những khách nước ngoài thường dành cả ngày ở Bell Labs nhưng lại dành đến 6 tháng ở MIT. Điều này đem lại những cơ hội để trao đổi kinh nghiệm và ý tưởng thực tế. Khi tính đến tất cả những thuận lợi và khó khăn, tôi thấy Bell Labs và môi trường chuyên môn cao ở đây đều quan trọng như nhau, nhưng 15 năm tại Bell Labs khiến tôi cảm thấy mình trở nên hơi nhàm chán và làm việc kém hiệu quả hơn. Tôi nghĩ một sự thay đổi về môi trường nghiên cứu và những đồng nghiệp mới sẽ kích thích tôi làm việc tốt hơn.” Shannon giải thích với Bode.

Shannon tiếp cận MIT với đề nghị về một công việc ổn định và lâu dài tại đây. Tiền bạc không phải là vấn đề. Bell Labs đã đề  nghị một mức lương vô cùng hấp dẫn nhưng Shannon từ chối (ông vẫn tiếp tục cộng tác với Bell Labs cho tới tận năm 1972). Mức lương khởi điểm của ông tại MIT là 17.000 đô la  một năm.

Shannon chỉ ưa thích sự khích lệ tại MIT ở một chừng mực nào đó. Thường thì ông làm việc hiệu quả nhất khi chỉ có một mình. Có lẽ ông đã đánh giá thấp mức độ phiền toái mà danh tiếng của một “huyền thoại sống” như ông đem đến khi ở ngôi trường rộng lớn giữa thành phố này. Shannon  “bắt đầu ít xuất hiện, cứ như là ông ta biến mất vậy.” Robert Fano nhớ lại.

Shannon nhận một vài sinh viên có bằng Tiến sĩ. Họ thường phải gặp ông ở nhà riêng để nghe ông chỉ bảo. Một sinh viên tên là William Sutherland nhớ mình đã hơn một lần đến nhà Shannon trong khi ông đang luyện kèn ô-boa. Betty kể: “Ông ấy ngủ bất kì khi nào thấy buồn ngủ và thường ngồi hàng giờ ở cái bàn ăn và suy nghĩ.”

Shannon không còn cho xuất bản những công trình nghiên cứu của mình nữa. Cuốn sách mà ông nói đến không bao giờ được hoàn thành. Những tài liệu của ông ở Thư viện Quốc hội không có gì ngoài mấy bản viết tay liên quan đến kế hoạch này.

Người đi tiên phong trong lĩnh vực trí thông minh nhân tạo Marvin Minsky cho rằng Shannon ngừng những công việc liên quan đến lĩnh vực thông tin vì ông cảm thấy những gì cần phải chứng minh ông đều đã hoàn thành. Sự độc lập trong công việc là điều mà không ai hơn được Shannon. Ý Fano muốn nói đây là một hiện tượng không bình thường. Trong một vài trường hợp ngoại lệ hiếm hoi, khi một nhà khoa học về lý thuyết  thông tin đề cập đến một lỗi sai nào đó với Shannon, (a) – ông đã biết về lỗi đó và (b) – ông đã sửa lại, nhưng không có ý định thông báo điều đó.

“Tôi chỉ theo đuổi và phát triển những sở thích khác nhau của bản thân. Vì cuộc sống luôn luôn thay đổi nên bạn cũng sẽ phải thay đổi hướng đi của mình.” Shannon nói về sự phóng túng trong cách làm việc của mình.

Một trong những sở thích này là trí thông minh nhân tạo. Shannon tổ chức cuộc hội thảo chuyên môn đầu tiên về đề tài này ở Dartmouth, năm 1956. Tiếng tăm của Shannon là một yếu tố khiến người ta quan tâm nhiều đến đề tài này.  Một vài thiết bị mà Shannon tạo nên, bao gồm chiếc máy tính biết chơi cờ đầu tiên và một chiếc máy thông minh khác, là những viên gạch đầu tiên trong lịch sử của học máy (machine learning). Shannon là một người ủng hộ ăn nói lưu loát, có tầm nhìn đủ để biết điều tuyệt vời nào có thể thành hiện thực đồng thời cũng thực tế để hiểu rằng chúng sẽ không xuất hiện trong cuộc đời ông. Ông có tài thiên bẩm trong việc né tránh những câu hỏi vụng về vẫn thường thấy:

Hỏi: Ông có nghĩ những con rô bốt sẽ đủ thông minh để làm bạn với con người không?

Trả lời: Tôi nghĩ là được. Nhưng tương lai ấy vẫn còn khá xa.

Hỏi: Ông có thể tưởng tượng ra một rô bốt làm Tổng thống Mỹ sẽ như thế nào không?

Trả lời: Có, tôi có thể tưởng tượng được. Còn bây giờ tôi nghĩ anh không nên nói về Hoa Kỳ nữa. Đó là một vấn đề hoàn toàn khác.

Rất nhiều thư từ, tài liệu, điện thoại từ những nhà khoa học nổi tiếng thế giới đổ về văn phòng của Shannon.  Họ muốn Shannon duyệt hộ một bài báo hay viết cho họ một bài, muốn mời ông đến nói chuyện, bày tỏ quan điểm hay cho một lời khuyên. Shannon từ chối hết những lời đề nghị, yêu cầu này. Khi tên tuổi  của Shannon được công chúng biết đến một cách rộng rãi, ông bắt đầu nhận được những lá thư từ các trường học đang xây dựng dự án khoa học cho bọn trẻ và những kẻ lập dị theo đuổi nhiều ý tưởng hoang đường về khoa học, máy tính và công ty điện thoại (“Thưa ngài” – dòng mở đầu một lá thư, “Con rô bốt Bel của ngài, một biểu tượng (Daniel 14) trong Kinh thánh, là một cỗ máy quái vật,… Ông đang chế tạo nên một kẻ phản bội, giúp sức cho Tổng thống Hoa Kỳ và FBI bằng cách để cho con rô bốt lừa gạt mình. Tôi sợ là mình sẽ phải kiện công ty Điện thoại New York  và tôi sẽ làm nếu như ông không thức tỉnh”).

Thỉnh thoảng CIA và các cơ quan khác vẫn tìm đến Shannon mỗi khi gặp phải khó khăn trong việc giải những bức mật mã. Shannon đã lịch sự nhắc họ rằng mình đã nghỉ hưu. Trong một bức thư của Philip H.McCallum – một nhân viên CIA viết năm 1983 có đoạn: “Chúng tôi hoàn toàn không lựa chọn ngài một cách ngẫu nhiên. Chúng tôi cần một bộ óc siêu phàm với những ý tưởng độc đáo và chúng tôi phải chấp nhận rằng ngài luôn luôn là người đầu tiên mà chúng tôi nghĩ tới… Mặc dù chúng tôi hiểu rằng ngài không quan tấm tới tiền bạc, nhưng chúng tôi vẫn sẽ trả tiền thù lao cho ngài.”

Shannon không thích trả lời một lá thư đến khi nào ông soạn được một câu trả lời hoàn hảo. Để làm được điều này phải mất một thời gian, vì thế ông sắp xếp những lá thư thành từng ngăn một, trên đó có dán những cái nhãn như “Thư chưa trả lời quá lâu”. Những lá thư này hiện được lưu giữ cẩn thận cùng các tài liệu khác của Shannon ở Thư viện Quốc hội, rất nhiều lá vẫn còn đợi được hồi âm.

Khi về nghỉ hưu sớm và không chính thức, Shannon mới 40 tuổi. Sau Shannon là một nhân viên khác của MIT, Bartleby, người có câu trả lời rất cá tính “Tôi không thích làm việc này nữa” – nghĩa là chức danh thư ký phòng quản lý những lá thư không ai nhận.

Trích từ “Công thức của vận may”, NXB Trẻ, Hoàng Trung, Hồng Vân dịch

(Còn nữa)

Kỳ tới: Người mách nước cá ngựa trong bộ com-lê xấu xí

Tuấn Phong

Apache Mahout

For the past two years, I have been working with this amazing bunch of people whilst, being paid by Google in their summer of code program in a project called Mahout. And like the name says, it is trying to tame the young beast known as Hadoop. I have received a lot from the community. Being part of the project, I have got some real exposure to Java, data mining, machine learning and hands on experience over distributed systems like Hadoop, Hbase, Pig.  The project is still in its infancy, but, its ambitions are high in the sky. I am happy to announce the second release of the project, and proud to be a part of it. I hope people will adapt it in their projects and that it becomes the defacto standard machine learning library the way lucene and hadoop has become in their respective focus areas.

If you are already excited and want to take it for a ride, read Grant’s article on IBM developerworks here
The release announcement below

Machine Learning, which nowadays also merged with Pattern Recognition, has many successful applications. Nevertheless, the two questions above are always difficult to explain to others. My probability of giving good answers to these two fundamental questions to laymen are very low.

Fortunately, I discover Prof. Nello Cristianini’s talk on the topics. Here they are:

The Analysis of Patterns

Who needs Patterns?

These two are also very interesting.

Are We There Yet?

Nello Cristianini asking Gregory Chaitin about “Pattern”

Have Fun!

Someone created a list of machine learning and related books which are very useful:
http://www.amazon.com/Pattern-Analysis-Foundations-and-Mathematics/lm/R2L8XSDG2DT9MR/ref=cm_lm_byauthor_title_full

As part of my ongoing agent training, I re-read the agent environment algorithm in Chapter 2 of AIMA (Russell and Norvig, 1995). The key challenge is how to convert an algorithm with repeat and for loops into viable Prolog code. At first inspection, the implementation is best suited for a language with control loops built into the programming language (e.g., LISP). My primary Prolog books [Bratko 1990] and [Sterling and Shapiro 1994] did not provide the required help. However, upon rereading Prolog Programming In Depth (Covington, Nute, and Vellino 1997), I was able to figure out how to write loops.

Hence, I wrote a draft of the prolog code in paper. Next, I prepared my test Linux environment and using a text editor put the code into files. Next, I had to write a test file to test the individual functions. After much debugging, revising, and rewriting, I had a functional running environment for an agent framework.

Enclosed is my code (only displayed for educational, noncommercial purposes):

/*
 * blockenv.pl
 */

run_environment(State, Updatefn, Agents, Termination) :-
   get_percepts(Agents, State, Percepts),
   get_actions(Percepts, Actions),
   call(Updatefn, Actions, State, NewState),
   !,
   (call(Termination, NewState) -> true ;
   run_environment(NewState, Updatefn, Agents, Termination)).

get_percepts([], State, []).

get_percepts([Agent|Agents], State, Percepts) :-
    get_percepts(Agents, State, Percepts1),
    append([percept(Agent, State)], Percepts1, Percepts).

get_actions([], []).

get_actions([Percept|Percepts], Actions) :-
    get_actions(Percepts, Actions1),
    arg(1, Percept, Agent),
    arg(2, Percept, AgentPercept),
    call(Agent, AgentPercept, Action),
    append([action(Agent, Action)], Actions1, Actions).

/*
 * blktest.pl
 */

:- consult('../environment/blockenv.pl').

test_percepts(Percepts) :-
   get_percepts([planner], [handempty, clear(a), ontable(a), counter(1)], Percepts).

test_actions(Actions) :-
   test_percepts(Percepts),
   get_actions(Percepts, Actions).

planner(_, noaction).

update_counter([], []).

update_counter([H|Rest], NewState) :-
   update_counter(Rest, NewState1),
   ( counter(X) = H -> Y is X - 1 ,
     append([counter(Y)], NewState1, NewState)
   ;
     append([H], NewState1, NewState)
   ).

update_state(_, State, NewState) :-
   update_counter(State, NewState).

termination(State) :-
   member(counter(0), State).

test_environment :-
   run_environment([handempty, counter(2)], update_state, [planner],
           termination).

Enclosed is a test trace.

[trace] 5 ?- test_environment.
 T Call: (7) test_environment
   Call: (7) test_environment ? creep
   Call: (8) run_environment([handempty, counter(2)], update_state, [planner], termination) ? creep
   Call: (9) get_percepts([planner], [handempty, counter(2)], _L214) ? creep
   Call: (10) get_percepts([], [handempty, counter(2)], _L238) ? creep
   Exit: (10) get_percepts([], [handempty, counter(2)], []) ? creep
   Call: (10) lists:append([percept(planner, [handempty, counter(2)])], [], _L214) ? creep
   Exit: (10) lists:append([percept(planner, [handempty, counter(2)])], [], [percept(planner, [handempty, counter(2)])]) ? creep
   Exit: (9) get_percepts([planner], [handempty, counter(2)], [percept(planner, [handempty, counter(2)])]) ? creep
   Call: (9) get_actions([percept(planner, [handempty, counter(2)])], _L215) ? creep
   Call: (10) get_actions([], _L237) ? creep
   Exit: (10) get_actions([], []) ? creep
   Call: (10) arg(1, percept(planner, [handempty, counter(2)]), _L238) ? creep
   Exit: (10) arg(1, percept(planner, [handempty, counter(2)]), planner) ? creep
   Call: (10) arg(2, percept(planner, [handempty, counter(2)]), _L239) ? creep
   Exit: (10) arg(2, percept(planner, [handempty, counter(2)]), [handempty, counter(2)]) ? creep
   Call: (10) planner([handempty, counter(2)], _L240) ? creep
   Exit: (10) planner([handempty, counter(2)], noaction) ? creep
   Call: (10) lists:append([action(planner, noaction)], [], _L215) ? creep
   Exit: (10) lists:append([action(planner, noaction)], [], [action(planner, noaction)]) ? creep
   Exit: (9) get_actions([percept(planner, [handempty, counter(2)])], [action(planner, noaction)]) ? creep
   Call: (9) update_state([action(planner, noaction)], [handempty, counter(2)], _L216) ? creep
   Call: (10) update_counter([handempty, counter(2)], _L216) ? creep
   Call: (11) update_counter([counter(2)], _L255) ? creep
   Call: (12) update_counter([], _L278) ? creep
   Exit: (12) update_counter([], []) ? creep
   Call: (12) counter(_G536)=counter(2) ? creep
   Exit: (12) counter(2)=counter(2) ? creep
^  Call: (12) _L280 is 2-1 ? creep
^  Exit: (12) 1 is 2-1 ? creep
   Call: (12) lists:append([counter(1)], [], _L255) ? creep
   Exit: (12) lists:append([counter(1)], [], [counter(1)]) ? creep
   Exit: (11) update_counter([counter(2)], [counter(1)]) ? creep
   Call: (11) counter(_G549)=handempty ? creep
   Fail: (11) counter(_G549)=handempty ? creep
   Call: (11) lists:append([handempty], [counter(1)], _L216) ? creep
   Exit: (11) lists:append([handempty], [counter(1)], [handempty, counter(1)]) ? creep
   Exit: (10) update_counter([handempty, counter(2)], [handempty, counter(1)]) ? creep
   Exit: (9) update_state([action(planner, noaction)], [handempty, counter(2)], [handempty, counter(1)]) ? creep
   Call: (9) termination([handempty, counter(1)]) ? creep
   Call: (10) lists:member(counter(0), [handempty, counter(1)]) ? creep
   Fail: (10) lists:member(counter(0), [handempty, counter(1)]) ? creep
   Fail: (9) termination([handempty, counter(1)]) ? creep
   Call: (9) run_environment([handempty, counter(1)], update_state, [planner], termination) ? creep
   Call: (10) get_percepts([planner], [handempty, counter(1)], _L237) ? creep
   Call: (11) get_percepts([], [handempty, counter(1)], _L261) ? creep
   Exit: (11) get_percepts([], [handempty, counter(1)], []) ? creep
   Call: (11) lists:append([percept(planner, [handempty, counter(1)])], [], _L237) ? creep
   Exit: (11) lists:append([percept(planner, [handempty, counter(1)])], [], [percept(planner, [handempty, counter(1)])]) ? creep
   Exit: (10) get_percepts([planner], [handempty, counter(1)], [percept(planner, [handempty, counter(1)])]) ? creep
   Call: (10) get_actions([percept(planner, [handempty, counter(1)])], _L238) ? creep
   Call: (11) get_actions([], _L260) ? creep
   Exit: (11) get_actions([], []) ? creep
   Call: (11) arg(1, percept(planner, [handempty, counter(1)]), _L261) ? creep
   Exit: (11) arg(1, percept(planner, [handempty, counter(1)]), planner) ? creep
   Call: (11) arg(2, percept(planner, [handempty, counter(1)]), _L262) ? creep
   Exit: (11) arg(2, percept(planner, [handempty, counter(1)]), [handempty, counter(1)]) ? creep
   Call: (11) planner([handempty, counter(1)], _L263) ? creep
   Exit: (11) planner([handempty, counter(1)], noaction) ? creep
   Call: (11) lists:append([action(planner, noaction)], [], _L238) ? creep
   Exit: (11) lists:append([action(planner, noaction)], [], [action(planner, noaction)]) ? creep
   Exit: (10) get_actions([percept(planner, [handempty, counter(1)])], [action(planner, noaction)]) ? creep
   Call: (10) update_state([action(planner, noaction)], [handempty, counter(1)], _L239) ? creep
   Call: (11) update_counter([handempty, counter(1)], _L239) ? creep
   Call: (12) update_counter([counter(1)], _L278) ? creep
   Call: (13) update_counter([], _L301) ? creep
   Exit: (13) update_counter([], []) ? creep
   Call: (13) counter(_G573)=counter(1) ? creep
   Exit: (13) counter(1)=counter(1) ? creep
^  Call: (13) _L303 is 1-1 ? creep
^  Exit: (13) 0 is 1-1 ? creep
   Call: (13) lists:append([counter(0)], [], _L278) ? creep
   Exit: (13) lists:append([counter(0)], [], [counter(0)]) ? creep
   Exit: (12) update_counter([counter(1)], [counter(0)]) ? creep
   Call: (12) counter(_G586)=handempty ? creep
   Fail: (12) counter(_G586)=handempty ? creep
   Call: (12) lists:append([handempty], [counter(0)], _L239) ? creep
   Exit: (12) lists:append([handempty], [counter(0)], [handempty, counter(0)]) ? creep
   Exit: (11) update_counter([handempty, counter(1)], [handempty, counter(0)]) ? creep
   Exit: (10) update_state([action(planner, noaction)], [handempty, counter(1)], [handempty, counter(0)]) ? creep
   Call: (10) termination([handempty, counter(0)]) ? creep
   Call: (11) lists:member(counter(0), [handempty, counter(0)]) ? creep
   Exit: (11) lists:member(counter(0), [handempty, counter(0)]) ? creep
   Exit: (10) termination([handempty, counter(0)]) ? creep
   Call: (10) true ? creep
   Exit: (10) true ? creep
   Exit: (9) run_environment([handempty, counter(1)], update_state, [planner], termination) ? creep
   Exit: (8) run_environment([handempty, counter(2)], update_state, [planner], termination) ? creep
 T Exit: (7) test_environment
   Exit: (7) test_environment ? creep
true.

Searching for new music can be a daunting task. Services like Apple’s iTunes or the website last.fm make it relatively easy to find music similar to songs or artists you already listen to. But type in “instrumentals for yoga class” and you probably won’t get very far.

Luke Barrington, a PhD student at the University of California San Diego, plans to change that with the beta version of his music search engine, Herd It, which he launched last week. His goal is to find and recommend music based on natural-language searches, providing users with both familiar and new songs that share acoustic qualities.

When he recently pitted his recommendation software against Apple’s music recommendation system, Genius, he found that users were equally satisfied with playlists suggested by both methods.

Genius is what Barrington labels a “metadata-based” system. The software associates music with information not necessarily related to the audio content, such as the name of the song, artist and album, and then averages statistics about how many users purchase and play each track. In other words, Genius is just telling you, “the people that listen to this song also listen to that song,” and suggests you do the same.

Barrington’s approach is more similar to the Pandora Radio model, or what he calls a content-based system, which builds playlists with songs that have similar sounds. Humans assign semantic descriptors to each of Pandora’s more than one million songs based on genre, emotion, instruments or vocals. For example, typing “Kings of Leon” into the search box produces a playlist featuring “electric rock instrumentation, a subtle use of vocal harmony, mild rhythmic syncopation, major key tonality and electric rhythm guitars.”

Herd It builds upon both Apple’s crowdsourcing mentality and Pandora’s natural-language song characterization, but incorporates machine learning to go beyond the capabilities of either system. Barrington created an algorithm to identify acoustic patterns in a song that predict a semantic tag. It then finds similar patterns in other songs and applies the tag automatically, eliminating the need to tag every song separately.

To this end, Barrington created a game on Facebook that allows users to ascribe qualities to thirty-second long clips of popular songs in a variety of genres. There are nearly 150 tags related to instrumentation, vocals, style, emotion, and even where you would prefer to listen to a song (relaxing at the beach, dancing at a party, while driving, etc.). When enough people independently agree on the same tag for a song, the algorithm learns to assign that tag to songs with similar acoustic patterns.

“Your definition of why a song is cool or why it goes well with another song may be quite different from mine,” Barrington said. “We’re hoping that the demographic information that we get from Facebook will help us to use Herd It data to learn demographic-specific representations of tags, like ‘teenage girls from San Diego think that this song rocks’ or ‘middleaged housewives from Europe would find this tune romantic’.” As the algorithm evolves, Herd It could one day provide personalized recommendations.

The machine gets smarter as more people play the game and as more music is available. Herd It’s current database is relatively small (about 10,000 songs) but Barrington hopes to partner with a major license holder for access to a more comprehensive collection. Since the algorithm is trained to read only acoustic qualities, it can be compatible with any music service. So while existing music recommendation engines build playlists around a specific song or artist, Herd It, in theory, should be able to create a playlist based on the query, “jazz trumpet melodies for a romantic dinner.”

There is even an option on Herd It for artists to upload their own music, which automatically receives relevant tags and is just as likely to appear on a playlist as any popular song with the same tags – an attribute not available through Genius, which requires that any new song they wish to add to their collection must first be listened to by a large number of users.

Barrington’s ultimate music recommendation engine would incorporate aspects of Genius’ collaborative filtering that leverage huge amounts of user ratings and information that is hard to extract from just listening to the audio (e.g., popularity, release data, artist similarity, etc.), into Herd It so that any new song can immediately be added and recommended in the same context as older or more popular tunes. “Our system doesn’t know anything that the average music fan is aware of,” said Barrington. “Once we add that information in, we think we can build something that is really smarter than Genius.”

Play the Herd It game on Facebook at http://apps.facebook.com/herd-it/ or request to try out the Herd It music discovery engine here.

I’ve spent the better part of the last few hours reinstalling my Ubuntu virtual machine from scratch after completely botching my previous install’s configuration. How? I was attempting to get Python 2.5 up and running with a few custom packages, and ended up accidentally removing everything Python-related, which included many rather important system packages. Synaptic then froze up, and most of the basic system operations stopped working.

Freaking awesome. Just how I wanted to spend my Saturday evening.

But now that I have it working again, I wanted to delve into my latest project: a semi-intelligent Twitterbot! There are three core components to this project:

1. Read the public Twitter timeline to accumulate posts.
2. Build a Markov Model out of all accumulated posts.
3. Use a cronjob to modulate the frequencies of #1 and #2.

I’ve posted about Hidden Markov Models before, and this is an example of theory put into practice. Granted, the utility of this application is questionable, but if for no other reason, it sure is entertaining. In fact, since activating my Twitterbot a little over a week ago, it’s already garnered a decent response. Here are some of my favorites thus far:

It’s endlessly amusing to me that so many people seem to think my bot is actually a person. At least a few also seem to be amused by its antics. Still others respond as though nothing is amiss. It’s also managed to flag down multinational Twitter users. And it’s even attracted the attention of other bots!

How does it work, ya say? Welllllll…

The underlying assumption of HMMs is that there is a hidden state that influences whatever the output we actively observe is. Within this context, it means we’re assuming there’s an unobservable pattern to the sentences Twitter users post that results in the actual words we can see. Thus, if we observe enough of these posts, we should, in theory, be able to infer those hidden states.

Yeah yeah, that wasn’t very simply put. Nevertheless, let’s move on.

The assumption my bot makes is pretty straightforward: each word that is observed depends only on the word before it. Put another way, this means that, given a single word, there is only a certain number of words that can come after it. Of these finite number of words that can come after it, some are much more likely than others.

This makes intuitive sense. Take any one of the sentences in this post, for example: after you read one word, you’re already expecting a certain word or number of words that could follow it (it’s how we read, in fact; ever heard that humans only actively read about 70% of the words on a  page? all the others are inferred by this same method). It’s basically a primitive form of contextual analysis.

From a technical standpoint, this dependence on only a single previous word is called a “first-order” Markov Model. HMMs can go as high as you’d like. There is another similar Twitterbot built by a friend of mine which uses a “second-order” Markov Model, in that each word depends on the two previous words, resulting in a sentence that probably makes more sense than mine will. But for those of you who ahead of me, this also means much more of the original posts used to build the HMM will show up in the generated posts.

And honestly, I wanted my bot’s posts to be as random as possible while still kind of making sense Hence I purposefully implemented a first-order model.

My bot accumulates 800 posts from the public feed over 20 minutes, then uses those posts to build a first-order HMM. From that model, it then constructs a post by sampling from the model, and posts it to the Twitter account.

If you’re interested in following the bot, you can find it here.

I’m in the process of refining the current model, perhaps a hybrid first-second order HMM. I may also include some topics that are weighted more heavily than others, so the generated posts more accurately reflect the trending topics. And of course, I’m open to suggestions!

Yes, this bot provides a wonderful source of amusement, particularly since I am way over my head in schoolwork. Applying for jobs, applying to PhD programs, conducting research with Dr Murphy, and actually keeping up with my coursework is all proving very difficult to juggle these last few weeks of the semester. So it’s nice to have a new joke to read every 20 minutes!

I also want to mention that, a few days ago, my total hits on this blog surpassed 20,000. Thank you again to all those who seem to find something on this blog interesting

One of the readers of this blog (skauf) suggested looking into KRLS (Kernel Recursive Least Squares), which is an “online” algorithm for multivariate regression closely related to SVM and Gaussian Processes approaches.    What struck me as clever in the KRLS algorithm is that it incorporates an online sparsification approach.  Why this is important, I will attempt to explain briefly in the next section.

The sparsification approach is similar in effect to PCA in that it reduces dimension and determines the most influential aspects of your system.  I had been thinking about a combined PCA / basis decomposition approach for some time, and it struck me that KRLS might be the way to do it.

Kernal-based Learning Algorithms
KRLS and SVM algorithms use a common approach to classification (or regression).   In the simplest case, let us assume that we want to find some function f(x) that will classify n-dimensional vectors as belonging to one of 2 sets (apples or oranges), which we will distinguish by + and – values from f(x):

Assuming the vectors X are linearly separable, we could try to find a hyperplane on the n-dimensional space that would separate the vectors such that there is a balance of distance between the two categories (and one that is maximal subject to some constraints).   The distance between the hyperplane (or in 2 dimensions a line) and the points is determined by a margin function.

Optimization is accomplished by maximizing the margins subject to constraints.  The optimization solves for the weights α  such that the sum of the weighted dot product of  X with each Xi yields the predicted value.  The form of f(x) is the following:

Now the above graph shows a very well-behaved data set, with a clear boundary for classification.  Many data sets, however, will have significant noise in the data and/or outliers that refuse to be categorized correctly.   This data has an overall impact on the regression line.

Supposing we have N samples in our training set {{X1,Y1}, {X2,Y2}, … {Xn, Yn}}.   In simple terms, sparsification is the process of removing (or discounting) samples that are already represented in the existing set, reducing the bias of the regressor.   One approach to determining the degree of representation is to observe the “degree of orthogonality” represented by a given vector with respect to the existing set.   Sparsification also points to an approach to evolving the regressor over time relative to new observations.

The Kernel
Above gave a brief overview of how SVM uses the notion of margin to classify data.  A kernel is not necessary for data that is linearly separable.   The kernel is “simply” a function that maps data from the “attribute” space to “feature” space.   We design or choose the kernel function so that our data is largely linearly separable in the “feature” space and with the constraint that the covariance matrix of all possible vectors mapped from attribute to feature space will be positive semi-definite.

Since our linear SVM equations are expressed in terms of inner products, given a feature mapping function Φ(x) mapping X from non-linear space to “linearly separable space”, we can express the kernel as a function of the inner product of two vectors, later to plug in to our linear equations:

The trick is to choose a kernel function that maximizes dispersion of the data into sets that are linearly separable.

How does this relate to an explicit probability based regression?
It turns out that the process of optimizing the margin function on a Gaussian kernel  is equivalent to finding the unnormalized maximum likelihood, whereas the Gaussian Process approach makes this explicit.

Which model does one choose?
The Gaussian Process approach is strictly bayesian.   It has the upside of providing explicit probabilities / confidence measures on predictions.  If one knows the likelihoods of the desired labels with respect to the attribute vectors, the model works very well and provides more information than the SVM family.

The SVM family of approaches use a margin function to determine the similarity between vectors (for the purpose of classification).   This does not explicitly involve probabilities, but for the Gaussian kernel can be shown to be equivalent to the Gaussian Process approach.   The SVM family has the upside that one does not need to know the likelihoods.

Finally, both models allow the use of kernels to map data from a non-normal or non-linear space to a linear or gaussian space.   Some have shown that there is a degree of equivalence between the likelihood function in the Gaussian Process algorithm and the kernel function used in SVM.

V. Vapnik has created a new field of statistical learning theory to guarantees the accuracy of Support Vector Machines (SVMs ) on future data (a generalization ability), i.e. he gave proofs of an error bound of SVMs. His proofs are in fact generalizations of classical laws of large number. Note that the classical laws of large number can be employed to predict generalization abilities of a finite set of predictors from their statistics on training data, but not for a continuum set of predictors. A situation like this is common for “learning the best predictor” tasks of machine learning and pattern recognition. The new laws of large number for some common continuum sets allow us to solve this prediction tasks and hence advance both machine learning and statistics to another level.

Classification and regression are prime examples of this learning problem. I want to note that, for a regression problem, which is one important and common problem in human history, is indeed advanced by this result as, since the time of Euler, Laplace and Gauss who invented least square algorithms, we have had no guarantee on a generalization ability of any regression predictors at all.

Consequently, a strong theoretical link between machine learning and statistics has created under the name of statistical learning theory. Recent publications on this topic can be found via the Journal of Machine Learning Research (JMLR) and the Computational Learning Theory (COLT) conference (see note 3).

NOTE

1. Ones interested in a bound of SVMs can look at R. Schapire’s excellent class notes: http://www.cs.princeton.edu/courses/archive/spring08/cos511/schedule.html

2. There are other prominent researchers who also make some connections between machine learning and statistics. For examples, J. Friedman and L. Breiman invented, among others, a regression tree algorithm named CART, and recently ensemble methods such as bagging and boosting. T. M. Cover, P. E. Hart and L. Devroye established a generalization ability of classifiers such as nearest neighbors, and L. Valiant introduces a PAC learning framework.

3. Actually, in the current situation, computer scientists seem to be active in learning theory more than statisticians, and hence they merge the PAC framework with statistical learning theory. We then have the COLT community as a result.

4. Useful Links

JMLR: http://jmlr.csail.mit.edu/
COLT: http://www.learningtheory.org/

In my opinion, a good way to describe today machine learning is to say that it is a “modern  statistics“. Although this is not an accurate definition, it can give a good first impression to ones outside the field. In the first few articles, I try to explain the historical connection between the two research fields.

According to the contents in Tom Mitchel’s Machine Learning book, machine learning (ML) originally belongs to computer science, and its emphasis on symbolic learning/processing make ML different from statistics. Examples of popular symbolic learners are decision tree, version space, and inductive logic programming.

Perceptron, the ancestor of the famous neural networks, is also invented by ML researchers. Although, perceptron is designed to attack one classical problem of statistics, namely, (linear) classification. The original goal of perceptron does not relate statistical ideas. The goal of perceptron is: if an input data is linearly separable, perceptron guarantees to find one linear separator; however, perceptron is not interested in “which separator is the best for unseen future data”. It is Vladimir Vapnik and colleagues who later in 1990s present the way to select the best separator, the legendary support vector machines (SVMs), by using a rigorous statistical idea, and connect ML to statistics ever since.

SVM has dramatically impacted the whole field of machine learning. Various ideas of SVM establish their own areas, which are new and big branches in ML. Popular examples which are extremely active in 2000s include “large-margin predictors”, “kernel machines”, “convex-programming learners” and “statistical learning theory”. Because of these newcomers we may say that classical ML research on symbolic learning are gradually died out as in best ML conferences and journals such as JMLR, ICML and NIPS, we can hardly see any papers on symbolic learning at all.

In fact, Bayesian probability and statistics also have their important places in ML originated from Bayesian casual networks and Bayesian neural networks in 1990s. I will postpone this to the next article so that this article will not be too long.

NOTES

1. In this blog, I may use machine learning, pattern recognition, pattern analysis and data mining interchangeably since nowadays it is difficult to make a sharp boundary among them. Note that I am not an expert so my explanation here can be inaccurate in some degrees.

2. Note that, unfortunately, there is no exact definition of statistics as statisticians are divided into many groups depending on what they believe about the philosophy of induction. For examples, there are Bayesian statistics, frequency statistics, decision-theoretic statistics and measure-theoretic statistics. This will be an interesting topic to discuss later, but as most of the people do not realize about this severe divisions of statistics anyway, so I will just say “statistics” here.

Main web page : http://mallet.cs.umass.edu/

http://mallet.cs.umass.edu/sequences.php

I found this (…while looking for a semi-gift-resonant way to make some money) and had to add my $0.02.

http://baris.typepad.com/venture_capitalist/2009/10/the-internet-was-designed-for-the-pc.html

My Comment:

“I live’n’breathe in the “why” and the “design/purpose” and the technology space – I love it here.

My work and writing speaks to essential and magical pragmatism (Yes, pragmatics can be magical, ask Williams James) of the “why” of the internet” as a function of an innate and unstoppable, although seemingly detour prone, process of individual, innate and gift-based human evolution… and whole human-systems gift-based or gift-centric social evolution, and more specifically, the evolution of 1) an individual and naturally disposed and gifted human mind, and 2) all of these human minds doing that same thing.

… and a “why the internet exists” design/purpose founded on the innate NEED of all these evolving and inherently gift-driven-at-the-core human minds finding fuller and better and more synergy-rich “connecting” along the way.

My innately disposed preference is to think of the internet as simply a mind-mirror and interactive functional projection and manifestation thereof, the human mind. And over time, guess what, it actually looks more and more like a human cortex, this felt-work of connective tissue, of fibrils, like glial cells and dendridic networks, enmeshed and encapsulating directly, and indirectly and over time, all minds on the planet. And these minds “connected” – consciously, most not – are trying really, really hard – as it is what they are designed to do… is their ultimate and pre-ordained purpose – to be all they can be… to be innately and naturally gift-driven and along the way, trying very, very hard to more effectively resonate (socnet) and connect (socnet) to optimizing “partners” along the way, and over generations of our humanity.

It is this fundamental process and direction of human evolving that feeds the evolution of da’Net itSelf, and reciprocally, the further evolution of the natural and innately disposed and gifted human Self.

Nice that the internet helps in all this, and was mirror-designed in fact, for that express purpose, whether we see it as such or not, i.e. it’s purpose within and without and pertaining to the unknown.

Okay, done spewing.

Thanks
Brian”

http://videolectures.net/mlss09uk_cambridge/

Approximation Algorithms for the Large-Scale Application of Physiognomy

A plate of facial-character templates from Johann Caspar Lavater 1826 treatise 'Physiognomy.'

Abstract

Over the past 15 years, advances in computer vision have enabled impressive progress in computer-based security technology and vastly improved human security. Computers can now identify weapons in images and video, track individuals through a series of scenes in order to predict their potentially nefarious aims, and collate individuals in live video against suspects in a massive database. Though there is no doubt that we are safer as a result of these advances, they fall short of providing true security. While identifying individuals in video is helpful, one still requires a database of suspects, which can be inaccurate or out of date. Wouldn’t it be simpler and more effective to dispense with the database entirely and inspect the character of the individual by the form of their face and skull? Thanks to recent advances in machine learning and computer vision technologies, this is now achievable. This paper presents GNOMON, a system for the application of the ancient science of physiognomy to live video feeds.

Some time ago began using bayesian networks (decision trees with conditional relationships) to boost signal for strategies, the idea being that a combination of related observations can be combined conditionally to provide a posterior conclusion with a higher degree of confidence.

In the context of trading our bayesian network tells us:

1. given the coincident set of events should we trade and if so, in what direction
2. with what degree of confidence

My approach up until now had been to carefully construct the relationships.  It requires painstaking research and a lot of trial and error.   Why not use an algorithm to assemble / find relationships, and classify data.

General Approaches
The more general decision tree algorithms allow me to present many factors (even with high dimension), determine which factors have the highest degree of information relative to our classification targets (buy, sell, don’t-trade), and formulate a classification or regression that models this.

Key to the assembly on the tree are measures taken from information theory.  We want to make arrangements in the tree such that the “entropy” of the tree is minimized or in other words information is maximized.   This is often calculated with a discrete form of the Kullback-Leibler divergence metric.

Algorithm
In particular the “Random Forest” approach is quite appealing.   Multiple decision trees are constructed against training subsets drawn randomly from a larger training set.   These ultimately produce many variations on the “true” model.   The approximation to the true model is made by observing the mode (similar to a robust mean / expectation) across the random trees.

Taking advantage of the classification ability of the algorithm is going to allow me to try many new inputs in my strategies without the huge research overhead and without the near-intractable multivariate optimization required in other approaches.   I’ll post some results on this soon.

In this month, I continued my studies in Relational Reinforcement Learning by reviewing the article Towards Informed Reinforcement Learning from the proceedings of the 2004 Machine Learning workshop of Relational Reinforcement Learning.   Basically the articles summarizes that an agent with limited information can find an optimal policy and can achieve a goal or goal states with limited information about its environment.  The experiments reported seems to suggest this type of exploration is possible.  According to Google Scholar search, there are 11 subsequent articles that reference this one.  In the RRL arena, my goal is to repeat the block’s world experiment as reported in Relational Reinforcement Learning article by Dzeroski, De Raedt, and Blockeel.

Also, I have been focused on Robert Winkler’s book An Introduction to Bayesian Analysis and Inference.  From the Winkler book, my goal is to gain a better understanding of Bayesian Inference to better understand the Decision Theoretic models of machine learning.  In the machine learning world, Bayesian approaches to various problems are resulting interesting solutions to various problems in Multiagent Reinforcement Learning.

There are some interesting articles in the Journal of Bayesian Analysis and in JAIR.

In another topic of interest, I am continuing my reintroduction of LISP by reading the Patrick Winston and Berthold Horn classic.

From a statistical point of view, this month achieve a new record number of hits with 806 hits, boosting my total for the current year to over 3200 hits.  The hotest pages are my About page, followed by Wumpus World and Wumpus World Revisited.

Thank you for your interest and support.

keynote by Tom Mitchell about populating semantic web using machine Learning

Points made:

manually enter by users
convert from Database
computer reading of unstructured data:

         use an initial ontology
         leverage redudancy available on web
        try to target the ontology to populate
        semi-supervised learning : couples semi-supervised learning (bootstrap learning) for category and relation    extraction.
        use seed examples from Freebase or dbpedia.

here comes o my mind can we populate existing LSCOM ontology from youtube data ?  this can be good candidate for the next paper.

I’ve identified a promising stochastic search heuristic called hyperclimbing for optimizing over discrete product spaces (e.g. the space of binary strings of some fixed length) with noisy objective functions. Hyperclimbing works by recursively sifting through large numbers of partitions of the search space and by identifying ones with variegated expected objective values. Because hyperclimbing is sensitive, not to the local features of a search space, but to certain more global statistics, it is not susceptible to the kinds of issues that waylay local search heuristics. The only visible barrier to the wide and enthusiastic use of hyperclimbing is that it seems to scale poorly with the size of the search space; when one heeds the seemingly high cost of applying hyperclimbing to large search spaces, this heuristic looses its shine. A key conclusion of my doctoral work is that this seemingly high cost is illusory. I have uncovered evidence that strongly suggests that genetic algorithms can implement hyperclimbing extraordinarily efficiently.

As readers of this blog probably know, genetic algorithms are search algorithms that mimic natural evolution. These algorithms have been used in a wide range of engineering and scientific fields to quickly procure useful solutions to poorly understood (i.e. black-box) optimization problems. Unfortunately, despite the routine use of genetic algorithms for over three decades, their adaptive capacity has not been adequately accounted for. Given the evidence that genetic algorithms can implement efficient hyperclimbing, I’ve proposed a new explanation for the adaptive capacity of these algorithms. This new account—the generative fixation hypothesis—promises to spark significant advances in the fields of genetic algorithmics and discrete optimization.

The discovery that hyperclimbing is efficiently implementable also promises to have a non-negligible impact on the ecology of machine learning research. Optimization and machine learning are, after all, intimately related. The practice of machine learning research, can broadly be characterized as the effective reduction of difficult learning problems to optimization problems for which efficient algorithms exist. In other words, the machine learning problems that can effectively be tackled are, in large part, those that can effectively be reduced to optimization problems that can be tackled efficiently. Currently, this largely limits the class of tractable machine learning problems to the class of learning problems that can effectively be reduced to convex optimization problems [1].  The identification of general-purpose non-convex optimization heuristics with efficient implementations (e.g. hyperclimbing), thus, has the potential to greatly extend the reach of machine learning.

For a description of hyperclimbing, and evidence that genetic algorithms can implement this heuristic efficiently, please see my dissertation

[1]  Kristin P. Bennett and Emilio Parrado-Hernandez. The interplay of optimization and machine  learning research. Journal of Machine Learning Research, 7:1265–1281, 2006.

Well again parts of my notes (modified suitably to be blog posts) for a discussion session!

This post would be the first in a series of four posts. The objective of each post would be as follows:

1. This post would introduce Learning Theory, the bias-variance trade-off and sum up the need of learning theory.

2. This would discuss two simple lemmas : The Union Bound and the Hoeffding inequality and then use them to get to some very deep results in learning theory. It would also introduce and discuss Empirical Risk Minimization.

3. Continuing from the previous discussion this post would derive results on uniform convergence, tie the discussions into a theorem. From this theorem we would have made formal the bias-variance trade-off discussed in the first post.

4. Will talk about VC Dimension and the VC bound.

Basically all the results are derived using two very simple lemmas, hence the name of these posts.

______

Introduction:

Learning theory helps give a researcher applying machine learning algorithms  some rules of the thumb that tell how to best apply the algorithms that he/she has learnt.

Dr Andrew Ng likens knowing machine learning algorithms to a carpenter acquiring a set of tools. However the difference between a good carpenter and not so good one is the skill in using those tools. In choosing which one to use and how. In the same way Learning Theory gives a “machine-learnist” some crude intuitions about how a ML algorithm would work and helps in applying them better.

A lot of people still think of learning theory as a method for getting papers published (I’d like to use that method, I need papers , as it is considered abstruse by many and not of much practical value. A good refutation of this tendency can be seen here on John Langford’s fantastic web-log.

______

As put in a popular tutorial by Olivier Bousquet, the process of inductive learning can be summarized as:

1. Observe a phenomenon.

2. Construct a model of that phenomenon.

3. Make predictions using this model.

Dr Bousquet puts it very tersely that the above process can actually said to be the aim of ALL natural sciences. Machine learning aims to automate the process and learning theory tries to formalize it. I think the above gives a reasonable idea about what learning theory deals with.

Learning theory formalizes terms like generalization, over-fitting and under-fitting. This series of posts (read notes) aims to introduce these terms and then jump to a recap of some important error bounds in learning theory.

______

Training Error, Generalization Error and The Bias-Variance Tradeoff:

For simplicity let’s take something as simple as linear regression. And since I want this piece to be accessible, I assume no knowledge of linear regression either.

Linear Regression essentially models the relationship between one variable and another variable such that the model itself depends linearly on the unknown parameters to be estimated from the data. Let’s have a look at what this means:

Suppose you have a habit of collecting weird datasets and you end up collecting up a dataset that gives the circumference of biceps of many men and the distance a javelin is thrown by each of them. And you want to predict for an unknown individual, given the circumference of his biceps how far can he throw the javelin.

Ofcourse there would be a number of reasons that would affect the distance a javelin would go, such as skill (which is essentially non-quantitative?), height, the kid of footwear worn, run-up distance, state of health etc. These would be the some of the many features that would affect that end result (distance a javelin is thrown). What I essentially mean is that the circumference of the biceps isn’t a realistic feature to predict how far a javelin can be thrown. But let’s assume that there is only one feature and it can make reasonable predictions. This over-simplification is only made so that the process can be visualized in a graph.

Suppose you collect about 80 such examples (which you call the training examples) and plot your data as such:

Now the problem given to you is: Given you have the bicep-circumference measurement of an unknown individual, predict how far he can throw the javelin.

How would one do it?

What we would do is to fit in some curve in the above training set (the above plot). And when we have to make a prediction we simply plug in that value in our curve and find the corresponding value for the distance. Something illustrated below.

The curve can be represented in a number of ways. However, if the curve was to be represented linearly (that’s why it’s called linear regression) it could be written as :

Where is the hypothesis, and  are unknown parameters which are to be learnt from the data and is the input feature. It is noteworthy that this is like the slope intercept form of the line.

In the above, for simplicity I considered only one feature, there could be many more. In the more general case:

The ’s are called the parameters (to be learnt from the data) that will decide the nature of the curve.

We see that the equation involves features of the training examples (’s), therefore using this, the task of the learning algorithm will be to decide the most optimum values of using the training set. This can be easily done by something like Gradient Descent.

For any new example, we’d have the features and parameters would already be known by running gradient descent using the training set. We simply have to plug in the value of in equation to get a prediction.

To sum up : Like I mentioned, we use the training set to fit in a optimal curve and then try to predict unseen inputs by simply plugging in its values to the “equation of the curve”.

Now, it goes without saying that we could fit in a “simple” model to the training set or a more “complex” model. A simple model would be linear say something like:

and a complex model could be something like this:

.

It’s to be noted that in the above the same feature is used in different ways, the second model uses to create more features such as , and so on. Clearly the second representation is more complex than the first as it will exploit more patterns in the data (it has more parameters).

However this increase in complexity can lead to problems, in the same way if the model is too simple it can lead to problems. This is illustrated below:

[Fig 1 (Left) and Fig 2 (Right)]

The figure on the left has a “simple model” fit into the training set. Clearly there are patterns in the data that the model would never take into account, no matter how big the training set goes. Paraphrasing this in more concrete terms, it’s clear that the relationship between and is not linear. So if we try to fit in a linear model to it, not matter how much we train it, there would always be some patterns in the data that the model would fail to subsume.

What this means is, what is learnt from the training set will not be generalised well to unknown examples (this is because, it might be that the unknown example comes from that part of the distribution that the model fails to account for and thus the prediction for it would be very inaccurate).

The figure on the right has a “complex” model fit into the same set, clearly the model fits the data very well. But again it is not a good predictor as it does not represent the general nature of the spread of the data but rather takes into account the idiosyncrasies of the same. This model would make very good predictions on the data from the training set itself, but it would not generalize well to unknown examples.

A more appropriate fit would be something like this :

Now we can move to a definition of the generalization error, The generalization error of a hypothesis is its expected error on examples that are not from the training set. For an example on understanding generalization refer to the part labeled “Van-Gogh Chagall and Pigeons” in this post.

The models shown in figures 1 and 2 have HIGH generalization errors. However each suffer from entirely different problems.

______

Bias : Like already mentioned : In the model shown in fig. 1, no matter how much the model is trained, There would always be some patterns in the data that the model would fail to capture. This is because the model has a high BIAS. Bias of a model is the expected generalization error even if we were to fit in a very large training-set.

Thus the linear model shown in figure 1 suffers from high bias and will underfit the data.

Variance : Apart from bias there is another component that has a bearing on the generalization error. That is the variance of the model fit into the training set.

This is shown in fig. 2. We see that even though that the model fits in very well in the training set, there is the risk that we are fitting patterns that are idiosyncratic to the training examples and may not represent the general pattern between and .

Since we might be fitting spurious patters and exaggerating minor fluctuations in the data, such a model would still give a high generalization error and will over-fit the data. In such a case we say that the model has a high variance.

The Trade-off : When deciding on a model to fit onto the training set, there is a trade-off between the bias and the variance. If either is high that would mean the generalizing ability of the model would be low (generalization error would be high). In other words, if the model is too simple i.e if it has too few parameters it would have a high bias and if the model is too complex it would have a high variance. While deciding on a model we have to strike a balance between the two.

A very famous example that illustrates this trade-off goes like this:

[Suppose there is an exacting biologist who studies and classifies green trees in detail. He would be the example of an over-trained or over-fit model and would declare if he sees a tree with non-green leaves like above that it is not a tree at all]

[An under-trained or under-fit model would be like the above biologist's lazy brother, who on seeing a cucumber which is green declares that it is a tree]

Both of the above have poor generalization. We wish to select a model that has an appropriate trade-off between the two.

______

So why do we need Learning Theory?

Learning theory is an interesting subject in its own right. It, however also hones our intuitions on how to apply learning algorithms properly  giving us a set of rules of the thumb that guide us on how to apply learning algorithms well.

Learning theory can answer quite a few questions :

1. In the previous section there was a small discussion on bias and variance and the trade-off between the two. The discussion sounds logical, however there is no meaning to it unless it is formalized. Learning theory can formalize the bias variance trade-off. This helps as we can then make a choice on choosing the model with just the right bias and variance.

2. Learning Theory leads to model selection methods by which we can choose automatically what model would be appropriate for a certain training set.

3. In Machine Learning, models are fit on the training set. So what we essentially get is the training error. But what we really care about is the generalization ability of the model or the ability to give good predictions on unseen data.

Learning Theory relates the training error on the training set and the generalization error and it would tell us how doing well on the training set might help us get better generalization.

4. Learning Theory actually proves conditions in which the learning algorithms will actually work well. It proves bounds on the worst case performance of models giving us an idea when the algorithm would work properly and when it won’t.

The next post would answer some of the above questions.

______

Onionesque Reality Home >>

]]>