Software For The Deaf Online

by Benedict Perez (hosted by Bruce Oliver, http://multiplestreamincomeapproach.com )

Software for the deaf is proliferating in the market either online or offline. Yes there are now many different types of software for the hearing impaired. Manufacturers of these kinds of software saw the need for people hounded by this physical handicap. Thanks to their noble purpose of having to manufacturer software that does not only make lives of the hearing impaired easier but also aides those in learning anything they wish to learn.

Yes, you can now have the wonders brought about by modern technology if you are one of the many individuals having problem with their sense of hearing. Hearing impaired all over the world can now enjoy and learn things other than the traditional braille. They can now have the modern touch to learn and enjoy things sans their sense of hearing by having the perfect software for the deaf that perfectly addresses their needs.

You just need to be extra cautious in having to choose these kinds of software. This is especially so if you so desire to purchase them online. Software for the deaf is now so easy to find. Many of these now abound the market especially in the internet.

While it would be much fast and easy to look and purchase software for the deaf online, it also tags along a lot of risk. Well not the kind of risk you may have in mind. Having it from just about any of the websites you will bump in while browsing for it can be that risky.

Risky in the sense that you could lose money if you will not be extra careful with it. It is therefore advised that you should make sure to have this kind of software only from online vendors known for good quality products that are also proven efficient. You can easily identify these vendors by carefully scrutinizing their products and observing the kind of customer care it has. These are just a few of the many things you need to look out for in purchasing software for the deaf.

About the Author:

You can find available in the market some software for the deaf. This has been intentionally created for those hearing impaired individuals so that they learn things like normal individuals do. You should also try learn to type software for both young and adults.

I recently wrote about Criticisms of Lakoff’s Theory of Metaphor and how some of these criticisms can be addressed by integrating the work of Lakoff and Genter. Perhaps the most controversial aspect of conceptual metaphor theory (CMT) is the idea of embodied cognition, that metaphor is grounded in the sensory experiences of our bodies. I went looking for experimental evaluations of this idea and found some interesting papers.

The Boston Globe recently had a story on this topic:

Thinking literally: The surprising ways that metaphors shape your world
Drake Bennett

Drawing on philosophy and linguistics, cognitive scientists have begun to see the basic metaphors that we use all the time not just as turns of phrase, but as keys to the structure of thought. By taking these everyday metaphors as literally as possible, psychologists are upending traditional ideas of how we learn, reason, and make sense of the world around us.

Several researchers are mentioned, but no references are given, so I tracked down some of the references:

Experiencing Physical Warmth Promotes Interpersonal Warmth
Lawrence E. Williams and John A. Bargh

… we hypothesized that experiences of physical warmth (or coldness) would increase feelings of interpersonal warmth (or coldness), without the person’s awareness of this influence …

Contemporary cognitive linguists have advanced similar arguments that people conceptualize their internal, mental worlds by analogy to the physical world (9–13). Applied to the question of how warm objects can produce the same affective states as a “warm” person, embodiment theorists have noted how objects and events that produce the same quality of affective response are associated (categorized) together in memory (14).

In summary, experiences of physical temperature per se affect one’s impressions of and prosocial behavior toward other people, without one’s awareness of such influences.

Cold and Lonely: Does Social Exclusion Literally Feel Cold?
Chen-Bo Zhong and Geoffrey J. Leonardelli

Metaphors such as icy stare and cold reception depict social exclusion using cold-related concepts; they are not to be taken literally and certainly do not imply reduced temperature. Two experiments, however, revealed that social exclusion literally feels cold.

The Smell of Virtue: Clean Scents Promote Reciprocity and Charity
Katie Liljenquist, Chen-Bo Zhong, and Adam D. Galinsky

Two experiments demonstrated that clean scents not only motivate clean behavior, but also promote virtuous behavior by increasing the tendency to reciprocate trust and to offer charitable help. Capitalizing on the fact that abstract concepts are often symbolically derived from the concrete environment (Emerson, 1836), our results suggest that olfactory cues can trigger virtuous behaviors that are related to cleanliness at only a symbolic level. The link from cleanliness to virtuous behavior appears to be a nonconscious one: in neither experiment did participants recognize an influence of scent on their behavior, and in Experiment 2, perceived cleanliness did not differ by condition nor correlate with the effects.

Weight as an Embodiment of Importance
Nils B. Jostmann, Daniël Lakens, Thomas W. Schubert

Four studies show that the abstract concept of importance is grounded in bodily experiences of weight. Participants provided judgments of importance while they held either a heavy or a light clipboard. Holding a heavy clipboard increased judgments of monetary value (Study 1), and made participants consider fair decision-making procedures to be more important (Study 2). It also caused more elaborate thinking as indicated by higher consistency between related judgments (Study 3) and by greater polarization between strong versus weak arguments (Study 4). In line with an embodied perspective on cognition, these findings suggest that, similar to how weight makes people invest more physical effort in dealing with concrete objects, weight also makes people invest more cognitive effort in dealing with abstract issues.

Here are the home pages of some people who are doing research in this area:

• John A. Bargh
• Daniel Casasanto
• Lawrence Williams
• Chen-Bo Zhong
• Nils B. Jostmann

Here’s a career that wouldn’t occur to most kids: cryptographer. Greg Laden’s blog features a link to a CIO (that’s a trade magazine) slideshow on the history of computer cryptography.

If you know a middle or high school student interested in this sort of thing, they might also like computational linguistics – and there’s an annual contest they can enter. The contest site also has lots of practice questions, which are a lot of fun to work through for anyone who enjoys codes, puzzles and the like.

In response to my previous post, Criticisms of Lakoff’s Theory of Metaphor, Cosma Shalizi suggested that I should look at some criticisms from Chris and Murphy:

• Explaining the War of the Metaphors, Chris (2007)
• Idioms, Metaphors, and Lakoff, Oh My!, Chris (2006)
• On Metaphoric Representation, Murphy (1996)

Here are my thoughts on these criticisms.

This is what I make of Murphy’s paper: What Murphy calls Structural Similarity (Sections 3.3 and 6) is the right approach to metaphor. Murphy’s idea of Structural Similarity is that (something like) Gentner’s Structure Mapping Theory (SMT) [1] is the underlying mechanism for Lakoff’s Conceptual Metaphor. I support this view [2], as does Gentner [3,4,5]. The other views Murphy talks about, strong and weak metaphoric representation (Sections 3.1 and 3.2), are strawmen.

However, it seems to me that Murphy does not fully understand Gentner’s work. In Section 3.3, he says, “The structural similarity view is not a theory of novel verbal metaphor.” This is wrong. The structural similarity view is exactly a theory of novel verbal metaphor [3]. Murphy doesn’t seem to understand that the point of Gentner’s SMT is to transfer knowledge from one domain to another. Murphy seems to believe that the two domains (source and target) must be completely structured before SMT can be applied, and that SMT does not add any new structure. If this were so, then the mapping process in SMT would be no more than an entertaining exercise, rather than a method to generate new knowledge.

Summary: the bulk of Murphy’s paper is spent on presenting and refuting strawmen. His proposed solution is the right way to go, but he doesn’t fully understand his own proposal.

[1] Structure-mapping: A theoretical framework for analogy, Gentner (1983)
[2] The latent relation mapping engine, Turney (2008)
[3] The career of metaphor, Bowdle and Gentner (2005)
[4] Metaphor is like analogy, Gentner, Bowdle, Wolff, and Boronat (2001)
[5] Metaphor as structure-mapping, Gentner and Bowdle (2008)

In Explaining the War of the Metaphors, Chris writes:

“I and others think that CMT and blending would benefit greatly from research on analogy. If these theories had a testable model of how mappings between conceptual domains are formed, interpreted, and used — a model that structure mapping theory can provide — it would be possible to make some concrete, experimentally testable predictions about things like inference, object recognition, etc.”

I completely agree that CMT (conceptual metaphor theory) would benefit greatly from research on analogy; in particular, the research of Gentner and her colleagues.

In Idioms, Metaphors, and Lakoff, Oh My!, Chris writes:

“Sometimes I forget that not everyone who happens upon this blog today has been reading it from day one (I mean come on, why haven’t you?). It surprises me, then, when people tell me they’ve seen no evidence that George Lakoff and Mark Johnson’s conceptual metaphor theory is, well, wrong.”

I disagree that Lakoff and Johnson’s theory is (wholly) wrong. Some details may be wrong, but I think that many of the core ideas are correct and furthermore consistent with Gentner’s theories and experiments.

It seems to me that Chris is arguing two incompatible claims: (1) Gentner’s theory is the proper foundation for Lakoff’s theories. (2) Lakoff is wrong. (If not incompatible, these claims are at least dissonant.)

In Metaphor as structure-mapping, Gentner and Bowdle write:

“We have suggested that metaphor is like analogy — that the basic processes of analogy are at work in metaphor. Specifically, we suggest that structural alignment, inference projection, progressive abstraction, and re-representation are employed in the processing of metaphor and simile. This view can help resolve some tensions in the field: for example, on this view, metaphor both reflects parallels (Murphy, 1996) and creates new similarities (Lakoff, 1990) between the domain compared, via structural alignment and candidate inferences, respectively.”

I agree with Gentner that her view “can help resolve some tensions in the field”. It puzzles me that Chris simultaneously agrees with Gentner and yet seems to actively work to increase tensions in the field. Instead of attacking Lakoff, why doesn’t Chris work to integrate Lakoff’s and Gentner’s theories, as I attempt to do?

There are quite a few well-known libraries for doing various NLP tasks in Java and Python, such as the Stanford Parser (Java) and the Natural Language Toolkit (Python).  For Ruby, there are a few resources out there, but they are usually derivative or not as mature.  By derivative, I mean they are ports from other languages or extensions using code from another language.  And I’m responsible for two of them!

• Ruby Linguistics – this is one of the more ambitious projects, but is not as mature as NLTK
  • interface for WordNet
  • Link grammar parser
  • some inflection stuff
• Stanford Parser interface – uses a Java bridge to access the Stanford Parser library
• Mark Watson has a part of speech tagger [zip], a text categorizer [zip], and some text extraction utilities [zip], but I haven’t tried to use them yet
• LDA Ruby Gem – Ruby port of David Blei’s lda-c library by yours truly
  • Uses Blei’s c-code for the actual LDA but I include some wrappers to make using it a bit easier
• UEA Stemmer – Ruby port (again by yours truly) of a conservative stemmer based on Jenkins and Smith’s UEA Stemmer
• Stemmer gem – Porter stemmer
• Lingua Stemmer - another stemming library, Porter stemmer
• Ruby WordNet - basically what’s included in Ruby Linguistics
• Raspell – Ruby interface to Aspell spell checker

There are also a number of fledgling or orphaned projects out there purporting to be ports or interfaces for various other libraries like Stanford POS Tagger and Named Entity Recognizer.  Ruby (straight Ruby, not just JRuby) can interface just about any Java library using the Ruby Java Bridge (RJB).  RJB can be a pain, and I could only initialize it once per run (a second attempt never succeeds), so there are some limitations.  But using it, I was able to easily interface with the Stanford POS tagger.

So while there aren’t terribly many libraries for NLP tasks in Ruby, the availability of interfacing with Java directly widens the scope quite a bit.  You can also incorporate a c library using extensions.

Naturally, if I missed anything, no matter how small, please let me know.

I will attempt to blog some of things I attend during the semester.  One of them is a weekly seminar on sentiment analysis taught by Philip Resnik.  I am in it right now. This is therefore a liveblog and hence not guaranteed to make sense or be complete. Especially the latter, far from it.

Tim Hawes’ thesis and conversational analysis

The first thing we’re talking about is Tim Hawes work for his Master’s degree, which he defended just yesterday.  I attended it yesterday before I had even started this blog.  It was about predicting the outcome of Supreme Court (US) cases from transcripts of oral arguments.  This is particularly interesting today, as Philip just mentioned, as Sonia Sotomayor showed up for work today at SCOTUS for the first time.  I proposed by mailing list that one further means of predicting how individual would vote, even if they rarely say anything on the bench (true of some justices) would be their body of writing and argument prior to confirmation.  Philip proposed the use of a mixture model using prior argument, updated as the justice moves through his/her career.

In the case of legal arguments, we have to make some assumptions.  Hawes’ thesis mentioned a couple of textual assumptions: cohesion and coherence.  That is, we assume that there are topical and other elements that evolve through the text in a consistent way.  There are techniques we can use to measure and segment a document based on these kinds of assumptions, such as TextTiling and lexical chaining.

The distinction between cohesion and coherence: the latter is a semantic value that really must be judged by a person—it’s about interpretation.  It is possible to have cohesion without coherence. (We mentioned the word “zeugma”, look it up.)

The right place to look for discourse analysis from an NLP point of view is to start with Rhetorical Structure Theory (RST).  Daniel Marcu has written on this topic.

Hawes’ thesis had two kinds of results.  One of these was encoded in “rose diagrams.”  These are modified pie charts for which each slice varies in radius as though it were a petal of a rose, and each petal is coloured differently in a gradient of shades.  In this representation, we can visualize a large number of things at once.  In the case of SCOTUS, each justice can be represented as a petal, whose colour represents political leaning, radius agreement, and width the number of follow-up turns at questioning.  It’s a bit of a complicated representation and hard to describe without a diagram, which I’m not about to do on the fly.

While this form of visualization is quite complicated by itself, it can be used to make contrasts between types of cases and judicial situations, in which case it produces often very strong and visible contrasts.  Contrasts we can examine are between “liberal” vs. “conservative”, affirm vs. overturn, plaintiff win vs. lose, and so on.   It turns out that by this method, you can predict Clarence Thomas’ vote (who rarely speaks) to a high degree of accuracy.

We didn’t seem to get to the other technique Hawes’ used, but we had to change rooms.

Sentiment analysis

We didn’t end up changing rooms.

This part of the seminar we briefly touch on the basics of sentiment analysis, particular with reference to Bing Liu’s recent review article.  So we begin with a discussion of the general dimension and challenges of sentiment analysis, such as

• What do people think of _____?
• What features/facets/topics matter?
• Mixed and neutral sentiment.
• The effect of comparatives.
• How opinions change, and what influences this.
• Covert opinion/spin vs. overt expressions.
• The holders of opinion
• Multilingual and cross-language issues.

We then had a rather wide-ranging discussion of the different kinds of issues, including a detailed discussion on syntactic relationships within sentences that might relate opinion-holders to sentiments to opinion targets/objects. This discussion was so wide-ranging and yet very compressed that it is hard to represent it in a liveblog, but it covers issues that we will revisit in later sessions.

Lakoff’s theory of metaphor has been both highly praised and highly criticized. My own thinking about how the mind works has been greatly influenced by Lakoff’s books, yet I also agree with much of what his critics say. I would like to make a case here that his books are worth reading, although much of the criticism is correct.

These are the books by Lakoff and various co-authors that I have read:

• Metaphors We Live By, Lakoff and Johnson
• Women, Fire, and Dangerous Things, Lakoff
• Where Mathematics Comes From, Lakoff and Nuñez
• Philosophy in the Flesh, Lakoff and Johnson
• More than Cool Reason, Lakoff and Turner

I enthusiastically recommend all five of these books to anybody who has a strong interest in either cognition or language.

Below are excerpts from four opinions of the work of Lakoff and co-authors. After each excerpt, I give my reaction to the opinion.

I dislike Lakoff’s theory of metaphor, especially when compared to Sperber and Wilson’s. His work is deeply, purely speculative, which is fine, but he seems not to care at all about experimental controls, or even admit they’re an issue, which is not. And his manner of writing is most unpleasant, principally because it’s so grandiose (he’s overthrowing the whole tradition of western thought) and so dogmatic (e.g., he often writes “cognitive science has shown that” when what he means is “as I have often claimed, and many others vehemently denied”). I half suspect, given the subjects Lakoff is writing on, that he’s deliberately positioning himself to be the 21st-century version of Freud, the man who provides a educated non-specialists with a scientific-sounding vocabulary for mental life. But I hasten to add that (a) I really have no evidence for that, and (b) Lakoff is in person polite, affable and well-spoken. Still, it’s very hard for me to force myself all the way through one of his books. — Cosma Rohilla Shalizi, Analogy and Metaphor

It is true that Lakoff does not seem interested in experimental controls, but other researchers have been working to address this problem. See Chapter 6 of Philosophy in the Flesh for many references. It is also true that Lakoff is often grandiose and dogmatic, and he neglects to cite much related work, such as the work of Dedre Gentner and her colleagues. However, his basic claims, stripped of their abrasive presentation, are insightful and fruitful. If you can focus on the ideas and ignore the style of presentation, there is much to learn from Lakoff’s books.

No one has a problem with the idea that the lens of an eye and the lens of a telescope are two instances of the general category “lens,” rather than the telescope being a “metaphor” for the eye. Nor is there anything metaphorical going on when we refer to “the genetic code”: a code by now is an information-theoretic term for a mapping scheme, and it subsumes cryptograms and DNA as special cases. But do cognitive psychologists use the computer as a “metaphor” for the mind, or (as I believe) can it be said that the mind literally engages in computation, and that the human mind and commercial digital computers are two exemplars of the category “computational system”?

So the ubiquity of metaphor in language does not mean that all thought is grounded in bodily experience, nor that all ideas are merely rival frames rather than verifiable propositions. Conceptual metaphors can be learned and used only if they are analyzed into more abstract elements like “cause,” “goal,”, and “change,” which make up the real currency of thought. And the methodical use of metaphor in science shows that metaphor is a way of adapting language to reality, not the other way around, and that it can capture genuine laws in the world, not just project comfortable images onto it.

[...]

Though metaphors are omnipresent in language, many of them are effectively dead in the minds of today’s speakers, and the living ones could never be learned, understood, or used as a reasoning tool unless they were built out of more abstract concepts that capture the similarities and differences between the symbol and the symbolized. For this reason, conceptual metaphors do not render truth and objectivity obsolete, nor do they reduce philosophical, legal, and political discourse to a beauty contest between rival frames.

Still, I think that metaphor really is a key to explaining thought and language. The human mind comes equipped with an ability to penetrate the cladding of sensory appearance and discern the abstract construction underneath — not always on demand, and not infallibly, but often enough and insightfully enough to shape the human condition. Our powers of analogy allow us to apply ancient neural structures to newfound subject matter, to discover hidden laws and systems in nature, and not least, to amplify the expressive power of language itself. — Steven Pinker, The Stuff of Thought

Pinker agrees that metaphor is important, but he believes that abstraction is more important. He claims that abstraction captures the similarities and differences that are the basis of metaphor. However, Pinker neglects to explain how abstraction works. I believe that metaphor (more precisely, analogy-making) is the basis for abstraction. First we observe an analogy, such as the analogy between water waves and sound waves, and then we form an abstraction, wave theory, which includes both types of waves. Later we add light waves. Abstraction does not subsume metaphor, because analogy precedes abstraction and is the basis for abstraction.

Whereas the authors devote 44 pages to Chomsky, they cover all of “Anglo-American analytic philosophy” in 29 pages, while lumping together Frege, Russell, Carnap and the Vienna Circle, Quine, Goodman, Davidson, Putnam, Kripke, Montague, and Lewis. In the same chapter, they continue with ordinary language philosophy (Strawson, Austin, and the later Wittgenstein), which they consider to be based on the same metaphors. Yet these philosophers have expressed widely divergent views on the embodiment of mind, the nature of language, and Chomsky’s theory of autonomous syntax. By drawing finer distinctions, the authors might have claimed some of them as potential allies against Chomsky’s position. — John Sowa, Review of Philosophy in the Flesh

Like Shalizi, Sowa criticizes Lakoff’s presentation. Yes, Lakoff distorts the work of others and exaggerates the novelty of his own work. Yes, his ideas could be presented more clearly, more fairly, and with better discussion of related work. Nonetheless, if you can ignore these problems with presentation, you will find many interesting ideas in the work of Lakoff and his co-authors.

Lakoff and Nuñez have a very interesting general framework for approaching their topic: mathematics arises by the extension of innate human capacities (e.g. subitization) or basic universals of human experience (spatial and motor experience), and the means of extension is cognitive metaphors which preserve the basic inferential structure of the source domain. The first few chapters provide a plausible sounding, perhaps workable account of arithmetic, simple logic and set theory, but one that they should have developed in far more detail (e.g. their account of intersection in terms of container schemas is criminally underexplained).

[...]

In short, they have very interesting ideas, but the mass of technical vagueness and blunders, plus the big strawman that is their “philosophical” argument, suggests that they are more interested in passing off as intellectual revolutionaries among the pop-science book audience than in contributing to our understanding of the topic. — Idiosyncrat, Review of Where Mathematics Comes From

The final sentence of this review serves as a good summary: the ideas are very interesting, but the presentation of the ideas is impaired by a desire to seem revolutionary, at the cost of fairness, balance, rigour, and careful discussion of related work. For me, ultimately, ideas trump presentation. Until there is a better source for these ideas, I recommend Lakoff’s books.

When Lazyfeed announced a limited round of beta invites on TechCrunch, I admit, I lusted after them.  Only 250?  I wanted to be one!  But alas, I was put on the waiting list.  It’s a decent marketing strategy for building up some hype.  When I finally did get my invite, I tried them out for about 5 minutes and fell prey to the distractions of the internet.  That was a bad sign, though.  Usually a new service can hold my attention for a little while longer.  So what happened?

Lazy-wha?

Lazyfeed is a service that lets you enter topics, blogs, twitter, delicious and flickr accounts to form a live streaming lazyfeed.  You then get live updates in the form of your tags being updated.  Your main screen consists of a bunch of boxes with your topics and then things it guesses are related.

The hook

Lazyfeed’s marketing strategy succeeded again by giving me three invites to hand out to friends.  I offered them on Twitter, having only one person bite.  So here are the other two invites for the adventurous.  Get em while they’re hot.  If you manage to take one, please comment that you did so, so that I can at least know who you were and we can save someone else the wasted time.  I’m just throwing them into the ether like this because I don’t feel like pushing them on Twitter again.

NTI1MzMxMjc5ZVhmUTl5cDBiek1R
OTk5MTUwNjczN3JCLklmZHhjMDdV

The rub

Lazyfeed is a lovely service in terms of appearance and ajaxy goodness, but my initial impression is that it ends up being streaming information overload.  For one, the topic suggestion feature appears to be fairly naive.  Someone correct me if I’m wrong, but it looks a bit like document similarity for topics is done purely by one-for-one matching on tags.  Whatever the method, the result of their suggested topics (“Stuff for Lazy Jason”) is stuff like the following:

Lazyfeed sample suggested topics

Granted, it’s a hard problem, but those results are pretty bad.  So as I started to write this post lambasting this service, I considered that maybe I was just seeing cold-start problems, and I was being unfair.  So I trained it with some additional feeds and topics that are straight-to-the-point of stuff I’m interested in, like sigir2009, topicmodeling, recommendersystems, etc.  Tags can contain no spaces, btw, which is why those don’t.  When I tried using dashes, like I often do on delicious, it gives no results.  I also removed some things that were too general or contained too many spurious results.

The light

Things started improving here, and I actually began to understand what the point of Lazyfeed is.  My initial confusion was that “Stuff for Lazy Jason” is stuff that I would want to read right now.  Being lazy, I didn’t expect to have to do work to get those things.  But “Stuff for Lazy Jason” is a list of topics it thinks I might be interested in.  Saving any one of those puts it into my lazyfeed, which is in the bar on the left.

My lazyfeed topics

So now what happens is that occasionally it discovers something new related to my interests and it bumps that category to the top of the list and turns it bold again (grayed out topics have been read).  Most of my topics are low traffic, so add something like mariahcarey if you want to see this functionality in action.  Now we’re getting somewhere.  It has actually started being helpful and has found me some stuff that my Google alerts haven’t.  Which is weird, and is making me think I need to double check to make sure my Google alerts are working…

The end

My takeaway after using Lazyfeed for nigh on two hours is that it’s an interesting alternative (or even extension) to RSS, but one that still hasn’t crossed the bridge to the next stage in evolution.  The idea is solid.  Automatically discover stuff in the sea of human knowledge (or human idiocy) and serve it up fresh.  The implementation lacks robust topic detection which is unfortunately going to be necessary unless it is to become another source of information overload rather than a useful stream of relevant information. Relevance is an ephemeral thing, given that your information needs change from day to day.  Lazyfeed makes it pretty easy to get rid of old topics and add new ones, even if some of their suggestions are still wonky.  It’s an interesting recommender system problem with a lot of potential.

A while back I ported David Blei’s lda-c code for performing Latent Dirichlet Allocation to Ruby.  Basically I just wrapped the C methods in a Ruby class, turned it into a gem, and called it a day.  The result was a bit ugly and unwieldy, like most research code.  A few months later, Todd Fisher came along and discovered a couple bugs and memory leaks in the C code, for which I am very grateful.  I had been toying with the idea of improving the Ruby code, and embarked on a mission to do so.  The result is a hopefully much cleaner gem that can be used right out of the box with little screwing around.

Unfortunately, I did something I’m ashamed of.  Ruby gems are notorious for breaking backwards compatibility, and I have done just that.  The good news is, your code will almost work, assuming you didn’t start diving into the Document and Corpus classes too heavily.  If you did, then you will probably experience a lot of breakage.  The result, I hope is a more sensical implementation, however, so maybe you won’t hate me.  Of course, I could be wrong and my implementation is still crap.  If that’s the case, please let me know what needs to be improved.

To install the gem:

gem sources -a http://gems.github.com
sudo gem install ealdent-lda-ruby

Enjoy!

“Never swallow anything whole. We live perforce by half-truths and get along fairly well as long as we do not mistake them for whole-truths, but when we do so mistake them, they raise the devil with us.” — Alfred North Whitehead, Dialogues of Alfred North Whitehead

There are dozens of theories about meaning, but they share a common element: meaning (semantics) is about mapping. We understand a thing, we give it meaning, by mapping it to another thing. Furthermore, and this is a crucial point, one mapping is not enough. The more mappings we make, the better we understand. A single mapping only gives us part of the truth.

We understand things (words, events, perceptions, people, signs) by relating (connecting, mapping) them to other things. This analogy-making is how we understand both high-level ideas and concepts and low-level perceptions:

“We have repeatedly seen how analogies and mappings give rise to secondary meanings that ride on the backs of primary meanings. We have seen that even primary meanings depend on unspoken mappings, and so in the end, we have seen that all meaning is mapping-mediated, which is to say, all meaning comes from analogies.” [emphasis added] — Douglas Hofstadter, I am a Strange Loop

But one analogy is not enough. No analogy is perfect, and we compensate for their imperfections by using multiple analogies, and by blending analogies together. Marvin Minsky calls this panalogy (parallel analogy):

“If you ‘understand’ something in only one way, then you scarcely understand it at all—because when you get stuck, you’ll have nowhere to go. But if you represent something in several ways, then when you get frustrated enough, you can switch among different points of view, until you find one that works for you!” — Marvin Minsky, The Emotion Machine

If you want to learn more about how we blend analogies, I highly recommend The Way We Think: Conceptual Blending and the Mind’s Hidden Complexities, by Gilles Fauconnier and Mark Turner.

Perhaps the most important lesson from the Netflix Prize has been that many models are better than one:

• “the biggest lesson for ML from the Netflix contest has been the formidable performance edge of ensemble methods” — John Langford, Netflix nearly done
• “We have learned that ensemble methods are the solution for more accuracy.” — Daniel Lemire, After Netflix? What next?

I view this as confirmation of the Panalogy Principle: we don’t understand anything until we understand it many ways. There are no whole-truths, but we can get by reasonably well with a large number of half-truths.

An easy start to coding and thinking about NLP – Natural Language Processing or Computational Linguistics is to try and obtain things like vocabulary, frequencies of words etc from a text. One grand project is a text fingerprinting system that could derive a numerical index to characterise a given writer’s style.

One of the questions I posed about this was… “exactly how much information can you get from a text without ever having to interpret what it is actually talking about ?” I have likened this to trying to learn about a given animal by liquidising it completely and then analysing the liquid obtained.

Actual semantic understanding of a text is a genuine Hard Problem of AI. We can’t go that far yet, but there are still a lot of useful things simple code can do. Here I wanted to find the number of sentences in a text, I was using Dracula by Bram Stoker.

def sentfull(a):
    """ takes a text and returns
     list where sentences have been split
      with full stops"""
    
    b=open(a)
    c=[]
    for x in b.read().split('.'):
        c.append(x)
    return c

def sentques(b):
    """ takes list of strings and returns 
    list where sentences have been split 
    with question marks"""
    for a in range(len(b)):
        if '?' in b[a]:
            c=b[a].split('?')
            b.remove(b[a])
            for x in c:
                b.append(x)
    return b

def sentexcl(b):
    """ takes list of strings and returns 
    list where sentences have been split 
    with exclamation marks"""
    for a in range(len(b)):
        if '!' in b[a]:
            c=b[a].split('!')
            b.remove(b[a])
            for x in c:
                b.append(x)
    return b
            

def sentence(a):
    """ takes list of strings and returns list
     where sentences have been separated"""
    return sentexcl(sentques(sentfull(a)))

print (sentence('dracula.txt'))


A twitter friend (@communicating) tipped me off to the UEA-Lite Stemmer by Marie-Claire Jenkins and Dan J. Smith.  Stemmers are NLP tools that get rid of inflectional and derivational affixes from words.  In English, that usually means getting rid of the plural -s, progressive -ing, and preterite -ed.  Depending on the type of stemmer, that might also mean getting rid of derivational suffixes like -ful and -ness.  Sometimes it’s useful to be able to reduce words like consolation and console to the same root form: consol.  But sometimes that doesn’t make sense.  If you’re searching for video game consoles, you don’t want to find documents about consolation.  In this case, you need a conservative stemmer.

The UEA-Lite Stemmer is a rule-based, conservative stemmer that handles regular words, proper nouns and acronyms.  It was originally written in Perl, but had been ported to Java.  Since I usually code in Ruby these days, I thought it’d be nice to make it available to the Ruby community, so I ported it over last night.

The code is open source under the Apache 2 License and hosted on github.  So please check out the code and let me know what you think.  Heck, you can even fork the project and make some improvements yourself if you want.

One direction I’d like to be able to go is to turn all of the rules into finite state transducers, which can be composed into a single large deterministic finite state transducer.  That would be a lot more efficient (and even fun!), but Ruby lacks a decent FST implementation.

I just discovered this cool blogpost on OpenCog. It is a very interesting blog by someone who has both time and talent for nice experiments (and he seems to have the needed hardware as well…). Although he is not directly in my line of research, which is quantitative sociolinguistics, he touches with my fondness for computational linguistis, statistics and lexical semantics via NLP.

If symbols must be grounded in perception, how does this grounding happen? How do we learn to create mappings between language and perception? For example, how does the word “rabbit” get tied to the perception (visual, tactile, whatever) of a rabbit? AI algorithms for assigning textual labels to photographs are not yet able to approach human performance on this task. The problem is somewhat similar to statistical machine translation, which exploits parallel corpora to learn mappings between two different languages, although the difference between text and photographs is more extreme than the difference between any two written languages. Perhaps ideas from statistical machine translation are applicable to symbol grounding. The translation algorithm of Lepage and Denoual, based on proportional analogy, seems particularly appropriate, since it makes minimal assumptions about the structures of the languages.

There is a view that the meaning of words must be grounded in perception:

• Symbol grounding and meaning: A comparison of high-dimensional and embodied theories of meaning
• Perceptual Symbol Systems

But how do we connect a word with a perception? Quine gives the following example: suppose we hear a person say “gavagai” in the presence of a rabbit. How do we know that “gavagai” means “rabbit”? Other possible interpretations of “gavagai” are “Lo, food”, “Let’s go hunting”, and “There will be a storm tonight”. The possibilities are endless.

It is interesting that Quine links the symbol grounding problem to the problem of translation. In Quine’s thought experiment, a linguist must translate “gavagai” into English. Perhaps recent progress with machine translation algorithms is applicable to the symbol grounding problem?

Lepage and Denoual use proportional analogies to derive translations from parallel corpora. A proportional analogy has the form A:B::C:D, meaning “A is to B as C is to D“. For example, quart:volume::mile:distance means “quart is to volume as mile is to distance”. Consider the following proportional analogy:

A = “This is a rabbit.”

B = “This is a fox.”

C =

D =

The analogy A:B::C:D helps us to map “rabbit” to the image of the rabbit, whereas A and C alone leave the mapping indeterminate. Comparing A and B, we see the shared structure “This is a X.” and we note that X = “rabbit” for A and X = “fox” for B. Likewise, comparing C and D, we see the shared backgrounds and note the differing foregrounds. This helps us map the foreground of C to “rabbit” and the foreground of D to “fox”. Given only A and C, we have no reason to pick out “rabbit” from the sentence “This is a rabbit” in A and we have no reason to pick out the foreground rabbit from the background grass in C.

The core idea here is that we do not ground symbols in perceptions by noting correlations between symbols and perceptions; rather, we note meta correlations between relations between symbols (e.g., the relation between A and B) and relations between perceptions (e.g., the relation between C and D).

The ability to consider differences between differences is important because it lies at the heart of our abilities to solve new problems. This is because these “second-order-differences” are what we use to remind ourselves of other problems we already know how to solve. Sometimes this is called “reasoning by analogy” and is considered to be an exotic or unusual way to solve problems. But in my view, it’s our most ordinary way of doing things.
— Marvin Minsky, The Society of Mind, 1988

Related work: Deb Roy.

It is common to derive vocabulary from the usage at, for instance, Twitter. This only leads to the defintion of an artificial language, not identical to any kind of English. In the beginning, there should be ….

The SemLab semantic search platform is based on our Vicore™ data processing platform which, among others, is the basis of our leading semantic news analysis solution ViewerPro™.

The purpose of semantic search is to improve standard keyword-based search technologies by adding domain dependent, a-priory knowledge about the semantics of the concepts entailed in the documents that are analysed. This a-priory knowledge is contained in ontologies: formal descriptions of the concepts that exist in the domain of application. In general the ontologies play a pivotal role in a semantic enabled technology and have a profound effect on its overall quality. For a search platform, the common quality parameters Precision and Recall will largely depend on the quality of the ontologies used.

Semlab’s semantic platform fully acknowledges that ontologies are the key to a successful semantic analysis system and therefore support ontology maintenance as a core technology. This includes both expert knowledge expression as well as (supervised) automatic ontology learning from domain-specific documents. In addition the SemLab platform fully supports existing base ontologies and offers their own default domain ontologies for many areas of financial information management, exactly as are used every day by the ViewerPro semantic news analysis platform.

The SemLab semantic platform is in essence a data pipelining platform in the sense that the data sources are fed to the start of the line, various relatively independent subsequent operations are done on the data. The results of these are stored in various databases.

In the case of semantic search, the data sources consist of text documents (e.g. reports, news messages, e-mails) in various formats and the final result is an semantic index, modeling the occurrence of ontology terms in the documents. When a user presents the platform with search query, this query is treated as if it were another document and also expressed in terms of the ontology. Since both the document corpus and the user’s query are expressed within the same domain of discourse, they can be compared and ranked on similarity. Finally, using carefully designed threshold values, the resulting document sections are presented to the user in a GUI or available through an API to other applications. In addition to the retrieval of documents, the domain ontology can also be queried directly. In this way the semantic search platform also serves as a question answering tool, enabling users to quickly retrieve the corporate domain knowledge.

MT technology has come a long way since the Russian to English Georgetown experiment back in 1954 which at the time gave rise to an optimistic prediction of having it all sorted out within some 5 or so years. Progress turned out to be a bit slower than that with the complexities of natural language presenting quite a number of stumbling blocks.

Putting aside just for the moment the need for post-editing of MT output by a human translator to achieve publishable standard, would you agree and have you noticed that the results from online translation tools have improved over the years?

As long as your expectations are properly adjusted and while you have to be aware that not all of what you get may be accurate, these tools do offer anyone the option of at least getting the gist of what something written in a foreign language is about, which adds another dimension to the global availability and sharing of information.

I ran a little test and put one of my blog pages through Babelfish (http://babelfish.yahoo.com/), Google Language Tools (http://www.google.com/language_tools), and the Microsoft Live Translator (http://translator.live.com/). Aware that I hadn’t used anything even remotely approximating controlled language in the source text, it wasn’t surprising that the German MT output was anything but perfect in all three cases. However, all three renditions manage to give a reasonable idea as to what the page is about.

There are of course some obvious problems with disambiguation (the word “quotes” for example causing problems and in some instances translated incorrectly as “Anführungszeichen” instead of “Zitate” which would have been the right choice in the given context), and the intricacies of German grammar and syntax in particular result in some awkward and grammatically and syntactically wrong constructions that are difficult to make any sense of - the odd time to the extent that the translated phrase actually says the opposite to what the source states.

A quick test using technical specifications for a printer gave pretty good results, and that is due to standard terminology and simpler linguistic structures being used in this type of material.

MT systems are improving at what appears to be a faster pace now, with hybrid solutions combining rule based and statistical approaches as well as other techniques. The main prerequesites for successful application of MT to produce publishable outputs are

• controlled language used in source material
• preparation and tuning of the MT engine (for the specific domain)
• setting and agreeing quality expectations for the translated output (to control human post-editing effort)
• regularly re-training engine with corrections.

MT therefore looks to be most suited for specialised translations (e.g. technical documentation – user manuals, product specifications). To ensure that quality expectations for the translated output are agreed upfront is key, otherwise little or no benefits will be gained in terms of time, effort and cost compared to human translation. The post-edited final output will have to be accurate and linguistically correct. Understood that this is not synonymous with highest achievable quality standard (which is quite hard to define actually), but how important is stylistic excellence to someone installing a newly acquired piece of software and looking for guidance from the installation manual?

Perhaps you’ve heard of the latest brainchild of the Wunderkind Stephen Wolfram:  Wolfram|Alpha.  Matthew Hurst nicknamed it Alphram today and I agree that’s a much better name.   Wolfram|Alpha (W|A henceforth) is not a search engine, it’s a knowledge engine.  It will compete with Google on a slice of traffic that Google really isn’t all that hot in for now, comparative questioning answering.  When you ask Google something like “How does the GDP of South Africa compare to China?” you hope you get back something relevant in the first few results (spoiler alert:  you don’t).  When you ask that of W|A, you get exactly what you’re looking for.  Beautiful.  W|A’s so-called natural language interface isn’t perfect, though.  You get a lot of flakiness from it until you start to recognize what works and what doesn’t.

Now let’s be honest.  How often do we search for that kind of thing?  Not very often.  I think that’s partly because Google is notoriously bad at it.  Once we start to get a handle on what W|A is capable of, I think people will start expecting more of their friendly neighborhood search giant.  Google claims to have a few tricks up its sleeves, but everything I’ve seen out of Google lately has been such a disappointment I am deeply skeptical.  The new trick is called Google Squared and it returns search results in a spreadsheet format, breaking down the various facets of the things you are searching for.  In the demo, it shows stuff like rollercoaster drop speeds, heights, etc when you search for roller coasters.  You can add to the square and do some pretty nifty stuff.  TechCrunch claims this will kill W|A.  I think the two could be complementary.  Based on the demo, I expect W|A will return results of a higher calibre, but will miss out on a lot of queries because the knowledge is just missing.  Google Squared appears to be doing something fuzzier and will return results that might be really bad.  So while W|A just says it doesn’t know, Google Squared will let you pick through the junk to find the gem.  Google Squared is expected to launch later this month in Google Labs.

Many have said that where W|A will really compete is against Wikipedia and I am inclined to agree.  There are plenty of things I go to Wikipedia for now that I probably will switch over to W|A for, like populations of countries, size of Neptune’s moons, and so on.  Wikipedia still wins for more in-depth knowledge on a topic.  W|A also does some pretty cool stuff when you search for the definition of a word (use a query like “word kitten“).  You learn that kitten comes from Classical Latin, and entered English about 700 years ago.  You can find out a similar thing (and go further in depth for the etymology at least) using the American Heritage dictionary on dictionary.com, but W|A requires less digging.

And this brings me around to a key point with W|A.  It’s an awesome factoid answering service.  It does it well and it does it in a pretty way.  Stuff you can find in more depth elsewhere you can get quickly and easily, but only superficially via W|A.  There are links to more information, though, so you don’t lose much by relying on W|A — assuming it has knowledge about what you’re looking for.  You’re still going to be more likely to hit a brick wall with W|A.

And of course, since Wolfram developed Mathematica, W|A is backed by it.  Enter an equation and you get some really handy math info back.  Need to quickly know the derivative of a fairly complicated equation?  Presto.  Probably the most satisfying feeling I got today was from a query similar to “what is the area under x^4+3x^2+4 from 1 to 8?“  Let’s see you answer that, Google Squared.

This study investigated second language (L2) lexical development in the spontaneous speech of six adult, L2 English learners in a 1-year longitudinal study. One important aspect of lexical development is lexical organization and depth of knowledge. Hypernymic relations, the hierarchical relationships among related words that vary in relation to their semantic specificity (e.g., Golden Retriever vs. dog vs. animal), are an important indicator of both lexical organization and depth of knowledge. Thus, this study used hypernymy values from the WordNet database and a lexical diversity measure to analyze lexical development. Statistical analyses in this study indicated that both hypernymic relations and lexical diversity in L2 learners increase over time. Additionally, lexical diversity and hypernymic values correlated significantly, suggesting that as learners’ lexicons grow, learners have access to a wider range of hypernymy levels. These findings are discussed in relation to developing abstractness in language, extending hypernymic knowledge, and the growth of lexical networks.

from Language Learning

In predicate logic, the concept red ball is represented as a combination of the concepts of red and ball. We can define the predicate RedBall(x) as (Red(x) & Ball(x)). Logical atomism views the world in terms of compound predicates, such as RedBall(x), that are built up from atomic predicates, such as Red(x) and Ball(x). Good old-fashioned AI (GOFAI) research almost always assumes a kind of logical atomism. Cyc, for example, represents knowledge using a form of logical atomism. Even those researchers who reject GOFAI still tend to assume logical atomism. Statistical and connectionist models of concepts typically view red ball as a combination of red and ball. I believe that we should turn this view on its head. That is, red ball comes first (is more basic, more primitive); red and ball come later (are more complex, more refined).

In Latent Semantic Analysis (LSA), we represent the semantics of red and ball with vectors, in which the elements are derived from the frequencies of the terms red and ball in various contexts. For those who are familiar with logic, it is natural to think of representing red ball by some mathematical operation on the vectors for red and ball. For example, we might add the red vector to the ball vector. One problem with this idea is that addition is not sensitive to order; thus house boat and boat house would have the same vector, although they have different meanings. One solution to this problem is a mathematical operation on vectors that is sensitive to order, such as the tensor product or the circular convolution.

Several papers explore this vector combination approach to representing compound predicates:

• Plate (1991), Holographic reduced representations: Convolution algebra for compositional distributed representations
• Plate (1994), Distributed representations and nested compositional structure
• Smolensky (1994), Grammar-based connectionist approaches to language
• Plate (1995), Estimating analogical similarity by vector dot-products of holographic reduced representations
• Wilson, Street, and Halford (1995), Solving proportional analogy problems using tensor product networks with random representations
• Jones and Mewhort (2007), Representing word meaning and order information in a composite holographic lexicon
• Widdows (2008), Semantic vector products: Some initial investigations

Plate (1995) describes his approach as follows:

Suppose we have distributed representations for the concept “circle”, “triangle”, “small”, and “large”. We can represent a small circle by superposing the patterns for “small” and “circle”. However, when we try to represent a small circle and large triangle we have the problem that the superposition of the four patterns is ambiguous – the information that small is associated with triangle and large associated with circle is lost – it could be a large circle and a small triangle. The same problem arises when we try to represent conceptual relations. Suppose we have a predicate representation for “Spot bit Jane”: bite(spot, jane). Spot is the agent of this relation, and Jane is the object (or patient). A distributed representation of this relation must be careful to preserve the information about which person is associated with which role (agent or object) so that there is no confusion with “Jane bit Spot”. I refer to associations between roles and fillers as role/filler bindings.

The assumption here is that we build up the complex structure bite(spot, jane) from simple atomic elements bite, spot, and jane. But I suggest that the atomic element is bite(spot, jane), and that we construct bite, spot, and jane from this atomic element.

Suppose we have a third-order tensor of the form pattern × word × word. The predicate P(x,y) is a pattern (“x bit y“), x is a word (“spot”), y is another word (“jane”), and the triple <P(x,y),x,y> is a cell (<”x bit y“,”spot”,”jane”>) in this third-order tensor. The concept “bite” corresponds to the slice (a matrix cut out of the tensor) <”x bit y“,*,*>, the concept “spot” is the slice <*,”spot”,*>, and the concept “jane” is the slice <*,*,”jane”>.

In this representation, the atomic elements (tensor cells; scalars) are whole events (“I see a red ball”, “Spot bit Jane”), and abstract concepts (“red”, “bite”) are complicated structures (tensor slices; matrices), composed of these atomic elements. This turns the usual picture upside-down. There is nothing “simple” or “atomic” about the concept “circle”. It is a complex structure, composed of all of our atomic experiences of events that contained some aspect of circularity.

Using a vector combination approach, Jones and Mewhort (2007) report a score of 57.81% (see page 18) on multiple-choice synonym questions from the TOEFL (Test of English as a Foreign Language). Using the opposite approach (Turney, 2007), a third-order pattern × word × word tensor achieves a score of 83.75% (see page 22).

Weizenbaum devotes a good deal of space in the book to a chapter called “How Computers Work,” a useful introduction to programming for anyone who comes into his book with no tech background.   Some of the most famous (or notorious) quotes from the book stem from Weizenbaum’s description of programming, and programmers.  Discussing high-level programming languages, he notes that most programmers “have no knowledge whatsoever of their computer’s machine language or of the content and structure of the translators that mediate between them and their computers.”  Weizenbaum notes that, rather than being the perfect transcription of mathematical principles, programming is “like any form of writing, more often than not experimental.  One programs, just as one writes, not because one understands, but in order to come to understand.” 

And he devotes a chapter to the “compulsive programmer,” a type we’re all familiar with now:

Wherever computer centers have become established…bright young men of disheveled appearance, often with sunken glowing eyes, can be seen sitting at computer consoles, their arms tensed and waiting to fire their fingers, already poised to strike, at the buttons and keys on which their attention seems to be as riveted as a gambler’s on the rolling dice…they work until they drop, twenty, thirty hours at a time.  Their food, if they arrange it, is brought to them: coffee, Cokes, sandwiches.  If possible, they sleep on cots near the computer…their rumpled clothes, their unwashed and unshaven faces, and their uncombed hair all testify that they are oblivious to their bodies and to the world in which they move.  They exist, at least when so engaged, only through and for the computers.  These are computer bums, compulsive programmers.  They are an international phenomenon.

The only thing that has changed in the intervening decades has been the relocation of the console from the university computer center to the dark basement or bedroom.  Weizenbaum notes the difference between the compulsive programmer and the “dedicated, hard-working professional”:  The professional has a problem to solve, uses the tool at hand and moves on, “whereas the compulsive programmer sees the problem mainly as an opportunity to interact with the computer.”  Still, there’s no doubt many discoveries and inventions have come about because of some hacker tinkering endlessly with registries and routines, which the workaday programmer would never have discovered or created.  But Weizenbaum’s argument is valid – to wall one’s self off into a “hard-coded” world where, say, everyone has a PhD in math, engineering or computer science, and design problems which should be handed to “creatives” are left instead to microgranular analysis of preference polls of thousands of Pantone shades, is to deny “the whole man,” to declare that problems which can’t be solved with pure science are not interesting or relevant.

Moreover, it’s just as dangerous when a Science that is really more of a Humanity, such as psychology, attempts to become “scientific,” to convert human behavior into codifiable rules (see for instance the creation of new “illnesses” for the DSM-V like “apathy disorder” and “parental alienation syndrome” so that all human problems can be assigned a billing code – and a prescription). 

Weizenbaum is openly disdainful of the AI research of his time, using the unpleasantly Stalinist term “artificial intelligentsia” to describe its proponents.  He lays out the now well-known linguistic problems with AI, i.e. “the house blew it” is a statement which the human mind can decode by using context – “the house” is often used to refer to a casino, and if the conversation is about gambling, the listener may glean the meaning even without having heard the phrase before. 

Weizenbaum’s concern is that even a system like ELIZA, which isn’t the greatest conversationalist, provokes a willing suspension of disbelief in many of its users.  Even in our interpersonal dealings, we bring a “working hypothesis” about the other person, usually based on first impressions – i.e., I see a fat old man getting out of a car with a RUSH IS RIGHT sticker, and I fill out my “conceptual framework” on that person with my prediction of what he’s going to think and say.  These “acts of induction” are automatic and probably instinctive, and they come into play whenever we interact with another, whether that other is man or machine.  Weizenbaum seems off the mark when he says that people conversing with ELIZA “cling to the belief that they are being understood” not because of our lifelong habit of infilling detail on new people, but because they don’t understand the science behind it, and that “it is therefore clothed in the magical mantle of Science and all of Science’s well known powers may be attributed to it.” 

Next post: the conclusion of my review.