(ESPECIALES WWW2009 Conference)

En el vídeo del debate ‘Vivimos tiempos exponenciales‘ se afirma que el mundo global produce cada año 16 Terabytes de nueva información; mas de la que produjo la humanidad en sus anteriores 5.000 años de historia. Y que la información técnica nueva se duplicará en 2010 cada 72 horas. Ricardo Baeza-Yates, Vicepresidente de Yahoo! Research para Europa y Latinamérica no esta tan seguro del significado de estas cifras. Ricardo, afirma en el programa Tecnópolis UPTV de esta semana, que muchas de las afirmaciones que se hacen ahora sobre la información, tecnología e internet, confunden conceptualmente los significados de términos como ‘datos‘, ‘conocimiento‘, ‘información‘ o ‘sabiduría‘ y la relación que hay entre ellos ya que, en su opinión, no se usan correctamente

Es indudable que para vivir satisfactoriamente en un mundo como el nuestro, macerado de datos e información necesitamos nuevos y sofisticados instrumentos y herramientas de búsqueda o gestión para conseguir encontrar, en ese maremagnum de información y datos que nos envuelve -y de forma razonable y sin que se vaya nuestra vida en ello-,  lo que es relevante para nosotros, que una vez encontrado, deberemos procesar y convertir en conocimiento y sabiduría ”útil’ para nosotros. En los laboratorios de YAHOO! RESEARCH en Barcelona, que lidera Ricardo Baeza-Yates, se dedican  a encontrar esas nuevas herrmientas digitales y sociales mediante investigaciones que combinan el campo de la tecnologias digitales y de búsquedas avanzadas (search engines) con la semántica  y el lenguaje.

Para tratar estos temas, Tecnópolis UPTV ofrece esta semana un diálogo en profunidad con Ricardo Baeza-Yates que aporta su reflexión sobre la relación entre la tecnología digital y datos, información, conocimiento y sabiduría. Y tambien sobre conceptos como tagging, desambiguación (técnicas para eliminar ambigüedades de significado en una palabra polisémica), habilidades multiproceso (multitasking), la preservación digital o sobre las tecnologías de busqueda (search engines) masiva que manejan en los laboratorios de YAHOO! RESEARCH en Barcelona, Santiago de Chile y Haifa (Israel), que Ricardol lidera y donde estan trabajando en este campo de vanguardia y en cuya estrucura de invetigación manejan datos de mas de 500 millones de personas. También explican sus investigaciones dos miembros de su equipo del Grupo Multimedia de YAHOO! RESEARCH en Barcelona: el líder del grupo Roelof Van Zwol y el ingeniero de investigación Lluís Garcia Pueyo.

Esta entrega de Tecnópolis UPTV se emite el miércoles 20 de Mayo, de las 13.30 a 14.30 h. y de 21.30 a 22.30 h. y el domingo 24 de Mayo con el mismo horario (horas de España).  La televisión de la Universidad Politécnica de Valencia emite en abierto en el área metropolitana de Valencia, por el canal 45 UHF; en TV por cable en España, por el canal 989 de Ono y, por el satélite Hispasat para Iberoamérica.

Para ver el programa ahora mismo,  clickear sobre el gráfico:

Um excelente sensor para averiguar as tendências futuras da Internet são os departamentos de investigação de Yahoo e Google. Neste post tratamos de Yahoo Research.

A localização dos centros de investigação de Yahoo Research já diz muito da sua estratégia:

• 3 nos EUA (Nova Iorque, Califórnia e Silicon Valley),
• 1 na Europa (Barcelona),
• 1 no Médio Oriente (Israel),
• E destaco: 1 na América Latina (Santiago do Chile) e 1 na Índia (Bangalore).

As áreas geográficas emergentes – Ásia e América do Sul, serão determinantes no desenho da Internet, não só como mercados, mas também como dinamizadores e impulsionadores do seu desenvolvimento.

Se atentamos no programa e nas áreas prioritárias de investigação, depressa concluímos que se trata de uma continuidade, mas é uma continuidade exigente e rigorosa que modificará certamente a “face” da Internet que conhecemos hoje.

A aposta de Yahoo está a fazer-se: na pesquisa, sobretudo de imagens e vídeos, na extracção automatizada de informação das páginas web, na classificação e clustering também aplicada a páginas web. Mas uma área fascinante e que revolucionará muito do que conhecemos é a pesquisa na web y o web mining.

É precisamente nessa vertente que encontramos Ricardo Baeza-Yates. É um chileno, investigador e responsável pelas secções da Yahoo Research em Barcelona e em Santiago do Chile. Os seus centros de interesse estão todos orientados para o estudo da web e para a recuperação de informação na web. Ele e a sua equipa centram-se sobretudo nos algoritmos, na estrutura de dados, na pesquisa de bases de dados e multimédia e na melhor forma de visualizar e representar as redes, um trunfo importante neste tipo de estudos.

Nos últimos anos, dois estudos de fundo foram realizados e publicados sobre web de países – “Características de la web de España” (2005) e “Caracterización del espacio web en Argentina” (2007). É impressionante o número de dados que são manipulados nestes trabalhos: dezenas de milhões de páginas e milhares de domínios. É justo perguntar: mas esses números não são uma gota no oceano? Sabe-se que a Internet é uma rede livre de escalas e é redundante. Se a recolha é razoável (no caso de um país são necessárias dezenas de milhões de páginas, dependendo claro está do estado de desenvolvimento da rede), é perfeitamente possível extrapolar com toda a segurança, porque é como se se tivesse recolhido e analisado a totalidade das páginas existentes.

Na Rede, tal como na natureza, nada é aleatório. Leis muito simples regem a sua construção e crescimento e é sobre isso tudo que este ramo de investigação trabalha. São leis como a centralização, a intermediação, o grau de influência, tudo dimensões muito fáceis de compreender e muito similares ao comportamento das redes sociais de sempre.

O valor e uso da informação obtida é estratégico e tem múltiplas aplicações. E os resultados são deveras curiosos, seja pela surpresa seja pela reprodução da realidade na Web.

Por exemplo, é possível saber se um país está a apostar na criação de uma rede com tecnologia aberta ou não através da linguagem, dos servidores, etc., usados na criação das suas páginas. Sabe-se com que países esse país em particular tem relações mais fortes, porque os links o reflectem. Consegue-se medir o contributo de entidades públicas e privadas na construção da rede nacional, etc.

Vale a pena ler os estudos e conhecer um pouco a análise de redes.

Aquest és el tema d’una nova conferència de l’Any de la Computació que ens espera a la Sala de Graus de l’ETSE el dimecres dia 2 d’Abril a les 12:30h.

Conferència a càrrec de Ricardo Baeza-Yates, investigador i director de Yahoo! Research a Barcelona i Santiago de Chile, a més de professor a la Universitat de Chile i professor investigador de l’ICREA a la UPF.

+info (cartell de la conferència)

There’s a remarkable feature on Edge.org.  I point it out because it’s robust dialogue about collective behavior.  In particular, the discussion is decidedly not casual agentish, mind-body dualist nor monocasual.  Dr. Couzin is refreshing!  His approach ties very well to analysis of media (collective behavior!)  (Read his other stuff like this essay, too!)

how can a colony decide between two food sources, one of which is slightly closer than the other? Do they have to measure this? Do they have to perform these computations?

We now know that this is not the case. Chris Langton and other researchers have also investigated these properties, whereby individuals just by virtue of the fact that one food source is closer, even if they are searching more or less at random, have a higher probability of returning to the nest more quickly. Which means they lay more chemical trail, which the other ants tend to follow. You have this competition between these sources. You have an interaction between positive feedback, which is the amplification of information—that’s the trail-laying behavior—and then you have negative feedback because of course if you just have positive feedback, there is no regulation, there is no homeostasis, you can’t create these accurate decisions.

There’s a negative feedback, which in this case is the decay of the pheromone, or the limited number of ants within the colony that you can recruit, and this delicate balance of positive and negative feedback allows the colony to collectively decide which source is closest and exploit that source, even though none of these individuals themselves have that knowledge.

Great exposition on the web of contingencies and the feedback loops capable of reinforcing complex behavior that we typically claim is “free choice” like or conscious decision making.

One of the big challenges that still remains, and one that we’re beginning to address—I’m not saying it’s a new question; I’m not saying people haven’t addressed it—is the level at which selection is acting within populations. The view of individual level selection, and selection at the level of genes, of course, holds. But if you consider, say, a school of pelagic fish—these large schools that make these dramatic maneuvers—the individuals are unrelated to each other. They drift around as pelagic larvae, so when their schools are comprised as adults, they’re completely unrelated.And yet the individuals’ functioning is entirely within the context of these schools; you can see the integration of the behavior when they are attacked by predators, you can see why in the ’40s people thought there must be thought transference, must be telekinesis, because of these remarkable maneuvers. We now know that these maneuvers are created by the relatively local interactions among the individuals. But if you take an individual, say, a herring, from the school and isolate it, it will die of stress.It is a bit like taking cells from your body—when you take them outside the body, they are unable to function. Of course it is not as closely integrated as a body, it is not as closely integrated as an ant colony. But there is this high level of integration among unrelated individuals. And in terms of how the genes are going to propagate, genes that allow individuals to function collectively as a group are going to be extremely important. So one has to then begin to think about the level at which selection is actually functionally acting. There is nothing new in terms of the genetics here, but it just in terms of how you begin to understand how the collective behaviors emerge and evolve within these types of systems.

Yes! Selection by consequences. Selection happens at many levels – genetically, epigenetically, and behaviorally.  This is a clearer and more accurate description of collective behavior than some previous discussion on edge.org.

It doesn’t stop there though.  Dr. Couzin reminds us yet again observed behavior emerges from simple, often unexpected, contingencies.

Another example that we’ve been investigating arehuge swarms of Mormon crickets. If you look at these swarms, all of the individuals are marching in the same direction, and it looks like cooperative behavior. Perhaps they have come to a collective decision to move from one place to another. We investigated this collective decision, and what really makes this system work in the case of the Mormon cricket is cannibalism.

You think of these as vegetarian insects—they’re crop pests—but each individual tries to eat the other individuals when they run short of protein or salt, and they’re very deprived of these in the natural environment. As soon as they become short of these essential nutrients, they start trying to bite the other individuals, and they have evolved to have really big aggressive jaws and armor plating over themselves, but the one area you can’t defend is the rear end of the individual—it has to defecate, there has to be a hole there—and so they tend to specifically bite the rear end of individuals. It is the sight of others approaching and this biting behavior that causes individuals to move away from those coming towards them. This need to eat other individuals means you are attracted to individuals moving away from you, and so this simple algorithm essentially means the whole swarm starts moving as a collective.

I mean, really. Think about this… how different is human behavior? (I know, I know, it’s more complex, but…).   Consider the elections, consider online behavior, and consider office politics.  We move towards what we move away from and then you get this behavior that appears collective.  Perhaps in our rush to be anti-brand, unique, a cut above, a stand out, we all come together????

In the human sphere it takes us a great deal of effort and complex cognitive abilities to decide what to do if we are going to decide in a collective. What we can now show is that animal groups, with really simple cognitive powers, can actually perform these types of computations. What we now want to understand is under what conditions these types of cognitive capacities work. How can these animal groups take information from multiple sources; how do they filter out noise and yet amplify weak signals?

Right, we should get some data to back that up!  We are on to it…

Second idea that is interesting is one reason I’m interest in NKS Summer School - simple rules applied over and over and sometimes compounded can generate very complicated behavior.  Sometimes the behavior is so complicated we’re inclined to model it with complex rules.  I want to explore this in depth and showcase it visually.

Dr. Couzin makes a great case for why the study of behavior is key to understanding.

The locust is one of the best-studied organisms for physiology and neurobiology. It is really an amazing model system for looking at these principles. Yet the last time people looked at the swarming behavior of locusts in the lab was in 1954. So there is this dearth of information, and no matter how much you look at a locust, and how much you know about the biology of a locust, you cannot predict what will happen when you start to put these organisms in swarms. …. Issues like the dimensionality of the problem are important; issues such as certain details of the interactions are not important if you want to understand the general principles of how it changes—the phase transition is very important for us in the case of locusts because, as everyone knows, locusts are always around. But then suddenly there is this transition from one state to another almost liquid state—so its driven liquid-type state—where the swarms can become enormous.

He, too, arrives at a similar curiousity in the application of these abstractions to the study of media.

Individuals can have a certain opinion on certain topics, and we can allow individuals to interact across a social network, and of course the social network’s topology is partly defined by what opinions you have. You tend to interact more often with people who have similar opinions to yourself because you are more likely to meet them in your sphere of life. But interacting with people can change your opinions, which can then change your social network, which can change your opinions, so again we have this recursive feedback. And so we are using it to explore these types of properties?I am sure that these types of principles also would apply to understanding dynamics on the Web.

I know there’s some excellent work by Duncan Watts on how individuals buy on-line, or how they judge information that they have on-line, how your judgment of something is dependant on what previous people have said about it. What they used was an on-line music store, where you either have information about what previous people have thought about a song, or you have no information and you just have to rank the song without that previous buyer. This strongly changes people’s behavior because of course when you see what other people have been doing, you can have this autocatalysis, this positive feedback. You can tend to buy into that because you have seen other people do it.

Yes, like so many of my posts, we’ll leave with a simple take away:

We are constantly looking for areas where we can create a more data-driven science behind the spread of these normative behaviors.

And maybe this constant looking is also a spread of a normative behavior?

~R

p.s.
For info on Duncan Watts weave your way to his Kevin Bacon paper and his take on online music/social trends.  You may want to hit up his wikipedia page for more links.

And Yahoo! research, where he works, has some kickass concepts, publications, apps.

:: :: :: :: :: :: :: :: :: :: ::