Here is a Data Visualization graphic from Office of the Joint Chiefs of Staff showing the clear way for Afghanistan.

Reminds me of the quote mistakenly attributed to Shakespeare-

Oh what a tangled web ( ^+^) we weave
When first we practice to deceive. – Sir Walter Scott (Marmion, 1808)

we were lucky enough to have the fantastic David McCandless come in to the agency yesterday to talk about the wonders of data visualisation (what this space). if you haven’t already – check out informationisbeautiful

I’m not sure if the following meets David’s rules for what makes a great data vis but it’s a quite nice interactive data visualisation of 5 years of data visualisation:

I guess once I get going, I keep going for awhile. Recently, I put up some T-shirts for sale which use my art for designs. However, after a few years of showing them, I also wanted to get some of my data and security visualization art available as well and, yesterday, I finally did it.  You can click here to go to the store:

Data Visualization Art Prints

Some of these don’t look quite as surreal or “clean” as other data visualization art, but that’s because I’m very interested in the specific cross-section of usability and “prettiness” in the aesthetics of images: The place where what makes them useful is also what makes them attractive. Finding that line, in my mind, is what makes them “art”.  One could make some really cool looking images out of most semi-structured data, but it would cease to be useful. The ones here retain their function to security and data analysts while, at the same time, being attractive pieces.

If you’re interested in other security visualization information, try secviz.org

Well there’s lots of snow in DC! Reports say this will surpass total snowfall of any storm in the past decade, and we may have to look even farther back than that. Right now (9:30 AM ET) there is about 10 inches or so, and it’s still coming down fast and fluffy. Woohoo!

To put this in perspective, let’s look at some average monthly snowfalls for the Washington, DC area vs the rest of the United States. Data is from the National Oceanic and Atmospheric Administration (NOAA) National Climatic Data Center (NCDC) and represents the past 40 years of data for DC and (on average) 52 years for the rest of the United States. Total stations is 276, many from the weather stations at regional/national airports. Here is the raw data set (before I cleaned it up for visualization).

To note, the total average annual snowfall for Washington, DC is about 19.5 inches, while the total average annual snowfall for the rest of the United States is about 32.2 inches. This does, however, include some extreme values from Alaska (and Puerto Rico for some zeros too). The maximum annual snowfall was at Valdez, Alaska with 326.0 inches. If I was to do this comparison again, I might trim some extremes from both sides of the data set, but now it’s time to go play in the snow.

With my post on “Everything is Connected” I thought I’d investigate a bridge between happiness and the level of development in a country…

The Happy Planet Index (HPI)

• Link: http://www.happyplanetindex.org/
• Dimensions: Life expectancy, life satisfaction, ecological footprint
• Total Countries: 143
• Range: [0,100]

“The HPI is an innovative measure that shows the ecological efficiency with which human well-being is delivered around the world. It is the first ever index to combine environmental impact with well-being to measure the environmental efficiency with which country by country, people live long and happy lives.”

The Human Development Index (HDI)

• Link: http://hdr.undp.org/en/statistics/indices/hdi/
• Dimensions: Life expectancy at birth, knowledge and education, standard of living.
• Total Countries: 178
• Range: [0,1]

“The first Human Development Report (1990) introduced a new way of measuring development by combining indicators of life expectancy, educational attainment and income into a composite human development index, the HDI. The breakthrough for the HDI was the creation of a single statistic which was to serve as a frame of reference for both social and economic development. The HDI sets a minimum and a maximum for each dimension, called goalposts, and then shows where each country stands in relation to these goalposts, expressed as a value between 0 and 1.”

Thoughts and Hypotheses

There are two relationships we will want to consider:

• Correlation: Is there any direct relationship (positive or negative) between the values of the HDI and HPI?
• Clustering: By region (or other characteristic field) can we find any clusters in the data?

Since these are composite indices of several weighted variable inputs, hopefully this top-level approach can identify some possible matches and mismatches between underlying data fields too. Related to the HDI, I bet the UN’s HPI (Human Poverty Index) has a bridge to happiness… or most likely, unhappiness.

Data/Discussion

• There seems to be a connection between deviations in the data. When there exists a large deviation, for a specific region, for the HDI, there seems to also be a large deviation of values for the HPI. Notice that Africa, Australasia, and the Middle East all have similar double-digit deviations. What does this tell us about the range of development and happiness within a specific region? Perhaps this could be tested across many country-level metrics to see if the similar deviations occur more frequently.
• As with the above note, since we have these metrics on a same scale/range, let’s combine them to see who has the highest composite score. In alphabetical order we have: 84, 125, 138, 137, 133, 134, 126, 134, 119, 117, 119. There seem to be three groups here: High (>130), Medium (100-130), Low (<100). Depending on a user need, algorithms can be created to join metrics to provide a big picture representation of economic, political, sociological, etc metrics, and flexibility can be built to dig into the weeds on the underlying data. This would be a nice comprehensive framework for understanding how countries (and regions as a whole) change over time.

• Looking at the scatter plot, it is clear that some clusters may exist, for example with Africa (blue). Caribbean (orange), Europe (green), and Russia and Central Asia (purple) also show some quick visual clustering, while the Middle East (red) shows the opposite. What could this mean? That regional trade, policy, weather, etc are good supplementary foundations for providing happiness and development?
• We could add trend lines and quickly check for any linear (or logarithmic) relationships. If any relationship does exist as a whole or with a region, it is certainly not a directly proportional or inversely proportional one. This was expected as these metrics are quite different (despite the overlap in life expectancy as an input dimension).

Moving forward, the methodologies and underlying dimensions (with their sources) should be compared. Data is always good, but with good data one still must be careful. That being said, this is a good start for a much larger investigation into the connections between different country-level metrics, especially if they are to be used in international and national policy.

Ever get a letter from your alma mater asking what you’ve been up to since you graduated? That data is used to see how well the college has been doing. It’s especially important for public colleges to see if most graduates stick around. Ever send one back? No? Well, most don’t.

But that’s a pain for researchers who really want to know. The response are unlikely to be random so we can’t infer much about the results. Iowa Department of Education and Iowa Workforce Development teamed up to track alumni from Iowa community colleges using administrative records. That is, we don’t ask you, we just look in pre-existing databases to find you.

The result is an infographic [PDF] that explains the methodology and results. It’s part of an upcoming series on Iowa community colleges and labor supply.

Encontrei em um dos blogs da Harvard Business um post interessante sobre visualização de dados, com um exemplo de visualização que vem lá da segunda guerra mundial.

About ten years ago, I vividly remember visiting the Cabinet War Rooms in the basement of Whitehall, where Churchill had his war room during WW II. The desks were full of phones, and the walls covered with maps and information about troop levels and movements. These used color coded pieces of string to help Churchill’s team easily understand what was happening.

On the one hand, I was struck by how primitive their information environment was only sixty years ago. But on the other, I found it reassuring to see how similar their approach was to war fighting today. The mode, quality and speed of data capture has changed greatly from the 1940s, but the paradigm for visualization of the terrain, forces, and strategy are almost identical to those of WWII.

No post o autor ainda aponta 3 bons motivos para apostar nos dados:

Great visualizations are efficient — they let people look at vast quantities of data quickly.

Visualizations can help an analyst or a group achieve more insight into the nature of a problem and discover new understanding.

A great visualization can help create a shared view of a situation and align folks on needed actions.

Source

http://manyeyes.alphaworks.ibm.com/manyeyes/files/thumbnails/bb09d328-d863-11de-a602-000255111976.wm.png

Note this is from 2006 Data, so assume 2009 CO2 as more than this.

Data Source-

TN guys at ORNL at http://cdiac.ornl.gov/trends/emis/glo.html

Data Visualization: MANY EYES IBM

http://manyeyes.alphaworks.ibm.com/manyeyes/visualizations/2006-co2-emissions-by-country

There’s been a little hiatus here as I have been traveling. I recently learned that Microsoft has launched Codename “Dallas”, a service for purchasing and managing datasets and web services. It seems they are trying to provide consistent APIs to work with different data from the public and private sectors in a clean way. There’s an introduction here.

This type of online data repository seems to be an idea whose time has arrived – I have previously talked about resources like Infochimps, Datamob and Amazon’s Public Data Sets, and there is also theinfo.org, which I seem to have forgotten to mention. A recent commenter on this blog pointed me to the comprehensive knowledge archive network, which is a “registry of open data and content packages”. Then there are the governmental and municipal data repositories, such as data.gov.

Another interesting service, which may have a slightly different focus, is Factual, described by founder Gil Elbaz as a “platform where anyone can share and mash open data“. Factual basically wants to list facts, and puts the emphasis on data accuracy, so you can express opinions on and discuss the validity of any piece of data. Factual also claims to have “deeper data technology” which allows users to explore the data in a more sophisticated way compared to other services like the Amazon Open Data Sets, for instance.

Companies specializing in helping users make sense out of massive data sets are, of course, popping up as well. I have previously written about Good Data, and now the launch of a new seemingly similar company,  Data Applied, has been announced.  Like Good Data, Data Applied offers affordable licenses for cloud-based and social data analysis, with a free trial package (though Good Data’s free version seems to offer more – a 10 MB data warehouse and 1-5 users vs Data Applied’s file size of <100 kb for a single user; someone correct me if I am wrong). The visualization capabilities of Data Applied do seem very nice. It’s still unclear to me how different the offerings of these two companies are but time will tell.

Could be an interesting way to think about searching for information. Fontplore is an interactive application designed for searching and exploring font databases. Fontplore helps you to easily find the right typeface for your project in a collection of several thousands of fonts. It lets you browse, preview, compare and print the fonts you are interested in.

Some pseudo-random Info Viz links

Here is the best internet site of 2009.
It basically shows how many jobs have been created per dollar spent.
Funded by the debt of American Treasuries
sold to Chinese.

Remember the Chinese Opium Wars.
Well the Chinese are hooked to American Treasuries and they probably need a Warship with Admiral to open their markets and currency. Oui!

Well anyway the website is called http://Recovery.gov

Looking around the web for data visualization practices it becomes obvious that the future of ‘market research’ lies at the blurred border between quali and quanti, where content will not strive for supersimplification, BUT complexity because it can simply and intuitively be handled with the latest programming languages (i.e. Processing, Phyton). Thanks to some bored bright kids looking for beauty and attention, research will become much more fun and much more genuine. At this moment in time, programmers are still fighting to find the balance between beautiful visualization and effective visualization.

Here’s what I came across in ~2 hours!

Twistori.com displays tweets in real time based on your selection of a key word:

The digg arc (from digg labs) creates stories following threads of discussion:

We feel in colors. With Adobe Kuler people can tag the colors used in their projects, to visualize the feeling of the concept. How would you describe ‘a nice surprise’ in colors?

And what’s the shape of a song? Martin Wattenberg shares many examples of a song shape in blue circles:

Or you might enjoy a visualization of song lyrics here.

Watch the blog visitor’s behaviour with a heat map (brought to you by Crazy Egg):

Elastic Minds use the ‘elastic list’ principle to search through Noble prize winners. Very easy to use, very neat:

Karl Hartig’s percentage movement of US household use of consumer electronics between 1920-1998:

An unexpected ‘matrix’ if you may on the functionality/performance of The Splayd brought to us by Rag Bag:

And not a very beautiful, but a good story from facts on charity activities here. But below you will find a beautiful story on a Roadtrip (made by Oostring):

“15 Stunning Examples on data visualization” here or “50 Great Data Visualizations” here.

A data visualization platform for users and the curious at Many Eyes (btw, brought to you by IBM). You can also create your own visualizations with interactive available tools; they are public, if you want them for your eyes only, you must contact Many Eyes:

Or give a try Drastic Data.

Here for inspirational essential Inforgraphics and Data Visualization blogs to follow.

In the future  ‘market research’ will refer to information and not data. Data is numbers. Numbers with a message is information.

And before I end, wonderful work as usual from Stefaner Moritz – learning geography in a practical and neat way: