Now that ambition is an obvious target for this novice bereted librarian, so I went to find out more.

…
The basic idea is simple, as Donna Spencer put it:

Get people to put stuff into groups that make sense for them.^[1]

Online card sorting is useful, as Neil Turner concludes, when you need lots of participant sorts or when you can’t easily set-up, or need to supplement, face-to-face card sorting sessions^[2]. Online card sorting tools offer the added advantage of automated statistical analyses and rapid production of statistical reports. Not, as Courtney Clark points out, that such quantitative data could suffice to gain insight into audience mental models ^[3].

How Websort Works

As outlined at the very inviting website:

Every uncluttered item of text and image on landing page contributes to rapid impression of purpose, next steps and desirable information.

1. You enter the items you want sorted, refine the study parameters and invite participants;
2. Participants drag and drop items into groups that make sense to them – and in open sorts define their own group labels;
3. WebSort compiles the results into a variety of statistical reports.

WebSort‘s features & options

Easy and enjoyable to use

Getting straight to the point: it was easy to set up and operate (provided Flash is installed).  Of course, I cannot report on performance of features I did not use, however I was aware of each of them through the interface. Except how to track progress – I have since noticed the “track progress through RSS” link; but did not at first.

I dragged and dropped and labelled intuitively. I particularly enjoyed being able to move groups around the canvas as I would cards on a table – this helped me track the way I was thinking about the groups.

I enjoyed placing the stacks where I wanted, placing some close to each other or overlapping demonstrating their mutual relationship. (Screenshot of interface used with permission).

However:

Visual information not captured

Unfortunately, the mental connections I was making by inter-group placement would not appear in the quantitative results.  I wonder whether WebSort might find a way to capture the screen layout when participants click “I’m done”?

Also, when I did the study I did not see images I had added to some cards. I was able to watch one participant do my study and she could see the images. However all cards were sized as if they had images (that is larger) so fewer displayed in the panel.  This did not appear to bother her, and she appreciated the images.

“Leave a comment”

Few of my participants left comments.  The link “Leave a comment” appears to be an effort to capture some of the thinking that many reviewers mention being missed in online sorts compared to face-to-face. Opportunities for researcher-participant interaction such as clarifying cards or body-language indicators of points of struggle that could be investigated.

I believe when I tried a colleague’s study in Optimal Sort the invitation to comment was more obvious. The fact that I did make a comment to her might have been more because of my interest in the  methodology. Could this “other” thinking be better captured? Does it need to be?

Success, and the challenge…

is in the preparation…

My imaginary trial run highlighted to me that careful preparation is required. Particularly: Careful selection of card contents, and participants.

If you’re going to give it a trial run, use these lessons learned:

• Use images for all items or not at all. – maybe?
• Consider who you plan to ask to be your (up to 10) trial participants, and choose relevant content for them. I tested on an obscure hobby of mine, and the struggle to understand the items interfered with thinking about how they would go together.

If you’re ready to plan an online sort:

• Read both the support/FAQ page and the WebSort category of UXPunk’s blog. Some potentially vital advice is only in the blog. For example, blogged advice on careful selection of card contents could prevent some causes of wanting to change a study part way through.
• As the FAQ admits, a card sort is only one of several possible techniques for gaining understanding about your site’s users. The blog links to creative use cases which identify how card sorting works in with other research methods.

Without such preparation it is impossible to give a fair evaluation of WebSort’s usefulness.

- and in interpretation

What you want to be able to interpret really should have fed into how you prepare, and whether a card sort, or an online one, is really your solution.  The relative value of quantitative and qualitative information is not unique to card sorting, nor the online versions.

Requires Flash: ?accessibility

Tablet un-friendly

As Flash is already installed in my browser I did not, at first, give a thought to the code platform. When temporarily disabled I discovered that it is not operable via my Xoom – it would not let me enter an email address.

It was not possible to enter an address into that box. (Interface screenshot shared with permission)

While a mobile phone does not offer ‘table’ space for a sort, tablets may. Perhaps androids do not have sufficient distribution for it to be worth trying to make WebSort android-friendly. I suspect however, that the first online card-sorting tool to adjust for use on i-pads will be very popular, given a growing distribution of i-pads among mobile workers. Karen Jorgenson mentioned such a situation in the OptimalSort support forum. Optimal Sort is an alternative online card sorting tool – and the two teams are friends, how sweet .

Not for the visually impaired

Although content apparently can sometimes be developed for accessibility in Flash, I do not think that drag and drop functionality would be included. I pondered how do paper versions of card sorting cater to blind participants? Suppressing, for the moment, my curiosity over the feasibility of printing cards in braille; I wonder if card sorting is sufficiently vital a research technique to require such effort.  If so, and if anyone has or does any additional research on card sorting with the visually impaired, the team at WebSort would like to hear from you.

Price

Not something I can evaluate really.  For “all in” costs, assuming all other considerations (ie whether or not you need face-to-face information; the number of participants’ views you want) are equal:

• For a WebSort: add to one of the three price options any incentives that might be needed.
• For an in-person sort: add to the very cheap cost of paper/cards, time to physically prepare the cards, arranging participant schedules, transporting, feeding, compensating them.

Quick response to support requests

My queries: about typical usage; variations; about Flash accessibility; and permission to share interface screenshots received prompt replies.

Partial conclusions

Great free tool for small teams

I can see this being triply handy for teams of 10 or less.

1. it is free for up to 10;
2. they could sort separately to ascertain different perspectives first, and
3. screenshot their layouts to capture the additional inter-group relationship information to then discuss their models.

Beyond that I do not know

Sure, it is easy to use (on a PC with Flash, as long as you use a mouse and can see clearly). However I would need a real situation in which to test WebSort‘s relative (against other methods) value for designing user-friendly information architecture.  I’d welcome hearing from anyone who has gained insights from well-prepared studies in real contexts with WebSort (or other online card sorting) that they had not already been seeing before the study.

References

We had a blast.

OptimalSort and Websort meetup

It’s odd that as competitors there are a bunch of unspoken rules.  Like how you should never talk to each other, hang out together or openly share information.  When we started our first business 6 years ago, we told ourselves that not having a business background can work to our advantage.  We will try things that “smart” business people probably won’t do because that’s not what you do.  Making friends with our competitors is possibly one of those things.

We thoroughly enjoyed meeting Jed.  It was uncanny how similar we were, and how many things we had in common.  We shared ideas with each other, lamented about facing the same types of problems and even gave each other useful advice.  There was no ulterior motive or hidden agenda.  We just wanted to say hi and make a friend.

If you are in the position to do so, we encourage you to try it.  Call a competitor and offer to take them out for coffee.   If you’re a competitor, please talk to us – we would love to shout you coffee.

I simply entered the parameters of my study, sent the link to a bunch of potential users around the country, and watched the results come in. Then I used the super-cool reporting tools provided by Websort to make sense of all the data. I even used the free trial!

I haven’t seen a better, simpler, or more cost effective way to get this work done remotely.