Can Card Sorting Improve the Usability of Your Designs?

If you’re not familiar with card sorting at all, especially in the context of usability testing, you can read this introductory guide on Six Revisions: Usability Testing with Card Sorting.

Card Sorting Via Remote (Online) User Testing

For this article, I’m specifically talking about remote user testing.

Why focus on remote user testing?

Firstly, it’s where my experience lies. I build web-based usability testing tools in my spare time and an online card sorting tool is one of them.

But more importantly, card sorting gets many benefits if done through online remote user testing.

I’m often puzzled when I hear of usability studies involving the recruitment of local participants, even when the product is some sort of a web application or web service that will be used by people all over the world.

Image source: Yandle

Unless your website is geographically focused — for example, if you were building an online store that’s targeting only a particular country — why in the world would you recruit participants only from one area?

To me the answer to that question is simple: It’s the way it has always been done. So, if you had some formal training in usability, this is what you were often taught to do.

I do not discard the benefits of observing the reaction of participants and hearing them vocalizing their decision processes as they perform user testing tasks (though this can also be done remotely with web cams).

An experienced researcher can spot issues and derive information from a participant’s behavior that may not be obvious when you’re only relying on quantitative data and remote testing results.

But, if we’re talking about card sorting — and we are — I would argue that remote online testing is the best way to go.

For card sorting, the benefits of remote online testing are numerous. Here are a few benefits:

• Participant recruitment is a lot easier because you’re selecting from a larger pool of potential test users.
• Most of the time, recruiting participants online will be much cheaper.
• Participants can do the card sorting exercises in their own time and schedule.
• Often, remote usability testing is the only way for you to collect data from participants residing outside of your geographical area.
• Remote user testing gives you a bigger potential to diversify the sources of your results.

What is Card Sorting Good For?

The primary purpose of card sorting is to help you organize items into logical groups.

In the context of web design, it can aid you in developing your site’s information architecture. For the online card sorting tool I built, the largest use-case I see is the organization of web pages on websites; anything from navigation menu links to content categories.

However, this powerful usability testing technique is not limited to website design. For example, about a year ago, we were approached by an English teacher from New York city who wanted to use the card sorting tool to help his students better understand the characters in the literature he covered in class.

And more recently, a researcher from the University of Amsterdam shared with me that he was using the tool for psychological research and theory building.

So, basically, if you have things to organize you can use card sorting.

The limit is your own imagination.

Card Sorting for User Interface Redesigns

About a couple of years ago, I was asked to redesign a financial transaction reporting page.

When the reporting page was originally built, which was years ago, everything was able to fit in only one screen.

But as time passed, new features and sections were added to the reporting page. When I reviewed it, it was 5-6 screens deep.

The reporting page had everything on it, but it was really hard use. The challenge was to rework it in such a way that it became useful again, while not sacrificing any new features.

To better understand the problem, I proposed to start with a closed card sorting activity.

We took all the labels from all data fields of the page and put them on cards.

Then we created 4 categories. Each participant could sort the cards in one of those categories.

These were the categories we made:

• “I use it all the time”
• “I use it sometimes”
• “I never use it”
• “I have no idea what it is”

The “I have no idea what it is” category was added as a “just in case” category.

We were quite surprised, however, to discover that there were quite a few cards in the “I have no idea what it is” category after two teams of support engineers participated in the exercise. These staff members worked with the reporting page on a daily basis, so presumably they had the best understanding of it. Yet they didn’t know what some of the items were for.

If you have ever worked in the enterprise software business, you know that new features are added constantly, but the old ones are hardly ever retired.

So, even if there were a few items that were unrecognizable to its regular users, we couldn’t remove those items because they served some purpose.

But now — having a better understanding of which items were used the most because of the card sorting activity — we could rebuild the page by ordering the elements by priority, starting with the most important at the top and then tacking on the lesser known sections at the bottom.

Types of Card Sorting

Now that I have shared some of the benefits I have gotten from my experience with card sorting, perhaps you are now thinking about using it in future projects that involve the organization of things.

One of the first steps you will need to get out of the way is to determine what type of card sorting you will do.

There are two main types of card sorting.

Open Card Sorting

In open card sorting, each participant is given a stack of cards. The participant is then asked to group those cards together any way they want.

Then they make labels for the groups they created.

Closed Card Sorting

In closed card sorting, the researchers create the labels for the groups. Participants are given a stack of cards and are asked to put each card into a group.

When to Use Open Card Sorting

If you are staring at a new project and have many things to work with, consider doing open card sorting. This way you will not introduce your own biases into the grouping of items and will see the information organized from other people’s perspectives.

In addition, open card sorting also helps you get ideas for category names.

Disadvantage of Open Card Sorting

The downside of open card sorting is that there can be just as many combinations of categories as there are participants, because participants are tasked with coming up with the own grouping system.

For each participant, there can be variations in the number of groups created, as well as what the labels of each group are.

So, in open card sorting, viewing and making sense of the results usually requires a lot of effort. For example, if you perform open card sorting, you might see various labels that mean identical or similar things, such as “Customer Support”, “Help”, “Support”, etc.

You might also see one participant having only 2 card groups, while others have 5 card groups. The variability of the results is often the reason many usability researchers skip open card sorting.

When to Use Closed Card Sorting

If you already know what categories you want to sort items in, then closed card sorting is the obvious choice.

Closed card sorting additionally removes the burden from the participants having to come up with their own group labels, which simplifies the activity for them.

Disadvantage of Closed Card Sorting

Using pre-determined group labels gives you less information because the participants’ choices are confined by the category labels you create. That, in turn, limits the chance of you seeing alternative approaches to the categorization of your items.

Using Both Open Card Sorting and Closed Card Sorting

A potential testing strategy is to first do an open card sort to help you determine category names for each group of content, and to understand the different ways people can group the items.

Then, after analyzing the results, you can conduct closed card sorting to validate the interpretation of the results.

How Do You Interpret the Results You Get from Card Sorting?

Honestly, in my experience, this part has always been a challenge.

And not all of us are skilled statisticians.

The good news is that we can let our computers do all (or at least most) of the number-crunching work. (There’s a short list of tools and resources for card sorting below.)

And once you get the results organized, analyzed, and interpreted, apply the results to your project to improve its user-friendliness.

Card Sorting Resources and Tools

Here are some tools and resources related to card sorting that will help you learn more about this testing method and to help you perform it in your design and development projects:

• Card Sorting (Usability.gov)
• Card Sorting: How Many Users to Test (Nielsen Norman Group)
• Card Sorting (Interaction Design Foundation)
• WebSort
• OptimalSort
• usabiliTEST Remote (online) Card Sorting Tool
• Card Sorting (a book by Donna Spencer)
• Card sorting (Kindle edition)
• Usability Testing With Card Sorting
• Card sorting: a definitive guide

* Thumbnail image source: Yandle