Card sorting is a powerful way to understand your users’ mental models
In order to find trends and patterns, the data collected in a card sort can be viewed through a number of lenses, each represented by a different tab on the summary.
The summary for a card sort step contains four tabs, each of these provide a different lens to find trends and patterns in how your users categorize these cards.
What is agreement rate?
The agreement rate is a way of understanding whether participants agreed that cards belong in certain categories. Slightly different agreement rates are shown across a card sort summary, these are:
Individual agreement rate: This considers a card’s relationship to a certain category and the percentage of users who agreed with that sorting. For example if 3 out of 4 participants placed a particular card in a particular category, then the individual agreement rate would be 75%.
Category overall agreement rate: The overall agreement rate for categories takes an average of all individual agreement rates for cards added to that category.
Card overall agreement rate: The overall agreement rate for cards takes an average of all individual agreement rates for categories that a card was added to.
What happens to cards left unsorted?
Unsorted cards are reported in a similar way to sorted cards. Unsorted will appear as a category in the categories, cards and agreement tabs. Unsorted cards are excluded from the similarity tab metrics.
Categories tab
The categories tab displays data for categories predefined by the researcher and created by participants. At a high level, you will see number of cards that were sorted into each category and the overall category agreement rates.
By expanding each category you can see which cards were added to the category and how frequently, as well as the individual agreement rates for each card in that category.
Standardizing open and hybrid card sort results
Standardizing categories is the process of merging very similar or the same categories into one category for analysis. This is useful because open and hybrid card sorts allow participants to create their own categories, which may be very similar to each other.
In these cases, Ballpark will automatically merge identical user created categories like ‘Shoes’ and ‘shoe’, but participants may also label categories like ‘footwear’ and ‘shoe’ which you can standardzse for your analysis.
Once categories have been standardized, the agreement and similarity data will be updated across all four summary tabs.
How to standardize categories:
Go a project with an open or hybrid card sort step
At the top, press Summary
Scroll down to the card sort step
At the bottom of the Categories tab, press Standardize categories
On the left hand side of the step, press Add new card
Choose two or more categories that you would like to merge
Press Choose a name
Enter a name for this newly merged category
Note: You can use the name of one of the original categories that this is replacing
Press Save category
The aggregate data in the summary tabs will now be recalculated
Please note, that you aren’t able to modify an existing category standardisation. This is because analysis should be conducted once all responses have been received.
To change an already standardized category:
Next to the category press the ✏️ pencil icon
Select Reverse standardization
Create a newly standardized category using the steps above
Pause your project before standardising: This will ensure the integrity of your findings and save you from having to re-standardise your data when more results come through.
Cards tab
The cards tab looks at the same agreement and frequency data as the categories tab, but this time with a focus on the cards. At the top level, you are able to see the overall agreement rate for that card and the number of times that it was sorted.
By expanding each card on the list, you can see each category that the card was sorted to, how frequently and the individual agreement rate.
Agreement tab
The agreement tab contains a matrix which displays each card and category pairing along with with the individual agreement rates.
The horizontal (x) axis displays all of the categories and the vertical (y) axis shows each card. Within each cell is the agreement rate for that card and category pairing, with the highest agreement rate shown in the top-left corner.
If you have a large number of cards and categories, you can expand by pressing View full breakdown below the matrix.
Similarity matrix tab
Similarity is how often two cards were paired in the same categories, even if the actual categories varied between different participants.
Each cell reflects the relationship between two cards which is represented by the similarity rate – the percentage of times that two cards were paired together by participants.
In the below example, Socks and Sneakers were sorted together by 66.67% of participants, but the actual categories they were placed together could have been Footwear or Accessories.
If you have a large number of cards and categories, you can expand by pressing View full breakdown below the matrix.
Downloading card sort data
You are able to export your card sort response data in CSV format, so you can easily analyze your responses in various other ways.
How to export as CSV:
Open the Summary
Scroll down to the card sort step
Click Download CSV
Viewing individual card sort responses
You are also able to view how an individual participant completed a card sort step. This can include any video, audio and screen recording that was collected in the process.
How to view a card sort response:
Open the Responses tab
Press View next to the response you would like to open
Scroll down to the card sort step