When Averages Lie: MaxDiff vs. Rating Scales

The rating-scale ceiling

Ask people to rate ten features on a 1–5 importance scale and most land between 3.8 and 4.4. Everything is "important" when importance is free. The resulting bar chart ranks features by decimals of an average — differences that often sit inside the noise.

What MaxDiff changes

MaxDiff (best–worst scaling) shows respondents small sets of items and asks for the most and least important in each set. Because every choice is a trade-off, the method recovers a preference scale with real spread: you learn not just the order, but how much distance separates the top item from the middle of the pack.

• No scale-use bias — there is no scale to use generously.
• Scores are ratio-like: an item at 20 shares carries twice the preference weight of one at 10.
• Results survive translation across markets, where rating-scale norms differ.

When a rating scale is still fine

If you need an absolute read ("is this acceptable at all?") rather than a relative one, a rating scale answers it and MaxDiff can't. The honest move is matching the method to the decision — which is exactly what a forced ranking of averages fails to do.