Customer ratings are useful things, aren't they? They help when you're making a purchase decision, even if you end up disagreeing with the rating. And they're showing up more and more—in the form of stars, likes, favorites, and thumbs-up buttons showing up everywhere. Naturally, all this structured data makes us want to analyze it, but what do the ratings really mean?
What if people have figured out that these scores do something, and they're trying to make the algorithms work better for them?
Tipping the DJ
Here's an easy example. Pandora is a wonderful service. Tell it the name of an artist or song that you like, and it starts playing music that you'll probably also like. The new list becomes a "channel" that you can listen to any time you like.
As Pandora selects songs for you, you can rate the tunes with thumbs-up/down buttons. Thumbs up? You'll hear the song more often when you listen to that channel. Thumbs down? Never again. If you're going for a particular sound, you might give a thumbs-down to a song that you like, in order to remove it from that channel.
Before you analyze the ratings (assuming you could get the data), what do they mean? In this context, it does not mean "I like/don't like the tune." It means that the user wants to hear more or less (none) of that tune on this channel. If you create a quiet music channel, you might give your favorite artist/song a thumbs-down, because it doesn't fit with the channel.
iTunes does something similar with its option to play higher-rated songs more frequently in its shuffle mode. But its five-star ratings are global within a user's playlist, so it's trickier to fine-tune individual playlists. Still, more stars don't really mean "I like this more" if the system interprets them as "play this more often."
Media-sharing sites (YouTube, Flickr, SlideShare, and the like) give users the ability to mark an item as a "favorite." Which means the person really, really, likes it—or wants to be able to find it again. You'd need to do usability testing to figure out how people use the feature. My take is that favorite is like friend: overused and meaning something different in the online context.
Ever favorite a tweet? Was it really one of your favorites, or is that just how Twitter's bookmark feature works? Thought so.
Like a discount?
Who doesn't like fans, right? They're almost like friends. But a survey by ExactTarget and CoTweet found that 40% of Facebook fans are there for discounts and promotions. Maybe they're a lot like Facebook "friends," in that the label doesn't really describe what's happening.
Sometimes, a star is just a star
I'm more inclined to believe the product ratings people give on shopping sites, though there's always the risk that a "customer" review is from an employee (high score) or competitor (low score). Usually, the 5-star scoring system seems to be what it claims: people's opinions of the product.
Still, it's worth considering whether product ratings might be considered in sites' recommendation engines. I know that Amazon, for example, recommends products based on things you've bought or looked at, and I don't always want to see more of what I just bought. I don't know if product ratings figure into the mix, but what if people think they do? How does that affect your interpretation?
Photo by Adam Arroyo
Buzzword bingo bonus: use these words in a sentence: social, ratings, sentiment, influencer, customer, tomato.