Why is it that so many people talk about the effects of social media on reputation, but so few mention the more interesting models for measuring reputation? Instead, we argue over how to read the sentiment mood ring, or which media-oriented measurement tracks reputation. In most cases, I don't think we're measuring reputation at all. Instead, we're measuring media coverage.
Media analysis reports on published statements. In the recent past, it focused on media created by professionals, but even as it includes media created by everyone else, it's still mostly about reporting aggregate data based on coverage. The usual metrics—volume, sentiment, topics, and voices—reflect that media-centric view, which is now adapting to summarize consumer's opinions in online media. It's good data for some applications, but it's the shallow end of the pool for understanding reputation.
Wading into the deeper water, we find some companies that take a more nuanced view of reputation. These models start with survey research and are typically calibrated to focus on the relevant attributes for a specific company or industry.
- Reputation Institute: RepTrak
Measures 23 attributes of 7 dimensions: products/services, innovation, governance, workplace, citizenship, leadership and performance. - Harris Interactive: Harris Reputation Quotient
Measures 20 attributes of 6 dimensions: emotional appeal, products & services, social responsibility, vision & leadership, workplace environment, and financial performance. Harris recently released its 2009 report (PDF). - APCO: Reputation Insight (PDF)
Multi-factor models customized for each client. - Millward Brown: BrandZ (more)
Evaluates the financial return attributable to the company's brands, based on an analysis of financial data and consumer research.
I know that these models can be incorporated into routine measurement programs, but I almost never hear about that. I don't hear about these models in the usual PR and social media measurement discussions, either. Why is that?
Is this stuff not accepted? Is it too advanced? Maybe too confidential? Or is it just above the social media paygrade? The audience in the room where I first learned about this was rather senior.
Hat tip to Leslie Gaines-Ross for pointing out some research I hadn't seen.
Nathan, thanks for this post. The first step to discussing different approaches to measuring reputation requires an agreement on what constitutes reputation to begin with. At evolve24, we believe that the definition that reputation is ultimately “what people expect from us tomorrow based on what they have seen through today” should not leave too much room for disagreement.
If reputation is about expectations, then the measurement of reputation requires capturing and revealing those expectations. The four methods you have listed in this post utilize primary research techniques, i.e. surveys and focus groups, to collect expectations.
This offers the first insights about why these approaches might not be discussed in the arena of social media.
There is another important difference between the four models for measuring reputation you’ve listed in the post and methods that measure reputation within media. The methods you’ve provided collect insight into reputation according to their pre-defined models of attributes and dimensions. When they receive a response, they must place that response into a pre-existing category. In essence, these approaches tell companies how the things that people are saying about them fit into a pre-defined structure.
The approach to reputation measurement through “listening” takes a different view. In collecting the attitudes, perceptions and impressions shared through media, media-driven reputation analysis allows companies to build a structure based on categories, attributes and dimensions that emerge from the conversations. As opposed to fitting conversations into a pre-defined and fixed structure, media-based measurement builds the structure from the conversations.
A third key difference in these approaches comes to timing. Because the approaches you listed are driven by primary research, their insights are fixed to specific points in time. In contrast, with a listening-based method, perceptions and expectations can be measured and acted upon as they emerge.
I think we can agree that traditional and social media have a large role to play in creating perceptions and setting expectations about a brand, and that therefore many of the drivers of what is measured by asking someone how they perceive a brand reside in what they have heard about that brand before they were asked. (Herein lays the importance of understanding influence). And of course, with social media, we don’t even have to ask for these perceptions, as they are often freely shared.
Thus, utilizing social media reputation measures allows firms to evaluate perceptions and expectations amongst finely segmented audiences. The primary research reputation measurement approaches survey primarily customers – they do not tend to survey interest groups and NGOs, regulators and legislators, or suppliers and vendors, regarding their clients’ reputations. Media analysis allows companies to measure the media that influences these groups, and to quantitatively estimate the reputation of a company within these groups, based on the full-set of information that these groups might encounter around the brand.
You are correct to note that expectations, and thus reputation, will also be built around experiences that are not colored by or reflected in media conversations, such as direct experience with a product or service, or word-of-mouth conversations amongst family, friends, peers, etc. These aspects of reputation are effectively measured by asking questions directly.
We have worked with several clients who find the greatest understanding of reputation comes from coupling these two approaches. A media-based reputation measurement approach allows them to measure real-time shifts in perception and expectation amongst a broad range of stakeholder within content categorization structures that align reputational perceptions with their business structure and strategic initiatives. Understanding what is emerging in conversations in the context of business-structures can help drive the questions that are asked in the primary research approaches.
At evolve24, we are helping clients link together listening measurements and primary research measurements through correlations that look to find how strongly and how far in advance the opinions expressed through surveys or focus-groups can be found in certain sets of media. Social media reputation measurement puts companies ahead of the game in terms of responding to the full range of people and perceptions that can drive reputation. But utilizing primary research and standardized models can be helpful in setting reputational strategies and goals, so an integrated approach to measuring reputation should be considered by firms who place a premium on reputation.
Scot has an interesting idea there. The source of data is very different in context, origination, and timeframe.
Another item that isn't discussed is passive user analytics gained through monitoring vs. active reputation research. The difference in passive vs active collection methods are huge.
There are hundreds of models that can be tossed into the "reputation" category... most of the old-school models are simply process vs return models.
The ultimate business value of any of these models is found with the "acid test" of locating key conversion metrics (buying an item, choosing to refer, lifetime value) and correlating them to both perceptual awareness (active) and passive habits (site visits, search behavior, conversational context.)
I agree with Scot in the fact that you have to merge several method of research together and apply some actionable business steps to make any of it worthwhile.
Finally catching up from travel...
Thanks, guys, this makes a lot of sense. In a way, this is just another example of measurement silo blindness. Take a group that is familiar with a particular set of tools, ask them a question, and they'll try to answer the question with their tools.
I like the linkage between active research and passive analysis. I would expect that the results of each method could be fed into the other for testing or tracking, respectively.