Scaling Human Analysis


AEDE2AAA-9300-4636-80E9-CE2F90947F85.jpgOne thing about sentiment analysis: it really stirs up the opinions. Apparently, it's good for attention, too, because yesterday's post has gotten a lot of it. So what is it about automated analysis that's so controversial, and what can human analysts do to offset the advantages of automation?

Automation offers four basic benefits:

  • Scale
    Keeping up with all of the relevant conversations as volume grows.

  • Speed
    Processing new items sooner; computers "read" faster than humans read.

  • Consistency
    Software doesn't get less accurate with fatigue or mood, and it doesn't consider contextual knowledge if it's not supposed to. It just follows instructions, over and over.

  • Availability
    Automated systems don't sleep, so they won't be the limiting factor in determing your 24/7/365 operations plan.
The trade-off—or the development challenge, depending on your point of view—is in accuracy and interpretation. Most of the discussion focuses on these accuracy issues, so let's think instead about the less controversial benefits and how a software-assisted human analysis approach can compete with them (remember, SAHA is human analysis within a software-mediated environment for operational efficiency).

Competing with automated systems
Scale and speed are related, and the hands-on approach is simple: add more people to the process. Speed (latency) will still be limited by the ability of your analysts to read quickly. You won't compete in the sub-second latency market, but you can get ahead of the daily-update crowd.

Consistency in human scoring comes down to training and process. I won't pretend to teach the media analysis pros how to do that job, but it's going to be more formal than the eyeball ratings I gave out yesterday.

Availability is an interesting challenge, but it's not the first time companies have addressed the issue. If you're going to work through nights and weekends, your choices are to create some undesirable jobs at home or switch to follow-the-sun operations overseas.

The rise of offshore outsourcers
Combine the need for a lot of people (analysts) with the desirability of around-the-clock operations, and a lot of people will reach the same conclusion. From the beginning of the social media analysis business, some of the better-known vendors have had development and analyst groups in India. Now, I'm starting to hear from companies that are offering offshore human analyst services as a specialized service.

The interesting bit is that they're unbundling the content coding, so clients (or vendors) can add human-powered sentiment analysis to any platform that provides user coding or tagging.

This won't be an easy group to track. It's largely a traditional outsourcing approach, and any company that provides full-service social media analysis using human analysts could unbundle the coding piece, too. But if clients end up selecting human analysis over the automated version, expect more offshoring of the manual effort.



As always, an interesting read.

On this point:

"But if clients end up selecting human analysis over the automated version, expect more offshoring of the manual effort."

I see clients unique requirements dictating this more than any debate over human vs machine analysis. I don't see this being the defacto approach mainly because of the stigma against offshore services (i.e. language/cultural barriers are the big ones). Vendors with more comprehensive multi-language monitoring services might be the exception. However there is also the aspect of sensitive intelligence which might make clients feel uneasy about handing over details to a company with offshore dealings.

In our experience, the critical parts of providing the human review portion in a precise and reliable manner is found in leveraging the machine portion to its absolute limits, relieving the analyst with as much of the manual intensive portion as possible. The assembly line analogy doesn't fit entirely well here because some of the moving parts of SM require a trained eye to understand the more complex and layered incidents of risk.

As far as automation is concerned, the hit and miss tendencies far outweigh the benefits and I don't particularly view scaling as being nearly as problematic or challenging on the SM analysis and listening side as the human resources and staffing needed for higher visibility brands to manage crisis and treating the addition of human touch points as a habit rather than a campaign.


Thanks, Joseph. You make several points worth pondering. I won't attempt to draw a conclusion that fits every company, except that I think everyone agrees that the final step in the process involves a person looking at the data, deciding what it means, and making (or recommending) a decision. The disagreement is over just how early in the process the systems should hand control over to the analysts.

Proposed this very thing to a client in 2006. Unfortunately, a little too soon for the market.

Comments are now closed for this entry.

About Nathan Gilliatt

  • ng.jpg
  • Voracious learner and explorer. Analyst tracking technologies and markets in intelligence, analytics and social media. Advisor to buyers, sellers and investors. Writing my next book.
  • Principal, Social Target
  • Profile
  • Highlights from the archive


Monthly Archives