October 2012 Archives

I sometimes summarize the opportunity of social media analysis as using computers to "read the Internet." It's not an original idea, but it is one we still haven't mastered. I've seen many tools that find relevant content and apply some level of automated analysis, but we're not about to replace the analyst. One simple question I've started to think about is, "then what happens?"

The SocialSpook 9000 reads millions of blog posts, Facebook updates, and tweets every second. It finds every relevant mention in your space, extracting the facts, opinions, and needs that you're looking for. Its sentiment analysis engine provides 120% accuracy in 38 languages, and its graphics are so well designed that whole new awards contests have been created for it to win.

In 2007, I pointed out the need to link social media monitoring to customer service, because most of the problems that people were seeing as PR problems started with unhappy customers. Since 2010, I've been thinking about another application: blending social media data with other publicly available sources to create an automated view of what's happening in the world. It turns out to be a big challenge.

My own private news channel… or command center?
We can take this in several directions. At the low end, applications such as Flipboard generate personalized media based on activity in the user's social media accounts and selected topics or sources. In the middle, we might have a more dynamic version of the social media dashboard running in the conference room or reception area. It's the web-powered news channel that always shows something you might care about.

At the high end, we're looking at a valuable—but noisy and sometimes misleading—source of crowdsourced information about events in near-real time. The obvious applications are in government: national security, law enforcement, emergency management, and disaster response agencies are looking for fast and accurate information from social media sources. I see value in corporate applications, too, for functions like security, risk management, logistics, and business continuity that need information when things happen. Preferably without hiring an army of analysts to look at dashboards on the quiet days.

Now what happens?
The challenge in using social media for real-time awareness is that the volume level becomes overwhelming just when the information becomes most valuable. Forget looking for the needle in a haystack; this is the needle in the needlestack. Faster than you can read them, more messages arrive, and they're all relevant.

Existing tools generally emphasize either handling messages individually (think customer service or community engagement) or analyzing them in aggregate (think sentiment and leading topics). For this application, we want the system to help analysts deal with the volume without losing the detail, and that's where I started asking about what happens next.

For all the systems that can notice something happening and put it on a screen, I wanted a system that can notice and pay attention. So what would that look like?

Here's an idea (click to enlarge):

Computer Attention in Situational Awareness Applications

The inputs to this system can go beyond social media content; depending on your application, it might pick up data about natural disasters, weather, or market data. It might incorporate traditional news media, commercial intelligence services, or internal data. Its models will reflect the needs of its users, so a system that looks for, say, transportation-related incidents could be quite different from one looking for damage reports in weather emergencies.

This has a lot of moving parts, and it builds on what others have already built. The central idea is to go beyond the dashboard and think about how the system can relieve analysts of some of the burden of reading the alert queue. Step one is to consider what an analyst does with that information and how a computer could mimic that.

I'm sharing some of the frameworks that have been hiding on my whiteboard. Want the long version? Email me.

Be Careful with that Email

Intuit scam emailI'm noticing a big increase in fraudulent emails, and they look more convincing than ever. October is National Cyber Security Awareness Month (via Coretta Jackson), so let's talk about email safety. If you already know this stuff, maybe this is the time to talk to friends and family about it. Parents, this is you, too.

When was the last time you got one of these?

Within the last hour? I don't even count how many of these I get every day. But you already know that none of those is what it claims to be.

Fish in a barrel
The most common online scams target the most gullible people. Hey, fraud is a business, and when you're sending out millions of offers, you need to screen your leads well. According to a new study from Microsoft Research (PDF), that explains why so many emails are so obviously fraudulent: they're targeting people who are too gullible to notice the scam.

An email with tales of fabulous amounts of money and West African corruption will strike all but the most gullible as bizarre. It will be recognized and ignored by anyone who has been using the Internet long enough to have seen it several times. It will be figured out by anyone savvy enough to use a search engine and follow up on the auto-complete suggestions… It won’t be pursued by anyone who consults sensible family or fiends, or who reads any of the advice banks and money transfer agencies make available. Those who remain are the scammers ideal targets.

—Cormac Herley, Microsoft Research (emphasis added)

So when you quietly delete that obviously scammy email, you validate the scammer's optimization method. But delete it, anyway.

Going after smarter targets
While the mass-market scammers are going for the easy marks, a different style of criminal is getting more aggressive about smarter targets. They're getting trickier, personalizing attacks on strategically selected targets and masquerading as services you probably use. You won't fall for the secret treasure of Idi Amin, but how about this private message reminder from LinkedIn? The email looks right—or almost right—so you click the link to go to your LinkedIn inbox… and end up installing botnet software on your computer. Ooops.

Your company won't believe it's won the European lottery, so these attackers mimic legitimate business services:

The image at the top of this post is one of two fakes I got on Friday, sent to separate addresses. It presents as approval for some payment system at Intuit, but by now, you know that Intuit had nothing to do with that message.

What does it do, exactly? I don't know, but nothing good. It probably has something to do with stealing a password or installing malware on my computer. We'll never know. <Delete>

Think before you act
Email-borne attacks are serious business. It's not some bored kid messing with your computer; it's hacktivists, criminal organizations, and even governments. As you're going through the daily slog in the inbox, take a few, simple precautions:

  1. If an offer is too good to be true, it's not true.
  2. If a need is unusually urgent, confirm that it's real before you commit resources.
  3. You don't win contests you haven't entered.
  4. Be careful about links in email, even from companies you trust. Look at the URL the link wants to send you to, before you click on it.
  5. Even better, type in the main URL of the trusted site, and use their navigation to find your inbox, or account, or password reset, or whatever you think needs attention.
  6. Be extra alert about attachments, especially ones you haven't requested.
  7. Don't open compressed (.zip) or executable files from unknown sources.
Finally, if there's any doubt about something you get in email, stop and think before you do anything. Type the main keywords and "scam" into Google, and see if the results tell you something important. Look it up on Snopes, which has been investigating rumors and scams for years. Email can wait for a little due diligence, but it's hard to unfall for the trap once you start clicking on things. You have so much investigative power as close as the nearest web browser, why not use it?

You're good with all this? Haven't been tricked in a long time? Excellent. Go share your wisdom with someone this month. Keep your family and friends from becoming victims.

Whole Lotta Influence Going On

What do you do when you disagree with a conference speaker? Do you tune out, start checking email, check the schedule for when the next session starts? Do you post snarky comments to Twitter and Facebook? Do you challenge the speaker in the Q&A time? What if the topic is well worn, and you're getting tired of hearing the same points you disagree with? Are you tired of the influence arguments yet?

While sitting through a presentation for nonprofits on communicating with online influencers, Heidi Massey got that familiar feeling, so she challenged the speaker, Justin Ware, in the Q&A. They continued the discussion after the session, and it's led to a great pair of point/counterpoint posts on using Klout scores:

I like this. Not only do they go to the effort of thinking through their arguments and writing something coherent, they do a service to everyone else by linking to each other's posts. It's a nice idea: post your position, and link with someone with a different opinion. Do you think it will catch on?

Speaking of influence, was there a memo about putting influence startups in Portland? First Tellagence, now Little Bird? If you need to identify the right people for spreading your message, the available tools are multiplying fast.

There should be a rule that if you have an argument on a Friday, you have to cite Monty Python.

The Summer Reading Post

It seems that I'm late posting this year's "what I read this summer" post. That's to be expected, since I'm behind on the reading pile, too. But summer is giving us an encore this week, so here's my chance to share some of the interesting things I've read with you.

Kill decisionAttack of the killer drones
I read mostly nonfiction these days, but I did take a summer diversion with two books by Daniel Suarez, who's making a run for the techno-thriller trophy. The first was Freedom™, which is more part two than sequel to Daemon (recommended in the 2010 list). If you liked Daemon and haven't read Freedom™ yet, you need to find out how the story ends.

Suarez's new book, Kill Decision, takes the same approach of combining current technology with a dose of near-future science fiction, but this time the threat is from autonomous swarms of killer drones. This one's weaker as a novel, but it raises serious issues: Black-hat PR in social media. The use, abuse, and proliferation of armed UAVs, persistent surveillance, and open-source intelligence. The attribution challenge of cyber warfare. By pushing these themes (and others) to an extreme, Suarez creates an opening to think about where the limits are, and where they should be.

If the dawn of the drone age interests you, you can balance the fictional portrayal with Wired for War, P.W. Singer's 2009 book on UAVs and ground-bound robot warriors. You'll wonder what war even is, when one side is far enough from the action to avoid harm. From there, pick up Rachel Maddow's Drift (2012), which—despite the author's well-known leanings—is a generally conservative take on the vanishing checks (in the U.S.) on executive power to make war.

What I like about Suarez is that his novels tie into real-world issues in a way that gets you thinking. These other books prove that the topics Suarez raises are real, even if the specifics include some science fiction.

Drunkards walkRandom difficulty
How improbable is an enjoyable read in statistics and probability? Leonard Mlodinow's The Drunkard's Walk: How Randomness Rules Our Lives (2008) demonstrates how much of what we interpret as cause and effect may be the result of the expected variation in random processes. Baseball stars and hedge-fund winners look a lot like coin tosses, if you look forward into the future instead of backwards into the hindsight.

The Drunkard's Walk gives the best explanation of Bayesian reasoning I've yet encountered, even while using the cancer-screening example that must be required. You also get a chapter on the Monty Hall problem, which is a bit of a mind-bender even after the light comes on. We're really not used to problems that break the rules of the discipline.

Nassim Nicholas Taleb's first book, Fooled by Randomness (2001), covers much of the same territory—people misinterpret randomness habitually—but it returns too often to the financial markets for its lessons and examples. Taleb's forthcoming Antifragile, on the other hand, looks like a must-read this fall. Systems that actually benefit from chaos and black swans? Time for some solid-surface counterintuitive.

Everything is obviousHow's that prediction working out?
Flipping the empirical method around, Duncan Watts tears into our habit of backfitting our analysis to fit past events and calling it common sense in Everything is Obvious (Once You Know the Answer) (2011). In so many different ways, the world is too complex to predict—oh, and randomness is a problem that we don't handle well.

The answer is to respect the unknown, build flexibility into our plans, and get better at reacting quickly instead of trying to predict the future. These are themes I keep running into, and they make a lot of sense. Or is that just my confirmation bias speaking?

The queue is winning
I started Rohit Bhargava's Likeonomics: The Unexpected Truth Behind Earning Trust, Influencing Behavior, and Inspiring Action (2012) this summer, and while I got surprisingly useful ideas from the first couple of chapters, events got in the way. I'll get back to you when I finish that one.

This post has become a bit of a tradition. If you like this, you might enjoy these posts from previous years, too: 2011, 2010

About Nathan Gilliatt

  • ng.jpg
  • Voracious learner and explorer. Analyst tracking technologies and markets in intelligence, analytics and social media. Studying complexity and futures.
  • Principal, Social Target

Subscribe