Recently in Social media analysis Category

It's January, which means that I've been working on my annual posts on investments and M&A in the social media intelligence market. As always, I find myself mentally dividing the transactions into several buckets: the serial acquirers, the complementary products, the geographic expansions. While I was working on last year's post, I sketched out a matrix to summarize some of the logic of what I was seeing. This year, I'm sharing it with you.

The matrix is built on two variables: customer base and company capabilities. For each, are the merging companies the same or different? Combined, these go a long way in understanding the logic of a deal.

Customers
Start with customers, which might be characterized by industry, location, functional role, or (likely) a combination of these. If the companies serve the same customers, does the combination bring new capabilities to those customers, or is it more about scale or reducing competition? If the companies serve different customers, is the combination about access to new markets or diversifying more generally?

Capabilities
Second, compare the companies' capabilities, especially their products, services, and underlying technologies. If the companies have similar strengths, do they bring different customers or markets to the combination? If they bring different qualities, do their strengths combine to better serve an overlapping customer base, or do they do different things for different customers?

Analyzing the Matrix
At one extreme, companies that have the same capabilities and the same target customer combine to build scale and consolidate their position in the market. At the other, dissimilar companies serving different customers may combine as an investment strategy or to create something entirely new. The Same/Different corners, on the other hand, represent the most-common deal logic, in which a larger company acquires a capability or customer segment.

Every year, I hear from people who expect a big increase in sector consolidation this year. Which types do you expect to see?

Shortly after last month's announcement of the new Facebook topic data service from DataSift, another kind of change showed up in my inbox: the impending disappearance of Facebook post data from social media monitoring tools. The search functions of the API that developers use to monitor public posts in Facebook are going away at the end of April, and the notices and workarounds are going out to customers now. I'm also hearing from software companies looking for alternative sources, though I have not heard of any such alternatives.

The vendor announcements say to expect fewer results with the switch to version 2.x of Facebook's Graph API (optional until April 30). The new version restricts access to information about users' friends, and it eliminates the Public Post search and News Feed search options from the Graph API. Monitoring of posts and comments on specified Facebook pages (including competitors' pages) is still supported, which creates a partial workaround.

As for the broader set of Facebook post data, DataSift's new PYLON API is the so-far exclusive source for most developers in the social media analysis business. The data includes private posts, but everything is anonymized and aggregated, and it doesn't include verbatim text. It's meant for broad analysis, not monitoring or engagement. Access is limited to the US and UK for now, so the answer for the European software developer who emailed me appears to be “no.”

Finding information about what a company doesn't do is tricky, especially with a segment of the industry that likes its trade secrets, but Facebook's announcement makes it pretty clear how they're thinking about user privacy in the data market:

We are not disclosing personally identifying information to anyone, including our partners and marketers. And, the results delivered to marketers are analyses and interpretations of the information, not actual topic data.

One of the nicest compliments I've received over the years came from a company founder who read one of my reports and said I'd summarized his company's work better than they did. It's just one of the things I do—take a pile of information and figure out what it's about. I summarize. So if you need to tease out the short version of something complicated, call me. But I've also been accumulating data on an industry for years, which gives me the material for a different view—the annual recap. Roll tape…

The Year in M&A, Social Media Analysis 2014
I've been tracking companies that extract meaning from social media data since 2006 (it stays interesting if you let the definitions evolve with the market). One way to tell how things are changing is to watch where the money goes, and in 2014, more money flowed to consolidation. VC and PE money funded multiple acquisitons by companies staking out hoped-for prominent positions. Big companies tucked SMA into their products and portfolios, and smaller companies chose "buy" over "build" for key capabilities.

Add some actual mergers and a few acquihires, and we get more transactions than in 2013. In other news, it takes longer to write a recap of 38 deals than one with 18 deals, which is how a year-end post shows up in early January. :-)

More Than $420 Million Invested in Social Media Analysis Companies in 2014
New investments in SMA companies were slightly below 2013 levels in dollar terms, although when you consider deals of unannounced size, we're probably close to the window of uncertainty on that. Some of that money has gone to fund acquisitions, and anybody who took a round of more than $20 million bears watching, but we're also still seeing funding for interesting and innovative companies in the space where social media and data analysis intersect.

Based on the last year's investment activity, look for continued product innovation and market evolution, in addition to ongoing consolidation.

So here's a summary: The opportunities in social media analysis are evolving, and heavy bags of money are being directed toward exploiting them. For the long version and its application to your situation, contact me about becoming a client.

Today's announcement that Twitter is buying Gnip raises big questions about the market for social media data. While it's too early to know how things will fall out, the deal changes the shape of the playing field for everyone involved—publishers, data resellers, software developers, and corporate customers.

Twitter has bought other companies in the social media analysis space—BackType (2011), Bluefin Labs (2013), Trendrr (2013)—but Gnip is a bigger deal. Gnip competes with other Twitter partners, and Twitter competes with other Gnip partners. If you weren't sure, things just got interesting.

As a reminder, here's my view of the social data ecosystem:

Anyone who works with data from social media sources has an interest in how the rest of the ecosystem reacts to the Gnip acquisition. Here's my initial take on what to watch for:

• Twitter competitors
  Twitter isn't the only data source for Gnip. Gnip's sources include full feeds from Tumblr, Foursquare, WordPress, and more. It also manages API access for Facebook, Google, and others that probably see Twitter as a competitor. How will these companies ("publishers" in the data market) react to the deal? Will access to data from Twitter competitors remain available through Gnip?

• Gnip competitors
  Twitter has offered its data through multiple data partners; how will DataSift, Dataminr, and NTT Data fit into the revised model? What impact will that have on their customers? (In a post, DataSift says its "relationship, contract and data resyndication partnership" are unchanged.)

• Other data providers
  There are other companies in the social data business, mainly those specializing in collecting data from blogs and forums. Will they add (or drop) services in response to the changing market?

I've asked Twitter for a comment, but I suspect we just have to wait for the answers.

Get the latest industry news at Social Media Analysis.

Social Media Analysis is my attempt at a sort of online industry trade journal covering the companies that work with social media data. Last year, I started a recap of the financial transactions in the business, so let's catch up with 2013.

2013 Saw More, Bigger Investments in Social Media Analysis
First, where the investment money went. And boy, did it go, more than $465 million. The champion fundraiser this year—by far—was HootSuite, with $165 million added to its runway.

The Year in M&A (and an IPO), Social Media Analysis 2013
Once all those companies are funded, some of them get acquired. One even went public. The big theme seems to be consolidation, as buyers picked up companies with complementary technology, products and people. At this rate, we should finish concentrating the industry by about 2080.

SMA would be better with more content, but I need help if it's going to get it. I have ideas for new sections, including opinion columns, product reviews, how-to articles and more. Anyone interested in becoming a contributor?

If you want to work with social media data, you first need some data. But "social media data" isn't a single thing, and sourcing it involves decisions about what you need and where you get it. Those decisions have technical, business, and even legal implications, which is why I've been working on a new research theme for Social Target: the social data ecosystem.

The project grew out of a whiteboard session with a client last year. I showed them how social media data—Twitter content in their example—is available from multiple sources, but your choice affects what you get and creates requirements for your systems to handle it.

The first draft
I've turned that original sketch into this map of the industry, which I'm treating as a hypothesis in the research phase. As I talk with companies in the various categories, I expect to validate the model and get a better understandinging of how the interfaces work.

This is what I mean by the social data ecosystem. It starts with the companies who collect data directly from their users, and it ends with the analyst or manager who is looking for information in social media. In the middle is where all the data changes hands and software turns it into something useful.

Exactly what happens in between is interesting and a bit complicated—but perhaps a bit less complicated once this project is complete.

What's your experience?
I'm interviewing companies throughout the ecosystem now. In addition to understanding the different business models in play, I'm also asking about current issues in the market. I'd like to know what's working, and what's not.

I'd like to hear from you, too. What's been your experience in working with social media data? Comment here or contact me privately, and let's find out together what's going on in this fast-evolving market.

Trust is an issue for an industry based on extracting meaning from what people share in social media. People don't have to use these services, and if they decide that their information might be used against them, they can stop. This week's revelations about the US intelligence agency monitoring social networks (among other sources) creates a massive trust issue for everyone who works with social media data. What now?

(This won't be an analysis of the NSA and Prism. I'm working from the same sources you are, and we'll probably have new information by the time I finish writing, anyway.)

The world reacts to US actions
David Meyer points out a threat to cloud computing vendors as customers and governments react to the news. US-based vendors can expect special challenges selling in Europe, where privacy is more protected and signs of a blowback from Prism are already appearing. In cloud computing, the trust issue relates to custody of the data—do you trust your vendor to keep your data safe and secure?—and the government version translates as a question of US-based vendors' ability to keep commitments to foreign governments.

But cloud computing is essentially just data center outsourcing. What does it mean for an industry that exists because of people's willingness to share publicly?

Access to the data is everything
The challenge to the social data industry is different. It's indirect, but potentially existential. What happens to your business as a result of the reaction to Prism? Will social networks tighten their terms of use to block data mining? Will EU safe harbor agreements create new requirements to protect user data (possibly by keeping it outside the US)? Will new legislation designed to limit government abuses include new limits on private-sector users?

Secret collection of private data by government agencies is fundamentally different from social media monitoring outside government. In business, we're working with publicly available data, which anyone can access without breaking the law or hacking a system. It's not espionage, but the facts aren't the problem.

The problem, as ever, is perception. The NSA is all over the news, and in the heated environment of a breaking story, subtle distinctions can get lost. The risk to the social data industry is that a reaction to government surveillance could become a problem for anyone doing the less intrusive type of monitoring.

How will you respond? What's your plan for minimizing the overreaction if it starts to get out of hand?

Responding as an industry
At its Big Boulder conference this week, Gnip announced the Big Boulder Initiative, which is an effort to start an industrywide discussion of the issues it faces. Trust is one of five issues they highlighted as starting points for discussion. The other news this week highlights the wisdom of the choice.

I'd go farther and ask a question I've asked before: should the companies who work with social data form an association to coordinate these discussions, codify standards, and speak for the group?

The ethics of social data
Let's go back to trust and consider the ethics of working with social data. Bob Gourley at CTOVision recently gave me a copy of Ethics of Big Data, a short e-book that lays out a process for establishing ethical limits to the use of big data. It's a worthy challenge, but I think the first step in the process—exploring an organization's values—will lose everyone. The Friedmanesque view that a business exists only to make a profit is common, which leaves only the law as a restraint on what can be done. "Be profitable" isn't the sort of value that will drive a hearty discussion of ethics.

I do think it's possible to have ethics of listening, but I don't see an existing standard that really applies. I don't see, for example, how ethical standards for social scientists, with their strict limits on personally identifiable information (PII) apply to social media monitoring in customer service. The standard for competitive intelligence boils down to "don't break the law," which appears to be the relevant limit on secret government programs, too.

Here's a starting point for discussion
I suggested a set of ethical standards for listening vendors in 2010 as a starting point, but the discussion went nowhere. Maybe it's time to try again. Comments are closed on the old post, but I'd welcome any discussion of the draft here.

The usual defense of social media monitoring in the private sector is that we're working with publicly available data, but monitoring public data can still be creepy. What's the plan for protecting public-source data mining from an overreaction to something far more invasive?

Photo by Marsmettnn Tallahassee.

Asking a computer to make sense of everyone's written opinions is a big challenge, but it's not the last one that social media will impose on anyone who wants to analyze it. We're sharing a lot of pictures in our virtual hangouts lately, which means it's time to update the old question. Instead of "what are people saying about us," the new question is something like, "what do people's pictures tell us about what they think of us and how they use our products?"

Just as the shared images give us access to new types of information about people, their tastes, and more, emerging technologies offer the promise of helping us understand the images at scale. To the vocabulary of text analytics or natural language processing, add computer vision. As with its text-processing cousin, it's not as evolved as your eyes, but it doesn't blink, and it doesn't sleep.

Looking at the photo directly
Let's say you want to track publicly shared photos that contain your company's logo. Without image analysis, monitoring depends on keywords in posts and photo descriptions, filenames, tags, and other metadata. It's better than nothing, but it has limitations. You're going to pick up images that don't actually include your logo, and you'll miss photos that include your logo but aren't about your logo.

If your tool can "see" product logos in photographs, you get access to a different type of information. You start to catch products and logos in the wild, where people really use them. The brand protection guys will like enhanced abilities to track counterfeits and parodies, but maybe this opens the door to a new kind of online ethnography, too.

Finding the technology
As demand picks up , you can expect the serious competitors in social media analysis to add image search capabilities. Already, Ninestars has added image recognition from a partner, and Meltwater's OculusAI acquisition suggests future capabilities with images. They won't be the last.

These companies are going at the image recognition challenge directly:

• Curalate

• Ditto

• gazeMetrix

• LTU Technologies

• TinEye
  

What's next?
Computer vision has lots of potential beyond spotting logos in photos. I imagine that this sort of product/logo identification will extend to video, though I'll need to talk to an expert to understand when to expect that.

And then there are people. We already have identity tagging in Facebook, and big money is going toward advancing facial recognition. I also found Real Eyes, a company that analyzes emotional responses from video, so visual analysis of faces isn't limited to identifying their owners.

The computers aren't just reading. They're starting to watch, too. Can you do something good with that?

This is one of those list posts that will grow as people point out more companies. Who'd I miss?

Quick, name a social media monitoring tool that can monitor Instagram. Got one yet? Not sure? I found four in seconds. Here's how.

I launched Social Media Analysis in 2009 to move industry news coverage from my personal blog to its own site. A little over a year ago, I added a free directory of social media analysis companies, which continues to grow as I discover more companies in the market. In yesterday's webinar on choosing social media monitoring tools, I realized that the news archive is the better tool for finding specific product capabilities.

SMA's directory has its own search feature, which knows a few tricks, such as finding companies based on a search for their old names. But if you're searching for a feature, the directory is only as good as the descriptions that the vendors have written about themselves (in this challenge, a directory search found one result). For something as specific as covering a particular network, it's not likely to be a big help.

If you write long enough, you build a history
The good news is that the main site also has a search feature and nearly four years' archive of industry news. The weekly roundups of product updates are particularly rich in keywords for the search engine to use. A quick search for "Instagram" revealed four monitoring tools that have announced Instagram coverage.

Even industry observers who make a point of keeping up with the tools market can't remember every detail of what 400+ companies are doing. Is SMA on your go-to list of resources for keeping up with the social media tools market?

Vendors, have you checked your company's entry recently? Is it complete and up to date? Does it contain the right keywords for searchers?

So much of the public discussion of social media monitoring and analysis focus on commercially available platforms that are more or less ready to go to work. Somebody has to set up the queries, and there may be some dashboard configuration, but the tools are generally pay-and-go propositions. A recent article points out that some companies are going for something much more customized to their needs.

In its big list issue on companies doing IT well, Information Week mentioned Toyota's new social media and CRM tool:

The tool took 60 hours to develop, largely using software Toyota already had. Oracle Endeca Discovery handles data discovery and search analytics, WiseWindow and DataSift aggregate social data, and Lexalytics analyzes sentiment.

Toyota is using the tool to improve customer service, product forecasting and quality, and lead generation. It plans to feed information to dealers in the future.

The opposite of off the rack is bespoke, which in this context means completely custom software development. Even using some of the open-source components that everyone uses as important building blocks, it's a lot of work. Unless you're in the software business (or might enter it), I'm not sure why you would bother. It's hard to think of even narrowly specialized applications for social media analysis that don't have someone trying to address the market.

Custom is expensive, which is why the volume market goes for off-the-shelf products. But given a hint of what one company is doing to tailor a system to fit their needs, I want to know more. Do you know other examples that people can talk about?

Photo by Douglas LeMoine.