Recently in Tools Category

Text Analytics in the Cloud

| 2 Comments

politicsbig.jpgIf you're a software developer (I'm not), you might find it easy to throw together a new application. You can handle pulling in data, moving it around, and displaying results—the normal software stuff. If you're analyzing social media content, though, the core functions of text analytics might be something you'd rather not learn. There's an API for that.

These services—some free, most commercial—allow you to skip the R&D and plug text analytics into your system. In a market crowded with me-too platforms, it might be the step that gets you off the entry-level rung of the ladder.

There, that should be enough for another 300 social media analysis startups. :-)

More posts in the "Build or Buy?" series:

Visualization by Thomas Jenkins.

Analytics as a service: AaaS? Wouldn't want to hazard a pronunciation on that one.

Ugh. Yahoo is killing off Delicious, the social bookmarking service that I've used since 2005. This is inconvenient. I don't need to spend my time rebuilding stuff that already works.

First priority is saving all the bookmarks. Frank Gruber posted a list of backup options to cover the basics. At least I don't have to worry about losing the bookmarks entirely. The challenge here is to replace what Delicious does in my publishing workflow. Just saving links doesn't meet the requirement.

I don't have the answer for that, yet. So far, I'm collecting names of services that might work. Here's what I've seen mentioned:

Any other suggestions? I use Delicious to support dynamic content on multiple websites. Replacing it kind of matters to me.

I don't suppose anyone thought of offering Delicious Pro accounts. I do pay for some of the other services I'm using. Would revenue make a difference to Yahoo?

Update 12/17: Now Yahoo says they're selling Delicious, not shutting it down. That would be better.

Note to self: Figure out which other Yahoo properties I use. Migrate. Don't look back.

img_data.jpgToo much information. And increasingly, too many disparate sources of data, many with their own analytical tools. So it's interesting to see a new crop of startups offering tools that pull analytics data from multiple sources into a single dashboard for analysis and reporting.

This is one of those posts that started as a more detailed look at a few tools, but as I was looking around, more platforms kept popping up on the radar. So it's become a list, which is probably just as well. Some of these guys are semi-hidden in beta testing, so any detailed description is going to be out of date soon, anyway.

If you're spending too much time trying to corral performance data from multiple online sources, try these on for size:

(Also available as a Twitter list.)

Analytics mashup
What these platforms have in common is the ability to create charts and dashboards that combine data from web analytics and social media sources (Leftronic is different, because of its emphasis on large-screen dashboards for public view). So if you want to see the correlation of Twitter followers and website visitors, you can. If you want to track multiple accounts on one dashboard, you can. If you want to stir in data from your internal databases, some of them will let you do that, too.

What if
Remember the RSS tricks post from a couple of years ago, how you can assemble useful applications by using RSS inputs and outputs as a pipeline between services? With so many APIs going in and out these days, one of these dashboards could be the user interface for some interesting manipulations. For example:

What if you were to combine online sources (social media) and internal company data, run them through some text analytics, and pipe selected metrics out to one of these dashboards to mash them up with web analytics (which you've already linked to business performance). Would you find the elusive social KPI you've been looking for?

It's a list. I've missed somebody. Tell me, and I'll add them.

Let's say you're ready to get serious about responding to customer needs expressed in social media. You're answering questions and responding to complaints, and you want to move that capability into full production. Here's a question for you: what software platform do you want to build the operation on, one built for social media or one built for managing customer interactions?

When building a customer service capability for social media, you can start with a social media analysis platform or a customer relationships management platform. Some of the SMA platforms have CRM-like workflow features, and some CRM platforms are getting social media features.

This is the Reese's Cup Question. As social media become more important as a customer service touchpoint, how much social media do you want in your CRM, and how much CRM do you want in your social media?

From the vendor side, it's easy. You start where you are. For the buyer, it's trickier. Overlapping capabilities raise questions of purchasing rationalization and integration of both systems and processes. I don't think we've reached the major decision at most companies, but it's coming.

As companies scale up their commitment to provide customer service through social media, what do you expect? More chocolate in the peanut butter, or more peanut butter on the chocolate?

Related: Social Media for CRM or Workflow for SMA?

I'll take Outdated Cultural References for 400, please, Alex.

The entanglement of social media analysis (SMA) and customer relationship management (CRM) is moving right along. It was inevitable, really, as companies realized the need to interact with customers through social media. The interesting question becomes, will they want engagement features in an SMA platform or monitoring and analytics in a CRM platform?

Adding process to SMA
From an SMA perspective, process management is an obvious addition to social media monitoring capabilities—once you find a customer with a problem, you want to fix the problem. Scale that beyond a handful of customers, and you need a system that can track your progress. Pieces to consider:

  1. Discovery
    Finding relevant mentions of a company is a fundamental building block of SMA.

  2. Tagging
    While we're in an analytics platform, let's assign some metadata to the item for future analysis.

  3. Triage
    Decide which items require a response and prioritize. You might filter on sentiment, topic, or influence. Some items might receive an automated response at this stage.

  4. Assignment, delegation and reassignment
    Who owns the response? It's not just for accountability, ownership relieves others of spending time on an item. The system should be able to handle ownership changes to support escalation and assignment to specialists.

  5. Track to closure
    Classic customer service management—what's the status of an open item? Open-item analysis gives management a tool to manage workload and understand current incidents (the metadata from step 2 will be useful).

  6. Measure results
    Now, we're back in the analytics realm and can look at the results through customer-service and media-analysis lenses.
From a client perspective, the question becomes, which platform do you use as a base—SMA, CRM, or something else?

The workflow features in some SMA platforms support this type of process, which can also be used in media relations or other contexts (action items aren't just for customer service). Other vendors provide tagging that can be kludged into workflow features. So you could choose to build your processes on an SMA platform.

Adding social media to CRM
The heavyweights of CRM are beginning to add social media features to their products, creating new options. Their existing installations will help them, especially in more conservative customers.

You got peanut butter on my chocolate!
Larry Dignan says of SAP's Twitter demo,

...it does show an increasing amount of integration with social networking tools. If corporate data is merged with the anecdotal tips from customers and partners there could be real insight.

This Holy Grail of insight is what a lot of vendors–Salesforce, Oracle and SAP–are chasing.

A few SMA companies have been talking about integration with other enterprise systems for a while. It's time to think seriously about how things fit together when established enterprise software companies start adding features that are the core of more specialized systems.

One of the more interesting sessions at BarCamp Charlotte was on using social media for social change. We didn't make much progress for the non-profits in attendance; mostly we learned that they need to connect with people who would like to help them. The session did, however, prime me to notice when two different programs focused on truly global issues wandered across my awareness the same day. What started as a discussion about building word of mouth for a fundraiser shifted to something much more ambitious.

zyOzy
I learned about zyOzy (zee-Oh-zee) when @zyOzyfounder followed me on Twitter. For me, at least, that still gets some attention. zyOzy applies a mix of events, social media and entrepreneurship to support efforts to end extreme poverty in Africa and India.

In addition to their blog, the site links to an extended online presence that includes Squidoo, MySpace, Facebook, YouTube, Flickr, Twitter, and a wiki. Who says you need a budget for an integrated media campaign?

Ushahidi
An old friend who now works in the NGO world pointed me toward Ushahidi, a platform for crowdsourcing crisis information. Ushahidi's original project compiled and mapped incident reports in Kenya during its 2008 post-election crisis. Reports were collected from citizen reporters using mobile phones.

The underlying technology is now being developed into an open-source platform that will be available for public and private monitoring of active situations anywhere. While in private beta, Ushahidi is being used for current projects focusing on Gaza, Congo, and South Africa, as well as a follow-up Kenya project.

What can you do with almost no budget?
"Social media are free" is the first myth to be busted—especially in a corporate marketing context—but most of the costs are driven by the need to spend time building social media programs. Free social media tools and open-source platforms, such as Ushahidi or the BuzzMonitor, put a lot of capability in the hands of NGOs that are more likely to have time and volunteers than a big budget.

Monitor Web Pages with RSS

| 1 Comment

I reprised the RSS tricks session at BarCamp Charlotte on Saturday, where a popular trick was generating RSS feeds for sites that don't provide them. I use this to keep up with companies who don't have feeds for their press releases; it's also good for finding out about the public launch of stealth-mode companies.

Here are the companies I've found who create RSS feeds from web pages.

If you know of others, mention them in the comments, and I'll add them to the list.

Dapper can create feeds from sites without existing feeds, too, but it's more complicated than the change alerts that other services generate. Mozenda goes farther into data scraping with its commercial service, but it's overkill for basic site monitoring.

A site-provided feed is better than this scraping approach—especially when it comes to sites with rotating content (such as client quotes). But when there's no feed and you really want to know when new content is added, these tools are helpful for pulling data into an RSS-centric information environment.

A few recent projects have reminded me how much value I get from Delicious, a useful service with a funny name (though not as funny as the original del.icio.us). It's a prime example of a social computing tool whose value isn't immediately obvious to newcomers, who probably get lost as soon as it's described as (take your pick) a social tagging, sharing, or bookmarking site. Delicious is increasingly important in my own work, so I decided to share some specifics on using it to find, save and publish information.

  1. Save bookmarks for my own use.
    Let's start with the obvious. Tagging is more effective than filing bookmarks, or at least it's a lot easier to find them later. Remember to back them up to your computer occasionally, though. Free services have been known to shut down.

  2. Autopost bookmarks to my blog.
    Those "links for 1/25/09" posts come from Delicious, which automatically creates a post based on each day's tags. Lists of links with no comments aren't too useful, so I make a point of adding original commentary to almost every item (a trick I learned from the bloggers of RedMonk—thanks, guys). That original content also makes it possible to...

  3. Use Delicious feeds to build dynamic web sites.
    This is one of my favorite RSS tricks: use Feed Informer to place feeds from Delicious on web sites (look for the link—virtually every page on Delicious has a feed). The commentary is already there for the link posts on the blog, so each link has original content to go with it. Delicious offers linkrolls that do the same thing, but Feed Informer offers more editorial control and the ability to manage feed content. The real secret here is that every tag in your account has its own RSS feed, so you're not limited to the everything-you-tag feed.

    Tag feeds let me send different content to multiple sites with just one workflow. So, for example, all of my tagged items show up on my vanity site, while subject-specific tags cause items to appear at Social Media Analysis, Managing Social Media or International Affairs. There's no extra work associated with supporting multiple sites—just add the right tag.

    If I tag something I don't want to publish, I just check the "Do Not Share" box to keep it private.

  4. Subscribe to Delicious feeds from interesting people.
    I subscribe to the feeds of a few people who pay attention to subjects I care about. When they tag something, there's a good chance it will be useful. If you find yourself following multiple people this way, add them to your network on Delicious and subscribe to that feed. Subscribing to individual user feeds doesn't alert your source that you're following them like the network feature does.

    If reading other people's tag list seems like an intrusion, remember that they have the Do Not Share box, too. Delicious users have complete control over how much they share. And if you're tagging items yourself, remember that your shared items really are public.

  5. Subscribe to tag feeds to find new sources.
    This little tip potentially puts every Delicious user to work for you: subscribe to the feed for tags in your field (for example, web analytics). Delicious suggests related tags when you search, so it's easy to find likely keywords. Now, when anyone on Delicious finds an item in your area of interest, you'll find out about it. Sometimes, this leads to the discovery of interesting people, too.

    I have all of Delicious finding things for me. How big is your research staff?

That's how I use it. How's it working for you?

What do you want that you can't have today? Specifically, what would you like to see in a social media analysis tool or service that you can't get (as far as you know)? Most of the vendors are probably reading this blog, so here's a chance to air your wish list while I get out of the way. I'll start with one that Lee Odden mentioned on Twitter yesterday.

SMA for SEM
Lee wants an automated keyword analysis tool for social media sites (process described by Marty Weintraub):

Really need to the social media monitoring industry to step up and come up with a good keyword analysis tool specific to social media sites
@leeodden
Lee heard from three companies in response (ok, the rest of you need to monitor social media keywords on Twitter). Is anyone else working on that application or want to talk about it (publicly or privately)?

What else do you need in 2009?
Open mic time. What's on your wish list? Mention it in the comments, and I can guarantee that companies in a position to offer it will see your suggestions. You might even hear from someone who already does it.

Following the announcement of the Lexalytics merger, I wrote that the availability of off-the-shelf text analytics and custom software development suggested some questions about technology strategy for social media analysis vendors. The main question is an old one in software: Build or buy? The follow-up flows from a value chain analysis, and it sets up a series of decision points for the build/buy decision. It turns out that a social media analysis platform is built from a few, basic building blocks, and a decision to build a custom system doesn't imply a decision to build every part.

Three basic building blocks
The technology behind monitoring and analyzing social media content fits into three basic categories: content sourcing, analytics, and the software application that provides end-user features, such as dashboards, reports, and alerts. Many service providers add value beyond what is baked into their software, of course, but these are the basic technology building blocks. What makes them interesting is that each generates a potential product that can be offered as the "buy" option to other companies.

  • Content sourcing
    The first step in analyzing social media content, sourcing includes source discovery, content aggregation, filtering and metadata tagging. The result is a feed of relevant content data, with duplicates, spam and other noise removed. Feeds of traditional media sources have been available for years; syndicated content sourcing is available for social media types now. (See a list of social media aggregators)

  • Analytics
    The next step is analysis of the content, also known as text analytics. The system extracts key words, concepts and sentiment from individual messages; depending on the vendor, it may also evaluate messages and sources for influence, demographics, location, audience size, and more. This is the hard part, which inspires the question of whether computer analysis is good enough.

    Perhaps assuming that text analytics must be homegrown, many vendors skip the analytics entirely, using human analysts or leaving metrics out of their product. But text analytics has its own build or buy decision. The technology is available from software companies in the business intelligence market, and at least two companies focusing on social media, Lexalytics and Leximancer. (See a list of text analytics options)

  • Application
    Content sourcing and analytics constitute checklist features in a social media analysis platform: "Sentiment? Check!" They represent a lot of work, but it's hard to tell whose works better. The application software, on the other hand, is what clients see, and it provides many opportunities for evaluation. Is the user interface dated and clunky or sleek and current? How is content presented? What can users do with it? How easy is it to explore the data or create reports?

    The once clear line between dashboards and social media analysis platforms for workgroups has blurred over the past year. Outside sofware development services will tend to increase the capabilities of dashboards from less traditionally technical vendors.

Room for differention
Syndicated and licensed components don't lead directly to commmoditization, but the threat is there for vendors who fall behind in capabilities. There's still room for differentiation, especially in application features, not to mention the research and consulting services many vendors offer in addition to their systems. But separating the build or buy decision into at least three discreet decisions does have effects:
  • Lower barriers to entry. New entrants—startups or established players in adjacent markets—can launch social media analysis tools without building everything from the ground up.

  • Higher baseline expectations. Dashboards from companies without strong technology traditions will soon include advanced technologies. Basic monitoring dashboards are becoming commodities (witness the $5/month entry pricing).

  • Fewer reinvented wheels. Vendors can compete on their strengths in content sourcing, analytics or application development. External sources remove the need to develop their own capabilities in areas outside of their core strengths.
This should be good news for customers: the tools continue to improve, and vendors will compete on what really adds value. For vendors, the question is back to basics: which parts of your system are really special, and which might be better outsourced? It's a question to reconsider from time to time as the market for building blocks matures.

More posts in the "Build or Buy?" series:

About Nathan Gilliatt

Subscribe

  • Subscribe by email


Recent Comments

  • Nathan Gilliatt: Good point. If you're buying a model (and with influence, read more
  • Tonia Ries: Great post, Nathan. Another missing element (aside from @theresa's hypothesis read more
  • Nathan Gilliatt: Thanks, Theresa. I wasn't entirely sure I was being clear, read more
  • Theresa Doyon: Excellent post! The missing ingredient is theory and hypothesis testing. read more
  • Nathan Gilliatt: Anne, I think the key takeaway is to step back read more
  • Larry Levy: Nathan, Thanks for including us on your list. Appinions is read more
  • tom.obrien@nmincite.com: Hi Nathan: Yes, Duncan Watts fairly demolishes most established "influencer" read more
  • Anne Weiskopf: Great article Nathan - thanks for the posting. It will read more
  • Nathan Gilliatt: Thanks, Lisa. I actually think Klout's interesting, but it's more read more
  • Lisa Thorell: Fab piece. What i like is, whether intentional or not, read more

New on SMA