Social Media Monitoring Tools In Action: Radian6 vs Alterian / Techrigy SM2

Not too long ago I followed a nice discussion on Ken Burbary’s blog about social media monitoring tools. Ken had come along some noticeable inconsistencies in the results reported by two leading social media platforms, namely Radian6 and Alterian/Techrigy SM2. Since we have been working here at Covisio with both of these tools for a while now, I agreed to post a few numbers based on our own experience from a real-world listening campaign that we are currently running for one of our clients in the enterprise software space.

Before moving on, a couple of disclaimers:

• My purpose is not to rate one tool vis-a-vis the other. As a matter of fact we have used both tools successfully to help clients with their marketing plans and that’s what really matters at the end. For this analysis here, we used the standard Radian6 version with one (paid) user account login to have access to the system (but no paid topic profile) and the Freemium version of SM2 – a setup similar to what a user evaluating both platforms might come up with (though both Radian6 and SM2 do offer 7-day trial versions of their full products).
• The results are from just a single project; they by no means constitute an all-inclusive comparative analysis of the two platforms. By the way, here’s a pretty detailed post by Jason Falls actually comparing the use of Radian6 to ScoutLabs, another social media monitoring tool.
• I understand there are differences between the two technologies in terms of what sources they use to pull data from and what internal technologies they use to classify the data and produce their reports. So I am aware that some variation in the reported results is to be expected. But how much is too much?

Now, we monitored data from 4 searches across all media types and for a period of 30 days:

• Search #1: company name.
• Search #2: industry term.
• Search #3: industry term.
• Search #4: product acronym (with a set of keywords to exclude as well as a set of source urls to filter out).

So, let’s look at what the tools came back with.

The first overview graph shows significant variations in the number of results returned from each tool with Radian6 consistently returning more items than SM2 (from 50% more to 450% more). The overlap (defined as per search, the # of same results divided by the # of total results from the tool that produced the lesser # of results) in the results of the two tools ranges anywhere between 30% (for a product acronym search with lots of noise) to 100% (for a straightforward company name search).

The second and third drill-down graphs show serious inconsistencies as well. Radian6 indicates that the predominant media type is blogs, while SM2 indicates that the predominant media types are either Twitter or “other”.

The bottom-line is that there are both quantitative and qualitative variations in the results reported by Radian6 and Alterian/Techrigy SM2 for the same search profiles. And there is limited overlap in the reported results, which seems to indicate that the search spaces covered by the two tools are very dissimilar. This to me is understandable, but I would expect to be able to adapt the tools (one way being by configuring their url filters, i.e. what urls to include and what to exclude) so that they start converging at least to identifying and tracking the most relevant conversations within an industry. At the end, although it is clear that social media technologies are still evolving, it is also clear that social media monitoring tools can be made extremely useful to (marketing, branding, customer service, R&D, legal, etc.) organizations, but not without a well-thought-of human investment to understand the results, interpret the findings, and take proper actions.

P.S. I am planning a follow-up post to discuss some of the things we learned when evaluating these social media monitoring tools and planning social media listening campaigns.

UPDATED (9/22/09): For a more complete analysis on the above social media monitoring results from Radian6 and Alterian / Techrigy SM2 towards the quality vs. the quantity of the results, here’s another graph on the relevancy of the results reported from each tool.

The graph shows that both tools deliver over 85% of relevant results for normal searches, but noise margins rise significantly for more involved searches (for example in search #4 the main keyword is a software product acronym that is our interest, but the keyword also matches names of religious groups, scientific techniques, and schools; despite the fact that the search has been configured in both tools to exclude results that contain certain keywords or are sourced from certain urls, the noise margin is still very high). More involved searches obviously require better optimized and more regularly updated search configurations to filter out irrelevant items based on context and source in order to maintain acceptable relevancy in the tools’ reported results.

Technorati Tags: Alterian, listening campaign, Radian6, Social Media Monitoring, social media tools, Techrigy SM2

9 Responses to “Social Media Monitoring Tools In Action: Radian6 vs Alterian / Techrigy SM2”

Leave a Reply

Name (required)

Mail (will not be published) (required)

Website

CAPTCHA Code