Skip to content →

Social Media Analytics Demystified: Are You Looking At The Big Picture?

Social Media Analytics DemystifiedAs with most, if not all, web marketing programs, people and businesses want to know whether something is succeeding and returning the right values and numbers to justify any attempt at marketing to their customers. In the world of social media, this is no different. For companies interested in executing social media campaigns, finding the right social media analytics tool is sometimes hard. Unfortunately, existing web analytics tools like Google Analytics or WebTrends do not support or have the capability to monitor the chatter happening in the interactive web. Besides, social media analytic metrics are different from what you would typically expect from a web analytics product – instead of hits and time spent on site, it’s more concerned about the number of blog posts, tweets, share of voice, etc.

Wait…what are we talking about? What is social media analytics?

Social Marketing Analytics is the discipline that helps companies measure, assess and explain the performance of social media initiatives in the context of specific business objectives.
as defined by Jeremiah Owyang & John Lovett

So what makes finding a social media analytics tool so difficult? There are a flurry of them out there on the market so what advice should you be given in order to find the right tool? Well one bad advice that you could get is to pick the one that is the cheapest in price – why? Because you’ll never really be happy with the metrics since it doesn’t meet your need, aside from the low price. Don’t just look at your budget for using a social media analytics tool. Instead, take these steps offered by the Altimeter Group and Web Analytics Demystified in their latest report: “Social Media Analytics:  A New Framework for Measuring Results in Social Media

To help make this an accurate assessment on the industry as a whole, the authors of this report, Jeremiah Owyang (Partner, Altimeter Group) and John Lovett (Senior Partner, Web Analytics Demystified), solicited input from dozens of companies and individuals in the ecosystem, including folks like Connie Benson, David Berkowitz, Shiv Singh, Blake Cahill, and from companies like Biz360, Omniture, Radian6, ScoutLabs, SWIX and many more. During the course of their research, Owyang and Lovett discovered four main things:

  • Many companies are “stumbling blindly” into social media marketing, largely without measurements in place.
  • A pragmatic approach using metrics derived from sound business objectives minimizes confusion about the value of various social efforts.
  • Existing social marketing measures and metrics fail to deliver actionable insights and offer little more than digital trivia.
  • Technologies exist to facilitate data collection in diverse social media, but there are no silver bullets.

IS IT THE BLIND LEADING THE BLIND?

It seems to me that the first point is a powerful statement. The fact that companies are embarking on a social media campaign hoping that their Twitter program or YouTube videos will catch on like fire, is definitely troublesome. For those who are interested in releasing some news to the press and bloggers, simply sending the news out without understanding how you’re going to quantify the results to your supervisors or to your client is absolutely ludicrous. Like in carpentry, the adage is “measure twice, cut once”. In social media, the point here is to measure twice…before your campaign and then after your campaign to make sure that your efforts not only made a difference, but how much of a difference. Did you change anyone’s sentiment about your brand and/or product? What are people’s reaction to what you’re doing with your new campaign? Is there more “buzz” surrounding your promotion? What is the ever-pervasive Return on Investment that you need to make and report back to the stakeholders? These are some questions that social media analytics can provide you with & that’s why it’s daunting to hear Owyang & Lovett report that companies are “stumbling blindly”. Just when will they get it right?

MARKETERS ALWAYS WANT THE PERFECT SOLUTION – IT DOESN’T EXIST.

Owyang & Lovett report there’s currently no single vendor that can “effectively measure all aspects of social media.” I think while this is true, another point to tack onto this is that there’s no single vendor out there that will provide YOU, the customer, with the measurements that you’ll want exactly in order to make sure you can provide the right results to your company. What we would be talking about is having these established names like Radian6, Omniture, Biz360, etc going out there and creating customized modules. While companies and brands might like that, what they probably would not like is the cost associated with this & these social media analytic companies may disapprove of this process because it requires more work on their end, plus the cost-benefit ratio may not be to their liking. So unless you have a lot more budget and can afford a customized build, similar to what SAS is offering with their enterprise module, you’re going to have to do some sort of work-around.

In fact, the report is quick to point out that “the reality is that businesses turn to multiple solutions for capturing, analyzing and interpreting their social media activities. Most use an amalgamation of commercial solutions geared for capturing social buzz, free tools offering limited information and a whole lot of manual intervention for aggregating and analyzing social media data. Don’t expect this to change in the near-term.

SOCIAL MEDIA NEEDS TO CREATE STANDARDIZED MEASUREMENTS.

In the heart of the Social Media Analytics report, Owyang & Lovett talk about helping companies establish standardized measures within social media to help make sure it’s consistent to show how effective a campaign really is. According to them, by following these measures, companies will have greater gains through their social media initiatives & develop stronger and more effective programs for their customers. The problem social media analytics is facing is commonality and consistency. It’s not readily apparent unless you look at multiple vendor services, but like web analytics tools, things are calculated differently because everyone has their own methodology. Sure, the general premise and idea is the same with everyone, but the way you actually calculate an impression or how influential someone is on Twitter and display it on your social media analytics dashboard can vary from service to service. To show you how different calculations can be across an industry, read this blog post I authored while at Stage Two on Twitter tools that determined a person’s “influence”.

  • Step 1 – Revisit Tradition for Solid Innovation…merely collecting data without a reason for “why” is a disaster waiting to happen, says Owyang & Lovett. Organizations that can align their social media campaigns and strategies (including key social media metrics) with their business objectives will wind up being much better off. If you don’t embrace new media, then you’re going to risk becoming extinct. But before you jump into social media, make sure that you don’t dishonor and ignore the traditional business rules. These rules must not be tossed out, but integrated with your web plans.
  • Step 2 – Make Learning Your Primary Goal…there’s a whole lot of talk about selling and listening in social media. With social media analytics, learn to…well…learn. Owyang & Lovett believe that social media analytics will provide a “scalable vantage point” to understand consumer behaviors, test new strategies/initiatives, and improve the effectiveness of your social marketing activities. Don’t think that just because you have results of a campaign that it’s over. Take the numbers and information garnered and learn from it. What insights can you garner from it to improve your efforts?
  • Step 3 – Define Requirements First, Then Select Vendors…probably a very good step. Don’t pick a vendor because you like the name, know someone that works there, or that they have a very appealing price point: low. Instead, look at why you need social media analytics and what measurements and features are important to YOU. Once you figure that out, research the vendors and find out what their product is all about. Don’t be afraid to look at a demo and ask their salesperson questions that matter to YOU – not what matters to the industry. Ask for a free trial period so you can make sure you’re happy with the service before committing any dollars. Perhaps most important, find out whether the service vendor has the technology to work with your goals and requirements. Said best: Don’t fall victim to vendors that require you to conform to their capabilities, but rather, work with those that offer flexibility and customization opportunities.
  • Step 4 – Develop Your Social Media Measurement Playbook…this is similar to your tactics. After figuring out the strategy, you’re going to need to determine how you’re going to go about to execute. Owyang & Lovett suggest that you first align your organization with the goals, objectives expectations and actions of the social media marketing efforts. Once done, map that out with the measurement technologies at your disposal. These actions will help make sure everyone is on the same page.
  • Step 5 – Make the Altimeter Group’s Framework Your Own…in the Social Media Analysis report, Owyang & Lovett take the time to develop a framework towards social media analytics that I’ll look at a bit further in my next post, but while it’s Altimeter’s work, it is suggested and recommended that you adopt certain parts piecemeal to your own organization and create your own custom framework. Not all the pieces will work for all companies in all aspects. Companies are going to need to develop specific aspects of their framework that aligns with their own objectives.

So now you have a general idea about what’s the premise in social media analytics. There are some serious things you need to consider when choosing to engage in social media analytics. Not all services are the same calculations and thorough planning is involved before you simply throw out a social media campaign. Are these efforts in concert with your business objectives?

In my next post, I’ll look through the Social Media Analytics report and review their framework, key performance indicators and how it matters for your company.

You can read more about the Social Media Analytics report on Jeremiah Owyang’s blog here or view the entire report on Slideshare here.

Published in Featured Industry Analysis Social Media Web Analytics

  • ben

    I use clicktail, it helps me know what is happening on my site and its heatmaps and videos show me i can increase my conversions

  • KDPaine

    Why is no one talking about is the accuracy and reliability of the data provided.Everyone focuses on the bells and whistles, but I heard one vendor tell my client that they could guarantee that they were 65% accurate. Is that really okay? In my world, (okay I live and breathe research for a living) anything under 88% isn't acceptable. But somehow everyone seems to trust the data despite the evidence that it's not reliable. Sure there's a trade off between cost and accuracy — it costs more to do things right, what astounds me is the companies that are spending a fortune and getting unusable or unreliable data.

  • KD – Thanks for the comment.

    RE: accuracy and reliability of the data, I think people are just not educated in how to tell or ask the questions. In fact, when picking out a vendor, how often do people really ask “So what makes your service really good and HOW ACCURATE is your data?”. The salespeople would probably then make a *great* excuse and say it's pretty accurate.

    Now you bring up the accuracy and reliability issue, I wonder how this affects web analytics as well (e.g. Google Analytics, WebTrends, etc.). How can we as marketers and users of these services figure out how good of data we're getting.

    And I agree with you that there is definitely a trade off between cost and accuracy – people don't want to pay more for a service that will generate the information they want…unless they have the thousands of dollars per month to spend on assuring that the data is somewhat more reliable and significant.

    So what are some tips/pointers people need to be aware of when choosing a social media analytics tool in order to find out how accurate the data is? Are there some tell-tale signs?

  • KDPaine

    In content analysis the bare minimum level of acceptable reliability is 88% — that means that if two or more people read the same 100 blog posts they should agree on its tonality (and any other parameters) 88% of the time . You get to that level of agreement by having really good training, and really good coding instructions. Automated sentiment can do the same thing (or at least SAS's Social Media Analytics can) they train the system, then they compare it to humans, and when humans and machine agree 90% of the time, they declare it ready to go.
    Anyone can do the same thing. Read 100 blog postings tweets, etc. that have been automatically coded by Radian 6 or whomever you're using. See how often you agree with the way its coded. If you disagree on more than 12 of them, it's not accurate.
    The other test you should always do is compare what your service gets on a daily basis to what Google and Twitter search returns. Google and Twitter will always get at least 10% more. If it's more than 20% more, you're not getting the full picture.

  • Ken – great post. The Social Media Monitoring space has grown significantly over the last few years, as has social media as a whole. The points you bring up about the blind leading the blind, as well as your step by step guide to building an analytics framework that fits your business are two key points that help illustrate where we are on this cycle. KD's points about data reliability is a key point to bring up, as this relates to many topics inside the analytics space (from which data you pull in to how you analyze that data for sentiment, workflow, etc).

    Looking forward to the next post in the series :).

    Katie
    Community Manager | Radian6
    @misskatiemo

  • Hi Katie –

    Thanks for the comment and glad you and Radian6 is monitoring what's being said in your industry. I agree that KD's comments RE: data accuracy is very important. How does Radian6 address this concern? I've sat in on a demo once with several analytics companies like Radian6 and, probably like a social media newbie or someone who expects the functionality of the service to do what I want (along with some cool bells & whistles), wouldn't pose the question KD has asked – but what IF the question was posed…while Radian6 is probably robust enough to handle any work that a company might need…how do you stack up in terms of accuracy?

    Is this accuracy measurement something that Radian6 currently or should leverage over their competitors?

    My next post on this series is up: http://blog.thelettertwo.com/2010/04/24/social-

  • Michael Fraietta

    Ken,

    Interesting post. I work for and use Jive's Software's monitoring platform, Filtrbox. I like the fact that you bring up defining your measurement requirements first and not bend over to the what the company offers. I use our analytical charts daily and love them, but there are some things missing that would be helpful outside our company (geographical and language breakdown, for example). As vendors, I think it is important that we are aware of our weaknesses and recommend the appropriate competitor if we don't offer what their campaign/company seeks. For example, if you're looking for automated sentiment , I'd recommend talking to @ConnieBensen at Alterian SM2 (I'm not a fan of automated sentiment. Sorry Connie! 🙂 ). If you need to measure results in Mandarin, Russian and French, talk to @michmski (Michelle Chmielewski) at Synthesio. I was about to suggest that it would be nice to see a non-bias chart of the strengths and weaknesses or the vendors, but I'm now reading your follow-up post that covers such.

    Thanks for including us in your vendors to watch list.

    Like Sean Carter, I'm on to the next one….

    Mike Fraietta I Director of Social Media I Jive Software I @MikeFraietta

  • Hi Michael –

    Thank you for your comments and I appreciate your feedback. It's very interesting that you suggest that it's important that you're aware of your weaknesses as a vendor and that you recommend competitors to the client if you can't meet their needs. I think that is a very much how the culture is at Zappos.com. I recall their culture being that if they can't find what meets your needs as a customer, they'd research the product for their nearest competitors and give them the sale without blinking an eye..excellent customer service – shows that you're more interested in the relationship and not the sale. If this is what you're doing at Jive Software/Filtrbox, then I commend you for that…people should start viewing you at an informative person as it relates to analytics.

    As a vendor, though…how are you reaching out to current and potential customers to help them plan accordingly for their needs? If you're all about the customer service, as stated earlier, then what steps are you taking to educate them? And are your clients mostly larger, established brands or are they small business owners? How would you tailor this advice to them?

  • Michael Fraietta

    Thanks for the reply Ken. Our focus has shifted greatly since the acquisition of Filtrbox by Jive from agencies to established brands. I'm great at explaining how to measure B2B companies the size of the former Filtrbox and now Jive because of my direct experience doing so. But when it comes to others (the vast majority), I have to do my research, and often that means acting as the “Chief Listener” of a brand for a few weeks to see what they would have to go through. For example, tracking online mentions of Nike (the first big brand I could think of and already was a Jive client), I found it difficult measure sentiment of the overall brand and it appeared as if it would be extremely difficult to respond each post without a large team. However, when tracking an individual shoe (i.e. LunarGlide) it was relatively simple to measure the public's overall opinion of see what words (lightweight, stability) they associated with the product. If their goal was for LunarGlide to be known as “lightweight” and over X% mentioned “lightweight” when talking about that shoe, they could see if their benchmark was hit. We obviously can not do this for every company, but we can dip our toes in different industries and use case studies from our clients.

    Jive is well experienced in the big brand business as the Filtrbox crew is with monitoring social media. However, since we Jive is new to monitoring and Filtrbox is new to fortune 500 brands, we all have a lot to learn. I think the marriage will work well as Jive is looking at monitoring as a piece of the puzzle (like micro-blogging, instant messaging, ideation, etc.) for the internal community, not a separate platform.

    Mike Fraietta I Director of Social Media I Jive Software I @MikeFraietta I 215.596.9009

  • Hey there, Ken. A few notes about data accuracy…

    First – the tool you choose is a launch pad. Yes, we do cover a lot. However, we're always happy to add a source which we don't cover “out of the gate” if your business requires it. We are also constantly adding sources that we go out in search of, on top of that.

    You mentioned sitting in on a demo. I'm happy to talk to a company about what we do (and don't) cover. I, in fact, did this tonight for a customer at a tweetup, and it helped them better understand how their PR agency and their own measurement practices tied in to what our tool provides.

    Furthermore, we're happy to look over things such as niche blog, forum, or network sites (to name a few), especially for industries that may not be mainstream, but nevertheless have a large presence online.

    It's really a two-fold concept. One – make sure you go out there and cover things you should be covering. Two – provide some flexibility for the cases where people really need you to cover sites that aren't in your current scope (this comes up most often in niche industires, but isn't limited to them in any way).

    You're correct in the fact that a person who is new to social media monitoring wouldn't necessarily ask the question about “what do you cover”. That's partially why software trials are great tools, as companies can set up identical (or as close to identical as possible) searches and compare results, as well as really get a feel for what using the tool would be like on a daily basis.

    I'm getting off on a tangent now :).

    Best,

    Katie
    Community Manager | Radian6
    @misskatiemo

    P.S. – Of course we're listening! 🙂

  • Hi Katie,

    Thanks for your reply…appreciate the conversation – totally think it's worth the long comments. 🙂

    Agreed that the social media analytics tool is a launching pad and that's really great that Radian6 is being a listener and adapter as well. But I refer you to KD Paine's comment where she talks about accuracy and reliability of the data to make the customer feel better about the numbers that are being represented…in my opinion with WEB analytics (not social media), the web traffic numbers can fluctuate between the different tools – Google Analytics could display a different number of page views/month than, say, WebTrends or Omniture…which one is correct? I might start off with one and choose a value that I might achieve using Omniture, but mid-way might switch to Google Analytics, but their back-end calculates a view or impression differently. Which one is more accurate and how can I guarantee that reliability to my superiors. You know what I mean?

    So what if I, as a potential customer, sat in on a demo (again), and talked to you and asked “How accurate is this sentiment analysis of my company's product?” How could you explain that to make me feel better about the numbers I get versus from one of your competitors like Sysomos, ScoutLabs, Attensity360, etc.?

    When you also say that “software trials are great tools”, is Radian6 advocating that customers run simultaneous trials between your product and a competitor to see which results are better? What numbers would they use to benchmark to determine how great the variance is?

  • Ken,

    Hey again! Your previous comment hit upon a bunch of points, so forgive me if my response gets a bit lengthy.

    I'll start with your first comment about how SMM tools, like web analytics tools such as Google Analytics or Omniture may return different results. We have our own unique system of going out and searching the social web, which means that from the very start, you may be dealing with an uneven data set. Say that there's a site that one company covers but another doesn't, for example. Or, even something as simple as what bits of information each company pulls from various social sites. It is important to ask, especially if you have questions about what a company does/doesn't cover, or why they do/don't seem to pick up bits of information that other companies do/don't. This is again, assuming I'm speaking to someone interested in using an SMM tool and going through the evaluation process.

    Sentiment analysis is the current “hot topic” in our industry, it seems. Instead of writing out a lengthy reply to that myself, I'm instead going to link you to a few posts which I think better explain the various bits and pieces involved with sentiment analysis.

    First – from Chris Newton, our CTO – explaining a bit about our automated sentiment analysis capabilities – http://www.radian6.com/blog/2009/12/on-automate

    Second, our PDF on exactly how this works in our product – http://www.radian6.com/wp-content/uploads/2009/

    Third, from Jason Falls, which sparked a LOT of discussion, so check out the comments – http://www.socialmediaexplorer.com/2010/04/26/t

    Again, just as you should ask questions about data accuracy, asking questions about how a company assigns sentiment (if that's important to you as a prospect) is good. There are many ways to achieve the same goal (assign some level of sentiment to a post) and just like companies each go out and dig for information a bit differently, each is also likely to have built that particular feature a bit differently.

    Furthermore, even if you take humans and sit them down, each is likely to score the posts slightly different, so it's important not to rely on machines to do things for you in this capacity. Double-check the results, and go in and change them if need be. Sentiment analysis is far from a perfect science, and it's equally as far from being the holy grail or magic bullet of SMM.

    When I say that software trials are great tools, I mean that trying out the software you're going to use (getting a demo, using a free trial if it's offered) is often a critical step in a buying process. If I were to buy a piece of clothing, I would likely try it on before buying. I want to test drive a car before signing, and I want to test an email marketing system with a sample list to see the user experience from all angles before entering into a contract. We offer free 7-day trials as well as an introductory demo to people for that purpose. Try us out, see if we fit your organization. Go ahead and set up the system and start collecting results during the trial, and ask us questions based on the results. Dig, do your research, and make sure that the tool you're choosing (just like the car you buy or the pair or jeans you select) fits you/your company.

    To answer your software trial question – I've come across a few post series dedicated to doing exactly what you describe, and a quick Google search turned up quite a few pages of results (“social media monitoring tools comparison” was my search term, in case you're interested).

    Katie
    Community Manager | Radian6
    @misskatiemo

  • I totally agree that Social Marketing Analytics is very important for companies measurement.This tool is useful to explain the performance of social media initiatives in the context of specific business objectives.
    I think Social Media really needs to create standardize measurements.

  • @Jack – thank you for your comment. I appreciate that you agree that social marketing analytics are important for companies to measure. Do you have any other feedback on what companies could do to leverage these analytics?

  • @Jack – thank you for your comment. I appreciate that you agree that social marketing analytics are important for companies to measure. Do you have any other feedback on what companies could do to leverage these analytics?

  • lacoste polos

    I wonder how this affects web analytics as well (e.g. Google Analytics, WebTrends, etc.). How can we as marketers and users of these services figure out how good of data we’re getting.

  • The importance of accessories in fashion is well known so no wonder jewelry/accessory designers are highly appreciated for their work.