For nearly two years, I’ve been researching alternative approaches to analyzing social media “chatter.” Frankly, as the social media movement started gaining momentum more than five years ago, many of us knew the challenge ahead. And I’ll be honest when I say that I’m still not sure there’s a good answer out there.
Discourse analysis, sentiment analysis, text analytics: whatever you call it, you’re looking at a huge volume of information. Unstructured information.
When CRM started in the 1990s, the challenge was to link customers to behaviors and make recommendations to keep them happy/buying more. Think of the Amazon “recommendation” feature: “people who looked at this, bought this” or “if you liked this, may we recommend…” They were able to “mine” purchase data to make better shopper suggestions.
Text mining is infinitely more complicated. The longer a blog post, Facebook comment, or product review, the more difficult it is to categorize. Twitter posts and other microblogs are theoretically easiest because they’re so short.
How It’s Done. Comments are filtered based on keywords (e.g., brand name). Next, as I understand it, using proprietary algorithms (depending on the software), comments are grouped into three buckets: positive, negative, and neutral.
The longer the post, the more likely the comment is to be classified as neutral. This isn’t surprising. Think of the complexity in nuance, sarcasm, slang, humor and differences in word usage across different cultures speaking the same language. (Anyone watch the Emmy’s Sunday night and see Ricky Gervais? He was laughing at Bucky Gunts’ name and it’s taken me 10 minutes to figure out why – so bad I can’t repeat it here!)
I’ve seen statistics showing that between 65-80% of all comments are tagged as neutral. The number is somewhat lower among the microblogs – maybe only 40%. Nevertheless, it’s obvious that huge biases can be introduced using this approach.
Elsewhere I’ve read that humans aren’t very accurate either. Using Mechanical Turk (which isn’t something I’d personally recommend), humans agree only 79% of time on how items should be coded.
We should be moving forward in this discipline. Tom Anderson, of Anderson Analytics, agrees. In a correspondence with him several months back, Tom said that he was using three different software packages and gained confidence based on the intersection their joint findings. I’m not sure if more data is successfully coded or if this just provides more confidence in what he reports as findings to his clients. I think this is very responsible, given the accuracy of single-solution approaches today.
Currently, I’m leaning in the direction of relying more on Market Research Online Communities (or Insight Communities), which straddle the space between pure discourse analysis and more structured qualitative or quantitative marketing research. I view them as a hybrid, where I can get out of the way and follow the discussion. With only 200-500 members, keeping up and staying accurate is more manageable. Further, when custom research is needed, the recruit is a snap.
And sometimes we don’t need to build communities. One may already exist because the product or person built their following online. In this case, the fan group may be the “go to” community because it’s larger, more representative and more cost effective than creating a new community. Think Justin Bieber, who was “discovered” by his YouTube videos. Those subscribing to his stream, particularly in the early days, would have been a great insight-mining resource.
I’ll be keeping a close eye on both these topics as they appeal to both my quant and qual sides. And if you have any thoughts, I’d love to hear and learn!
Some reading:
The best of the articles I read, BrandSavant: http://brandsavant.com/the-hidden-bias-of-social-media-sentiment-analysis/
August 2010 Quirks has several articles on social media research
Mashable: http://mashable.com/2010/04/19/sentiment-analysis/
I got a DVR five or six years ago. I wasn’t exactly an early adopter, but I must say I love it. I record everything, often watching a program a mere 20 minutes after it starts to avoid the commercials. I know I’m not alone.
Don’t get me wrong. Good advertising I really like. But so much of it is bad – or stupid. The DVR lets me watch commercial-free if I want to.
Among DVR owners in Nielsen markets (37%), Nielsen recently reported that 40-50% of commercials are watched during playback, up from earlier estimates of 30-40%. (Source: MediaDailyNews, 8/5/10)
I’m not sure what to believe about this: Commercial viewership up among DVR owners? Really? Or did Nielsen change how they’re measuring things? One hypothesis I had was that more recent DVR adopters could be less sophisticated/less comfortable using the fast forward/jump feature on their remote controls. But Nielsen says that DVR playback is happening more among younger and higher-income viewers, so I can’t imagine my hypothesis is true.
While DVR viewership isn’t universal, these “facts” got me asking a few questions:
- Who’s not watching ads – consistently? Do they have any identifiable characteristics that we can plan around?
- What are the best, most effective ways to reach these ad-skippers in other media and/or via other ways on TV (e.g., product placement, sponsorships, etc.)?
- What defines “stopping power” in a TV ad today, and how is it measured, particularly among DVR viewers? I would think that the first and last ads in a pod would have the highest viewership, yet is this reflected in the numbers agencies are getting to calculate impressions? Related to this, for researchers/account planners, is there a method out there for testing ads in a program-embedded clutter and letting people skip as they would at home, to get at stopping power and ad effectiveness? Plus, can ads be tested at fast-forward speed to assess their stopping power?
- What impact is all this having on brand awareness or brand image? Have any tracking studies been done among ad-skippers vs. those who fast-forward through ads vs. ad-watchers to see what impact ad viewing behavior has on these critical brand measures?
As I write these questions from my logical left brain, I have to remind myself that how we take in information isn’t very well understood. I might not “know” the latest musical phenom, but I always seem to have heard the tune before I learn the artist’s name or see their face – and this is without kids at home. What goes on around us seems to sink in and have an impact.
I don’t know the answers to the questions above, and maybe I’ll dig around to learn a bit more. Yet it’s clear that whether TV ads are skipped or seen, consumers continue to form impressions of brands. And since no matter how hard we try, not everything that impacts them will be known or measurable, I feel more strongly than ever that having consistent brand values which shine in all we do – and being present where our target lives – is the best foundation for the relationships we want to have with them.
I saw this yesterday in “SmartBrief on Social Media”:
No wonder marketers are confused about how to use social media effectively! The juxtaposition of the two article titles made me laugh. (Actually, the FeverBee.com piece is rather good, offering some creative pointers on how to engage community members.)
Staying with the subject of online communities, this month’s “Quirk’s” piece on the 2009 Globalpark Market Research Software Survey reported that only 17% of companies worldwide manage at least one online community and that 56% have no plans for starting one. (A bit more detail: 60% of small and medium companies say they have no plans, with 41% of large companies responding the same way).
Bottom Line: From their actions, most marketers seem aware of the a fine line between staying attuned to customers and invading their privacy. Sure, an online community is a major investment of both time and money. But there are other ramifications that marketers are having to consider which may be slowing down this new research method’s adoption.
While I do a lot of qualitative research in Los Angeles (where I’m based), getting a good cross-section of the client’s target takes me to other cities. We pick those cities with care, yet often there are major-market favorites. For example, Chicago gets picked far more often than St. Louis or Kansas City.
A project last week reminded me once again how nice it is to include even smaller markets in the mix. I drove to Bakersfield from Los Angeles, only an hour and a half drive. (Bakersfield is much smaller than Kansas City, with slightly less than 400,000 in the urban area.) The client picked the market as it’s a good market for them. There isn’t a focus group facility in the area and there is no longer a recruiter based in the market. But I got great respondents (using my LA recruiter) who didn’t have prior research experience, who showed up on time (a good incentive strategy worked!), and who really helped to give us the needed insights.
I had a similar experience many years ago when I had a client who loved the Atlanta market, yet after three projects in a row where we had “dud” groups, we both decided to drive up to Chattanooga, TN. The facility was in someone’s house. Again, the respondents were great, candid, and while not as sophisticated, they gave us a perspective that we hadn’t heard before. (And what a lovely city!)
Online qualitative research does afford us the opportunity to include more of these participants in the mix, but a lot of the smaller markets may still experience some problems with Internet connection speeds.
BTW, I’m not recommending that small markets always be included in market selection – that depends on the client and the target. However, if you’re going to three or more markets, you might want to consider one of the smaller gems to add perspective to your exploratory undertaking.
I’m a huge pro cycling fan. Pro cycling is the only sport where races play out over many days or weeks and all the teams are on the same field at the same time. It is the most strategic sport I know. And that sounds similar to the competitive business environment the rest of us play in all the time.
So, as we are in the month between the Giro d’Italia and the Tour of California and Le Tour de France (and as the Tour de Suisse is running now), I have a few observations that if corporations would follow, would make for a happier, healthier, and more successful business environment.
- There are teams for a reason. You can’t win by yourself.
- There are no silos within teams; no great team has internal cliques. At some point, you’ll need your teammates to win.
- Everyone has specialties. And there are different types of races. This means that each team member has a chance to shine (win) at some point during the season.
- Training is very important. Train toward goals and don’t burn yourself out needlessly.
- “Leaders” change depending on the race. Lance, for example, played “super-domestique” for Levi Leipheimer again this year at the Tour of California; it was Levi’s race for leadership.
- Teammates sacrifice for their leaders. HTC Columbia’s Mark Cavendish is a sprinter whose record last year was simply awesome (six stage victories at the Tour de France alone). Mark has his lead-out men – a whole line of them – who get him into place to get what he calls a team victory. One by one these men sacrifice themselves at the front of the line, working until they can work no more, to ensure that victory. All of his lead-out men can win sprints on their own, but when it’s Mark’s turn, they sacrifice.
- Sometimes you need a mate to get you through a rough patch. A leader will often be paced up a tall mountain by someone who is feeling stronger at the moment; and a leader never gets left behind.
- Sometimes there are crashes. Get back up and keep riding.
- Sometimes the weather is really bad. Stay on the bike and keep riding.
- Sometimes there are small break-away groups – riders who go out in front of the Peloton (main group) hoping to stay away and win the stage/race. It rarely works, but sometimes it does. It gets you noticed and sometimes this break from the norm (AKA innovation) prevails. Risks bringing rewards.
- Instincts are important. Sense what’s going on around you and know where you need to be at any given moment. If you’re caught off-guard, your competitor can make a move and leave you in the dust.
- Team directors can often see a bigger, longer-term picture than the riders are able to. No matter how important you are, remain coachable. We all can and should continue to learn.
Are you on a winning team? Are you leading when you should lead, and actively supporting when that’s your job? Do you agree that these lessons from cycling really do apply to business?
