Everyone is out there monitoring social media for understanding trends around their customers, competitors and products. It is getting increasingly difficult to gain an competitive edge by solely relying on social media monitoring capabilities. Progressive Market Intelligence teams are moving towards website monitoring.
Monitoring social media is similar to taking vitamin pills. Vitamin pills will help you get good health in the long run but are unlikely to provide an immediate cure for any ailment that you are currently facing.
Similarly, developing extensive social media monitoring capabilities will help your marketing and sales efforts become more targeted in the long run. But in the short term, they are unlikely to give you satisfying answers to the questions you have about your customers and competitors.
Think of the most strategically important questions around customers and competitors which leadership typically asks the Market Intelligence / corporate strategy teams to analyse. Questions like -
Which new customer segments should we target for the next leg of growth?
Are my customers increasingly preferring lower-priced products compared to what my company offers?
What new features are my competitors adding to their products and why?
Social media monitoring provides a very limited value add in answering such questions. We will tell you all the reasons after just a handful more paragraphs.
In the short term, social media will not help you get actionable insights about your customers and competitors.
On the other hand, here is something that can unlock significant value for you. And it’s right in front of your eyes. It is nothing but website monitoring, also referred to as web-scraping.
Your competitors’ websites, the news publications and the marketplaces where your customers purchase from are a goldmine of relevant information. Carefully analyzing this information will help you understand the answers your competitors have come up with for questions similar to yours about the customers and the market.
On top of monitoring specific websites, the website monitoring tools can also analyze news coverage and gather insights from marketplaces like Amazon, eBay, Etsy etc. Because of this, they help you evaluate which strategies/capabilities are working and which are not as your competition goes about addressing the evolving customer needs and business environment.
McKinsey says that even in the world of B2B businesses where customer intelligence is typically centred on client interactions, the most forward-thinking companies are heavily investing in data and analytics. Mckinsey mentions that website monitoring is being leveraged as one of the core capabilities for gathering crucial external data about customers and competitors.
Before we show you how website monitoring unlocks significant value for you, here are all the reasons why we believe that the potential for social media monitoring has potential is getting increasingly limited for addressing your Market Intelligence needs.
1. Most of your competitors are already out there monitoring social media conversations. So, you cannot expect that doing the same will give you a competitive edge in any way.
In the wake of the new way of life driven by COVID-19, almost all businesses are leveraging social media for understanding consumer trends, marketing products and building customer relationships.
This shows that social media has great use for you if you have customer experience or product marketing related responsibilities.
But if your primary responsibility is to generate customer/Market Intelligence that gives your company an edge over the competition then social media cannot be your go-to place.
This is because most of your competitors are likely to be broadly monitoring what people are publicly discussing on social media about the products and customer experience offered by them and the competition. And there is a very low chance that you will discover a significant insight from social media that your competitors don’t already know.
This is even more likely if you lack sophisticated software tools and successful past experience of harnessing insights from social media.
2. Important conversations on social media are moving away from public posts to private messages. And no social media monitoring system can get insights from these private messages.
The days of people publicly sharing their opinion about products they used are slowly getting behind us.
Meta (erstwhile Facebook) says that the future of communication will increasingly shift to private messaging. And the central theme of Meta’s growth strategy is to allow people and businesses to have secure, private conversations through Instagram, Whatsapp and Facebook Messenger. Read this note from Zuckerberg - A Privacy-Focused Vision for Social Networking
Every month, businesses exchange more than 20 billion messages with people on Facebook Messenger. That volume is huge. And these are conversations that are centred on products and services and therefore contain invaluable insights for business.
But unfortunately, no social media monitoring tools can draw even an iota of insights from what customers are saying about products and brands in private conversations on platforms like Facebook, Twitter and private communication channels (e.g., chatbots on websites) hosted by businesses.
Hence, for most topics, the pool of data (public posts and comments) that fuel the social media monitoring tools is getting smaller. And the more direct and valuable customer insights will increasingly lie in private conversations.
3. Niche events do not get discussed on social media. So, you cannot rely on social media to help you consistently discover events involving your customers, partners and competitors.
Typically you only get insights from social media only on topics and events which are considered relevant and ‘worth sharing’ by influencers or a significant number of people.
Now let’s say that there is a huge drop in the share price of a competitor or say the government just released a new tax regulation clause that impacts your business. There are several events like these that you deeply care about but which are unlikely to get adequately ‘shared’ on social media for your monitoring tool to notice and tell you.
Some of these events might totally miss you. For the remaining, you may learn of the days or weeks later when you hear about them in the News or see a significant reaction in your competition.
So your reliance on social media for monitoring important events around your business can make you lose the precious opportunity to meaningfully respond before your competitors.
To stay aware of critical events around your customer and competitors, you will be better served by directly monitoring the source where such events are likely to be released. At the source, you get the most authentic and comprehensive information.
And this is where website monitoring solutions like Midesk excel. In Midesk you can set triggers to notify you whenever an important change is published on your competitors’ websites, news or any other web portal. This is as real-time as it gets. We will be telling you more about this in a later section of this article.
4. Social Media insights are usually vague and, at the very best, directional. So, without adequate experience, the right social media monitoring tools, supplementary customer data sources to integrate, and dedicated resources, social media is for gathering Market Intelligence.
You need extensive experience in using social media insights to understand which questions social media can help you answer and to what extent. If you are starting out fresh, it is likely that you will have to invest considerable time and run experiments before you get a good sense of where social media monitoring can help you.
Also, extracting insights from social media cannot be someone’s part-time job. You cannot expect someone to give you good insights by manually looking up keywords and people on social media. You need to have dedicated resources who are extensively trained on social media tools. Moreover, you need to invest time to identify the right tools for your needs.
Lastly, you need to have additional data sources about your customers to supplement what you are hearing on social media. Without these sources, there is a high chance that you will not fully comprehend the context behind the social media insights.
For example, social media may tell you that there is a 200% increase in people searching for “gift cards” in London in the last two months. You might see this observation as indicating a secular increase in demand for gift cards. But if you had been monitoring your competitors, you would know that a couple of gift shop chains folded their stores in the last few months and moved to online retailing. And with this context, you are better positioned to evaluate if the growth trend shown by social media was secular or transient.
5. To understand the broad trends regarding consumer behaviour in your target market, there are paid reports available. These reports are much more comprehensive and actionable compared to insights you typically get from social media
If your questions are not fixated on your current customers and require you to have a macro sense of what’s happening in your target market, then research reports are best positioned to serve you. These reports can be free or paid.
You can easily find a report that explains the most significant consumer behaviour trends in your target market (or its superset).
These reports are typically prepared quarterly or annually. And they derive their consumer insights not just from monitoring social media conversations. The editors talk to the consumers, the market experts, and the VPs/ CXOs of major companies in the market to present a comprehensive analysis of what’s going on around a consumer segment. Here is a free McKinsey report that tells you what matters the most to customers buying mortgage loans currently.
Hence, whenever you think of getting broad insights about consumer preferences and purchase patterns, you should evaluate if the insights can be quickly gained by purchasing one of these reports or you need to squeeze them out from social media.
6. If you happen to have a product e.g., a B2B solution that rarely gets discussed on social media, then your social media analysis is likely to be ridden with ‘small sample size bias.
You might have a very popular product, say Notepad. But it could be that very few people are discussing such products on social media. So, in spite of the sophisticated social media monitoring capabilities, you may have developed, the conclusions you draw from listening to social conversations around your product/product category will only offer you insights based on a very small section of your customers. And these insights are unlikely to be representative of what a majority of your customers feel.
So, what can provide better consumer and competitor insights than social media?
Progressive Market Intelligence teams are increasingly seeking insights directly from the websites of their competitors, partners, and marketplaces.
Here is why…
1. Tracking what your competitors are saying on their websites tell you a great deal about how your customers’ needs, motivations and product preferences are evolving.
As you think about monitoring your competitors’ websites, you may feel that you are not directly connected with your customers. But that is far from the truth.
By monitoring the content your competitors are releasing publicly, you are actually viewing the results of the most thorough analysis of all the insights available about your customers.
Think about it. Your competitors are only going to update their product, pricing, blog, and support pages based on what the customers want the most right now. And in understanding what the customers want they would have leveraged social media insights as well as proprietary insights that they have themselves collected from their customers.
Website monitoring solutions like Midesk can easily scale and get you consolidated insights from the websites of 100s of your competitors. And by monitoring and analyzing the online footprint (i.e., websites, news coverage etc.) of such a large number of your competitors you will get a robust understanding of important developments around your customers. Developments like:
- Which new features/add-ons/services are being preferred by the customers?
- How is the customers’ price preference evolving? Are they opting for lower-priced products?
- Which benefits are being valued more by the customers? What updates are being made to the positioning/value prop of the products out there to address these benefits?
Midesk and other advanced website monitoring tools ensure that you get the insights exactly where you need them to be - in your Inbox or added to a database.
2. Now let’s shift the focus from your customers to your competitors. Are you always aware when your competitors make key moves? Do you understand all the new capabilities your competitors are bringing to the market to keep up with the evolving customer needs?
The competitors’ websites are a goldmine of relevant data points that can propel the analysis.
In the last point, we talked about the demand side - how website monitoring helps you understand evolving customer needs. Now let’s talk about the supply side - understanding how your competitors are addressing those needs.
Advanced web-scraping tools like Midesk can consolidate and analyze content from websites of hundreds of firms (your competitors and beyond) to help you get a good view into areas like:
- What new products/services/features is your competition bringing to the market?
- Who is leveraging advanced technologies like Machine Learning, cloud computing to improve their products and how are they doing it?
- Which of your competitors are aggressively hiring? Which roles are they hiring for?
- Who is opening new physical offices?
- Who is developing new partnerships and why?
Social media can also provide insights to answer some of these questions. But the accuracy and timely delivery of the insights will always remain uncertain.
3. You want to get real-time notifications whenever significant developments happen around your customers, competitors, partners or government regulations. And you want those notifications to be based only on the criteria that matter to you.
With sophisticated tools like Midesk, you can choose to get notified exactly based on what matters to your business. So you can ask to be notified when a key competitor adds a new product to their website. Or when a competitor secures a round of funding. Or when someone slashes their prices by more than 20%. Or maybe when there is a regulatory update. The possibilities are endless.
Bottom line is that through sophisticated website monitoring tools you will get real-time notifications whenever something relevant happens around you.
These tools will only trigger notifications when the valid criteria are met from analyzing the content on relevant websites and news sources. This is much more accurate and powerful than social media-based triggers. Social media monitoring tools typically allow you to set triggers only when milestones are reached around a narrow set of variables, for example, your LinkedIn page followers increasing by 500.
4. Website monitoring tools can ensure that you never miss out on new business opportunities that rise up around you.
Maybe you get a lot of your business through tenders or RFPs published on specific websites. It could also be that your agency gets clients by looking at project opportunities on UpWork or Fiverr.
Basically, if you find yourself or your team regularly visiting a set of websites and making a note of specific information from there, then website monitoring tools can be quite useful. Tools like Midesk can fully automate your workflow, analyze and consolidate the insights, ship them to your email or a database, and send you notifications.
5. Understand what the wisest voices (the journalists, CXOs, seasoned professionals etc.) are saying about your industry, competitors and customers.
Needless to say, a big picture view of your business helps you put into perspective the diverse developments around your customers, competitors or partners.
Website monitoring tools like Midesk can help you track what the journalists and industry influencers are saying about your business in news publications as well as personal blogs. Midesk ensures that you get notified whenever a relevant opinion piece gets released and receive a summary in your email.
So, in which scenarios are social media still useful for gathering Market Intelligence?
If you have a good past record of gathering insights from social media and successfully acting upon them, then social media can continue to be useful for you to answer questions similar to those you have answered in the past. One thing is crucial here. You need to supplement social media insights with as much additional customer and competitor data as you can. This ensures that you utilize the insights in the right context.
Secondly, if you are currently running campaigns then you can use the campaign insights to get a deeper understanding of your customers. Your campaigns can help you figure out which benefits and messaging stick with your customers. They can help you create more useful customer segments. Campaigns could also help validate your TAM estimation. And, lastly, campaigns can give you a broad answer to many customer intelligence questions which are pertinent for your sales and marketing efforts, for example, campaigns could help you estimate the customer acquisition costs.