Competing in the 21st century requires a 360º vision of our markets (customers, competitors, suppliers, technologies, regulation). Market Intelligence is becoming increasingly important; but, its dependence on manual processes, that are not very scalable, renders its application in decision making difficult. Deep text analytics allows for more scalable and actionable Market Intelligence.
Market Intelligence: benefits and limitations
Market Intelligence consists of collecting information about a certain market in a broad sense (i.e. its customers, competitors, partners and supply chain, investors, economic, legal and technological environment…) to extract actionable insights that can be used for strategic decision-making.
The value of Market Intelligence is enormous: understanding the needs, perceptions and opinions of customers, discovering developments in competitors, detecting investment and collaboration opportunities or identifying emerging technologies and relevant legislation allows us to achieve competitive advantages, refine and optimize our business model and, more generally, respond quickly to changing scenarios (and which one is not changing currently?).
Market Intelligence (and to some extent its subset of Competitive Intelligence, which is more focused on competitors) differs from traditional Business Intelligence in that it mainly collects and analyses information from external and unstructured sources.
Now, more than ever, it can be said that “the truth is out there” and that no organization can afford to disregard the information published on social networks, forums, blogs, traditional media platforms, and, generally-speaking any external websites. Market Intelligence must apply a Capture – Analyze – Update process in which a wealth of external content is collected and analyzed to extract insights, which are shared and distributed throughout the organization to make strategic decisions and more intelligent processes possible.
Nonetheless, achieving this is not easy and many current Market Intelligence tools and technologies have revealed their limitations in this new scenario. In particular, with regard to the analysis of unstructured content, some tools are limited to relatively superficial analyses such as classification and extraction of elementary information from the text. For example, knowing that a news item concerns the launch of a new product and that it mentions companies A and B is valuable, but it does not directly provide us with actionable information: Who launched the new product? What is the name of the product? What category does it fall into? These shortcomings mean that manual analysis by human experts must be used to digest this content and extract the detailed and structured information that will enable decision making.
However, it is not only a question of the depth and quality of the analysis. In many cases we find that the tools are “closed” and do not even incorporate all the relevant data sources for an industry (e.g., specific forums, specialized media). In short, not only are we finding that there is a problem regarding recall (we miss information) but that there is a problem regarding precision (it is not analyzed correctly) as well. All of this hinders the effectiveness and efficiency of our Market Intelligence initiatives.
Applying deep text analytics to Market Intelligence
The MeaningCloud approach for Market Intelligence incorporates two areas of improvement that allow scaling the analysis with the necessary volume, speed, quality and cost requirements:
- Incorporation of any source.
- Generation of actionable insights.
Incorporating any source
While other tools are limited to the incorporation of content from the most common social networks and media, MeaningCloud is totally flexible in this respect, including:
- Predefined standard integrations, with a variety of social networks and information providers: Twitter, Facebook, Instagram, Twingly, DataSift, Webhose.io.
- Web scraping technology for extracting content from any website, browses such websites as a human user: authentication, session management, querying and data extraction.
These possibilities allow us to greatly extend the panorama of information that can be extracted, enabling previously unapproachable analysis:
- News: competitive analysis, financial events.
- Social networks: customer insights, competitive analysis, influencer analysis.
- Forums, review sites: customer insights, competitive analysis, financial sentiment.
- Competitors: new products, projects and partnerships.
- Clients (e.g. buyers): new projects and partnerships, procurement solicitations.
- Sector sites: supply chain events, shortages.
- Government and regulators: legislation, administrative authorizations.
Generating actionable insights
MeaningCloud understands natural language. Its engines carry out a deep morphosyntactic and semantic analysis of the text and the disambiguation technology is able to discriminate between the meaning of the expressions according to their context (e.g., discerning whether “Washington” refers to the city, the football team or a surname). It incorporates standard functions to detect themes, entities, concepts, sentiment, emotion, intention, discovery of topics… Furthermore, its customization tools and APIs allow you to adapt it to your application/domain to increase the accuracy of the analysis.
As if that were not enough, Deep Semantic Analytics tools allow you to discover the deep meaning of complex documents through functions such as the extraction of passage-level categories or semantic relations.
Areas of application
This section explains some applications of deep text analytics and the MeaningCloud approach to Market Intelligence.
Understanding our customers in depth
Customers are the essence of a market: without customers there is no market worthy of the name. That is why a thorough understanding of our customers is a major component of any market intelligence solution. Nowadays, forums, review sites, communities… serve as a huge source of information about this group and an opportunity to better decipher them. MeaningCloud can extract information from those sources and apply out-of-the-box products, to extract Sentiment Analysis, Emotion Recognition, Intention Analysis, Multidimensional Satisfaction or Corporate Reputation (this was described in detail in this article and recorded webinar). However, in addition, through specific developments and configurations we can analyze:
- Concerns and delights: what do they care about and what do they love about our category?
- Key purchase criteria: what attributes are most relevant to the purchase decision?
- Perceptions: how do they perceive us, compared to the competition, with respect to a set of relevant attributes?
- Preferences: why do they buy our products, and those of the competitors?
Discovering business opportunities
Customers are buying, but these processes often go unnoticed by suppliers. Wouldn’t it be extraordinary to have a service that alerts us to open purchasing processes in a certain industry or for a supplier’s products? MeaningCloud can connect to buyers’ websites (e.g., https://sam.gov/), authenticate itself, navigate, consult open processes, download documentation and extract structured data: product name, presentation, number of units, target price, contact information, offer date… Not to mention, it does this comprehensively across hundreds of sites in any language. In this way, instead of an incomplete stack of unstructured documents, we can have a detailed set of qualified and quantified sales opportunities. This has an immediate impact on user turnover.
Analyzing the environment
The analysis of the economic, legal or technological environment is what truly distinguishes comprehensive market intelligence solutions. The ability to digest thousands of reports, scientific articles, regulatory documents or patents cannot be based on slow, error-prone and non-scalable manual procedures. MeaningCloud provides the functionality to automatically analyze and understand such content at various levels:
- Classification according to relevant (custom) taxonomies, e.g.: related to our product categories.
- Identification of relevant topics (customized), e.g.: medical vocabulary extraction.
- Grouping documents within a collection according to their similarity and the discovery of topics emerging from it, e.g.: coronavirus.
- Automatic summary: extraction of significant phrases.
- Passage-level categorization: structure of subtopics, e.g., provisions in a law.
- Extraction of complex insights, e.g., semantic relationships between topics expressed in a document.
Detecting signs of growth
A very valuable use of Market Intelligence is when it is used to identify companies to partner with or invest in (e.g. private equity, VC). In this scenario, detecting “growth signals” (identifying companies in a sector that are evolving positively in terms of new clients, projects, etc.) is a very useful indicator. MeaningCloud can analyze news sites, social media, employment sites and sectorial sites to analyze:
- Media presence: share of voice (SOV), sentiment, etc. as an indicator of the volume of conversation around the company.
- New customers with whom the company starts to collaborate.
- The launch of new products.
- The launch of new projects: investment in new technologies, opening of new countries, etc.
- New alliances with partners, suppliers, distributors, joint ventures.
- New additions to management and technical staff, etc.
- New investments: entry of new capital into the company.
- Mergers and acquisitions.
Understanding influencers and audiences
In today’s hyper-connected world, no vision of a market is complete without including the analysis of its influencers: the individuals who, with their social comments, prescribe and guide the public’s purchasing and consumption behavior. MeaningCloud allows you to connect to networks and forums and profile market influencers: identifying the topics of their publications and those that interest your audience, and the engagement that those publications generate in that audience. We can also prospect for influencers, starting from “seed” themes and scopes and identifying individuals with potential to influence them. In this way, not only do we help with understanding the landscape of influencers in a market, but also with discovering possible influencers in it, with whom companies can collaborate in commercial campaigns. Finally, in a closely related field, we can detect “false news”: content invented with the aim of misinformation, damaging the reputation of others or obtaining personal benefit. Our technology detects signals, both internal to the content and from its context, which allow it to be classified as potentially false.
Conclusion
The integration of various information sources and the capacity to extract structured insights from such varied content allow MeaningCloud to analyze the complete life cycle of any industry with levels of automation and depth that were previously unheard-of : from the detection of financing events and early rumors of mergers and acquisitions, through the granting of patents or the development of pre-commercial trials and pilot tests, to the granting of administrative licenses, the launching of new products and user feedback about them.
As we have seen, the limitations of current technologies limit the value of Market/Competitive Intelligence by restricting the exploitation of information sources and making it dependent on excessively manual and inefficient processes. MeaningCloud, thanks to the integration of a wide variety of sources and the automatic extraction of structured and deep insights from them, makes Market Intelligence more scalable and actionable.
[PS: Webhose has been rebranded as Webz.io.]
