The What, Why and How of Advanced Analytics

The business value of data has exploded over the last decade. This is happening for two reasons: the internet enables people to continually create vast quantities of data, and computers have progressed to the point where they can store and analyze this data. 

Modern organizations have two primary sources of data. These include internal data, such as from clients, customers, internal employees, partners, or website visitors, and external data, such as from influencer reviews, social media, and competitors. Historically, data was collected and analyzed by human hands, but drawing practical insights manually is a daunting, and in many cases impossible, task. Advanced analytics as a discipline is maturing but remains focused on structured sources, which are also mostly internal. 

Enter advanced analytics, the automated examination of data or content using state of the art tools and techniques. Traditional business intelligence focuses on historical patterns, while advanced analytics uses collected data to uncover and predict future trends. 

To help make sense of what advanced analytics is, let’s explore how it works in practice.

The Six Steps in Advanced Analytics

Advanced analytics is the central engine of a data fabric, that includes multiple elements, including data collection, data classification and data access. Advanced analytics follows a precise sequence: 

1. External Data Pipeline
   
   Advanced analytics looks to automate the collection of data. Data in its original form is still typically entered by a human, such as a review that somebody writes on a product or a social media post. Other sources include abstracts, current events, directory registries, public records knowledge, and patents. Data can be manually collected, but this is inefficient and does not scale. Automated data collection continuously gathers data through the use of APIs, scraping, and third-party integrations. This provides the capability of flowing a wide range of up-to-date data at scale into the advanced analytics pipeline, a must-have for success.
2. Data Preparation
   
   External data is created by people going about their daily lives, so it’s not always entered in a specific form. It's sometimes in different languages, in different structures such as graphics, sometimes completely unstructured, and from various types of data such as products in-market, patents, and business activity. Some types of data may be more valuable than others. A post from a key opinion leader, for example, will make a much larger impact than a post from a general member of the public. At this second stage, the machine prepares the data into a form in which it can be analyzed. This also includes cleaning data, which consists of identifying and correct (or deleting) incomplete, inaccurate, or irrelevant data, and normalizing the data and identifying key structural elements within the unstructured data.
3. Content & Sentiment Extraction
   
   At this step, machine learning and Natural Language Processing (NLP) algorithms are being applied to the key data elements in order to extract meaning from the data. The meaning, for example, could be a value that represents a specific benefit or feature or ingredient that a data element referred to, and the sentiment related to that value. This complex automation mimics the human process of content understanding and sets the bar for the quality of advanced analytics. The accuracy of advanced analytics is highly dependent on the accuracy of this step.
4. Data Fusion
   
   After meaning from the numerous sources is extracted, it is then integrated into a unified data set. This is more powerful than looking at each set of data on its own. Think of how humans combined data from our five senses to improve our perception of the world. In a similar way, automated processes combine data to uncover relationships data scientists wouldn’t have otherwise noticed.
5. Data Modeling
   
   Here’s where it all comes together. Once that unstructured, unclean data is cleaned and transformed, analysts can get to the fruitful part of their work: connecting things together and developing models that show impact, dependencies, trends and predictions. This surfaces actionable, predictive insights that brands can use to see around corners and future-proof their business. Doing this accurately, at scale, is impossible without the aid of powerful AI-related techniques and widely-diversified data.
6. Data Management
   
   After data is used to build a model, it can then be stored for future refined models. This requires a well-managed system for processing, storing, and authenticating data. At this point, the sequence starts over, as analytics platforms collect data and repeat the process.
   
   It’s important to note that there can be human touch points in each of these steps. People can decide what data to collect, then inspect it, clean it, structure it, and derive actionable insights from it. The challenge advanced analytics addresses is: how can we automate this process to uncover valuable insights the traditional approach might miss?

The Business Value of Advanced Analytics

Consider a relevant example from the coronavirus pandemic. Data scientists might look at, for example, the number of times people mention the immune system, then look at the number of products that talk about strengthening the immune system. If they discover that the immune system gets a lot of mentions, but there are very few products that claim this feature, that would indicate an unmet need in the market. There’s a lot of interest from the public, but very few companies talking about it.

This process is partially human, since the thought behind looking at the immune system is manual. 

With advanced analytics, we go beyond this traditional business intelligence to uncover connections no one foresaw. Suppose, for example, we follow product sales, then we list the ingredients of those products. We may discover a lot of growth in products that contain Vitamin C. The system can surface something like that, and maybe connect it to trends like the coronavirus and interest in the immune system. Nobody has to think about that in advance. The computer is able to uncover this type of connection by breaking down sales data and connecting it to the ingredients of each product.

The real power comes from combining the human model and machine model. Data scientists can develop models for connecting things together and trying to understand what may impact what. They can use their industry knowledge to understand what attributes or characteristics, like ingredients or patents, are relevant. Analysts can then develop a model that breaks down incoming data to help uncover connections and connectivity they wouldn’t have discovered on their own.

This extra connectivity is a primary driver of business value in advanced analytics. The mix of human insights and powerful machine algorithms allows analysts to dig through mountains of data and find what’s relevant. They can then extend the impact of the data and analytics throughout the enterprise. This is one of the biggest challenges, because oftentimes the analytics is too generic to be applied to specific decisions. Moreover, when this is done in an automated manner, with real-time streaming from the constellation of external data sources, brands can uncover and respond to real-time shifts and trends.

How Advanced Analytics Works in Practice

Advanced analytics uses a variety of techniques to automate this six-step process. 

These include: data/text mining, pattern matching, forecasting, visualization, semantic analysis, sentiment analysis, cluster analysis, multivariate statistics, graph analysis, simulation, complex event processing, and neural networks.

Two of the most powerful techniques are natural language processing (NLP), machine learning, and image processing.

• NLP breaks down a raw sentence into its taxonomic components. This allows analysts to transform unstructured data - such as social media posts or product reviews - into a simplified, structured form that’s readily analyzed. Signals Analytics’ patented NLP algorithms go beyond open source NLP methods. We deconstructed standard NLP, and rebuilt it to extract granular context and sentiment across many different sources and data types.

• Machine learning is an application of AI that teaches computers to understand data based on what it knows about previous data sets. Is a review positive or negative? A machine learning algorithm can take what it knows about previous positive or negative reviews and categorize a new review based on the past reviews. In addition to understanding the meaning of sentences, machine learning can also uncover current trends and predict how they will evolve in the future.

As a whole, these techniques apply the most cutting-edge mathematics, computer science and linguistics concepts. This empowers people across an organization to make evidence-based decisions rather than relying on conjecture.

Use Cases for Advanced Analytics

Advanced analytics is applied across industries, but Signals Analytics focuses on three:

• Food & Beverage

Advanced analytics platforms continually track consumer sentiments to uncover unaddressed needs, such as ingredient or category trends. They go beyond traditional market research methods to give evidence-based recommendations for marketing and messaging strategies. Product development gets strategic insights from tracking ingredient trends and predicting competitors’ future moves.

• Consumer Packaged Goods

Consumer goods manufacturers leverage data created by customers, partners, and competitors to uncover trends and guide disruptive innovation. They uncover hidden needs, allowing brands to prioritize product development and identify fruitful areas of pursuit. Continual external data monitoring helps craft an evidence-based response to real-time disruptions in the marketplace. A deep understanding of consumer sentiments helps maximize marketing ROI.

• Pharmaceuticals

Pharmaceutical companies tap into advanced analytics to assess, predict, and act on macro trends in the market and industry dynamics. Common data sources include patent filings, research papers, clinical trials, drug listings and more. Advanced analytics platforms can uncover underserved opportunities to enable strategic product development. Insights into new capabilities and drug effectiveness help identify opportunities for partnerships and M&A.

Key Points to Consider When Evaluating an Advanced Analytics Platform

Advanced analytics platforms tend to be very complex. Building and maintaining them requires advanced skill sets. Yet they also serve to democratize data, empowering decision-makers across your organization to utilize data insights for evidence-based decisions. 

To ensure an advanced analytics platform is well-suited to deliver the most impactful insights possible, ask:

1. Can it support unstructured, external data feeds?
   
   The internet provides a constant stream of data surrounding technologies, key opinion leaders, events, products, consumers, and sales. To leverage this data, brands need a way to collect that external data and structure it. This requires reaching deep into external sources, then extracting the data into its proper context to generate actionable insights. The best platforms automate the collection and structuring of data to ensure you can surface insights and predictions in real-time. Signals Analytics is one of those unique platforms - we provide both the data and the analytics in a single, configurable package
2. How does the platform structure that data?
   
   To leverage the stream of data being created every second, advanced analytics platforms need a standardized way to clean and structure that raw data into a usable form. This requires a deep expertise of the industry to help understand the context of data, along with cutting-edge NLP and machine learning algorithms. These algorithms go beyond open source NLP techniques, and allow brands to gain accurate insights at scale from consumer discussions or other data sources.
3. Is it transparent with its data collection and analytics processes?
   
   Relying on an analytics platform for decision-making generates skepticism among stakeholders. To be confident in your advanced analytics platform, ensure it’s transparent with how it collects and analyzes data. Ask what sources it uses for collecting data, how it cleans, structures, and analyzes it. Ensure you can rely upon the platform to deliver data-based insights you can act on.
4. Is it configurable?
   
   To get the most out of an analytics platform, look for a platform that supports custom data sources, taxonomies, models and outputs. Ensure you can configure it to deliver the kind of insights you need to make fast, accurate business decisions.

To learn more about what factors to consider as you compare platforms, check out our blog post on this topic.

About Signals Analytics: Signals Analytics is an advanced analytics platform built to empower business decision makers with real-time, actionable market intelligence. Our founders, two Israeli military intelligence officers, realized they could apply the same processes they used on the battlefield to make better decisions in the boardroom. We’re unique in our ability to connect more than 13,000 external data sources using proprietary NLP and machine learning techniques. Our clients include many of the world’s leading pharmaceutical and consumer brands.

To see how Signals Analytics works in practice, schedule a demo today.