The analysis and interpretation of large datasets, aka big data analytics, increasingly provides a means of achieving a competitive advantage as the amount of data available to businesses continues to grow in size and sophistication.
It is said that the volume of accessible data doubles every three years given the wealth of information captured by digital platforms, cellphones, sensors, surveys etc. As individuals and organisations, we are – often unknowingly – creating gigabytes of data every day which has huge value to those with access to this information.
The level of insight from big data analytics depends on how many fields of useful information are available and the quality (accuracy, completeness etc.) of this data. Sometimes companies are sitting on multiple datasets that are siloed by function or department (e.g. a CRM system which isn’t integrated with customer / technical support data or inbound marketing/sales platforms, etc.) or siloed by brand (e.g. a company that has been through a merger or acquisition, with separate datasets by brand).
Whether the analysis is based on in-house data alone, or also combines additional information from third parties or other external sources, big data can create value in numerous ways:
The more customer-centric organisations are those who have a wealth of information at a specific customer level and act on this to enhance communications to the customer and how that customer is served. In the more advanced cases, big data is used to drive product development to improve the customer experience – for instance, manufacturers using data from sensors embedded in products to inform after-sales service such as proactive or preventative maintenance.
In circumstances where companies have masses of data on customers, it can be feasible to run a segmentation without the need for primary market research. Sometimes the data are housed in different platforms, so they first need merging and cleaning, along with analytics to incorporate hypothesized data into the missing cells. A range of statistics can then be used to arrive at a segmentation. Latent class clustering is usually the more sophisticated solution as it detects latent variables in the data (like factor analysis) to easily identify an appropriate number of segments, and it can work on almost all types of data (unlike K-means which is limited to scale data).
Through this method, each customer on the database can be assigned a segment, and a tool can be developed to distinguish the segments that prospective or new customers would fall into. Additional data is sometimes appended to existing databases – whether purchased or acquired through primary research – and this can feed into the segmentation model. Whatever the method, big data analytics can be used for more optimal customer targeting, and therefore more tailored products / services and efficient communications to different audiences.
A common use of big data analytics is for interrogating and modeling data to better understand customers, markets or business performance. For instance, big data can be used to forecast demand and profitability of specific products over a given time period, to help write business plans and better plan manufacturing and inventory management. Big data can also provide insights into the competitive landscape, especially in markets where data are readily available such as pharmaceuticals and securities – for example, the anticipated impact on revenue from a rival launching a new competing drug. Big data analytics are also of value following mergers and acquisitions, when a company likely has a myriad of data on various brands, housed in different data platforms.
Substantial insights can be obtained by exploring synergy between the datasets, identifying patterns in the data, and inferring the opportunity based on factors of relevance to business success. A range of statistical methods are used depending on the amount and type of data available. Multinomial logistic regression models are used, for example, to predict outcomes (such as the product bundle most likely to enjoy the greatest price uplift based on a combination of factors such as demand, manufacturing costs, the competitive landscape, etc.) Monte Carlo simulation is another type of modeling technique used, especially in more risky business situations as the tool explores not only what the outcomes could be, but also the actual likelihood of each outcome occurring.
Numbers alone don’t tell the full story. As market research consultants, we are skilled at deriving value from big datasets including recognising patterns in the data that ultimately provide insights, and empowering businesses to take action based on informed predictions and smarter decision-making.