By: Richard Boire, Senior Vice President, Environics Analytics
As most data science practitioners know, artificial intelligence (AI) is not new and has been explored by academia back as far back as the fifties. The real core of AI is the branch of mathematics related to neural nets which have been explored both by academia as well as data science practitioners. A number of practitioners including myself familiarized ourselves with these techniques which became one more item within the data scientist toolkit. For those of us involved in using predictive analytics to predict consumer behaviour related to marketing and risk, logistic regression and decision trees in many cases performed at about the same level as neural nets. In some cases such as fraud where there were typically a much larger volume of records, neural nets did exceed the more traditional type of modelling techniques.
But the appetite for AI deployment was always negated by its lack of its explainability to the business stakeholder and as mentioned above the minimal examples of its superior performance relative to the more traditional techniques.
So what changed and what has led to all the excitement about AI. In order to better understand this evolution, one needs to focus on the research. Research in this area for decades always focussed on how these tools could better classify images. Back in the nineties, I remember reading numerous articles from publications where the ability to classify images was approximately 40%-50%. In the last 5 years, though, this accuracy has now achieved levels of 95%+. This game breaking change was caused by two factors with the first factor being related to technology and how data could be processed.
Data and Big Data could now be processed and consumed using parallel processing as opposed to sequential processing. Meanwhile, this newfound technical capability allowed practitioners to consume exponentially much larger volumes of data for analytics (both advanced and non-advanced) purposes. The consumption of these extremely large volumes then allowed users to explore the notion of more complex type neural nets or deep learning, which is the ability to utilize many hidden layers and many nodes as opposed to a single hidden layer with few nodes that was the common occurrence within a restricted sequential data processing environment. This ability to more fully leverage the power of artificial intelligence was the second factor which now improved the image classification accuracy to 95%+.
With this breakthrough, AI had to be more seriously explored as another option in improving results. But does that mean that we should blindly adopt AI in all our business processes. Certainly, we are seeing the emergence of applications to better detect fraud through improved image recognition while enhanced customer service is the outcome of improved AI-developed chat boxes. Many more applications are being explored and which are expected to provide further disruption to an already changing economy. But let’s discuss the notion of AI within the world of predicting consumer behaviour both from a marketing as well as a risk behaviour.
The use of data science and machine learning to predict consumer behaviour has been an ongoing business discipline for many decades. Success for seasoned data science practitioners in this area was never..Continue Reading