Predictive Analytics using Power BI

Predictive capabilities have always enamoured humankind - from astrologers to data scientists. Just like our galaxy – there are patterns, clusters and trends in data, which hold astronomical value for businesses. Data driven insights could be descriptive, prescriptive or predictive and in this article my focus is Predictive Analytics. Stated simply, predictive analytics analyses current and historical facts to make predictions about future or otherwise unknown events, using patterns found in historical and transactional data.

The business value of predictive analytics

To help illustrate the tremendous value of predictive analytics in business context, here are a few real examples I have encountered while working with customers (anonymised)

Examples of predictive analytics in business

_____________________________________________________________________________________________________________

Contoso is an enterprise manufacturing aircraft engine, burdened by a mounting cost of equipment failures. The ability to proactively predict equipment failure by tapping the millions of signals being generated by machinery – transforms their service operations and uptime. The failure predictions automatically create contextual workorders in the service management system, helping technicians proactively service the faltering parts before the failure occurs. Their transformed and more profitable business now sells ‘uptime’ instead of engines. (predictive analytics examples in manufacturing)

_______________________________________________________________________________________________________________

Contoso is a banking institution – designing a campaign to influence existing customers to invest in a newly launched financial instrument. They use predictive analytics to segment customers who are most likely to invest, using socio demographic factors, their relationship with the bank and how they interacted with previous campaigns. The dynamic segment has the membership automatically fed by predictive engagement propensity. Predictive insights led marketing automation has reduced marketing cost, improved marketing performance, and contributed to the success of the financial instrument. (predictive analytics examples in manufacturing)

______________________________________________________________________________________________________________

Contoso is an electric utilities company, facing high levels of customer churn, as customers switch to competitors as soon as a more suited tariff is offered. Using predictive analytics Contoso uses customer data (like tariff, interactions, demographics etc.) to predict customer churn, which is directly actioned by the customer retention team – who proactively engage the customer to deliver a more suitable tariff. An insight into exactly what factors influence customer churn serves as a cherry on the cake. (predictive analytics examples in utilities)

________________________________________________________________________________________________________________

Predictive analytics – The litmus test

While predictive analytics holds tremendous value and potential – organisations have struggled to get it right. A litmus test for any analytics exercise to derive value from data, should ask the following 3 questions:

• Are your data collection experiments problem driven and designed with a business judgement?
• Are you able to uncover Insights like - clusters, anomalies and trends by connecting disparate data?
• Are you able to embed & action the insights in your business process fabric – delivering tangible value?

As you would have gathered from the litmus test – business Judgment and business processes must intricately influence any Analytics project, however it’s not uncommon for organisations to undergo huge analytics projects, with business judgement and business processes being more of an afterthought. The examples in the previous section illustrate, how planned analytics weaved in the business process fabric is the key to unlock business value. (predictive analytics best practices) (advanced analytics best practices)

Predictive Analytics done right – what does it take ?

As you would have imagined, predictive analytics is a complex science which encompasses a variety of statistical techniques from data mining, predictive modelling to machine learning. There are 4 steps to any successful advanced analytics project.

1. Define the business problem and outcome which necessitate predictive analytics
2. Design data collection/experimentation – clean, merge and map data, remove bias
3. Design and train an accurate predictive model
4. Business process automation to embed and action insights in your business applications.

Getting to clean, map and merge data from a variety of sources, design and train an accurate predictive model and thereafter embedding the insights in business process context has been costly in terms of time, skillset and complexity. However you would be pleased to note that disruptive innovation in analytics technology, advancement in machine learning algorithms and availability of planet scale compute on the cloud is reducing the entry barrier to simpler applications of predictive analytics.

I recently decided to experiment with recently launched Power BI AutoML and AI visualisations, and was thoroughly impressed by the fluidity with which Power BI aims to tackle the entry barrier to Predictive analytics – addressing the issues I listed in the ‘litmus test’. Following sections provides a high-level guide to bring these capabilities to life.

Predictive Analytics with Power BI 

Predictive analytics using Power BI : With Power BI AutoML, the data science behind the creation of Machine learning models is automated by Power BI, ensuring that business analysts, data professionals and developers without any data science background can build high quality predictive models. The AI visualisations highlight the key features among your inputs that most influence the predictions returned by your model. And most importantly the predictive insights can be actioned in business process context. Listed below is a step by step guide. Please note that data ingestion and refresh currently requires a premium Power BI workspace. (advanced analytics with power bi)Predictive analytics power bi(power bi machine learning)

Step 1: Prepare Data

Dataflows in Power BI help organisations unify data from disparate sources and prepare it for modelling. You can use dataflows to ingest data from a large and growing set of supported on-premises and cloud- based data sources including Dynamics 365, Salesforce, Azure SQL Database, Excel, SharePoint, and more. Dataflows can be used to ingest, transform and map data from more than one sources, however in this example I have used single data source Dynamics 365 (CRM)

1. Create an app workspace, or go to an existing app workspace
2. Create a Dataflow and establish connection with the desired entity

Power BI and Dynamics 365 share the ‘common data model’ ensuring a seamless connection without the need to map data. Data flows also allow limited data cleansing and transformation capabilities.

Step 2: Train, Review and Apply ML models

1. Configure the desired prediction and machine learning model

Open the newly created dataflow and navigate to the machine learning tab. Click on get started to apply the most appropriate ML model. Power BI users supervised machine learning - which means that they learn from the known outcomes of past observations to predict the outcomes of other observations. The input dataset for training an AutoML model is a set of records that are labelled with the known outcomes. In this step you must select the field which is desired to be predicted.

Power BI uses 3 ML models – the choice is usually governed by the datatype of desired prediction.

Binary Prediction model: Used to predict events that can have a binary outcome. Example: Whether an opportunity will convert). The outcome is a probability score which identifies the likelihood that the outcome corresponding to the label value being true will be achieved.

Classification model is used to classify a dataset into multiple groups or classes, used when the predicted outcome can have one of the many possible outcomes.

Regression model is used to predict a value – like customer sentiment, revenue realisation etc.

2. Refresh and Train data

Power BI splits the historical data you provide into a training and testing datasets. In some cases, the final model generated may use ensemble learning, where multiple models are used to deliver better predictive performance. You would notice that Power BI AutoML automatically picks the input fields from your dataset, which could be further tuned.

3. Review the ML performance

AutoML generates a Power BI report that summarizes the performance of the model during validation, along with the global feature importance. You can review the model report to understand its performance. You can also validate that the key influencers of the model align with the business insights about the known outcomes.

4. Apply the AutoML model to your data – generate predictions

If you're satisfied with the performance of the ML model created, you can apply it to new or updated data when your dataflow is refreshed. You can do this from the model report, by selecting the Apply button in the top-right corner. Applying the ML model creates a new dataflow entity with the suffix enriched <model name>, which can be used for visualisations.

Step 3: Deliver and action the insights

1. Use the AI visualisations in Power BI to surface insights (Power BI Key influencers)

The Key influencers visual in Power BI helps you understand how various factors influence the metric you are interested in. While you have made the predictions in the previous steps, key influencers visualisations can be used to analyse how different fields in your data impact the predicted value – which is quite valuable for the business decision makers. From my earlier examples – being able to understand the factors most likely to cause customer churn could bring about business process improvements which hold tremendous value.

Use the enriched entity created as an outcome of the machine learning exercise, select the ‘Key influencer’ visualisation, drag the predicted value in the ‘Analyse’ section and drag the likely influencing factors in the ‘Explain by’ section. Power BI does all the statistical hard work to surface the factors influencing the predictions. Moreover, it also segments your data and highlights clusters to provide hidden insights into your data.

2. Embed the insights in business process context

Power BI reports and dashboard can be embedded in Business application like Dynamics 365, Microsoft Teams and PowerApps – ensuring that the predictive insights are now directly available to the business users and can be actioned in context. (dynamics 365 predictive analytics).

Hope this article helped, please feel free to share with the larger community, comment, feedback and reach out for help if needed. If you like to evangelize with your professional network - A version of this BLOG is also available on LinkedIn.

Collective wisdom is good for this planet!

Any opinions expressed herein are solely those of the author