Examples of extending Dynamics 365 Customer Insights with Azure ML

Dynamics 365 Customer Insights offers a platform to integrate customer data from various sourcesinto one unified view.This unified data presents a perfect starting point for building custom machine learning(ML)models to generate key business metrics. In an earlier blog, we saw how to bring custom ML models from Azure Machine Learning (AML) Studio to work with the unified data from Customer Insights. To accelerate the initial model development, we offer AML Studio–based model pipelines for three frequently encountered use cases:

1. Churnanalysis
2. Customerlifetimevalue (CLTV) prediction
3. Productrecommendations

We will go over these models inmoredetail in this blog.

Hotel Scenario

We will use the Contoso Hotelscenario for the models discussed in this article.AttheContoso Hotel,CRM data is gatheredandconsists of hotel stay activity. In this data, the information about the dates of stay are logged for each registered customer. Along with this, the data also contains information about the booking, room types, details of spend,etc.The data spans aboutfouryears from January 2014 to January 2018.

The second set of datais from customer profiles. These profiles contain personal information for each customer since they are registered with the hotel or since their first stay. This information includes name, birth date, postal address, gender, phone number,etc.

The third set of data includes useof other services offered by the hotel. For example, the use of spa, laundryservices,WiFi, courier,etc. This information is also logged for each registered customer. Typical use of services is linked with the stay, but it is not required;in some cases, the services can be used by customers without actually staying in the hotel.

Churn Analysis

Churn analysis can be performed in different situations (e.g., retail churn, subscription churn, service churn, etc.). In this example, were going to look at service churn, specifically in the context of hotel servicesas described above.Although it may not cover allofthe scenarios explicitly, the model will provide insights into all the steps in building a custom model using Azure ML andCustomer Insights. It will also provide a working example of an end–to–end model pipeline that can be used as a starting point for any other type of churn model.

Definition of Churn

The definition of churn can differ significantly based on the scenario. In this case of hotel activity, our definition isif a customerhas not visitedthe hotel inthe pastyear,s/he should belabeled as churned.

As described inourprevious blog,the AML Studio experiment can be imported fromthegallery.Below is a screenshot of the import blocks that read these tables from the Azure blob storage location.

Featurization

Based on the definition of churn,we first identify the raw features that will have some causal effect or impact on the label. Then, we further process these raw features into numerical features that can be used with ML models.Data integration happens in Customer Insights so as a result, joining these tables can be done by simply using the Customer ID created byCustomer Insights.

The featurization for building the model for churn analysiscan be a little tricky. The problem is typically cast as a static problem like image classification; however, the underlying data is not static like a set of images. The data is a function of time with new hotel activity recorded on daily basis. Hence, featurization should take this into consideration and generate static features from the dynamic data. In this case, we generate multiple features from hotel activity with a sliding window ofoneyear. We also expand the categorical features like room type, booking type, etc. into separate features using the technique of one-hot encoding.

The final list of features isshown below:

Model selection

Once the feature set is ready, the next step is to choosetheoptimal algorithm to use. In this case, we have majority of features coming out of categorical features.Typically,in such situations,decisiontree–based models perform well. In case of purely numerical features,neural networks could be a better choice. Support vector machine (SVM) also is a good candidate in such situations; however, it needs quite a bit of tuning to extract the best performance. We choose gradient boosted decision tree asthefirst model of choicefollowed by SVM asthesecond model. AML Studio lets you performA/B testing of two models to compare and contrast. To make better use of this, its always beneficial to start with two models rather than one.

The following image shows the model training and evaluation pipeline from AML Studio.

We also apply atechniquecalled “Permutation Feature Importance. This is a crucial aspect of model optimization. Allofthe built-in models present a black box model with little to no insight into the impact of any specific feature on the final prediction. The feature importance calculator uses a custom algorithmto compute the influence of individual features on the final outcome for a specific model. The feature importances are normalized between +1 to -1. The negative influences essentially mean that the corresponding features have counter intuitive influenceon the outcomeand should be removed from the model. Higher positive influence indicates that the feature is contributing heavily towards the prediction. These values should not be confused with correlation coefficients as they are completely different metrics. For more details, see Permutation Feature Importance.

Completing the integration withCustomer Insights

Once the training experiment is complete and the resulting metrics are acceptable, we can create a predictive service as described in the previous blog.Make sure the predictions are accompanied with the customer IDs fromCustomer Insights. After that the predictions can be exported to the same blob storage and ingested back intoCustomer Insights.

The entire churn experiment is available in the Azure ML gallery.

Customer lifetime value (CLTV)prediction

The customer lifetime value (CLTV) calculation is one of the key metrics that a business can use to assessand segmentits customers. Forthehotelbusiness, it iscriticalto know their customers.For instance, understanding the differencebetweencustomers thatbring more valueversusthe customers that dontcan provide crucial information to hotel management. This type of segmentation can help hotel management assess which features they need to focusonand improve to satisfy their high paying customersagainstfeaturesthatarelessimportant. These decisions can have a direct impact on sales and earnings. In this example,we will define the CLTV as the cumulative amount brought in by the customer in the designated time frame.

Definition of CLTV

In this context we are going to define the CLTV of a customer as of today, as the total dollar amount the customer is expected to spend in the next365 days or 1 calendar year. We are going to use the past three years’ worthof data for all customers to predict this value.

Featurization

In this case, featurization is going to be quite similar to the churn scenario; however, the label and predicted values are different than defined above.

Model selection and training

Predicting the CLTV is a regression problem as the predicted value is a positive valued continuous variable. Based on the feature properties, we select Boosted Decision Tree Regression as one algorithm and Neural Network Regression as another algorithm to train the model.

Completing the integration with Customer Insights

As described in the churn model, the output of the CLTV model is tied with Customer IDs and ingested back into Customer Insights for further analysis.

The entire CLTV experiment is available in the Azure ML gallery.

Productrecommendation or Next Best Action

Product recommendation in the context of the hotel scenario is interpreted as recommending services offered by the hotel to the customers. The objective is to choose the appropriate services for customers so that their usage is maximized. The problem is similar to the problem of movie recommendations for video streaming service users.

Definition of Product Recommendation or Next Best Action

We define the goal as maximizing the dollar amount of service usages by offering the best matching services to hotel customers according to their interest.

Featurizationand Training

Similar to the churn model, we are joining the hotelServiceCustomerIDwithCustomerIDin order to build recommendations consistently perCustomerID.

As described in the beginning, the data is sourced from threedifferent entities and features are derived from them. The featurization for the problem of recommendation is different compared to churn or CLTV scenarios. The recommendation model needs input data in the form of threesets of features. One set of features represents the past usage of services by the customers, and thesecondset representsthe details of each service itself and the third set represents the details of the customers.

We use the algorithm called Matchbox Recommender to train the recommendation model.

From the figure above, thethreeinput ports for the Train Matchbox Recommender model takes in the training service usage data, customer description (optional), and service description.There arethreedifferent ways of scoring the model. Oneis for model evaluation where an NDCG score is computed to evaluate the rated items rank. In this experiment, we have NDCG score as 0.97.The other two options are scoring the model on the entire recommendable service catalog or scoring only on items that users have not used before depending on the business requirement.

Looking further on the distributions of the recommendations on the entire service catalog, we notice thatphone,WiFi, andcourier are the top services to be recommended. This is consistent with what we found from the distributions of the service consumption data:

The entire product recommendation experiment can be accessed in Azure ML gallery.

More information

Readour Team blog to stay up-to-date with the latest features and ongoing innovation.

Engagewith us on our Community siteand provide ideas and feedback on the product.

The post Examples of extending Dynamics 365 Customer Insights with Azure ML appeared first on Dynamics 365 Blog.