Skip to main content

Notifications

Customer experience | Sales, Customer Insights,...
Suggested answer

Churn model best practices

(1) ShareShare
ReportReport
Posted on by 5

Hello Everyone, 

We are trying to implement the Churn prediction model in CI for a customer. We have many questions about it in term of best practices:

    • What is the best way to test the Churn model ? To train it on a part of the dataset (all data except last 6 months for example) and then compare results with the other part (last 6 months data) ?
    • Does the event parameter or activity type in activities has an impact on the model training ? What about the detail parameter ? Can we use a continuous variable or only categorical ones ? 
    • Should we split activities to obtain better results, ex: slit entity tasks into phone calls and email correspondence or tickets into incident and asks
    • Why event activities for subscription is set by default to “Is a recurring subscription” when we created it from the Churn model?

If you can help me on one of this point I will appreciate a lot !

Thanks in advance :)

    • Zhi Profile Picture
      Zhi on at
      RE: Churn model best practices

      You're welcome!

      Unfortunately, currently only the detail parameter is used in the churn model to take additional info (i.e., satisfaction). In the future version, we will consider to use event for model improvement. Thanks a lot for your feedback!

    • Dimitri Cbd Profile Picture
      Dimitri Cbd 5 on at
      RE: Churn model best practices

      Thank you very much Zhi for these informations !

      I'm just a little surprised for the event parameter: you mean that no matter which variable we choose it doesn't impact the model performance. For example for a ticketing activity it would be great to bring more information than just the creation date, but also that this ticket has been created by email/phone or add the customer satisfaction rate on this ticket...  So it is the same for detail parameter ? Whatever we fill in it does't impact the Churn model performance ?

    • Suggested answer
      Zhi Profile Picture
      Zhi on at
      RE: Churn model best practices

      What is the best way to test the Churn model ? To train it on a part of the dataset (all data except last 6 months for example) and then compare results with the other part (last 6 months data) ?

      • Yes, this is one way for validating out of time performance. Internally, the model is validated at different time points through a sliding window approach.

      Does the event parameter or activity type in activities has an impact on the model training ? What about the detail parameter ? Can we use a continuous variable or only categorical ones ? 

      • Event parameter won’t impact on the model training.
      • Activity type will. If you specified different activity types for different activity data, model features will be created for each type separately and combined.
      • Details: Yes, you can use continuous variable for details to record amount/value about customer activity.

      Should we split activities to obtain better results, ex: slit entity tasks into phone calls and email correspondence or tickets into incident and asks

      • Split activities will increase number of model features as mentioned above. Theoretically, if you have sufficient data, the model should produce better results as you bring in more information to the model by splitting activities. In practice, I would suggest considering the actual data size and how relevant the activity is to the churn risk (ex. If coupon is universal, differentiating receival and redeem will be useful) when making this decision.

      Why event activities for subscription is set by default to “Is a recurring subscription” when we created it from the Churn model?

      • will come back to you on this one
    • Nimrod Magen Profile Picture
      Nimrod Magen on at
      RE: Churn model best practices

      Hi Dimitri,

      Thank you very much for raising it.

      We will come back to you soon.

      Yours,

      Nimrod

    Under review

    Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

    Helpful resources

    Quick Links

    Announcing Our 2025 Season 1 Super Users!

    A new season of Super Users has arrived, and we are so grateful for the daily…

    Announcing Forum Attachment Improvements!

    We're excited to announce that attachments for replies in forums and improved…

    Vahid Ghafarpour – Community Spotlight

    We are excited to recognize Vahid Ghafarpour as our February 2025 Community…

    Leaderboard

    #1
    André Arnaud de Calavon Profile Picture

    André Arnaud de Cal... 291,965 Super User 2025 Season 1

    #2
    Martin Dráb Profile Picture

    Martin Dráb 230,836 Most Valuable Professional

    #3
    nmaenpaa Profile Picture

    nmaenpaa 101,156

    Leaderboard

    Featured topics

    Product updates

    Dynamics 365 release plans