[Best Practices] Dynamics 365 Release Management

If you are a technical or functional consultant working for a partner or developer working at a client site, at some point in your career, you may be asked to perform a production deployment.  Depending on the size and the nature of the organisation and total business value of the features you are deploying, the change you are making to the Dynamics 365 production environment may have a significant impact on a large number of users and the organisation’s bottom line.   It is extremely important to take this task seriously.   YOUR FATE MAY DEPEND ON IT! 

Ok.. that’s a bit too dramatic.  But you get what I am trying to say.  It is important!

Regardless of the size of the change you are making it’s a good idea to follow the same process every single time.  I have seen on many occasions, people taking shortcuts, without following the right process.  Maybe your client is pressuring you to make the change quickly because he/she getting pressured by the business to fix a critical bug.  Maybe it’s a simple change to a security role.  As a Dynamics 365 professional, you need to make sure to do the right thing and say “NO”.  Follow the right process for the benefit of your employer, client, your colleagues, and most importantly, yourself.

Who are the stakeholders? Communicate with them.

Ask yourself, who will be impacted by this production deployment.

• Users of the system – Users are directly impacted by a change to the production environment.  They need to know what’s coming, when they are coming, and the impact of the change.
• Senior Executives of the impacted Business Unit/Organisation – End of the day, Senior Executives are ultimately responsible for all the activity in the organisation.  If something goes wrong, they will surely be involved to make sure it won’t happen again.
• Product Owner – PO owns the features of the system and he/she must be comfortable with what’s included in the deployment package.
• SMEs/Super Users – SMEs/Super Users need to be involved to perform User Acceptance Testing (UAT) and Production Verification Testing (PVT).  They need to be involved and should be well aware of their responsibilities.
• IT Manager – IT Manager is responsible for all IT infrastructure and applications.  He/she needs to be aware of the deployment and provide the approval to proceed.
• Project Manager – Project Manager is responsible for managing all activities of the project.  Production deployment is no different.  They need to be available during the deployment period, onsite or remotely, to liaise with relevant stakeholders when required.
• Development Team – This sounds like an obvious stakeholder to be involved but I’ve seen a number of times that some components of the solution were not included in the solution package or critical post-deployment steps were missing.  Every single person in the development is responsible for the final outcome of the deployment.   They may not be working on the project during the deployment period but may have implemented features which should be included in the deployment package.  Make sure they are informed.  If they are onsite, tap on the shoulder to discuss.  ALWAYS, get written confirmation from the team that they have checked-in their code to source control and the solution includes all the changes required for the deployment.  Don’t take their word for it.  Get written confirmation.  Did I mention, get written confirmation?
• Customers – Customer Portal maybe down for some time.  New changes may require customers to follow a new process.  Depend on the situation, customers may need to know about the deployment.  It’s the responsibility of the Project Manager/Product Owner and the Senior Executives to make that decision.  Don’t send emails to the customers yourself but send an email to the Project Manager/Product Owner, reminding them to do so if required.
• Practice Manager – If you are working for a partner and working at a client site, your Practice Manager may not be aware of the production deployment.  Please let him/her know.  Practice Manager will remind you to follow the process and can support you through the deployment period if there are any issues.  If there are any issues arise during deployment, let him/her know.  If you make a mistake, let him/her know.  If the other stakeholders not responding to your emails, let him/her know.  It is not a burden for the Practice Manager.  It won’t reflect badly on you.  Sooner the issue is resolved lesser the negative impact.  Just keep them in the loop.  Call him/her if needed.

Yes, it is your responsibility as a Dynamics 365 Professional, to communicate with all the relevant parties and/or make sure all the relevant parties are informed.  You may be a Junior or Senior.  If you are clicking that innocent Import Solution button, Y..O..U    A..R..E    R..E..S..P..O..N..S..I..B..L..E   !!!!

Plan the deployment

Make sure to have a clearly defined plan/process for the deployment.  The plan should include;

• Step by step tasks required for the deployment (Deployment Checklist)
• Who is responsible for each task
• Expected duration for each task
• Estimated time to complete each task
• Roll back process

Deployment Checklist

I would typically use a checklist similar to below list.  You can include more information that would help with the deployment process.

Pre-deployment

• Prepare the deployment tasks
• Communicate with Product Owner/Project Manager/IT Manager to find out people who will be involved in the deployment process.  For example, who would perform the deployment?  Who will perform the PVT? Who is authorised to make the go-no-go decision?  Collect mobile phone numbers and email address of all involved.
• Contact everyone who will be actively involved in the deployment and provide them with the deployment plan.  Make sure they have confirmed their availability during the deployment period.  This is critical if the deployment is scheduled after hours or over the weekend.
• Contact everyone in the development team to make sure they have checked-in their code and all components are in the Dynamics 365 solution/s.  Make sure you know what components should be included in the solution.
• Send an email to the development team, Product Owner/Project Manager/IT Manager the features that will be deployed.  This can be a list of user stories completed in the sprint.
• Prepare any Dynamics 365 solution/s, SSIS packages, PowerShell scripts, or any other scripts that will be used to automate the deployment process.
• If using AzureDevOps (formerly VSTS) for building and deploying solutions, make sure the build process is tested and working correctly.
• Prepare any reference data imports required as part of the deployment.  Typically, Dynamics 365 Portals, Field Service, and Project Service Automation solutions may require reference data import.  There are a number of tools available to perform this task.  Make sure you have tested your process in a non-production environment.
• Prepare any master data imports, if required.

Deployment

During the deployment, send an email to everyone actively involved in the process after each step with the status of the current and previous steps.  If the person responsible for the next step does not respond within a few minutes or previously agreed time, ESCALATE immediately!  Don’t wait.

• Create a backup of Dynamics 365 Instance (if Online), or database (if On-Premises) – Dynamics 365 Consultant
• Import the Dynamics 365 solution/s – Dynamics 365 Consultant
• Validate all components are imported successfully – Dynamics 365 Consultant
• Validate all Workflows which should be active are actually active.  Make sure the plugins are active.  Don’t assume. – Dynamics 365 Consultant
• Import data – Dynamics 365 Consultant
  • Turn off Workflows, Flows, and Plugins.  If you have Workflows, Flows, or Plugins triggering on creation or update of any records you are importing, even if you are importing a few thousand records, the impact can be catastrophic.  I have seen a number of times, millions of system jobs were created when importing data due to triggering of Workflows or Plugins, bringing the system to a halt and storage consumption increase exponentially.  Unless you have a very good reason AND you have tested the Workflows and Plugins and the import process 100% on a non-production environment, I would highly recommend to turn off all Workflows, Flows, and Plugins before any data import.  This also improves the speed of the data import.
  • Import reference data if required.
  • Import master data if required.
  • Turn on Workflows, Flows, and Plugins.
• Validate the data are imported correctly without errors. – Dynamics 365 Consultant
• If Dynamics 365 Portal, Voice of the Customer, Field Service Mobile App, Connect Field Service, or any other applications and solutions are required to be updated, make sure those are also deployed (additional deployment steps may be required). – Dynamics 365 Consultant
• If any configurations are required on the production instance, make the relevant configurations.  For example, configuring URLs of external APIs.
• Resolve any errors – Dynamics 365 Consultant – Send an Email with the status and handover to SMEs/Super Users performing PVT
• Start Production Verification Testing – SMEs/Super Users
• Complete PVT – SMEs/Super Users
  • Successful – Send an Email with the status and confirming the deployment is a success.  The person with the authority to make the GO-NO-GO decision must provide written confirmation (Email) with the GO decision.
  • Unsuccessful – Send an Email with the status and details of the issues and handover to Dynamics 365 Consultant
    • Investigate and resolve any issues – Dynamics 365 Consultant – Send an Email with the status and handover to SMEs/Super Users performing PVT
    • Repeat until all issues are resolved.  If the issues can’t be resolved within the previously agreed time frame, ESCALATE.  The person with the authority to make the GO-NO-GO decision will discuss and make a decision.
• If the deployment is successful, send an email to all relevant parties confirming the deployment is successfully complete – Dynamics 365 Consultant
• If the deployment is unsuccessful, perform rollback steps – Dynamics 365 Consultant
  • This may involve, deleting Managed solutions, deleting imported data, and/or restoring the system from the backup and many other steps if external systems are also involved – Send an Email with the status and handover to SMEs/Super Users performing PVT
  • If the rollback process failed, ESCALATE.  Ask for help.  Get a Senior Consultant/Architect involved to help resolve the issues.

Post-Deployment

Once the deployment is complete, it is a good idea to be available to resolve any issues that arise once the regular users start using the system.

Other Useful Resources

There is a great article with a slide deck from Razwan Chowdry which details best practices of Dynamics 365 Release Management.

CRM Release Management Best Practices at CRMUG Reading

The following video from Ivan Kurtev is a few years old but still provides valuable information on best practices for Dynamics 365 release management.

Thank you for visiting Dyn365Apps.com.

Follow me on Twitter – Follow @dyn365apps

Until next time…

About the Author

Nadeeja Bomiriya is a Microsoft MVP, Chapter Lead – Dynamics 365 Saturday – Australia, Committee Member – Melbourne Dynamics 365 User Group, Technical Architect, and Dynamics 365 Practice Lead who lives in Melbourne, Australia.

Disclaimer: This blog post contains opinions of my own and not the views of my employer.