Personalized Community is here!
Quickly customize your community to find the content you seek.
Choose your path Increase your proficiency with the Dynamics 365 applications that you already use and learn more about the apps that interest you. Up your game with a learning path tailored to today's Dynamics 365 masterminds and designed to prepare you for industry-recognized Microsoft certifications.
Visit Microsoft Learn
2020 Release Wave 2Discover the latest updates and new features to Dynamics 365 planned through March 2021.
Release overview guides and videos Release Plan | Preview 2020 Release Wave 2 TimelineWatch the 2020 Release Wave 1 virtual launch event
Ace your Dynamics 365 deployment with packaged services delivered by expert consultants. | Explore service offerings
Connect with the ISV success team on the latest roadmap, developer tool for AppSource certification, and ISV community engagements | ISV self-service portal
The FastTrack program is designed to help you accelerate your Dynamics 365 deployment with confidence.
FastTrack Program | Finance TechTalks | Customer Engagement TechTalks | Upcoming TechTalks
Hope you all are doing well.
Here looking for a best solution to address one business need. Business want to send SMSs to customer's to participate in the Poll.
In CRM we have a custom entity to control SMS content, URL and reminder and an external database to manage the transactions.
From CRM side I need to process all records and update the SMS content and Shorten the URL and update the same to an external database. To shorten the url using 3rd party api.
My ask is how to design the solution which can process millions of records without any issue, currently thinking to develop a console app to handle all this requirement.
How to avoid performance issue, If I put this much population in a foreach loop/parallel foreach loop to process the records and perform update on CRM entity record, calling 3rd party api to get shorten url in response & updating in CRM entity and updating external database.
Appreciate your help to design a best solution.
If you are talking about millions of records, you could consider using a queue/service bus based approach to reliably process these records. With this approach, as and when the records become available, you could push it into the queue and multiple services can process items from the queue allowing you to scale as an when you need.
If you are completely On-Prem, and not ready to pay for cloud bases services, you could consider following option
01. You could use RabbitMQ as your queue management system and push all the SMS records that needs to be sent to the queue - you can push (publish) them in batches and to multiple queues - You can have this logic in CRM in plugin or workflow with CWA to push SMS record for processing as and when it becomes available
02. You could then use activity brokers like Hangfire Service(https://www.hangfire.io/) or MassTransit (masstransit-project.com/.../quickstart.html ) to process messages from the queue - This processing could involve updating content and shorting URL and sending to the external database. You can have multiple instances of these running, same server or different servers, all processing items from RabbitMQ queues.
03. Service mentioned I point 2, can also do the updating back to CRM
04. You could consider storing the SMS records in an intermediate database or some indexing and search technologies like Lucene (https://lucenenet.apache.org/)
One thing to consider, try and find ways to batch and push records to queues as and when they are ready for processing. Even if loads of them become available at once to be processed, you can batch them into in let's say 1000s or 10000s
Business Applications communities