My company has been using dynamics CRM on premise for several years. We are, of course, now migrating to the 365 platform.
I am not an office 365 or dynamics 365 specialist. I am a back end developer who specialises in SQL Server and C#. I have been given this task purely because I am the most experienced developer in my organisation and the consultants initially asked to provide a quote for the work quoted something on the order of a 6-9 month contract, which our CIO could not accept as being reasonable.
I am thoroughly confused by the evolution of the dynamics 365 platform that seems to have happened over the past 1-2 years. As a developer, the mechanism for data integration with our on premise dynamics CRM was clear: Data in the CRM system could be read via the Filtered Views in the CRM database. Write operations, on the other hand, went through the XRM SDK, which exposes the CRM data as entities whose data can be manipulated and then written back via service endpoints, ensuring that any business logic written into the application for those entities was executed. So, databases and web services. Simple.
The equivalent application surface area now seems almost deliberately obtuse, possibly due to the evolution into CDM/CDS and now dataverse. In particular, the overwhelming majority of documentation is very much sales-focused-jargon (telling me all about how it is now easy for "citizen developers" or "makers" to build productive business focused value-adding secure scalable blah blah blah), without giving me any actual technical details. This is further complicated by the emergence of "dataverse for teams" which - in true microsoft form - has made specific information extremely difficult to find because of the naming collisions one gets out of a google search, and again, the "documentation" focused on trying to persuade decision makers to move to the platform rather than providing specific information relevant to an actual developer.
As you can tell, I am becoming more and more frustrated by microsoft's opacity when it comes to their azure products.
So, my specific question is this: Suppose I have an existing integration between some application which is NOT in any way related to the dynamics 365 suite of applications, and a "legacy" dynamics CRM installation. This integration synchronises data via a SQL Server integration services package, which reads from some source, and uses a script task to write that data into dynamics CRM via the XRM SDK. Another solution moves data in the opposite direction, reading from Filtered Views in the CRM database, and sending that data out to some unrelated application. These SSIS packages run via SQL Agent job schedules. For the record, each of these integration solutions individually took one to two hours to develop, just to demonstrate the ease and simplicity of the "old" way of doing things.
What is the appropriate architectural pattern, and set of technologies, that one should use to replace the above solution if the dynamics CRM platform has been replaced with a dynamics 365 product? If the answer is to use OData via dataverse, what are the licensing implications? Is the "user account" which connects to dataverse in order to read or write the data considered a licensed user, with an associated cost of, IIRC, about 30 US dollars per month?
Further to this, I am currently watching this video https://www.youtube.com/watch?v=N73riCrC4ws which indicates that dataverse is "not populated automatically" - so I gather that dataverse is *not* merely a layer of indirection that sits on top of our "CRM" data and provides various mechanisms of access. It is apparently an independent repository that must be populated by us in some way. So, if that's the case, how would I access our "CRM" data without first spinning up a dataverse model and wiring that up to our actual data? Or is this not even possible? Is our own data effectively hidden from us *unless* we go through the additional expense, in time and licensing, of creating a dataverse and populating it?