RE: Role of 'Sandbox Processing Service' and 'Asynchronous Processing Service'
Think of it like this:
The Sandbox Service is simply a home for the safe execution of Custom .NET code you have written, to stop you from potentially doing anything dangerous to the platform. Whether that custom code is a Sync Plugin, Async Plugin or Custom Workflow Step.
The Asynchronous Processing Service handles the queueing and execution of asynchronous processes.
These can be either Background Workflows or Asynchronous Plugin Steps. Both appear as "System Jobs" in the Dynamics UI.
Example: Background Workflow that contains a custom workflow step written in C#.
The Background Workflow will be handled initially by the Asynchronous Service.
There might be simple logic and updates defined in the Workflow Builder in the UI that it would process.
If I had a Custom Workflow Step written in C# as a step in that workflow definition. When it gets to that point, I would expect the Sandbox Service to handle the execution of that piece of custom code.
I am not sure of what happens exactly where for each process in the platform. Only Microsoft would know.
But I can see the confusion.
Async plugin steps behave an awful lot like background workflows.
Real-Time workflows behave an awful lot like Synchronous plugins (execute in transaction as part of the Execution Pipeline etc...)
Async Plugin Steps when triggered create a special type of workflow job on the Async Queue. That is why they appear as system jobs.
A more "out-there" guess (and this is definitely only a guess) is that real-time workflow definitions are actually executed by some kind of internal Microsoft plugin and that is why they behave an awful lot like plugins.
See the documentation:
https://docs.microsoft.com/en-us/dynamics365/customerengagement/on-premises/developer/create-real-time-workflows?view=op-9-1