If you're serious about finding the root cause of this issue, you're going to have to start to divide and conquer the problem.
Performance issues in a complex n-tier system like Dynamics AX are not always easy to figure out. However, most performance problems related to some resource being severely constrained.
Most of the time the resource is either CPU, I/O, memory, or network. In an n-tier system like AX, the resource can be either on the client machine (RDP server in this case), the AOS, or the database, and in some cases there can be quite a few machines in that mix. In a virtualized environment, you have to consider the host(s) as well. Yes, a host can have a severely saturated CPU and it will be felt by the virtual machines, and not be immediately obvious from any performance measurement done within the virtual machine itself.
Because your problem seems somewhat random, start by setting up Perfmon to do standard logging to disk. There are many good articles on technet regarding standard performance metrics to log. This way if you can't figure it out while the problem is actually happening to you, there's a chance that enough information was captured that you can review it after the fact.
You can gain some good clues by commonality. For example, when one client is suffering, are others as well? If so, what do they share? Are they on the same Remote Desktop server, or are they connected to the same AOS? Or, are they running reports that share the same data at SQL Server. In each case, you can start to focus a little closer on a suspect machine based on how performance issues are shared among many users.
Once you have figured out on what tier the problem lives, and what resource is constrained, you stand a good chance of solving it. For example, if you knew for sure the problem was SQL Server and a poorly performing query, there are many ways to find, analyze, and improve the performance of SQL Server and specific queries.
Start by ruling out what is not the problem until you understand better where it lies in your infrastructure or application.