hey!
Interesting approach. As the records are assigned to the same user (despite different team), there shouldn't be any access lost to the previous record. However, you should test this on a sandbox before doing the Production change. Perhaps the best approach would be that both teams exists for some period of time.
One important thing: when you create an AAD Team on Dynamics, the users are not populated immediately with the information from Azure. Instead, it will be populated as the users login to the environment.
best regards,