If a user as no permissions to a shared mailbox in exchange and that mailbox as a queue linked to it in Dynamics, any user that can see that queue will be able to send emails from that queue.

How can or can we prevent a user that as no permission to a mailbox in exchange to send email even if he is a member of the queue and no permission in exchange?