Hello Joe,
IFD (Internet Facing Deployment) uses SAML:Password as authentication method. It is expected that the App prompts for a Form Based login to request the Username and Password.
From security perspective this is expected and enforced to control the access as we explicitly want users to provide the data.
A pass-through of credentials (like Windows Authentication / WIA) is not recommended. (Changing the IIS configuration which changes the Web.Config / Config of Dynamics CRM Is unsupported Supported extensions (Developer Guide for Dynamics 365 Customer Engagement (on-premises)) | Microsoft Learn)
In addition you should also check the Post-Installation Guide regarding the App for Outlook and OAuth authentication. Post-installation and configuration guidelines for Dynamics 365 Customer Engagement (on-premises) | Microsoft Learn
The final question you need to ask yourself if Usability is more important than Security? ;)
If you want to reduce the amount of login prompts you can modify the token lifetime values on ADFS to generate authentication token for multiple days till a new prompt appears.
Example:
Additional settings (values are in Minutes)
Set-AdfsProperties -SsoLifetime 10080
Set-ADFSRelyingPartyTrust -Targetname "external crm" -TokenLifetime 7200
on top you need to run below statement on CRM
Import-Module 'C:\Program Files\Microsoft Dynamics CRM\Tools\Microsoft.Crm.PowerShell.dll'
$s = Get-CrmSetting OAuthClaimsSettings
$s.SessionSecurityTokenLifetimeInHours = 24 * 7
Set-CrmSetting $s
iisreset