InvalidSecurity SOAP error when sending a request to Dynamics with ADFS, IFD/on premise

(0) Share

Report

Posted on by Community Member

There is another post here which talks about this subject, but I looked at all of those answers and none seemed to apply to my case. Specifically I looked at the code here and the SOAP messages seemed to be different from mine. I don't have a <VsDebuggerCausalityData> element for example. And I do have a <SignedInfo> element which is not present in the example.

I have been using this page as a guide. I am using Java / HttpClient, not Javascript. I send the Secure Token request to https://myadfsserver.com/adfs/services/trust/13/usernamemixed like so:

<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://www.w3.org/2005/08/addressing" xmlns:u="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
<s:Header>
<a:Action s:mustUnderstand="1">http://docs.oasis-open.org/ws-sx/ws-trust/200512/RST/Issue</a:Action>
<a:MessageID>urn:uuid:d4447e4f-fc51-4a48-aef6-3483cc99d563</a:MessageID>
<a:ReplyTo>
<a:Address>http://www.w3.org/2005/08/addressing/anonymous</a:Address>
</a:ReplyTo>
<a:To s:mustUnderstand="1">https://myadfsserver.com/adfs/services/trust/13/usernamemixed</a:To>
<o:Security xmlns:o="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" s:mustUnderstand="1">
<u:Timestamp u:Id="_0">
<u:Created>2015-01-27T09:18:04.242Z</u:Created>
<u:Expires>2015-01-27T09:23:04.242Z</u:Expires>
</u:Timestamp>
<o:UsernameToken u:Id="uuid-213b1e6e-aac4-4781-a5d4-151921b05b99-2">
<o:Username>********</o:Username>
<o:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText">********</o:Password>
</o:UsernameToken>
</o:Security>
</s:Header>
<s:Body>
<trust:RequestSecurityToken xmlns:trust="http://docs.oasis-open.org/ws-sx/ws-trust/200512">
<wsp:AppliesTo xmlns:wsp="http://schemas.xmlsoap.org/ws/2004/09/policy">
<a:EndpointReference>
<a:Address>https://mydynamicsserver.com/XRMServices/2011/Organization.svc</a:Address>
</a:EndpointReference>
</wsp:AppliesTo>
<trust:RequestType>http://docs.oasis-open.org/ws-sx/ws-trust/200512/Issue</trust:RequestType>
</trust:RequestSecurityToken>
</s:Body>
</s:Envelope>

And this is the response from the server:

<s:Header>
<a:Action s:mustUnderstand="1">http://docs.oasis-open.org/ws-sx/ws-trust/200512/RSTRC/IssueFinal</a:Action>
<a:RelatesTo>urn:uuid:d4447e4f-fc51-4a48-aef6-3483cc99d563</a:RelatesTo>
<o:Security xmlns:o="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" s:mustUnderstand="1">
<u:Timestamp u:Id="_0">
<u:Created>2015-01-27T09:17:24.275Z</u:Created>
<u:Expires>2015-01-27T09:22:24.275Z</u:Expires>
</u:Timestamp>
</o:Security>
</s:Header>
<s:Body>
<trust:RequestSecurityTokenResponseCollection xmlns:trust="http://docs.oasis-open.org/ws-sx/ws-trust/200512">
<trust:RequestSecurityTokenResponse>
<trust:Lifetime>
<wsu:Created xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">2015-01-27T09:17:24.259Z</wsu:Created>
<wsu:Expires xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">2015-01-27T11:17:24.259Z</wsu:Expires>
</trust:Lifetime>
<wsp:AppliesTo xmlns:wsp="http://schemas.xmlsoap.org/ws/2004/09/policy">
<wsa:EndpointReference xmlns:wsa="http://www.w3.org/2005/08/addressing">
<wsa:Address>https://mydynamicsserver.com/</wsa:Address>
</wsa:EndpointReference>
</wsp:AppliesTo>
<trust:RequestedSecurityToken>
<xenc:EncryptedData xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" Type="http://www.w3.org/2001/04/xmlenc#Element">
<xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#aes256-cbc" />
<KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#">
<e:EncryptedKey xmlns:e="http://www.w3.org/2001/04/xmlenc#">
<e:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-oaep-mgf1p">
<DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
</e:EncryptionMethod>
<KeyInfo>
<o:SecurityTokenReference xmlns:o="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
<X509Data>
<X509IssuerSerial>
<X509IssuerName>CN=Symantec Class 3 Secure Server CA - G4, OU=Symantec Trust Network, O=Symantec Corporation, C=US</X509IssuerName>
<X509SerialNumber>12345</X509SerialNumber>
</X509IssuerSerial>
</X509Data>
</o:SecurityTokenReference>
</KeyInfo>
<e:CipherData>
<e:CipherValue>xxxxx</e:CipherValue>
</e:CipherData>
</e:EncryptedKey>
</KeyInfo>
<xenc:CipherData>
<xenc:CipherValue>xxxxx</xenc:CipherValue>
</xenc:CipherData>
</xenc:EncryptedData>
</trust:RequestedSecurityToken>
<trust:RequestedProofToken>
<trust:BinarySecret>12345</trust:BinarySecret>
</trust:RequestedProofToken>
<trust:RequestedAttachedReference>
<o:SecurityTokenReference xmlns:o="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:k="http://docs.oasis-open.org/wss/oasis-wss-wssecurity-secext-1.1.xsd" k:TokenType="http://docs.oasis-open.org/wss/oasis-wss-saml-token-profile-1.1#SAMLV1.1">
<o:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/oasis-wss-saml-token-profile-1.0#SAMLAssertionID">_123-45-6789-accd-12345</o:KeyIdentifier>
</o:SecurityTokenReference>
</trust:RequestedAttachedReference>
<trust:RequestedUnattachedReference>
<o:SecurityTokenReference xmlns:o="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:k="http://docs.oasis-open.org/wss/oasis-wss-wssecurity-secext-1.1.xsd" k:TokenType="http://docs.oasis-open.org/wss/oasis-wss-saml-token-profile-1.1#SAMLV1.1">
<o:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/oasis-wss-saml-token-profile-1.0#SAMLAssertionID">_123-45-6789-accd-12345</o:KeyIdentifier>
</o:SecurityTokenReference>
</trust:RequestedUnattachedReference>
<trust:TokenType>urn:oasis:names:tc:SAML:1.0:assertion</trust:TokenType>
<trust:RequestType>http://docs.oasis-open.org/ws-sx/ws-trust/200512/Issue</trust:RequestType>
<trust:KeyType>http://docs.oasis-open.org/ws-sx/ws-trust/200512/SymmetricKey</trust:KeyType>
</trust:RequestSecurityTokenResponse>
</trust:RequestSecurityTokenResponseCollection>
</s:Body>
</s:Envelope>

So according to the article you have to, you have to create a SOAP message that will have to be attached to all subsequent requests to https://mydynamicsserver.com/XRMServices/2011/Organization.svc:

1. Get the following values from the Security Token response and stick them in the header of all subsequent requests:

<e:CipherValue>

<xenc:CipherValue>

<o:KeyIdentifier>

2. Create a <Timestamp> with the current date/time, save it as a variable "timestamp".

3. Create a SHA1 hash of the timestamp variable, store it in a variable named "digestValue".

4. Create a <SignedInfo> block, include the value of "digestValue" in it. Save value in a variable named "signedInfo".

5. Base64 encode "signedInfo", save value in variable named "b64SignedInfo".

6. Get the value of the <trust:BinarySecret> element, store it in variable named "serverSecret".

7. Create HmacSHA1 hash of "b64SignedInfo", using "serverSecret". Store value in a variable named "signatureValue".

8. Put value of "signatureValue" in <SignatureValue> of new request.

And that's it. That's enough! So all subsequent requests should have the following SOAP Envelope:

<s:Header>
<a:Action s:mustUnderstand="1">http://schemas.microsoft.com/xrm/2011/Contracts/Services/IOrganizationService/Execute</a:Action>
<SdkClientVersion xmlns="http://schemas.microsoft.com/xrm/2011/Contracts">6.1.1.132</SdkClientVersion>
<a:MessageID>urn:uuid:c89ee2a4-7c37-4c00-8337-26da941bb93a</a:MessageID>
<a:ReplyTo>
<a:Address>http://www.w3.org/2005/08/addressing/anonymous</a:Address>
</a:ReplyTo>
<a:To s:mustUnderstand="1">https://mydynamicsserver.com/XRMServices/2011/Organization.svc</a:To>
<o:Security xmlns:o="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" s:mustUnderstand="1">
<u:Timestamp u:Id="_0">
<u:Created>2015-01-30T11:32:51.614Z</u:Created>
<u:Expires>2015-01-30T11:37:51.614Z</u:Expires>
</u:Timestamp>
<xenc:EncryptedData xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" Type="http://www.w3.org/2001/04/xmlenc#Element">
<xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#aes256-cbc" />
<KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#">
<e:EncryptedKey xmlns:e="http://www.w3.org/2001/04/xmlenc#">
<e:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-oaep-mgf1p">
<DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
</e:EncryptionMethod>
<KeyInfo>
<o:SecurityTokenReference>
<X509Data>
<X509IssuerSerial>
<X509IssuerName>CN=Symantec Class 3 Secure Server CA - G4, OU=Symantec Trust Network, O=Symantec Corporation, C=US</X509IssuerName>
<X509SerialNumber>12345</X509SerialNumber>
</X509IssuerSerial>
</X509Data>
</o:SecurityTokenReference>
</KeyInfo>
<e:CipherData>
<e:CipherValue>xxxxx</e:CipherValue>
</e:CipherData>
</e:EncryptedKey>
</KeyInfo>
<xenc:CipherData>
<xenc:CipherValue>xxxxx</xenc:CipherValue>
</xenc:CipherData>
</xenc:EncryptedData>
<Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
<SignedInfo>
<CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/>
<SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#hmac-sha1"/>
<Reference URI="#_0">
<Transforms>
<Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/>
</Transforms>
<DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/>
<DigestValue>abcd1234</DigestValue>
</Reference>
</SignedInfo>
<SignatureValue>fghi5678</SignatureValue>
<KeyInfo>
<o:SecurityTokenReference xmlns:k="http://docs.oasis-open.org/wss/oasis-wss-wssecurity-secext-1.1.xsd" k:TokenType="http://docs.oasis-open.org/wss/oasis-wss-saml-token-profile-1.1#SAMLV1.1">
<o:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/oasis-wss-saml-token-profile- 1.0#SAMLAssertionID">12345
</o:KeyIdentifier>
</o:SecurityTokenReference>
</KeyInfo>
</Signature>
</o:Security>
</s:Header>
<s:Body>
<Execute xmlns="http://schemas.microsoft.com/xrm/2011/Contracts/Services">
<request xmlns:b="http://schemas.microsoft.com/xrm/2011/Contracts" xmlns:c="http://schemas.microsoft.com/crm/2011/Contracts" xmlns:i="http://www.w3.org/2001/XMLSchema-instance" i:type="c:WhoAmIRequest">
<b:Parameters xmlns:d="http://schemas.datacontract.org/2004/07/System.Collections.Generic" />
<b:RequestId i:nil="true" />
<b:RequestName>WhoAmI</b:RequestName>
</request>
</Execute>
</s:Body>
</s:Envelope>

I follow all of the instructions above, plug in all the values in the correct places, and I still get:

<s:Code>

<s:Value>s:Sender</s:Value>

<s:Subcode>

<s:Value xmlns:a="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">a:InvalidSecurity</s:Value>

</s:Subcode>

</s:Code>

<s:Reason>

<s:Text xml:lang="en-US">An error occurred when verifying security for the message.</s:Text></s:Reason>

And I have no idea why. But for starters ... are all of the steps outlines above correct? Am I doing the hash and hmac on the right values (<Timestamp> and <SignedInfo>, respectively? Or is there perhaps some other detail I am overlooking? Is this documented anywhere?

Many thanks for any help, I've been at this for quite some time ...

*This post is locked for comments

I have the same question (0)

All responses (6)

Answers (1)

Suggested answer

ScottDurow 21 on at

Like (0)

Report

I would suggest using a WCF Security library such as http://axis.apache.org as suggested by msdn.microsoft.com/.../jj602979%28v=crm.5%29.aspx

Hope this helps,

Scott

Was this reply helpful? Yes No
Verified answer

jlattimer 24,562 on at

Like (1)

Report

Here is some sample code I came up with for generating the the SOAP header using Java. Hopefully it will help.

https://github.com/jlattimer/CRMSoapAuthJava

Was this reply helpful? Yes No
Community Member on at

Like (0)

Report

Hello, and thank you so much for posting your Java port of the JS code you wrote over on your blog (which I referenced above). I actually have written code very similar to yours, and it's that code that produced the SOAP messages I posted above. The difference is, your code works! That's right, I can now connect to my ADFS-protected Dynamics back-and and run a "WhoAmI" query.

I'm still looking at your code, trying to find the differences, and the only thing I see that is different is the way you create the <SignedInfo> block. For one thing, you Base64.decode the "binarySecret", something that I don't see you doing in the .js version. Anyway, what's most important is, it works!

My question now is, how do I execute a different query, say, get a list of Contacts? In theory I should be able to simply call:

XRMServices/2011/OrganizationData.svc/ContactSet

passing along the SOAP header with the security info in it, ask for the response to be json (or xml but I'd prefer json), and I should be able to get a list of Contacts, no? I tried simply concatenating "ContactSet" onto the url and the response I am now getting the login page html as the response, the exact same page I was getting when I was only sending my login credentials, without doing the whole ADFS handshake process.

Anyway, I'll have to keep messing with the code. I just wanted to confirm that what I'm trying to do is at least feasible. My understanding was that running the "WhoAmI" query was just to demonstrate that the login process worked. Now that your code got that working, I'd like to work with Entities with json responses.

Anyway, thanks again so much, I was going absolutely nuts with this. I actually tried to get your js version working and had browser/CORS problems -- the browser was always sending OPTIONS requests and the ADFS server (apparently ... I don't own the server) isn't configured to work with OPTIONS requests, so it was choking.

Was this reply helpful? Yes No
jlattimer 24,562 on at

Like (0)

Report

Basically with this approach you should be able to execute any type of SOAP request - in the sample I just used WhoAmI because it is simple and could apply to any environment.

You have a couple choices on how you can figure out which SOAP XMl to replace the WhoAmI request with.

In the CRM SDK there is a project called SOAP Logger with lets you execute C# code using the SDK and it outputs the raw SOAP XML. So that might be helpful if you have some .NET background. Otherwise you can check out Jamie Miley's blog and he has the SOAP XML generated for most of the requests you would need. Look for the JavaScript examples and pull out the XML.

So once you have the SOAP request created you should be able to pass it to my ExecuteSoapRequest function and get back the XML SOAP response - then you'll just need to parse it out and find whatever bit of data you need.

The JS version will only work inside a mobile application that doesn't use a standard browser where the cross-origin requests are allowed.

Was this reply helpful? Yes No
Community Member on at

Like (0)

Report

Uh-oh. I mean, thanks again! But my understanding this whooooole time was that after I got went through the whole ADFS login process -- send my credentials, request security token, receive security token -- that I would be able to construct the SOAP header with all of the correct info -- X509 info, ciphers, signed info, key value -- and make a simple call to the oData endpoint like so:

myserver.com/.../OrganizationData.svc/ContactSet?$filter=Address1_City eq 'Redmond'

That's what I've been doing up until now, on a non-ADFS secured Dynamics server. It was much simpler of course; there was no security token request. I would just send my credentials, and it would work.

On this blog (bingsoft.wordpress.com/.../odata) the guy says:

"I use the HttpWebRequest class to perform SOAP requests to perform the authentication part of this integration ... Once you get authenticated and obtain the cookies you are free to fire REST calls off by simply providing the cookies along with the request."

Ever worked with something like that? I think working with the oData endpoint is simpler than working with the SOAP endpoint. But it seems I've been wrong about one thing: you have to use SOAP to get the security info but it seems you can't just send the SOAP header to the oData endpoint (duh, I'm totally new to Dynamics as you can tell). You have to send it to the SOAP endpoint, which means you have to work exclusively with SOAP. Which I'd like to avoid.

Once again, thanks again for the explanation, and of course the code. It's nice to see a server response that isn't 500 - Internal server error.

Was this reply helpful? Yes No
Community Member on at

Like (0)

Report

Hello again. I accepted your answer above, with the code sample. I found the problem with my code (I wasn't properly formatting the Timestamp 'created' and 'expires' dates properly, among other things) and now have it working, the same as yours, only it uses HttpClient instead of HttpURLConnection.

As to my question below, I'm going to open a new issue, because my initial problem was solved. Now I am trying to call the oData REST endpoint, passing it the ReqClientId cookie returned with the WhoAmI request, and I am once again getting the login screen.

Anyway, that's a problem for another issue! I hope to be able to use the oData endpoint. I really, really don't want to use SOAP, even though I could use Axis to automatically generate the SOAP requests based on the wsdl. I find SOAP to be clunky.

Of course, I'd love to be able to use OAuth as well, and it seems to be theoretically possible but there isn't much info on the 'net about using it with Dynamics and I don't think the Dynamics admins have it configured.

Thanks again!

Was this reply helpful? Yes No