Tags

, , , , , ,

I encountered a situation in BizTalk 2010 where when using the WCF-Custom send port with a sapBinding that was subscribing to a WCF-CustomIsolated with wsHttpBinding was throwing back a Microsoft BizTalk Server Negative Acknowledgement SOAP messages as below.  These occurred after about 15 minutes of the request.

<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://www.w3.org/2005/08/addressing">
 <s:Header>
 <a:Action s:mustUnderstand="1">http://schemas.microsoft.com/net/2005/12/windowscommunicationfoundation/dispatcher/fault</a:Action>
 </s:Header>
 <s:Body>
 <s:Fault>
 <s:Code>
 <s:Value>s:Receiver</s:Value>
 <s:Subcode>
 <s:Value xmlns:a="http://schemas.microsoft.com/net/2005/12/windowscommunicationfoundation/dispatcher">a:InternalServiceFault</s:Value>
 </s:Subcode>
 </s:Code>
 <s:Reason>
 <s:Text xml:lang="en-NZ">
 <![CDATA[<SOAP:Envelope xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/" SOAP:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"><SOAP:Body><SOAP:Fault><faultcode>Microsoft BizTalk Server Negative Acknowledgment </faultcode><faultstring>An error occurred while processing the message, refer to the details section for more information </faultstring><faultactor>sap://CLIENT=470;LANG=EN;@a/SAPSERVER/00?RfcSdkTrace=False;AbapDebug=False</faultactor><detail><ns0:NACK Type="NACK" xmlns:ns0="http://schema.microsoft.com/BizTalk/2003/NACKMessage.xsd"><NAckID>{90F540BE-B738-45FC-8BC1-01F5FD3577B3}</NAckID><ErrorCode>0xc0c0167a</ErrorCode><ErrorCategory>0</ErrorCategory><ErrorDescription>System.Transactions.TransactionException: The operation is not valid for the state of the transaction. ---&gt; System.TimeoutException: Transaction Timeout
--- End of inner exception stack trace ---
at System.Transactions.TransactionState.EnlistVolatile(InternalTransaction tx, IEnlistmentNotification enlistmentNotification, EnlistmentOptions enlistmentOptions, Transaction atomicTransaction)
at System.Transactions.Transaction.EnlistVolatile(IEnlistmentNotification enlistmentNotification, EnlistmentOptions enlistmentOptions)
at Microsoft.BizTalk.Adapter.Wcf.Runtime.TxnBatch2..ctor(IBTTransportProxy transportProxy, ControlledTermination control, Transaction transaction, ManualResetEvent orderedEvent, Boolean makeSuccessCall)
at Microsoft.BizTalk.Adapter.Wcf.Runtime.BizTalkDeleteTxn.Execute(BizTalkExecutionContext executionContext)</ErrorDescription></ns0:NACK></detail></SOAP:Fault></SOAP:Body></SOAP:Envelope>]]></s:Text>
 </s:Reason>
 <s:Detail>
 <ExceptionDetail xmlns="http://schemas.datacontract.org/2004/07/System.ServiceModel" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
 <HelpLink i:nil="true"/>
 <InnerException i:nil="true"/>
 <Message>
 <![CDATA[<SOAP:Envelope xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/" SOAP:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"><SOAP:Body><SOAP:Fault><faultcode>Microsoft BizTalk Server Negative Acknowledgment </faultcode><faultstring>An error occurred while processing the message, refer to the details section for more information </faultstring><faultactor>sap://CLIENT=470;LANG=EN;@a/SAPSERVER/00?RfcSdkTrace=False;AbapDebug=False</faultactor><detail><ns0:NACK Type="NACK" xmlns:ns0="http://schema.microsoft.com/BizTalk/2003/NACKMessage.xsd"><NAckID>{90F540BE-B738-45FC-8BC1-01F5FD3577B3}</NAckID><ErrorCode>0xc0c0167a</ErrorCode><ErrorCategory>0</ErrorCategory><ErrorDescription>System.Transactions.TransactionException: The operation is not valid for the state of the transaction. ---&gt; System.TimeoutException: Transaction Timeout
--- End of inner exception stack trace ---
at System.Transactions.TransactionState.EnlistVolatile(InternalTransaction tx, IEnlistmentNotification enlistmentNotification, EnlistmentOptions enlistmentOptions, Transaction atomicTransaction)
at System.Transactions.Transaction.EnlistVolatile(IEnlistmentNotification enlistmentNotification, EnlistmentOptions enlistmentOptions)
at Microsoft.BizTalk.Adapter.Wcf.Runtime.TxnBatch2..ctor(IBTTransportProxy transportProxy, ControlledTermination control, Transaction transaction, ManualResetEvent orderedEvent, Boolean makeSuccessCall)
at Microsoft.BizTalk.Adapter.Wcf.Runtime.BizTalkDeleteTxn.Execute(BizTalkExecutionContext executionContext)</ErrorDescription></ns0:NACK></detail></SOAP:Fault></SOAP:Body></SOAP:Envelope>]]></Message>
 <StackTrace>at Microsoft.BizTalk.Adapter.Wcf.Runtime.BizTalkServiceInstance.EndOperation(IAsyncResult result)
at AsyncInvokeEndEndTwoWayMethod(Object , Object[] , IAsyncResult )
at System.ServiceModel.Dispatcher.AsyncMethodInvoker.InvokeEnd(Object instance, Object[]&amp; outputs, IAsyncResult result)
at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeEnd(MessageRpc&amp; rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage7(MessageRpc&amp; rpc)
at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)</StackTrace>
 <Type>Microsoft.BizTalk.Adapter.Wcf.Runtime.BizTalkNackException</Type>
 </ExceptionDetail>
 </s:Detail>
 </s:Fault>
 </s:Body>
</s:Envelope>

Extracting the CDATA from the Fault\Reason\Text and formatting it for readability gives you.

<SOAP:Envelope xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/" SOAP:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
 <SOAP:Body>
 <SOAP:Fault>
 <faultcode>Microsoft BizTalk Server Negative Acknowledgment </faultcode>
 <faultstring>An error occurred while processing the message, refer to the details section for more information </faultstring>
 <faultactor>sap://CLIENT=470;LANG=EN;@a/SAPSERVER/00?RfcSdkTrace=False;AbapDebug=False</faultactor>
 <detail>
 <ns0:NACK Type="NACK" xmlns:ns0="http://schema.microsoft.com/BizTalk/2003/NACKMessage.xsd">
 <NAckID>{90F540BE-B738-45FC-8BC1-01F5FD3577B3}</NAckID>
 <ErrorCode>0xc0c0167a</ErrorCode>
 <ErrorCategory>0</ErrorCategory>
 <ErrorDescription>System.Transactions.TransactionException: The operation is not valid for the state of the transaction. ---&gt; System.TimeoutException: Transaction Timeout
--- End of inner exception stack trace ---
at System.Transactions.TransactionState.EnlistVolatile(InternalTransaction tx, IEnlistmentNotification enlistmentNotification, EnlistmentOptions enlistmentOptions, Transaction atomicTransaction)
at System.Transactions.Transaction.EnlistVolatile(IEnlistmentNotification enlistmentNotification, EnlistmentOptions enlistmentOptions)
at Microsoft.BizTalk.Adapter.Wcf.Runtime.TxnBatch2..ctor(IBTTransportProxy transportProxy, ControlledTermination control, Transaction transaction, ManualResetEvent orderedEvent, Boolean makeSuccessCall)
at Microsoft.BizTalk.Adapter.Wcf.Runtime.BizTalkDeleteTxn.Execute(BizTalkExecutionContext executionContext)</ErrorDescription>
 </ns0:NACK>
 </detail>
 </SOAP:Fault>
 </SOAP:Body>
</SOAP:Envelope>

And the important bit extracted is

System.Transactions.TransactionException: The operation is not valid for the state of the transaction

Hunting around I came across this blog Timeout issue with WCF-SAP BizTalk Adapter. However the solution proposed there seemed a bit too much of an overkill as it would affect all transactions.

Add the below entries in framework64 .net 4.0  machine.config file(C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config) which will change the timeout setting for entire System.Transactions Namespace.
<configuration>
<system.transactions>|
  <machineSettings maxTimeout="00:59:00" />
</system.transactions> 
</configuration>

In the end I found a setting on the send port called Use Transaction and as the error mentioned a Transaction.Exception I though this would be worth trying.

Use Transaction

Un-ticking this did indeed solve the initial error in the test environment.  However I got a different error after 20 minutes (I was using a larger data set in test, hence it taking longer) as per below.

<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://www.w3.org/2005/08/addressing">
 <s:Header>
 <a:Action s:mustUnderstand="1">http://schemas.microsoft.com/net/2005/12/windowscommunicationfoundation/dispatcher/fault</a:Action>
 </s:Header>
 <s:Body>
 <s:Fault>
 <s:Code>
 <s:Value>s:Receiver</s:Value>
 <s:Subcode>
 <s:Value xmlns:a="http://schemas.microsoft.com/net/2005/12/windowscommunicationfoundation/dispatcher">a:InternalServiceFault</s:Value>
 </s:Subcode>
 </s:Code>
 <s:Reason>
 <s:Text xml:lang="en-NZ"><![CDATA[<SOAP:Envelope xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/" SOAP:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"><SOAP:Body><SOAP:Fault><faultcode>Microsoft BizTalk Server Negative Acknowledgment </faultcode><faultstring>An error occurred while processing the message, refer to the details section for more information </faultstring><faultactor>/Client_BT_Application/Service.svc</faultactor><detail><ns0:NACK Type="NACK" xmlns:ns0="http://schema.microsoft.com/BizTalk/2003/NACKMessage.xsd"><NAckID>{3D9A129A-27EC-47ED-A9DC-70454AF4DD59}</NAckID><ErrorCode>0xc0c01675</ErrorCode><ErrorCategory>0</ErrorCategory><ErrorDescription>A request-response for the "CustomRLConfig" adapter at receive location "/Client_BT_Application/Service.svc" has timed out before a response could be delivered. </ErrorDescription></ns0:NACK></detail></SOAP:Fault></SOAP:Body></SOAP:Envelope>]]></s:Text>
 </s:Reason>
 <s:Detail>
 <ExceptionDetail xmlns="http://schemas.datacontract.org/2004/07/System.ServiceModel" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
 <HelpLink i:nil="true"/>
 <InnerException i:nil="true"/>
 <Message><![CDATA[<SOAP:Envelope xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/" SOAP:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"><SOAP:Body><SOAP:Fault><faultcode>Microsoft BizTalk Server Negative Acknowledgment </faultcode><faultstring>An error occurred while processing the message, refer to the details section for more information </faultstring><faultactor>/Client_BT_Application/Service.svc</faultactor><detail><ns0:NACK Type="NACK" xmlns:ns0="http://schema.microsoft.com/BizTalk/2003/NACKMessage.xsd"><NAckID>{3D9A129A-27EC-47ED-A9DC-70454AF4DD59}</NAckID><ErrorCode>0xc0c01675</ErrorCode><ErrorCategory>0</ErrorCategory><ErrorDescription>A request-response for the "CustomRLConfig" adapter at receive location "/Client_BT_Application/Service.svc" has timed out before a response could be delivered. </ErrorDescription></ns0:NACK></detail></SOAP:Fault></SOAP:Body></SOAP:Envelope>]]></Message>
 <StackTrace>at Microsoft.BizTalk.Adapter.Wcf.Runtime.BizTalkServiceInstance.EndOperation(IAsyncResult result)
 at AsyncInvokeEndEndTwoWayMethod(Object , Object[] , IAsyncResult )
 at System.ServiceModel.Dispatcher.AsyncMethodInvoker.InvokeEnd(Object instance, Object[]&amp; outputs, IAsyncResult result)
 at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeEnd(MessageRpc&amp; rpc)
 at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage7(MessageRpc&amp; rpc)
 at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)</StackTrace>
 <Type>Microsoft.BizTalk.Adapter.Wcf.Runtime.BizTalkNackException</Type>
 </ExceptionDetail>
 </s:Detail>
 </s:Fault>
 </s:Body>
 </s:Envelope>

The relevant bit out of this is

A request-response for the "CustomRLConfig" adapter at receive location "/Client_BT_Application/Service.svc" has timed out before a response could be delivered.

After a lot of looking at settings, Googling, trying out different solutions with no success I and finally consulting my colleagues I gave this article a second look “Exchange Patterns for Receive Adapters” which at the end of it mentions Request-Response Message Time-Outs. However the registry keys it mentions MessagingReqRespTTL for the hosts no longer exist in BizTalk 2010, so I initially discounted it but went back for a second look later when one of my colleagues thought this might be the solution.  I verified again that the Registry Keys didn’t exist.  I then used the search term “MessagingReqRespTTL biztalk 2010” and found this blog Response Timeout in BizTalk 2010   This showed a screen shot highlighting a request response timeout on the host settings.  Now I had checked various settings including this but somehow hand’t spotted this.

Response timeout

 

Changing this setting to 60 (which is the maximum allowed) and restarting the Application Pool the Receive Location was running under finally resolved the issue.

Advertisements