BizTalk 2020 – CU4

BizTalk 2020 come out last week, you can find the details here

There are Adapters fixes for 2 x WCF-SAP, WFC-OracleDB, Office365 (Outlook & TMS) & Event Hub.

For the runtime FIX: Excessive CPU usage in kernel mode of Windows when you run multiple BizTalk host instances

A fix for how maps treat whitespaces, this one a fix for where it ignored the default element value if it was only whitespace. FIX: Default element value of spaces or tabs is ignored when using the .NET transform engine in a BizTalk map

A FIX: Turning on auditing causes errors in the BizTalk Server Administration Console

Also of note are the version of components you will need.

Updated External Dependencies

DependencyUpdated version
WinSCP5.19.2
Newtonsoft.Json13.0.1
WindowsAzure.ServiceBus6.2.0
Saxon-HE9.9.1.7
Microsoft.Identity.Client4.36.0
Microsoft.ApplicationInsights2.18.0

If you have BizTalk Developer Tools installed, you must install the latest version of BizTalk Server Visual Studio extension (build 3.13.2.0) in addition to this CU package. The extension can be installed from https://marketplace.visualstudio.com/items?itemName=ms-biztalk.BizTalk or from within Visual Studio through Manage Extensions.

Always check Service Pack and cumulative update list for BizTalk Server for the latest CU for both core BizTalk and Adapters for Enterprise Applications, for HIS check HIS Update Center

Advertisement

Book Review: Migrating to BizTalk Server 2020

Review of the Book, Migrating to BizTalk Server 2020

This book is written by Sandro Pereira, Tom Canter and Lex Hegt, all well known people in the BizTalk sphere. After the dedication about the Authors the introduction starts with an interesting brief history of the various BizTalk Server and then dives straight into what this book is for and what it covers, which unsurprisingly is about how to migrate from earlier version of BizTalk to the latest version Microsoft BizTalk Server 2020. And that is just the introduction.

The next chapters give an Overview of BizTalk Server 2020, what might be some of the reasons why you would want to upgrade; what you need to do to prepare for the migration; the options of doing an in-place upgrade (but also covers why you shouldn’t do this); migrating in a side by side way; and also looks at the option of migrating to Azure. Lastly it has an Appendix that list the BizTalk Dependencies their EOL (End Of Life), which can be one of the justifications you need for an upgrade.

All in all a very comprehensive book and something that you should read if you do have some older BizTalk environments that you are planning to migrate away from. As I’m currently in the midst of a project to migrate of both BizTalk 2010 and BizTalk 2016 to BizTalk 2020, this book has already proven valuable.

This book is available on Shopify

BizTalk 2020 CU 3

BizTalk 2020 CU 3 is now available at, KB5007969 – Cumulative Update 3 for Microsoft BizTalk Server 2020

Fixes for adapters include

  • WCF fix for proxy settings
  • 2 fixes for MQSeries
  • 2 fixes for WCF-SAP
  • 1 fix for the SharePoint adapter
  • 1 fix for the Office365 Outlook Email adapter

Improvements

On the blog post that announced it, they also say

As a quick reminder, for BizTalk Developer Tools please also update/install version 3.13.2.0 of the BizTalk Server Visual Studio extension in addition to installing the CU package. The extension can be installed from https://marketplace.visualstudio.com/items?itemName=ms-biztalk.BizTalk or from within Visual Studio – Manage Extensions. Please note that we have not updated the extension since CU1 so you can skip this step if you have already installed it previously.

BizTalk 2020 CU1 released

BizTalk 2020 CU1 was released on July 28th 2020 as announced here

As per previous version of BizTalk it has a number of SAP/IDOC fixes.

Fixes

  • 4 SAP/IDOX
  • 2 fixes related to the WCF-SQL adapter (order of elements and FILESTREAM operations)
  • 1 for the Oracle DB adapter to handle Database artifacts with nested types in Oracle 18c and later versions
  • Fix for predefined WCF Bindings not taking in account a default Web proxy configuration
  • 2 JSON Encoder fixes (1 one which was fixed for BizTalk 2016 in CU7)
  • Fix for XLTS errors not being reported in XSLT stylesheet
  • Fix for EDI receive location with named SQL instance
  • Fix for ESB Toolkit to use gMSA.
  • Fix to remove dependency on VS 2019 version 16.3.9
  • FIX: Mapper toolbox not working for WCF Workflow Service Application after upgrade to BizTalk 2020

Improvements

  • FTP improvement (macros for Before Put and After Put)
  • Dynamic send ports able to use Officer 365 Outlook adapter types.

Visual Studio Extension

In addition to installing CU package, also update/install new version of BizTalk Server Visual Studio extension (3.13.2.0). The extension can be installed from https://marketplace.visualstudio.com/items?itemName=ms-biztalk.BizTalk or within Visual Studio – Manage Extensions.

As per other CUs, you can find them on the page Service Pack and cumulative update list for BizTalk Server

Max worker threads for SFTP, MQSC or DB in BizTalk

A about March or earlier, the latest rules for BizTalk Health Monitor started to show the following important warning.

BizTalk Health Monitor Important warning – The host instances of ‘ReceivingSFTP’ need more worker threads per cpu to run correctly SFTP Receive Locations. Increase so the “Maximum Worker Threads” property to 500 for these host instances and be sure they are dedicated for this SFTP Receive Locations

I had already been advising clients to run SFTP receive locations on a separate host instance due to issues where it would silently stop receiving requiring a host instance restart to resolve, possibly related to FIX: SFTP adapter receive locations stop working intermittently during higher load in BizTalk Server

BHM rule 637 – Suggest to Increase MaxWorkerThreads for SFTP, MQSC or DB2 Receive Locations

The rule itself also talks about MQSC and DB, but as I didn’t have any receive locations for those it was only warning about SFTP.

I’ve seen that warning even on a BizTalk 2010 environment that has a Custom SFTP adapter installed, possibly because it was also called simply SFTP. However it does not warn about third party SFTP adapters such as the nSoftware one which has a different name.

BizTalk Host Instance Settings – Maximum worker threads.

However, even after changing the Max worker threads to 500 and restarting the Host Instance and re-running BHM, the warning did not go away. Even moving the receive locations running on the nSoftware onto another host instance, and un-configuring the adapter on that host instance didn’t make any difference.

The BizTalk adapter’s default configuration is to poll every 5 seconds. This is rather excessive and should be reduced.

Recommendations for SFTP Receive Locations

  1. Have a dedicated host for receive locations using SFTP
  2. Increase the Maximum Worker threads setting to 500 on that host instance
  3. Check how frequently you poll (the default is 5 seconds), increase the delay if that is not needed.
  4. Put a schedule on to only poll during the periods you need.
  5. Disable message body tracking if it is not needed.

Cumulative Update 7 for Microsoft BizTalk Server 2016

CU 7 for BizTalk Server 2016 has been released, and also Microsoft BizTalk Server 2016 Feature Update 3 with CU7

Important: Do not mix installing CU’s and Feature packs.

This CU contains

  • Four fixes/improvements for the SFTP adapter.
  • One fix for the IBM MQ.
  • One improvement for the the MQSeries adapters.
  • One fix for SAP (NCo) adapter.
  • One fix for the HL7 BizTalk Server Accelerators.
  • Five fixes around applying CU’s, Feature Packs and other setup and configuration changes (not counting the SFTP one which I’ve counted under adapters).
  • And a JSON FIX: JSON encoder unable to handle XML schema with the same name for record and one of its elements which is an issue that has caused grief for a few people.

Note: It has been reported on Twitter that “We have some issues with the new #biztalk CU7. The json encoder has started formatting boolean values as string values, like “true” instead of true.” and “As we wanted the FTP fixes in CU7 we installed it anyway and replaced the jsonextension dll with an older version.”

In that same thread “Little warning, rolling back CU7, does not rollback the jsonextension version.”

Also JSON encoder pipeline doesn’t create json array after installing CU7

Finding the SSH server host key fingerprint — BizMunch

Up till, and including, BizTalk Server 2016, the SFTP adapter included with BizTalk Server, has only supported SSH server host key fingerprints using the RSA algorithm, also called an RSA key fingerprint. Previously, you could connect to an SFTP server using FileZilla, and get the RSA key fingerprint that way. In recent versions, FileZilla will […]

Finding the SSH server host key fingerprint — BizMunch

BizTalk – WinMgt error when refreshing Group Hub – The parameter is incorrect #msbts

Recently when refreshing the Group Hub in BizTalk 2013 R2 in a QAT environment for a client we encountered an error when refreshing in the group hub.

Doing a search on this quickly brought up a threads in Technet, where I had actually posted the accepted answer for, The parameter is incorrect (WinMgmt) and also pointing to an older thread WinMgt error when refreshing Group Hub where a colleague (at that time) Mark Brimble had posted both the question and the answer. He also blogged about it The parameter is incorrect (WinMgt) error when refreshing the BizTalk Group Hub

In all cases this error was resolved by first moving any ports onto another handler and then removing an adapter from a host, restarting the host, testing that the console works and then re-adding them.

In our case we had added two new host instances recently for SFTP adapters to use, both the out of the box Microsoft one and the nSoftware one, and somehow the out of the box one had corrupted some settings on one of the hosts. The trick of course is identifying which adapter, we assumed correctly that it would be the last ones that were changed.

There is also this blog post The parameter is incorrect (WinMgt)” error when refreshing the BizTalk Group in BizTalk Administration Console that gives some steps to achieve the same via the Windows Management Instrumentation Tester if it errors out in the Adapter configuration in the Console.

A possible (unconfirmed) way to identify the offending Adapter is to run the following queries in the WMI Tester

  1. Go to Run and type : wbemtest
  2. Click on connect and give the namespace as : root\MicrosoftBizTalkServer
  3. Click on query and run the cmd

SELECT * FROM MSBTS_ReceiveHandler where MgmtDbServerOverride <> "" or MgmtDbNameOverride  <> ""

And for the send handlers run

SELECT * FROM MSBTS_SendHandler where MgmtDbServerOverride <> "" or MgmtDbNameOverride  <> ""

BizTalk SFTP – Open SFTP connection error

TLDR Update: If you get this error, then it probably is the fact that BizTalk 2013 R2 uses an older version of Putty and doesn’t support the Cipher as per this thread Open SFTP connection error and the solutions as per that are

– Upgrade to BizTalk 2016/2020
– Change SFTP server to have matching algorithms as the SFTP client
– Use external script to do the transfer from/to file location, e g scheduled task psftp.exe
– 3rd party/own SFTP adapter, e g BizTalk community based Blogical SFTP adapter

NiklasEngfelt MSFT (MSFT)

Biztalk 2013 R2 SFTP adapter only supports hmac-sha1, hmac-sha1-96 and hmac-md5 MACs and KEX algorithm

BizTalk 2013 R2 SFTP adapter MACs or KexAlgorithms

The long story as it unfolded.

I was asked to set up some interfaces connecting a on premise FTP server and to send and receive files from two SFTP servers using BizTalk 2013 R2 (CU6).

The first one used a username & password combination and worked without any issues.  The other that used a username & public key however is throwing errors as below.

I can connect with FileZilla from the BizTalk server using the same username & .ppk file that are configured in BizTalk.

I’ve tried various things including the below.

  • Set Accept Any SSH Server Host Key = True 
  • Change Encryption Cipher from Auto to AES (same error), TripleDES (Cipher error)
  • Setting the Folder Path to / or .
  • Changing the Connection Limit = 1
  • Setting Polling Interval = 5 minutes
  • Making sure that it is running on a 64 bit host
  • Googling to try and find others that have encountered this and found a solution

I’ve posted about this on the thread Initialize error for SFTP adapter in BizTalk 2013 R2 which was very similar and on Open SFTP connection error which looks to be an identical error.  There was one fix FIX: “Open SFTP connection” error when you use the SFTP adapter in BizTalk Server, which was part of CU1, so either the issue has regressed, or it is a different issue that throws the same error.

Any clues as to what the cause might be and how to resolve this apart from?

  • Upgrading to BizTalk 2016
  • Using the bLogical adapter
  • Raising an issue with Microsoft (which will have to do if this cannot be resolved)
Receive Location:

The Messaging Engine failed to add a receive location "{ReceiveLocation}" with URL "sftp://{SERVER}:{PORT}/{FILEMASK}*.xml" to the adapter "SFTP". Reason: "Microsoft.BizTalk.Adapter.SftpInvoker.SftpException: Open SFTP connection error.
   at Microsoft.BizTalk.Adapter.SftpInvoker.SftpInvoker.Open()
   at Microsoft.BizTalk.Adapter.Sftp.SftpConnection.OpenUnderlyingConnection(SftpConnectionProperties connectionProperties)
   at Microsoft.BizTalk.Adapters.CommonHelpers.Connection`3.ConnectionPool`3.GetConnection(T1 connectionProperties, TimeSpan timeout)
   at Microsoft.BizTalk.Adapters.CommonHelpers.Connection`3.GetConnection(T connectionProperties, TimeSpan timeout)
   at Microsoft.BizTalk.Adapter.Sftp.SftpRLConfig.ValidateConfiguration(SftpReceivePropertyBag receivePropertyBag)
   at Microsoft.BizTalk.Adapter.Sftp.SftpRLConfig.CreateBinding(RHConfig rhConfig)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.BtsServiceHostBase.InitializeRuntime()
   at System.ServiceModel.ServiceHostBase.OnOpen(TimeSpan timeout)
   at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfReceiveEndpoint.Enable()
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfReceiveEndpoint..ctor(BizTalkEndpointContext endpointContext, IBTTransportProxy transportProxy, ControlledTermination control)


Send Port:

The adapter failed to transmit message going to send port "{SENDPORT}" with URL "sftp://{SERVER}:{PORT}/%SourceFileName%". It will be retransmitted after the retry interval specified for this Send Port. Details:"Microsoft.BizTalk.Adapter.SftpInvoker.SftpException: Open SFTP connection error.

Server stack trace: 
   at Microsoft.BizTalk.Adapter.SftpInvoker.SftpInvoker.Open()
   at Microsoft.BizTalk.Adapter.Sftp.SftpConnection.OpenUnderlyingConnection(SftpConnectionProperties connectionProperties)
   at Microsoft.BizTalk.Adapters.CommonHelpers.Connection`3.ConnectionPool`3.GetConnection(T1 connectionProperties, TimeSpan timeout)
   at Microsoft.BizTalk.Adapters.CommonHelpers.Connection`3.GetConnection(T connectionProperties, TimeSpan timeout)
   at Microsoft.BizTalk.Adapter.Sftp.SftpOutputChannel.Send(Message message, TimeSpan timeOut)
   at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Object[]& outArgs)
   at System.Runtime.Remoting.Messaging.StackBuilderSink.AsyncProcessMessage(IMessage msg, IMessageSink replySink)

Exception rethrown at [0]: 
   at System.Runtime.AsyncResult.End[TAsyncResult](IAsyncResult result)
   at System.ServiceModel.Channels.ServiceChannel.SendAsyncResult.End(SendAsyncResult result)
   at System.ServiceModel.Channels.ServiceChannel.EndCall(String action, Object[] outs, IAsyncResult result)

Exception rethrown at [1]: 
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at System.ServiceModel.Channels.IOutputChannel.EndSend(IAsyncResult result)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfClient`2.SendCallback(IAsyncResult result)".

As we were getting no were fast resolving this, we also tried the nSoftware adapter from a BizTalk 2010 server.  Unfortunately this also didn’t work, but at least gave a meaningful error.

The adapter “nsoftware.SFTP v3” raised an error message. Details “Transmission failed for message “f99728a0-8791-42d7-b27e-c1d53feabd3e”: Error uploading SFTP file: Could not negotiate key exchange algorithm. Remote host supports the following algorithms: “diffie-hellman-group-exchange-sha256″.”.

According to the nSoftware documentation, this is supported (in V4), however the client only has V3 of that adapter (for which I cannot find the documentation so cannot confirm what is has to say about supporting that).  I’ve confirmed it does work with V4 now.

Cumulative Update package 7 for BizTalk Server 2013 R2

Microsoft have just released CU 7 for BizTalk 2013 R2

These CU includes 6 fixes.

I got rather excited when I saw that they had fixed an issue with the WCF-SQL adapter polling.  However the cases the symptoms they describe aren’t quite the same as one we’ve encountered in BizTalk 2010 (so I don’t think this fix is going to help us even when do they get around to patching other BizTalk version).  But it may fix things for others or spur them to look a bit closer at this adapter to fix other issues.

As per nearly every CU, it contains a iDoc/SAP/NCo fix and one for EDI.

There are also two fixes for two-way send ports (one deployment issue and the other a runtime one), and finally a TDDS/database fix.

 

Mike the Tester

A blog about all things testing. Views are my own

Nick's Blog

Biztalk gotcha!

Whatever

FURIOUSLY REASONABLE

A Different Kind of Query

Technology, music, life, and musings

Vierodan IT Space

Spread up technology

BizTalk Server Help book!

Made easy and simple

nethramysooru

Blog on BizTalk Server

Blog Of the Serverless Spirit

Stray Notions on All Things Microsoft Azure and BizTalk

Microsoft Azure/BizTalk_Read

Let's learn and share !

Boutaleb Hicham

Biztalk & Azure Integration Architect : Logic Apps, Serverless, Azure Service Bus, BizTalk Server, and Hybrid Integration

Pieter Vandenheede

Stories, tips & tricks for BizTalk Server, Azure, Data Science & Machine Learning

Glenn Colpaert

Blogging with my head in the (Hybrid) Cloud and my feet on premises! Azure / IoT / Integration

BizMunch

BizTalk blog by Knut Urke

Dragon's BizTalk Blog

A blog about BizTalk, and other integration tools.

The Deployment Bunny

OS Deployment, Virtualization, Microsoft based Infrastructure...

Integration Made Easy

Demystify Integration Development