Sending files to Azure Blob Storage from BizTalk Server 2020

Until BizTalk 2016, there was no direct way to communicate with Azure Blob storage to send files. Hence, we had to use WCF-WebHttp adapter to send files to the blob storage. It also required us to write a custom pipeline component to strip the filename from the file path (example – “file.txt” from “c:\files\file.txt” ) before sending it to Azure Blob storage else the file name in Azure blob storage would end up being “c:\files\file.txt” instead of just “file.txt”. However, BizTalk Server 2020 comes in with a built in Azure Blob Storage adapter using which you can directly communicate with the blob storage. This also eliminates the need for a custom pipeline. Click here to read the article.

Next version of BizTalk Server for Windows Server 2019 announced!

Microsoft’s Jon Fancey BizTalk Server vNext announced on Windows 2019 work has begun on the next version called BizTalk Server “vNext” which will include support for upgrades from BizTalk Server 2013 R2 and BizTalk Server 2016. This is welcome news for our customers that have invested in building their mission critical transactional systems on BizTalk Server as we now have a clear understanding of Microsoft’s road map for BizTalk Server on the Windows Platform. Expect to hear more news on BizTalk and the value it can bring to organizations trying to connect their different applications, whether they’re on premises or in the Cloud.

Using the Microsoft BizTalk Server Enterprise Service Bus (ESB) Toolkit to deploy enterprise canonical schema changes with no down time

Most integration projects use an enterprise canonical schema (or CDO – canonical data object), so that messages coming in from (or going out to) an external integration point (e.g. shipping orders coming in and shipping details going out) can be mapped to the CDO and routed internally to different applications that all understand the CDO. The CDO acts as common data structure that all applications reference. By using the BizTalk ESB toolkit, you can avoid having to delete many applications when there’s a change to the CDO. The ESB toolkit allows you to use itineraries that can call maps dynamically (using .NET reflection). In addition, you can use XML and multipart messages so that schemas are not tightly bound to ports and orchestrations. To learn more and view code samples, click here to download a PDF document.

Microsoft BizTalk Server 2016 and Host Integration Server 2016 released!

Microsoft announced the general availability of Microsoft BizTalk Server 2016 and HIS 2016 in December of 2016. We can now leverage:

  • Hybrid Connectivity: BizTalk Server 2016 not only provides comprehensive on-premises application integration capabilities, but can now connect to SaaS applications, use enterprise cloud messaging to interact with partners and vendors and take advantage of the numerous Azure services like Functions, Cognitive Services, Machine Learning and more – all through the Azure Logic Apps adapter available in BizTalk Server 2016.
  • Integration with first-party Microsoft platforms: BizTalk Server 2016 seamlessly integrates with first party platforms such as Windows Server 2016, Visual Studio 2015, SQL Server 2016 and Office 2016 – enabling customers to easily integrate services they are already using.
  • SQL Server AlwaysOn Availability Groups: This is an important feature that provides a highly available solution in Azure, and on-premises.

For more information on What’s new in BizTalk Server 2016, please contact us at info@enkaytech.com.

Microsoft Integration Newsletter – September 2016, Issue 24

EnkayTech was mentioned in Microsoft’s Integration Newsletter. Here’s the extract from the newsletter.

Monitoring on premise BizTalk applications using cloud based Power BI SaaS
In this session, Jaidev Kunjur demonstrated how they implemented a BizTalk application monitoring solution using Azure cloud based Power BI and multiple on premise data sources. The goal was to quickly build an easy to use monitoring solution for the Network Operations Center team (that didn’t know BizTalk) but also provide more visibility into business processes so that Production support staff (who knew BizTalk) could use this as an additional tool to track and debug performance issues. The Power BI dashboard was also used to provide daily reports to business and IT so they could see the KPIs and the value provided by BizTalk that was being used as an enterprise service bus for processing claims, billing, and policies. You can watch the video of the session here.

Please contact us if you’d like to see a live presentation of the work we’re doing with Power BI SaaS and analytics.

BizTalk Performance Benchmarking or Baseline Performance Analysis

We recently completed benchmarking the performance of our customers BizTalk Server environment. After several runs and tweaks to both BizTalk Server host settings and SQL Server settings, and ensuring all BizTalk SQL jobs were running successfully, we were able to get results that were acceptable to the customer. We recommend running a benchmark before installing any BizTalk applications so that you get a good baseline and also verify that BizTalk, SQL, and storage have been setup correctly. With tracking turned on, we were getting around 175 messages/second. When tracking was turned off, we were getting around 400 messages/second. There was a significant degradation in performance when all tracking is turned on, so our guidance was to keep tracking turned off as much as possible and use BizTalk BAM Portal Views and Activities for tracking message details. SQL Server was running at 99% CPU utilization during the test runs. However, we repeatedly ran the load on and off for an hour and BizTalk would occasionally throttle but recovered gracefully so we weren’t too concerned that SQL Server CPU was running at near max during the test.

BizTalkBenchmark7
BizTalkBenchmark8
BizTalkBenchmark1
BizTalkBenchmark2

BizTalk SQL error “The join order has been enforced because a local join hint” during testing.

While doing performance benchmarking of our customers BizTalk environment, one of the developers ran the bts_CleanupMsgBox script directly via SSMS to clean up history before starting a new test run. We started seeing the following error in the event log soon after.

The following stored procedure call failed: ” { call [dbo].[bts_UpdateMsgbox_TxHost]( ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}”. SQL Server returned error string: “Warning: The join order has been enforced because a local join hint is used.;Warning: The join order has been enforced because a local join hint is used.;Duplicate key was ignored.”.

With BizTalk 2013 R2, we should not run bts_CleaupMsgBox stored procedure directly from SQL Server Management Studio. Instead, run this cleanup using the Health Monitor (which internally calls the bts_CleaupMsgBox stored proc) to purge all entries in MsgBox database and also use Health Monitor to delete and purge for all the other entries (e.g. tracking). This caused the error to go away and restored my environment.

BizTalk ESB send port failure with SQL adapter error “Could not find stored procedure ‘usp_insert_Fault'”.

Reconfigured ESB toolkit since code was missing in SQL Server. This happened because the accounts security settings were incorrect (local were being used instead of domain) and when applying configuration, we saw errors regarding group name incorrect etc. SQL DB ‘EsbExceptionDb’ credentials setup for host account access ‘db_owner’ for host account, bts admin group and bts app user group.

When using select for Oracle within send port, you get a Microsoft.ServiceModel.Channels.Common.MetadataException error.

The reason for this is because the generated binding file created the BtsActionMapping with a default operation name (in this case GET_AGGREGATE_VALUES). But this operation name has to match the operation name used by the logical send port in the orchestration.

In BizTalk Administration Console, go back to the send port’s transport properties and modify the SOAP Action header. Change the operation name to match the logical send port’s operation name in your calling orchestration.