Microsoft BizTalk Server 2016 and Host Integration Server 2016 released!

Microsoft announced the general availability of Microsoft BizTalk Server 2016 and HIS 2016 in December of 2016. We can now leverage:

  • Hybrid Connectivity: BizTalk Server 2016 not only provides comprehensive on-premises application integration capabilities, but can now connect to SaaS applications, use enterprise cloud messaging to interact with partners and vendors and take advantage of the numerous Azure services like Functions, Cognitive Services, Machine Learning and more – all through the Azure Logic Apps adapter available in BizTalk Server 2016.
  • Integration with first-party Microsoft platforms: BizTalk Server 2016 seamlessly integrates with first party platforms such as Windows Server 2016, Visual Studio 2015, SQL Server 2016 and Office 2016 – enabling customers to easily integrate services they are already using.
  • SQL Server AlwaysOn Availability Groups: This is an important feature that provides a highly available solution in Azure, and on-premises.

For more information on What’s new in BizTalk Server 2016, please contact us at info@enkaytech.com.

Microsoft Integration Newsletter – September 2016, Issue 24

EnkayTech was mentioned in Microsoft’s Integration Newsletter. Here’s the extract from the newsletter.

Monitoring on premise BizTalk applications using cloud based Power BI SaaS
In this session, Jaidev Kunjur demonstrated how they implemented a BizTalk application monitoring solution using Azure cloud based Power BI and multiple on premise data sources. The goal was to quickly build an easy to use monitoring solution for the Network Operations Center team (that didn’t know BizTalk) but also provide more visibility into business processes so that Production support staff (who knew BizTalk) could use this as an additional tool to track and debug performance issues. The Power BI dashboard was also used to provide daily reports to business and IT so they could see the KPIs and the value provided by BizTalk that was being used as an enterprise service bus for processing claims, billing, and policies. You can watch the video of the session here.

Please contact us if you’d like to see a live presentation of the work we’re doing with Power BI SaaS and analytics.

Cloud based EDI on Microsoft Azure using Enterprise Integration Pack (EIP)

With Azure and Logic Apps, we can now implement “serverless”, cloud-based business-to-business (B2B) communication, without the need for BizTalk Server or SQL Server. We can quickly and reliably process business transactions, easily track and troubleshoot B2B events and leverage more out-of-the-box connectors. Some key features include:

  • Electronic Data Interchange (EDI) capabilities on Azure: The integration accounts within EIP quickly creates and manages cloud based B2B related artifacts such as maps, schemas, trading partners, agreements and certificates. We can send, receive and troubleshoot B2B transactions across a wide variety of protocols including AS2, EDIFACT and X12.
  • Management capabilities: We can track B2B events in a number of flexible ways like built-in tracking which can be routed to Microsoft Operations Management Suite (OMS) using the out-of-the-box tracking portal. We can easily view and troubleshoot B2B transactions over AS2 and X12 formats (with EDIFACT too).
  • Enterprise connectors: With Logic Apps, we can use pre-built enterprise connectors that make it simple and faster to establish connections with business applications. There are several connectors we can use, for example, MQ Series and SAP ECC connectors. For a full list of all our connectors, please visit the Logic Apps connectors reference.

Microsoft Integration Roadmap

Microsoft has published a new integration roadmap, which can be downloaded from here:

https://www.microsoft.com/en-us/download/details.aspx?id=50408

The article describes how the platform can now support both the Pro integrator and the Citizen integrator based on use cases each has to develop. We can now build increasingly sophisticated solutions (e.g. mobile applications, internet of things, mobile engagement, software as a service, service bus etc.) on Azure using App Services (Web Apps, Logic Apps, API Apps, and Mobile Apps). We also learn more about Microsoft BizTalk Server 2016 and Host Integration Server 2016.

BizTalk Performance Benchmarking or Baseline Performance Analysis

We recently completed benchmarking the performance of our customers BizTalk Server environment. After several runs and tweaks to both BizTalk Server host settings and SQL Server settings, and ensuring all BizTalk SQL jobs were running successfully, we were able to get results that were acceptable to the customer. We recommend running a benchmark before installing any BizTalk applications so that you get a good baseline and also verify that BizTalk, SQL, and storage have been setup correctly. With tracking turned on, we were getting around 175 messages/second. When tracking was turned off, we were getting around 400 messages/second. There was a significant degradation in performance when all tracking is turned on, so our guidance was to keep tracking turned off as much as possible and use BizTalk BAM Portal Views and Activities for tracking message details. SQL Server was running at 99% CPU utilization during the test runs. However, we repeatedly ran the load on and off for an hour and BizTalk would occasionally throttle but recovered gracefully so we weren’t too concerned that SQL Server CPU was running at near max during the test.

BizTalkBenchmark7
BizTalkBenchmark8
BizTalkBenchmark1
BizTalkBenchmark2

BizTalk SQL error “The join order has been enforced because a local join hint” during testing.

While doing performance benchmarking of our customers BizTalk environment, one of the developers ran the bts_CleanupMsgBox script directly via SSMS to clean up history before starting a new test run. We started seeing the following error in the event log soon after.

The following stored procedure call failed: ” { call [dbo].[bts_UpdateMsgbox_TxHost]( ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}”. SQL Server returned error string: “Warning: The join order has been enforced because a local join hint is used.;Warning: The join order has been enforced because a local join hint is used.;Duplicate key was ignored.”.

With BizTalk 2013 R2, we should not run bts_CleaupMsgBox stored procedure directly from SQL Server Management Studio. Instead, run this cleanup using the Health Monitor (which internally calls the bts_CleaupMsgBox stored proc) to purge all entries in MsgBox database and also use Health Monitor to delete and purge for all the other entries (e.g. tracking). This caused the error to go away and restored my environment.

Power BI Dashboards

We recently built Power BI Dashboards hosted in the Cloud for surfacing BizTalk BAM aggregates that are tracking near real-time performance metrics. The Dashboard will be used by the customer to monitor BizTalk application performance for all integrations flowing through the Enterprise Service Bus. For example, we track durations for each integration, message sizes, etc., so that the Network Operations Center (NOC) can quickly review the health of the system when issues are reported. For example, when end users complain that the response times are very poor etc., the NOC can go to the Power BI BAM Dashboard and quickly review historical charts to identify potential root causes and then contact the BizTalk DevOps team to look into a specific application that may not be meeting SLAs. We’re also building alerts so that, for example, if we detect a particular integration is performing poorly, or if messages sizes are larger than expected etc., or percentage change in the number of messages received is greater than or less than expected, we send a notification to the NOC. We’re also able to use Power Query to quickly search for a particular transaction, and can either display it in tabular form, or create a chart as part of the query.

Below are some sample screen shots showing how we’ve used Power Query, Power Maps, and Power View to display BAM aggregates.

PowerBIDashboard1
PowerBIDashboard2
PowerBIDashboard3
PowerBIDashboard4
PowerBIDashboard5
PowerBIDashboard6
PowerBIDashboard7
PowerBIDashboard8

BizTalk ESB send port failure with SQL adapter error “Could not find stored procedure ‘usp_insert_Fault'”.

Reconfigured ESB toolkit since code was missing in SQL Server. This happened because the accounts security settings were incorrect (local were being used instead of domain) and when applying configuration, we saw errors regarding group name incorrect etc. SQL DB ‘EsbExceptionDb’ credentials setup for host account access ‘db_owner’ for host account, bts admin group and bts app user group.

When using select for Oracle within send port, you get a Microsoft.ServiceModel.Channels.Common.MetadataException error.

The reason for this is because the generated binding file created the BtsActionMapping with a default operation name (in this case GET_AGGREGATE_VALUES). But this operation name has to match the operation name used by the logical send port in the orchestration.

In BizTalk Administration Console, go back to the send port’s transport properties and modify the SOAP Action header. Change the operation name to match the logical send port’s operation name in your calling orchestration.