Can I have 100 BizTalk Server Host Instances?
06 December 13 10:26 AM | Johan Hedberg | 1 comment(s)

On occasion you see these questions. Can I run X number of host instances? What will happen? Without diving deep into the reason why you think you need to, or the details of what is happening inside BizTalk Server when you do, I will present some results of doing that.

First, I needed a machine to play around with. I also wanted a reasonably powerful machine, so I where better to go than to Windows Azure? I selected a BizTalk Server 2013 Evaluation Edition pre-configured image.

image

Choose the Extra Large size – that’s 8 cores and 14 GB of RAM.

image

When that machine was provisioned for me, this was how the performance of it looked:

image

Now I am fully aware that viewing only the Task Managers idea of the processor is a very limited view of “performance” but I am purposefully using that view so that you, the reader, understand that this is NOT intended to be a DeepDive. It is merely an indication.

So, I configured a script I have to create 100 hosts and host instances for me, along with handlers for FILE Adapter for those hosts. But so far no ports and no traffic. This is how that looked.

image

You can see the Processor churning away at about 20% utilization, while Memory is largely unaffected.

Taking it one step further I wanted to make sure that the host instances actually had something to check for, so I created 1 receive port and 100 send ports. One send port for each of the host instances (the Send Port has that hosts handler for the FILE transport).

image

That put the machine under a little more pressure. Obviously it is doing something more when it actually has ports configured. Processor is at ~60%. Memory again not really affected.

Remember now, this is all just doing nothing at this point. Let’s see what happens if we actually do something:

image

That piece of the graph marked in blue above, that’s when I dropped a file into the receive port and receive location that all the 100 send ports (one for each of the hosts) were subscribed on. It went to 100% quickly. All files went through quickly though, and the event didn’t last long enough for BizTalk to start throttling.

So what about the Memory though. Is that really not affected? How much does a BizTalk instance use and wont 100 of them make an impact? Well, it turns out that each host instance will, at this particular point, only use about 20 MB.

image

But ok, 100 host instance is a lot. What about 50? Still the same config as above, but only 50 host instances are started.

image

A bit jagged, but still, running only 50 takes Processor utilization down from 60% to 20%. Now what if we send something through?

image

A short spike when sending the document through to its 50 subscribers.

Taking a little bit of a deeper look at that spike we can see that SQL Server is the main contributor to that spike.

image

Hmmm. Ok, “But” says the customer, I want all of this to be Low Latency. 50 ms polling. Do it!

image

CPU goes up from ~20% to a little under 40%. But it also changes characteristics. When it before was jagged, it now becomes more or less a straight line. The processor does not get to rest, and it does not get spikes in the same way.

Unless of course you send a message through, in which case it does again spike.

image

But it’s just a very short spike. Nothing at all in the way that you can or cannot say that this would be a bottleneck based on this simple test. I have not done any extended tests to see what the MST would be for this machine.

What if we raise the number of host instances just slightly? To 75.

image

See that marked point in time above? That’s where I enabled the host instances. Proc goes to about 60%. 100 again you say? Let’s try it…

image ¨

Again a visible increase in power needed. And the proc now at about 70% and pretty flat.

image

Sending a message through again spikes it.

image

And SQL is the thing that is grabbing most of that processing power.

image

So there you go. That’s what would happen if you run 100 hosts and 100 host instances on a single machine, and if you put them all to poll the database at 50 ms.

This was done on a Windows Azure Virtual Machine with the BizTalk Server Evaluation image in its default configuration. I did nothing to it. No updates, no tuning, no alterations. I know for certain that I can improve the performance of what we see above.

You can draw your own conclusion from the above. My own conclusion is “in-conclusive” ;). That is - I can see that running 100 host instances with 50ms polling on a machine where both BizTalk and SQL share the same machine and the machine is not optimized does not bring down the machine by the share volume of polling alone. However when running even simple traffic through we hit the roof. If this load would be placed on a distributed environment, with SQL and BizTalk on separate machines and SQL with a more optimized storage architecture etc, BizTalk with other configuration such as Global Tracking disabled etc, I should think that the scenario is doable.

I would however highly question why you think you need 100 hosts and 100 host instances. There are a lot of functionality in BizTalk, for example SSO Affiliate Applications, that solve some of the reasons why you would think you need that many. My recommendation is certainly not to go there unless absolutely necessary.

HTH,
/Johan

BizTalk Server 2013 Developer Edition arrives
05 November 13 01:47 PM | Johan Hedberg | with no comments

Microsoft BizTalk Server 2013 Developer Edition New SKUs and Changes

Effective November 1, 2013, Microsoft BizTalk Server (BTS) 2013 Developer Edition licenses will be available under the Developer Tools license model in the Open, Select Plus and Worldwide Government Partner programs. Previously offered as a free download in prior BTS versions, the new BTS 2013 Developer Edition offers the full functionality of the
BTS 2013 Enterprise Edition, licensed for development and test use only, and includes the newly released Host Integration Server (HIS) 2013 software.

Source: http://microsoft.also.ch/fileadmin/Dateien/Dokumente/Pricelist/November_2013_Price_List_Guide.pdf, http://www.enpointegov.com/newsletter/october2013

You will also find it if you run the Microsoft License Advisor at http://mla.microsoft.com/.

image

The Developer Tools license model

Previously with BizTalk Server 2010 you have bought BizTalk Server Licenses for servers only (Branch, Standard and Enterprise). The Developer Edition was free. With 2013 you must purchase this license per user, if they do NOT have MSDN. This is a per user license.

You must acquire a license for each user you permit to access or use the software. You may install any number of copies on any number of devices for access and use by one user to design, develop, test and demonstrate programs. Only licensed users may access the software.

Source: https://www.microsoft.com/licensing/about-licensing/product-licensing.aspx

My interpretation of “access the software” (but I am not a license expert!) is that it is ok for BizTalk Server to exist in a test environment where it routes traffic and perform integrations to and from other systems that at their end have users, devs or admins that are not licensed. It is also ok for other users of the same server to access the server where BizTalk is installed to, for example, administer Windows (as long as that in itself is properly licensed). The limiting factor are the users that in some form access the BizTalk Server software itself. Such as deploy, configure, administer or in other ways interact with the BizTalk Server GUIs or services.

Price Lists

https://mspartner.microsoft.com/en/us/pages/licensing/price-lists.aspx

Target prices seems to be around $37 Estimated Retail Price (ERP). https://mspartner.microsoft.com/en/us/Pages/Licensing/Downloads/open-license-estimated-retail-price-list.aspx

image

I have also seen $36 for price level C. https://mspartner.microsoft.com/en/us/Pages/Licensing/Downloads/open-license-level-c-estimated-retail-price-list.aspx

MSDN

To those with an MSDN subscription (http://msdn.microsoft.com/subscriptions) the list of subscription types that has access to this seems to be quite large:

VS Pro with MSDN (VL)
VS Pro with MSDN Premium (Empower)
VS Pro with MSDN Premium (MPN)
VS Test Pro with MSDN (Retail)
VS Test Pro with MSDN (VL)
VS Ultimate with MSDN (MPN)
VS Ultimate with MSDN (NFR FTE)
VS Ultimate with MSDN (Retail)
VS Ultimate with MSDN (VL)
BizSpark
BizSpark Admin
Designer AA
DreamSpark Premium
DreamSpark Standard
MSDN Platforms
VS Premium with MSDN (MPN)
VS Premium with MSDN (Retail)
VS Premium with MSDN (VL)
VS Pro with MSDN (Retail)

See screenshot:

image

How do I get it?

Summarizing from above you would either have an eligible MSDN subscription, or you get it through traditional volume licensing channels.

HTH,
/Johan

BizTalk Send Ports, WS-Addressing, ClientVia and non-http prefixed To headers, Part 2
22 February 13 06:54 AM | Johan Hedberg | with no comments

In a previous post I explained how we had a need to use the WS-Addressing To header to send a non-http prefixed URI, such as urn:company/path/subpath/Service1, and how that was supported, after a fashion, out of the box in BizTalk Server. It did however come with the limitation of not being able to edit the WCF config in the BizTalk Server Administration Console GUI once you loaded it from a binding file. I don’t like limitations.

In this post I’ll show you how you can create a very simple WCF behavior to help you to set a RemoteAddress EndpointAddress Uri to be able to accomplish the same thing, while still being able to continue to edit the port configuration.

WCF behaviors allows you to intercept and inspect or alter messages or metadata, or in other ways modify the runtime behavior or processing of the same as they are sent or received. In this case we are creating a client behavior.

The behavior is in essence very very simple, it’s only purpose is to alter the endpoint address at runtime. The place where I choose to implement this is in the ApplyClientBehavior method of the IEndpointBehavior interface.

void IEndpointBehavior.ApplyClientBehavior(ServiceEndpoint serviceEndpoint, ClientRuntime behavior)
{
    serviceEndpoint.Address = new System.ServiceModel.EndpointAddress(this.Uri);
}

Incidently, I borrowed this implementation with pride from the ClientVia behavior that comes with the .NET Framework. Apart from the fact that that behaviors sets the ClientRuntime.Via property and this sets the ServiceEndpoint.Address property the implementation is very close to exactly the same.

This allows you to configure BizTalk in the following manner.

The “Address (URI)” property can be set to anything (as long as it is http or https prefixed), since it will later be overridden.

image

In the behaviors section we now have two behaviors, clientVia:

image

and the new one I created, which I called remoteAddress:

image

clientVia is configured as the actual URI of the service we need to call, while the remoteAddress behaviors is configured with the value we want to have in the To header.

The solution contain three files of interest.

image

The App.config file holds a snippet of configuration that needs to be placed in machine.config or in the BizTalk WCF-Custom WCF extension part of the Send Handler of the host that handles the WCF calls being made. It exists to make the behavior available to the BizTalk process and looks like this:

<extensions>
  <behaviorExtensions>
    <add name="remoteAddress" type="bLogical.BizTalk.WSAHelper.RemoteAddressElement, bLogical.BizTalk.WSAHelper, Version=1.0.0.0, Culture=neutral, PublicKeyToken=3672865486d21857"/>
  </behaviorExtensions>
</extensions>

It points to the RemoteAddressElement class, whose responsibility it is to point out the type of the behavior and create new instances.

The RemoteAddressBehavior then in turn does the already above explained logic.

The project and code is available here.

I suppose a custom pipeline component setting the Address, or a Dynamic Send port for easier cases of configuration might also do the trick.

Filed under: ,
BizTalk Send Ports, WS-Addressing, ClientVia and non-http prefixed To headers
21 February 13 10:04 PM | Johan Hedberg | with no comments

Through WS-Addressing services can require a certain value in the ws-addressing <wsa:to> SOAP header. WCF has full support for this. This support is inherited by the WCF-Adapters. When using WS-addressing in a BizTalk Server Send Port what you enter in the “Address (URI)” of the WCF adapters configuration will also be the value that ends up in the <to> header.

Like so:

image 

This will produce the following. Other headers and body removed for clarity.

<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://www.w3.org/2005/08/addressing">
  <s:Header>
    <a:To s:mustUnderstand="1">http://localhost:8990/Service1</a:To>
  </s:Header>
  <s:Body>
    ...
  </s:Body>
</s:Envelope>

If you need to have a different value in the <to> header than the actual address that the service is hosted at it becomes a little bit trickier. You need to use the WCF-Custom adapter and add the ClientVia behavior. The value configured as the “Address (URI)” will still end up as the value in the <to> header, but the actual URI that the call will be made to will be the value you configure in the ClientVia’s viaUri property.

Like so:

image image

This will produce the following (again cleaned for clarity):

<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://www.w3.org/2005/08/addressing">
  <s:Header>
    <a:To s:mustUnderstand="1">http://somedummyvalue/</a:To>
  </s:Header>
  <s:Body>
    ...
  </s:Body>
</s:Envelope>

Now, as long as the value that you want in the <to> header is http or https (depending on the bindings security settings) then you are fine. However, if you end up needing to have a value in your <to> header that looks for example like this: urn:company/path/subpath/Service1, then you’re in trouble.

You will get an error dialog saying that The specified address is invalid. Invalid address scheme; expecting “http” scheme.

image

Why? Because BizTalk Server in its diligence to help you configure things correctly will force you to enter an URI that is prefixed with either http or https (again, depending on the security setting of the binding). There is no way for you to configure a non-http prefixed port in the adapter GUI (that I know of).

A colleague of mine, Gustav Granlund, experienced this issue and found a solution: you can coup it in through a binding file, and the runtime will accept it. Doing this you can enter a “Address (URI)” like so:

image

And you are then able to send a message that looks like this (to an address of http://localhost:8990/Service1):

<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://www.w3.org/2005/08/addressing">
  <s:Header>
    <a:To s:mustUnderstand="1">urn:company/path/subpath/Service1</a:To>
  </s:Header>
  <s:Body>
    ...
  </s:Body>
</s:Envelope>

The caveat is that after having done this you cannot open and change WCF adapter settings in the port through the administration console GUI and keep the urn:company/path/subpath/service1 style URI. But, as mentioned, BizTalk will happily run with it. In a follow up post I examine another option.

HTH,
/Johan

Filed under: ,
BizTalk Server futures–Presentations from TechEd North America
26 June 12 12:43 AM | Johan Hedberg | 1 comment(s)

I have already relayed this information to so many, and given the links to more, that I though I’d put them up here for easy access. There is much and more to be written about the content, but I’ll settle for this. Information has been available around BizTalk Server 2010 R2 for some time, but it got much more real and saw some things unveiled not previously mentioned or detailed. In short:

Application Integration Futures: The Road Map and What's Next on Windows Azure: Video Slides

Building Integration Solutions Using Microsoft BizTalk On-Premises and on Windows Azure: Video Slides

Filed under: , , ,
Creating a new BizTalk machine from a sysprep image in Windows Azure Virtual Machines, and making it work
24 June 12 05:47 PM | Johan Hedberg

In its simplest single machine using default configuration BizTalk Server 2010 comes with a sample of how to use sysprep, that resides in the C:\Program Files (x86)\Microsoft BizTalk Server 2010\SDK\Samples\Admin\Sysprep folder. It uses an unattended installation answer file that among other things tells windows to run a number of scripts that updates the database, SSO, BAM etc, where references to the machine name exists. Full description of that “sample” is described here.

Windows Azure on the other hand does not (as far as I can figure out) support running sysprep and supplying an unattend file. The way to properly sysprep a Windows Server VHD running on Windows Azure Virtual Machines is described here. If developing a machine locally the same general steps apply.

The problem with this is that they clash, they mix like cream and lemon - not at all. (I am sure there are ways to combine these, but its not openly apparent how, but if someone knows feel free to Comment on that).

To make a BizTalk Server image created on Windows Azure fully usable we can run the scripts and commands that the unattended configuration would have. When running these scripts you need to know the name the computer had when sysprep was used and the image captured.

There are three steps to this:

  1. Update SQL Server User Groups 
  2. Update SQL Server metadata
  3. Update BizTalk Server (including SSO and other components)

Update SQL Server User Groups

Simply rename the groups.

image

Updating SQL Server Metadata

The problem when logging on to SQL Server after the name change is that you no longer have access with the administrative account, Administrator, since this is really oldmachine\administrator. So you need another administrative account, ie SYSTEM to fix your login for you.

One way to run as system and do the things described for step 1 is to add a Scheduled Task. Set it to run as SYSTEM, since that account will be sysadmin in the database and set it to run a script that fixes the things necessary, ie:

REM UpdateSqlLogins.bat

SET oldcomputername=JEHBTS5
pushd %programfiles(x86)%\Microsoft BizTalk Server 2010\SDK\Samples\Admin\Sysprep\scripts

SqlCmd -s . -d master -A -Q "sp_dropserver %oldcomputername%" -o UpdateSQLLogins.log
SqlCmd -s . -d master -A -Q "sp_addserver %computername%, local" -o UpdateSQLLogins.log
SqlCmd -s . -d master -A -Q "drop login [%oldcomputername%\Administrator]" -o UpdateSQLLogins.log
SqlCmd -s . -d master -A -Q "create login [%computername%\Administrator] from windows" -o UpdateSQLLogins.log
REM SqlCmd -s . -d master -A -Q "EXEC sp_changedbowner [%computername%\Administrator]" -o UpdateSQLLogins.log
SqlCmd -s . -A -Q "EXEC sp_addsrvrolemember @loginame = N'%computername%\Administrator', @rolename = N'sysadmin'" -o UpdateSQLLogins.log

popd

Update BizTalk Server

As the start of the article mentions, Microsoft has briefly covered applying sysprep to a BizTalk Server machine, but since that procedure does not map to this we need to take a somewhat different approach.

  1. Update the file UpdateInfo.xml. In particular remove the <DeploymentUnit Name="Alert"> section, since I am not using BAM Alerts.
  2. Create a file called Replace.vbs and insert the following code.
    'Usage: Replace.vbs <text file to open> <string to be replaced> <string to replace it with>
    
    Dim sOutput, reader, readerStream, writer, writerStream, Wshell
    
    set WshShell = WScript.CreateObject("WScript.Shell")
    
    Set reader = CreateObject("Scripting.FileSystemObject")
    set readerStream = reader.OpenTextFile(WScript.Arguments(0), 1, , -2)
    
    sOutput = Replace(readerStream.ReadAll, WScript.Arguments(1), WScript.Arguments(2))
    sOutput = Replace(sOutput, UCase(WScript.Arguments(1)), WScript.Arguments(2))
    sOutput = Replace(sOutput, LCase(WScript.Arguments(1)), WScript.Arguments(2))
    
    readerStream.Close
    
    Set writer = CreateObject("Scripting.FileSystemObject")
    Set writerStream = writer.CreateTextFile(WScript.Arguments(0), true, False) ''Write the file in ASCII
    writerStream.Write(sOutput)
    writerStream.Close
  3. Create a file called BizTalkSysPrepRestore.bat and place the following code in it.
    REM BizTalkSysPrepRestore.bat
    
    REM SET /P oldcomputername=<test.txt
    SET oldcomputername=JEHBTS5
    pushd %programfiles(x86)%\Microsoft BizTalk Server 2010\SDK\Samples\Admin\Sysprep\scripts
    
    REM First run UpdateSQLLogins.bat. Once. To provision the account as Admin to be allowed to do the below.
    
    net stop BTSSvc$BizTalkServerApplication
    
    net stop RuleEngineUpdateService
    
    net stop ENTSSO
    
    cscript.exe "Replace.vbs" "UpdateInfo.xml" $(NEWCOMPUTERNAME) %computername%
    cscript.exe "Replace.vbs" "UpdateInfo.xml" $(OLDCOMPUTERNAME) %oldcomputername%
    cscript.exe "Replace.vbs" "UpdateInfo.xml" $(OLDCOMPUTERENAME) %oldcomputername%
    
    cscript.exe "UpdateRegistry.vbs" "UpdateInfo.xml" > "UpdateRegistry.log"
    cscript.exe "UpdateDatabase.vbs" "UpdateInfo.xml" > "UpdateSqlServerDatabase.log"
    cscript.exe "UpdateBAMDb.vbs" "UpdateInfo.xml" > "UpdateBAMDb.log"
    
    "UpdateSSO.cmd" > "SSO.log"
    
    REM Update path to SSOXXXX.bak or place in local folder with this name
    "%CommonProgramFiles%\Enterprise Single Sign-On\ssoconfig.exe -restoreSecret SSO.bak
    
    net stop SQLAgent$SQLEXPRESS
    net stop sqlserveragent
    
    net stop MSSQL$SQLEXPRESS
    net stop mssqlserver
    
    cscript.exe "Replace.vbs" "%programfiles(x86)%\Microsoft BizTalk Server 2010\Tracking\bm.exe.config" %oldcomputername% %computername%
    
    net start mssqlserver
    net start MSSQL$SQLEXPRESS
    
    net start sqlserveragent
    net start SQLAgent$SQLEXPRESS
    
    net start RuleEngineUpdateService
    
    net start BTSSvc$BizTalkServerApplication
    
    popd
    
    pause
  4. Run the BizTalkSysPrepRestore.bat as Administrator.
  5. Open SQL Server Management Studio and runt the following SQL script to update Agent jobs.
    USE [msdb]
    GO
    EXEC msdb.dbo.sp_update_job @job_name=N'Backup BizTalk Server (BizTalkMgmtDb)', @owner_login_name=N'sa'
    EXEC msdb.dbo.sp_update_job @job_name=N'CleanupBTFExpiredEntriesJob_BizTalkMgmtDb', @owner_login_name=N'sa'
    EXEC msdb.dbo.sp_update_job @job_name=N'DTA Purge and Archive (BizTalkDTADb)', @owner_login_name=N'sa'
    EXEC msdb.dbo.sp_update_job @job_name=N'MessageBox_DeadProcesses_Cleanup_BizTalkMsgBoxDb', @owner_login_name=N'sa'
    EXEC msdb.dbo.sp_update_job @job_name=N'MessageBox_Message_Cleanup_BizTalkMsgBoxDb', @owner_login_name=N'sa'
    EXEC msdb.dbo.sp_update_job @job_name=N'MessageBox_Message_ManageRefCountLog_BizTalkMsgBoxDb', @owner_login_name=N'sa'
    EXEC msdb.dbo.sp_update_job @job_name=N'MessageBox_Parts_Cleanup_BizTalkMsgBoxDb', @owner_login_name=N'sa'
    EXEC msdb.dbo.sp_update_job @job_name=N'MessageBox_UpdateStats_BizTalkMsgBoxDb', @owner_login_name=N'sa'
    EXEC msdb.dbo.sp_update_job @job_name=N'Monitor BizTalk Server (BizTalkMgmtDb)', @owner_login_name=N'sa'
    EXEC msdb.dbo.sp_update_job @job_name=N'Operations_OperateOnInstances_OnMaster_BizTalkMsgBoxDb', @owner_login_name=N'sa'
    EXEC msdb.dbo.sp_update_job @job_name=N'PurgeSubscriptionsJob_BizTalkMsgBoxDb', @owner_login_name=N'sa'
    EXEC msdb.dbo.sp_update_job @job_name=N'Rules_Database_Cleanup_BizTalkRuleEngineDb', @owner_login_name=N'sa'
    EXEC msdb.dbo.sp_update_job @job_name=N'TrackedMessages_Copy_BizTalkMsgBoxDb', @owner_login_name=N'sa'
    GO
  6. You should now be set to run BizTalk.

In Summary

I am sure there are better ways to do some of these things. This was a PoC of exactly this, and nothing else. I know there are ways to simplify and automate further. I am sure this is not the best possible solution, but it is one possible solution. It is also a work in progress. I do not have time to take it further right now, but I still wanted to release this post to perhaps help someone else along.

I have hardcoded the old computer name in these scripts, you need to replace that with whatever your original machines name was when you created the image.

There are a couple of things here where I have taken the easy road. All services are run by Administrator, which of course is an administrator. The same is a a member of the groups SSO Administrator, SSO Affiliate Administrator, BizTalk Server Application Users and BizTalk Server Administrator, as well as being assigned sysadmin in the scripts above. So not the ideal best practice security there, and it was outside my scope to make it that.

For the moment, you can also create an image from a virtual machine in Windows Azure without using sysprep. This will work as well, but it’s a quirk, not really what we want here for several reasons.

Since the sysprep support for BizTalk Server is a SAMPLE, I am not sure how “supported” using sysprep on BizTalk Server really is at the moment. The team will have to solve this on their way to offering BizTalk Server as stock images on Windows Azure Virtual Machines, but we are not there yet.

SQL Server does not officially support sysprep in this manner. Instead another procedure is detailed, which includes not fully installing SQL at all before you sysprep. This does not seem to have changed with SQL Server 2012. It will be interesting to see how the team works around this limitation for BizTalk Server 2010 R2. I am guessing that is what the “provisioning” stage that virtual machines go through is for – finalizing installations.

Perhaps not really installing SQL, and following that products offical way to do it, as well as just installing but not configuring BizTalk Server, is the easiest way to do it at the moment. You be the judge.

Further reading

Steef-Jan has a post that gives a better overview of the procedure of creating a virtual machine from an image, without going deep on the sysprep step as I did.

Saravana has a series of posts, starting here, that takes us through setting up a full multi-machine environment, including AD and Virtual Networks.

SQL Server VC++ Installation voes
23 June 12 12:30 PM | Johan Hedberg | 19 comment(s)

I’ve installed SQL Server any number of times over any number of versions, but I have never had this problem before, and I am not sure why I got it now. However, since searching the web gave me very little in the way of a direct, working, solution I thought I’d write mine down. I was using Windows Server 2008 R2 Datacenter edition, SP1, patched to May 2012 standard, aka The Windows Server 2008 R2 image available in the Windows Azure Virtual Machine preview and installing SQL Server 2008 R2 Developer edition onto it. Now, I don’t think they are related, but I am not ruling it out that there is an issue in some way with that image. Since I am not doing anything but starting the image and running the installation which I have previous downloaded and un-packed from its ISO on a separately attached data drive.

The error I am getting is this:

The application has failed to start because its side-by-side configuration is incorrect. Please see the application event log or use the command-line sxstrace.exe tool for more detail. (Exception from HRESULT: 0x800736B1).

I tried several times, and often this error would occur during install, but on the fifth (or so) attempt the install was successful and all looked to have installed fine, until I tried opening SQL Server Management Studio (SSMS). And got the exception there instead.

Now following the instructions in the exception text I did two thing, first – check the event log, where I found this:

Activation context generation failed for "C:\Program Files (x86)\Microsoft SQL Server\100\Tools\Binn\VSShell\Common7\IDE\Ssms.exe".Error in manifest or policy file "C:\Windows\WinSxS\manifests\x86_microsoft.vc80.atl_1fc8b3b9a1e18e3b_8.0.50727.4053_none_d1c738ec43578ea1.manifest" on line 0. Invalid Xml syntax.

Now looking through the event log I could see that I got this error for a number of other applications and services as well, and that ssms wasn’t alone in this.

Next, I ran sxstrace, ie (from an elevated command prompt):

sxstrace trace –logfile:trace.log

The I tried to start ssms to produce the error, which it did. So then I ran:

sxstrace parse –logfile:trace.log –outfile:trace.txt

(More on the sxstrace tool here).

The trace file, among other things, gave me this (similar) information:

INFO: Parsing Manifest File C:\Windows\WinSxS\manifests\x86_microsoft.vc80.atl_1fc8b3b9a1e18e3b_8.0.50727.4053_none_d1c738ec43578ea1.manifest.
    INFO: Manifest Definition Identity is (null).
    ERROR: Line 0: XML Syntax error.
ERROR: Activation Context generation failed.

This file is from the Visual Studio C++ 2005 Service Pack (SP) 1 Redistributable Package. So I proceeded to download and install both the original 2005 Redistributable (x86, x64) and SP1 (x86, x64), hoping that would fix the problem and correct the manifest file. Not so for me.

I still wanted to see if the error could be fixed by “normal” procedures so I ran System File Checker (SFC). It produced the following result:

sfc /scannow

Beginning system scan.  This process will take some time.

Beginning verification phase of system scan.
Verification 100% complete.
Windows Resource Protection found corrupt files but was unable to fix some of th
em.
Details are included in the CBS.Log windir\Logs\CBS\CBS.log. For example
C:\Windows\Logs\CBS\CBS.log

The log file contain this (snipped somewhat for readability):

Manifest hash for component [ml:280{140},l:152{76}]"x86_microsoft.vc80.atl_1fc8b3b9a1e18e3b_8.0.50727.4053_none_d1c738ec43578ea1" does not match expected value.
Expected:{l:32 b:43e8b1d9f404eb67105ab15282fd01f5bf4cd30f7f0c5d1250d11e9384ae9cc5}
Found:{l:32 b:d47fec989a9ad0351d4effd5984343181925f15919245da2a0609e1c5d68f280}.
Unable to load manifest for component [ml:280{140},l:152{76}]"x86_microsoft.vc80.atl_1fc8b3b9a1e18e3b_8.0.50727.4053_none_d1c738ec43578ea1"
[SR] Cannot verify component files for Microsoft.VC80.ATL, Version = 8.0.50727.4053, pA = PROCESSOR_ARCHITECTURE_INTEL (0), Culture neutral, VersionScope neutral, PublicKeyToken = {l:8 b:1fc8b3b9a1e18e3b}, Type = [l:10{5}]"win32", TypeName neutral, PublicKey neutral, manifest is damaged (TRUE)

At this point I gave up on any form of allowing installers or the system to fix the problem for me and went at the file myself using Advanced guidelines for diagnosing and fixing servicing corruption. The file is readable (although empty), but I cannot edit it (even if I am an administrator). Only SYSTEM has access to the file. So to be able to edit it I must first take ownership of it and grant ACLs:

>takeown /f C:\Windows\winsxs\Manifests\x86_microsoft.vc80.atl_1fc8b3b9a1e18e3b_8.0.50727.1833_none_d1c5318643596706.manifest

SUCCESS: The file (or folder): "C:\Windows\winsxs\Manifests\x86_microsoft.vc80.atl_1fc8b3b9a1e18e3b_8.0.50727.1833_none_d1c5318643596706.manifest" now owned by user "JEHBTS5\Administrator".

>icacls C:\Windows\winsxs\Manifests\x86_microsoft.vc80.atl_1fc8b3b9a1e18e3b_8.0.50727.1833_none_d1c5318643596706.manifest /grant administrators:F
processed file: C:\Windows\winsxs\Manifests\x86_microsoft.vc80.atl_1fc8b3b9a1e18e3b_8.0.50727.1833_none_d1c5318643596706.manifest
Successfully processed 1 files; Failed processing 0 files

>takeown /f C:\Windows\winsxs\Manifests\x86_microsoft.vc80.atl_1fc8b3b9a1e18e3b_8.0.50727.4053_none_d1c738ec43578ea1.manifest

SUCCESS: The file (or folder): "C:\Windows\winsxs\Manifests\x86_microsoft.vc80.atl_1fc8b3b9a1e18e3b_8.0.50727.4053_none_d1c738ec43578ea1.manifest" now owned by user "JEHBTS5\Administrator".

>icacls C:\Windows\winsxs\Manifests\x86_microsoft.vc80.atl_1fc8b3b9a1e18e3b_8.0.50727.4053_none_d1c738ec43578ea1.manifest /grant administrators:F
processed file: C:\Windows\winsxs\Manifests\x86_microsoft.vc80.atl_1fc8b3b9a1e18e3b_8.0.50727.4053_none_d1c738ec43578ea1.manifest
Successfully processed 1 files; Failed processing 0 files

Now I can edit the file. As for the content I simply took it of another machine in which it existed and did not seem to have any issues. The content is this:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<!-- Copyright © 1981-2001 Microsoft Corporation -->
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
    <noInheritable/>
    <assemblyIdentity type="win32" name="Microsoft.VC80.ATL" version="8.0.50727.4053" processorArchitecture="x86" publicKeyToken="1fc8b3b9a1e18e3b"/>
    <file name="ATL80.dll" hash="6d7ce37b5753aa3f8b6c2c8170011b000bbed2e9" hashalg="SHA1"/>
</assembly>
After saving the file I am (at least seemingly to this point) rid of the problems.
Filed under: , , ,
You do not (always) need a correlation set to promote properties in a message sent from an orchestration
03 June 12 10:48 PM | Johan Hedberg

It is common knowledge that you use a Correlation Set to correlate message by creating an instance subscription that subscribes to hopefully unique properties. The subscription is created by pointing out a number of properties that you want to use for the correlation. Then when a message is published to the MessageBox that matches that subscription it is delivered to the orchestration. What most experienced developers knows is that Correlation Sets also promote properties as they are initialized; as the first message is sent by the orchestration. There is no other way to manually select properties to be promoted in a message sent from an orchestration. However, do not think that manual promotion through the use of Correlation Sets is the only way that properties will get promoted when published by an orchestration. It is not.

If the property has a Property Schema Base of MessageDataPropertyBase as below, then you do not need a correlation set. You only need to send the message and promotion will be taken care of automatically.

image

Only if you want to promote a property that has a Property Schema base of context type (MessageContextPropertyBase or PartContextPropertyBase) do you need to manually provide a Correlation Set to make sure promotion happens.

image

By *ContextPropertyBase we mean that its origin cannot be found within the data of the message, but is expected to be promoted to the context by another component, such as the adapter, a pipeline component or something else – in this case an orchestration, do you need to manually take action to make sure it is promoted as you publish a message to the MessageBox from an orchestration. And in this case you need a Correlation Set.

Of course if you were to be actually doing correlation with the Correlation Set and not only using it to promote a property then you would still want to Initialize a Correlation Set containing all properties you want to use regardless if these have their origin in the message data or not.

Sommarkollo 2012–The Microsoft Integration Story
03 June 12 06:58 PM | Johan Hedberg | 1 comment(s)

Ever updated, The Microsoft Integration Story, in an extended 3h format, joins the lists as one of the available topics in Microsoft Swedens Summercamp (Sommarkollo) 2012. Two stops in Stockholm (27/6, 21/8) and one in Helsingborg (26/6). I hope to see you there.

Enjoy
/Johan

clip_image001

Additional info (in Swedish):

Snart är sommaren här och med den Microsofts uppskattade evenemang Sommarkollo. För tionde året i rad har vi massor av spännande seminarier och produkter att presentera. Delta i så många seminarier du vill – helt utan kostnad! Passa på att träffa oss när vi besöker Stockholm, Göteborg och Helsingborg på vår turné genom Sverige i sommar.

Sommarkollo är ett evenemang för dig som vill bli inspirerad och påläst inför hösten. Du blir väl insatt i nyheter, teknik och annan intressant och användbar information som rör våra senaste och hetaste produkter. Seminarierna riktar sig både till dig som är kund och partner till Microsoft.

Vi kommer bland mycket annat presentera nyheterna System Center 2012, Windows Server 2012 och SQL Server 2012. Vi ger dig även unik inblick på hur Windows 8 kommer se ut och hur du och ditt företag kan arbeta enklare och effektivare med hjälp av Microsofts produktivitetsplattform. Vi bjuder också på flera målgruppsanpassade seminarier för exempelvis utvecklare, it-proffs och säljare.
Välj och kombinera de seminarier som intresserar och passar just dig. Om du bokar både för- och eftermiddagspass bjuder vi på en lättare lunch.

Det här är ett strålande tillfälle att under avslappnade former diskutera, få inspiration och utveckla din kompetens.

Anmäl dig här!

Published: (MCTS): Microsoft BizTalk Server 2010 (70-595) Certification Guide
03 June 12 04:53 PM | Johan Hedberg | 5 comment(s)

The book has finally been published. I couldn’t be happier about it. It’s been a fun ride but reaching the finish line is always sweet.

The book itself is targeted at the BizTalk Server certification 70-595. We have done our very best to be as brief and as focused on the areas of the certification as possible, while still keeping it far away from being verbatim or a cheat sheet. We want people to come away having learned useful pieces of BizTalk Server, not only be able to successfully answer the certification questions. The book does contain a healthy number of certification style (but not certification copied) question and answers, with a short explanation of why and why not the correct answer applies. All you need to know is there, explained and put into context. Hand in hand with being certification bound and focused also goes carrying a thread throughout and across chapters that will make the book an easy and time efficient read. I hope, in the end, that we succeeded in that.

You can read all about the book and it’s content, as well as making purchases ;) on either Packt or Amazon (or if you prefer and are in Sweden from Bokus).

4927EN_cov

Authoring a book was more of a challenge than I anticipated. Not so much the writing in itself, but all the things that goes around it. All the editing and proof reading and keeping track of changes etc. The latter took basically half of the time if not more, both calendar wise and in number of spent hours. I would do it again if the title was interesting enough, but right now I have other plans for the summer.

Completing a book is a real team effort, and without the team at Packt and our reviewers the quality would not be at the level it is. Thanks to Kent and Morten, to all the people at Packt, Kerry especially, and to our reviewers, Jan Eliasen, Mikael Håkansson, Steef-Jan Wiggers, Todd Uhl among others.

Kent has written two post (1, 2) on the progress of the book as well.

TechDays 2012–The Microsoft Integration Story
26 May 12 08:12 AM | Johan Hedberg

At TechDays 2012 in Sweden we did a presentation called “The Microsoft Integration Strategy” (or Story), something that we have done in variations for BizTalk User Group Sweden and other events.

I am delighted to have gotten back the review which puts the session in the top 10 sessions (7th). Glad you liked it. The presentation slides are here (as a pdf). I’ll be happy to share the pptx if you want it, just ask (It keeps me up to date with other interesting presenters and events). The video should be available shortly (in Swedish though). The video is available here.

Microsoft Integration Story

How the per-core licensing model will (by my guess) affect how we build BizTalk environments today
10 February 12 03:37 PM | Johan Hedberg

SQL Server 2012 comes with a new licensing model. Instead of per processor socket you will pay per core. Licenses are sold in two core packs with a minimum of four core licenses needed per physical processor (even if that processor only has two cores). The price per core is about one quarter (1/4) of the price per processor today. This means that the price for anything but quad core processor architectures will either go up, in the case of hexa core processors, or will give you less value then what you are paying for, in the case of dual core processors.

As far as how you would design your environments to keep the value without increasing the cost that means going with quad core as far as possible instead of for example hexa core. Especially if you are not sure you need that extra bit of processing power.

Although this change is explicitly announced for SQL Server and so far does not extend to BizTalk Server, the way the development of processors is and has been moving over the last couple of years it stands to reason that this may very well be (and by my guess will be) extended to other products in the future. The change is inevitable.

Therefore, today I would opt for quad core processors over hexa core if I use physical servers, in both SQL and BizTalk, to get a smooth transition to new licensing models.

Or at the very least, think twice before you go on old preferences and choose as many cores as possible just to get the most value out of the current licensing model.

Virtualization stays pretty much the same, that is, either pay for the virtual cores or for the physical cores (under the same rules as stated above).

At this point in time this is not new information. For licensing technicians this is a well known fact, but for other technicians or architects, this information might not have surfaced yet.

HTH,
/Johan

Filed under:
BizTalk Server and named SQL Server Analysis Services instances
09 January 12 10:21 PM | Johan Hedberg | 1 comment(s)

The documentation for BizTalk Server 200X is very clear, and has been since BizTalk Server 2006. See the 2010 versions of technet wiki and download. It states:

Named instances of SQL Server 200X Analysis Services are not supported

But why is that? I set out to investigate if I could find a reason for it still being true.

Examining the history

In the beginning, there was a limitation with SQL Server 2000 Analysis Services (SSAS). It was not instance or cluster aware (much like SSIS today). Some time later with the help of service packs (if I can recall and use my search engine correctly) SSAS became cluster aware, but only supported running on the default instance. Thus the requirements and the reason that was true with BizTalk Server 2006 is crystal - due to the support of the underlying technology, namely SQL Server 2000, named instance of Analysis Service was not supported.

Present

I later versions of SQL Server, and the latest of SQL Server 2008 R2 specifically, there is no such limitation. Analysis Services is both cluster and instance aware and can be installed in a named instance. So the underlying limitation just isn’t there. But what about BizTalk? Does it have some built in limitation to how it uses Analysis Services that doesn’t allow working against a named instance?

The scenario

Two BizTalk Server Group installations – connected to their respective database instance - side-by-side. Both SQL Server instances has the database, agent and analysis services services installed. In the default instance I have a BizTalk Server Group configured with BAM tracking and Analysis, and a deployed activity and view when we start our journey. For the named instance I wanted to configure the same thing to see that there were no issue with

  • running Analysis Service in a named instance, and
  • running it side by side with the default instance and its Analysis Services on the same SQL Server machine.

Configuring BizTalk Server against a named instance of SQL Server 2008 R2 Analysis Services

First configuring the feature using BizTalk Server Configuration.

BTS2_BAM_config_seperateAS_2

Then applying the configuration and view the result.

BTS2_BAM_config_seperateAS_2_configDone

No issues.

Deploy BAM activities and views

Using bm.exe to deploy my BAM activity and view to my named instance.

BTS2_BAM_config_seperateAS_2_deployOK

No issues.

Using Tracking Profile Editor to connect my activity to an orchestration.

BTS2_BAM_config_seperateAS_btt

No issues.

Running some data through the integration

BizTalk Server behaves as expected. After running some sample data through the integration I could open my BAMPrimaryImportDb and run a Select against it to view my data.

BTS2_BAM_config_seperateAS_Select

Running my BAM_AN_<View> SSIS package

This was one of my prime candidates beforehand of where it could possibly go wrong. However as I used Business Intelligence Development Studio to inspect the package I could see that it was configured with datasources for the databases that it worked with, including the Analysis Services database, and it clearly points to the correct configured instance.

BTS2_BAM_config_seperateAS_BIstudio_BAM_AN_OrderView

Running it caused no incidents that I could detect. After having run the package and processed the cube I was ready to look at the data in the cube to see if it got there as expected.

Viewing the Live Workbook

The first thing to check was the live workbook (useless as I find this feature I wanted to see if it could read the data as expected).

BTS2_BAM_config_seperateAS_ExcelLiveWoorkbook

It could. And again, if we view the data connection properties here it point to my named instance.

BTS2_BAM_config_seperateAS_ExcelLiveWoorkbook_ConnectionProperties

(sorry about the swedish locale here, but you get the point)

Viewing data in the BAM portal

Next up, what about the BAM portal – would it be able to understand the named instance of SSAS and retrieve my data?

BTS2_BAM_config_seperateAS_BAMPortalView BTS2_BAM_config_seperateAS_BAMPortalActivitySearch

Yes. Both the Activity Search and the Aggregations show the data expected.

Conclusion

I can find no reason why SQL Server 2008 R2 Analysis Service cannot be used on a named instance with BizTalk Server 2010. Granted, this was just a small test and I certainly haven’t investigated this through every aspect (as per the usual disclaimer), but it “Works on my Machine”.

Do you know a reason that this will not work? Are you running this configuration yourself and can confirm further that it does work? Let me know…

HTH
/Johan

Filed under: , ,
Why BizTalk Server 2010 R2 should be BizTalk Server 2013
03 January 12 07:33 PM | Johan Hedberg | 7 comment(s)

That BizTalk Server 2010 will have a successor and that the working title of that release is BizTalk Server 2010 R2 was announced at the BizTalk Server team blog in december 2011 and re-announced by folks like Kent Weare, Charles Young, Saravana Kumar, Steef-Jan Wiggers among others.

What some of them hints at but doesn’t discuss further is that the version naming “2010 R2” might not stick. There are good grounds for such guesses. Historically BizTalk Server 2009 was initially called BizTalk Server 2006 R3 before the renaming was announced and BizTalk Server 2010 was called BizTalk Server 2009 R2 when first announced before it too was renamed.

One might argue that the R2 is a good suffix in this release since it is a minor release, without an abundance of new functionality. That’s true.

One might argue that there is only so much to a name, the important thing is that Microsoft is showing that it will continue to maintain and carefully evolve the product. Not so I say.

There is one very important thing that goes into that name that should not be overlooked or underestimated – the support lifecycle. Microsoft’s support lifecycle policy says that products will have 5+5 (mainstream+extended) support. However, that applies to major versions. An excerpt:

“Minor releases follow the same Support Lifecycle as the major product release.

An example of this is Windows Server 2003 R2 which has the same Mainstream Support phase and Extended Support phase dates as the parent product, Windows Server 2003. Likewise, Windows Server 2008 R2 follows the same Support Lifecycle dates as the initial release of Windows Server 2008”

If the product will be BizTalk Server 2010 R2 (and assuming it will follow the general rule), it will not get an extended support lifecycle end date. Except for what is stated in the general policy you can see examples of this scheme throughout the product chain. BizTalk Server 2006 and 2006 R2 are the closest examples, but also Windows Server 2008 and 2008 R2, and SQL Server 2008 and 2008 R2 both follow suit. In all these cases the R2 version does not start a new 5+5 year period. Where as with BizTalk Server 2006 R2 to BizTalk Server 2009 and again to BizTalk Server 2010, we got new lifecycle support dates.

One of the major asks around BizTalk Server before the announcement was for Microsoft to clearly show its continued support for BizTalk Server as a product – they did that, but here is hoping that they will strengthen that statement further by giving us a BizTalk Server 2013 (or 2012).

HTH
/Johan

The rogue agent that brought BizTalk to its knees
02 January 12 10:16 PM | Johan Hedberg

To help others that might find themselves in a similar situation I am posting this odd experience we had with a BizTalk environment during the fall of 2011.

We had a pretty standard setup with good hardware to back it up all the way, set up after best practices. We were using the BizTalk Benchmark Wizard (BBW) to benchmark our environment and were comming up short at around 70 msg/s.

We should have had values that were around 900 msg/s. Overall, from scrutinizing the performance logs using Performance Analysis of Logs (PAL) as well as our own best judgement, we at first couldn't find anything alarming. Processor, Memory, disk, network etc. All good. We also ran things like the BizTalk Best Practice Analyzer (BizTalk BPA), the MessageBoxViewer tool (MBV), the Monitor BizTalk Server SQL Server Agent job, but it all came back looking good. The environment just seemed... slow.

As it turns out the processor was especially interesting knowing what turned out to be the final finding. The processors (two of them per server each of them with 6 cores per processor) was on an average very low, but as it turns out there was one process that was taking the equivalent of 1 full core of power (its Process % Processor time was at 100), but since it didn't stay on one core it was hard to spot the problem. PAL doesn't have an alert for this, and finding the one process and performance counter among all of them is not so easy.

The process was the "HP Insight Server Agents" (cqmgserv.exe). The theory goes that as it was failing, recovering and retrying it was pumping the machine full of events and clogging up the underlying bus.

The closest we got to a match in the form of a support document from HP was this. Once the service was disabled the tests ran as expected att around 1000 msg/s. Later the service was updated to a newer version and started again without causing the same issues.

 

The purpose of this post is not to lay the blame on HP's door but instead to enlighten readers that similar situations can occur and to highlight the value of a tool like BBW, since without it this exception would have likely never got caught and this server would have gone into production delivering much less value on the investment than it should.

HTH
/Johan

Filed under: ,
More Posts Next page »

This Blog

News

    Messenger

    Twitter Updates

      Follow me on twitter

      Visitors

      Feedburner Subscribers

      Locations of visitors to this page

      Disclaimer

      All material is provided AS IS voiding any thinkable or unthinkable effect it might have for any use whatsoever. There... is that clear enough ;)

      Pages

    Syndication