January 2008 - Posts

BizTalk File Adapter and Windows Fileshare Limitations
30 January 08 07:38 PM | Johan Hedberg | 1 comment(s)

This post is based on something we experienced recently. When we activated some more file receive locations against a machine to which we previously had a bunch of receive locations configured, after a while they began to become disabled, one after the other (but not all). The message in the eventlog states:

The receive location “Xxx Receive Location” with URL “\\server\share\*.*” is shutting down.
Details:”The FILE receive location \\server\share\*.* exhausted the network retry attempts.”

The network was up, security settings was unaltered and besides - some of the receive locations still worked. Curious.

Turns out, it's as much a Windows Server 2003 (and other OS'es) issue as it is a BizTalk issue, and you can just as likely get it in another type of application as in a BizTalk application. The problem is based upon limitations in the number of open requests or commands between two machines. There are limits to both how many connections you are allowed to another computer and how many connection you allow another computer to make to you (ie both outgoing and incomming).

The reason this is more likely to occur in a BizTalk application is that when listening on a share BizTalk registers for change events, and for each receive location one such event will be register. The default limit for the number of active commands is 50. Luckily there is a knowledgebase article (ID:810886) describing how to get around this issue (ie raise the limit). Also for those of you reading the Pro BizTalk 2006 book, there is a section in the book called File Tuning (pages 445-447) that talks about this and also highlights some other issues with the file adapter. The book suggest setting the values to 2048 instead, but doesn't really say why. The highest possible value is 65000. My suggestion is to set it to a value where you are sure you have covered the number of active receive locations and have room for additional connections used by send port and other type of access using fileshares - only you know what that is. Also for a BizTalk environment, do this to both the BizTalk server and the fileserver you are connecting to. 

Below is the summary of what the knowledgebase describes, in registry file format (valid for Windows 2003 Server machines):

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\lanmanserver\parameters]
"MaxMpxCt"=dword:00002048

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\lanmanworkstation\parameters]
"MaxCmds"=dword:00002048
Filed under: ,
ESB Guidance Exception Management and Pro BizTalk 2006
28 January 08 04:26 PM | Johan Hedberg

The Exception Management part of the ESB Guidance was first introduced (to me at least) through the book Pro BizTalk 2006 by George Dunphy and Ahmed Metwally. I didn't immediately make the connection, but when looking at the ESB Guidance solution something felt familiar and as it turns out I had seen it before. So for those of you that are wondering what the ESB Guidance holds as far as Exception Handling and Management goes - but haven't had the time to sink your teeth into it yet - if you have read and understood the Exception Management scenario within Pro BizTalk 2006 (pages 225-250), you understand the basics of what the ESB Guidance Exception Management does and the goals of the implementation. Some of the text and images in the book, when compared to the ESB Guidance documentation, is exactly the same. The ESB Guidance however contains more than is covered in the book - in fact, in addition to the part that the book describes, it also contains an implementation for the parts that the book does not describe, but mentions as design goals for a complete Exception Management scenario built upon BizTalk and the Messaging and Orchestration sub-systems.

Filed under: ,
How-to: Use BizTalkMgmtDb to get referenced assemblies
26 January 08 08:46 PM | Johan Hedberg

Sometimes when you are trying to remove an assembly from within the Administration Console you get and error saying that it's being used. Most often you'll get information on where it's being used at the same time, but not always. These queries are for those times. With the help of these queries, and the knowledge gained from working with these tables you could also lookup things like assemblies that aren't being used or on which ports a particular map or pipeline is used - information that is not easily accesible through the Administration Console.

Get Assemblies referenced by Maps on ReceivePorts

Select    ass.nvcName as Assembly, 
        itm.Name as Map, 
        rcv.nvcName as ReceivePort 
from    bts_receiveport_transform tr
join    bt_MapSpec map on tr.uidTransformGUID = map.id
join    bts_item itm on map.itemid = itm.id
join    bts_assembly ass on map.assemblyid = ass.nID
join    bts_receiveport rcv on rcv.nID = tr.nReceivePortID
order by Assembly, Map, ReceivePort

Get Assemblies referenced by Maps on SendPorts

Select    ass.nvcName as Assembly, 
        itm.Name as Map, 
        snd.nvcName as SendPort
from    bts_sendport_transform tr
join    bt_MapSpec map on tr.uidTransformGUID = map.id
join    bts_item itm on map.itemid = itm.id
join    bts_assembly ass on map.assemblyid = ass.nID
join    bts_sendport snd on snd.nID = tr.nSendPortID
order by Assembly, Map, SendPort

Get Assemblies referenced by Pipelines on ReceiveLocations

select    ass.nvcName as Assembly,
        pipe.Name as Pipeline,
        loc.Name as ReceiveLocation
from    adm_receiveLocation loc
join    bts_pipeline pipe on pipe.ID = loc.ReceivePipelineId
join    bts_assembly ass on ass.nID = nAssemblyID

Get Assemblies referenced by Pipelines on SendPorts

select    ass.nvcName as Assembly,
        pipe.Name as Pipeline, 
        snd.nvcName as SendPort
from    bts_sendport snd
join    bts_pipeline pipe on pipe.ID = snd.nSendPipelineId
join    bts_assembly ass on ass.nID = nAssemblyID
Filed under: , ,
How-to: Find the host that BTSNTSVC.exe belongs to
25 January 08 08:01 PM | Johan Hedberg | 2 comment(s)

Often times when you are looking at your environment from a performance perspective - perhaps you are testing or perhaps you are only trying to find out which host is responsible for the process running at 100% for 5 minutes - you need to know which BizTalk process has been started by what host. Another common reason is to know which service to attach to when debugging. There are a couple of ways to know this, and which one is best is both preference and related to environment and scenario. Regardless of scenario I prefer to start as few, quick and small applications as possible. I've put what I consider the best options in the order which is my preference.

  1. Use the tasklist command in a command window: tasklist /svc /fi "imagename eq btsntsvc.exe"
  2. View the BizTalk:Messaging \ ID Process perfomance counter in a command window: typeperf "\BizTalk:Messaging(*)\ID Process" -sc 1
  3. If you would like a more graphical interface there are tools that wraps the same data up in a prettier format, such as PerfMon and Windows SysInternals Process Monitor.
  4. I'm sure there are several more ways to do this... as I mentioned, it's a matter or preference.

If you want to add this as an External Tool in Visual Studio to have it easily accessible for when you want to attach a host process to debug you can add it like this:

Of course, you could just always attach to all hosts. If you want to do this or not depends on the number of hosts you have running among other things, but why do so if you don't need to?

Filed under: ,
Enabling the BAM Excel Add-In for Excel 2007
19 January 08 11:05 PM | Johan Hedberg | 2 comment(s)

Much has changed in the GUI from Office 2003 and earlier version to Office 2007. For many, menu option are hard to locate. This post is meant to be a short pictorial guide to one way of enabling the BizTalk Server 2006 BAM Add-In for Excel 2007. I'll assume that BAM and Excel are already installed.

What we need to do is to add it to the ribbon so we can access it. We do that by going to More Commands...

This will open the Excel options dialog box and from there you can select Add-Ins. You should see Business Activity Monitoring listed.

Next, select Manage: Excel Add-Ins, and press Go... This will bring up a familiar dialog where you need to check the Business Activity Monitoring Add-In.

Having done this the BAM Add-In will now be accessible through the ribbon and you can edit your BAM Activities and Views.

Filed under: ,
Trouble with Excel automation in a non-English locale
19 January 08 10:35 PM | Johan Hedberg | 1 comment(s)

Deploying BAM definitions from Excel files means automating Excel. Automating Excel has it’s issues when you work in a multilingual environment, like me. Specifically, when you use a non-English locale for your OS while having an English localized version of Excel installed may cause you to get the exception: 
ERROR: Failed to open BAM Excel workbook file - 'C:\Xxx.xls'.
Old format or invalid type library.

I always use the English versions of applications, and the Office 2007 Suite is no exception. This problem isn't new, but it's still with us. According to the KB that deals with this issue there are three possible solutions.

  1. Set the locale in the application doing the automation to English
  2. Set your environments (OS) locale to English

Option one isn’t really an option in this case since it’s bm.exe doing the automation. Option two isn’t really something you’d want in the long run, but it’s ok if you just need it as a temporary workaround during deployment. Option three - getting a language pack is really the preferred one and if you’re a MSDN Subscriber like me, the language packs are just a few clicks away. For the rest of you, sorry, you’ll have to buy them (they are $24.95 at the moment).

Filed under: ,
Nystart för BizTalk Usergroup Sweden
10 January 08 12:27 PM | Johan Hedberg | 1 comment(s)

Det är med nöje jag kan tillkännage...

Nystart för BizTalk Usergroup Sweden

 

Målet med BizTalk Usergroup Sweden är att genom en community driven insats öka förståelsen och kunskapen kring Microsofts produkter och lösningar från Connected Systems Division, främst kring varumärket BizTalk och närliggande. Information kommer att förmedlas främst genom presentationer och demo, genomförda av såväl frivilliga bland deltagarna själva som av speciellt inbjudna erkända svenska och internationella talare och experter. Det kommer även att finnas utrymme för andra typer av evenemang.

 

Målgruppen är individer med teknisk bakgrund eller intresse, såväl från linje som från konsultorganisationer, intresserade av systemintegration på Microsoft plattformen.

 

Vår målsättning är initialt att vi ska ha ett möte ungefär varannan månad.

 

Vår första träff kommer att gå av stapeln den 12:e februari, där innehållet i ESB Guidance och nyheterna kring Oslo kommer vara fokus.

                      

Agenda för kvällen är som följer:

17:45             Samling

18:00             Start

                      Introduktion till Usergroupen och dess arbete

                      ESB Guidance (Mikael Håkansson och Johan Hedberg)

19:00             Paus, mingel, diskussioner. Något att äta och dricka kommer finnas.

19:30             ESB Guidance avslutas

                      Oslo (Alan Smith)

Q & A, avrundning, diskussioner

 

Vi kommer att hålla till i KnowITs lokaler på Klarabergsgatan 60. Tack vare att antalet platser är begränsade och att mat ska planeras vill vi ha ditt svar senast den 31:a januari till bugs.info@gmail.com.

 
Oslo är en samling tekniska lösningar som ska förenkla byggandet av applikationer för en tjänsteorienterad arkitektur. Med Oslo förflyttar vi oss från att modeller beskriver applikationer till att modellen är applikationen. Läs mer på http://www.microsoft.com/soa/products/oslo.aspx.
 
ESB Guidance är ett steg i riktningen att lyfta BizTalk till mer av en Enterprise Service Bus (ESB) med utökat stöd för löst kopplade tjänster. Läs mer på http://msdn2.microsoft.com/en-us/library/bb931189.aspx. Presentationen kommer belysa hur BizTalk relaterar till SOA, SOI och inte minst ESB. Vi kommer gå igenom stöd, riktlinjer, komponenter och tillhörande portal, och ge en bild av hur ESB Guidance kan konfigureras, användas och utökas.
Filed under: , ,
Removing Xml Namespace in a pipeline component
07 January 08 11:11 PM | Johan Hedberg | 18 comment(s)

Richard Hallgren recently blogged about how to remove Xml namespaces from Xml documents, as has Kirk Allen Evans earlier here and here. While the latter writes from a general .NET and ASP.NET perspective Richard's post is from a BizTalk perspective and in his post he asks for alternative ways, other then the ways he is presenting, of doing it. 

The root issue is how do I turn

<ns0:Blah xmlns:ns0="http://RemoveXmlNamespace.BTS.BlahMessage">

into

<Blah>


First of all, let me just say that I understand namespace and I don't think they should be removed if at all avoidable. This is not my first choice of a solution, as it probably isn't for anyone working with BizTalk. The fact remains though that in some cases it still turns out to be necessary. My option of doing this would then be by using a pipeline component and the classes available to us from Microsoft.BizTalk.Streaming.dll. My main reason for this is performance. I would really hate to have to keep either the source document or the resulting document in memory using XmlDocument and MemoryStream like the XslTransform pipeline component sample from the BizTalk Server 2006 SDK does. I'm pretty certain that particular sample can be enhanced using XPathDocument, XslCompiledTransform and VirtualStream (to keep down the impact on memory) but it still isn't as good if your only purpose is removing the namespace. That sample can however do many other things that my option can't, since it does an XslTransform and this doesn't.

So using the streaming classes in general and XmlTranslatorStream in particular we can override some of it's methods to create a streaming xsl transformation doing what we want. The resulting code is suprisingly simple. Here is the Execute method:

public Microsoft.BizTalk.Message.Interop.IBaseMessage Execute(
    IPipelineContext pContext, 
    Microsoft.BizTalk.Message.Interop.IBaseMessage pInMsg)
{
    pInMsg.BodyPart.Data = new XmlNamespaceRemoverStream(
        pInMsg.BodyPart.GetOriginalDataStream());
    return pInMsg;
}

And here is the class that does the actual work (although as you can see it doesn't really do much):

public class XmlNamespaceRemoverStream : XmlTranslatorStream
{
    protected override void TranslateStartElement(
        string prefix, string localName, string nsURI)
    {
        base.TranslateStartElement(null, localName, null);
    }

    protected override void TranslateAttribute()
    {
        if (this.m_reader.Prefix != "xmlns")
            base.TranslateAttribute();
    }

    public XmlNamespaceRemoverStream(Stream input)
        : base(new XmlTextReader(input), Encoding.Default)
    { }
}

This sample is available for download: RemoveXmlNamespacePipeline.zip.

Getting formatted c# code into CommunityServer
06 January 08 02:38 PM | Johan Hedberg

After looking at a couple of different solutions I ended up using csharpformat for one of my previous posts, and I will probably keep using it. It's not perfect, most of all I would have liked getting CopySourceAsHtml working, since judging from the description that would make the snippets in the blog look exactly like in Visual Studio, which is what I really want. However it just keeps throwing exceptions and I am in no mood of debugging it at the moment. Csharpformat with some small adjustments to the css file does the trick, and most importantly it makes the code itself searchable and copyable instead of it being an image - which isn't really helping anyone. CSharpformat seems to also be mirrored here, and is available as a VS.NET 2005 plugin here, although since it builds on the same principles I am not sure what extras it gives you, but then I haven't tried it.

Grab BizTalk Hotrod to start of the new year
06 January 08 01:52 PM | Johan Hedberg

The free online magazine BizTalk Hotrod has been published again, this time we are looking at issue number three. It contains more then ever, and the size of the download shows it. The PDF weighs in at just over 18MB! Supposedly the size is also attributed to the fact that you will now also be able to see the content of all images and read the sourcecode contained there-in, which was my only problem with the previous issues. Go get it (links directly to the PDF). In case you haven't read the past two issues, those are well worth your time as well.

Filed under: ,
Consuming Pipeline Component issue fixed in R2
05 January 08 10:11 PM | logical

I previously experienced an issue with a general purpose (not an assembler/disassembler) consuming pipeline component in BizTalk Server 2006. For those of you unfamilliar with the term a consuming pipeline component is just that, a component that consumes the messages and returns null to stop pipline and message processing. Having tested this on R2 I'm happy to report that this issue doesn't exist any more.

In short, my previous issue was that in some situations a consuming pipeline component might cause an endless loop to occur within BizTalk, which in any scenario is undesirerable. The cause of this was if the pipeline threw an exception making the message become suspended on the first time around, and then returned null the second time around. Granted, not a very common scenario, so chances are good that it might not have happened to you.

This code in the execute method of the pipeline component would allow you to reproduce the error:

public IBaseMessage Execute(IPipelineContext pContext, IBaseMessage pInMsg)
{
    if (pInMsg.MessageID != new Guid((string)pInMsg.Context.Read(
        "InterchangeID", "http://schemas.microsoft.com/BizTalk/2003/system-properties")))
    {
        System.Diagnostics.Trace.WriteLine("EndLess: Return null");
        return null; // resume, return null
    }
    else
    {
        System.Diagnostics.Trace.WriteLine("Ex");
        throw new Exception("First time in pipeline, throwing exception");
    }
    return pInMsg;
}

Just stick this code in the appropriate place in the custom pipeline component, create a receive pipeline, and run a message though. First time around it will get suspended. Now go ahead and resume it. Wham! There you go - endless loop in 2006. However like I said, R2 does not display this behavior, and with any luck some hotfix or update for 2006 might contain the fix as well.

The code above also shows another small trick: How to know if a message is a resume in a receive pipeline. When a message is first received the MessageID and InterchangeID are the same, however on a receive the MessageID will be a new one, but the InterchangeID will remain the same as it was the first time.

Programming WMI - The Visual Studio Server Explorer Way
03 January 08 10:39 PM | logical

Recently Niklas posted about a tool called WMI explorer. It's a nice and easy tool to use if you want to explore the WMI interfaces and their values on any computer, especially any non-development computer. However as far as development, debugging and using WMI goes, I've come to be more in favour of another way of doing WMI programming. Namely by using the Visual Studio .NET Server Explorer. There is a great article/blogpost at El Grego's BizTalk Blog highlighting the procedure. Granted, the post isn't recent, but the approach is still valid. The only update for the 2005 version of Visual Studio .NET is that the Management extensions isn't needed. The less custom code the better.

Filed under: ,
Custom Pipeline Components Development Best Practices
02 January 08 12:05 AM | Johan Hedberg | 4 comment(s)

There is much to be said about pipeline component development. Online, in the helpfiles, in books there are step by step guidance on how to build pipeline components of different types (Assembers, Dissassemblers, General Purpose etc.). I'm not going to repeat any of that. Instead I'm going to list the things that I feel are paramount to think about when doing pipeline component development, regardless of type of component and its purpose.

  1. As long as it's possible - keep it forward only streaming. Learn and know the contents and techniques of Microsoft.BizTalk.Streaming.dll.
  2. In case you can't keep it forward only, make sure you have a seekable stream, by wrapping it in such (ReadOnlySeekableStream), which in turns creates a VirtualStream that overflows to disk instead of filling up your memory.
  3. If the above streaming classes are not enough, and you need data from the stream, try to build your own Stream implementation and perform your logic as it is being read. Be a copycat, use Reflector.
  4. Do not load the contents of streams into memory. MemoryStream, XmlDocument, string and ReadToEnd and such is therefore generally a sign of bad practice. Keep your components impact on memory as low as possible.
  5. Don't start new threads - doing so interferes with BizTalks internal threadpool management and thus impacts performance.
  6. Try to stay away from database calls or calls to WebServices. If you must do them, be sure to cache the response if at all possible (and reasonable). Keep the pipeline lean and mean.
  7. Test. Test early, test often, test as a unit, test in conjuction with other components in a pipeline, test logic, test performance. And when testing, don't test on your development laptop and go "that seems to work fine", keep it real (as real as possible).

If you think of these things you'll be better off. There are still mistakes you can make doing BizTalk custom pipeline component develpment, and of course all general .Net coding best practices apply here as well, but if you think of these things, and can clearly motivate when and why you deviate from them, you're one step closer to a successful component implementation.

This Blog

News

    Messenger

    Twitter Updates

      Follow me on twitter

      Visitors

      Feedburner Subscribers

      Locations of visitors to this page

      Disclaimer

      All material is provided AS IS voiding any thinkable or unthinkable effect it might have for any use whatsoever. There... is that clear enough ;)

      Pages

    Syndication