Having recently discovered that this Azure thing is just about ready for actual release, I have picked it up. So after spending an evening installing everything needed to run samples and tutorials, Microsoft releases version 1.5 of the SDK and I have to reinstall everything.

Voicing my angst about this on Twitter I got an answer from the newly appointed Evangelist Thiago Almeida: There is a better way, use Web Platform Installer!

Just point your browser to the Windows Azure Team blog here and off you go. Using this tool you can even install separate labs and it detects dependencies and lets you install them

Now if we can just get them to release a VHD-file as well.

A new experience for me today: You have to use English in the config-file for BizTalk. The thing was that we store some custom settings in the <appSettings>-tag. Quite useful in this particular scenario. So I added some information in Swedish (using the letter ö) and later restarted a host-instance.

I got this error message:

image

It reads:

A failure occurred when executing a Windows service request.
Service request: Start
BizTalk host name: BizTalkServerApplication
Windows service name: BTSSvc$BizTalkServerApplication
Additional error information:
Error code: 0xc0c0153a
Error source: BizTalk Server 2009
Error description: A BizTalk subservice has failed while executing a service request.
Subservice: XLANG/s
Service request: Start

Note that it’s XLANG/s that does not start.

It took me a while to figure out but it was due to the ö. The strange thing is that the letter was used in a comment! So even there you have to use English.

The Swedish Azure User group (or SWAG!) will host their forth event on the 29th of September in Stockholm. Detailed info and booking here.

Alan Smith and Barry O’Reilly have invited Björn Eriksen from Microsoft to speak about Azure Storage and the implications it has on programming and architecture.

When coding for the cloud you cannot utilize the traditional file system, what do you use instead and how? Also, how do you make efficient use of other storage capabilities, such as database, blob and queues?

Given the location and the vicinity to bars, my guess is that there probably will be some time for a relaxed chat, after the event.

Due to the large amout of spam I get, I have removed the comment-feature from the blog.

If you have any feedback about a post, I’d love to hear it. Use the contact form.

logo-mix-ibm-microsoft The last couple of days I have been working with IBM connectivity. We use standard IBM components to bridge SQL Server SSIS and DB2. The component has been working 100% in all but one environment.

So, everything is installed and seems to be configured correctly. The package is installed and every possible setting checks out compared to the other (working) instances. Still I get the error message above.

A google search points me to fixpacks, upgrades and horrific security settings (sharing the %systemroot%)! Even the support forums at IBM turns up this little sad, unanswered, post.

The simple answer was found by a colleague: Run SSMS as administrator! The elevated rights make it possible for SSMS to utilize the IBM provider.

Then it hits me: In no other of the working environment are we trying to execute the SSIS package from SSMS. We use jobs for that… Now I will celebrate and possibly kill myself

Or, as it was called: “Top 10 Things to Know When Integrating with Line of Business Systems”.

JohanIntroducingThe quick Saravana Kumar beat me to posting about this, but that does not mean I cannot reiterate it. The presentation was hosted by the Swedish BizTalk User Group (Johan and Mikael of course) and was presented by Richard Seroter, BizTalk MVP and nice guy, and Kent Weare, also a BizTalk MVP and nice guy (I see a pattern here).

Notice that a white shirt was the thing to wear on stage. They looked a bit like a cult…

So, what were the top 10 things to know? Here they are:

#1: Strategically use adapters, proxies and direct APIs.

This means that you should be aware of the fact that there might often be more than one way to integrate with a system. Depending on your needs (QoD or EAI) you should opt for different strategies.

#2: Be creative when client library can’t be used for security.

The most obvious of this is letting credential information tag along though a Azure Service Bus service, but there are other examples. Creative does not have to mean “not secure at all and totally dangerous”, but you have to present it well to the security guys for it to seem creative and not barking mad.

#3: Sometimes you need a data/protocol proxy in front of BizTalk

I guess we have all been there. You are presented with a service, or a schema or file that simply will not work with BizTalk. The most famous of these are web services that use the xs:any to be “dynamic” and “agile”.

In these cases you might need to build something to serve BizTalk in a way that the data is digestible. A personal example is a service that read data from a com-port and sent incoming messages to an MSMQ-queue (thanks to Kjell Almgren).

#4: Reliability may not be pretty

A picture of an old Volvo was the symbolism of choice. But again it ties into #3 and highlights the solution using MSMQ, but hey it worked and still does. You might not create a new database just to act as a buffer as a first step during development but as you encounter problems with lost messages and overflow, you implement it. Much to the annoyance of the people that takes over after you. But: It does not have to be pretty.

sav_and_richardAfter this we had a bit of a break, during which Saravana had the opportunity to demo his new creation BizTalk 360. It’s well worth a look. Personally I wonder what the next version will be called, 360 2.0, or BizTalk 720. I also had the pleasure of discussing the presentation with both Richard and Saravana.

So, after some salad, soft drinks and socializing Kent started round 2.

#5: Consider both Polling and Notification techniques

This, for me, is something to remember. Often I don’t even consider notification as it tends to be a bit hard to do. However Kent reminded us how easy it is to use notification from an SQL server using the WCF-SQL adapter. It might still be hard using other adapters though.

#6: Understand when to use Batch and Real time interfaces

Once again this is a very good thing to remember when you try to architect a solution. “Does this have to run in a batch?” “Can I use real time or will it bog down the AS/400 machine?”.

#7: Use asynchronous communication when possible

Another great tip and something to live by in order to shorten response times and latency in every solution. If the message does not have to be validated and possibly sent back for revision, then use asynchronous communication. Please.

#8: Data access strategies

tip 8This really needs to be illustrated by the great picture from the lecture.

It shows how data access might not be as straight forward as one might think.

Kent works in the utility business and integrates with one of the most up-tight systems around the world (not his words).

Now this system has fairly good ways for integration but … it has it’s issues, as presented by himself in a really useful webcast from 2009.

He talked about an example in which they had to exchange data in triplicate every time something was updated. One file containing the data, one for archiving and another to signal the system that the first file had been posted. A smart solution to a big problem, but not the straightest line of the three on the picture.

#9: Securing LOB interfaces

If I am honest I cannot remember the particulars about this point.

#10: Don’t integrate anything that feels unnatural

Illustrated by fries and mayonnaise (which is great by the way). Kent’s main point here, I would say, is that if it something in your gut is telling you that these systems should not integrate, then you are probably right. They might just be too different or handle completely different data.

Lastly I would like to a nice guy as well and point out that the coming book is a must read for those of us integrating LOB systems. I will surely order it since I, as a swebug attendee, get a discount.

Now I am off to view some movies and also give Richard a tip.

I got a question through my e-mail today. The problem was that he hade to use 3rd party schemas (papiNet). These schemas contained a lot of words that BizTalk does not allow. Well rather it is the c# compiler that does not allow it. However:  the schemas would not compile and so he was stuck.

The question was, in short, how do you use reserved c# or BizTalk words in schemas? Words like “schema” or “Int”?

The answer is quite simple: Just select the node in question and take a look at the property list. There is a property called RootNode TypeName. This is the type name that will be used when the schema is compiled. Change it to something that will compile.

In the case, the node is called “int”. Since this will not compile, the RootNode TypeName is set to “SchemaTypeInt”.

image

In this way you can still be true to the original format and still use the schema in BizTalk. Also note that the TypeName will never be visible in BizTalk maps nor when you work with promoted properties or distinguised fields.

An important note though: You cannot solve the issue with reserved xpath words using this. If you have a schema that contains one of those words and you need to promote, or use the xpath-function in an orchestration, you need to map the external schema into an internal, compatible, one.

Thanks to @cndamin for the feedback.

Not quite, but it will be a great event for those of us that get exited about words like “LOB”, “integration”, “pattern” and “cloud”.

BizTalk User Group Sweden is hosting an event called “Top 10 Things to Know When Integrating with Line of Business Systems”. It will feature the ever book-plugging Richard Seroter and co-writer Kent Weare. Together they have written a book called “Microsoft BizTalk 2010: Line of Business Systems Integration” (has a nice ring to it, doesn’t it).

Other confirmed guests are Saravana Kumar (BizTalk MVP and BizTalk 360 developer) and also that Dutch guy that keeps tweeting about football; Steef-Jan Wiggers.

…and me of course.

Now, here’s the thing: There are still seats left! Go! Grab yourself a seat.

Way back in 2005 Stephen W. Thomas wrote the #1 Google hit for “recoverable interchange processing”. This was added as a very welcome feature to the 2006 edition. As such I knew about it when designing the following information flow.

  1. Data is polled from a database (about 500 parent rows at a time). SQL server then adds the child data for each parent and returns it as XML.
  2. The receive pipeline is set to XMLReceive and the message is debatched when received by BizTalk. This is because the data is wrapped in an envelope by the adapter.
  3. Each individual message can then be mapped to an internal format.
  4. Each message is the mapped again and sent to the receiving party.

Easy peasy!

Now what happens if row 486 contains some kind of error and cannot be mapped on the receive port? The answer: The default behavior of BizTalk is that it will suspend the whole batch. Enter recoverable interchange processing (or RIP for short).

So I simply turned on RIP on the receive pipeline in step 2, but to no avail. That did not work. Turns out that you need to add a bit of code in a custom pipeline component for this to work.

Simply add:

inmsg.Context.Write("SuspendMessageOnMappingFailure", 
"http://schemas.microsoft.com/BizTalk/2003/system-properties", true);

to a pipeline component and add it to a pipeline in the validate stage. Don’t forget to add an XmlDisassembler to the disassemble stage, since you want to keep the debatching behavior.

The result will look like this: pipeline

If you create the component, make sure to have it be configurable from Visual Studio as well as BizTalk Management Console.

The end result might look like this: image

You can see that there is a property called SuspendBatchOnFail. If this is set to false, BizTalk will process the next message in the batch even if mapping fails.

BizTalk also marks the message with a sequence number. This number that is displayed in the eventlog if it is suspended. The error will read: “The Messaging Engine failed while executing the inbound map for a recoverable interchange message on Receive Port [---] The input message to the map is being suspended.  The sequence number of the suspended message is 8.”

I have uploaded some code that might help you.

I had no idea these existed, but like Rob Schneider, I’ll take what I can get!

MCC-fullcolor

So, I got an award! Whohoo!

A real bonus is that this includes total access to the Microsoft Press E-Reference Library for a whole year. The only downside to this is that there are no good books on BizTalk nor AppFabric from Microsoft Press, but still…

The point of the award system is to “to recognize notable contributions to Microsoft online community forums such as TechNet, MSDN® and Answers.”, so kind of like MVP then I’ll guess. Probably not! Probably Mikael Håkansson and Johan Hedberg will have my head for even comparing the two.

If you would like some more info on the subject visit the MCC home page.

This is mostly one of those “retweet"-posts”, since the publication of the new exams are yesterday’s news (literally it was published on March 30th).

Both fellow bloggers Steef-Jan and Mick Breeze has posted on it and Steef-Jan points out the new parts of the exam. Check out the page here.
(Update 2011-04-01) Be sure to read Steef-Jan's later post about available resources about what to learn before taking the test.

The BRE has been made less important as the exam has the new section: “Implementing Capabilities” that includes RFID (does anyone besides K-mart use that?), EDI and BRE.

Also, there is a new section about WCF and it’s adapters.

Interestingly; nothing is mentioned about AppFabric integration. I find this strange, since it is an obvious stepping stone towards integrating with cloud based services. Also: there is nothing about integrating cloud based services… My theory is that these services are still in a kind of beta stage and are subject to change.

Now to find some reading material as well as certified examiner.

The problem

To quote the online help: “…excluding trailing blanks”.

The issue

I was writing some position based datastrings using SQL server and was continually using the Len function to evaluate the total length of the string. This is a useful technique when the total string length is about 660 positions.

Sometimes when I used Len I got strange results. The strings where much shorter than they where supposed to be.

This was, as I noticed, down to the len-function ignoring trailing blanks. *sigh* SQL server did that thing again that gives me a headache.

Examples

select len(‘ ’) –- returns 0
select len(‘ X’) –- returns 2
select len(‘ X ’) –- returns 2

A solution

A useful solution to this problem is given in the books online article: replace the blank, using the CHAR-function when evaluating the string length:

LEN(REPLACE(' x ', CHAR(32), '_')) -– returns 3

A better solution

A better and simpler solution was provided by commenter "iain", thanks! Simply use the DATALENGTH-function instead.

select DATALENGTH(‘ ’) –- returns 1!
select DETALENGTH(‘ X’) –- returns 2
select DATALENGTH(‘ X ’) –- returns 3!

“Effect” is not a good word for it. Maybe “messes up”, “disturbs”, or “kills” is better.

After installing the Cumulative Update (CU) referred to as KB 2429050 everything seemed ok until some of the jobs collecting in/out statistics from the DTA database failed. Actually, they did not fail; they simply reported that no messages had passed through BizTalk. This was, of course, wrong!

If you want the quick fix: scroll to the end.

How does basic tracking work?

As you might know there is something called BAM tracking. BAM is richer, better and more flexible than basic tracking. By default BizTalk tracks all events of messages going in and out of the message box as well as instances of services. All that data is useful for debugging, statistics and to answer the age old question from the people that we develop our applications for: “Has my file passed through BizTalk?”.

image

In order to make this possible and still not weigh down the message box with data that really should not be there, the data is moved to a separate database called BizTalkDTADb. This database is famous for bloating up, especially it’s log datafile.

The job of getting the data from the message box into the DTA falls to one or more hosts in BizTalk that are marked with “Allow Host Tracking”. Note the word host and not host instance. One host must be configured to handle this job. You cannot configure BizTalk hosts without defining at least one tracking host.

One very good optimization is to configure a host that does all the tracking and to set this host to be the only one handling tracking. MSDN has a good article about it. This way you always have tracking active (since you never need to stop that host) and all the other hosts do not get “bogged down” by tracking overhead.

You can turn tracking off all together if you really want to though. The downside to this is that you will most likely regret the decision.

How do I know if I have a problem?

You might not use the info in the tracking but BizTalk still tracks data. If you have this issue – the data will not be moved from the message box! Your message box will continue to grow and performance will go down.selectfromtdds

There is a quick and easy way to find out if you have a problem. BizTalk dumps all failed tracking to a separate table called TDDS_FailedTrackingData in the BizTalkDTADb. This table should be empty if your DTA clean up jobs are running. Just run a simple select * and order descending. Look at the newest row. If the dtLogTime is resent compared to when a message lastly was submitted to BizTalk; you might have a problem my friend.

The table contains the error that caused the event tracking to fail. My error messages was not exactly helpful but it pointed me to a forum post that helped solve the issue. The message reads:

TDDS failed to execute event. Stored Procedure dtasp_MsgOut failed to run. Stored Procedure dtasp_MsgOut Parameter Count and IPersistQueryable Item count not the same, Expected Count: 0, IPersistQueryable Count: 14

Other procedures than dtasp_MsgOut are mentioned in other error messages, like: dtasp_ServiceInsert, dtasp_CtxInsert or dtasp_ServiceEnd.

Perhaps someone other than me can make sense of it.

selectfrommessageAnother way is to take a look at one of the tables that should contain tracking data. Do a simple select * on the dta_MessageInOutEvents table. Order descending by dtInsertionTimeStamp. The newest row should be fairly resent depending on the load on your system. If the last inserted row was on the day you installed the CU, then you know what caused it.

Yet another way is to look at the DTA database size and compare it to how it looked before you installed the update. If it is much smaller than in should be, then you might have a problem.

Enough! How do I solve the issue?

This solution does not answer why. It only shows you how.

You have to configure a new tracking host with, and this is important, a new name. No joke! If you simply restart the host or delete it and create a new one, then it will not work.

Follow these simple steps (step 7 and 8 are optional):

  1. Go get your passwords and BizTalk users and groups since you probably have not used them in a while.
  2. Stop the host instance(s) that handle the tracking.
  3. Create a new host (not instance) and call it something different compared to the existing one. Remember to check the option “Allow Host Tracking”. Later update: This might not be necessary. Check out the update below.
  4. Create a new host instance for the host. Depending on your setup, you might need to create more than one. Just make sure that the setup looks like it does for the existing host and host instances.
  5. Start the new host instance(s).
  6. Delete the old host. This will automatically delete all host instances.
  7. Open regedit and do a search for your now deleted host name. Notice that it is not gone from the registry.
  8. Run the select-query mentioned above on the TDDS_FailedTrackingData. If new data passes though the system and you do not get any more rows here, everything works fine.

Some of you might think that there must be a simpler, better way of doing this. I have yet to find it. If you however have this issue and feel that you do not want to manually update you can always write a script. There are good ones for creating hosts and host instances out there.

Update!

Forum user Phil D Smith posted that he managed to recreate a dedicated tracking host with the same name. He claims to have succeeded by

  1. Moving the tracking to the default host by allowing that on that host.
  2. Restarted that host (has to be done anyway).
  3. Stopped the dedicated tracking host instance.
  4. Deleted the tracking host.
  5. Created a new tracking host with the original name.

Now I was pretty sure I tried this but I was obviously wrong.

Closing statement

Further research is needed since failed tracking does not seem to be an issue for everyone. There are as always a lot of variables when you compare different setups, we all know this.

I would love feedback! Both about facts that I might have gotten wrong, but also about your setup and your experiences. Most importantly: the answer to the question “why”.

Have you ever gotten an error message something like this:

The XLANG/s runtime failed in terminating the instance “looong ugly GUID” of service type "Full name of orchestration goes here". Exceptions.Messages.Preparation

Exception type: BTXServiceTerminateFailedException
Source: System.Xml

Well I did, and I also discovered that very few people have. If you “bing” for the exception type you will only score one good hit. During my search for the answer I actually learned something new about BizTalk; if an orchestration persists, all messages, that are to be persisted, must validate.

In my case I took over an orchestration at a client and started to “optimize” it. I added a terminate shape in a catch all branch of a decision shape and when the shape was hit I got the error message above…a lot of them. In fact one instance of the orchestration would generate hundreds of these error messages.

The trouble I had was easily solved as a message of type XmlDocument was initialized by simply calling the New(). This creates an empty class. A totally empty document cannot be serialized (as the stream is empty) and therefore the orchestration could not be persisted.

The solution

It was simple once I understood what had gone wrong and knew about the ins and outs of persistence. I simply set the XmlDocument using the Load method and feed it a “dummy”-tag.

The way I found this solution was by looking further down in the error message. After all the gibberish about BTXServiceTerminateFailedException there is additional error information.

Additional error information: The part 'part' of message 'outDoc' contains zero bytes of data.

This is the error that the validation triggered and if you solve that error the persistence error will also be solved.

Thanks

I would like to thank Guo Ming for being the only guy who had ever posted on this error message before.

Inspired by all the usual "turn a page" and "a new life starting today" I decided to update the theme on the blog.

So, to everyone using the gregorian calendar: Happy New Year!

More Posts « Previous page - Next page »