The REST Start Kit for BizTalk Server is about providing support for GET, POST, PUT and DELETE operation, for both Receive and Send ports together with support for both XML and JSON.

In a former post, I showed how to expose REST services from BizTalk. However I never got it to work for GET operations and did not cover consuming other REST services through Send Ports. Consuming REST services was however greatly covered by Nitin Mehrotra in his post “Invoke ReSTful Web Services with BizTalk Server 2010”. In fact I’ve included Nitins code in this solution to compete the REST Start Kit for BizTalk Server.

Is this post interesting to me?

We are seeing a broader diversity among clients, hosted on various smart devices, running on different runtimes. Commonly among all these, is that applications are becoming commodity with a relatively short life-cycle. These applications needs to be rapidly developed using frameworks that can be re-used on other platforms. These and other factors are reasons we see a broad adoption of HTML5 and JavaScript today.

When developing these kind of applications, REST-based services are the preferred choice by far. Not only because it’s easy to consume using JQuery (resource orientated JavaScript library), but equally important is the fact that REST is lightweight. -Bandwidth is not infinite on mobile devices!

Is this important to BizTalk? Yes it is! BizTalk Server is Microsoft’s middleware platform. And as such it should manage REST. Arguably, many web-based application are built together with the service layer in the same application. But what happens when the client needs to access a back-end system like SAP or Oracle EBS? BizTalk can handle these with ease, however it can’t bridge them to REST. 

I’ve tried my best to make the start kit easy to use. However I do realize that many of the topics mentioned in the post are outside the comfort zone for many. But if you only care about HOW to use it, just skip the “How it works” section of the post.

What is REST?

I assume you already know what REST is, but never the less… Representational state transfer (REST) is a light weight protocol introduced by Roy Fielding around the year 2000. REST exposes services over HTTP, and can receive the request either through the URI, as in the case of a GET or DELETE operation, or through the payload (POST and PUT).

HTTP verbs are essential to REST, as REST is focused on entities or resources. For instance, you wouldn’t expose a DeleteCustomerOrder method using REST. Instead you’d let the client access your recourses using the URI, Eg http://somedomain.com/Customers(1234)/Orders. If the user was to delete an Order (with an id of 5678), he or she would call http://somedomain.com/Customers(1234)/Orders(5678) using the HTTP DELETE verb. Using the GET verb, would return the actual order (given that it was not already deleted). If the resource was deleted (or never created), the user should receive a 404 status code.

POST (and PUT) works a bit different, as it receives it’s request through the payload. This behavior is similar to SOAP (which only uses POST), however REST does not require a specific format (as opposed to SOAP which uses the SOAP envelope). In fact, you don’t even have to use XML. Arguably, most clients call REST services using the JSON format.

If you don’t know about JSON, it’s a lightweight data format, commonly used by JavaScript and JQuery. Part from being less verbose then XML, it can be parsed to an object on the client which makes it easier to navigate (as oppose to using XPath).

As such, REST is all about HTTP, and works very much like the architectural principles of the World Wide Web. This is not an accident, as Roy Fielding was one of the principal authors of the HTTP specification.

Many more things could (and should) be said about REST (and RESTFul services), but this is not the place. However, it’s important to point out the challenges we face using BizTalk:

1. BizTalk requires a Message!

This is somewhat problematic as all parameters for a GET or DELETE request is passed along with the URI, not the payload. Because of this, we need to intercept the request and create a “real” message using the parameters passed on by the URI. Using a dynamic URI’s reveals yet an other challenge, as URI’s are generally of a static nature for BizTalk. This is where the WebHttpBinding comes in to place, as it let’s us define a base URI which our service (Receive Location) will listen to, regardless of what proceeds the base URI, Eg http://somedomamin.com/Customers(1234)

2. BizTalk only accepts POST!

This was the most difficult problem to solve, and I’ll get more into detail on this later. But the basic principle is that upon creating the message described before, it will add necessary meta data to the message to let BizTalk think this is a POST message.

3. BizTalk only understand XML!

If the consumer sends a GET request to the service, it also passes along information in the HTTP header, letting us know what format is expected. This is done through the “Accept” header, and could be either “application/xml” or “application/json” (there are others, but xml and json are the only supported ones using the REST Start Kit).

In case of a GET request, the outgoing message will get converted from XML to JSON using JSON.Net. If the consumer sends a POST request, the header could state: Content-Type: application/json. If this is the case, the incoming request will also be converted.

How it works

To make use of the REST Start Kit for BizTalk you need to expose your service or consume another REST service using the WCF-Custom adapter with the binding set to WebHttpBinding. In addition to selecting the binding, you also need to add a behavior; BizTalkRESTRequestHandler for the Receive Location and BizTalkRESTTransmitHandler for the Send port. These behaviors takes care of underlying pluming needed to make BizTalk accept REST based messages. This is all you have to do.

Exposing REST services (Receive Locations)

GET and DELETE

Regardless of what HTTP verb is used, BizTalk only exposes one usable method, – the “TwoWayMethod”. This is because BizTalk does not really care about the actual message until it hits the pipeline. The TwoWayMethod does however care about the HttpRequestMessageProperty.Method to be “POST”.

In order to send anything but a POST request to BizTalk, we need to intercept the message and convert it to a POST, or create a new message (GET & DELETE) and set the HttpRequestMessageProperty.Method to “POST”. This action needs to be done in the SelectOperation method of an IDispatchOperationSelector behavior. As we need to use the WebHttpBinding, the behavior I’ve created inherits from WebHttpDispatchOperationSelector.

As said before, a GET or DELETE request passes it’s parameters through the URI. You may have a base URI (declared on the Receive Location) set to http://localhost:8080/MyService while the consumer may call your service using http://localhost:8080/MyService/Bookings/2012/11, expecting all bookings from November 2012. I this case the BizTalkRESTRequestHandler behavior parses the URI and passes a BizTalkWebHttpRequest message to BizTalk that would look like this:

<ns0:bizTalkWebHttpRequest 
  method="GET" 
  xmlns:ns0="http://bLogical.RESTSchemas.BizTalkWebHttpRequest/1.0">
  <ns0:params>
    <ns0:param>2012</ns0:param>
    <ns0:param>11</ns0:param>
  </ns0:params>
</ns0:bizTalkWebHttpRequest>

On the other hand, the user might use a bit more complex URI such as: http://localhost:8080/MyService/Employees?firstname=Caren&lastname=Smith  

In this case you need to instruct the behavior how to parse the URI. If you’ve ever created a REST service using only WCF, you’re probably familiar to the UriTemplate class, which is also used with the BizTalkRESTRequestHandler behavior. In the sample above, the UriTemplate could be: 
Employees?firstname={fname}&lastname={lname} 

Which in turn would give you BizTalkWebHttpRequest looking like this:

<ns0:bizTalkWebHttpRequest 
  method="GET" 
  xmlns:ns0="http://bLogical.RESTSchemas.BizTalkWebHttpRequest/1.0">
  <ns0:params>
    <ns0:param name="fname">Caren</ns0:param>
    <ns0:param name="lname">Smith</ns0:param>
  </ns0:params>
</ns0:bizTalkWebHttpRequest>

Lastly, if the consumer requires an XML formatted response, the response from BizTalk will be sent straight back to the consumer. However, if the user expects JSON, the response need to be parsed. The DispatchOperationSelector behavior, described above, is only executed on the incoming request, so we need to add an other behavior, –a IDispatchMessageInspector to handle the outgoing response. This behavior is called BizTalkRESTResponseHandler and is added by the BizTalkRESTRequestHandler. In that sense it’s internal and never exposed to you. Never the less, an IDispatchMessageInspector exposes two methods; AfterReceiveRequest and BeforeSendReply. The incoming header is added to the operation context in the  AfterReceiveRequest method, and if the “Accept” parameter is set to “application/json”, the response is converted to JSON and sent back, along with the WebBodyFormatMessageProperty set to WebContentFormat.Json.

PUT and POST

If the client sends a POST or PUT request with a content type set to “application/xml”, nothing is done but passing the message to BizTalk. Only it the content type is set to “application/json”, it is then parsed to XML using JSON.Net. If you expect a message to match a BizTalk generated schema like this:

<ns0:Tweet xmlns:ns0="http://yourns.Tweet">
  <Author>wmmihaa</Author>
  <Text>XML Rocks!</Text>
</ns0:Tweet>

…the JSON message needs to included the namespace as:

{"ns0:Tweet":{"@xmlns:ns0":"http://yourns.Tweet","Author":"wmmihaa","Text":"XML Rocks!"}}

Also, all JSON properties will be parsed as Xml Elements, not Attributes!

Consuming REST services (Send Ports)

As I said before, I’ve “borrowed” the consuming part from Nitin Mehrotra's sample, and added support for POST and PUT. To consume a REST service from BizTalk, the experience is pretty much the same as when exposing a service using a Receive Location. Set the Send Port Transport to WCF-Custom and the binding to WebHttpBinding. As with the Receive Location, we also need to add a behavior, - this time the BizTalkRESTTransmitHandler. This is a IClientMessageInspector. The actions of this behavior differs depending on the SOAP Action Header which you state in the WCF-Custom Transport Properties:

GET and DELETE

To consume a REST service you have to pass a BizTalkWebHttpRequest message to the adapter.

<ns0:bizTalkWebHttpRequest 
  method="GET" 
  uriTemplate="/Events/{id}" 
  xmlns:ns0="http://bLogical.RESTSchemas.BizTalkWebHttpRequest/1.0">
  <ns0:params>
    <ns0:param name="id">123</ns0:param>
  </ns0:params>
  <ns0:headers>
    <ns0:header name="Content-Type">application/xml</ns0:header>
  </ns0:headers>
</ns0:bizTalkWebHttpRequest>

The UriTemplate together with your parameters and the Send Port URI, will make up the actual To URI of the message. For instance, if the Send Port URI is set to http://somedomain.com the message above would be sent to http://somedomain.com/Events/123.

The response is at this moment never parsed, since most REST services does respond XML if asked to…

POST and PUT

Except for adding necessary headers, the POST and PUT request does very little. Keep in mind though, you will always post a message to the URI set on the Send Port. You may find it tempting to use the same Send Port for all calls to a particular REST service, and although this will work for GET and DELETE, it wont work for POST and PUT. This is because the behavior has no knowledge of your intentions other than the payload. I could have solved this using SOAP Action Headers or some promoted property, but I didn’t… (sorry). 

How to use it

1. Download the bits from CodePlex.

The solution has four projects:

2. Register the behavior

In order to see the behaviors in the Select Behavior Extension dialog you need to register the behaviors. This can be done in different places such as in the machine.config or the BTSNTSvc(64).exe.config file. However I find it more convenient to set it on the Receive- and Send handler in the BizTalk Administration Console, as you’d otherwise need to update the config files on all BizTalk machines (given you have more then one)

To do this, select the WCF-Custom adapter under Platform Settings –> Adapters in the BizTalk Administration Console. Double-click the Receive Handler and click the Properties button. Then click the Import button and browse to the WCF-Custom.BindingsExtension-ReceiveHandler.config found in both the solution folder and the installation folder. Repeat the steps for the Send handler using the WCF-Custom.BindingsExtension-SendHandler.config file.

If you want to register the behaviors in the machine.config file(s), just copy the behavior extensions to the <behaviorExtensions> element in the appropriate files.

3. Install the BizTalkWebHttpRequest schema

The BizTalkWebHttpRequest is part of the bLogical.RESTSchemas project. Update the Application Name in the project properties, and deploy it to BizTalk.

4. Use the behaviors

Receive

1. Create a two-way Receive Port and Receive Location
2. Set the Transport to WCF-Custom
3. Set the URI to something like http://localhost:9090/MyService, and select the webHttpBinding in the Binding tab
4. Select the Behavior tab, and right-click the EndpointBehavior node on the left, and select Add extension
5. Select the BizTalkRESTRequestHandler

6.  Set the UriTemplate to match your expected incoming request. You can have multiple UriTemplates delimited using a pipe, eg

/Person/id={pId}| /Person/firstname={fname} | /Person/lastname={lname}.

This would make your service accept incoming GET or DELETE request like these:

http://localhost:8080/myservice/Person/id=123
http://localhost:8080/myservice/Person/fistname=Caren
http://localhost:8080/myservice/Person/lastname=Smith

Send

1. Create a new Static Solicit-Response Send Port
2. Set the Transport to WCF-Custom
3. Click the Configure button and set the URI to the base URI of the service. Eg http://externalservice/service.svc. Select the webHttpBinding in the Binding tab
4. Select the Behavior tab, and right-click the EndpointBehavior node on the left, and select Add extension
5. Select the BizTalkRESTTransmitHandler from the list of extensions

5. Install and use the Sample Application

The BizTalkSampleApplication project has six samples:

The BizTalkSampleApplication is deployed to an appplication with the same name. The bindings file are included in the project. Before you try out the Consume* scenarios, make sure you’ve started the ExternalSampleApplication (just right-click the RESTService.svc file and select View in browser).

After you’ve deployed the solution to BizTalk, you need to redirect all FILE ports to the FILEDROP folder (part of the zip file)

All Consume* samples uses the same Receive Location; ReceiveConsumeREST and submits the message to the RESTService.svc in the ExternalSampleApplication.

To test the ExposeGET sample, you can use the browser and hit http:localhost:9999/Events/id=123 (any number will do)

To test the ExposeDELETE and ExposePOST you’ll need to use some other tool like Fiddler.

To use the sample with Fiddler:

1. Select the Composer tab
2. Select the Verb and set the URI
3. Set the Request Headers (sample file included in the solution)
4. When testing POST, generate an instance of the Event schema in the BizTalkSampleApplication or copy the content from the Event_output.xml file in the FILEDROP folder.

That’s it. Let me know if you run into any issues. If you find it useful TWEET it!

HTH

//Mikael

This ports is no longer valid. An update can be found here:

Solutions for consuming REST services from BizTalk has been around for a while, and Jon Flanders has an excellent post about it.  However, very little has been told about exposing REST endpoints, and even less using JSON. If you don’t know about JSON, it’s a lightweight data format, commonly used by JavaScript and JQuery. Part from being less verbose then XML, it can be parsed to a object on the client which makes it easier to navigate (as oppose to using XPath). This can come to good rescue for UI devs who apparently don’t understand XPath ;)

I haven’t yet been in a situation where I’ve had to expose REST/JSON endpoints from BizTalk, but as Kent Weare was being hackled by Bil Simser (MS Word MVP), I was eager to help out.

I begun by creating a custom WCF MessageInspector. My plan was to parse the incoming JSON message to an XML message, and also to change the HTTP verb from GET to POST if the client sent a GET request (BizTalk requires POST). As it turns out, the HTTP verb/Method, can not be changed in the IDispatchMessageInspector. If it was to be changed it would have to be earlier in the channel stack.

Prior to the MessageInspector is the OperationSelector, so I went on creating one implementing the IDispatchOperationSelector interface. After moving the logic from the inspector to the SelectOperation method in the OperationSelector, I ran into a new problem. The method was never called. It seems BizTalk is adding it’s own OperationSelector through its HostFactory. As I wanted to host the Receive Location in a In-Process host (no IIS), making my own HosFactorythis wouldn’t work either…

I was forced to dig even deeper in the WCF channel stack. Next step was a custom Encoder. Luckily I found a sample in the SDK which was pretty easy to use. The only problem was I couldn’t access the HTTP verb. However after all this, I was willing to accept this trade-off.

Next up was the serialization and deserialization of JSON. Bil Simser pointed me to the JSON.Net project on codeplex, which made it very easy:

XmlDocument doc = new XmlDocument();
doc.LoadXml(xmlString);

string jsonString = JsonConvert.SerializeXmlNode(doc, Newtonsoft.Json.Formatting.None, true);

How to use the sample:

1. Download the sample here
2. Either run the bLogical.JsonXmlMessageEncoder.Setup.msi or build and add the bLogical.JsonXmlMessageEncoder to the global assembly cache (GAC).
3. Open BizTalk Administration Console. Browse to Adapters and right-click the WCF-Custom Receive Handler. Select Properties.
4. Click the Import button, and select the WcfExtensions.config file found in the project.
5. Deploy the FortisAlberta project to BizTalk.
6. Import the FortisAlberta.BindingInfo.xml to the FortisAlberta Application
7. Start the FortisAlberta Application.
8. Run the WebApplication1 project, and submit an OutageReport.

How to configure a Receive Location manually

1. Add a Request/Response Receive Port and Location.
2. Set the transport to WCF-Custom (no need to host it in IIS).
3. Set the binding to customBinding, and remove the existing binding elements.
4. Add the "jsonXmlMessageEncoder" and the "http transport" extensions.
5. Enable the port.
6. You can use the XmlToJSONConverter that comes with the project to generate the expected JSON format from an XML instance, or use any of the online conversion sites like this one.

How to call the service

<script type="text/javascript">

    jQuery.support.cors = true;
    var jsonRequest = '{"Tweet":{"Author":"wmmihaa","Text":"BizTalk Rock!"}}';

    function ReportOutage() {
        $.ajax({
            type: 'POST',
            url: http://yourdomain.com/submitTweet,
            data: jsonRequest,
            contentType: "application/json; charset=utf-8",
            dataType: "json",
            success: function (msg) {
                alert(msg);
            },
            error: function (xhr, ajaxOptions, thrownError) {
                alert('error: ' + thrownError);
            }
        });
    }
</script>

Please not the JSON above:

“{"Tweet":{"Author":"wmmihaa","Text":"XML Rocks!"}}”. This is going to be translated to:

<Tweet><Author>wmmihaa</Author><Text>XML Rocks!</Text></Tweet>

As there are no namespace, you’d need to add one in the receive pipeline. Alternatively, you could add the namespace in JSON:

{"ns0:Tweet":{"@xmlns:ns0":"http://yourns.Tweet","Author":"wmmihaa","Text":"XML Rocks!"}}

Which would come out as:

<ns0:Tweet xmlns:ns0="http://yourns.Tweet">
  <Author>wmmihaa</Author>
  <Text>XML Rocks!</Text>
</ns0:Tweet>

How to call the service without using parameters

function ReportOutage() {
    $.ajax({
        type: 'POST',
        url: "http://yourdomain.com/submitTweet",
        data: '{}', // empty parameter
        contentType: "application/json; charset=utf-8",
        dataType: "json",
        success: function (msg) {
            alert(msg);
        },
        error: function (xhr, ajaxOptions, thrownError) {
            alert('error: ' + thrownError);
        }
    });

Empty parameters are casted to a message that looks like this: <EmptyJsonMessage/>. As you won’t have an equivalent schema in BizTalk, you can’t parse it using an XmlReceive pipeline. If you want to process the message in an orchestration, you’d need to set the message type of the incoming message to System.Xml.XmlDocument.

Restrictions

1. Does not support HTTP GET.
2. Does not support Uri parameters, Eg. http://server/Customers?id=16.
3. The encoder supports both XML and JSON, but not both. It will be restricted to the media type set on the encoder.
4. In this sample the response can not be handled in a streaming manner. If the size of the response message is bigger then what is read from the client, this is likely to cause a problem. I haven’t experienced this myself, but if you get into this problem, contact me and I’ll look into it.

HTH    

In this blog post we are going look at how to manage routing in the new Azure ServiceBus EAI CTP.

As a scenario, I’m going to send a request for information (RFI) to some of my fellow MVP’s. To do that, I’m going to create a One-Way Xml Bridge, to receive the messages. After receiving the RFI message I tend to route it to one of three queues.

1. Create a ServiceBus project

If you haven’t already downloaded the SDK, you can do this here. After you’ve installed the SDK, you can sign in to the labs environment using a Windows Live ID.

Open Visual Studio 2010, and select Create Project. In the list of project templates, select ServiceBus, and Enterprise Application Integration. Give it a name and click Ok.

2. Create a Message Type

Right-click the project and select Add->New Item. At this time there are two types of artifacts you can add; Schemas and Maps. Select Schema and sett an appropriate name. In my case I set the name to RFI.xsd. Continue building up your schema. Notice, you don’t have to promote any nodes as you’d have to do in BizTalk.

3. Designing the Bridge

Double-click the BridgeConfiguration.bcs and drag a Xml One-Way Bridge from the toolbox to the canvas. This is going to be your entry point to your process, similar to a Receive Location in BizTalk. Set the name appropriately, and notice the Router Address which is going to be your endpoint in Azure ServiceBus.

4. Add Queues

As stated before, the incoming RFI message is going to be routed to any of the three queues. You might not your message relayed to a queue, and could there for use any of the other Destinations such as Relay- or External Service EndPoints. Either way, the principle of routing is the same.

Connect the Bridge with all Destinations.

5. Configure the Bridge

Next we’ll define the incoming message type(s). Double-click your Bridge (ReceiveRFI in my case). Click the plus button in the Message Types stage. Select the Message Type you created earlier, and click the arrow button on the right.

6. Enrich the Message

This is the interesting step, where we are going to promote some fields in the payload so that we can route on these in the next step.

First open your schema and select the node you care to use for routing (in my case Receive). Copy the Instance XPath from the Property window.

Then double-click the Bridge, select either of the Enrich stages, and then click the Property Definition button in the Property window. There are two Enrich stages, as you might be using a Transformation, in which case you might want to promote fields from either the original message or the transformed message.

For more information about transformations, have a look at Kent’s post.

Set the Type to XPath, and paste the XPath expression in the Identifier text box. Select the Message Type and set the name of the property. Finish be setting the data type and click the Add button (+). Close the dialog by clicking the Ok button.

7. Set the routing conditions

As you have promoted your property (or properties), you’re now ready to set the Filter Conditions on each of the Connectors. Select one of the selectors and type the Filter in the Property window. Eg. receiver=’SteefJan’ or customerId=1234.

8. Create the Queues

Before we deploy the solution, you need to create the queues. There are several tools for this, but with the samples comes a MessageReceiver project you can use.

Type MessageReceiver.exe <Your namespace> owner <Your issuer key> <Queue name> Create 

After creating the queues verify they are created in the portal.

9. Deploy your solution

Right-click the project and select Deploy. Supply the secret.

10. Test the solution

Along with the MessageReceiver tool, you’ll find a MessageSender project as well. Just type:

MessageSender.exe <Your namespace> <Your issuer key> <Your endpoint> <Path to sample file> application/xml

Use the MessgeReceiver to get the messages from the queues:

HTH

The WCF-SQL adapter provides support for multiple inserts through the Consume Adapter Service feature:

However, sometimes you might want to validate the data on the SQL side before before making the insert. For instance, if you have a collection of Customers, where some of them might already exist in the database, and should only be updated. In such a case, you’d have to first make a database lookup, to determine the state of the Customer and then make either an insert or update.

In such a case, using user-defined table types might be your solution. User-defined tables are similar to ordinary tables, but can be passed in as a parameter.

In my sample, I have a Contacts table, and I’m receiving a collection of Persons where some entities are new and some are to be updated.

Create the User-Defined Table Type

The user-defied table type will serve as our contract.

CREATE TYPE [dbo].[InsertContactRequest] AS TABLE
(
    [PersonNo] [varchar](50) NOT NULL,
    [FirstName] [varchar](50) NOT NULL,
    [LastName] [varchar](50) NOT NULL,
    [Phone] [varchar](50) NOT NULL,
    PRIMARY KEY CLUSTERED ([PersonNo] ASC)WITH (IGNORE_DUP_KEY = OFF)
)

Create the Stored Procedure

The stored procedure takes the user-defined table type as a parameter (@insertContactRequest), then updates all existing rows and inserting all new once.

CREATE PROCEDURE [dbo].[sp_InsertContacts] @insertContactRequest InsertContactRequest READONLY
AS
BEGIN
    
    UPDATE dbo.Contacts 
    SET Phone = r.Phone
    FROM dbo.Contacts c
    JOIN @insertContactRequest r on r.PersonNo = c.PersonNo

    INSERT INTO dbo.Contacts (PersonNo, FirstName, LastName, Phone)
    SELECT r.PersonNo, r.FirstName, r.LastName, r.Phone
    FROM    @insertContactRequest r
    WHERE    r.PersonNo not in(SELECT PersonNo FROM dbo.Contacts)
       
END

Generate BizTalk artefacs

1. In you Visual Studio, right-click the BizTalk project and select Add->Add Generated Items. Select Consume Adapter Service.

2. In the Consume Adapter Service dialog, click the configure button to set the credentials. Click Ok, and then Connect.

3. In the tree-view, select Strongly Typed Procedures, and select your stored procedure in the right pane. Click Add and Ok to generate the schemas.

4. Make your transformation, and complete your solution.

Here is the sample source.

HTH

(Kudos Daniel Östberg)

If you’re a consultant like me, you’ve probably got similar calls from some key account manager, as I did yesterday:

KAM: Hi Mikael. I’m just about to close this super big deal, with this super important Customer.
Me: Really! Good for you.
KAM: Yeah, we’re really close. But to rap it up, I was wondering if you could help me out a bit…
Me: Sure. What do you have in mind?
KAM: Could you come with me to meeting with the customer on Monday? (this happens on Thursday 6PM)
Me (getting suspicious): mmm…What do you want me to do?
KAM: A demo!
Me: Demo of what?
KAM: The customer want us to show how to integrate Dynamics CRM Online with SAP using BizTalk.
Me: What?
KAM: Yes, yes. The customer wants us to show it live! You know, they want to see you do it…
Me: ARE YOU HIGH? (I didn’t actually say that, but I was thinking it)
Me: It will not happen! I haven't worked with SAP since 3-4 year ago. I’ve never worked with CRM Online (or off-line for that matter). I’m fully booked tomorrow, and I want to spend the weekend with my family as X-mas is coming up.
KAM: But we need to close this deal…
Me: NO!
KAM: Please…
Me: No way!
(Yada, yada, yada)
Me: Ok, I’ll give it a try (I’M SUCH AN IDIOT!!!!!)

So here it is: How to connect to Dynamics CRM Online from BizTalk Server

To begin with, if you want to integrate with CRM Online, you have two options. Either use an un-typed web-service API or use a tool called CrmSvcUtil.exe to create a proxy class for you. Each of these comes with some challenges and limitations:

Using an un-typed web service, can of course be somewhat messy, but the SDK provides you with the schemas you need (more on that later). The biggest challenge, however, is to authenticate to the service as it assumes you’re using Windows Live Id. Authenticating against the service would require an additional four calls to the service to finally get the authentication tokens needed to create the security header. An then figure out a way to to add the headers in a pipeline. The steps needed are described by Girish Raja here.

The proxy created using the CrmSvcUtil is quite nice, since it’s typed, but of course I can’t use it in a send port. I would  have to make the call using the inline-send approach from within an expression shape in an orchestration. And thereby loose the built in re-send functionality and more, that ships with BizTalk.

As none of these approaches was acceptable, I begun looking for other alternatives. What I really wanted was an authentication behavior, that I could add to my WCF-Custom send port adapter. 

Building the Custom WCF Behavior

What I needed was a Message Inspector that would build up the security header as Girish Raja did in his sample, and then add that header to the SOAP envelope. This class is called LiveIdAuthenticationMessageInspector and inherits from IClientMessageInspector. This gives two methods to my class: BeforeSendRequest and AfterReceiveReply.

public object BeforeSendRequest(ref System.ServiceModel.Channels.Message request, 
    System.ServiceModel.IClientChannel channel)
{
    string securityHeader = HeaderHelper.GetSecurityHeader(this._username, this._password, this._crmUri);
            
    request.Headers.Add(MessageHeader.CreateHeader("Security",
        WSSecurityUsernameTokenProfileNamespace,
        string.Empty,
        new SecurityHeaderSerializer(securityHeader),true));

    return null;
}

In the BeforeSendRequest method is where I can add the security header to the message before the message is sent out. The the BeforeSendRequest method I call a helper class returning the actual header. The GetSecurityHeader method is going through four steps to build up the header:

1. Get Windows Live Device Credentials
2. Register Device Credentials and get binaryDAToken
3. Get Security Token by sending WLID username, password and device binaryDAToken
4. Build up the security header with the token from previous step.

(This sample does not cache the tokens! I strongly suggest you add some caching logic before you run this in production)

After the header is created it is added to the request, using a custom serializer, as it would otherwise be HTML encoded.

public void AfterReceiveReply(ref System.ServiceModel.Channels.Message reply, object correlationState)
{
    Trace.WriteLine("[bLogical] LiveIdAuthenticationMessageInspector:AfterReceiveReply called");
    int index = reply.Headers.FindHeader("Security", WSSecurityUsernameTokenProfileNamespace);
    reply.Headers.RemoveAt(index);
}

When BizTalk (or any other WCF client) receives the response it will throw an exception, as it doesn’t understand the Security header. I might have gone away with adding the http:http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd schema to BizTalk, but as I don’t need it, I just removed it from the header on the way back.

Part from the Message Inspector, I also added an EndpointBehavior and a BehaviorExtensionElement. The LiveIdAuthenticationBehaviorExtensionElement needs to be registered in the configuration using the behavior in BizTalk

After you have registered the behavior, you can focus on the normal BizTalk tasks like building orchestrations and mappings. To get started with consuming the CRM Services, have a look at Richards post. The only thing I’d like to emphasis is that the correct schemas are part of the SDK (sdk\schemas). After you run the Consume WCF Service Wizard, remove all schemas and replace them with the the once in the SDK. Better yet, put all those schemas in a separate schema project, and reference that project from other projects where you’re using them.

1. Add the behavior to the Global Assembly Cache

Open up the LiveIdAuthentication project, build and add it to the global assembly cache.

2. Register the behavior in the configuration

(sorry about the formatting).

<extensions>
  <behaviorExtensions>
    <add name="liveIdAuthentication" type="LiveIdAuthentication.LiveIdAuthenticationBehaviorExtensionElement, LiveIdAuthentication, Version=1.0.0.0, Culture=neutral, PublicKeyToken=698ceec8cebc73ae"/>
  </behaviorExtensions>
</extensions>

You can do this either in a config file (machine.config or BTSNTSvc[64].exe.config) or in BizTalk (WCF-Custom Send handler):

I prefer the later as I would otherwise need to make the changes to all the servers in the group. Just copy the extension element above, into a config file, and import the file from the Transport Properties dialog above. Or you can point to the app.config file in the sample.

3. Add the Endpoint Behavior

Open the send port and click the Configure button to open the WCF Transport Porperties. Select the Behavior tab and right-click the Endpoint behavior node, and select Add extension. Select the liveAuthentication extensions. Select the extension and set the properties.

You’re done.

Use this code as you like, and on your own risk. If you make improvements, I’d appreciate if you notify me. One thing I know could be done better, would be to cache the tokens and re-use them for the next call.

Download the Dynamics CRM LiveId Authentication Behavior sample here.

HTH

Every year my employer hosts a two day event all related to integration or service orientation. Around 450 people attended this year, which I assumes makes it the biggest event of the year in the integration space, with more then 25 sessions on four different tracks. All Microsoft related session were recorded and are now available.

• Microsoft BizTalk Server and Microsoft’s Middleware vision (Marcus Gullberg, Microsoft) –SWEDISH
• Microsoft BizTalk Server & Windows Azure AppFabric (Mikael Håkansson, Enfo Zystems) –SWEDISH
• Windows Azure AppFabric Platform futures (Johan Hedberg, Enfo Zystems) –SWEDISH
• BaseLine for BizTalk Hands On (Martin Rydman & Mikael Håkansson, Enfo Zystems ) –SWEDISH
• Using AppFabric Cache to Maximize the Performance of Your Windows Azure and On Premises WCF Applications (Paolo Salvatori, Microsoft) –ENGLISH
• How to integrate BizTalk Server with Windows Azure Service Bus Messaging. (Paolo Salvatori, Microsoft) –ENGLISH

Enjoy, an we hope you join us next year.

Welcome to Integration days 2011!

If you are in the integration space, you’ll find all kinds of interesting and valuable sessions in any of the four tracks; Strategy, Public, Microsoft and IBM. Each track has six sessions with speakers from both Enfo Zystems and other partner organizations such as Microsoft.

At Thursday evening, you’re invited for dinner with entertainment, which of course will be a great time to meet up with other integration geeks (such as myself…)

And the best of all… –It’s all free, so sign up now!

The event starts on the 13^thof October and covers four tracks, each with six sessions. The Microsoft platform track will cover the following six sessions:

Microsoft BizTalk Server and Microsoft’s Middleware vision

BizTalk Server has been at the center of Microsoft’s Middleware platform for a number of years, to provide a rich set of capabilities for services and integration. AppFabric, both on-premise and on Windows Azure provides additional capabilities as well as some overlapping ones. So what is the strategy here, what is Microsoft up to long term and short term? How will this affect solutions you create and what opportunities will it create for your company? In this session, you will get the answers to these questions.
Presenter: Marcus Gullberg, PM Microsoft Sweden

Microsoft BizTalk Server & Windows Azure AppFabric

Microsoft’s Middleware platform is currently undergoing a change, which in turn offers different solutions with unique capabilities. What is available today, and how can we today make these solutions work together? This session will cover Microsoft BizTalk Server, Windows Server AppFabric and Azure AppFabric, to show how you can extend the reach of your integration platform outside your own domain.
Presenter: Mikael Håkansson, Solution Architect, Enfo Zystems

Windows Azure AppFabric Platform futures

Where is the future of Microsoft’s Middleware platform going? How will we design, build and monitor our solutions in the future? What capabilities will we have in our tool box? These and many other questions will be addressed in this session, which will focus on Microsoft Azure AppFabric Platform and emerging capabilities such as Composite Application, Access Control Center, Caching, ServiceBus Topics & Queues and other enhancements, and Integration.
Presenter: Johan Hedberg, Solution Architect, Enfo Zystems

Using AppFabric Cache to Maximize the Performance of Your Windows Azure and On Premises WCF Applications

Caching is an integral part of an overall scaling strategy. By properly utilizing caching you can radically increase the number of concurrent users your application can service. Much of the caching information available to users today only focuses on server side caching. Server side caching is important, we will cover it in this session, and we will show concrete techniques to maximize its effectiveness . However, this session will also cover client side caching techniques. Client side techniques are often overlooked in spite of the fact that in order to truly hit extreme scale those techniques are nearly always necessary and often end up being bolted on after the fact. After attending this session, the attendees will walk away with the concrete knowledge and code necessary to immediately improve their WCF application performance.
Presenter: Paolo Salvatori and Mikael Håkansson

Deep dive: How to integrate BizTalk Server with Windows Azure Service Bus Messaging

The Windows Azure AppFabric Service Bus and Windows Azure Connect are the foundation for building a new class of distributed and hybrid applications that span the cloud and on premises environments. The Service Bus is an Internet-scale Service Bus that offers secure, scalable and highly available connectivity and messaging capabilities. Windows Azure Connect provides a network-level bridge between applications and services running in the cloud and on-premises data centers. Windows Azure Connect makes it easier for an organization to migrate their existing applications to the cloud by enabling direct IP-based network connectivity with their existing on-premises infrastructure. In this session you will see how to integrate these technologies with BizTalk Server to create solid and cloud-ready solutions.
Presenter: Paolo Salvatori, Senior Program Manager Microsoft

Baseline for BizTalk Hands-on

Baseline provides a comprehensive framework that supports the design, development and maintenance of systems integration solutions. In this session we will provide a practical example of how to use the Baseline methodology and tools to refine project requirements into a working BizTalk solution – tested, documented and packaged, ready for deployment in BizTalk Server 2010. In the process we will use Baseline documents and the Baseline Portal to highlight the main strengths of Baseline.
Presenter: Martin Rydman and Mikael Håkansson

Thanks to everyone attending my TechEd session on Thursday. You can find the scripts I was running here (sorry I didn’t put them up earlier). Let me know if you have any trouble.

If you didn’t attend, you can view the session online here.

Good luck

//Mikael

Thanks to everyone attending my session on integration with Office365 and on-prem applications.

All demos can be downloaded from here: http://blogical.se/files/folders/downloads/entry25152.aspx

I recommended you to start downloading the AppFabric SDK CTP, in which you’ll find the ClientAccessPolicyPublisher sample I was running in the last sample:   http://www.microsoft.com/downloads/en/details.aspx?FamilyID=d89640fc-c552-446e-aead-b1e0d940f31b

We are once again fortunate to have prominent speakers visiting our user group in Sweden. This time it's Kent Weare and Richard Seroter. Both of them have been here before, and have both been much appreciated speakers. (BTW if you are aware of any challenges happening in Sweden at the time of the event such as Iron man, marathon or cage fighting let me know so we can sign up Kent).

Kent Weare recently led a team of authors in their production of the book Microsoft BizTalk 2010: Integrating with Line of Business Systems (Packt Publishing, 2011).  This book walks through multiple technologies and how to integrate with them via BizTalk Server 2010.  Join Kent and Richard as they deliver the Top 10 Things to Know When Integrating with Line of Business Systems.
In these sessions, they will walk us through numerous key principles to follow when doing system integration and they will draw inspiration from their new book.  These principles will be demonstrated by integrating BizTalk Server 2010 with SharePoint, Windows Azure, SAP and software-as-a-service providers.

Monday the 13th of June 18:00 at Microsofts office in Akalla

If you have an opportunity to join us, sign up. If you need help booking travel arrangements such as hotel, let us know by dropping us an email.

Welcome

Configuring the backup job for BizTalk is a fairly simple task, while restoring it is a bit more complicated. By default the BizTalk backup job makes a full backup once a day, and a log backup every 15 minutes. When backups are done, a mark is set on each file. This mark is the same across all databases, and should be used to restore all databases to the same point in time and keeping all databases in a consistent state. –Also, by default, all backups are made to the same directory folder.

The only supported disaster recovery procedure from Microsoft is log-shipping. Nick Heppleston has gone through the trouble of describing this in great detail, and I strongly recommend to read these post before you choose to use any other approach.

A few weeks ago, I sent out a question on twitter, asking whether people used log-shipping or not. I got 24 responses where only 4 used log-shipping.

Although log-shipping comes with many advantages, it is still expensive since it requires a secondary SQL cluster. Most of the people I asked confirmed this was the main reason why they had chosen other alternatives. Since BizTalk doesn’t come with any restore scripts/features other than log-shipping, everyone is left to fix this on their own.

If you’re in the same situation, feel free to download this sample. If it doesn’t fit your solution, it might at least be a good starting point.

The sample comes with two cmdlet’s: Get-Marks and New-RestoreDatabaseFromMark. The first one gives you a list of all marks from all log files. The second one, as the name implies, restores a database to a specific mark. When doing so, the database will be restored from the last full backup before the mark. After that, all log files will be restored in order from the full backup. The last log file will only be restored to the specified mark.

The Get-Mark cmdlet queries the backup output folder to retrieve all marks. The mark is part of the name of each backup file:

Each file is made up of the following parts:

[Server]_[Instance*]_[Database]_[Full|Log]_[Mark]

Eg: SERVER001_DTA_BizTalkDTADb_Log_BTS_2011_01_18_12_06_50_22.bak
* The instance is only present for none default instances.

You can use the New-RestoreDatabaseFromMark cmdlet with or without specifying the mark. Leaving the mark empty is eqvivilent to last mark.

The New-RestoreDatabaseFromMark cmdlet is called per database, why it's easier to create a script for restoring all databases together. The sample comes with a RestoreScript.ps1 script file, which could work as a good start:

$backupPath = "X:\BizTalkBackUp";
$dataPath = "E:\SQL Server 2008\MSSQL10.MSSQLSERVER\MSSQL\DATA";
$logPath = "E:\SQL Server 2008\MSSQL10.MSSQLSERVER\MSSQL\DATA";

  

$mark = Read-Host "Specify mark (use the Get-Marks cmdlet to get all marks or blank to use last mark)";

if ($mark.Length -eq 0)
{
    Write-Output "Restoring to last mark...";
    $mark="";
}

New-RestoreDatabaseFromMark SSODB $backupPath $dataPath $logPath $mark;
New-RestoreDatabaseFromMark BAMPrimaryImport $backupPath $dataPath $logPath $mark;
New-RestoreDatabaseFromMark BizTalkDTADb $backupPath $dataPath $logPath $mark;
New-RestoreDatabaseFromMark BizTalkMgmtDb $backupPath $dataPath $logPath $mark;
New-RestoreDatabaseFromMark BizTalkMsgBoxDb $backupPath $dataPath $logPath $mark;
New-RestoreDatabaseFromMark BizTalkRuleEngineDb $backupPath $dataPath $logPath $mark;

trap [Exception] 
{
    Exit;
}

Write-Output "Done restoring all BizTalk databases" -foregroundcolor "yellow";

The first section of the script defines a set of path variables. In my simple sample, all database files are located in the same folder. This is never a good practice, why you probably have different paths for data- and log files for each database.

To use the sample, open PowerShell and navigate to the sample output folder. Eg:

PS C:\> CD “c:\Program Files\bLogical.BizTalkManagement”

Before you can use the cmdlets, you need to install them. You can do this using the install script:

PS C:\Program Files\bLogical.BizTalkManagement> Install.ps1

After you’ve installed the snapins, you can start using the commands:

PS C:\Program Files\bLogical.BizTalkManagement> Get-Mark "X:\BizTalkBackUp"

or

PS C:\Program Files\bLogical.BizTalkManagement> MyRestoreScript.ps1

Downloads 

HTH

Paolo Salvatori and I, had this idea about using AppFabric Cache as a workflow persistence provider. As we’ve been working on this for quite some time now, we’re happy to finally share the work with everyone else thinking about how workflow persistence work, or perhaps thinking of building your own provider.

Developing your own persistence provider is not the simplest task you can take on, but it’s perfectly doable. I was surprised by the absence of persistence provider samples on the web. The only helpful one I could find was the MemoryStore, which helped me get started.

The funny thing with this sample was that I spent most of the time, trying to figure out why it worked, not why it didn’t.

I want to thank Paolo Salvatori, Manu Srivastava and Ruppert Koch from the AppFabric CAT and Product group for helping out and clarifying how the underlying plumbing works. More on that at the end of this post.

What is Workflow Persistence?

Unless we enable persistence, a workflow instance is only stored in memory. If the host goes down, it will take any evidence of your workflow instance with it. The process of persisting a workflow to a durable repository or storage is known as dehydration while restoring it, is generally referred to as rehydration. If you are a BizTalk person, you are probably aware that the persistence store for BizTalk is the BizTalkMsgBoxDb database.

Much like BizTalk, Workflow Foundation comes with a SQL persistence store, - the SqlWorkflowInstanceStoreProvider. However, with WF it’s optional. -Not only to use persistence, but you are also free to choose other providers, or even build one yourself.

You can enable the persistence either through configuration or code. If you’re hosting your workflow in IIS/AppFabric you can even manage your persistence through the IIS Manager.

Why and when would we use persistence?

You might end up using persistence for different reasons, which might also have an impact on which provider you’d choose. As I’m a “BizTalk guy”, I generally think of persistence in terms of 1) A safety-net in case something goes wrong and 2) Resource management, off-loading running instances to prevent memory consumption. In most cases you’re looking for a safe and robust provider to make sure you are not going to lose sensitive data along with the internal state of the workflow instance.

AppFabric Cache might not be your first choice if you are looking for a secure and reliable store provider. As Stephen W. Thomas put it: “Never put anything in the Cache that MUST not be lost – it is a non-transactional cache, not a database”.

However, persistence is also about scalability. If you, for example, are using WF to control the page flow of an ASP.Net application running on multiple web servers, you’d need persistence to make sure one server can continue a workflow which initiated on another server. Furthermore, in any scenarios where workflow services are accessed in a session-less manner, it is also important to persist immediately when the workflows gets idle. Otherwise the second request to the workflow might be re-directed to the other server, while it’s still active in memory on the first server. We can control this behavior using the WorkflowIdleBehavior.TimeToUnload setting in the web-/app.config:

<behaviors>
  <serviceBehaviors>
    <behavior>
      <sqlWorkflowInstanceStore connectionStringName="persistenceStore" />
      <serviceMetadata httpGetEnabled="true"/>
      <workflowIdle timeToUnload="00:00:00"/>
      <serviceDebug includeExceptionDetailInFaults="true"/>
    </behavior>
  </serviceBehaviors>
</behaviors>

The default value for the property is 1 minute. Unloading a workflow implies that it is also persisted. If TimeToUnload is set to zero the workflow instance is persisted and unloaded immediately after the workflow becomes idle. Setting TimeToUnload to MaxValue() effectively disables the unload operation. Idle workflow instances are never unloaded.

In some cases, for example when using WF to control the UI of your application, , you may consider using a less reliable provider. In such cases storing your workflow state in AppFabric Cache could be a good fit. In fact I would not be surprised if Microsoft shipped an AppFabric Cache provider in a near future.

But I won’t lie to you, – I think AppFabric Cache is super cool, and would make up any excuse to play with it!

What is correlation?

First lets have a look at the sample workflow. It’s a very simple sample to simulate a process of working with some sort of document form. The user may save the document, to come back later to continue working on its content.

1. The client calls the workflow service through the CreateDocument method. This will cause the runtime to create a new instance of the workflow.
2. After receiving the new Document, the workflow assigns it to a local variable.
3. As the response is sent back to the client, the workflow initializes the correlation on the Document.Id (string)
4. While the workflow waits for the client to send an updated document it will become idle, and therefore persisted to the AppFabric Cache store.
5. The client invokes the UpdateDocument method. As the workflow instance is not in memory, the runtime will ask the persistence provider for the serialized workflow and will then rehydrate it.
6. The workflow instance will perform the following actions:
• update the local document variable;
• return the response;
• persist its internal state,
• continue to listen for updates until the user submits “I’m done”. 

The only “complex” part of this workflow is making the second call (and all subsequent calls) to be “routed” to the right workflow instance. Keep in mind that there might be any number of concurrent documents being processed in the system. This is solved using the WF Correlation and in particular Content-Based Correlation.

If you’re a BizTalk guy, this is nothing new; however when I teach BizTalk classes people seems to have a bit of trouble to understand the concept. A correlation set is basically a “primary key” for the instance of the workflow. The id is created on the first call to the workflow, and needs to be passed on to the runtime for all subsequent calls. Using WF, this is very simple, all you need to do is to set the CorrelationInitializers for instance a SendActivity:

This will cause the Workflow runtime to create the instanceId (Guid), and add it to the binding context of the client. This way the instanceId is passed along with the soap header.  Although this might work fine in most situations, there are some limitations. -If the client does not have the instanceId, as when different clients are interacting with the same workflow, this won’t work. You are therefore better off using the Content based correlation type; Query correlation initalizer. For more information about the different kinds of correlation that WF provides, have a look at Paolos post.

This correlation type lets you define the id of your workflow, based on the content of the message. But keep in mind it needs to be unique.

So how does a persistence provider work?

The simple answer is: It serializes the workflow along with its metadata to a store. The persistence provider could use any durable media such as a file, database or queue. The AppFabric Cache does not fall in the category of being a durable media, but we will ignore Stephens lame warnings for now, and just concentrate on the sweet coolness of AppFabric Cache.

To begin with, a persistence provider needs to inherit from InstanceStore (System.Runtime.DurableInstancing), which is an abstract class with some operations you need to overload. The most important one is the BeginTryCommand. This is a universal operation which will be called from the runtime. With it comes an InstancePersistenceCommand parameter which tells you what the runtime expects you to do. This can be any of the following:

• CreateWorkflowOwnerCommand – Registers the host as the owner of the workflow instance
• LoadWorkflowCommand – Creates a new instance of the workflow, and returns it to the runtime.
• LoadWorkflowByInstanceKeyCommand – Loads the instance of the workflow
• SaveWorkflowCommand – Saves the instance together with correlation keys.

(there are more commands, but this will do for now…)

If the last SaveWorkflowCommand has the CompleteInstance set to true, this indicates the workflow is done. In my case, this is where I clean up the cache, even though the cache will eventually clean itself up, as I add to the cache using the timeout parameter. You can set this yourself in the web.config.

...
<behaviors>
  <serviceBehaviors>
    <behavior>
      <bLogicalPersistenceStore cacheName="bLogical" timeout="00:05:00"/>
      <serviceMetadata httpGetEnabled="true"/>
      <workflowIdle timeToUnload="00:00:01"/>
      <serviceDebug includeExceptionDetailInFaults="true"/>
    </behavior>
  </serviceBehaviors>
</behaviors>

The LoadWorkflowByInstanceKeyCommand is where I got stuck. As I said in the beginning, –it worked, I just couldn’t figure out why. Below is the code in my LoadWorkflowByInstanceKey method:

private IAsyncResult LoadWorkflowByInstanceKey(InstancePersistenceContext context, 
    LoadWorkflowByInstanceKeyCommand command, 
    TimeSpan timeout, 
    AsyncCallback callback, 
    object state)
{

    Key key = _cacheHelper.GetKey(command.LookupInstanceKey);
    Instance instance = _cacheHelper.GetInstance(key.Instance);

I’m loading the workflow using the LookupInstanceKey value, which is a Guid. Where did that come from? I was expecting my correlation key, – the Document.Id. I was starting to believe the correlation key was stored (together with the LookupInstanceKey) somewhere else. I started to trace the message to see if there was anything hidden in the header, but there wasn’t. I tested different client, collaborating on the same workflow instance, and that worked too. I even set up an environment with multiple servers –AND IT STILL WORKED! How was this possible?

As I was losing sleep over this, I turned my faith to Paolo Salvatori, hoping he could share some light on this. Eventually, Manu Srivastava and Ruppert Koch explained how the magic works. It turns out the LookupInstanceKey is a 128-bit hash of the actual content based key.

Manu Srivastava:

“The Hash == InstanceKey.Value == CorrelationKey == LookupKey. These terms all mean the same thing. InstanceKey.Value contains the value of the hash, which is of type GUID and represents the correlation / lookup key [http://msdn.microsoft.com/en-us/library/system.runtime.durableinstancing.instancekey.aspx]”

“It is the responsibility of the Provider to store the Workflow Instance, the Correlation Keys *and* the mapping between the two. The LoadWorkflowByInstanceKeyCommand has a LookupKey as an argument. This is the correlation key. This is the key the custom provider implementation must use to identify and then return the Workflow Instance. The hashing algorithm used is irrelevant to the implementation of the Provider. The Provider only has to worry about storing the InstanceKey.Value and retrieving an instance via the InstanceKey.Value. The transformation between Document.Id and the InstanceKey is a WF runtime detail that you do not need to worry about for your implementation; thus, you do not need to worry about the hashing algorithm itself. When you persist an instance, save the InstanceKey.Value and its mapping to InstanceId. When you load an instance, use the LookupKey to find the correct InstanceId and return the WorkflowInstance. That's it. You don't need to go from Document.ID to hash or vice-versa...thats handled at the WF Runtime Layer. Just focus on saving InstanceKey.Values and loading by LookupKey. =) InstanceKey.Value == LookupKey in this particular scenario”

In other words: – Concentrate on the problem, and leave the plumbing to us! –Point taken.

Running the sample

Download and install:

Before running the sample, you need to properly configure the environment and in particular you have to create the cache used by the custom Persistence Provider.

To accomplish this task, you can perform the following steps:

1. Download the Windows Server AppFabric.

2. Read Scott Hanselman’s post as to how to set it up.

3. Open Caching Administration Windows PowerShell through the Start menu, and run the following commands:

• Start-CacheCluster
• New-Cache bLogical

4. Download and run the sample the sample.

This is probably a good time to point out, that the sample provided with this post is not production ready. However, If you want to take it further, I’d be happy to help out!(</disclaimer>)

Again, many thanks to Paolo Salvatori, Manu Srivastava and Ruppert Koch from the AppFabric CAT and Product group.

HTH

I’ve taken the time to reflect over the blogs I follow, and though about the impact it made to my work over the last year. I’ve also credited post for being just interesting or cool, and perhaps useful in the future.

I’ve done this post as a tribute to those who took there valuable time to share their thoughts and experience with the rest of us.

How to exploit the Text In Row table option to boost BizTalk Server Performance

by Paolo Salvatori

As I’ve spent much of the last year focusing BizTalk performance, I found these settings to make a huge difference. Applying the “text in row” table option on all tables storing the actual message, improved not only the message throughput but also greatly reduced the CPU utilization.   
http://blogs.msdn.com/b/paolos/archive/2010/03/16/how-to-exploit-the-text-in-row-table-option-to-boost-biztalk-server-performance.aspx

Creating a bootable VHD the easy way

by Hugo Häggmark

Creating a bootable virtual hard drive is not as easy as it’s made to believe! I struggled quite a bit, and as I was complaining about it in the office one day, Hugo said: “I’ve written a blog post about it…”. These posts are great, and it really is the EASY way!

http://www.hugohaggmark.com/2010/09/23/creating-a-bootable-vhd-the-easy-way/
http://www.hugohaggmark.com/2010/11/15/creating-a-bootable-vhd-the-easy-waycontinued/

How To Boost Message Transformations Using the XslCompiledTransform - Series

by Paolo Salvatori

Paolo Salvatori’s blog should be on every BizTalkers feed subscription. I’m amazed by the details and the effort of his research. In this post he shows how to use the XslCompiledTransform class instead of using the traditional XlsTransform which is used by the Transformation shape in orchestrations. I found there to be an issue of memory consumption that needs to be sorted out. Never the less, it’s a very interesting concept, and a great series of posts.

http://blogs.msdn.com/b/paolos/archive/2010/01/29/how-to-boost-message-transformations-using-the-xslcompiledtransform-class.aspx
http://blogs.msdn.com/b/paolos/archive/2010/04/08/how-to-boost-message-transformations-using-the-xslcompiledtransform-class-extended.aspx

WCF-SQL Adapter Table Operations

by Steef-Jan Wiggers

As I was working with the WCF-SQL adapter a couple of months ago, I was really happy to find Steef-Jans post on the subject. The post shows how to use all the table operations using the “new” sql adapter. Thanks Steef-Jan, and again –Congrats on the MVP award.
http://soa-thoughts.blogspot.com/2010/08/wcf-sql-adapter-table-operations.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+blogspot%2FLrBC+%28SOA+Thoughts%2C+EAI+Challenges%29

Four Questions - Series

by Richard Seroter 

Every time my feed reader notifies me of a new “Four Question” post, I instantly have to read it. It’s always interesting to hear what other people within the community and Microsoft are up to, and Richards questions are always relevant to what is currently happening in the field. And of course it always ends up with the last question, where Richards humor and sarcasm comes to good use.

I was especially intrigued by the last post where he interviews one of the true heroes of the BizTalk community – Ben Cline.   

http://en.wordpress.com/tag/four-questions/

Mapping in BizTalk 2010: My favorite new features - Series

by Randal van Splunteren

Randal, also a newly decorated MVP was kind enough to share the features of the new mapper that shipped with BizTalk 2010. If you’re a BizTalk dev, I really recommend you to read these posts as you’re likely to find out some features you didn’t already know about.

http://biztalkmessages.vansplunteren.net/2010/05/07/mapping-in-biztalk-2010-my-favorite-new-features-overview/
http://biztalkmessages.vansplunteren.net/2010/05/07/mapping-in-biztalk-2010-my-favorite-new-features-part-2-functoid-intellisense/
http://biztalkmessages.vansplunteren.net/2010/05/07/mapping-in-biztalk-2010-my-favorite-new-features-part-1-intelligent-linking/
http://biztalkmessages.vansplunteren.net/2010/05/07/mapping-in-biztalk-2010-my-favorite-new-features-part-3-relevance-view/
http://biztalkmessages.vansplunteren.net/2010/05/07/mapping-in-biztalk-2010-my-favorite-new-features-part-4-suggestive-matching/
http://biztalkmessages.vansplunteren.net/2010/05/07/mapping-in-biztalk-2010-my-favorite-new-features-part-4-suggestive-matching/
http://biztalkmessages.vansplunteren.net/2010/05/07/mapping-in-biztalk-2010-my-favorite-new-features-part-5-optimized-display-of-links/
http://biztalkmessages.vansplunteren.net/2010/05/07/mapping-in-biztalk-2010-my-favorite-new-features-part-6-highlighting-selected-objects/
http://biztalkmessages.vansplunteren.net/2010/05/07/mapping-in-biztalk-2010-my-favorite-new-features-part-7-bringing-selected-objects-in-view/
http://biztalkmessages.vansplunteren.net/2010/05/07/mapping-in-biztalk-2010-my-favorite-new-features-part-8-%E2%80%93-search%C2%A0support/

BizTalk 2010: Musing of the ‘new’ SharePoint 2010 WS Adapter

by Mick Badran

As I was putting together a SharePoint / BizTalk lab using the new SharePoint adapter, I was happy to have found this great great post. Saved my lots of time, –Thanks Mick!

http://blogs.breezetraining.com.au/mickb/2010/10/12/BizTalk2010MusingOfTheNewSharePoint2010WSAdapter.aspx

ShareTalk Integration (SharePoint/BizTalk) series

by Kent Ware

Yet another series of really good posts if you plan to integrate with SharePoint. Despite being a compulsive gambler from Canada, Kent is a great guy who shares a lot of BizTalk experience through his blog. Rumors has it he is also writing on a book…And besides his interest in BizTalk he also shares his thoughts on Windows Phone 7 through his new WP7 blog.

http://kentweare.blogspot.com/2009/10/sharetalk-integration-sharepointbiztalk.html
http://kentweare.blogspot.com/2009/10/sharetalk-integration-sharepointbiztalk_11.html
http://kentweare.blogspot.com/2010/05/sharetalk-integration-wss-adapter-web.html
http://kentweare.blogspot.com/2009/10/sharetalk-integration-sharepointbiztalk_4024.html
http://kentweare.blogspot.com/2009/10/sharetalk-integration-sharepointbiztalk_27.html
http://kentweare.blogspot.com/2010/01/sharetalk-integration-sharepointbiztalk.html
http://kentweare.blogspot.com/2010/01/sharetalk-integration-sharepointbiztalk_16.html
http://kentweare.blogspot.com/2010/05/sharetalk-integration-wss-adapter-web.html

BizTalk Adapter Pack 2.0/SAP Adapter series

by Kent Ware

I’m not currently working with SAP even though know I’ll probably be in the future. By then I’m sure I’ll thank Kent again for taking the time to write these posts. 

http://kentweare.blogspot.com/2010/01/biztalk-adapter-pack-20-sap-adapter.html
http://kentweare.blogspot.com/2010/03/biztalk-adapter-pack-20-sap-adapter.html
http://kentweare.blogspot.com/2010/04/biztalk-adapter-pack-20-sap-adapter.html
http://kentweare.blogspot.com/2010/04/biztalk-adapter-pack-20-part-4-sending.html

Benchmark your BizTalk Server (Part 3)

by Ewan Fairweather

If you’re a true hard-core BizTalker and think performance is important, then this is a “must read” article. Take your time and read it a couple of times as it’s very detailed. Ewan have also been kind enough to share his script files for helping you identifying bottlenecks.

http://blogs.msdn.com/b/ewanf/archive/2010/01/12/benchmark-your-biztalk-server-part-3.aspx

Large Message Transfer with WCF-Adapters – Series

by Paolo Salvatori

I find it funny that Paolo sometimes makes an effort to split a topic into two parts. Every one of those posts could easily be split into five parts, and together make up for small book. Paolo does not write posts – he writes essays! 

Transferring large messages is a common challenge using BizTalk. This post cover the subject in detail, and shows how to effectively minimize the use of recourses, while still transferring large files through BizTalk. The content of these posts was also demonstrated by Ewan Fairweather during his talk at the Swedish BizTalk User Group.  
http://blogs.msdn.com/b/paolos/archive/2010/05/25/large-message-transfer-with-wcf-adapters-part-1.aspx
http://blogs.msdn.com/b/paolos/archive/2010/09/09/large-message-transfer-with-wcf-adapters-part-2.aspx

XmlDisassemble in a passthrough pipeline?

by Johan Hedberg

The discovery of BizTalk adding an Xml disassemble stage to a passthrough pipeline, was an interesting fact to say the least. Frightening might be a better choice of words.  Johan explain under which circumstances this happens. “Funny” enough, I’ve come across this issue twice this year.

http://blogical.se/blogs/johan/archive/2010/09/11/xmldisassemble-in-a-passthrough-pipeline.aspx

Modernizing BizTalk Server BAM with PowerPivot

by Jesus Rodriguez

I couldn’t agree more, – BAM is one of my favorite features of BizTalk. With the release of SQL Server 2008 R2 came PowerPivot for SharePoint and Excel, and even though I haven’t got around to test PowerPivot  yet,I really find this interesting, and I’m sure I’ll get back to the post later on.  
http://weblogs.asp.net/gsusx/archive/2010/09/28/modernizing-biztalk-server-bam-with-powerpivot.aspx

Less Virtual, More Machine - Windows 7 and the magic of Boot to VHD

by Scott Hanselman

With Windows 7 came the “boot to VHD” feature. I generally don’t want to install anything but the Office suite on my laptop. This is because I tend to try out a lot of CTP releases, along with the fact that I work with different customers where it’s a good practice to separate the environments. I solve that by always working in virtual environments. Using Windows Virtual PC is only supported in 32 bits, and even if I’d use VMWare or Virtual Box, I would not fully utilize the capacity of my laptop.

I’ve come across many high level demos showing the “boot to VHD” feature in action. But it’s not as easy as it seams. Every time I need to add a VM to my boot menu, I return to this post.

http://www.hanselman.com/blog/LessVirtualMoreMachineWindows7AndTheMagicOfBootToVHD.aspx

Nesting Scope shapes more than 19 levels

By Jan Eliasen

The knowledge of this limitation is not likely to come to any good use (for anyone (I hope)). But the effort of finding this flaw can not go unnoticed! – Thank you Jan. 
http://blog.eliasen.dk/2010/04/24/NestingScopeShapesMoreThan19Levels.aspx

We are in desperate need of developers! If you point me to anyone (yourself included), I’ll split the finder’s fee of 20.000 SEK with you, if we hire that person.

Who are “we”?

Enfo Zystems is a company with a long standing commitment to integration and service orientation. In fact – It’s all we do! We are currently expanding on the Microsoft platform with focus on BizTalk, AppFabric and the cloud offerings from Microsoft.

The commitment and focus of this company, lead Johan Hedberg and myself to join Zystems. We were, and still are, amazed by the dedication by everyone we’ve met. Everyone from dev’s to sales, knows and understand integration and service orientation. We’ve even had discussions about BAM with our CEO!   

What do we offer?

Right now we are looking to set up a delivery center, from which we’ll work as a unit, delivering solutions to projects, opposed to selling consultants per hour. This means you’ll be working on-site from our office in Kista, together with your colleagues, delivering solutions to many customers.

Don’t know BizTalk? –Not a problem, we’ll provide you with necessary education and training. We require you to either have a couple of years experience from .Net, or from working with other integration- or ESB platforms.

Let me know if you find anyone...

On the 29th of September we had yet another fantastic event at the Swedish BizTalk User Group. This time the agenda was the Future directions of Microsoft Application Platform, which we did together with the Swedish SQL user Group.

Unfortunately only one session was recorded and that was the “Management Tasks Made Simpler in Microsoft BizTalk Server 2010” with me and Paolo Salvatori. The good news is that is been published on Channel9:

http://channel9.msdn.com/Blogs/MSCOMSWE/Management-Tasks-Made-Simpler-in-Microsoft-BizTalk-Server-2010