August 2008 - Posts

Using Team Foundation Build for BizTalk projects (Part 2)
14 August 08 11:11 PM | wmmihaa | 7 comment(s)

In my last post I wrote about the requirements and challenges we encountered, implementing Team Foundation Server to create an automated build process. This post is meant to leave a more detailed view. 

Design Code structure

The main concern when it comes to design was to make deployment as easy as possible.  To enable this, we separated all BizTalk projects by types of artifacts. That left us with three different types of projects Orchestrations, Transformation and Schemas. No reason for Pipeline projects since we created generic pipelines to be configured at runtime. This way we got a chain of dependencies from Orchestrations -> Transformations -> Schemas. This would prove very important upon automating the undeploying of these assemblies.

We used the notion of BizTalk Applications to represent actual systems from which we'd receive and send messages to. The Application name would be used in the name of the BizTalk project, together with the information type. Eg. SAP.SalesRevenue.Orchestrations.proj.

Every BizTalk project where then grouped into a ILS solution, which in this case could be GeneralLedger.sln. To make sure each ILS solution could be built and deployed independent of other solutions, there could never be any references between these solutions.

Custom build tasks

Custom build tasks extends Team Foundation Build, and is necessary when you'd want to perform actions part from what is included in the product. There are lots of custom tasks, made available through the community, such as Microsoft.Sdc.Tasks. This library has lots of nice features for many of the products from MS, among them BizTalk.

Unfortunately, we couldn't quite get them to work with our requirements. But they where much helpful as a starting point.

We ended up creating four custom build tasks:

UndeployBindings

Undeploys ports and orchestrations from BizTalk applications using binding files. For this to work, the name of the binding file(s) are expected to end with target environment, Eg Sap.BindingFile.BUILD.xml.

<VIPN.Shared.Tools.BizTalkTfsBuildTasks.UndeployBindings
                TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
                BuildUri="$(BuildUri)"
                Server="$(BizTalkServerName)"
                RootPath="%(SolutionToBuild.Identity)"
                Database="$(BTServerDatabase)"
                ProcessTimeOutMilliseconds="60000"
                TargetEnviroment="DEV"/>
UndeployAssembly

The UndeployAssembly Task is only used internally from the UndeployBinding Task,  and is used for undeploying a single assembly from an BizTalk application. This is done after all ports and orchestrations are removed. The resources needs to be removed in the right order, assemblies, with references to others needs to be removed first. This is accomplished by encapsulating schemas, transformations and orchestrations in separate projects, and naming them thereafter. Eg. [Application].[InfomationType].Transfomations

ImportBindings

If the solution path includes a "Bindings" folder, all binding files will be imported into the application. The target environment will be set, if included in the name of the binding file. Eg Sap.BindingFile.TEST.xml.

Usages:

<VIPN.Shared.Tools.BizTalkTfsBuildTasks.ImportBindings
                TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
                BuildUri="$(BuildUri)"
                Server="$(BizTalkServerName)"
                Database="$(BTServerDatabase)"
                TargetEnviroment="$(TargetEnviroment)"
                RootPath="%(SolutionToBuild.Identity)"/>
ExportPackage

The ExportPackage exports one package per BizTalk Application, but only the assemblies stated in the binding file(  + all binding files for that application).

Usages:

<VIPN.Shared.Tools.BizTalkTfsBuildTasks.ExportApplication
                        TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
                        BuildUri="$(BuildUri)"
                        Server="$(BizTalkServerName)"
                        RootPath="%(SolutionToBuild.Identity)"
                        Database="$(BTServerDatabase)"
                        TargetEnviroment="$(TargetEnviroment)"
                        OutputPath="$(OutDir)"/>

Download the task library, and make sure it builds. You might have to reset some of the Team Foundation references. Copy all the assemblies to a directory on the build server.

Team Foundation Build Type

To make use of all this, create a new Build type, and open the TfsBuild.proj file.  Copy the lower part of this template ( from <!-- Blogical configuration --> and down). You need to change some of the configuration in the <!-- BizTalk Variables --> property group.

  • <Blogical_CustomTasks> should be set to the directory on the build server where you copied the task library.
  • <TargetEnviroment> should be set to the name of your target environment, and correlate to the binding file name to use.
  • <BizTalkServerName> is the name of the database server.

That's it.

Summary

I realize that this is a complex solution, and it may not fit within your design. But hopefully it will give you some inspiration of how to solve the problem.

Using Team Foundation Build for BizTalk projects (Part 1)
13 August 08 06:48 PM | wmmihaa | 28 comment(s)

I've been working in a fairly big BizTalk project since early this year, and wanted to share my experience working with Team Foundation together with BizTalk projects. The solution became more complex than originally anticipated. So hopefully you can learn from this, and maybe reuse some of the code and scripts I'll share in this series of articles.

But before we get into the details, let me fill you in on the requirements and challenges...

Easy and automated Deployment
  • We have five types of environments: Development (several), Build, Test, Acceptance and Production. We have about 30 BizTalk applications with a total of just about 300 ports on each environment.
  • We have normalized our BizTalk projects as much as possible, to reduce impact on redeployment, which left us with just over 200 Visual Studio projects. The amount of projects, together with its dependencies, would  make deployment quite challenging.
  • Each developer must be able to build and deploy their solutions, independently from other developers.
  • Due to the complexity of the environments, the deployment process has to be rigid or preferably automated.

Looking at Microsoft Team Foundation (from a far distance), it seems to be the solution to our problems. However, on a closer look there where many obstacles to overcome, before we'd get it to work.

Team Foundation Server

Team Foundation is a server product that integrates with products like MS Excel, MS Project and of course Visual Studio. This is accomplished through Team Explorer which adds functionality such as WorkItem Tracking and Source Control to these products.

Team Foundation also provide the functionality for Team Build, through which Build Managers can compile the projects, run associated unit tests, perform code analysis, release builds on a file server, and publish build reports.

 

The basic practice is this:

  1. Create a Visual Studio Solution with your projects, and check them in to the source control.
  2. Create a Build, and select your solution, along with the Build server to use.
  3. You may optionally choose triggers for when you want the Build to start. Useful for nightly builds etc.(only for Team Explorer 2008)
  4. Run the Build.

When running the Build, a Team Build Service on the selected Build server will get the latest code, label it, and build it. This is all done according the tfsbuild script that was created for you when you created the Build.

However...

Everything works fine unless, of course, you're trying to build BizTalk projects... As it turns out, Team Build is very similar to MS Build, and normal Visual Studio projects like C# projects, are just MS Build scripts. BizTalk projects, however, are not, and are therefor ignored by the build engine.

We didn't want to invest in Team Foundation 2005, but since we can only use Visual Studio 2005(VS 2008 does not support BizTalk projects yet), we had to use Team Explorer 2005, which gives less functionality. This left the Build Manager to have both Team Explorer 2005 and 2006.

Furthermore, there are no built-in support to deploy BizTalk projects.

Solution

To meet our requirements and utilize the functionality of Team Foundation, we had to start out by setting some design rules.

Since each developer must be able to build and deploy there solutions, independently from other developers, we had to group our projects in groups without dependencies to other groups. This was implemented by grouping the BizTalk projects in to ILS solutions. ILS is a term sometimes used in earlier BizTalk version and stands for Integration Service Layer. But in this case, it's better read as Isolated Service Layer, since there can be no references to projects in other ILS solutions.

Each ILS solution also included binding files for each BizTalk Application and environment.

A BizTalk project has by default two build configurations, Development and Deployment. We used these configurations to set the Target Environment (Development == (local), Deployment == Build server). Later on, when deploying the project through Team Build, we used the Deployment configuration. The actual compiling of each BizTalk project was done by DevEnv.bat rather than MSBuild. By using the /Deploy flag, we where able to build, deploy and create the BizTalk Application in one line of code.

"C:\Program Files\Microsoft Visual Studio 8\Common7\IDE\devenv" "MySolution.sln" /Deploy "Deployment"

We ended up creating a handful custom build tasks, which we could use in our build script to:

  • Undeploy ILS projects, ports and artifacts
  • Build and deploy all ILS projects
  • Import Binding files to Build environment
  • Generate MSI packages for deployment in other environments with only the deployed artifacts and binding files.

This all led us to be able to be able to, through Team Build, build all ILS projects, deploy them to the build environment, label all sources and open up for an easy deployment for the other environments.

In the next article I promise to get into more details, of the solution.

Jon Flanders will speak at BizTalk User Group Sweden
06 August 08 10:36 AM | wmmihaa | 2 comment(s)

Our user group has been fortunate enough this year to sign speakers like Darren Jefford and Dwight Goins. On the 4th of September we'll add Jon Flanders to that list. 

Jon is an MVP BizTalk, instructor for Pluralsight and is well known a cross the world as an author, speaker, instructor and for his commitment to the community. You might have seen him talking at events such as TechEd, about WF, WCF, BizTalk, BAM or any other related technologies from the Microsoft Connected System Division. If you have, you know that he is not only competent, but uses a great deal of irony and jokes to make his presentation even more interesting.

There will be two, one hour, sessions, and we've left it up to Jon to deside what the sessions should be about, and just set the topic to -"In the head of Jon Flanders"

So if you are in Sweden on the 4th of September, make sure to let us know you are comming.

For more information: www.biztalkusergroup.se

This Blog

News

    MVP - Microsoft Most Valuable Professional BizTalk User Group Sweden BizTalk blogdoc

    Follow me on Twitter Meet me at TechEd

    Visitors

    Locations of visitors to this page

    Disclaimer

    The opinions expressed herein are my own personal opinions and do not represent my employer's view in anyway.

Syndication