Compiling WIX projects via TFS Builds.

In order to get WIX projects compiling with the default setting for TFS builds a couple of things need to be changed.

 

One option is be to make the build target the x86 platform, but I quite often have issues with this. The other option is to alter the build definitions to force the Installer to build with the Any CPU target. This is achieved by changing the following:

 

1. In the configuration manager of Visual Studio. Ensure that the Installer project is selected to build for the Any CPU platform. By default this is unselected (as shown).

image 

 

2. Edit the Installer project definition so that it will always build under the Any CPU platform:

  • Remove the following line from the top of the definition:

    <Platform Condition=" '$(Platform)' == '' ">x86</Platform>

  • Change the following property groups associated with the different configurations and platforms removing the platform conditions. E.g. change:

<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|x86' ">
  <OutputPath>bin\$(Configuration)\</OutputPath>
  <IntermediateOutputPath>obj\$(Configuration)\</IntermediateOutputPath>
  <DefineConstants>Debug</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|x86' ">
  <OutputPath>bin\$(Configuration)\</OutputPath>
  <IntermediateOutputPath>obj\$(Configuration)\</IntermediateOutputPath>
</PropertyGroup>

to:

<PropertyGroup Condition=" '$(Configuration)' == 'Debug' ">
  <OutputPath>bin\$(Configuration)\</OutputPath>
  <IntermediateOutputPath>obj\$(Configuration)\</IntermediateOutputPath>
  <DefineConstants>Debug</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)' == 'Release' ">
  <OutputPath>bin\$(Configuration)\</OutputPath>
  <IntermediateOutputPath>obj\$(Configuration)\</IntermediateOutputPath>
</PropertyGroup>
<ItemGroup>

The installer should now build with a TFS build targeting the Any CPU platform, the MSI appearing in the build output.

Use System.Web.DynamicData.MetaModel with ADO.NET Entities Outside a Web Application

The Dynamic Data set of technologies Microsoft have introduced to speed web site development are build on top a set of libraries that provide meta data about a database model. Lots of this information such as maximum field lengths can be very useful within your own code. Below is an example of how to access this data in order to perform some processing on it.

 

I add a property to the partial class we have for the entity context:

/// <summary>
/// Gets the context meta model which can be used for finding out the model
/// meta data, such as length of fields.
/// </summary>
/// <value>The context meta model.</value>
internal static MetaModel ContextMetaModel
{
    get
    {
        // there doesn't appear to be a way to test if a model has been registered
        // without an exception being thrown. The web application normally registers the context
        // when it starts, we need to ensure its registered when we are operating outside
        // the web application
        MetaModel model;
        try
        {
            model = MetaModel.GetModel(typeof(MyContextContainer));
        }
        catch (InvalidOperationException)
        {
            model = new MetaModel();
            model.RegisterContext(typeof(MyContextContainer),
                new ContextConfiguration() { ScaffoldAllTables = false });
        }
        return model;
    }
}       

      

For example to find the maximum length of a name field:

int maxLength = MyContextContainer.ContextMetaModel.GetTable("Customer").GetColumn("Name").MaxLength

 

Although I haven't tested this, I assume the above works just as well with LINQ for SQL contexts.

Ordering EntityCollection sets in the ADO.NET Entity Framework

Following on from a recent post regarding XML serialisation of ADO.NET Entity Framework one issue I've come across is getting entities within the XML in the correct order.

The main issue is that the EntityCollection<IEntityWithRelationships> type used to maintain references within the entity framework is that its un-ordered. This makes perfect sense of course as entities in a database have no reliable order.

Even though this makes sense its very annoying if you just want your entities to come out in a defined order within an object structure and the any XML serialised from it. You could of course order the XML once it has been serialised, either via a transform, or in code, but having data contract references makes this pretty difficult.

I've come up with a pretty hacky mechanism that forces the order that objects are returned via the EntityCollection Enumerators. The solution uses reflection and won't be guaranteed to work in future versions of the framework. Essentially it finds the HashSet object used by EntityCollection to store its values and re-orders the data structure that the HashSet loops through in its enumerator.

Once a sort order has been applied to the entity collection the only code that runs against the collection must only work via the hashsets enumerator. I.e. use a context object solely to get all the objects for the re-order process and only ever read from the collections that have been re-ordered after that. Any attempt to add or remove items will cause the HashSet object to blow up as its underlying data structures have been messed with.

The code is:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data.Objects.DataClasses;
using System.Collections;
using System.Reflection;

namespace Examples

{
    public static class EntityUtil
    {
        /// <summary>
        /// Reorders the entities in an entity collection by the keySelector function.
        /// </summary>
        /// <remarks>
        /// This class uses reflection to re-write the slots in the underlying hashset used by the entity
        /// collection to contain the references.
        ///
        /// The only action that should be taken on the entity collection after this method has been called
        /// is via its enumerators, as the hash set is not left in a consistant state.
        /// </remarks>
        /// <typeparam name="TSource">The type of the entity objects.</typeparam>
        /// <typeparam name="TKey">The type of the key to order by.</typeparam>
        /// <param name="source">The entity collection source to re-order.</param>
        /// <param name="keySelector">The key selector function that returns the key to order by.</param>
        public static void ReorderEntities<TSource, TKey>(
            this EntityCollection<TSource> source, Func<TSource, TKey> keySelector)
            where TSource : class, IEntityWithRelationships
        {
            HashSet<TSource> relatedEntities = (HashSet<TSource>)
                typeof(EntityCollection<TSource>).GetProperty("RelatedEntities", BindingFlags.Instance | BindingFlags.NonPublic).GetValue(source, null);
            // re-order the sets slots to fix the ordering, (but mess up everything else probably)
            Type slotType = typeof(HashSet<TSource>).Assembly.GetType("System.Collections.Generic.HashSet`1+Slot").MakeGenericType(new Type[] { typeof(TSource) });

            FieldInfo slotField = typeof(HashSet<TSource>).GetField("m_slots", BindingFlags.Instance | BindingFlags.NonPublic);
            FieldInfo lastIndex = typeof(HashSet<TSource>).GetField("m_lastIndex", BindingFlags.Instance | BindingFlags.NonPublic);
            System.Array slots = Array.CreateInstance(slotType, relatedEntities.Count);
            int index = 0;
            foreach (TSource obj in new List<TSource>(relatedEntities).OrderBy(keySelector))
            {
                object test = Activator.CreateInstance(slotType);
                slotType.GetField("hashCode", BindingFlags.Instance | BindingFlags.NonPublic).SetValue(test, obj.GetHashCode());
                slotType.GetField("value", BindingFlags.Instance | BindingFlags.NonPublic).SetValue(test, obj);
                slotType.GetField("next", BindingFlags.Instance | BindingFlags.NonPublic).SetValue(test, -1);
                slots.SetValue(test, index++);
            }
            slotField.SetValue(relatedEntities, slots);
            lastIndex.SetValue(relatedEntities, slots.Length);
        }
    }

}

 

The reflection code itself is quite interesting as it find and constructs an inner type (Slot) of a generic type. The inner type is a struct so the Activator class is used as struct's don't have a constructor to invoke.

An example use of the code is:

Customer customer;
using (ContextContainer context = new ContextContainer())
{
    // cause a load of static data types first
    customer = context.Customers
        .Include("Addresses")
        .Include("Addresses.AddressLines")
        .First(e => e.CustomerId == customerId);
}

// reorder the structure in memory
customer.Addresses.ReorderEntities(e => e.Order);

foreach (Address a in customer.Addresses)
{
    a.AddressLines.ReorderEntities(e => e.Order);

}

 

The structure will now have the correct serialisation order. FYI an example xslt snippet that works correctly with ordered referenced entites is:

<xsl:for-each select="do:Addresses/do:Address">
  <xsl:apply-templates select="self::node()[@z:Id]"/>
  <xsl:for-each select="self::node()[@z:Ref]">
    <xsl:variable name ="ref" select="@z:Ref"/>
    <xsl:apply-templates  select="//do:Address[@z:Id = $ref]"/>
  </xsl:for-each>
</xsl:for-each>

Data Authorisation with ADO.NET Entity Framework

An application I'm writing has a requirement to only allow authorised users access to particular areas of data, generally referred to as 'Data Authorisation'. Generally when restricting access to data it's a good idea to create a framework that applies authorisation at the layer that the rest of the application uses, i.e. all data access is protected, stopping coding mistakes made at the higher layers exposing unauthorised data.

The application is a typical data entry web site using ADO.NET entities so I've decided to apply data authorisation to data in the ADO.NET Entity classes. Whilst trying to do this it turns out that Entities doesn't appear to allow a fully featured mechanism to prevent data access. For example, you cannot add the ability to restrict queries from returning un-authorised data.

The web site has many EntityDataSource's that generally load data via the entity set name plus a few where clauses. As we cannot generically alter the data returned from these queries I found that if you want to keep on using EntityDataSource's you will need to add joins to the tables containing the access control lists (ACL's) for an entity at the web tier. This isn't ideal but the application is mainly data entry orientated so having the web tier close to the data tier isn't too bad.

What I have found you can do though is to throw exceptions when un-authorised data has been loaded from the database. Although a developer has to know about the access control tables when writing queries, if they get things wrong exceptions will be thrown, highlighting any incorrect code. I.e. even though the web tier defines the join's to the access control tables, if it gets it wrong then the data tier throws exceptions. In addition pages that allow editing of a single entity that load the entity via a query string are protected as the underlying data layer will throw an exception if incorrect data is loaded via the manipulation of the query string.

The application then uses three main mechanisms for authorising data access:

  • All queries are written to join to the access control tables when they need to display lists filtered by ACL's.
  • All entity objects requiring data authorisation have methods added in partial files to check the ACL's when those entities are loaded from the database.
  • When the context saves changes checks are performed to ensure the user is authorised to perform those changes.

 

The code added to each entity to check access control lists is in general as below. It assumes that each entity has a single identifier, in this case called EntityTypeId (we name our identifies in relation to the type name, for example CustomerId, AddressId...). The code below is invoked whenever an entity is constructed from the database, it makes an assumption that the id is never set when creating new objects (our identifiers are auto generated).

/// <summary>
/// An entity identifier is changed when the object is materialised
/// from the database. Therefore check for permissions here.
/// </summary>
/// <param name="value">The id being changed</param>
partial void OnEntityTypeIdChanging(long value)
{
    if (this.EntityState == System.Data.EntityState.Detached)
    {
        DataAccessControl.AssertEntityTypeAccess(value);
    }
    else
    {
        // all other access potentially alters the acl's so re-cache
        DataAccessControl.AccessCache.ClearThreadAssociations();
    }
}

In my implementation the DataAccessControl class maintains a thread local cache of id's that can be accessed, this will be loaded on each web request and cleared down at the end of a request, or when a different user identity is found. Alternative mechanisms can be used though.

 

We can monitor when a context has it changes saved through the following code added to the partial class for the context:

partial void OnContextCreated()
{
    this.SavingChanges += new EventHandler(ContextContainer_SavingChanges);
}

void ContextContainer_SavingChanges(object sender, EventArgs e)
{
        List<ObjectStateEntry> checks =
            new List<ObjectStateEntry>(ObjectStateManager.GetObjectStateEntries(EntityState.Added));
        checks.AddRange(ObjectStateManager.GetObjectStateEntries(EntityState.Modified));

        PerformAccessChecks(checks);
}

The above code just gathers up all the changes and ensures that the user has access to objects being added or modified (deleted could be added to if required).

 

In general a generic data access authorisation mechanism can be added to ADO.NET Entities by performing the following:

  • Perform an access check when the entities primary key is changed, when the entity is detached via the partial OnChanging methods.
  • Perform an access check to all entities when being saved by listing to the SavingChanges event on the context.
Asp.Net HttpSessionState in Microsoft Unit Tests Framework

We have some unit tests that tests code which relies on a variable stored in the Asp.Net session state (HttpSessionState) accessed from the HttpContext.Current.Session property.

Our unit tests run within the Microsoft Unit Test framework that comes with Visual Studio, but I think that the technique should apply to other frameworks such as NUnit.

Without doing any work HttpContext.Current returns null. It appears quite easy to make set a current HttpContext to contain a session:

HttpContext.Current = new HttpContext(new HttpRequest("", "http://localhost/", ""), new HttpResponse(new System.IO.StringWriter()));

Now HttpContext.Current returns a context, but HttpContext.Current.Session still returns null. You can attach a basic session object with the following line:

System.Web.SessionState.SessionStateUtility.AddHttpSessionStateToContext(
 HttpContext.Current, new HttpSessionStateContainer("",
  new SessionStateItemCollection(),
  new HttpStaticObjectsCollection(), 20000, true,
  HttpCookieMode.UseCookies, SessionStateMode.Off, false));

Now HttpContext.Current.Session can be assigned to and read from allowing my unit tests to run.

I think the code is best put in the a method marked with a [TestInitialize] attribute. The context should also be cleared when a test has completed:

[TestCleanup]
public void Clean()
{
 HttpContext.Current = null;
}

I'm not expecting the HttpContext to work well for any other calls, but it might allow more than just interaction with the session.

Ado.Net Entity Framework with Many to Many Relationships and Eager Load

I've switched a relationship in a model from one to many to many to many and its effecting how the ObjectQuery.Include method is eager loading objects from the database.

An example Module before adding a many to many looks like:

Company -(1..*)-> Cars -(1..*)-> Passengers
Company -(1..*)-> Passengers

A simple query such as:

company = context.Company.Include("Cars").Include("Passengers").First(e => e.CompanyId = companyId);

both populates all the passengers under the company, but also all the passengers under the cars. I.e. company.Cars[0].Passengers is populated.

But when the model is changed to:
  Company -(1..*)-> Cars -(*..*)-> Passengers
  Company -(1..*)-> Passengers

performing the same query doesn't populate the passengers under the cars.

Adding an additional Include("Cars.Passengers") corrects this.

I initially thought this was a bug, but it makes complete sense when you realise that before the entity framework had enough information to construct the graph. It will query all cars and passengers associated with the company, and each passenger row/object references the car it belonged to, therefor it can assign passengers to cars.

With a many to many relationship added there is an additional link table that will not query unless told to. Without the relationship between cars and passengers the framework cannot reconstruct the graph.

Debugging Dot Net Release Configuration Applications

This might be obvious to many people but I've found its not entirely easy to debug a dot net framework executable (i.e. not asp.net) that has been build in its Release configuration. Attaching a debugger to an already running exe build in release mode will not work well as the exe has been optimised.

VS 2005 onwards generated pdb files by default in Release configuration (it's also possible to create them in VS 2003 also) so it is possible to debug these executables, but only if they have been started by a debugger which stops optimisation happening.

I've found the easiest way to do this is to first attach Visual Studio as a debugger to the running exe that you want to debug. Either do this when the application crashes and provides a debug dbox, or whilst it is running eg:

debug exception window

Then choose a new instance of visual studio:

attach vs 2008

Visual studio starts and creates a solution for the executable. At this point the debug session isn't much good as the code is optimised.

The trick is just to stop the debug session and restart it again in visual studio (e.g. pressing ctrl-shift-F5 should achieve this). The application is now started in such as way that optimisation is disabled, VS will load the symbols and allow you to attach the source file when you pause the application or when it encounters an exception.

In addition the solution VS generates can be saved and reloaded so a debug session can be started directly. Much easier than using cordbg.exe.

Asp.Net request thread associated ADO.NET Entity Framework Context

A current project I'm working on is using ADO.NET Entities to enable a web site to collect form based information. One of the first things I've tried to do is associated the ADO.NET Entities ObjectContext with a thread static associated with the current page request.

So far this approach appears to be working well. We define a context factory as follows, all access to the context is now via the factory which creates it on demand and associates it with the current thread.

 public static class ContextFactory
 {
     [ThreadStatic]
     private static ModelContextContainer context;
     private static object staticLock = new object();

     public static ModelContextContainer Context
     {
         get
         {
             // Warning. Adding a watch to the context may change the behaviour
             lock (staticLock)
             {
                 if (context == null)
                     context = new ModelContextContainer();
                 return context;
             }
         }
     }

     public static void Clear()
     {
         lock (staticLock)
         {
             if (context != null)
                 context.Dispose();
             context = null;
         }
     }
 }

In addition a HttpModule is used to dispose the current context when the page request cycle has completed:

Which is configured in the Web.config:

<system.web>
  <httpModules>
    <add name="ContextDispose" type="VisuLive.Studio.Web.ContextHttpModule"/>
  </httpModules>
</system.web>

So far we haven't had any issues following this mechanism, and it allows us to stop creating and disposing context's if more than separate operation on the database is required per page request. 

public class ContextHttpModule : IHttpModule
{
   void IHttpModule.Dispose()
   {
   }

   void IHttpModule.Init(HttpApplication context)
   {
      context.EndRequest += new EventHandler(context_EndRequest);
   }

   void context_EndRequest(object sender, EventArgs e)
   {
      ContextFactory.Clear();
   }
}

XML Serialize ADO.NET Entity Framework Generated Data Object for XSLT

As far as I can tell the MSDN docs don't explicitly show how to generate XML from an entity framework data object, although its very simple.

It appears that as long as you use the DataContractSerializer associated with WCF it all works fine. The example code below shows can be added to any data object partial code.

public XmlDocument SerialiseAsXml()
{
   XmlDocument doc = new XmlDocument();
   using (XmlNodeWriter writer = new XmlNodeWriter(doc, false))
   {
      new DataContractSerializer(this.GetType()).WriteObject(writer, this);
   }
   return doc;
}

Note that XmlNodeWriter is an example bit of code from gotdotnet.com, but it could be replaced with other kinding of writers.

All the data objects are marked with the data contract attribute as follows:

[DataContract(IsReference=true)]

 

The IsReference is quite important in that it means the XML generated works with object graphs with cycles (e.g. a customer has an address property, and the address has a customer property).

More information about is in the MSDN docs, but essentially the first time a data object is serialized an Id attribute is added. The next time the object is serialized an empty node is created with a Ref attribute with the same value as the Id attribute for the XML node containing the objects XML.

One real issue with this mechanism is found when it comes to performing an XSLT transform on the generated XML. You can't totally guarantee where in the XML tree structure the full data objects information can be found as it is dependent on where an object first appears in the object tree. I've come up with a mechanism in the XSLT to perform a lookup when the ref attribute is found which should work in all cases:

  <xsl:template match="do:Customer">
    Customer '<xsl:value-of select="do:Name"/>'

    <!-- decend into the dashboard collection either directly or through a lookuk on the ref -->
    <xsl:for-each select="do:Addresses/do:Address[@z:Id]">
      <xsl:apply-templates select="."></xsl:apply-templates>
    </xsl:for-each>
    <xsl:for-each select="do:Addresses/do:Address[@z:Ref]">
      <xsl:variable name ="ref" select="@z:Ref"/>
      <xsl:apply-templates select="//do:Address[@z:Id = $ref]"></xsl:apply-templates>
    </xsl:for-each>
  </xsl:template>

WCF and Custom Behaviors

I had quite an issue getting a custom behavior registered within the system.serviceModel configuration section of the App.config

I was attempting to add an endpoint behaviour using a class that extended BehaviorExtensionElement eg:

public sealed class MCWcfSessionBehaviourElement : BehaviorExtensionElement

{

public override Type BehaviorType

{

get

{

return typeof(MCWcfSessionBehaviour);

}

}

 

protected override object CreateBehavior()

{

return new MCWcfSessionBehaviour();

}

}

Using the configuration:

 

<behaviors>

<endpointBehaviors>

<behavior name="MetaCoreServiceBehavior">

<MCWcfSessionBehaviour />

</behavior>

</endpointBehaviors>

</behaviors>

 

<extensions>

<behaviorExtensions>

<add name="MCWcfSessionBehaviour"

type="com.iMeta.iBureau.InternalContract.MCWcfSessionBehaviourElement"/>

</behaviorExtensions>

</extensions>

 

Once the configuration is altered everything starts correctly. The answer came from a closed bug which has no information regarding if the problem has been fixed, found at http://connect.microsoft.com/wcf/feedback/ViewFeedback.aspx?FeedbackID=216431

The issue is that the type didn't have the full type name with assembly information, WCF could lookup the MCWcfSessionBehaviourElement type and call the property BehaviorType but then threw an exception when parsing the MCWcfSessionBehaviour element when adding the behaviour. It turns out that when adding behaviour extentions you need to specify the exact type name, i.e. 

com.iMeta.iBureau.InternalContract.MCWcfSessionBehaviourElement, com.iMeta.iBureau.InternalContract, Version=1.0.0.0, Culture=neutral, PublicKeyToken=068e3941efd78a6f