Tatham Oddie

Quite a while back I posted an entry about rejoining the pack, and it’s been pretty common since for me to face the question “how’s it going” and “why did you move”. Well, this post is a well overdue response about the decisions I’m following now. This particular post focuses on my move from ActiveRecord, however I’m planning to write similar posts for the other areas discussed in my original post.

I hate datasets. Adam Cogan once said in a presentation, “if you like fat women, you’ll like datasets”. Even that statement doesn’t go far enough to describe how much I don’t like datasets. But that’s another story… For this post, lets just assume that you’re sold on the concept of domain objects.

ActiveRecord gave me a quick and relatively simple way of filling my application with what I then thought were nice domain objects. I didn’t need to write any SQL. I could place validation on a property using a single attribute like [ValidateNotEmpty] or even write custom ones like [ValidateAustralianBusinessNumber]. Life was good.

In then end, I built two major sites using ActiveRecord, and they are still using their ActiveRecord implementations today. Via Virtual Earth serves 5,000 visitors an average day and 2,000,000 on a big day (conferences or product launches). Male Order Only is a fully fledge e-commerce site that’s starting to get some heavy press in international fashion circles.

Unfortunately, ActiveRecord then started to show it’s faults. One of my biggest motivations to move was code flexibility – with AR I almost feel backed into a corner when it comes time to do maintenance on these sites. The other major motivation was performance - ActiveRecord sits on top of NHibernate which sits on top of ADO.NET, and there’s a ton of reflection mixed in along the way. With a framework that deep, performance is always going to be an up-hill battle and currently it’s a battle that’s hurting us.

To be fair, I’m not an AR guru and some of these issues are probably completely my fault. On the same note however, there are a number of use cases I have since come to discover that AR just doesn’t handle nicely.

Where to from here? Much of where I’ve ended up today in the realm of domain models is derived from Paul Stovell’s Trial Balance project. I think he’s finally come to the realization that he will never finish the product, but in the seven or so re-writes so far he’s had the opportunity to try a lot of different techniques. I like where he’s ended up, and this is the architecture I’ve started implementing for my latest round of projects. Considering Hammett lives in Brazil, and Paul lives 5m away in the other room, Paul probably had a considerably easier job of convincing me, but consider me convinced.

Lets take a look at some code…

The whole domain model (entities and data providers) sits within FuelAdvance.<ProjectName>.DomainModel.

Each of my entities have their own class that inherits from DomainObject (I’ve trimmed a number of the properties to simplify the sample):

namespace FuelAdvance.<ProjectName>.DomainModel.Entities
{
    public class Voucher : DomainObject
    {
        private Guid _voucherId;
        private string _toName;
        private string _fromName;

        internal Voucher(IDataProvider dataProvider)
            : base(dataProvider)
        {
            _voucherId = Guid.NewGuid();
        }

        [DataProperty("Voucher Id")]
        public Guid VoucherId
        {
            get { return _voucherId; }
            set
            {
                AssertCanEdit();

                if (_voucherId != value)
                {
                    _voucherId = value;
                    NotifyChanged("VoucherId");
                }
            }
        }

        [DataProperty("To Name")]
        public string ToName
        {
            get { return _toName; }
            set
            {
                AssertCanEdit();

                if (_toName != value)
                {
                    _toName = value;
                    NotifyChanged("ToName");
                }
            }
        }

        [DataProperty("From Name")]
        public string FromName
        {
            get { return _fromName; }
            set
            {
                AssertCanEdit();

                if (_fromName != value)
                {
                    _fromName = value;
                    NotifyChanged("FromName");
                }
            }
        }   

        public override void CopyTo(object targetObject)
        {
            base.CopyTo(targetObject);

            Voucher targetVoucher = targetObject as Voucher;
            if (targetVoucher != null)
            {
                targetVoucher.VoucherId = this.VoucherId;
                targetVoucher.ToName = this.ToName;
                targetVoucher.FromName = this.FromName;
            }
        }
    }
}

There are a few things to note here:

• The entity has an internal constructor, and it keeps a reference to the data provider that created it.
• The propeties seem a bit wordy, but if you take a close look you’ll actually see they’re doing quite a bit such as asserting whether the object is read-only, and providing INotifyPropertyChanged notifications. This is helped with a simple snippet:

• Each property is decorated with a custom DataProperty attribute. This is entirely optional – but I’ll explain it more later.

The entity inherits from an abstract DomainObject class within the same project. All of the underlying logic actually sits in the very simple FuelAdvance.Components.Modeling.DomainObject<T>, however we need this interim class to keep some of the properties internal to our domain model.

namespace FuelAdvance.<ProjectName>.DomainModel
{
    public abstract class DomainObject : DomainObject<IDataProvider>
    {
        internal DomainObject(IDataProvider dataProvider)
            : base(dataProvider)
        {
        }

        internal new IDataProvider DataProvider
        {
            get { return base.DataProvider; }
        }

        public new bool IsReadOnly
        {
            get { return base.IsReadOnly; }
            internal set { base.IsReadOnly = value; }
        }
    }
}

All pretty simple so far right? Now, lets take a look at how we interact with these objects as a consumer from say FuelAdvance.<ProjectName>.WebUI.

Retrieving an object is pretty straight forward – we just call the data provider:

IDataProvider dataProvider = new SqlDataProvider(connectionString);
Voucher voucher = dataProvider.GetVoucher(voucherId);

By default though, our object is read only so this would throw an exception (remember that AssetCanEdit earlier?):

voucher.ToName = "Tatham Oddie";

This is where the ChangeCoordinator steps in. The change coordinator maintains a link to the original item, as well as a working copy where we perform our edits. It is the data provider’s responsibilty to provide us with the change coordinators. An edit might look like this:

ChangeCoordinator<Voucher, IDataProvider> changeCoordinator = dataProvider.AcquireChangeCoordinator(voucher);
changeCoordinator.WorkingCopy.ToName = "Tatham Oddie";
changeCoordinator.PushChanges();

The PushChanges() method is responsible for pushing the changes from the working copy to the original item, and then firing a ChangesPushed event. When the data provider made our change coordinator, it subscribed to this event so that it could persist the changes at this time.

The actual process of pushing the changes from the working copy to the original item is achieved using the CopyTo() method on our entity.

PushChanges() can only be called once – after this time the working copy is read only, and any subsequent calls to PushChanges() will fire an exception.

There are lots of advantges to this approach:

• In a winforms secnario we might have a list of records. When we double click a record in the grid, a form is opened to edit the individual record. If the user makes a change and clicks Ok, we expect this change to be visible in the original listing, but no earlier than this. There are a number of approaches here:
  1. We pass the voucher ID to the detail form. The detail form loads it’s own instance of the object. When the user clicks Ok, the detail form persists the changes then notifies the original form there has been a change. The original form reloads its list. There’s lots of reloading going on here…
  2. We pass an instance of the voucher to the details form. The detail form uses data binding. When a field is changed in the detail form, that change is instantly visible in the list (not good as they haven’t clicked ok yet). To support the cancel button, we keep a copy of the original values somewhere.
  3. We pass an instance of the voucher to the details form. We manually populate the fields from the instance (no databinding). When the user clicks Ok we manually copy the values back.
  4. We use a change coordinator. The details form binds to changeCoordinator.WorkingCopy. When the user clicks the Ok button, we call changeCoordinator.PushChanges(). When the original object gets updated the INotifyPropertyChanged interface means the original list gets immediately updated without having to re-poll the database. The if user clicked Cancel, we jsut abandon the change coordinator.
• As part of the PushChanges event we actually receive a List<Change> where each change includes the property name (eg. “FromName”), a friendly name retrieved from the DataProperty attribute (eg. “From Name”), the original value (eg. “Tom Harvey”) and the new value (eg. “Tatham Oddie”). Only properties that actually changed are included in this list. We can use this list for a number of things:
  1. Implementing auditing. (I have some really cool stuff in this space that I will cover in a later post)
  2. Complex security policies, for example: “The ‘from name’ on a voucher can only change if the voucher has not yet been claimed or expired.”
  3. Complex validation, for example: “If the expiry date is changed, it must be equal to or greater than it’s original value.”
• We can persist changes without having to go back to the original data provider and call something like Save(voucher), and we don’t need need to make the entity aware of how to persist itself either.

We use this same approach to create new instances as well:

ChangeCoordinator<Voucher, IDataProvider> changeCoordinator = dataProvider.CreateVoucher();
changeCoordinator.WorkingCopy.FromName = "Tatham Oddie";
changeCoordinator.WorkingCopy.ToName = "Tom Harvey";
changeCoordinator.PushChanges();

In this case, changeCoordinator.OriginalItem is null. When the data provider receives the ChangesPushed event it can easily detect this case and act accordingly.

Alright, so lets take a look at what’s under the covers of one of these data providers then!

To start off with, we define an interface for our providers:

namespace FuelAdvance.<ProjectName>.DomainModel
{
    public interface IDataProvider
    {
        Voucher GetVoucher(Guid voucherId);
        ChangeCoordinator<Voucher, IDataProvider> CreateVoucher();
        ChangeCoordinator<Voucher, IDataProvider> AcquireChangeCoordinator(Voucher voucher);
    }
}

When defining your data provider interface, it’s crucial that you only create the methods you need. It’s very easy to say “well we need a GetAll, a Get, a Save, a Create, a …”. No you don’t! Start with nothing, and add the methods as you need them. You’ll also notice at this point that our data provider interface doesn’t have any save methods. This is because the concrete implementation is responsible for providing the change coordinators, and thus provides it’s own persistence logic at this point.

For our example, the SqlDataProvider implementation might look like this:

namespace FuelAdvance.<ProjectName>.DomainModel
{
    public class SqlDataProvider : IDataProvider
    {
        private StoredProcedureHelper Helper;

        public SqlDataProvider(string connectionString)
        {
            this.Helper = new StoredProcedureHelper(connectionString);
        }

        public ChangeCoordinator<Voucher, IDataProvider> CreateVoucher()
        {
            return CreateChangeCoordinator<Voucher>(null, new Voucher(this));
        }

        private void CreateVoucher(Voucher voucher)
        {
            Helper.ExecuteNonQuery(
                 Helper.CreateCommand("[dbo].[CreateVoucher]",
                      Helper.CreateParameter("@VoucherId", voucher.VoucherId),
                      Helper.CreateParameter("@FromName", voucher.FromName),
                      Helper.CreateParameter("@ToName", voucher.ToName)));
        }

        private void SaveVoucher(Voucher voucher)
        {
            Helper.ExecuteNonQuery(
                 Helper.CreateCommand("[dbo].[SaveVoucher]",
                      Helper.CreateParameter("@VoucherId", voucher.VoucherId),
                      Helper.CreateParameter("@FromName", voucher.FromName),
                      Helper.CreateParameter("@ToName", voucher.ToName)));
        }

        public Voucher GetVoucher(Guid voucherId)
        {
            return Helper.ExecuteSingleSelectCommand<Voucher>(
                 Helper.CreateCommand("[dbo].[GetVoucher]",
                      Helper.CreateParameter("@VoucherId", voucherId)),
                      delegate()
                      {
                          Voucher voucher = new Voucher(this);
                          voucher.IsReadOnly = false;
                          return voucher;
                      },
                      GetDefaultSealObjectInstanceDelegate<Voucher>());
        }

        #region Change Coordinators

        public ChangeCoordinator<Voucher, IDataProvider> AcquireChangeCoordinator(Voucher voucher)
        {
            return CreateChangeCoordinator<Voucher>(voucher, new Voucher(this));
        }

        protected ChangeCoordinator<T, IDataProvider> CreateChangeCoordinator<T>(T originalItem, T workingCopy) where T : DomainObject
        {
            if (originalItem != null && originalItem.DataProvider != this) throw ExceptionHelpers.NewInconsistentDataProviderException();
            if (workingCopy == null) throw ExceptionHelpers.NewArgumentNullException("workingCopy");

            ChangeCoordinator<T, IDataProvider> coordinator = new ChangeCoordinator<T, IDataProvider>(originalItem, workingCopy);

            coordinator.ChangesPushing += new ChangeCoordinatorChangesPushingEventHandler<T, IDataProvider>(ChangeCoordinator_ChangesPushing<T>);
            coordinator.ChangesPushed += new ChangeCoordinatorChangesPushedEventHandler<T, IDataProvider>(ChangeCoordinator_ChangesPushed<T>);

            return coordinator;
        }

        private void ChangeCoordinator_ChangesPushing<T>(object sender, ChangeCoordinatorChangesPushingEventArgs<T, IDataProvider> e) where T : DomainObject
        {
        }

        private void ChangeCoordinator_ChangesPushed<T>(object sender, ChangeCoordinatorChangesPushedEventArgs<T, IDataProvider> e) where T : DomainObject
        {
            if (typeof(T) == typeof(Voucher))
            {
                if (e.ChangeCoordinator.OriginalItem == null) CreateVoucher(e.ChangeCoordinator.WorkingCopy as Voucher);
                else SaveVoucher(e.ChangeCoordinator.WorkingCopy as Voucher);
            }
            else throw ExceptionHelpers.NewNotSupportedException();
        }

        #endregion

        #region Stored Procedure Helpers

        private StoredProcedureHelper.SealObjectInstanceDelegate<T> GetDefaultSealObjectInstanceDelegate<T>() where T : DomainObject
        {
            StoredProcedureHelper.SealObjectInstanceDelegate<T> result = delegate(T target)
            {
                target.IsReadOnly = true;
            };

            return result;
        }

        #endregion
    }
}

A few things to note:

• Most of the code there consists of helper methods useds to create our change coordinators
• We’re using stored procedures to load everything
  • Much more effecient than the dynamic SQL that automated ORM systems like to use
  • Lets a DBA go an restructure as much as they want without affecting the program (while most developers won’t admit it, they really do suck at SQL and thus should be leaving the opportunity there for a DBA to clean it up later)
• The CreateVoucher(Voucher) and SaveVoucher(Voucher) methods are both private to the data provider implementation – our consumers don’t need to use these
• In this case we had separate methods for creating and saving vouchers, however we could have very easily used the same method – this is a case that many systems don’t support very well. It doesn’t always have to be CRUD you know …
• The SQL code is very succinct thanks to the StoredProcedureHelper class.

The more I think about it, I just don’t see the value that a mammoth framework like Active Record or NHibernate offers. Sure, I have an underlying component library (FuelAdvance.Components.Modeling) but that’s a total of 8 classes with maybe 500 lines of code. The power is in the architecture, not the underlying codebase.

I’d like to make some more posts about this and cover things like relationships and security. Let me know which areas interest you most.

You can download the latest version of the code related to this post from our SVN repository here:

• FuelAdvance.Components.Data/
• FuelAdvance.Components.Modeling/

If you find a bug, please email me, or better yet, email me the patch.

Usually whenever I write something semi-cool at work, I get home and tell my flatmate “Hey – check out this semi-cool thing I did today”. He replies “Yeah – that’s cool – you should blog it” to which I respond “Hmmm … maybe later – I’m hungry”. Well, here’s the first post from what is hopefully a series of semi-cool things Tatham did at work one day.

Many of these posts will cover very simple concepts, but they are concepts or code snippets that often get skipped over and thus just aren’t that well documented around the web.

One of the things that occassionally bugs me is that I can’t use a string as a base type for an enumeration. I’m sure there’s a good underlying reason, but when working with existing systems it’d be nice to write something like this:

using FuelAdvance.Components.Common;

namespace FuelAdvance.Components.Payments.Gateways.PayPal
{
    public enum PaymentStatus : string
    {
        CanceledReversal = "Canceled-Reversal",
        Completed = "Completed",
        Denied = "Denied",
        Expired = "Expired",
        Failed = "Failed",
        InProgress = "In-Progress",
        PartiallyRefunded = "Partially-Refunded",
        Pending = "Pending",
        Processed = "Processed",
        Refunded = "Refunded",
        Reversed = "Reversed",
        Voided = "Voided",
        Unknown = "Unknown"
    }
}

Well, .NET doesn’t work that way so behold FuelAdvance.Components.Common.EnumerationHelpers.

Now, we can write our enumeration like this:

using FuelAdvance.Components.Common;

namespace FuelAdvance.Components.Payments.Gateways.PayPal
{
	public enum PaymentStatus
	{
		[Value("Canceled-Reversal")] CanceledReversal,
		Completed,
		Denied,
		Expired,
		Failed,
		[Value("In-Progress")] InProgress,
		[Value("Partially-Refunded")] PartiallyRefunded,
		Pending,
		Processed,
		Refunded,
		Reversed,
		Voided,
		Unknown
	}
}

Really, the code is nicer as we only need to include strings for the three values which include dashes. The Value attribute is about a basic as attributes get, and it doesn’t need to be any more complex.

The EnumerationHelpers class provides the logic for mapping string values to enumeration values and back again:

PaymentStatus status = EnumerationHelpers.GetEnumValue<PaymentStaus>("In-Progress")

In writing the EnumerationHelpers class, I discovered a rather annoying limitation of generic constraints – you can’t constrain to an enum like so:

public static T GetEnumValue<T>(string value) where T : enum

{

    ...

}

Instead, you can only narrow the field as far as a struct, then test the type manually:

public static T GetEnumValue<T>(string value) where T : struct
{
    if (!typeof(T).IsEnum) throw ExceptionHelpers.NewArgumentNotAnEnumException(typeof(T).FullName);

    ...

}

UPDATE: I received an email this morning from Sonny Malhi saying “I’m not sure of the complete intent of the post ‘.NET Enumerations: Generic Constraints and Custom Values’ but if it is to return a description based on a title I believe you should use the [description] attribute if your goign to use a enum type. This way you get a name, value and description from your enum type.“

The point of the [Value] attribute is not for UI display – you would still use the builtin [Description] attribute for that. The usefulness of the [Value] attribute is when you’re trying to integrate with an existing API or data source. You’ll notice that the example above is a PaymentStatus enumeration from my PayPal components namespace. PayPal return values from their API such as “In-Progress” which aren’t valid as enumeration item names, and I wanted to avoid writing a ParsePaymentStatus(string) method which just contained a massive switch statement.

You can download the latest version of all the code related to this post from our SVN repository here:

• FuelAdvance.Components.Common.EnumerationHelpers
• FuelAdvance.Components.Common.ValueAttribute
• FuelAdvance.Components.Common.ExceptionHelpers
• FuelAdvance.Components.Common.Tests.EnumerationHelperTests

If you find a bug, please email me, or better yet, email me the patch.

When writing my Local proxies with IE7 – Solved post, Windows Live Writer helpfully turned http://localhost:xyz/ into a link for me. Unfortunately, this resulted in my WordPress page throwing JS errors on every load.

It turns out that the invalid port number breaks the IE7 DOM, and the WordPress devs (who replied within about 4 hours!) haven’t been able to find a way around the condition.

I have posted a test case here:

http://tatham.oddie.com.au/files/InvalidPortNumberInHrefAttributeBreaksDom/

Annoylingly the Microsoft Connect site for IE7 is closed. I’ve emailed a link to the test case to the IE team via their blog, but if anybody knows of a better way I’d be keen to know.

UPDATE 25/4/07: Links like http://anonymous:@svn.fueladvance2.com/ seem to break it as well as demonstrated if you view my next post in IE7.

Here’s some food for thought for anybody who develops server software… I like to know what’s going on without having to jump through hoops.

This post was provoked after spending my weekend on a Server 2003 / Active Directory / Exchange deployment. As part of the system, we are using Ed Forgacs’ wonderful Exchange Connector. It sits in the “it-just-works” category, but I have no idea what the hell it’s doing. If I want to see a log file, I need to give it a SQL database and then I need to go and query the SQL database myself. For 99.999% of the time this is fine, but sometimes I just want to quickly check up on what’s happening.

On one of the servers I manage, we host email for 50 or so of our client’s domains with hMailServer. It’s a great little mail program that just works. (In a hosting environment like this, Exchange is too tightly coupled with Active Directory to be useful.)

One of the nicest features to the whole program for me is the logging. Sure, I can save a log to a file … like so:

BUT – I can also go to the “Logging” tab quickly and just click “Start”. This is immensely useful for quick little debugging tests.

This has been bugging me …

IE7 always bypasses the proxy when connecting to localhost, irrelevant of what options you choose. This makes life painful if you’re trying to use some like Fiddler when developing a site locally.

If you’re hosting locally with IIS, you can just connect using http://machinename/ instead of http://localhost/.

Unfortunately we don’t have this flexibility with Casini (the one included with VS 2005) as it only accepts connections to http://localhost:xyz/.

Today, I came across the solution thanks to Dominick Baier!

Use http://localhost.:xyz/ (notice the added period between the hostname and the colon).

• IE7 is no longer sure if it’s local or not, so we get the proxy,
• it still gets routed correctly, and
• Casini still accepts the connection.

In the real world of HTML, you use a separate <form> tag for each form on your page. If I have a currency converter in my travel website’s sidebar, it has its own form tag. As a result, when I press “enter” in a text field, the right submit button is pushed and only that form is submitted.

In ASP.NET we just wrap the whole page in one massive hack of a <form>-esque tag.

<rant>That bit always annoys the hell out of me – but I guess that what’s you get when you let architects try and replicate a windows development model onto the web.</rant>

The most annoying thing however has always been how to map the “enter” effect to the right button. Well, in ASP.NET 2.0 a feature shipped that I never noticed – and it’s pretty cool.

    <asp:Panel DefaultButton="MyButton" runat="server">
        ... my form here ...
    </asp:Panel>

It still submits everything to the server, which is a bit wrong, but at least you get the right UI model.

It omits horendous output:

    <div onkeypress="javascript:return WebForm_FireDefaultButton(event, 'ctl00_ctl00_ContentPlaceHolder_ctl10_ConvertButton')">

…but I guess with a megahack like <form runat="server"> there’s not much you can do besides hack it further.

My excitement about this feature, mixed with the disappointment of <form runat="server"> has left me in the same mood I started in.

Even more file types now supported!

Read more about the Vista and Office 2007 Code Preview Handler Pack

Download the Vista and Office 2007 Code Preview Handler Pack

This is what CSS files look like:

And this is what patch files look like:

This build also includes some preliminary support for batch files, however I’m not yet happy with the colors and some things aren’t being colorized (like the labels):

I’m pleased to announce that Ivan Porto Carrero and I have managed to add support for Ruby and JScript files to the Preview Handler Pack.

The updated pack is available for download here: PreviewHandlerPackSetup.msi

Ivan added support for Ruby (.rb, .rhtml, and .rjs) files:

And I addedsupport for JScript (.js) files:

It’s a pretty common requirement these days to send a nicely formatted email, yet I’ve come across very few “nice” solutions.

The solution I’ve come to use is simple, and documented across the web, but rarely used in the real world.

The most common code I see (and what I used to write back in the days of .NET1.0 powered dinosaurs roaming the earth) revolves around ugly use of the StringBuilder:

MailBody.Append("<html><head></head><body> \n");
MailBody.Append("<span style=\"font-size: 11 px; font-family:Verdana,Helvetica, sans-serif\">Hi,<br><br>Your account information as it has been retrieved from the database:</span><br><br> \n");
MailBody.Append("<div style=\"font-size: 11 px; font-family:Verdana,Helvetica, sans-serif\">Username: " + UserData.Username + "<br><br>" + "Password: " + UserData.Password + "</div> \n");
MailBody.Append("</body></html>");

Ugly. Unmaintainable.

Instead, create a user control for your email template using standard ASP.NET controls:

<%@ Control Language="C#" AutoEventWireup="true" CodeBehind="PasswordRecovery.ascx.cs" Inherits="SampleApp.WebUI.EmailTemplates.PasswordRecovery" %>
<h1>Password Recovery</h1>
<p>Your new password is <asp:Literal ID="NewPassword" runat="server" />.</p>

Expose a property for each value that you need to include in the email:

public string Password
{
     get { return NewPassword.Text; }
     set { NewPassword.Text = value; }
}

Now just render the control on demand:

private string RenderControl(Control control)
{
    StringBuilder stringBuilder = new StringBuilder();
    StringWriter stringWriter = new StringWriter(stringBuilder);
    HtmlTextWriter htmlTextWriter = new HtmlTextWriter(stringWriter);

    control.RenderControl(htmlTextWriter);

    return stringBuilder.ToString();
}

How simple is that.

UPDATE: I should mention how to instantiate the control – just calling the default constructor isn’t good enough as you’ll find that all of your controls are null. This is the magic line you’ll need:

EmailTemplates.MembershipApplication emailTemplate = (EmailTemplates.MembershipApplication)LoadControl("~/EmailTemplates/MembershipApplication.ascx");