Sunday, 14 June 2009

Managing Loosely-Ordered Collections : Lopsided Fun with NHibernate

NHibernate’s One-To-Many mapping allows you to map database one-to-many relationships into various kinds of .NET collections, but I recently hit a bit of a snag regarding the ordering of the collections. For example, consider a simple content management system for a music store’s website. This table:

Id Name ParentPageId Position
1 Home null 0
2 Products 1 0
3 Guitars 2 0
4 Basses 2 1
5 Keyboards 2 3
6 Drums 2 4
7 Services 1 0
8 Guitar Tuition 7 0
9 Guitar Repairs 7 1

represents a simple page-tree structure that looks like this:

    +  Products
        + Guitars
        + Basses
        + Keyboards
        + Drums
    + Services
        + Guitar Tuition
        + Guitar Repairs

The key here is that the order of the elements is controlled by the Position column, so I can change the order of any page’s child pages. NHibernate can cope with this just fine by using an indexed collection – but this only works as long as the Position column is always populated with unique, sequential, non-null values. Null Position values cause it to blow up. It’ll also leave gaps in the list if there’s gaps in your sequence – five records with Position values 1,2,4,5,8 will be mapped into a nine-element list with NULL entries at indices 0,3,6,7, which turns your foreach() loops into little baby minefields and requires liberal use of guard clauses.

That’s not quite what I’m after here. All I want to do is preserve the order of elements if an order has been defined. There’s other apps talking to this database that aren’t using NH, which respect ordering by Position when retrieving records but don’t necessarily use sequential zero-based indices when saving changes. The actual data I’m dealing with is often going to look more like this:

Id Name ParentPageId Position
1 Home null 0
2 Products 1 0
3 Guitars 2 2
4 Basses 2 null
5 Keyboards 2 5
6 Drums 2 6
7 Services 1 7
8 Guitar Tuition 7 null
9 Guitar Repairs 7 4

All I’m after is that when I retrieve elements, they are ordered by the Position column (using whatever ORDER BY semantics are in use on the database server), and that if I move things around in the list and then save it, the order of my list is preserved when saving. I don’t care about null values – if there’s no explicit positions defined, just stick whatever you’ve got in a list in any old order and give it back to me. Likewise duplicate values are OK, and if there’s missing values, don’t give me NULLs, just skip to the next element.

One possible solution I’ve come up with looks like this. First, define a Position property on the entity. The getter returns the item’s current index in the parent’s children collection; and the setter is private and does nothing (but NHibernate won’t let you leave it out)

public virtual int Position {
    // Getter returns the current index of this element in its parent's children collection.
    get {
        if (this.Parent == null) return (0);
        if (this.Parent.Children == null) return (0);
        return (this.Parent.Children.IndexOf(this));
    // Setter does nothing - order is determined by the "order-by" attribute in NHibernate mappings,
    // but NH requires that the setter exists.
    private set { }

Then there’s the actual mapping. We want to map this Position property to a DB column, so when doing insert/update operations, the value is persisted to the Position column, and we want to add an order-by attribute to the NHibernate mapping so that when we retrieve the collection, it comes back in the right order. If you’re using the lovely functional goodness that is Fluent NHibernate, the mapping looks like this:

public class CmsPageMap : ClassMap<CmsPage> {
    public CmsPageMap() {
        Map(page => page.Position);
        HasMany<CmsPage>(p => p.Children)
            // Using SetAttribute() here because Fluent NHibernate doesn't support order-by yet.
            .SetAttribute("order-by", "Position");

and if you’re mapping it using XML files, you’ll need something like this:

<property name="Position" type="Int32">
  <column name="Position" />
<many-to-one name="Parent" column="ParentPageId" />
<bag name="Children" inverse="true" cascade="all-delete-orphan" order-by="Position">
  <key foreign-key="Page_Parent_Children" column="ParentPageId" />
  <one-to-many class="NinjaCms.Shared.Model.CmsPage, NinjaCms.Shared" />

The asymmetry of having a property that’s an order-by on the way out and a column mapping on the way back is slightly weird, but it works.

Tuesday, 9 June 2009

What If They Ran The Trains For Free?

Sexy Dynamite is on the Bakerloo line, but it's in Zone 漢 字. (Just kidding. It's actually a shop in Tokyo.)London is gearing up for a two-day Tube strike – industrial action by London Underground staff means tomorrow morning, three million people who’d normally take the tube will be walking, cycling, packing themselves into crowded buses or overground trains, or just pretending they’re sick and staying at home. Papers say the strike is going to cost £100m in lost productivity. The net is a-twitter with harassed Londoners who want to kick Bob Crow in the bollocks.

Now, I recall once hearing a story, which I cannot now find, so the details are somewhat hazy, but I’ll tell it like I remember it and you can tell me if I’m wrong.  It’s about a railway strike. I believe it happened in Japan sometime after the war, but the details aren’t really important. It started out like most industrial disputes – the railway staff wanted something, and the Powers That Be didn’t want to give it to them.

So the staff resorted to industrial action. They didn’t strike, picket, protest, or stay at home. They went to work early, they cleaned the trains, fuelled them, and ran them as usual – but they opened all the barriers, shut off all the ticket machines, and let everybody travel for free.

Imagine if the RMT tried this. There’d be trains full of happy, supportive passengers right now. By Thursday, every commuter, employer and tourist in London would be cheering the drivers, encouraging the staff, showing their support and praising Bob Crow as a public hero. Meanwhile, London Underground are panicking, watching their lost revenue shoot into the tens of millions, and probably hammering out a deal pretty damn sharpish so they can get the ticket barriers switched back on.

Surely that’s a nicer way of making your point than pissing off three million people?

Sunday, 7 June 2009

White Noise and Chaos

It’s been observed, both in science fact and in science fiction, that the radio transmissions of any sufficiently advanced civilization are indistinguishable from random noise. Think about this for one second. The most efficient way of utilising available bandwidth is to encrypt the stuff that needs to stay secret, and compress the hell out of everything else. End result – every repeating pattern in the signal becomes obliterated, either because it’s been deliberately hidden to prevent the signal being decrypted, or because compression algorithms have exploited – and thus eliminated – the redundancy in the repeating patterns. If E.T. is out there, he’s probably using gzip and PGP, and so even if we could pick up his transmissions they’d look exactly like random noise coming from every other corner of the universe.

Does the same thing apply to software development teams? What happens once you’ve established a solid system for code reuse, a culture of refining and refactoring, implemented continuous integration for  builds, testing and deployment, and generally automated or eliminated all the routine, repetitive manual processes you can find? When you reach a point where every predictable aspect of the build and release process has been automated, you’ve effectively guaranteed that the only bits left for the people to do are the creative, unpredictable, subjective, random aspects of the process.

Does that mean that, to an outside observer, the behaviour of any sufficiently advanced software team is indistinguishable from chaos?

(And if is is… what does mean for new hires trying to learn your culture and processes?)

Photo from oooJasonOoo via Flickr, used by permission under Creative Commons. Thanks.

Friday, 5 June 2009

NHibernate / Castle ByteCode provider – are you running the right number of bits?

Just had one of those head-scratching moments… checked out a known working project including NHibernate and ActiveRecord DLLs, built the whole thing – which built without errors - and then got this lovely message:

Unable to load type 'NHibernate.ByteCode.Castle.ProxyFactoryFactory, NHibernate.ByteCode.Castle' during configuration of proxy factory class.
Possible causes are:
- The NHibernate.Bytecode provider assembly was not deployed.
- The typeName used to initialize the 'proxyfactory.factory_class' property of the session-factory section is not well formed.

Confirm that your deployment folder contains one of the following assemblies:

I’ve seen that message quite a lot recently, but thought we’d well and truly sorted it… well, it turns out there’s another possible cause – you’re building for “Any CPU” on a 64-bit machine and your copy of NHibernate.ByteCode.Castle.dll was built on a colleague’s 32-bit machine. Because this DLL is loaded at runtime there’s no compile-time checking that it’s the right bitness – so it’ll build fine, and then blow up.

Setting VS2008’s platform target to “x86” has solved it in this case – I guess doing a 64-bit build of the relevant DLL would also work.

Thursday, 4 June 2009

Just for Fun: The Webcam Teleprompter Shroud Experiment

My first experience of video conferencing was on a £10,000 dual ISDN Pentium system back in 1997, and it was horrible – low quality, unresponsive, and this weird problem where the person on the other end never actually looked AT you, they always looked slightly off to one side or above your head. Nowadays, of course, the video and audio quality are massively improved, and even a £300 netbook has a pretty good built-in webcam – but they still haven’t solved the problem of the camera and the screen not being in the same place – so if we’re chatting on Skype, and I’m looking at you, I’m not looking at the camera – so to you, I appear to be looking slightly above your head, or off to one side, or whatever. Does this look familiar? If this was a face-to-face conversation, I’d be staring at your chest/cleavage/awesome ninja T-shirt… not ideal.


TV teleprompters have solved this problem – they use a piece of glass, tilted at 45°, inside a darkened “shroud” ; the camera lens is hidden behind the darkened glass, so the newsreader sees the reflection of the hidden screen, and the camera doesn’t see the screen at all – it sees straight through the darkened glass to where the newsreader is sitting. Yes, this is a bit magic (or maybe it’s just sufficiently advanced?) It’s worth remembering that a sheet of glass which is lit on one side and dark on the other actually behaves like a mirror – notice how you see your reflection when you look out of your house windows after dark, but you never see your reflection when it’s bright outside?

Point is, teleprompters work on some pretty basic optics. I’ve wondered for a long time whether the same principle could solve the video-conferencing problem, so I decided to hack together a prototype to see if it would work… and although the image quality is pretty rough, it does actually work. I was quite surprised. In this screenshot, I’m looking at Skype  - NOT at the webcam, which is flipped 90° and pointing straight down from inside the cardboard gizmo on top of the screen – and yet the video feed shows me looking straight at the camera.


Optically, here’s what’s going on. Once Skype’s running, just drag the remote video window so it lines up behind the glass exactly (my webcam has a tiny light next to the lens that shows up in the reflection, which makes this part pretty easy), and then when you look at the person you’re talking to, you’ll actually make eye contact instead of staring over their head or down at their chest.


and here’s the actual contraption itself; the glass is from one of those cheap postcard picture frames, and it’s just held in place with blu-tak inside a home-made cardboard “shroud”. The webcam’s a Logitech Communicator Deluxe, which has a neat flexible base/clamp/grip thing that makes it pretty easy to flip, and the last thing I had to do was tweak the webcam settings in Skype to flip the video 90° vertically.


If anyone else fancies hacking together one of these, drop me a line – I’d be curious to see how well it works when you’ve got eye contact and line of sight on both ends of the video feed.