Tuesday, 11 August 2015

Dynamics CRM Online: Any SMTP Server You Like! (as long as it's Gmail or Yahoo)

UPDATE: I take it all back. Following a really useful call from Dynamics CRM support, it turns out there are three different mail integration scenarios, only there's no reference to the third one in the CRM Online configuration screens.

  1. Server-side sync using Exchange
  2. Server-side sync using POP3/SMTP - this is the one that's restricted to Yahoo/Gmail, presumably because it IS doing something very clever.
  3. Dynamics CRM Email Relay - this is a completely separate tool that you need to download and install, and use it to configure your email routing.

I'm going through the mail relay config as we speak - will let you know how it goes. Sorry, Microsoft. :)


We are doing a lot of work at the moment with Microsoft Dynamics CRM Online. It's generally very nice - as a business tool, it's excellent; the UI is great, it's a good fit for our requirements, and despite a couple of headaches caused by the restrictions of the CRM Online hosting environment, we've now got a pretty elegant two-way data sync system up and running using EasyNetQ and webhooks.

Now, one of our key requirements for CRM is being able to send email to our customers. Yeah, I know - audacious, right? CRM Online is our first foray into the cloudy world of Microsoft Office 365, and all the rest of our infrastructure (Exchange, Active Directory, etc.) is still hosted on-premise. For testing and staging systems, we use mailtrap.io - a really slick service that will accept incoming mail and store it so you can test things are relaying properly without actually sending any real emails. For production, we use Mandrill, which is a commercial mail relay service - high availability, reputation management, excellent delivery statistics. We send about a million emails a month through Mandrill, and it works beautifully.

So... this morning I log into CRM Online, go into Settings => Email Configuration => New POP3/SMTP Server Profile. Looks pretty straightforward, right? I enter some details, click "save" and get this:

image

Weird. I don't want to set up server-side synchronization - I just want to send email. So I start poking around, Googling, that kind of thing, and find this article, which says:

You may have read in the documentation that GMail and Yahoo are listed as supported pop3/smtp providers for Microsoft Dynamics CRM Online and

"Although other POP3/SMTP systems may work with Microsoft Dynamics CRM, those systems were not been tested by Microsoft and are not supported."

Let’s be clear about “not supported“. In this context it means precisely “you will not be able to go past server profile screen as we will reject any pop3/smtp provider that is not GMail or Yahoo.”

And that is exactly what the email relay screen does. If you enter any value other than "smtp.gmail.com" or "smtp.mail.yahoo.com" in the Outgoing Server Location field, you get the "unsupported email server" message. I've even tried modifying the configuration using the SDK instead of the UI, but get the same response - "the email server is unsupported".

There's two possibilities here:

  1. Microsoft have worked closely with Yahoo and GMail to provide first-class business email support, with all sorts of clever features and proprietary extensions that aren't supported by any other SMTP mail servers. (UPDATE - yes, it appears this is more-or-less what they've done. See above.)
  2. Somebody has arbitrarily decided to support GMail and Yahoo and exclude all other SMTP servers. Including, by the way, Hotmail (owned by Microsoft), Outlook.com (owned by Microsoft), and our on-premise SMTP relay (powered by Microsoft Exchange)

If I were a betting man, I know which one I'd be putting money on. I'm really disappointed about this. CRM Online isn't a toy. It's a seriously powerful platform, with a hefty price tag attached... and just when everything is going really nicely, something like this comes along and our beautiful product vision is completely hamstrung by the fact that if we want to email our customers, we need to do it via GMail or Yahoo - and there's absolutely no rational justification for this.

Anyone else encountered with this particular restriction? Anyone have any bright ideas as to how we can work around it?

Tuesday, 4 August 2015

As a Developer, I Want to Abolish User Stories, So That I Can Ship Great Products Faster.

Once upon a time, when you programmed by the seat of your pants and the handlebars of your moustache, lots of people wrote specifications. And there was this lovely idea that you could get a Business Analyst to write you a specification, which laid out in very precise detail exactly how The Software was going to function, and then you would give the specification to a development team and they would build it for you, and Everything Would Be Lovely. Except, of course, it wasn't. Only one team ever shipped a massively successful project using this sort of specification-driven waterfall approach, and trust me - they were smarter than you, and they had a much bigger budget. So some bright folks started wondering why software always went wrong, and suggested a whole load of things that would probably help, and came up with things like scrum, and unit testing, and continuous integration.

One of the great ideas that came out this movement was the idea of user stories. See, specifications tended to be incredibly dry and formal, and often did a really bad job of communicating exactly why you were doing something. Joel Spolsky wrote a great article years ago on writing painless functional specifications, but user stories takes this idea even further. And, like a lot of the good ideas that came out of agile, there's a descriptive element to it and a prescriptive element to it. The descriptive part says "write short, simple stories, not detailed specifications" - and the prescriptive part suggests some 'templates' to help you get the hang of this. There's two formats that became popular for working with user stories - given-when-then and as-a-I-want-so-that.

As-a-I-want-so-that is pretty good for describing, at a very high level, what you are trying to do:

As a marketing coordinator, I want to send email to everyone on our mailing list, so that I can tell them about our big summer sale.

And then you'll add a couple of acceptance criteria to the story:

Given I am not subscribed to the mailing list, when the marketing coordinator sends an email about the summer sale, then I should not receive the email.

Given I have received a newsletter email, when I click the unsubscribe link, then I should be removed from the mailing list.

This sort of clarity makes it easy to build the feature, easy to test the feature, and easy to understand exactly what you're doing and why. See? Simple.

Right. Imagine we have a spec from the Olden Days, and Volume 6, Section 8, Subsection 14, Paragraph 9 says:

The handset will feature a Maplin RK82D component installed on the side of the unit. The RK82D will be positioned exactly 45mm from the top of the unit. When activated, the RK82D component will cause the internal speaker to emit a square wave signal at 16Hz, at a volume of not less than 90dBA as measured from a distance of 1 metre.

Now, let's take an old-school project manager and turn them into an agile product owner. Probably by sending them on a three-day course with a certificate and everything.  When they get back, they'll dig out the old spec, and they'll laugh at how dry and opaque it all is. And they'll sit down, all excited, and start writing stories that look like this:

As a handset user, I want a Maplin N27QQ component installed on the side of the unit, so that when the component is activated the device will emit a square wave signal at 16Hz at a volume of not less than 90dBA measured from a distance of 1m

And then they'll add acceptance criteria:

Given I am a handset user, when I activate the N27QQ component, then the frequency of the square wave signal will be 16Hz

Given I am a handset user, when I activate the N27QQ component and measure the signal volume at a distance of 1m, then the volume of the square wave will be not less than 90dBA

Given I am a handset user, when I examine the handset, then the distance of the N27QQ component shall be 45mm from the top of the unit

and everything is AWESOME because we're being AGILE! The story will sit there in the icebox for a couple of weeks, until it gets bumped up the list and included in a backlog refinement meeting. At which point this conversation will happen:

Dev: "Er... why are we doing this? I don't understand the justification for including this feature."
PM: "Sorry, what?"
Dev: "Well... do you know what a Maplin N27QQ component is?"
PM: "It's the component specified in the specification"
Dev: "Yes... it's also a big round red plastic button the size of a softball"
PM: "Oh. Well, it's in the specification now."
Dev: "Right. Explain to me what happens when you press it"
PM: "Oh, easy. The unit emits a square wave signal at 16Hz, at a volume of..."
Dev: "Yeah, yeah. Do you know what that sounds like?"
PM: "Er... it's probably some sort of ring tone"
Dev: "No, it's a fart noise as loud as a motorcycle"
PM: "Oh. Well, can we estimate the story, please?"

At which point the developer will start flicking through LinkedIn on their phone and wondering how long it is until they can go to the pub.

You know what the story should have said? It should have said something like:

As Bongo the Clown, I want my new phone handset to feature a massive red fart button, so that I can make children laugh at parties when I answer it.

First of all, unless you happen to be in the circus supply business, someone's going to say "hang on - why are we making phone handsets for clowns? Is that a thing now?"

Second, anybody who reads that story can immediately picture what they're building, who wants it, and why. It leads to creative discussion and suggestions. Someone might have a brilliant idea about replacing the fart noise with a trombone, or making the button light up when you press it. One of your team might remember how Dad used to get mad every time Tony Blair came on the radio, but the volume knob on Dad's stereo fell off when he tried to turn the radio off, and how hilarious it was watching him chase it across the kitchen while cursing and muttering under his breath, and maybe we should make the fart button actually fall off when you press it so Bongo has to chase it around the room? Clear stories let you cut through the waffle and get straight to the important bit - what is this? How do we build it? How might we make it better?

Now, compare these two sentences:

  1. As a handset user, I want a Maplin N27QQ component installed on the side of the unit, so that when the component is activated the device will emit a square wave signal at 16Hz at a volume of not less than 90dBA measured from a distance of 1m
  2. We'll put a giant red fart button on the side of the phone, so that Bongo the Clown can use it to make kids laugh when he's doing children's parties.

Which one makes more sense to you, as a reader? Which one is going to lead to better decisions, better estimation and less time wasted in meetings?

As-a-I-want-so-that is not some sort of witchcraft. It doesn't magically translate your dry, meaningless specification into readable English. Like writing unit tests, it can help to keep you honest when you're breaking bad habits, and it's one more tool to keep handy when you're doing this stuff for real, but it is not why we're here. It's the story that matters, not the syntax. And if you can't tell me a short, simple story about what you want and why, then maybe you don't actually understand the thing you're trying to build, and no amount of syntactic convention is going to fix that.