Wednesday, 20 July 2016

ASP.NET Core 1.0 High Performance

I have exciting news. My friend and erstwhile colleague James Singleton has just published his first book, ASP.NET Core 1.0 High Performance. James was gracious enough to invite me to contribute the foreword – and since the whole point of a foreword is to tell you all why the book is worth buying, I figured I’d just post the whole thing. Read the foreword, then read the book (or better still, buy it then read it.)

TL;DR: it’s a really good book aimed at .NET developers who want to improve application performance, it’s out now, and you can buy your copy direct from

And that foreword in full, in case you’re not convinced:

"The most amazing achievement of the computer software industry is its continuing cancellation of the steady and staggering gains made by the computer hardware industry."
- Henry Petroski

We live in the age of distributed systems. Computers have shrunk from room-sized industrial mainframes to embedded devices smaller than a thumbnail. However, at the same time, the software applications that we build, maintain and use every day have grown beyond measure. We create distributed applications that run on clusters of virtual machines scattered all over the world, and billions of people rely on these systems, such as email, chat, social networks, productivity applications and banking, every day. We're online 24 hours a day, 7 days a week,  and we're hooked on instant gratification. A generation ago we'd happily wait until after the weekend for a cheque to clear, or allow 28 days for delivery; today, we expect instant feedback, and why shouldn't we? The modern web is real-time, immediate, on-demand, built on packets of data flashing round the world at the speed of light, and when it isn't, we notice. We've all had that sinking feeling... you know, when you've just put your credit card number into a page to buy some expensive concert tickets, and the site takes just a little too long to respond. Performance and responsiveness are a fundamental part of delivering great user experience in the distributed age. However, for a working developer trying to ship your next feature on time, performance is often one of the most challenging requirements. How do you find the bottlenecks in your application performance? How do you measure the impact of those problems? How do you analyse them, design and test solutions and workarounds, and monitor them in production so you can be confident they won’t happen again?

This book has the answers. Inside, James Singleton presents a pragmatic, in-depth and balanced discussion of modern performance optimization techniques, and how to apply them to your .NET and web applications. Starting from the premise that we should treat performance as a core feature of our systems, James shows how you can use profiling tools like Glimpse, MiniProfiler, Fiddler and Wireshark to track down the bottlenecks and bugs that are causing your performance problems. He addresses the scientific principles behind effective performance tuning - monitoring, instrumentation, and the importance of using accurate and repeatable measurements when you’re making changes to a running system to try and improve performance.

The book goes on to discuss almost every aspect of modern application development - database tuning, hardware optimisations, compression algorithms, network protocols, object-relational mappers. For each topic, James describes the symptoms of common performance problems, identifies the underlying causes of those symptoms, and then describes the patterns and tools you can use to measure and fix those underlying causes in your own applications. There’s in-depth discussion of high-performance software patterns like asynchronous methods and message queues, accompanied by real-world examples showing how to implement these patterns in the latest versions of the .NET framework. Finally, James shows how you can not only load test your applications as part of your release pipeline, but can continuously monitor and measure your systems in production, letting you find and fix potential problems long before they start upsetting your end users.

When I worked with James here at Spotlight, he consistently demonstrated a remarkable breadth of knowledge, from ASP.NET to Arduinos, from Resharper to resistors. One day he’d be building reactive front-end interfaces in ASP.NET and JavaScript, the next he’d be creating build monitors by wiring microcontrollers into Star Wars toys, or working out how to connect the bathroom door lock to the intranet so that our bicycling employees could see from their desks when the office shower was free. Since James moved on from Spotlight, I’ve been following his work with Cleanweb and Computing 4 Kids Education. He’s one of those rare developers who really understands the social and environmental implications of technology - that whether it’s delivering great user interactions or just saving electricity, improving your systems’ performance is a great way to delight your users. With this book, James has distilled years of hands-on lessons and experience into a truly excellent all-round reference for .NET developers who want to understand how to build responsive, scalable applications. It’s a great resource for new developers who want to develop a holistic understanding of application performance, but the coverage of cutting-edge techniques and patterns means it’s also ideal for more experienced developers who want to make sure they’re not getting left behind. Buy it, read it, share it with your team, and let’s make the web a better place.

Check it out. The chapter on caching & message queueing is particularly good :)

Wednesday, 15 June 2016

Why I'm Voting Remain: Social Media Redux

So, I wrote a thing about why I'm voting to remain in the EU, and lots of people liked it, but pointed out it was quite long. And they're right. It was. So here's a really nice short manifesto-style version that I think captures the gist of the thing.


If you want to share it, please go ahead

Regular readers – don't go away. Normal tech blogging will resume on the 24th, one way or another. But this is a once-in-a-generation event, and it's important.

Why I’m Voting to Remain in the European Union.

There's a referendum coming up. I'm not going to talk about why it’s happening. I'm not going to share any “facts” about politics or the economy, beyond a few historical details that are the basis for my opinion on this. But I am going to tell you why I'm voting for the UK to remain in the European Union.

I live in London. I’m British, though I grew up in Africa. I’ve never been much of a patriot, although I’ll confess to the odd surge of pride when it comes to Pink Floyd, the BBC and Thrust SSC. I’ve spent most of my adult life working in software development, and I'm lucky enough to work with an outstanding team of people – some of the brightest, smartest people you'll ever meet. More than half of our development team are EU citizens who are living and working in London thanks to the UK being part of the EU. I love working with these people, and there's numerous studies supporting my own anecdotal experience that working with people you like is a massive factor in how happy you are. And, like many Londoners, my friends are a motley assortment of people from all over the world, and a lot of them are here as citizens of the EU. On a very immediate level, it would really suck if they all had to either leave the UK or go through the merry hell of applying for permanent leave to remain, and potentially have to choose between their native citizenship and becoming British citizens. But that's both a very personal and a very temporary thing. I’m sure we'd all get over it eventually.

In the last year, I've been lucky enough to visit Lithuania, Denmark, France, Belgium, Norway and Ukraine. France and Belgium were founder members of the EEC, the predecessor to the European Union. Denmark and the UK joined in 1973. Lithuania, part of the USSR until 1991, became a member of the EU in 2004. Norway is a bit special, but it is a part of the European Economic Area and the Schengen visa area. Ukraine has gone, within my lifetime, from being part of the Soviet Union to winning the Eurovision Song Contest. Ukraine borders four EU member states (Romania, Hungary, Poland, and Slovakia), and whilst I don't think it will be admitted as a member of the European Union any time soon, the prospect of closer integration with the EU is a huge factor in Ukrainian politics. Whilst the UK was gearing up for a referendum about whether to leave the EU, two hundred thousand Ukrainians were marching through Independence Square in protest at their government's failure to sign a trade agreement that would have represented one small step towards eventual EU membership. I've visited all these countries without applying for a visa. I've spent time in these remarkable places with friends and colleagues from all over the world.

The Kon-Tiki, the balsa-wood raft that Thor Heyerdahl sailed from Peru to Polynesia. Photograph © National Geographic

In Oslo, there is a museum dedicated to the great Norwegian explorer Thor Heyerdahl, who dedicated his life to the study of human migration, researching the astonishing stories of people who crossed Earth's vast oceans on flimsy wooden boats in search of a better life. This wanderlust, the urge to venture beyond our comfort zone, is as old as humanity. Along one wall is a quote from Heyerdahl:

“Borders - I have never seen one. But I have heard they exist in the minds of some people.”

He's right. Borders aren't physical things. Sure, some of them run along coastlines and rivers, but that's not the point. Borders are where, once upon a time, we drew a line in the sand and agreed that as long as your lot stayed on your side and our lot stayed on our side, it would be OK if maybe we stopped killing each other for a bit. And, slowly, as we learned to put down the pointed sticks and talk to each other instead, the world has changed. We can cross those borders now. We can live, work and study abroad. We can make friends, fall in love, start families, start businesses. Fifty years ago, someone from Birmingham falling for someone from Bucharest would have been a Pulitzer Prize-winning novel. Today it's just one more data point on Tinder (Or Grindr, depending who's playing.)

And that's why this matters to me. The impact of next week’s referendum will last for generations. It isn't about the petty power struggle between David Cameron and Boris Johnson. It isn't about Jeremy Corbyn or UKIP or immigration or the "war on terror" or economic subsidies. It's about peace, and stability, and the opportunities that affords all of us. The author Charlie Stross captured perfectly my thoughts about this when he wrote:

"The EU is the current incarnation of an institution established in 1947 to ensure that never again would the nations of western Europe go to war with one another."

The European Union is the legacy of a group of nations who, still reeling from the horror and the bloodshed of World War II, agreed that whatever the challenges the future might hold, we would be stronger facing those challenges together. They believed, as I do, that a unified Europe represents a significant step on the road towards a better world for humankind. Next week, the UK goes to the polls to decide whether we want to step off that road or not. Whether we want to close our borders, lock our doors and go it alone. To me, this is tantamount to a decision about whether we, the citizens of the United Kingdom, believe that we deserve to prosper while others are suffering, and there's really only one way I can answer that question and still sleep at night.

Neither side is presenting a terribly cohesive argument. Speculation, scaremongering and misinformation are absolutely rife on both sides. Nobody knows what'll happen if we leave. Nobody really knows what'll happen if we stay. I'm not quite sure what "vote remain" are campaigning on, but from the leaflets I’ve been getting through my front door, it looks to me very much like the "vote leave" camp is campaigning on fear. Fear of immigration and poverty and hardship. Fear of terrorism. Fear of things which are a fact of life for billions of people around the world. I am not scared of these things, because I believe that if we work together, we can solve them, just as we brought lasting peace to the countries of western Europe after centuries of frequent, bloody conflict. Twenty years ago, most British people’s idea of a terrorist was an Irishman in a ski mask. Things can change, and they will.

That doesn’t mean I’m not scared. I'm scared that fear and misinformation are going to rule the day. I'm scared for my friends and colleagues for whom a "leave" result on the 23rd could have a real, immediate impact on their lives and livelihoods. I'm scared that, a century from now, some kid is going ask grandad how the war started, and he’ll tell them a story that starts with Brexit. If you don't think that's a possibility, consider that it's less than a hundred years since the Battle of the Somme - nearly a million British, French and German soldiers killed and wounded on the fields of northern France. No Islamic State, no al-Qaeda, no "war on terror" - just honest God-fearing white guys killing one another for the hell of it. Within thirty years, they'd do it all over again – nearly a thousand veterans of that conflict are still alive today.

Since the 1950s, the EEC, and later the EU, has grown from six countries to twenty-eight, and there has never been an armed conflict between two EU member states. And if you think that EU membership is expensive, remember that World War I nearly bankrupted the UK, and World War II saw us borrow so much money from the United States that we didn't finish paying them back until 2006. Peace is small potatoes compared to the cost of war.

Different people care about different things, and I respect that. Ask yourself what matters to you, research your position, and vote accordingly. Maybe even draw up a list of things you hope are going to happen as a result, so you can validate your assumptions a few years from now. But these are the things that matter to me, and I haven’t heard a single plausible argument that leaving the EU will improve any of them.

I believe that peace is more important than politics. I believe that the desire to travel in search of new ideas and new experiences is fundamental to what makes us human. I believe that open borders and common markets give us unprecedented freedom to explore the world we live in. I believe that membership of the European Union embodies all of these principles, and I’m voting to remain.


Monday, 13 June 2016

Join us at the Progressive.NET Tutorials 2016!

Next week sees the return of the Progressive.NET Tutorials at SkillsMatter here in London.

Prog dot net postcard A6 date sticker.indd

Progressive.NET is a unique event. It goes beyond the high-level slides & code demos of most conferences, and offers a series of hands-on, in-depth workshops with some of the best speakers and experts in the .NET world. This year we’ve got a great lineup of speakers and workshops. On Wednesday, Glenn Block will be showing you how to run C# outside your IDE using ScriptCS, Ian Cooper will be demonstrating high-availability patterns for distributed systems, Toby Henderson will be showing you how to get up and running with .NET Core, and I’ll be running a new workshop about asynchronous programming patterns, async/await and how you can use them to deliver more responsive apps.

Image result for ian cooper

Thursday, we’ve got Ashic Mahtab showing you how to process millions of messages a second using Apache Kafka, we’ve got Mark Gray talking about machine learning in F#. After lunch Ben Hall will be showing you how to get up and running with .NET and Docker, and Phil Trelford will be building a compiler with F#, starting with an abstract syntax tree and a parser, and ending up with a compiler that generates .NET IL code or JavaScript. Thursday wraps up with the Progressive.NET Party – food, drinks, and a chance to hang out with the Progressive.NET speakers and developers in the Space Bar at SkillsMatter’s amazing CodeNode building.

On Friday, we round out the event with a day of talks from some of the best speakers in .NET

  • Rachel Reese on event-driven microservices
  • Charalampos Karypidis on isomorphic JavaScript applications and .NEt
  • Sebastien Lambla on API versioning (and why it’s evil)
  • Rajpal Singh Wilkhu on OpenID Connect and Identity Server
  • David Whitney on metaprogramming
  • Harry Cummings on Node.js for .NET developers
  • Barbara Fusinska talking about predicting the future with Azure machine learning
  • Sam Elamin talking about metrics-driven development
  • Liam Westley on App 2.0 and why the web lost
  • Evelina Gabasova on how to spice up your website with machine learning


Progressive.NET has been running since 2009. It’s an excellent event, and it’s also a great way to keep up to date with everything that’s new in the .NET world. If you’ve been hearing about things like .NET Core, Docker, Identity Server, async and await, machine learning – and you’re not quite sure what they are or how they apply to you – come along to Progressive.NET. By the end of the week, you’ll have a head full of new ideas, a laptop full of working code you can refer back to, and a load of new friends who can help out when you get stuck.

The event is next week, 22-24th June, at the SkillsMatter CodeNode here in London. Tickets are normally £875+VAT, but you can use the code SPECIAL_LDNUG_PROGNET to get 40% off - £525 instead of £875 – so what are you waiting for? Get your ticket now and we’ll see you there!

Friday, 13 May 2016

The Next Big(int) Thing

One of our systems here uses a bigint identity column as a database primary key – because we knew when we built it, back in 2010, that we were going to end up with more than 2,147,483,647 records.

Well, that happened at 12:02 today, and a couple of systems promptly failed – because, despite the underlying database being designed to handle 2^63 records, the POCOs that were being mapped to those classes were using a regular C# int to store the record ID, and so as soon as they got an ID from the database that's bigger than Int32.MaxValue, they blew up. Thanks to the underlying DB schema already supporting 64-bit IDs, the fix was pretty simple – just change int to long in a few carefully-selected places and redeploy the applications – but it's still annoying that something we knew about, and planned for, still came back to bite us. So I started thinking – how could we stop this happening?

The problem is that, despite being a bigint column, we just accepted SQL Server's default identity setting of (1,1) – i.e. start counting at 1, and increment by 1 each time. Which means that until you hit 2-billion-and-something records, it doesn't actually make any difference - and that takes a while. In our case, it took 5 years, 8 months and 26 days. During that time we've made hundreds of changes to our code, and in a handful of those cases, we've mapped that bigint ID onto a regular C# Int32 – and so inadvertently planted a little time-bomb in our production code. Tick, tick, tick…

So here's a nice neat solution, that I wish I'd thought of five years ago. Anytime you create a bigint identity, seed it with (2147483648, 1) – so that right from day one, it's already too big to fit in an Int32. Any system that tries to store an ID in a regular int variable will fail immediately, not in five years when someone creates that magic 2.14-billion-and-somethingth record. Even though you've effectively thrown away 2^32 possible values, you have another (2^64 - 2^32) values to play with, so you've lost a tiny, tiny fraction of the available keyspace in exchange for immediate feedback if any of your client apps can't cope with 64-bit ID values.

Thursday, 5 May 2016

We're Gonna Build a Framework!


Check it out! I made a thing. A musical thing. Well, Billy Joel made the musical thing (which is copyright © 1989 Columbia Records, btw) and I wrote and recorded a new set of lyrics for it, all about software frameworks and how lovely it is to have so many to choose from.

And, for anyone who doesn't believe that every single one of those really is a software development framework, here's the Googlified lyrics.

Handlebars, Hibernate, Solar, Activate,
Phalcon, Flask and Silverstripe and TYPO3 Flow,
Agavi, Pixie, Hazaar MVC,
CodeIgniter, Lithium and PRADO.

Raphael, Bobo, Bottle and Tornado,
Django, CherryPy and WSGI
Glashammer, WebSphere, RedBean, TurboGears,
Albatross, Aquarium, Selenium,


We’re gonna build a framework,
‘cos we wanna use one, but don’t wanna choose one,
We’re gonna build a framework,
we didn’t like the others, so we’ll write another…

SiteCore, Tapestry, Maverick and JSP,
Barracuda, Ay Caramba, Groovy on Grails,
Intercal on Interstates, Cascade, NHibernate,
JDBC, Ruby on Rails
Jasmine, Doctrine, Java Forms Engine,
Active Record, D3, Dapper and Velocity,
Thymeleaf, TopLink, Pyramid, Rethync,
Aura, Rico, Midori and Mojito,

Sitemesh, Cymbeline, Enterprise Java Beans,
Hug, Grok, Boost, Click, anything by Telerik,
Rango, Dojo, LLBLGen Pro,
Carbonado, Seaside, Pylons and Pyroxide

Fusebox, Flight, Flex, ServiceStack, Silex,
Carbon, Cocoa, Ample and Giotto,
Banshee, Symfony, Laravel, Fat Free,
Mocha, Pecker, Hobo, Cuba and Rialto

OpenRasta, Nancy, ASP net MVC,
Kendo, Zend, ODBC and Tempo,
Java Server Faces, Entity Spaces,
Cappuccino, SpecFlow, Polymer, JDO,
Google Web Toolkit, no need to learn Javascript,
JDK, code away, what else do I have to say?

Prototype, Boilerplate, jQuery, animate,
Mustache, Wijmo, Ionic and Allegro,
Bootstrap, Backbone, running on an iPhone;
Mobile apps, going live, built in HTML5
Angular, Scriptaculous, react, redux,
Knockout, ember, does anyone remember,
It’s just like in the browser wars,
IExplore and Netscape 4,
lodash vs underscore,
and I can’t take it any more!

(yes, I know ODBC and JDBC aren't strictly speaking frameworks.)

(yes, I also know the recorded version uses Tempo twice. Relax. It's OK.)

London.NET User Group – A Progressive.NET Special!

This July, the Progressive.NET Tutorials returns to SkillsMatter – for the eighth year running, we’re bringing together some of the best speakers and experts in the .NET community for three days of in-depth workshops and hands-on tutorials. The team at SkillsMatter are working hard to finalize what promises to be a great programme, covering all the latest platforms, patterns and ideas in the world of .NET development.

To coincide with this, we’re running a special London.NET User Group meetup at SkillsMatter the night before the conference – Tuesday 21st June.

imageWe’re going to be hosting a series of short talks and demos, around a theme of “zero footprint” development tools. Over the years, we’ve had a lot of really excellent talks covering languages, platforms, patterns, architectural styles – but quite often, we’ll be in the pub afterwards chatting about the talk we’ve just seen, and you’ll hear people saying “it looks really great; maybe I can try it out on my next project” or “I love the idea but I can’t see the rest of the team going for it.”

So, for our Progressive.NET special, we want to showcase the very best tools, utilities and ideas that you can start using right away – they won’t modify your code, they won’t break your build pipeline, they won’t interfere with what the rest of your team are doing, but they might make your day just a little bit more pleasant – and we want your help!

Come along and give us a 10-minute demo of something you think is awesome. Maybe it’s an open source tool, maybe it’s one of your company’s products, maybe it’s just something you use daily and can’t live without – but remember: zero footprint. Don’t break the build, don’t don’t move the cheese.

In the spirit of things like Scott Hanselman’s 2014 Ultimate Developer and Power Users Tool List, we’re looking for talks about:

Whether you’re a first-timer speaker or a fulltime product evangelist, we’d love to have you on board, so have a think, get in touch – by email or find us @LondonDotNet on Twitter - and get involved!