No matter what your preferred software architecture is — whether it’s n-tier, CQRS, Clean Architecture or whatever — if you’re a .NET developer, you need to watch this video. No exceptions, no excuses. It’s by Udi Dahan, one of the “founding fathers” of CQRS. He makes exactly the same points as I’ve been making about software architecture over the past few years:
Throughout my career, I’ve worked on many projects, in .NET as well as with other platforms and frameworks. One particular practice that I’ve encountered time and time and time again in .NET, which I rarely see elsewhere, is that of having a separate identical set of models for each layer of your project, mapped one to another by rote with AutoMapper.
It’s a practice that I detest with a passion. It adds clutter and repetition to your codebase without delivering any benefit whatsoever, and gets in the way of important things such as performance optimisation. In fact, if you suggested it to a Python developer or a Ruby developer, they would probably look at you as if you were crazy. But many .NET developers consider it almost sacred, justifying it on the grounds that “you might want to swap out Entity Framework for something else some day.”
But why should this be? How did speculative generality end up being viewed in the .NET ecosystem as a Best Practice™? In actual fact, there are historical reasons that, in the dim and distant past, were very real concerns.
Back in the early days of .NET, round about 2001/2002, the best practice that Microsoft recommended was to use stored procedures for everything. It didn’t take long for everyone to start complaining on the ASP.NET forums about how cumbersome this was. Half of the .NET community had come from Borland Delphi, with its RAD tools letting you drag and drop data sources and data grids onto a form, while the other half had come from Java, which already had O/R mappers such as Hibernate. To go from either of these approaches to hand-cranking stored procedures, with all the tedious repetition that it involved, was like going back into the stone age.
Naturally, a whole lot of two-guys-in-a-garage ISVs were more than willing to step into the gap with a slew of ORMs. By 2004, we had Entity Broker, Pragmatier, WilsonORMapper, Objectz.net, Sisyphus, NPersist and a host of others that have long since been forgotten. They were coming and going like nobody’s business, and you couldn’t rely on the one you chose still being around six months later. With this being the case, abstracting out your ORM “just in case” you needed to swap it out for something else seemed like an eminently sensible — if not vitally necessary — suggestion.
Within a couple of years, things started to settle down, and two market leaders — the open-source NHibernate and the commercial LLBLGen Pro — emerged. These both quickly gained a solid backing, and they are both still going strong today.
But there was nothing from Microsoft. In the early days they promised us an offering called ObjectSpaces, but it was subsequently abandoned as vapourware.
This was a problem for some people. Right from the beginning, the majority of .NET developers have worked in companies and teams that wouldn’t touch anything that didn’t come from Microsoft with a barge pole if they didn’t have to. But working with DataSets and stored procedures was so painful that they held their noses and used NHibernate anyway — but wrapped it in an abstraction layer in the hope that they could swap it out for Entity Framework the moment that the latter became stable enough for them to do so.
Entity Framework finally appeared in 2008, but the first version was so bad that many in the .NET community started up a vote of no confidence in it. It was 2011 — ten years after .NET 1.0 was first released to beta — before Entity Framework was good enough to see serious use in production, and a further two years before it reached a similar level of functionality to NHibernate.
Nowadays, of course, Entity Framework is well established and mature, and although there are differences between EF6 and EF Core, the only thing these days that you’re likely to want to swap it for is hand-crafted SQL for performance reasons — and that usually means cutting right across your neat separation between your DAL and business layers altogether. Even testing is scarcely a reason any more now that EF Core has an in-memory provider for the purpose.
But old habits die hard, and by the time we got here the practice of abstracting your O/R mapper on the grounds that “you might want to swap out your data access layer for something else” had become deeply entrenched as a Best Practice. Many of its advocates are too young to remember its historical context, so they aren’t aware that it is aimed at a use case whose likelihood has nosedived. Nor are they aware that although we once had a good idea of what we’d have to swap our DAL out for, nowadays all we can talk about are unknown mystery alternatives. But this is why we constantly need to be reviewing our best practices to see whether they still apply. Because if we don’t, they just fossilise into cargo cult programming. And that benefits nobody.
- 4 May 2019: Grace now passes all Microsoft’s tests as of version 7.0.0. Updated other containers to the latest versions.
Since my last post on the state of IOC containers in .NET Core, I’ve ended up going down a bit of a rabbit hole with this particular topic. It occurred to me that since Microsoft has come up with a standard set of abstractions, it is probably best, when choosing a container, to pick one that conforms to these abstractions. After all, That Is How Microsoft Wants You To Do It.
But if you want to do that, what are your options? Which containers conform to Microsoft’s specifications? I decided to spend an evening researching this to see if I could find out.
Rather helpfully, there’s a fairly comprehensive list of IOC containers and similar beasties maintained by Daniel Palme, a .NET consultant from Germany, who regularly tests the various options for performance. He currently has thirty-five of them on his list. With this in mind, it was just an evening’s work to go down the list and see where they all stand.
I looked for two things from each container. First of all, it needs to either implement the Microsoft abstractions directly, or else provide an adapter package on NuGet that does. Secondly, it needs to pass the specification tests in the
In the end of the day, I was able to find adapters on NuGet for twelve of the containers on Daniel’s list. Seven of them passed all seventy-three test cases; five failed between one and four of them. They were as follows:
|AutoFac 4.9.2||AutoFac.Extensions.DependencyInjection 4.4.0||All passed|
|Castle Windsor 5.0.0||Castle.Windsor.MsDependencyInjection 3.3.1||All passed|
|DryIoc 4.0.4||DryIoc.Microsoft.DependencyInjection 3.0.3||All passed|
|Grace 7.0.0||Grace.DependencyInjection.Extensions 7.0.0||All passed|
|Lamar 3.0.2||Lamar 3.0.2||2 failed|
|LightInject 5.4.0||LightInject.Microsoft.DependencyInjection 2.2.0||4 failed|
|Maestro 3.5.0||Maestro.Microsoft.Dependencyinjection 2.1.2||4 failed|
|Microsoft.Extensions.DependencyInjection 2.2.0||All passed|
|Rezolver 1.4.0||Rezolver.Microsoft.Extensions.DependencyInjection 2.2.0||All passed|
|Stashbox 2.7.3||Stashbox.Extensions.Dependencyinjection 2.6.8||All passed|
|StructureMap 4.7.1||StructureMap.Microsoft.DependencyInjection 2.0.0||2 failed|
|Unity 5.10.3||Unity.Microsoft.DependencyInjection 5.10.2||All passed|
Which tests failed?
It’s instructive to see which tests failed. All but one of the failing tests failed for more than one container.
ResolvesMixedOpenClosedGenericsAsEnumerable. This requires that when you register an open generic type (for example, with
svc.AddSingleton(typeof(IRepository<>), typeof(Repository<>))) and a closed generic type (for example,
IRepository<User>), a request for
IEnumerable<IRepository<User>>should return both, and not just one. Lamar and StructureMap all failed this test.
DisposesInReverseOrderOfCreation. Does what it says on the tin: last in, first out. Lamar, Maestro and StructureMap fail this test.
LastServiceReplacesPreviousServicestests that when you register the same service multiple times and request a single instance (as opposed to a collection), the last registration takes precedence over the previous registrations. LightInject fails this test.
ResolvesDifferentInstancesForServiceWhenResolvingEnumerablechecks that when you register the same service multiple times, you get back as many different instances of it as you registered. LightInject fails three of the test cases here; Maestro fails two.
DisposingScopeDisposesServicechecks that when a container is disposed, all the services that it is tracking are also disposed. Maestro fails this test — most likely for transient lifecycles, because different containers have different ideas here about what a transient lifecycle is supposed to mean with respect to this criterion.
These failing tests aren’t all that surprising. They generally concern more complex and esoteric aspects of IOC container functionality, where different containers have historically had different ideas about what the correct behaviour should be. They are also likely to be especially difficult for existing containers to implement in a backwards-compatible manner.
Nevertheless, these are still tests that are specified by Microsoft’s standards, and furthermore, they may cause memory leaks or incorrect behaviour if ASP.NET MVC or third party libraries incorrectly assume that your container passes them. This being the case, if you choose one of these containers, make sure you are aware of these failing tests, and consider carefully whether they are ones that are likely to cause problems for you.
The most surprising result here was Lamar. Lamar is the succesor to StructureMap, which is now riding off into the sunset. It was also written by Jeremy Miller, who has said that two of his design goals were to be fully compliant with Microsoft’s specification from the word go, while at the same time having a clean reboot to get rid of a whole lot of legacy baggage that StructureMap had accumulated over the years and that he was sick of supporting. It is also the only container in the list that supports the DI abstractions in the core assembly; the others all rely on additional assemblies with varying amounts of extra complexity. However, the two failing tests in Lamar were exactly the same as the failing tests in StructureMap, so clearly there has been enough code re-use going on to make things difficult. Furthermore, the tests in question represent fairly obscure and low-impact use cases that are unlikely to be a factor in most codebases.
Most of the IOC containers on Daniel’s list for which I couldn’t find adapters are either fairly obscure ones (e.g. Cauldron, FFastInjector, HaveBox, Munq), dead (e.g. MEF), or not actually general purpose IOC containers at all (e.g. Caliburn Micro). There were, however one or two glaring omissions.
Probably the most prominent one was Ninject. Ninject was the first IOC container I ever used, when I was first learning about dependency injection about ten years ago, and it is one of the most popular containers in the .NET community. Yet try as I might, I simply have not been able to find a Ninject adapter for the .NET Core abstractions anywhere. If anyone knows of one, please leave a note in the comments below and I’ll update this post accordingly.
Having said that, it isn’t all that surprising, because Ninject does have some rather odd design decisions that might prove to be a stumbling block to implementing Microsoft’s specifications. For example, it eschews nested scopes in favour of tracking lifecycles by watching for objects to be garbage collected. Yes, seriously.
Another popular container that doesn’t have an adapter is Simple Injector. This is hardly surprising, though, because Simple Injector has many design principles that are simply not compatible with Microsoft’s abstraction layer. The Simple Injector authors recommend that their users should leave Microsoft’s built in IOC container to handle framework code, and use SimpleInjector as a separate container for their own application code. If SimpleInjector is your personal choice here, this is probably a good approach to consider.
Finally, there doesn’t seem to be an adapter for TinyIOC, which is not on Daniel’s list. However, since TinyIOC is primarily intended to be embedded in NuGet packages rather than being used as a standalone container, this is not really surprising either.
Some final observations
I would personally recommend — and certainly, this is likely to be my practice going forward — choosing one of the containers that implements the Microsoft abstractions, and using those abstractions to configure your container as far as it is sensible to do so. Besides making it relatively easy to swap out your container for another if need be (not that you should plan to do so), the Microsoft abstractions introduce a standard vocabulary and a standard set of assumptions to use when talking about dependency injection in .NET projects.
However, I would strongly recommend against restricting yourself to the Microsoft abstractions like glue. Most IOC containers offer significant added value, such as convention-based registration, lazy injection (
Lazy<T>), interception, custom lifecycles, or more advanced forms of generic resolution. By all means make full use of these whenever it makes sense to do so.
For anyone who wants to tinker with the tests (or alert me to containers that I may have missed), the code is on GitHub.
One of the first things that I had to do at my new job was to research the IOC container landscape for ASP.NET Core. Up to now we’ve been using the built-in container, but it’s turned out to be pretty limited in what it can do, so I’ve spent some time looking into the alternatives.
There is no shortage of IOC containers in the .NET world, some of them with a history stretching as far back as 2004. But with the arrival of .NET Core, Microsoft has now made dependency injection a core competency baked right into the heart of the framework, with an official abstraction layer to allow you to slide in whichever one you prefer.
This is good news for application developers. It is even better news for developers of libraries and NuGet packages, as they can now plug straight into whatever container their consumer uses, and no longer have to either do dependency injection by hand or to include their own copies of TinyIOC. But for developers of existing containers, it has caused a lot of headaches. And this means that not all IOC containers are created equal.
Conforming Containers in .NET Core
Originally, the .NET framework provided just a simple abstraction layer for IOC containers to implement: the
IServiceProvider interface. This consisted of a single method,
GetService(Type t). As such, all an IOC container was expected to do was to return a specific service type, and let the consumer do with it what it liked.
But there’s a whole lot more to dependency injection than just returning a service that you’re asked for. IOC containers also have to register the types to be resolved, and then — if required to do so — to manage their lifecycles, calling
.Dispose() on any
IDisposable instances at the appropriate time. When you add in the possibility of nested scopes and custom lifecycles, it quickly becomes clear that there’s much more to it than just resolving services.
And herein lies the problem. For with the introduction of
Microsoft.Extensions.DependencyInjection and its abstractions, Microsoft now expects containers to provide a common interface to handle registration and lifecycle management as well.
This kind of abstraction is called a Conforming Container. The specification that conforming containers have to follow is defined in a set of more than fifty specification tests in the ASP.NET Dependency Injection repository on GitHub. It includes such requirements as:
- When you register multiple services for a given type, when you request one, the one that you get back has to be the last one registered.
- When you request all of them, they have to be returned in the order that they were registered.
- When a container is disposed, it has to dispose services in the reverse order to that in which they were created.
- There are also rules around which constructor to choose, registration of open generics, requesting types that haven’t been registered, resolving types lazily (
Lazy<TService>) and a whole lot more.
These specification tests are also available as a NuGet package.
There are two points worth noting here. First, conforming containers MUST pass these tests otherwise they will break ASP.NET Core or third party libraries. Secondly, some of these requirements simply cannot be catered for in an abstraction layer around your IOC container of choice. If a container disposes services in the wrong order, for example, there is nothing you can do about it. Cases such as these require fundamental and often complex changes to how your container works that in some cases might be breaking changes.
For what it’s worth, this is a salutary lesson for anyone who believes that they can make their data access layer swappable simply by wrapping it in an
IRepository<T> and multiple sets of models. Data access layers are far more complicated than IOC containers, and the differences between containers are small change compared to what you’ll need to cater for if you want to swap out your DAL. As for making entire frameworks swappable, I’m sorry Uncle Bob, but you’re simply living in la-la land there.
All containers are equal, but some are more equal than others
So should we just stick with the default container? While many developers will, that is not Microsoft’s intention. The built in container was explicitly made as simple as possible and is severely lacking in useful features. It can not resolve unregistered concrete instances, for example. Nor does it implicitly register
Lazy<T> (though the latter can be explicitly registered as an open generic). Nor does it have any form of validation or convention-based registration. It is quite clear that they want us to swap it out for an alternative implementation of our choice.
However, this is easier said than done. Not all IOC containers have managed to produce an adapter that conforms to Microsoft’s specifications. Those that have, have experienced a lot of pain in doing so, and in some cases have said that there will be behavioral differences that won’t be resolved.
For example, the authors of SimpleInjector have said that some of their most innovative features — specifically, those that support strong, early validation of your registrations — are simply not compatible with Microsoft’s abstractions. Travis Illig, one of the authors of Autofac, noted that some of the problems he faced were incredibly complex. Several commenters on the ASP.NET Dependency Injection GitHub repo expressed concerns that the abstraction is fragile with a very high risk that any changes will be breaking ones.
There are also concerns that third party library developers might only test against the default implementation and that subtle differences between containers, which are not covered by the specification, may end up causing problems. Additionally, there is a concern that by mandating a standard set of functionality that all containers MUST implement, Microsoft might be stifling innovation, by making it hard (or even impossible) to implement features that nobody else had thought of yet.
But whether we like it or not, that is what Microsoft has decided, and that is what ASP.NET Core expects.
Build a better container?
So what is one to do? While these issues are certainly a massive headache for authors of existing IOC containers, it remains to be seen whether they are an issue for authors of new containers, written from scratch to implement the Microsoft specification from the ground up.
This is the option adopted by Jeremy Miller, the author of StructureMap. He recently released a new IOC container called Lamar, which, while it offers a similar API to StructureMap’s, has been rebuilt under the covers from the ground up, with the explicit goal of conforming to Microsoft’s specification out of the box.
Undoubtedly, there will be other new .NET IOC containers coming on the scene that adopt a similar approach. In fact, I think this is probably a good way forward, because it will allow for a second generation of containers that have learned the lessons of the past fifteen years and are less encumbered with cruft from the past.
Whether or not the concerns expressed by authors of existing containers will also prove to be a problem for authors of new containers remains to be seen. I personally think that in these cases, the concerns may be somewhat overblown, but whether or not that turns out to be the case remains to be seen. It will be interesting to see what comes out in the wash.
A colleague asked me the other day what I thought about “Uncle Bob” Robert C Martin’s Clean Architecture.
It’s admittedly not something to which I’ve given much thought. I’ve always had a lot of respect for Uncle Bob and his crusade for greater standards of professionalism and craftsmanship in software development. I have two of his books — Clean Code and The Clean Coder — and I heartily recommend them to software professionals everywhere.
But I hadn’t given much thought to what he says about architecture in particular, so I thought I’d check it out.
He has written a whole book about the subject. I haven’t read it in its entirety yet, but he also wrote a short summary in a blog post back in 2012. He illustrates it with this diagram:
It’s basically a different way of layering your application — one that rethinks what goes where. That’s fair enough. One of the things that a clean architecture needs to deliver is clear, unambiguous guidelines about what exactly goes where. A lot of confusion in many codebases arises from a lack of clarity on this one.
But the other thing that a clean architecture needs to deliver is a clearout of clutter and unnecessary complexity. It should not encourage us to build superfluous anaemic layers into our projects, nor to wrap complex, unreliable and time-wasting abstractions around things that do not need to be abstracted. My question is, how well does Uncle Bob’s Clean Architecture address this requirement?
Separation of concerns or speculative generality?
Uncle Bob points out that the objective at stake is separation of concerns:
Though these architectures all vary somewhat in their details, they are very similar. They all have the same objective, which is the separation of concerns. They all achieve this separation by dividing the software into layers. Each has at least one layer for business rules, and another for interfaces.
But it’s important to remember that separation of concerns is a means to an end and not an end in itself. Separation of concerns is only a useful practice when it addresses requirements that we are actually facing in reality. Making your code easy to read and follow is one such requirement. Making it testable is another. When separation of concerns becomes detached from meeting actual business requirements and becomes self-serving, it degenerates into speculative generality. And here be dragons.
The classic example of speculative generality is the idea that you “might” want to swap out your database — or any other complex and fundamental part of your system — for some other unknown mystery alternative. This line of thinking is very, very common in enterprise software development and it has done immense damage to almost every codebase I’ve ever worked on. Time and time again I’ve encountered multiple sets of identical models, one for the Entity Framework queries, one for the (frequently anaemic) business layer, and one for the controllers, mapped one onto another in a grotesque violation of DRY that serves no purpose whatsoever but has only got in the way, made things hard, and crucified performance. Furthermore, this requirement is seldom needed, and on the rare occasions when it is, it turns out that the abstractions built to facilitate it were ineffective, insufficient, and incorrect. All abstractions are leaky, and no abstractions are more leaky than ones built against a single implementation.
You really don’t want to be subjecting your codebase to that kind of clutter. It is the complete antithesis of every reasonable concept of “clean” imaginable. In any case, if it’s not a requirement that your clients are actually asking for, and are willing to pay extra for, it is stealing from the business. It is no different from taking your car to the garage with nothing more than faulty spark plugs and being told you need a completely new engine.
Ground Control to Uncle Bob
So how does Uncle Bob’s Clean Architecture stack up in this respect? It becomes fairly clear when he lists its benefits.
1. Independent of Frameworks. The architecture does not depend on the existence of some library of feature laden software. This allows you to use such frameworks as tools, rather than having to cram your system into their limited constraints.
2. Testable. The business rules can be tested without the UI, Database, Web Server, or any other external element.
3. Independent of UI. The UI can change easily, without changing the rest of the system. A Web UI could be replaced with a console UI, for example, without changing the business rules.
4. Independent of Database. You can swap out Oracle or SQL Server, for Mongo, BigTable, CouchDB, or something else. Your business rules are not bound to the database.
5. Independent of any external agency. In fact your business rules simply don’t know anything at all about the outside world.
Points two and three are good ones. These are points that true separation of concerns really does need to address. We need to be able to test our software, and if we can test our business rules independently of the database, so much the better — though it should be borne in mind that this isn’t always possible. Similarly, just about every application needs to support multiple front ends these days: a web-based UI, a console application, a public API, and a smattering of mobile apps.
But points 1, 4 and 5 are the exact problem that I’m talking about here. They refer to complex, fundamental, deeply ingrained parts of your system, and the idea that you might want to replace them is nothing more than speculation.
Point 1 actually makes two mutually contradictory statements. A “library of feature laden software” is the exact polar opposite of “having to cram your system into their limited constraints.” In fact, if anything is “cramming your system into their limited constraints,” it is attempting to reduce your system to support the lowest common denominator between all the different frameworks.
When I get to point 5, I have to throw my hands up in the air and ask, what on earth is he even talking about here?! Business rules are, by their very definition, all about the outside world! Or is he trying to tell us that we need to abstract away tax codes, logistics, Brexit, and even the laws of physics themselves?
When I read things like this, it reminds me of one thing. This essay by Joel Spolsky:
When great thinkers think about problems, they start to see patterns. They look at the problem of people sending each other word-processor files, and then they look at the problem of people sending each other spreadsheets, and they realize that there’s a general pattern: sending files. That’s one level of abstraction already. Then they go up one more level: people send files, but web browsers also “send” requests for web pages. And when you think about it, calling a method on an object is like sending a message to an object! It’s the same thing again! Those are all sending operations, so our clever thinker invents a new, higher, broader abstraction called messaging, but now it’s getting really vague and nobody really knows what they’re talking about any more. Blah.
When you go too far up, abstraction-wise, you run out of oxygen. Sometimes smart thinkers just don’t know when to stop, and they create these absurd, all-encompassing, high-level pictures of the universe that are all good and fine, but don’t actually mean anything at all.
These are the people I call Architecture Astronauts…
I’m sorry, but if making your business logic independent of the outside world isn’t architecture astronaut territory, then I don’t know what is.
Clean means less clutter, not more
There are other problems too. At the start, he says this:
Each has at least one layer for business rules, and another for interfaces.
This will just encourage people to implement Interface/Implementation Pairs in the worst possible way: with your interfaces in one assembly and their sole implementations in another. While there may be valid reasons to do this (in particular, if you are designing some kind of plugin architecture), it shouldn’t be the norm. Besides making you jump around all over the place in your solution, it makes it hard to use the convention-based registration features provided by many IOC containers.
Then later on he speaks about what data should cross the boundaries between the layers. Here, he says this:
Typically the data that crosses the boundaries is simple data structures. You can use basic structs or simple Data Transfer objects if you like. Or the data can simply be arguments in function calls. Or you can pack it into a hashmap, or construct it into an object. The important thing is that isolated, simple, data structures are passed across the boundaries. We don’t want to cheat and pass Entities or Database rows. We don’t want the data structures to have any kind of dependency that violates The Dependency Rule.
This is horrible, horrible advice. It leads to the practice of having multiple sets of identical models for no reason whatsoever clogging up your code. Don’t do that. It’s far simpler to just pass your Entity Framework entities straight up to your controllers, and only transform things there if you have specific reasons to do so, such as security or a mismatch between what’s in the database and what needs to be displayed to the user. This does not affect testability because they are POCOs already. Don’t over-complicate things.
Of course, there may be things that I haven’t understood here. As I said, I haven’t read the book, only the blog post, and he no doubt mentions all sorts of caveats and nuances that need to be taken into account. But as they say, first impressions count, and when your first impressions include a sales pitch for layers of abstraction that experience tells me are unnecessary, over-complicated, and even outright absurd, it doesn’t exactly encourage me to read any further. One of the things that a clean architecture needs to deliver is the elimination of unnecessary and unwieldy layers of abstraction, and I’m not confident that that is what I’ll find.