If your tests aren’t hitting the database, you might as well not write tests at all
Posted at 07:00 on 14 August 2014
Out of all the so-called “best practices” that are nothing of the sort, this one comes right up at the top of my list. It’s the idea that hitting the database in your tests is somehow harmful.
I'm quite frankly amazed that this one gets as much traction as it does, because it's actively dangerous. Some parts of your codebase require even more attention from your tests than others -- in particular, parts which are:
- easy to get wrong
- tricky to get right
- not obvious when you're getting it wrong
- difficult to verify manually
- high-impact if you do screw up.
Your data access layer, your database itself, and the interactions between them and the rest of your application fall squarely into all the above categories. There are a lot of moving parts in any persistence mechanism — foreign key constraints, which end of a many-to-many relationship you declare as the inverse, mappings, migrations, and so on, and it’s very easy to make a mistake on any of them. If you've ever had to wrestle with the myriad of obscure, surprising and gnarly error messages that you get with both NHibernate and Entity Framework, you'll know exactly what I mean.
If you never test against a real database, but rely exclusively on mocking out your data access layer, you are leaving vast swathes of your most error-prone and business-critical functionality with no test coverage at all. You might as well not be testing anything.
Yes, tests that hit the database are slow. Yes, it's off-putting to write slow tests. But tests that don't hit the database don't test things that need to be tested. Sometimes, there are no short cuts.
(Incidentally, this is also why you shouldn’t waste time writing unit tests for your getters and setters or for anaemic business services: these are low-risk, low-impact aspects of your codebase that usually break other tests anyway if you do get them wrong. Testing your getters and setters isn't unit testing, it's unit testing theatre.)
“But that rule just applies to unit tests. Integration, functional and regression tests are different.”
I agree there, and I'm not contradicting that. But if you're saying "don't hit your database in your unit tests" and then trying to qualify it in this way, you're just causing confusion.
Regardless of what you are trying to say, people will hear “don’t hit the database in your tests, period.” People scan what you write and pick out sound bites. They see the headline, and skip over the paragraph about integration tests and so on as if it were merely a footnote.
By all means tell people to test their business logic independently of the database if you like, but phrase it in a way that’s less likely to be misunderstood. If you’re leaving them with the impression that they shouldn’t be testing their database, their data access layer, and the interaction between them and the rest of your application, then even if that isn’t your intention, you’re doing them a serious disservice.