Posts tagged: lab notes
My decision to start keeping lab notes, research scientist style, in my work a couple of months ago has started to pay off. By constantly writing down everything I do, when I run into trouble with what I’m doing, I am able to go back over my notes quickly and easily and see exactly what I did. As well as making for a much more disciplined approach to working, it’s helped to clarify my thoughts, and to avoid situations where I might end up spinning my wheels rather than getting to the heart of the matter.
Here are some tips.
1. Choose the most low-friction solution you can get your hands on.
It is very important, especially in the early days when you are just getting used to the discipline, that you choose a solution that is as frictionless as possible.
You are establishing a habit. Those who study these things tell us that it can take several months for a habit to become established to the point at which it seems easier to do it than not to do it, and until you get to that point, anything that tempts you to take shortcuts needs to be resisted and ruthlessly eliminated.
For me, I find that Microsoft Word works fine. Our project manager tried to persuade me to use Jira, so as to have everything in one place, but Jira’s quirky markup syntax, combined with having to type into a tiny input field in a web form, made this a frustrating experience. It’s better to write it in Word and then copy and paste into Jira later if that is what is needed.
2. Write down everything you do, as you do it.
Lab notes work best if you write down what you’re going to do before you do it rather than after the fact. In that sense it’s a bit like test-driven development: write your tests first, then write the code that makes them pass.
This has two benefits. First, it is more accurate and more complete, because you’re less likely to miss out details. Secondly, it has the benefit of forcing you to think clearly about what you’re going to do.
This also makes your lab notebook a record of your stream of consciousness. As such, you will be spending at least as much time in your lab notebook as you are in Visual Studio, or vim, or whatever your IDE of choice happens to be. This is another reason why you need to go for the lowest-friction solution you can get your hands on: even small amounts of friction will chafe at you, and will tempt you to ease off and take shortcuts.
When you’re waiting for a slow compilation or test run to complete, use the time to write down what you’re going to do next. This is a far more effective use of the resulting downtime than reading Reddit or Hacker News, and it’s also a whole lot easier to context-switch back to what you were doing once you’ve finished.
3. You can not be too detailed.
The purpose of lab notes, as with any other form of auditing, is to answer questions. For example, one such question that we all ask, time and time again, is, “I encountered this error before. What did I do to fix it?”
So for instance, if you encounter an error message, copy and paste the message into your notes. Write down exactly what steps you took when it happened. Write down exactly what steps you need to take to reproduce the error. Write down exactly what tests you carried out in order to troubleshoot it, and exactly what the error messages were. Write down any relevant Google searches, and copy and paste the URLs of the web pages that they led you to.
4. Write down your train of thought when you’re planning and designing your code.
Use your lab notes as a scratchpad to plan what each script, each method, and each command should do. How does it store state? How does it identify if it’s been run before? What should it prompt the user for, how, and when? What should the default values be? Why did you give things the names that you did?
It’s actually a lot easier to write code this way, because you’re holding less information in your head at once, and it’s much easier to pick it up again if you break off. It’s also easier to comment your code, because you can just copy and paste some of your design decisions into your source code or your wiki if necessary.
5. Don’t worry too much about making your notes look good.
You aren’t writing documentation, or a final report, or a thesis, or a sales pitch, or a blog post. You are only writing rough notes to remind yourself, or keep your colleagues up to date, about what you’ve done.
Just write it down in the first person (“I ran this command on the command line.”) There’s no reason why you should get all formal and write it down in the passive voice like you typically encounter in scientific literature.
6. Use a searchable text format.
I personally use a Microsoft Word document. Some people use plain text files, or a note-taking application such as OneNote or Evernote. You can also get specialised applications devoted specifically to laboratory note-taking. LabArchives is one such example—a web-based SAAS lab notebooking platform.
While you may be tempted to record error messages or other data as screenshots, bear in mind that you will almost certainly want to search for them again in the future. Record them as text wherever possible.
For the same reason, it’s best not to use a physical notebook unless you need to do so for legal reasons. Scientists often use handwritten lab notes for patent protection purposes, but this isn’t really necessary, since there are electronic solutions that can provide a dated audit trail if needed.
7. Make your notes append-only.
You may be tempted to go back and tidy up your lab notes afterwards, but this is a temptation you need to resist. They are an audit trail, not a design document, nor a presentation, nor a manual, nor a sales pitch. Tidying them up may make them look pretty, but it undermines their integrity, and besides, it’s a waste of time.
You may want to copy and paste from your lab notes into your presentation, your design document, your wiki, or your code, but the tidying up should take place in those other documents. What you have written in your lab notes, you have written. Just move on.
8. Use your notes as a source for documentation, commit summaries and pull request descriptions.
A well maintained set of lab notes will make it much easier to compile a detailed commit summary, or a wiki page, or a pull request. Just copy and paste, and then edit to tidy up as necessary.
I frequently draft out a first version of documentation in my lab notes themselves. That way all I have to do is to copy and paste.
9. Share your notes with your whole team.
Your colleagues should all be able to refer to your notes should they need to. For example, a git bisect session may reveal that Alice was the author of a commit that introduced a bug in your payment processor on a certain date. If you can refer back to Alice’s notes on that day, you’ll have a much better idea of why she made the decisions she made, and you will be able to see how to fix the problem without breaking the new features she was working on herself.
Having said that, I’m not all that keen on the idea of having a single set of notes for the entire team. It can easily get unwieldy if everyone is working on different parts of the codebase, so it’s generally better for each individual to have their own set of notes that they write to. All in all, your mileage may vary on that one.
10. Don’t make your notes public.
If you are working on private, closed-source code, your lab notes will contain a whole lot of proprietary information. They will give your competition the heads-up about what you are working on, allowing them to beat you to market. They may also be of legal importance if your company is ever involved in a patent dispute.
Even if you are working on an open source project, there is another good reason why you should keep your lab notes private: security. They will contain a record of the exact details of any security vulnerabilities that you investigate, and any steps that you take to reproduce them. If hackers get hold of this information, you’re asking for trouble.
11. Learn from your mistakes.
If you ever find yourself asking a question about what you did last time, and your lab notes don’t answer it, that is an indication that there was some kind of information that you didn’t write down because you thought it was too trivial.
If this happens, make a note of exactly what kind of information you hadn’t written down that you should have, and write it down in future. Don’t assume that you won’t need the same kind of information again a third time. If you’ve needed it twice, the chances are pretty high that you will.
Keeping lab notes may come naturally to some programmers, while on the other hand, for others it may require considerable effort. But in the end of the day, practice makes perfect. The more you do it, and the more critically you examine your approach to it, the better you’ll get at it and the easier and more natural it will become.
I’ve recently been getting into various discussions about science and faith with friends at church and on Facebook, and one of the things I’ve been doing is explaining to them exactly how the scientific method works. This got me thinking—there are some aspects of the scientific method that we don’t always apply to software development, that maybe we should.
People with no scientific training often don’t appreciate just how rigorous and exacting the scientific method is. It has a lot of very strict protocols to ensure that research meets stringent standards of quality control. One of these protocols is keeping detailed laboratory notes. In other words, you document everything you do. In meticulous detail. As you do it—not after the fact.
There are several benefits to keeping a detailed record of everything you do in this way. It helps you to keep focused on the task at hand. It gives you something more constructive to do during slow edit/compile/test loops than reading Reddit. It helps you to pick up where you left off after your lunch break or when you get into work first thing in the morning. It jogs your memory for your daily stand-up meeting. It provides you with a searchable audit trail that you can consult to figure out where to find things, why things were done in a certain way, or what exactly happened when you tried X. It provides source material for when you need to write up the documentation proper. And it covers your back. For example, if you’re working from home, it provides evidence—should you need it—that you were actually working and not slacking off.
Of course, if you’re a fan of the Agile Manifesto, you are probably cringing at this idea. Aren’t we supposed to favour working software over comprehensive documentation? In fact, the whole idea no doubt sounds like anathema to many software developers, who hate writing documentation with a passion.
But if we were research scientists rather than software developers, we would be doing this as a matter of course, and we wouldn’t think twice about it. Besides, as Martin Fowler says, if it hurts, do it more often.