An update on Lambda Tools

Posted at 08:00 on 23 July 2018

A little under year ago, I started work on a new open source project to manage deployment of serverless code to AWS Lambda. This grew out of a task that I'd started at work, where we had a number of Lambda functions managing various features of our infrastructure. At the time, they were being managed rather chaotically through Terraform and I wanted to get a Continuous Delivery pipeline set up for them.

As I have since moved on to a new job, I thought I should probably say a word or two about it.

Use Serverless instead.

I was introduced to the Serverless framework by a colleague a few months before I left my last job, and I was immediately impressed. It does everything I'd envisaged for Lambda Tools, plus a whole lot more, and furthermore it is actively being developed by a full-time team with contributions from the open source community. As well as supporting AWS, it also supports Azure, Google Cloud Platform, and a whole lot more. The fact that Serverless is a thing saved me masses and masses of work on a project that I was struggling to fit in round everything else.

I'm particularly impressed by the way that Serverless works. Rather than manipulating AWS resources independently, as Terraform does, it works by generating CloudFormation templates. This makes things massively more robust than trying to configure different resources independently of each other. Since CloudFormation is built into AWS itself, and everything it does is transactional, you're a lot less likely to end up with things getting out of sync with each other when making changes.

When I left the Parliamentary Digital Service, the WebOps team was still using Lambda Tools for most of their existing code, though I had started the transition to Serverless with a Continuous Delivery pipeline for one particular project. However I don't know what their plans are for it in the long run.

As for myself, I don't have any plans to develop Lambda Tools any further. We aren't using a serverless platform at my present job and I don't anticipate us doing so in the near future either. Even if we did, the fact that there is a mature and robust alternative means that I would be using Serverless rather than trying to carry on reinventing the wheel.

A note on performance reviews

In many organisations, you are required to complete some form of annual performance review process in which you agree some objectives with your line manager to be completed over the following twelve months. In the Parliamentary Digital Service, this was called the Individual Performance Review, or IPR.

For a while now, I've wanted to release an open source tool or library and build an online community around it. I've thrown a few things against the wall over the years, but nothing has ever stuck. But the Powers That Be thought that to do something like that would be good for recruitment, so I put it down as one of my IPR objectives for the 2017-2018 reporting year. Lambda Tools was the result.

On the face of it, it sounds like a good idea. You have a personal objective that is closely aligned with the objectives of your employer. Why not combine the two and pick up some brownie points for doing something that you're passionate about anyway?

Unfortunately, it didn't work out that way.

It was always viewed as a low priority by the rest of the team, who gave me little or no encouragement to keep working on it, and who weren't well placed to pitch in and help anyway because they were Ops engineers rather than developers. In theory, we were supposed to have "10% time" to work on projects such as this, but while many other teams made full use of their 10% time, on my team it simply didn't happen. As a result, I ended up doing most of my work on it on the train and in the evenings, just to have something to put down on my IPR form. It ended up feeling like a lead weight round my shoulders, and to then discover that something already existed that did everything I wanted it to do and more left me feeling thoroughly discouraged. I'm sure you would feel discouraged too if you'd discovered you'd spent a whole lot of your own time reinventing the wheel just so that you could tick a box on a form.

If there's one lesson I've learned, it is this: if you have to set performance objectives at work, stick to what you can deliver in your 90% time. Annual performance review processes are nothing more nor less than bureaucratic enterprisey box-checking exercises that simply do not deliver the benefits that they claim to offer. Their feedback loops are far too slow. They suck the life out of everything they touch, and if you let them get their grubby paws on your 10% time or your pet projects, they will suck the life out of that too. Keep the beast locked up in its cage. Don't let it rob you of your passion.