james mckay dot net

A couple of months ago, I started migrating my personal projects from Mercurial to Git. This was not because I now prefer Git (I don’t) but because I’d been swayed by the popularity of GitHub and was beginning to doubt whether Mercurial could sustain its momentum in the long term.

But reading the 501 Manifesto has made me think again about this.

Most of the reactions I’ve seen to the 501 Manifesto seem to have completely misunderstood it. It’s not about having a go at open source or blogging or anything, and it’s not about wilfully switching out of code mode the minute you leave the office: the message of the 501 Manifesto is simply that we favour competence and professionalism over passion.

This is something that needs to be said, and forcefully. There is a noisy group of prima donnas on Twitter (they prefer to think of themselves as “high end developers” or “ninjas”) who are loudly promoting the toxic message that passion and competence are one and the same, or at the very least, closely correlated. If you don’t blog, if you are not contributing to open source projects, if you are not attending conferences in your spare time, if you are not on GitHub, if you don’t view software development as a lifestyle, then according to them, that automatically makes you incompetent.

The author of the 501 Manifesto calls these people “toilet coders.” People who push code to GitHub even while sitting on the toilet.

Here’s the deal: if you are using Bitbucket or Google Code or Codeplex, and you’re happy with it, stick with it. While you may get more contributions on GitHub, and while most Git users are fine, if it’s beneath anyone’s dignity to use Mercurial to send you a pull request, that automatically makes them a Toilet Coder and your community is probably better off without them. Heck, they should be thankful that you’re open sourcing your code at all and not charging a small fortune for it. The only valid reason for choosing Git and GitHub over one of the alternatives is that you prefer it.

Oh, and forget about all this “GitHub is your resume” nonsense. For every recruiter that follows this line of thinking, there are dozens of others who don’t. Besides, if they aren’t prepared to accept an online portfolio on Bitbucket or Google Code or Codeplex instead, then similarly, they are Toilet Coders, and you are probably better off not having them as a boss if you value quality time with your family.

The fact of the matter is that what you do with your personal projects, and in your spare time, outside of work, is entirely up to you. Just because you don’t want to use GitHub doesn’t make you incompetent. Just because you don’t contribute to open source projects doesn’t make you incompetent. Just because you don’t have any personal coding projects at all doesn’t make you incompetent. Just because you don’t use Node.js and CoffeeScript and Ruby on Rails and CQRS and all the latest over-hyped this, that and the next thing doesn’t make you incompetent. Just because you use Windows rather than Linux doesn’t make you incompetent. Just because you prefer Entity Framework to NHibernate doesn’t make you incompetent. Just because you don’t use vim doesn’t make you incompetent. Switching out of code mode when you go home doesn’t make you incompetent. The only thing that makes you incompetent is not knowing, or not being able to apply correctly, the baseline skill set required by your team.

Don’t burn yourself out trying to impress people you don’t work for who don’t understand the difference between passion, competence, and GitHub.

Comments (3) »

Posted at 07:00 on 23 April 2012.
Tagged: open source

Github, so we are told, is the most popular source code hosting website on the planet. With nearly a million registered users and 2.5 million repositories, it is certainly impressive. But if you look at what kind of projects are actually hosted on Github, a certain picture emerges.

Take a look at its list of about eighty or so “interesting” repositories, for starters. There’s a lot of cool stuff there: Prototype, Scriptaculous, Ruby on Rails, MooTools, Groovy, Sinatra, Redis, Symfony, and so on. The list changes from time to time, and I have no idea what criteria are used to construct it, but at the time of writing, with only one exception  — phpBB — every single repository on the list is either a programming language, or a programming framework or library, or a programming tool.

You can see something similar — and quite surprising — when you look at the list of most popular languages on Github. The dominance of JavaScript, Ruby and Python is not all that surprising, but the fourth most popular language on Github is Shell. In other words, bash scripting — the Linux equivalent of DOS batch files or PowerShell.

C# — probably the most sought-after programming language by employers in the Real World — does not appear in the top ten. Clicking through to it from the right hand column shows that it is number twelve. But which is number eleven? After a minute or two of clicking on educated guesses then completely at random, I discovered, much to my surprise, that it is actually VimL.

I kid you not. Github apparently has more repositories dedicated to the scripting language for Vim — a console-based text editor mainly used by hard-core geeks — than to C#.

Why am I highlighting these things? Because they are indicative of a problem that seems to be plaguing the open source world and the programming blogosphere these days. There is far too much of a focus on processes and tools, and not enough of a focus on individuals and interactions. I’ve been getting too much inclined that way myself in the past couple of years here on my blog and on Twitter, and I’m getting weary of it.

You can understand why this would be. Programming languages, tools and frameworks are fun to work on because they’re hard and make you a better programmer, they scratch a personal itch, and they look good if you’re trying to attract the attention of trendy startups in Silicon Valley. And of course these things are important. But they are written by open source developers, for open source developers. They are of no direct interest whatsoever to non-developers.

Where are all the WordPress plugins and themes? The personal productivity software? Personal finance? Health and fitness? Photo and video editors? Education? Motoring? Foreign language learning? E-commerce? Geolocation? Astronomy? Twitter and Facebook clients? Games? Screensavers? Android and iPhone apps? Yes, they’re out there, but you have to hunt for them. And among all the blog posts about how CQRS, NoSQL and IOC containers are so cool, where are all the success stories telling us how they’re being used in the Real World?

Programming does not exist in a vacuum, folks. If we were serious about the Agile slogan, “Individuals and interactions over processes and tools,” projects aimed at non-developers would predominate. As it is, a lot of it seems like a case of programming for programming’s sake.

A lot of developers are introverts, and find it easier to spend time writing code than interacting with people. But if you want to produce something that’s really useful, you need to spend some time getting out from behind the computer, developing other hobbies and interests, and interacting with people. After all, that’s where the ideas for useful software come from in the first place.

Being a successful developer really does require you to put individuals and interactions over processes and tools.

Comments (5) »

Posted at 08:00 on 15 August 2011.
Tagged: open source, tools

I sent a pull request to the lead developer of an open source project that I’m starting to contribute to. This is the default that both github and bitbucket give you, and the first thing that newbie open source contributors on those sites will think of doing.

I got this response from him:

I’d *massively* prefer patch submission via patchbomb to the … list – that’s generally where they come in, and sometimes people other than me find potential problems in the patch. Can you do that?

D’oh! My bad! It’s the same kind of thing as the rule that you should ask the whole community, not just one of its members. Admittedly I had noticed the existence of the mailing list, but only after I sent off the pull request, so his response did not surprise me.

Moral of the story: if you want to contribute to an open source project, look for a mailing list before you do anything else. And if there is one, use it.

Comments closed »

Posted at 08:47 on 4 September 2010.
Tagged: open source

(Update: I’ve updated these instructions for Mercurial 1.6/TortoiseHg 1.1.)

I’m going to get really controversial here and say that I think Mercurial is better than git. My reasoning (as with the reasoning of everyone else who takes sides in this particular debate) is entirely subjective, so we won’t belabour the point here too much. Nevertheless, some of us do have a preference for one over the other, and many Subversion refugees like me who do most of their work in Windows tend to lean towards Mercurial.

But there’s no denying that github is fast becoming the Facebook of open source programming (albeit hopefully without the unethical bits, Farmville, and people tagging you in embarrassing photos for all and sundry to see), and if you want to strut your stuff as a developer, that’s the place to do it. Github is, of course, a hosting facility for git repositories, as one would expect of a site whose name says what it means and means what it says.

Fortunately, it is quite possible to use Mercurial as a client against github repositories via the hg-git extension, and you can pull and push from one to the other pretty much losslessly.

However, setting it all up on Windows is not entirely straightforward, and there doesn’t seem to be a decent guide to it anywhere on the Internet: most of the instructions that you read assume that you’re using either (a) Linux or a Mac, (b) the command line, or (c) both. You also have to figure it out from various places all over the web, and searches on Google and Stack Overflow proved to be surprisingly fruitless. Furthermore, the most comprehensive howto that I came across elsewhere contained several instructions that were just plain wrong.

So, after spending two solid evenings struggling against a myriad of error messages and cryptic dialog boxes, I finally managed to get it working, and for future reference (and anyone else who wants to know how), I’ve documented what I’ve found actually works for me as best I can.

1. Install TortoiseHg and hg-git.

Install TortoiseHg 1.1 or later. If you are using an earlier version, upgrade: these instructions may work if you don’t, but I can’t make any guarantees.

I downloaded hg-git by cloning the repository. You can get it from either github and Bitbucket. The advantage of cloning the repository is that you can upgrade to the latest version quickly and easily by hg pull then hg update, or use the graphical tools if you prefer. You can also easily switch between the bleeding edge version of the code and a stable release if you like.

hg clone http://bitbucket.org/durin42/hg-git c:\abc\mercurial\hg-git

I downloaded hg-git into the directory c:\abc\mercurial\hg-git. If you put it elsewhere in your filespace, alter these instructions to suit.

2. Update to the appropriate version of hg-git.

If you are using TortoiseHg 1.1, you will need to use hg-git 0.2.3. If you ignored my advice to upgrade, and are still using version 1.0, you will need to use hg-git 0.2.1. Don’t use version 0.2.2: it doesn’t work with either version of TortoiseHg.

The official hg-git documentation tells us that we also need to download and install Dulwich 0.4.0 or later. The latest version of hg-git requires Dulwich 0.6.0. In any case, Dulwich is included with TortoiseHg (version 0.6.0 with TortoiseHg 1.1; version 0.5.0 with TortoiseHg 1.0) so you don’t need to do anything else there. Open up the TortoiseHg repository explorer on your clone of hg-git, choose the “Tagged” radio button to show only tagged releases, and update to version 0.2.3:

3. Configure Mercurial to use hg-git and an appropriate SSH client.

To do this, you need to edit your mercurial.ini file. You can get to this simply by choosing “Global Settings” on the TortoiseHg context menu in Windows Explorer, and clicking “Edit file” to bring it up in Notepad. Add the following lines to your configuration file:

[extensions]
hggit = C:\abc\mercurial\hg-git\hggit

[ui]
ssh = "C:\Program Files\TortoiseHg\TortoisePlink.exe"

The [extensions] section loads hg-git into Mercurial; the ssh option in the [ui] section specifies an SSH command line client to use to communicate with github. TortoiseHg gives us TortoisePlink, which works fine for me.

4. Create a public key/private key pair.

There are some instructions on github on how to create a public key/private key pair. Unfortunately, these don’t tell you that key pairs come in two formats: OpenSSH (as used by git itself and github), and PuTTY (as used by Tortoise Everything).

A simpler approach is to download PuTTY (you can get it from here) and use PuTTYgen to generate your key pair:

Once you have generated your SSH key, copy and paste the “Public key for pasting into OpenSSH authorized_keys file” into github. Save your public key and private key to your hard disk somewhere.

5. Start Pageant

Pageant is a program that stores all your private keys in memory, where the SSH client used by Mercurial, that we configured above, can find them. It comes with both PuTTY and TortoiseHg. You can set it to load in your private key(s) when you log on to Windows by creating a new shortcut in the Startup folder of your Start menu with this command:

"C:\Program Files\TortoiseHg\Pageant.exe" "c:\abc\github.ppk"

Note that if you don’t start Pageant first and load in your private key, you will not be able to push to github.

6. Clone a repository and start pushing!

You should make sure that you get the format of your repository URL correct. It should be:

git+ssh://git@github.com/your-github-username/your-repo-name.git

The rest from there on is all plain sailing. All being well, you should now be able to pull from your github repository and push changes back up as if it were a Mercurial repository.

Things to check if it goes wrong.

Now all this is a bit of a fiddly process, there is plenty of room for error, and some of the error messages you are likely to get can be a little bit cryptic. However, most of it was due to me trying things that weren’t properly documented, and they all boiled down to a few things that you can check if you’ve followed the above instructions:

• Are you using the correct version of hg-git? While you can use versions later than 0.2.1, you need to use a later version of Dulwich than that which comes with TortoiseHg 1.0.
• The “ssh” option in your mercurial.ini file should only specify the name of the executable, without command line options. Some articles tell you that you can fill in the path to your private key in this option. Personally, I couldn’t get this to work, so I just stuck with Pageant.
• Is Pageant running?
• Is your private key loaded into Pageant?
• Do your public and private keys match?
• Is your private key saved in PuTTY format? If you generated your key pair using git, as per the instructions on github, it will be saved in OpenSSH format instead, and Pageant can’t handle that.¹
• Have you specified the URL to your github repository correctly? The version I gave above works, while missing out various parts of the URL (e.g. using “github.com” instead of “git@github.com“) doesn’t.

-----BEGIN RSA PRIVATE KEY-----
<transmission line noise>
-----END RSA PRIVATE KEY-----

PuTTY-User-Key-File-2: ssh-rsa
Encryption: none
Comment: imported-openssh-key
Public-Lines: 6
<transmission line noise>
Private-Lines: 14
<transmission line noise>
Private-MAC:
<transmission line noise>

Comments (4) »

Posted at 08:00 on 21 June 2010.
Tagged: open source, source control

Jeff Atwood wrote an interesting post the other day where he asked the question, Is Open Source Experience Overrated? He quoted an anonymous developer who had lamented not being able to find a job despite being the architect of a couple of open source projects:

One company seemed impressed with my enthusiasm for the job but it was part of their policy to provide coding tests. This seemed perfectly reasonable and I did it by using the first solution I thought about. When I got to the phone interview, the guy spent about five minutes telling me how inefficient my coding solution was and that they were not very impressed. Then I asked whether he had looked at the open source projects I mentioned. He said no – but it seems his impression was already set based on my performance in the coding test. The coding test did not indicate what criteria they were using for evaluation but my solution seemed to kill the interview.

I must confess that I was a little bit incredulous when I read this. It seems the correspondent was expecting the recruiter to consider his open source contributions as a substitute for his poor performance in the coding test — and it would be an irresponsible recruiter who did that.

The problem with open source contributions is that they don’t necessarily tell a recruiter what they need to know about you. They indicate passion and enthusiasm for programming, which is a plus, but they don’t necessarily indicate competence — and passion is no substitute for competence. In fact, some open source projects, such as this example cited by Ayende the other week, are very badly written and do not reflect well on the developer. And in the recruitment process, it’s competence that a responsible recruiter will be looking for first and foremost. That’s why it’s so important to have a coding test. The team needs to have a certain baseline standard, and that standard needs to be treated as a “must have” — failing the technical test should be an automatic “no hire,” regardless of passion or experience. Yes, they should take a look at your open source contributions, but they only really become relevant in the later stages of the recruitment process, once they have carried out the technical screening and are getting into the final round of interviews.

Here are some examples of things that a recruiter may want to know about your abilities that he or she is unlikely to find in a typical open source project:

• How quickly you can write code
• How quickly you can troubleshoot code
• How quickly you can learn new technologies
• The extent of your understanding of the company’s core technologies (how much you have committed to memory, and how much you need to refer to the documentation)
• How well you can understand and implement customer requirements
• How clean you can keep your code when working to tight deadlines
• Whether you can grok recursion
• Whether you understand Big O Notation and its implications for performance
• How well you can integrate with the rest of the team

Different companies also have different attitudes towards open source. The ones that are most likely to be interested will advertise job openings in the open source community itself — they reason that that way they have a generally richer pool of talent to draw on than the hordes of unqualified and mediocre people that you get in places like monster.com and recruitment agencies. Larger organisations, on the other hand, are only likely to take notice if your contributions are to a project that they’re actually using, such as NHibernate or Tomcat, and some companies actually have a prejudice against it, so it pays to do some research and bear in mind that YMMV.

One other thing. If you want to pitch your open source contributions to your prospective employer, you need to make sure that your code is the best quality that you can give it. You might find that they take it into account in ways that you hadn’t expected, and if you’re doing stupid things such as silently swallowing exceptions, or writing thousand-line, hundred-parameter, multi-responsibility, copy-and-paste methods, or giving them names like DoIt(), or if your code formatting is sloppy and difficult to follow with inconsistent indentation, it may work to your disadvantage if they ask a technical reviewer to look at it. Open source coding isn’t like coding for work or profit: you have no deadlines, so you can afford to give it a lot more tender loving care.

Comments closed »

Posted at 08:00 on 20 April 2009.
Tagged: open source, recruitment

If you’ve ever had anything to do with modern church music, chances are you’ll have come across an organisation called Christian Copyright Licensing International. Their website has the strap line “encouraging the spirit of worship” and the idea is that rather than paying royalties to individual songwriters and their agents, you just pay one licence fee and that lets you sing whatever you like as often as you like in your church for a whole year. It helps with administration and makes it easier for your church to operate in righteousness, so it saves some time and hassle, though maybe not money. It’s a vast improvement over what we had in the early 80s with songbooks like this one that had a dozen or so entries that said “This song has been omitted for copyright reasons.”

However, it only covers church services, so if you are organising evangelistic events, or conventions like Faith Camp, or making your own worship album, or streaming your meetings live over the Internet, or making a mashup for something or other, or even playing tracks from your favourite Christian albums in a coffee shop, you need to go through the rigmarole of getting whatever other additional licences you need. And of course, all this costs more in terms of both money and time, and what might otherwise only take a couple of days can end up taking several weeks or even months while you’re waiting for permission to come through — if it comes through at all.

Now compare this “Christian” approach to copyright with the concepts that developers and geeks have come up with. I am talking, of course, about open source and Creative Commons.

If you’ve never heard of Creative Commons, you may want to take a look at this video, which explains it very simply and clearly:

The idea is for copyright owners to allow greater freedom and flexibility in what is done with their own intellectual property. Take my blog for example. I could put a notice on it saying you’re not allowed to copy it without paying me a fat fee, period, but I have deliberately chosen not to do so. Instead, I’ve released it under a licence that lets you reproduce it wherever you like as long as you aren’t doing so for profit, you acknowledge me as the original author, and if you make a derivative work, you grant others the same rights. You don’t even have to ask me — though it would of course be nice to know. The Creative Commons website allows you to choose a licence tailored to your needs from several different options.

The entire concept could have been lifted straight out of the New Testament, yet Christianity has had little involvement in it. It is a practical outworking of Jesus’ words, “Freely you have received, freely give” — indeed, in recent years, Bram Cohen, who is pretty much a poster child of the whole free content movement, made “Give and ye shall receive” the slogan for Bittorrent. It is a slight rewording of Luke 6:38.

Or what about Paul’s words in 2 Corinthians 2:17? “Unlike so many, we do not peddle the word of God for profit. On the contrary, in Christ we speak before God with sincerity, like men sent from God.”

So where on earth is the Body of Christ in all of this? Why are we dragging our heels when we should be forging ahead?

Worship leaders, church musicians and Christian authors have a lot in common with software developers such as myself. We tend to be very creative individuals, and what we do is often very much a labour of love. We write songs, books, blogs or computer code even if we’re not getting paid for it, and while it is nice to earn something from it, that is only a secondary consideration.

Yet while there are some people producing resources such as books, Bible studies and worship songs who have taken the concept of Creative Commons on board, they are very much on the fringes. Most, if not all, widely used Christian resources — including most modern translations of the Bible and nearly all songs that have a circulation beyond the songwriter’s home church — are only made available under restrictive commercial licences.

Is this encouraging the spirit of worship, or the spirit of mammon?

I would love to see some notable Christian songwriters distributing their compositions under licences similar to Creative Commons. I would love to see major ministries jumping on board, open sourcing their Bible study resources, and actively encouraging others to do the same.

I simply can’t accept the excuses that “it can’t be done” or “it’s impractical” or “worship leaders have to make money somehow.” The whole open source movement blows these claims completely out of the water. Some open source software packages have taken far longer to write than all the time that Graham Kendrick, Martin Smith, Tim Hughes, Matt Redman and the entire Hillsongs crowd have spent on all their songs put together — yet they are still made available for free, despite being mature and stable enough to power business critical servers. If software developers can do it, why can’t the Church?

Comments closed »

Posted at 14:00 on 4 June 2008.
Tagged: faith, open source