james mckay dot net

There’s a fairly long-standing bug in Django where initial SQL files break if they contain a double hyphen in a quoted string when you run manage.py syncdb. This bug also causes problems if you’re trying to create stored procedures in MySQL and need to change the delimiter to allow multiple statements.

Seems the problem is some overly naive code to split the SQL file into individual statements. Unfortunately there’s no easy fix without at least partially reinventing the SQL parser or spawning a separate mysql command line client process. Best approach in the meantime is to avoid using stored procedures and check any scripts that you do have using Django’s unit test framework.

Comments closed (1) »

Posted at 08:00 on 15 May 2009.
Tagged: Django, Python

Over the weekend I was doing a bit more Django development, and I ran into a rather strange problem. I had written a custom command for manage.py whose last task was to update some rows in a couple of database tables using some raw SQL commands. Strangely, the changes weren’t showing up when I looked at the data in MySQL Query Browser afterwards. Even more strangely, when I copied and pasted the raw SQL and ran it in Query Browser, it worked fine.

A little bit of extra code revealed that the SQL commands were being run, which absolutely screamed that the code was being run in a transaction that wasn’t getting committed. Fixing it was simply a case of adding these two lines at the end of my custom command:

from django.db import transaction
transaction.commit_unless_managed()

I’m not sure whether this is a bug or a feature. Django’s default transaction behaviour — in HTTP requests at any rate — is the autocommit model: every change you make is automatically committed as soon as it is run. I’d have thought it would be the same in management commands, but clearly it is doing something rather different here. If anyone can enlighten me, I’d be interested to know.

Comments closed »

Posted at 08:00 on 30 March 2009.
Tagged: Django, Python

I came across this error this morning on a Django project that I’m working on in my spare time. I was trying to add some custom permissions to one of the models, as outlined in the documentation, only to have it choke with this message when I ran manage.py syncdb.

A bit of Googling got me to this site, but unfortunately it’s in Polish. Never mind, Google Translate managed to give me enough information. Turns out that I had missed out a comma that I didn’t think was necessary:

class Meta:
  permissions = (
    ("can_drive", "Can drive"),
    ("can_vote", "Can vote in elections"),
    ("can_drink", "Can drink alcohol"),   # <== This comma is important!
  )

Hmmm, I always thought that last comma in Python was only an optional extra in case you decided to add an extra item to the end of a tuple and forgot to add it in to the previous line. Seems it does have some obscure but significant semantic importance after all.

Comments closed (3) »

Posted at 15:01 on 27 December 2008.
Tagged: Django, Python

I’ve been fiddling with Django a bit in my spare time recently, with a view to possibly using it for some side projects. It’s the Python web application framework, similar in many respects to Ruby on Rails but with a distinctive flavour of its own.

It looks fairly promising. I particularly like its built in administration interface — it’s better than Rails’s scaffolding, giving you an integrated interface to pretty much all the models that are mapped to the database straight out of the box. It also seems more flexible than Rails — there are a few decisions that it leaves up to you, such as which JavaScript framework to use, or how exactly to structure your application.

I first started learning Python about three years ago when I used it to write some maintenance scripts for the Kingdom Faith podcasts, and since then I’ve used it for various random odds and ends (our daily build script is written in Python for instance) but nothing majorly extensive. As far as scripting languages go it has a lot to offer — it is relatively fast, has pretty good Unicode support, and fully supports procedural, object oriented, aspect oriented and functional programming paradigms, though it could do with better support for multi-statement lambdas like in JavaScript, Ruby and C#. I also like the way it forces you to write code that is clean and easy to read and follow, by making indentation syntactically significant and by making package and module names follow filesystem names. It is also fairly well documented, though the index in the .chm help file on Windows is a bit quirky at times.

Another good thing about Python is that it is very much an all-rounder: like .net or Java, you can write web applications, console applications, services, GUI applications, maintenance scripts or whatever else takes your fancy with it. This is in contrast to PHP and Ruby, which tend to be dominated to a much greater extent by web development, though they are capable of being used for other things. Python also has fairly mature support in the .net ecosystem — IronPython is the most mature of the DLR languages, so integrating Django with .net framework code is a real possibility. It also seems to have a much smarter contingent of users on average than either .net, Java or PHP. I think this is because it is only infrequently the first programming language that people learn, and most Python developers already have quite a bit of experience with two or even three other languages.

Comments closed »

Posted at 00:48 on 30 June 2008.
Tagged: Django, Python

If I can give one single piece of advice to ASP.NET developers anywhere, it will be this:

Learn another web development environment.

I really can not emphasise this strongly enough. From what I’ve observed, developers who only work with ASP.NET seem to have quite a bit of difficulty thinking outside of the Microsoft box. I am frequently confronted with indiscriminate and even inappropriate use of aspects of the .NET framework that don’t scale, such as DataSets, view state, or drag-and-drop programming. There’s nothing wrong with all these per se, but one of the most important things you need to know about how to use them is when not to use them. When all you have is a hammer, everything starts to look like a nail.

The ASP.NET Web Forms model in particular was originally designed to make web development look like Windows development, and ease the transition for VB6 developers from programming for rich Windows clients to the web. The result of this is that it has made the easy aspects of web development almost brain dead, while introducing a horrendously leaky abstraction layer that makes the hard things even harder, with masses of gotchas and pitfalls to trip you up if you venture outside it.

Languages such as PHP, Ruby on Rails or Python don’t have the same leaky abstractions, so developers tend to not only program “closer to the metal” but to think closer to the metal as well. This is why most of the cool sites, with stunning Ajax effects, tend to be written in these languages and target these platforms, while ASP.NET is largely languishing in the enterprisey world of Dilbert-esque cubicle farms.

I recommend you choose your alternative carefully, however. Rails and Python are the best choices. They will teach you patterns, practices, conventions, O/R mapping, MVC, and all round agile and pragmatic programming, and they tend to be taken up by smart and experienced developers who know what they’re doing. I have mixed feelings about Java: while you can learn a lot from it, like .NET it is very enterprisey, and at a time when everyone is getting excited about dynamic languages, Java is heading in completely the opposite direction. And I certainly don’t recommend PHP as a learning exercise: it is a beginners’ language — and a mind-bogglingly badly designed one at that — and while PHP guys are generally pretty enthusiastic and some of them are quite smart, and there are some decent PHP frameworks such as CakePHP and Symfony, the overwhelming majority of the PHP community simply don’t have what it takes to be programmers. Having said that, you need to know it, simply because it’s so pervasive.

You should also learn Linux if you can. It will teach you about modular design and the value of scripting everything that can be scripted. This is right at the heart of why Unix is Unix: a large part of its philosophy involves chaining text-based programs where the output of one can be passed as the input to another, to produce some fairly powerful command-based functionality, and scripting repetitive tasks so that their outcomes can be reliably reproduced. These are philosophies that seem largely lost in the world of Windows, which relies much more heavily on the visual, drag, drop and click approach of dialog boxes and wizards, even though they are every bit as essential if you want to have robust procedures and practices in place.

And whichever platform you take on board, you simply must familiarise yourself thoroughly with CSS, DHTML, JavaScript and Ajax, and at least one JavaScript framework such as Prototype or jQuery.

Personally, I still think that ASP.NET is technically the best platform on which to develop scalable, high performance, reliable web applications. However, in order to make the most of it, you need to have a good feel for what approaches you can import and learn from other platforms. Otherwise you will be stuck with the limitations and leaky abstractions of Web Forms.

Comments closed »

Posted at 14:00 on 6 June 2008.
Tagged: .NET, best practices, Java, PHP, programming, Python, recruitment, Ruby

When I’m learning new programming languages, concepts and things, I like to have the documentation available for download as a self-contained unit that I can refer to on my computer, rather than having to connect to the Internet to browse through it all online.

There are various reasons for this. The first is speed: since many APIs are overwhelmingly humungous these days, a quick, easy to use search facility and straightforward overviews and walkthroughs are a must. A .chm file is the best option — you can search it quickly and easily, and it usually doesn’t take more than a few seconds to find what you’re looking for. Individual HTML pages are not quite so good though, since searching is less straightforward. And accessing something on your hard disk is much faster than surfing the web, due to network latency and other similar factors.

Secondly, I prefer to spend the bulk of my time offline to help eliminate distractions. I’m not saying you should work offline exclusively: I have found a lot of help from online resources such as the ASP.NET forums, experts’ blogs, tutorials and the like. However, I do like to be disconnected from the Internet when I’m trying to work my way through some tricky problems. It helps to reduce distractions such as those Wikipedia loops that it’s so easy to get into — you know the kind of thing I mean, where you end up clicking on one interesting looking link after another for a while and suddenly realise that you’ve spent two hours reading a whole lot of total drivel and completely forgotten what you went online for in the first place.

Furthermore, at present at least, we still only have dial-up connectivity to the Internet at home rather than broadband, so going online is slow and cumbersome, though hopefully this will all change after we move house at the end of this month.

A lot of programming languages and platforms do quite well in this respect. The .net framework is a good example: the SDK documentation comes with examples, quickstarts, howtos and API specifications that you can install on your computer to refer to as and when you need it. PHP and Python are also pretty good, with comprehensive, searchable, easy to follow documentation available for download in CHM format.

Java is a bit more awkward. You can download the J2SE and the J2EE documentation, but they are a bit more fragmented, and contain a lot of links back to the web — in particular, from J2EE stuff to J2SE stuff, which is a little bit annoying. I’ve also found Ruby on Rails pretty frustrating. It would be great to have one download that covers both Rails and the Ruby core API, rather than having to go online all over the place, but there isn’t one. You can get a number of CHM files, but they seem to be strewn all over the place and don’t cover everything. This is a real shame, because I would love to be able to get my teeth into Rails in my spare time, and it seems that you can’t do that quite so effectively offline.

Still, at least the documentation is comprehensible and reasonably structured, which is a lot more than can be said for Perl. I really, really hate the Perl documentation. Look at the index, for starters — a list of cryptic, obfuscated and sometimes downright misleading names such as perlboot, perltoot, perllol, perlre, perlrun, perlpod. I can just about make sense of it, but it takes twice as long to find what you’re looking for as anywhere else. To a beginner, it is probably the most confusing, incomprehensible and intimidating mess you are ever likely to encounter.

Comments closed (2) »

Posted at 21:00 on 11 August 2006.
Tagged: .NET, documentation, Java, Perl, PHP, Python, Ruby