Posts Tagged ‘django’

Database structure migration without code versioning

Thursday, July 30th, 2009

This is not intended to be a rant or even close to it. But I find myself with a problem so obvious that I’m annoyed by the lack of solution. I can’t be the only developer with this problem.

I use South to handle database structure migration and it works (for the most part) very well, and I’m happy to use it. One problem though. When you actually need to migrate backward and forward many times you will run into problems with missing fields in your Django models. Say, so have a field called “person” on a random model. You wish to rename this to “full_name”, so you write the straight forward migration to handle this and remove the field from your model. Now, if you wish to migrate back your code is outdated. This mean that you’ll be unable to use your Django system to inspect things or whatever you might wish to do.

The solution – from my point of view – for all of us using some versioning tool would be to bind the migration to a changeset and have the forward and backward migrations update the code as well. This is a bit more demanding to set up, but it’s properly the only solution you have if you want the possibility to migrate back and have a useable system.

If only I had the time to take a proper look at South to find the solution.

The status of Django 1.1

Monday, July 6th, 2009

On May 7 Jacob Kaplan-Moss wrote a blog post on djangoproject.com about the status of Django 1.1. One of the things you really like as a web developer using Django is:

We’ll ship Django 1.1 when it’s stable, and not a moment before.

This really makes sense and we love the django developers for it. But right after he writes:

Thanks for your understanding, and watch this space for updates.

There haven’t been any updates yet. And at that time – according to the blog post it self – there were about 50 open bugs. Now there is only one. Why in gods names haven’t there been any updates? We’re a lot of people waiting for the release and we understand why we’re waiting. At least give us something to read while we’re waiting.

Update: I regret to say it happening again. 8 days ago the 1.1 release candidate was born, and the promise was that we would see the final release if no show-stopping bugs where found in the mean time. As far as I can see in the issue tracker, no such bug has been found. But still no release, and more importantly, no blog post or other communication explaining why. I know it’s only one day overdue, but why not just make a blog post, a tweet.. something about the release status.

Update II: Fantastic. Django 1.1 was just released. Hopefully we’ll see regular updates in the future from the team.

A generic copy/clone action for django 1.1

Saturday, May 23rd, 2009

The last couple of days I’ve been working on customizing the django 1.1 administration to handle a number of specific permission and functionality wise things. Tonight I wrote a little function that should make your life much easier in the administration. I find, when I do administrative stuff, that it’s easier to copy something existing and work on that as a base. This is also true for my customers so why not make it possible to do so in the built in administration.

The idea is to write an admin action that takes a number of objects and clone them. The action function can be defined as:

#!/usr/bin/python/

from django.db.models.fields import CharField
def clone_objects(objects):
    def clone(from_object):
        args = dict([(fld.name, getattr(from_object, fld.name))
                for fld in from_object._meta.fields
                        if fld is not from_object._meta.pk]);

        return from_object.__class__.objects.create(**args)

    if not hasattr(objects,'__iter__'):
       objects = [ objects ]

    # We always have the objects in a list now
    objs = []
    for object in objects:
        obj = clone(object)
        obj.save()
        objs.append(obj)

def action_clone(modeladmin, request, queryset):
    objs = clone_objects(queryset)
action_clone.short_description = "Copy the selected objects"

It’s not that much code. And I made the basic clone_objects function so it can also take a single object if you want to use it somewhere else. Now you can just put the action in the ModelAdmins you’ll like or enable it site-wide.

The next thing is to make it possible to recognize the objects you have just copied. The method presented below is not very elegant but it works for most cases. I do it by looking for CharFields that have a specific name, like “title” or “name”:

#!/usr/bin/python/

from django.db.models.fields import CharField
def clone_objects(objects, title_fieldnames):
    def clone(from_object, title_fieldnames):
        args = dict([(fld.name, getattr(from_object, fld.name))
                for fld in from_object._meta.fields
                        if fld is not from_object._meta.pk]);

        for field in from_object._meta.fields:
            if field.name in title_fieldnames:
                if isinstance(field, CharField):
                    args[field.name] = getattr(from_object, field.name) + " (copy) "

        return from_object.__class__.objects.create(**args)

    if not hasattr(objects,'__iter__'):
       objects = [ objects ]

    # We always have the objects in a list now
    objs = []
    for object in objects:
        obj = clone(object, title_fieldnames)
        obj.save()
        objs.append(obj)

def action_clone(modeladmin, request, queryset):
    objs = clone_objects(queryset, ("name", "title"))
action_clone.short_description = "Copy the selected objects"

So now, if you clone a object with “Look ponies” as the title, the cloned objects will have “Look ponies (copy)” as it’s title. Here are some screen shots to show it in action:

Object list
Action selection
Copied object

Another thing you could do is to create a column in the admin list pages with a copy link that would copy-and-edit the object. This should fit a normal use case where you want to edit the object and actually differentiate the new object from the old one. It should be pretty basic to write, but it includes a view function so I won’t include it here. If you need such a link on a lot of models consider using an BaseAdmin class that all your admin classes inherit from. This way to can write the view function and link function there and have it automatic on all your admins.

Now this code has a number of drawbacks, and maybe more than I have just listed here:

  • The actual clone functionality will fail if the object you’re cloning don’t have an AutoField as it’s primary key
  • The “(copy)” append functionality is very limited and you’ll properly need change the tuple manually for every model you use unless you’re very consistent with your field naming
  • The clone functionality will properly fail hard if some of the fields are unique. The method could be made more robust by adding stuff to unique fields

Danger with django ModelForms and related names

Thursday, May 21st, 2009

There is a danger in extending your ModelForms with fields if you don’t take care of your naming. Take for example these models. The forms represents a simple batch emailing module.

#!/usr/bin/python/

class EmailBatch(models.Model):
    body = models.TextField()
    subject = models.CharField(max_length=400)

class Email(models.Model):
    batch = models.ForeignKey(EmailBatch, null=False, related_name="emails")
    recipient = models.EmailField()

I wrote a simple form field called MultipleEmailField so the users can input several email addresses in a simple textarea. The resulting form is something like this:

#!/usr/bin/python/

class SendingForm(forms.ModelForm):
    emails = MultiEmailField(required=True)

    class EmailBatch:
        model = Delivery
        fields = ('body', 'subject', )

This appears to be alright, but isn’t. If you use this form you’ll get an AttributeError. The problem is in handling the “emails” field on the form. It’s not bound to any model, but the model actually contains something called emails (it’s set in the related_name part of the Email model). This can take some time to find :(

The bug is already fixed in django trunk, but for those of us trying to use a stable version we’ll just have to wait a little longer.

A simple fix is just to rename your field or related name

Howto get custom GET variables into djangos admin list pages

Tuesday, May 12th, 2009

Yesterday I ranted a bit about how to insert extra GET parameters in the django change list page without having the admin blow up when it was not able to recognise them as filters. I’m using this for building a custom menu that uses GET parameters to fold / unfold a submenu. The menu is context aware so I can’t do this by looking up the path, ie. several submenu points can like to /admin/auth/user/, usually just with different filter parameters.

I haven’t been able to find a nice solution for this, but here is a dirty one. As always I find myself using middlewares when something takes a nasty turn. The idea is to put the information in GET parameters, but use a middleware to actually find the special parameters and insert them on the request object. A context processor can then later give template the variables. I’m controlling which parameters we should fetch with a simple settings tuple:

#!/usr/bin/python/
ADMIN_GET_VARS = (
    ('submenu', None),
)

In my case I have only defined the submenu parameter, but you could take as many as you’ll like. The second element in the tuple (The None after “submenu”) is a default value that will be used if the GET parameter is not present. The middleware looks like this:

#!/usr/bin/python/

from django.conf import settings

class ForwardGetMiddleware:
    def process_request(self, request):
        new_get = request.GET.copy()
        custom_vars = {}
        if hasattr(settings, "ADMIN_GET_VARS"):
            for name, default in settings.ADMIN_GET_VARS:
                custom_vars[name] = new_get.get(name, default)
                try:
                    del new_get[name]
                except KeyError:
                    pass
        request.ADMIN_GET_VARS = custom_vars
        request.GET = new_get

Here is something important. We need to remove the get parameter from the request.GET as it will otherwise make the admin blow up. It’s therefore important that you don’t use parameter names that will clash with the admin. I’m considering introducing a prefix on every parameter to leave the admin alone. The context processor is very simple due to the way we build the middleware:

#!/usr/bin/python/

def admin_get_vars(request):
    return getattr(request, "ADMIN_GET_VARS", {} )

So there it is in a very basic stage. It’s possible to introduce more advanced settings to enable finer control with when this functionality kicks in, but it’s already a hack, so I don’t think there is any need for it. Also, one might consider trying to limit this to specific urls to improve performance and limit the number of side effects this might have.