Django: Taking note of ManyToManyField changes

I’ve gone through a bit of a mess lately, trying to do something I considered very possible, only to ultimately fail to find a working solution. This write-up will hopefully save someone else several hours of effort, asking for help, waiting, etc.

Those reading the article, please be sure to also read any comments to the article, as others may have additional ideas, solutions, or helpful information.

As you can see from the documentation links in this post, I am working with version 1.3. I am not aware of any differences in 1.3.1 or 1.4.0 regarding the topic covered in this post.

Scenario

You have a Django model. That model has a field which is a ManyToManyField. When an instance of your model is modified via any mechanism (custom web form, Django admin interface, code using the ORM, etc), you want to execute some custom code based on what happened to the model instance’s fields (including the ManyToManyField).

Specifically, what if you need to keep a non-Django source in sync with a Django model’s data? In my case, I have a “simple” need to push changes to a Django model’s data to an LDAP server.

class Interface(models.Model):
    fqdn = models.CharField(primary=True,
                            verbose_name="Fully Qualified Domain Name",
                            max_length=80)

class Netgroup(models.Model):
    name = models.CharField(primary=True,
                            max_length=80)

    interfaces = models.ManyToManyField(model=Interface,
                                        null=True,
                                        blank=True)   

Did NetgroupInstance.interfaces change? What changed? Was 1 interface removed from 500? What was that 1 interface? Were 9 interfaces added to the existing 15 interfaces? What were the 9 interfaces? Was this a brand new Netgroup creation with interfaces added at creation-time? What were the interfaces?

Solution 1?

Just override the Netgroup save() method! Call the superclass’s save first, then run your custom code against the saved data.

FAILS: The ManyToManyField data is not reliable at this point in the code flow for some reason. See the first few posts in my thread here.

Solution 2?

Maybe try Django’s signals? Register your custom code as a callback function for the Django post_save signal?

FAILS: Does not work properly in the ‘Admin’ interface at a minimum. Creating a new Netgroup with some Interface relations selected does not show the Interface relations inside the post_save callback function. Another similar case to above. See the final post here (as of 1/18/2012).

Solution 3?

Try solution 2 in combination with another signal callback function for the m2m_changed Django signal. This was recommended to me in #django-users IRC as the way to do it.

FAILS: Does not work properly in the ‘Admin’ interface. Removing Interface relationships does not present itself as data associated with the pre_remove and/or post_remove “actions”. Instead, it appears the ‘Admin’ interface clears all interfaces from the field and then adds the correct interfaces back in. See bug 16073, bug 14482, and this thread.

Sad Conclusion

If there wasn’t a bug, we would obviously use solution 3, even though I find it goofy to have to catch an extra signal (m2m_changed) when a model happens to have a ManyToManyField on it.

As it is now, we just use “Solution 2” and tell users of the ‘Admin’ interface, via extra help_text on the Netgroup.name field, to enter the name, save, then add items to the Netgroup instance.

class Interface(models.Model):
    fqdn = models.CharField(primary=True,
                            verbose_name="Fully Qualified Domain Name",
                            max_length=80)

class Netgroup(models.Model):
    name = models.CharField(primary=True,
                            max_length=80,
                            help_text="WARNING: save new netgroups with just the name, then add interfaces and save again.  Sorry!")

    interfaces = models.ManyToManyField(model=Interface,
                                        null=True,
                                        blank=True)

from django.db.models.signals import post_delete, post_save, m2m_changed

def netgroup_delete(sender, **kwargs):
    pass # DO STUFF HERE

def netgroup_save(sender, **kwargs):
    pass # DO STUFF HERE

post_delete.connect(netgroup_delete, sender=Netgroup,
                    dispatch_uid="netgroup_delete")
post_save.connect(netgroup_save, sender=Netgroup,
                  dispatch_uid="netgroup_save")