Posts tagged with Python

Foreign key to Django CMS page ...

Published at July 14, 2011 | Tagged with: Python , Django , django-forms , django-cms , django-admin , django-models , usability , Development , Django CMS

... how to make usable drop-downs with Django CMS pages

Problem: some times when you create custom applications or plugins for Django CMS you need a property that connects the current item to a page in the CMS. Nothing simple than this - you just add a ForeignKey in your model that points to the Page model and everything is (almost)fine. Example:

from cms.models import Page

class MyModel(models.Model):
    # some model attributes here
    page = models.ForeignKey(Page)

If you registered your model in Django admin or just add a model form to it you will see something like this:

Django Admin Screenshot

Cool right? Not exactly. The problem is that these pages are in hierarchical structure and listing them in a flat list may be/is little confusing. So let's indent them accordingly to their level in the hierarchy. Solution: The easies way to achieve this indentation is to overwrite the choices list of the ForeignKey field in the ModelForm __init__ method.

class MyModelForm(forms.ModelForm):
    class Meta:
        model = MyModel
    
    def __init__(self, *args, **kwargs):
        super(MyModelForm, self).__init__(*args, **kwargs)
        choices = [self.fields['page'].choices.__iter__().next()]
        for page in self.fields['page'].queryset:
            choices.append(
                (page.id, ''.join(['-'*page.level, page.__unicode__()]))
            )
        self.fields['page'].choices = choices

The magic lies between lines 7 and 11, on line 7 we create a list with one element the default empty option for the drop down. The need to use "__iter__().next()" comes from the fact that the choices attribute of the fields is django.forms.models.ModelChoiceIterator object which is iterable, but not indexable i.e. you can not just use self.fields['url'].choices[0].
After we had the empty choice now it is time to add the real ones, so we iterate over the queryset(8th line) that holds them and for each item we add a tuple to our choices list(10). The first item of the tuple is the page id - nothing special, but the second one... here the python magic comes. We multiple the minus sign('-') by the page level and join the result with the page title. The only thing left is to replace the field choices(line 12) and here is the result:

Django Admin - usable drop-down

Final words: For me this is much more usable than the flat list. Of course you can modify the queryset to return only published pages or filter the results in other way and still use the identation code from above.
I'll be happy to hear your thoughts on this.

Caching websites with Django and Memcached...

Published at July 12, 2011 | Tagged with: Python , speed , semantic web , Django , cache , Development , Technologies , Web

... memcached installation, django configuration and how to use it for your website

After Caching web sites and web applications and When to use caching for web sites its time for a little sample. This sample shows the usage of Memcached for server-side application cache. The installation part is taken from my Ubuntu so it may differ depending from your OS/distribution.

What is Memcached: Memcached is a tool that allows you to store key-value pairs in you memory. The keys are limited to 250 Bytes and for better performance the value size is limited to 1MB(more details) but this size is fair enough for web usage.

Memcached installation:

apt-get install memcached
apt-get install python-memcache

The first line installs Memcached and the second one install Python API for communication between your application and Memcached daemon. After this the Memcached daemon is up and running. With default configuration it runs on port 11211 on localhost(127.0.0.1). If you want to modify this the configuration file(in my case) is situated in /etc/memcached.conf Django configuration: This one depends from the Django version that you use. For 1.2.5 and prior the next code should by added in your settings file(settings.py):

CACHE_BACKEND = 'memcached://127.0.0.1:11211/'

For 1.3 and development version add:

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
        'LOCATION': '127.0.0.1:11211',
    }
}

In both cases if you use different port and/or IP you have to replace them above. More info about cache backend configuration you can find in Django documentation docs. So now you have Memcached running and Django configured. If you have doubts about is this suitable/usable in you case take a look at the posts mentioned above or just add comment with your case and I will be happy to give you an advice. Now it is time to start using it. Cache usage(part I) - how to cache on Python level: If you have some heavy calculations in your view you can cache the result from this and use the calculated one to lower the load. Example:

from django.core.cache import cache

def heavy_view(request):
    cache_key = 'my_heavy_view_cache_key'
    cache_time = 1800 # time to live in seconds
    result = cache.get(cache_key)
    if not result:
        result = # some calculations here
        cache.set(cache_key, result, cache_time)
    return result

The process is simple, you ask the cache for a value corresponding to a given key(line 4). If the result is None you execute the code that generates it(line 8 ) and store it in the cache(line 9). My advice is to declare the key and time as variables cause this will ease their future changes. Cache usage(part II) - how to cache on template level: This is suitable for the cases when you have some heavy processing in the template(as regroup) or you want to cache only part of the template(as latest news section). Example:

{% load cache %}
 ... non cached content here ...
{% cache 1800 latest_news %}
    ... here are latest news - cached ...
{% endcache %}

The basic usage usage is {% cache time_in_seconds key %} ... {% endcache %} You can also cache code fragments based on dynamic properties, for example - current user recent conversations, just pass a 3rd param the uniquely identifies the code to be cached.

{% load cache %}
{% cache 300 recent_conversations request.user.id %}
    ... current user recent conversations - cached ...
{% endcache %}

Final words: as you see from the examples above using Django and Memcached is really easy. Using it correctly will speed up your website and respectively improve your user experience(UX) and SEO. Using it wrong will provide negative results. Just take a moment and think what can be cached, how long can it be cached and is there a reason to be cached. Try to avoid double caching - there is no need to use caching in templates and then cache the rendered template in the view too.

Django models ForeignKey and custom admin filters...

Published at June 1, 2011 | Tagged with: Python , Django , django-forms , dynamic values , django-admin , django-models , ForeignKey , Development

... or how to limit dynamic foreign key choices in both edit drop-down and admin filter.

The problem: Having a foreign key between models in django is really simple. For example:

class Question(models.Model):
    # some fields here

class Answer(models.Model):
    # some fields here
    question = models.ForeignKey(Question)

Unfortunately in the real live the choices allowed for the connection are frequently limited by some application logic e.g. you may add answers only to questions created by you. In some simple(rare) cases this can be easy achieved using choices or limit_choices_to attributes in the models.ForeignKey call. In the case of choices you just have to pass list/tuple each element of which contains the value to be stored and human readable name for the choice. Unfortunately this one is computed on server run and is not updated with new items during run-time. If you use limit_choices_to you may pass to it some kind of filter expression e.g. limit_choices_to = {'pub_date__lte': datetime.now} but this not always can do the job. Speciality: So if you want dynamic choices in the admin drop down you have to write a method that will return list with options and bind it to the form(as shown in Django forms ChoiceField with dynamic values…) which is used by admin. This will work great but if you decided to add this column in admin`s list_filter you will see all element from the connected model in filter. How to limit them to the same list used for the form choices? Solution: The simplest solutions is to extend RelatedFilterSpec, overwrite its default choices and add a single row to the model:

from django.contrib.admin.filterspecs import FilterSpec, RelatedFilterSpec

class CustomFilterSpec(RelatedFilterSpec):
    def __init__(self, *args, **kwargs):
        super(CustomFilterSpec, self).__init__(*args, **kwargs)        
        self.lookup_choices = get_questions() #this method returns the dynamic list

FilterSpec.filter_specs.insert(0, (lambda f: bool(f.rel and hasattr(f, 'custom_filter_spec')), CustomFilterSpec))

class Answer(models.Model):
    # some fields here
    question = models.ForeignKey(Question)
    question.custom_filter_spec = True # this is used to identify the fields which use the custom filter

Final word: The solutions is pretty simple and really easy to implement but should be used carefully. If your filtering method(in my case get_questions) is slow/resource consuming it may bring you troubles. Here is the place where and you should think about caching it. This is a place where a application cache can be used. Hope this will help you as much as it helped me.

Django forms ChoiceField with dynamic values...

Published at Dec. 5, 2010 | Tagged with: Python , Django , django-forms , dynamic values , ChoiceField

... or how to get a dynamic drop-down list with Django forms

The problem: Lets say that you need a form with a drop-down list that have dynamic values. With Django this can be done simple and fast, but if you are new you may get yourself into a trap. In a standard form(with static values in the drop-down) your code will be something like this:

MY_CHOICES = (
    ('1', 'Option 1'),
    ('2', 'Option 2'),
    ('3', 'Option 3'),
)
    
class MyForm(forms.Form):
    my_choice_field = forms.ChoiceField(choices=MY_CHOICES)

So if you want the values to be dynamic(or dependent of some logic) you can simply modify your code to something like this:

def get_my_choices():
    # you place some logic here
    return choices_list

class MyForm(forms.Form):
    my_choice_field = forms.ChoiceField(choices=get_my_choices())

and here you will fail(not absolutely). This is a common mistake and sooner or later you will see what you messed but it my be too late. Speciality: the trick is that in this case my_choice_field choices are initialized on server (re)start. Or in other words once you run the server the choices are loaded(calculated) and they will not change until next (re)start. This mean that your logic will be executed only once. This will be OK if your logic is server specific(depends from server settings) or "semi-dynamic" in any other way. But you need this list to be updated on every form load you will need to use something the form __init__ method. Solution: fortunately the form`s class has an __init__ method that is called on every form load. Most of the times you skipped it in the form definition but now you will have to use it.

def get_my_choices():
    # you place some logic here
    return choices_list

class MyForm(forms.Form):
    def __init__(self, *args, **kwargs):
        super(MyForm, self).__init__(*args, **kwargs)
        self.fields['my_choice_field'] = forms.ChoiceField(
            choices=get_my_choices() )

You first call the __init__ method of the parent class(Form) using the super keyword and then declare your dynamic fields(in this case my_choice_field). With this code get_my_choices is called on every form load and you will get your dynamic drop-down.

Final words: have in mind that this method also have one strong drawback. If your application have a heavy load, executing a logic on every form load may(will) bring you troubles. So some caching of the form or the logic result may be useful.
And remember If you have other approach, idea or question I am always ready to hear it.

Python comparisons speed depends from the result

Published at Aug. 18, 2010 | Tagged with: Python , testing , speed , Development

Recently I decided to check whether "less than or equal"(<=) is slower than "bigger"(>) and I was surprised from the result. In my case "bigger" was slower. I was amazed, according to the simple logic in the case of less then or equal we need one or two operation(for example we first check for equality and the for then for "is lower"), so I asked at stackoverflow what causes this and here is the answer.

The speed of the operation does not depends from the operation itself but from the result. When the result is false it is calculated faster the if it is true. You can see the disassembler trace here.

Have you found other operations when python acts "strange"?

GAE, Timezones and weekdays...

Published at May 23, 2010 | Tagged with: Python , Google App Engine , PyTZ , timezones , Development

... or how to find the date of week in specified timezone, no matter where your server is.

The problem: I maintain a site targeting user in single country(single timezone) and I have to create administration based on a day of week(show this on Monday, that on Tuesday etc.). The site is based on Google App Engine Platform.

The simplest and most obvious solutions is to take the current time and add/substract the difference between server timezone and your target timezone. Unfortunately this is not going to work in our case.

Speciality: As you may know google have multiple data centers all over the world.

(more info at
royal.pingdom.com.)

So you can not be sure which server is currently responsible for your app. Also you have a DST(daylight saving time) that also must be included in calculation. Fortunately you can use UTC time and
PyTZ for GAE.

Solution:

from datetime import datetime, timedelta
from pytz import timezone
import pytz
bg_zone = timezone('Europe/Sofia')
bg_zone_time = bg_zone.localize(datetime.utcnow())
day_of_week = bg_zone_time.strftime("%w")

Explanation:
#5) create timezone object corresponding to your timezone(in my case this is Europe/Sofia)
#6) get the correct time for the specified timezone calling localize method of the timezone object with current UTC time as parameter
#7) using time formatting to get the day of week. There is simple catch here, at first I try it with weekday() method and get incorrect results. It looks like the method ignores the timezone/DST correction, so you have to use strftime().

Final words: I didn`t test all methods so I couldn`t say which one ignores the timezone and which not, but I hope this article will be helpful for someone facing this or similar problem. Any correction and code improvements will be appreciated.

⋘ Previous page