Posts tagged with Django

Fabric & Django

Published at May 28, 2012 | Tagged with: , , ,

Or how automate the creation of new projects with simple script

Preface: Do you remember all this tiny little steps that you have to perform every time when you start new project - create virtual environment, install packages, start and setup Django project? Kind of annoying repetition, isn't it? How about to automate it a bit.

Solution: Recently I started learning Fabric and thought "What better way to test it in practice than automating a simple, repetitive task?". So, lets mark the tasks that I want the script to perform:

  1. Create virtual environment with the project name
  2. Activate the virtual environment
  3. Download list of packages and install them
  4. Make 'src' directory where the project source will reside
  5. Create new Django project in source directory
  6. Update the settings

Thanks to the local command the first one was easy. The problem was with the second one. Obviously each local command is run autonomously so I had to find some way have activated virtual environment for each task after this. Fortunately the prefix context manager works like a charm. I had some issues making it read and write in the paths I wants and voilà it was working exactly as I want.

The script is too long to place it here but is publicly available at
It is quite simple to use, you only need python, fabric and virtual environment. Then just use the following code.

fab start_project:my_mew_project

To Do: Here are few things that can be improved:

  • Read the packages from a file
  • Update to enable admin
  • Generate Nginx server block file

So this is my first try with Fabric, I hope that you will like it and find it useful. As always any comments, questions and/or improvement ideas are welcome.

Connecting Django Models with outer applications

Published at Jan. 23, 2012 | Tagged with: , , , ,

Preface: Sometimes, parts of the data that you have to display in your application reside out of the Django models. Simple example for this is the following case - the client requires that you build them a webshop but they already have CRM solution that holds their products info. Of course they provide you with a mechanism to read this data from their CRM.

Specialty: The problem is that the data in their CRM does not hold some of the product information that you need. For instance it misses SEO-friendly description and product image. So you will have to set up a model at your side and store these images there. It is easy to join them, the only thing that you will need is a simple unique key for every product.

Solution: Here we use the product_id field to make the connection between the CRM data and the Django model.

# in
class Product(models.Model):
    product_id = models.IntegerField(_('Original Product'),
    description = models.TextField(_('SEO-friendly Description'),
    pod_image = FilerImageField(verbose_name=_('Product Image'),
                                blank=True, null=True)

   def name(self):
       return crm_api.get_product_name(self.product_id)

# in
class ProductForm(forms.ModelForm):
    name = forms.CharField(required=False,
                               'readonly': True,
                               'style': 'border: none'}))

    class Meta:
        model = Product
        widgets = {
            'product_id': forms.Select(),
    def __init__(self, *args, **kwargs):
        super(ProductForm, self).__init__(*args, **kwargs)
        self.fields['product_id'].widget.choices = crm_api.get_product_choices()
            self.fields['name'].initial =

The form here should be used in the admin(add/edit) page of the model. We define that the product_id field will use the select widget and we use a method that connect to the CRM and returns the product choices list.
The "" check is used to fill the name field for product that are already saved.

Final words: This is a very simple example but its idea is to show the basic way to connect your models with another app. I strongly recommend you to use caching if your CRM data is not modified very often in order to save some bandwidth and to speed up your application.
Also if you have multiple field it may be better to overwrite the __getattr__ method instead of defining separate one for each property that you need to pull from the outer application.

P.S. Thanks to Miga for the code error report.

Django CMS Plugins with selectable template ...

Published at Oct. 9, 2011 | Tagged with: , , , , ,

... or how to reuse your plugins inside sections with different design

Problem: Frequently on the websites I am developing I need to display same set of data in several different ways. For example if I have a news box that needs to appear in different sections of the website e.i. in sidebar, main content etc. Using Django CMS plugins make this quite easy.
For simplicity we will take the following case. An image/text tuple with two layout variations - image on left of text and image on right.

Django CMS Plugins

Same data but different layout. All you need to do is just to allow your users to change the plugin template according to their needs. If you don't have experience with Django CMS Plugins I advice you to check how to create custom Django CMS Plugins before you continue with solution.

Solution: First you will have to create a tuple holding your templates(and their human readable names) and add a field that will hold the chosen template to the plugin model.
  ('image_on_left.html', 'Image on left'),
  ('image_on_right.html', 'Image on right'),

class SamplePlugin(CMSPlugin):
    # your plugin properties here
    template = models.CharField('Template', max_length=255,
                                choices = PLUGIN_TEMPLATES)
Now it is time to tweak the template render method too:
class CMSSamplePlugin(CMSPluginBase):
    model = SamplePlugin
    name = 'Sample plugin'
    render_template = PLUGIN_TEMPLATES[0][0]
    def render(self, context, instance, placeholder):
        if instance and instance.template:
            self.render_template = instance.template
        #your stuff here
        return context

Final words: Yep, this is all. Simple isn't it? It is amazing how sometimes such small things are so useful. If you are having bigger difference in the layout of your templates you will probably have to put a little more stuff in the context that some of your templates may not need but it is OK. Feel free to comment and if you are using this "trick" please add your use case - it will be interesting to see in how many different cases this works.

Language redirects for multilingual sites with Django CMS ...

Published at Sept. 11, 2011 | Tagged with: , , , , , , , ,

... or how to avoid duplicate content by keeping the current language in the URL

Preface: Earlier this year I posted about Django CMS 2.2 features that I want to see and one of the things mentioned there was that once you have chosen the language of the site there is no matter whether you will open "/my_page/" or "/en/my_page/" - it just shows the same content. The problem is that this can be considered both duplicate and inconsistent content.
Duplicate because you see the same content with and without the language code in the URL and inconsistent because for the same URL you can get different language versions i.e. different content.

Solution: This can be easy fixed by using a custom middleware that will redirect the URL that does not contain language code. In my case the middleware is stored in "middleware/"(the path is relative to my project root directory) and contains the following code.

from cms.middleware.multilingual import MultilingualURLMiddleware 
from django.conf import settings
from django.http import HttpResponseRedirect
from django.utils import translation

class CustomMultilingualURLMiddleware(MultilingualURLMiddleware): 
    def process_request(self, request):
        lang_path = request.path.split('/')[1]
        if lang_path in settings.URLS_WITHOUT_LANGUAGE_REDIRECT:
            return None
        language = self.get_language_from_request(request) 
        request.LANGUAGE_CODE = language
        if lang_path == '': 
            return HttpResponseRedirect('/%s/' % language)
        if len([z for z in settings.LANGUAGES if z[0] == lang_path]) == 0:
            return HttpResponseRedirect('/%s%s' % (language, request.path))
Now a little explanation on what happens in this middleware. Note: If you are not familiar with how middlewares work go and check Django Middlewares. Back to the code. First we split the URL by '/' and take the second element(this is where our language code should be) and store in lang_path(8). URLS_WITHOUT_LANGUAGE_REDIRECT is just a list of URLs that should not be redirected, if lang_path matches any of the URLs we return None i.e. the request is not changed(9-10). This is used for sections of the site that are not language specific for example media stuff. Then we get language based on the request(11-13). If lang_path is empty then the user has requested the home page and we redirect him to the correct language version of it(14-15). If lang_path does not match any of the declared languages this mean that the language code is missing from the URL and the user is redirected to the correct language version of this page(16-17). To make the middleware above to work you have to update your First add the middleware to your MIDDLEWARE_CLASSES - in my case the path is 'middleware.URLMiddlewares.CustomMultilingualURLMiddleware'. Second add URLS_WITHOUT_LANGUAGE_REDIRECT list and place there the URLs that should not be redirected, example:
Specialties: If the language code is not in the URL and there is no language cookie set your browser settings will be used to determine your preferred language. Unfortunately most of the users do not know about this option and it often stays set to its default value. If you want this setting to be ignored just add the following code after line 10 in the middleware above:
if request.META.has_key('HTTP_ACCEPT_LANGUAGE'):
    del request.META['HTTP_ACCEPT_LANGUAGE']

It removed the HTTP_ACCEPT_LANGUAGE header sent from the browser and Django uses the language set in its settings ad default.

URLS_WITHOUT_LANGUAGE_REDIRECT is extremely useful if you are developing using the built in dev server and serve the media files trough it. But once you put your website on production I strongly encourage you to serve these files directly by the web server instead of using Django static serve.

Final words: In Django 1.4 there will be big changes about multilingual URLs but till then you can use this code will improve your website SEO. Any ideas of improvement will be appreciated.

Foreign key to Django CMS page ...

Published at July 14, 2011 | Tagged with: , , , , , , , ,

... how to make usable drop-downs with Django CMS pages

Problem: some times when you create custom applications or plugins for Django CMS you need a property that connects the current item to a page in the CMS. Nothing simple than this - you just add a ForeignKey in your model that points to the Page model and everything is (almost)fine. Example:

from cms.models import Page

class MyModel(models.Model):
    # some model attributes here
    page = models.ForeignKey(Page)    
If you registered your model in Django admin or just add a model form to it you will see something like this:

Django Admin Screenshot

Cool right? Not exactly. The problem is that these pages are in hierarchical structure and listing them in a flat list may be/is little confusing. So let's indent them accordingly to their level in the hierarchy. Solution: The easies way to achieve this indentation is to overwrite the choices list of the ForeignKey field in the ModelForm __init__ method.
class MyModelForm(forms.ModelForm):
    class Meta:
        model = MyModel
    def __init__(self, *args, **kwargs):
        super(MyModelForm, self).__init__(*args, **kwargs)
        choices = [self.fields['page'].choices.__iter__().next()]
        for page in self.fields['page'].queryset:
                (, ''.join(['-'*page.level, page.__unicode__()]))
        self.fields['page'].choices = choices 

The magic lies between lines 7 and 11, on line 7 we create a list with one element the default empty option for the drop down. The need to use "__iter__().next()" comes from the fact that the choices attribute of the fields is django.forms.models.ModelChoiceIterator object which is iterable, but not indexable i.e. you can not just use self.fields['url'].choices[0].
After we had the empty choice now it is time to add the real ones, so we iterate over the queryset(8th line) that holds them and for each item we add a tuple to our choices list(10). The first item of the tuple is the page id - nothing special, but the second one... here the python magic comes. We multiple the minus sign('-') by the page level and join the result with the page title. The only thing left is to replace the field choices(line 12) and here is the result:

Django Admin - usable drop-down

Final words: For me this is much more usable than the flat list. Of course you can modify the queryset to return only published pages or filter the results in other way and still use the identation code from above.
I'll be happy to hear your thoughts on this.

Caching websites with Django and Memcached...

Published at July 12, 2011 | Tagged with: , , , , , , ,

... memcached installation, django configuration and how to use it for your website

After Caching web sites and web applications and When to use caching for web sites its time for a little sample. This sample shows the usage of Memcached for server-side application cache. The installation part is taken from my Ubuntu so it may differ depending from your OS/distribution.

What is Memcached: Memcached is a tool that allows you to store key-value pairs in you memory. The keys are limited to 250 Bytes and for better performance the value size is limited to 1MB(more details) but this size is fair enough for web usage.

Memcached installation:

apt-get install memcached
apt-get install python-memcache
The first line installs Memcached and the second one install Python API for communication between your application and Memcached daemon. After this the Memcached daemon is up and running. With default configuration it runs on port 11211 on localhost( If you want to modify this the configuration file(in my case) is situated in /etc/memcached.conf Django configuration: This one depends from the Django version that you use. For 1.2.5 and prior the next code should by added in your settings file(
CACHE_BACKEND = 'memcached://'
For 1.3 and development version add:
    'default': {
        'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
        'LOCATION': '',
In both cases if you use different port and/or IP you have to replace them above. More info about cache backend configuration you can find in Django documentation docs. So now you have Memcached running and Django configured. If you have doubts about is this suitable/usable in you case take a look at the posts mentioned above or just add comment with your case and I will be happy to give you an advice. Now it is time to start using it. Cache usage(part I) - how to cache on Python level: If you have some heavy calculations in your view you can cache the result from this and use the calculated one to lower the load. Example:
from django.core.cache import cache

def heavy_view(request):
    cache_key = 'my_heavy_view_cache_key'
    cache_time = 1800 # time to live in seconds
    result = cache.get(cache_key)
    if not result:
        result = # some calculations here
        cache.set(cache_key, result, cache_time)
    return result
The process is simple, you ask the cache for a value corresponding to a given key(line 4). If the result is None you execute the code that generates it(line 8 ) and store it in the cache(line 9). My advice is to declare the key and time as variables cause this will ease their future changes. Cache usage(part II) - how to cache on template level: This is suitable for the cases when you have some heavy processing in the template(as regroup) or you want to cache only part of the template(as latest news section). Example:
{% load cache %}
 ... non cached content here ...
{% cache 1800 latest_news %}
    ... here are latest news - cached ...
{% endcache %}
The basic usage usage is {% cache time_in_seconds key %} ... {% endcache %} You can also cache code fragments based on dynamic properties, for example - current user recent conversations, just pass a 3rd param the uniquely identifies the code to be cached.
{% load cache %}
{% cache 300 recent_conversations %}
    ... current user recent conversations - cached ...
{% endcache %}

Final words: as you see from the examples above using Django and Memcached is really easy. Using it correctly will speed up your website and respectively improve your user experience(UX) and SEO. Using it wrong will provide negative results. Just take a moment and think what can be cached, how long can it be cached and is there a reason to be cached. Try to avoid double caching - there is no need to use caching in templates and then cache the rendered template in the view too.

Custom 404 Not Found page with Django CMS...

Published at July 3, 2011 | Tagged with: , , , , , , ,

... or how to make user editable 404 page that stays in the pages tree of the CMS

Basics: Yes you need it! You need 404 page cause you never know what may happen to a link: bad link paste, obsolete or deleted article, someone just playing with your URLs etc. It is better for both you and your website visitors to have a beauty page that follows the website design instead of the webserver default one that usually contains server information which is possible security issue. With Django this is easy, just make a HTML template with file name 404.html, place it in you root template directory and voilà - you are ready. You will also automatically have a request_path variable defined in the context which caries the URL that was not found.

Problem: sometimes clients require to be able to edit their 404 pages. Or other times you need to use some custom context or you want to integrate plug-ins and be able to modify them easy trough the CMS administration. For example: you want do display your brand new awesome "Sitemap Plug-in" on this 404 page.

Solution: Django allows you to specify custom 404 handler view so you just need to define one, set it in and make it to render the wanted page:

# in
handler404 = 'site_utils.handler404' 

# in
from cms.views import details 
def handler404(request):
    return details(request, '404-page-url')
Where '404-page-url' is the URL of the page you want to show for 404 errors. So everything seems fine and here is the pitfall. If you use it this way your web page will return "200 OK" instead of "404 Not Found". This could kill your SEO(except if you want your 404 page as first result for your website). So you just need to add a 404 header to the response:
def handler404(request):
    response = details(request, 'novini')
    response.status_code = 404
    return response

Final words: Why are these HTTP status codes so important. The reason is that they tells the search engines and other auto crawling services what is the page status. Is it normal page, redirect, not found, error or something else. Providing incorrect status codes may/will have a negative effect on your website SEO so try to keep them correct, especially when it is easy to achieve as in the example above.

Note: If the code above is not working for you, please check Allan's solution in the comments

Django models ForeignKey and custom admin filters...

Published at June 1, 2011 | Tagged with: , , , , , , ,

... or how to limit dynamic foreign key choices in both edit drop-down and admin filter.

The problem: Having a foreign key between models in django is really simple. For example:

class Question(models.Model):
    # some fields here

class Answer(models.Model):
    # some fields here
    question = models.ForeignKey(Question)
Unfortunately in the real live the choices allowed for the connection are frequently limited by some application logic e.g. you may add answers only to questions created by you. In some simple(rare) cases this can be easy achieved using choices or limit_choices_to attributes in the models.ForeignKey call. In the case of choices you just have to pass list/tuple each element of which contains the value to be stored and human readable name for the choice. Unfortunately this one is computed on server run and is not updated with new items during run-time. If you use limit_choices_to you may pass to it some kind of filter expression e.g. limit_choices_to = {'pub_date__lte':} but this not always can do the job. Speciality: So if you want dynamic choices in the admin drop down you have to write a method that will return list with options and bind it to the form(as shown in Django forms ChoiceField with dynamic values…) which is used by admin. This will work great but if you decided to add this column in admin`s list_filter you will see all element from the connected model in filter. How to limit them to the same list used for the form choices? Solution: The simplest solutions is to extend RelatedFilterSpec, overwrite its default choices and add a single row to the model:
from django.contrib.admin.filterspecs import FilterSpec, RelatedFilterSpec

class CustomFilterSpec(RelatedFilterSpec):
    def __init__(self, *args, **kwargs):
        super(CustomFilterSpec, self).__init__(*args, **kwargs)        
        self.lookup_choices = get_questions() #this method returns the dynamic list

FilterSpec.filter_specs.insert(0, (lambda f: bool(f.rel and hasattr(f, 'custom_filter_spec')), CustomFilterSpec))

class Answer(models.Model):
    # some fields here
    question = models.ForeignKey(Question)
    question.custom_filter_spec = True # this is used to identify the fields which use the custom filter

Final word: The solutions is pretty simple and really easy to implement but should be used carefully. If your filtering method(in my case get_questions) is slow/resource consuming it may bring you troubles. Here is the place where and you should think about caching it. This is a place where a application cache can be used. Hope this will help you as much as it helped me.

Querying ManyToMany fields in Django...

Published at March 23, 2011 | Tagged with: , , , ,

... or how to select items that have one or more categories matching with other item categories

The problem: having a Many To Many relation(ManyToManyField) is really useful when you have to link one object to many other. For example one artist may play in several styles, or a teacher can teach more than one subjects. I`ll use the latter example for the post. If you want to get all teachers that teach (for example) "music" you just have to do something like:

Teacher.objects.filter(subjects = music_obj)
The real problem comes when you want to select all teachers that have subject matching one or more of another teacher subjects. For example "teacher_a" teaches several subject and we want to find all his colleges that teach at least one of his. Solution: to achieve this we have to use the Q object(imported from django.db.models) in a way similar to the making of "OR" query.
from django.db.models import Q
x = Q()
for subject in teacher_a.subjects.all():
    x = x | Q(subjects = subject) 
colegues = Teacher.objects.filter(x).distinct()

Lets see what happens line by line:

1) import the Q class
2) create empty Q object
3) iterate over every subject taught by the specified teacher
4) on every iteration we add an "OR"-clause to the object corresponding to the current subject
5) we use the generated query as a filter

Have in mind that after using the Q object you have to sort your result(manually add order_by clause), because when you use Q the default ordering is not considered.

Final words: The speed of this method is not tested but to be honest I suspect that this can bring performance problems on big tables. In that case maybe you can speed the things up with custom SQL query. So the choice is yours - database independence or speed. Both ways have their pros and cons and its case specific which one you will choose.
If you have found a better way or want to add something feel free to comment.

Post on Request

Published at Jan. 30, 2011 | Tagged with: , , , ,

In the last few weeks I have been little busy(and suffering from lack of inspiration) so my blog lacks of new posts but I see that there are people visiting it(especially for the Django CMS stuff) so I am willing to start a "Post on Request" practice.

The idea is simple - I will collect post theme requests in the first three weeks(or 21 days) of every month and after a choose based on your votes( and my decision ) I`ll go out with a post on the selected theme by the end of the week. Please keep your question in the following areas - Django, Django CMS, Web Development. Question in other areas will be considered too, but may stand out of my knowledge area. You can place your request here or check the list and vote for another one or just comment this post.
I hope that you will find this initiative interesting and join it.

⋘ Previous page Next page ⋙