Posts tagged with timezones

Django, pytz, NonExistentTimeError and AmbiguousTimeError

Published at March 29, 2015 | Tagged with: , , ,

Brief: In one of the project I work on we had to convert some old naive datetime objects to timezone aware ones. Converting naive datetime to timezone aware one is usually a straightforward job. In django you even have a nice utility function for this. For example:

import pytz
from django.utils import timezone


timezone.make_aware(datetime.datetime(2012, 3, 25, 3, 52),
                    timezone=pytz.timezone('Europe/Stockholm'))
# returns datetime.datetime(2012, 3, 25, 3, 52, tzinfo=<DstTzInfo 'Europe/Stockholm' CEST+2:00:00 DST>)

Problem: You can use this for quite a long time until one day you end up with something like this:

timezone.make_aware(datetime.datetime(2012, 3, 25, 2, 52),
                    timezone=pytz.timezone('Europe/Stockholm'))

# which leads to
Traceback (most recent call last):
  File "", line 1, in 
  File "/home/ilian/venvs/test/lib/python3.4/site-packages/django/utils/timezone.py", line 358, in make_aware
    return timezone.localize(value, is_dst=None)
  File "/home/ilian/venvs/test/lib/python3.4/site-packages/pytz/tzinfo.py", line 327, in localize
    raise NonExistentTimeError(dt)
pytz.exceptions.NonExistentTimeError: 2012-03-25 02:52:00

Or this:

    
timezone.make_aware(datetime.datetime(2012, 10, 28, 2, 52),
                    timezone=pytz.timezone('Europe/Stockholm'))

#throws
Traceback (most recent call last):
  File "", line 1, in 
  File "/home/ilian/venvs/test/lib/python3.4/site-packages/django/utils/timezone.py", line 358, in make_aware
    return timezone.localize(value, is_dst=None)
  File "/home/ilian/venvs/test/lib/python3.4/site-packages/pytz/tzinfo.py", line 349, in localize
    raise AmbiguousTimeError(dt)
pytz.exceptions.AmbiguousTimeError: 2012-10-28 02:52:00

Explanation: The reason for the first error is that in the real world this datetime does not exists. Due to the DST change on this date the clock jumps from 01:59 standard time to 03:00 DST. Fortunately (or not) pytz is aware of the fact that this time is invalid and will throw the exception above. The second exception is almost the same but it happens when switching from summer to standard time. From 01:59 DST the clock shifts to 01:00 standard time, so we end with a duplicate time.

Why has this happened(in our case)? Well we couldn't be sure how exactly this one got into our legacy data but the assumption is that at the moment when the record was saved the server has been in different timezone where this has been a valid time.

Solution 1: This fix is quite simple, just add an hour if the exception occurs.

try:
    date = make_aware(
        datetime.fromtimestamp(date_time, timezone=pytz.timezone('Europe/Stockholm'))
    )
except (pytz.NonExistentTimeError, pytz.AmbiguousTimeError):
    date = make_aware(
        datetime.fromtimestamp(date_time) + timedelta(hours=1),
        timezone=pytz.timezone('Europe/Stockholm')
    )

Solution 2: Instead of calling make_aware call timezone.localize directly.

try:
    date = make_aware(
        datetime.fromtimestamp(date_time, timezone=pytz.timezone('Europe/Stockholm'))
    )
except (pytz.NonExistentTimeError, pytz.AmbiguousTimeError):
    timezone = pytz.timezone('Europe/Stockholm')
    date = timezone.localize(datetime.fromtimestamp(date_time), is_dst=False)

The second solution probably needs some explanation. First lets check what make_aware does. The code bellow is take from Django's sourcecode as it is in version 1.7.7

def make_aware(value, timezone):
    """
    Makes a naive datetime.datetime in a given time zone aware.
    """
    if hasattr(timezone, 'localize'):
        # This method is available for pytz time zones.
        return timezone.localize(value, is_dst=None)
    else:
        # Check that we won't overwrite the timezone of an aware datetime.
        if is_aware(value):
            raise ValueError(
                "make_aware expects a naive datetime, got %s" % value)
        # This may be wrong around DST changes!
        return value.replace(tzinfo=timezone)    

To simplify it, what Django does is to use the localize method of the timezone object(if it exists) to convert the datetime. When using pytz this localize method takes two arguments: the datetime value and is_dst. The last argument takes three possible values: None, False and True. When using None and the datetime matches the moment of the DST change pytz does not know how to handle the datetime and you get one of the exceptions shown above. False means that it should convert it to standard time and True that it should convert it to summer time.

Why isn't this fixed in Django? The simple answer is "because this is how it should work". For a bit longer check the respectful ticket.

Reminder: Do not forget that the "fix" above does not actually care whether the original datetime is during DST or not. In our case this was not criticla for our app, but in some other cases it might be, so use it carefully.

Thanks: Special thanks to Joshua who correctly pointed out in the comments that I have missed the AmbiguousTimeError in the original post which made me to look a bit more in the problem, research other solutions and update the article to its current content.

GAE, Timezones and weekdays...

Published at May 23, 2010 | Tagged with: , , , ,

... or how to find the date of week in specified timezone, no matter where your server is.

The problem: I maintain a site targeting user in single country(single timezone) and I have to create administration based on a day of week(show this on Monday, that on Tuesday etc.). The site is based on Google App Engine Platform.

The simplest and most obvious solutions is to take the current time and add/substract the difference between server timezone and your target timezone. Unfortunately this is not going to work in our case.

Speciality: As you may know google have multiple data centers all over the world.


(more info at
royal.pingdom.com.)

So you can not be sure which server is currently responsible for your app. Also you have a DST(daylight saving time) that also must be included in calculation. Fortunately you can use UTC time and
PyTZ for GAE.

Solution:

from datetime import datetime, timedelta
from pytz import timezone
import pytz
bg_zone = timezone('Europe/Sofia')
bg_zone_time = bg_zone.localize(datetime.utcnow())
day_of_week = bg_zone_time.strftime("%w")

Explanation:
#5) create timezone object corresponding to your timezone(in my case this is Europe/Sofia)
#6) get the correct time for the specified timezone calling localize method of the timezone object with current UTC time as parameter
#7) using time formatting to get the day of week. There is simple catch here, at first I try it with weekday() method and get incorrect results. It looks like the method ignores the timezone/DST correction, so you have to use strftime().

Final words: I didn`t test all methods so I couldn`t say which one ignores the timezone and which not, but I hope this article will be helpful for someone facing this or similar problem. Any correction and code improvements will be appreciated.