Merge lp:~liuyq0307/lava-scheduler/fix-861149 into lp:lava-scheduler

[Yongqin Liu]

I am not very clear about the decorator of @transaction.commit_manually(),
but if I changed it to @transaction.commit_on_success(), there will be no this problem.

In my opinion, the submit_time and end_time is submitted in UTC, but the start_time is submitted in the local TIMEZONE of the server OS.
So if use utcnow to set start_time in getLogFileForJobOnBoard_impl method, it will be realized as local TIMEZONE time to commit to DB, instead of UTC time.

BTW,
1. after sentence "from django.db.models import Q", the timezone will be changed to UTC.
2. the TIMEZONE settings seems only useful for backend of PostgreSQL in django.db
   For the following description in django/db/_init__.py file.

# DatabaseWrapper.__init__() takes a dictionary, not a settings module, so
# we manually create the dictionary from the settings, passing only the
# settings that the database backends care about. Note that TIME_ZONE is used
# by the PostgreSQL backends.

[Zygmunt Krynicki (zyga)]

This is wrong, don't do this.

Using utcnow() is correct, rendering the timezone should be fixed instead. If we have legacy records that used now() vs utcnow() we can either fix them in the datbase or just ignore them.

Look up django's TZ setting for details.

review: Disapprove

[Yongqin Liu (liuyq0307)]

> This is wrong, don't do this.
>
> Using utcnow() is correct, rendering the timezone should be fixed instead. If
> we have legacy records that used now() vs utcnow() we can either fix them in
> the datbase or just ignore them.
>
> Look up django's TZ setting for details.

Do you mean to add one line [TIME_ZONE = 'GMT'] to the settings of lava-server
(file:lava-server/lava_server/settings/debian.py)?

If so, this doesn't make any change:(

[Spring Zhang (qzhang)]

the setting is https://docs.djangoproject.com/en/1.3/ref/settings/

[Zygmunt Krynicki (zyga)]

For the record:

1) lava-server already uses GMT as anything else is broken by django design
2) All timestamps and dates in the database must use GMT/UTC
3) We can render timestamps with any offsets, this is a user preference

There is a bug somewhere that, most likely, records the wrong date in the database but this patch does not address that IMO.

[Yongqin Liu (liuyq0307)]

On 13 October 2011 21:01, Zygmunt Krynicki
<email address hidden>wrote:

> For the record:
>
> 1) lava-server already uses GMT as anything else is broken by django design
> 2) All timestamps and dates in the database must use GMT/UTC
> 3) We can render timestamps with any offsets, this is a user preference
>
> There is a bug somewhere that, most likely, records the wrong date in the
> database but this patch does not address that IMO.
>
In method getJobForBoard_impl, the start_time is utc time, but the commit
use it as local time to store into database,
I will look into the code deeper to see why the transaction commit does as
such.

[Michael Hudson-Doyle (mwhudson)]

The problem turned out to be rather different from what might have been expected :(

Unmerged revisions

81. By Yongqin Liu on 2011-10-11

make start_time use local time to avoid the problem that start_time is before submit_time

=== modified file 'lava_scheduler_daemon/dbjobsource.py'
--- lava_scheduler_daemon/dbjobsource.py	2011-08-22 03:20:17 +0000
+++ lava_scheduler_daemon/dbjobsource.py	2011-10-11 14:49:24 +0000
@@ -2,6 +2,15 @@
 import json
 import logging
 
+import time
+old_altzone=time.altzone
+
+class TZ(datetime.tzinfo):
+    def utcoffset(self, dt):
+        return datetime.timedelta(seconds=-old_altzone)
+    def dst(self, dt):
+        return datetime.timedelta(0)
+
 from django.core.files.base import ContentFile
 from django.db import connection
 from django.db import IntegrityError, transaction
@@ -73,7 +82,7 @@
             if jobs:
                 job = jobs[0]
                 job.status = TestJob.RUNNING
-                job.start_time = datetime.datetime.utcnow()
+                job.start_time = datetime.datetime.now(TZ())
                 job.actual_device = device
                 device.status = Device.RUNNING
                 device.current_job = job