Merge lp:~veebers/autopilot/fix-1328600-large-datetime into lp:autopilot

Proposed by Christopher Lee
Status: Superseded
Proposed branch: lp:~veebers/autopilot/fix-1328600-large-datetime
Merge into: lp:autopilot
Diff against target: 622 lines (+375/-66)
6 files modified
autopilot/introspection/types.py (+97/-7)
autopilot/tests/functional/fixtures.py (+15/-1)
autopilot/tests/functional/test_types.py (+87/-17)
autopilot/tests/unit/test_test_fixtures.py (+20/-0)
autopilot/tests/unit/test_types.py (+153/-41)
debian/control (+3/-0)
To merge this branch: bzr merge lp:~veebers/autopilot/fix-1328600-large-datetime
Reviewer Review Type Date Requested Status
Nicholas Skaggs (community) Approve
PS Jenkins bot continuous-integration Needs Fixing
Thomi Richards (community) Needs Fixing
Robert Bruce Park Pending
Barry Warsaw Pending
Christopher Lee Pending
Review via email: mp+236815@code.launchpad.net

This proposal supersedes a proposal from 2014-07-18.

This proposal has been superseded by a proposal from 2014-10-22.

Commit message

Workaround around 32-bit platform limitations with regards to timestamps. Bug 1328600

Description of the change

Fixes bug 1328600. Includes a test for "large" timestamps as well

This works around 32-bit platform limitations.

Resubmitted with updates.

To post a comment you must log in.
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal

PASSED: Continuous integration, rev:506
http://jenkins.qa.ubuntu.com/job/autopilot-ci/804/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-utopic-amd64-ci/79
        deb: http://jenkins.qa.ubuntu.com/job/autopilot-utopic-amd64-ci/79/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-utopic-armhf-ci/78
        deb: http://jenkins.qa.ubuntu.com/job/autopilot-utopic-armhf-ci/78/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-utopic-i386-ci/78
        deb: http://jenkins.qa.ubuntu.com/job/autopilot-utopic-i386-ci/78/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-utopic-touch/2206
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-utopic-autopilot/219
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-runner-mako/2415
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-utopic-armhf/3380
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-utopic-armhf/3380/artifact/work/output/*zip*/output.zip
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/touch-flash-device/10094
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-utopic-autopilot/283
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-utopic-amd64/2059
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-utopic-amd64/2059/artifact/work/output/*zip*/output.zip

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/autopilot-ci/804/rebuild

review: Approve (continuous-integration)
Revision history for this message
Thomi Richards (thomir-deactivatedaccount) wrote : Posted in a previous version of this proposal

This doesn't actually fix the problem, it just creates new problems. Please see my comment on the bug report:

https://bugs.launchpad.net/ubuntu-calendar-app/+bug/1328600/comments/9

Thanks for the patch though, but it looks like this is going to be a bit more of an involved fix.

Cheers,

review: Disapprove
Revision history for this message
Nicholas Skaggs (nskaggs) wrote : Posted in a previous version of this proposal

So the idea of using tzlocal seems to suffer from daylight savings problem, unless I'm missing something.

>>> from datetime import datetime, time, timedelta
>>> from dateutil.tz import tzlocal, tzutc
>>> print(datetime.fromtimestamp(1377209927))
2013-08-22 18:18:47
>>> print(datetime.fromtimestamp(1377209927,tzutc()))
2013-08-22 22:18:47+00:00
>>> naive = datetime.fromtimestamp(0) + timedelta(seconds=1377209927)
>>> print(naive)
2013-08-22 17:18:47
>>> aware = naive.replace(tzinfo=tzlocal())
>>> print(aware)
2013-08-22 17:18:47-04:00
>>> print(aware.astimezone(tzutc()))
2013-08-22 21:18:47+00:00
>>> aware2 = datetime.fromtimestamp(0, tzlocal())
>>> print(aware2)
1969-12-31 19:00:00-05:00
>>> print(aware2.astimezone(tzutc()))
1970-01-01 00:00:00+00:00
>>> try2 = aware2 + timedelta(seconds=1377209927)
>>> print(try2)
2013-08-22 17:18:47-04:00
>>> print(try2.astimezone(tzutc()))
2013-08-22 21:18:47+00:00

Revision history for this message
Nicholas Skaggs (nskaggs) wrote : Posted in a previous version of this proposal

Going to grab the local offset at the time of the timestamp and apply it against the fromtimestamp date.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
Thomi Richards (thomir-deactivatedaccount) wrote : Posted in a previous version of this proposal

I have a couple of comments in the diff.

Overall, this looks excellent - thank you for taking this on.

There's one change I'd like you to make in the tests, and I'd really like Barry Warsaw to review the implementation.

I'm setting this to approve, so you don't need a re-review once those two things have happeend (I trust ya :D )

review: Approve
Revision history for this message
Thomi Richards (thomir-deactivatedaccount) wrote : Posted in a previous version of this proposal

Oh, also, CI has to pass (obviously)

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
Nicholas Skaggs (nskaggs) wrote : Posted in a previous version of this proposal

Tests don't want to run now?

/var/local/autopilot/setup.log: I: Running autopilot
/var/local/autopilot/setup.log: I: Using python2
tail: '/var/lib/lxc//utopic-amd64-20140619-0752/run/delta//var/local/autopilot/autopilot.log' has become accessible
/var/local/autopilot/autopilot.log: Loading tests from: /usr/lib/python2.7/dist-packages
/var/local/autopilot/autopilot.log:
/var/local/autopilot/autopilot.log: Did not find any tests
/var/local/autopilot/autopilot.log:
/var/local/autopilot/autopilot.log: ----------------------------------------------------------------------
/var/local/autopilot/autopilot.log: Ran 0 tests in 0.126s
/var/local/autopilot/autopilot.log:
/var/local/autopilot/autopilot.log: OK
/var/local/autopilot/setup.log: I: Using python3
tail: /var/lib/lxc//utopic-amd64-20140619-0752/run/delta//var/local/autopilot/autopilot.log: file truncated
/var/local/autopilot/autopilot.log:
/var/local/autopilot/autopilot.log: ----------------------------------------------------------------------
/var/local/autopilot/autopilot.log: Ran 0 tests in 0.110s
/var/local/autopilot/autopilot.log:
/var/local/autopilot/autopilot.log: OK
/var/local/autopilot/setup.log: I: No test left to run

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
Nicholas Skaggs (nskaggs) wrote : Posted in a previous version of this proposal

Why is this running 0 tests?

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
Nicholas Skaggs (nskaggs) wrote : Posted in a previous version of this proposal

I see a crash:

PythonArgs: ['/usr/sbin/aa-status']
Traceback:
 Traceback (most recent call last):
   File "/usr/sbin/aa-status", line 194, in <module>
     commands[cmd]()
   File "/usr/sbin/aa-status", line 17, in cmd_enabled
     if get_profiles() == {}:
   File "/usr/sbin/aa-status", line 92, in get_profiles
     for p in open(apparmor_profiles).readlines():
 PermissionError: [Errno 13] Permission denied: '/sys/kernel/security/apparmor/profiles'

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
Nicholas Skaggs (nskaggs) wrote : Posted in a previous version of this proposal

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/autopilot/tests/functional/test_types.py", line 28, in test_date
    datetime(2014, 1, 1, 0, 0, 0)
  File "/usr/lib/python3/dist-packages/testtools/testcase.py", line 338, in assertEqual
    self.assertThat(observed, matcher, message)
  File "/usr/lib/python3/dist-packages/testtools/testcase.py", line 423, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: DateTime(2014-01-01 00:00:00) != datetime.datetime(2014, 1, 1, 0, 0)

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
Christopher Lee (veebers) wrote : Posted in a previous version of this proposal

A couple of minor indent issues. Deeper review to come.

review: Needs Fixing
Revision history for this message
Robert Bruce Park (robru) wrote : Posted in a previous version of this proposal

Packaging looks good, thanks for the wrap-and-sort, makes for a noisy diff now but easier to read later.

review: Approve
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal

PASSED: Continuous integration, rev:535
http://jenkins.qa.ubuntu.com/job/autopilot-ci/865/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-utopic-amd64-ci/140
        deb: http://jenkins.qa.ubuntu.com/job/autopilot-utopic-amd64-ci/140/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-utopic-armhf-ci/139
        deb: http://jenkins.qa.ubuntu.com/job/autopilot-utopic-armhf-ci/139/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-utopic-i386-ci/139
        deb: http://jenkins.qa.ubuntu.com/job/autopilot-utopic-i386-ci/139/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-utopic-touch/4627
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-utopic-autopilot/303
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-runner-mako/4409
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-utopic-armhf/5879
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-utopic-armhf/5879/artifact/work/output/*zip*/output.zip
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/touch-flash-device/12911
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-utopic-autopilot/342
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-utopic-amd64/3727
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-utopic-amd64/3727/artifact/work/output/*zip*/output.zip

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/autopilot-ci/865/rebuild

review: Approve (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal

PASSED: Continuous integration, rev:536
http://jenkins.qa.ubuntu.com/job/autopilot-ci/866/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-utopic-amd64-ci/141
        deb: http://jenkins.qa.ubuntu.com/job/autopilot-utopic-amd64-ci/141/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-utopic-armhf-ci/140
        deb: http://jenkins.qa.ubuntu.com/job/autopilot-utopic-armhf-ci/140/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-utopic-i386-ci/140
        deb: http://jenkins.qa.ubuntu.com/job/autopilot-utopic-i386-ci/140/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-utopic-touch/4631
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-utopic-autopilot/304
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-runner-mako/4413
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-utopic-armhf/5883
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-utopic-armhf/5883/artifact/work/output/*zip*/output.zip
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/touch-flash-device/12920
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-utopic-autopilot/343
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-utopic-amd64/3732
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-utopic-amd64/3732/artifact/work/output/*zip*/output.zip

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/autopilot-ci/866/rebuild

review: Approve (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal

PASSED: Continuous integration, rev:537
http://jenkins.qa.ubuntu.com/job/autopilot-ci/867/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-utopic-amd64-ci/142
        deb: http://jenkins.qa.ubuntu.com/job/autopilot-utopic-amd64-ci/142/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-utopic-armhf-ci/141
        deb: http://jenkins.qa.ubuntu.com/job/autopilot-utopic-armhf-ci/141/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-utopic-i386-ci/141
        deb: http://jenkins.qa.ubuntu.com/job/autopilot-utopic-i386-ci/141/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-utopic-touch/4633
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-utopic-autopilot/305
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-runner-mako/4415
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-utopic-armhf/5885
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-utopic-armhf/5885/artifact/work/output/*zip*/output.zip
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/touch-flash-device/12923
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-utopic-autopilot/344
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-utopic-amd64/3733
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-utopic-amd64/3733/artifact/work/output/*zip*/output.zip

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/autopilot-ci/867/rebuild

review: Approve (continuous-integration)
Revision history for this message
Nicholas Skaggs (nskaggs) wrote : Posted in a previous version of this proposal

Can you provide an update on what you'd like to see with mp?

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
542. By Christopher Lee

Commented out part of test to explore failure.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
543. By Christopher Lee

Avoid use of calls that use fromtimestamp (or _isdst) as they aren't large timestamp safe.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
544. By Christopher Lee

Fix creation of localtime + re-added test.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
545. By Christopher Lee

Removed unneeded check

546. By Christopher Lee

Cleanup tz setting in functional test.

547. By Christopher Lee

WIP comments for working out the DST applications.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
548. By Christopher Lee

Rough (further)WIP. About to all change.

549. By Christopher Lee

Code confirmation and cleanup. Using aware datetime objects now

550. By Christopher Lee

Change __eq__ operator and update test

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
551. By Christopher Lee

Improve naming and comments explaining what is going on.

552. By Christopher Lee

flake8 cleanup.

553. By Christopher Lee

Commit and whitespace cleanup

554. By Christopher Lee

Re-enable test.

555. By Christopher Lee

Remove un-needed package from d/control

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
556. By Christopher Lee

Updated docstrings.

557. By Christopher Lee

Further docstring update

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Thomi Richards (thomir-deactivatedaccount) wrote :

Getting very close now, but still need to make a few changes

review: Needs Fixing
558. By Christopher Lee

Clarify docstrings, comments and variables

559. By Christopher Lee

Added SetTimezone fixture + tests for it

560. By Christopher Lee

Update tests skip tests only when the platform doesn't support the large time_t

561. By Christopher Lee

Clarified comment a little bit.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
562. By Christopher Lee

Added and clarified details around DateTime.timestamp

563. By Christopher Lee

Really minor comment change.

564. By Christopher Lee

Backout packaging changes to be put in a separate patch

565. By Christopher Lee

Cleanup double-up in debian/control

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Nicholas Skaggs (nskaggs) wrote :

I retested the original failing tests (both calendar as well as UITK) and confirmed this change fixes the test. We also tested the trunk version of these tests and the tests reacted the same way as trunk autopilot; no regressions on date handling.

review: Approve
566. By Christopher Lee

Minor whitespace fix

567. By Christopher Lee

Better name for Timezone fixture.

568. By Christopher Lee

Alter test to not use Europe/Moscow for now.

569. By Christopher Lee

Fix flake8 issue

Unmerged revisions

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'autopilot/introspection/types.py'
2--- autopilot/introspection/types.py 2014-05-29 19:44:44 +0000
3+++ autopilot/introspection/types.py 2014-10-22 20:43:57 +0000
4@@ -37,7 +37,9 @@
5
6 """
7
8-from datetime import datetime, time
9+from datetime import datetime, time, timedelta
10+from dateutil.tz import gettz, tzutc
11+
12 import dbus
13 import logging
14 from testtools.matchers import Equals
15@@ -547,6 +549,15 @@
16 """The DateTime class represents a date and time in the UTC timezone.
17
18 DateTime is constructed by passing a unix timestamp in to the constructor.
19+ The incoming timestamp is assumed to be in UTC.
20+
21+ .. note:: This class expects the passed in timestamp to be in UTC but will
22+ display the resulting date and time in local time (using the local
23+ timezone).
24+
25+ This is done to mimic the behaviour of most applications which will
26+ display date and time in local time by default
27+
28 Timestamps are expressed as the number of seconds since 1970-01-01T00:00:00
29 in the UTC timezone::
30
31@@ -588,10 +599,48 @@
32 Finally, you can also compare a DateTime instance with a python datetime
33 instance::
34
35- >>> my_datetime = datetime.datetime.fromutctimestamp(1377209927)
36- True
37-
38- DateTime instances can be converted to datetime instances:
39+ >>> my_datetime = datetime.datetime.utcfromtimestamp(1377209927)
40+ True
41+
42+
43+ .. note:: Autopilot supports dates beyond 2038 on 32-bit platforms. To
44+ achieve this the underlying mechanisms require to work with timezone aware
45+ datetime objects.
46+
47+ This means that the following won't always be true (due to the naive
48+ timestamp not having the correct daylight-savings time details)::
49+
50+ >>> # This time stamp is within DST in the 'Europe/London' timezone
51+ >>> dst_ts = 1405382400
52+ >>> os.environ['TZ'] ='Europe/London'
53+ >>> time.tzset()
54+ >>> datetime.fromtimestamp(dst_ts).hour == DateTime(dst_ts).hour
55+ False
56+
57+ But this will work::
58+
59+ >>> from dateutil.tz import gettz
60+ >>> datetime.fromtimestamp(
61+ dst_ts, gettz()).hour == DateTime(dst_ts).hour
62+ True
63+
64+ And this will always work to::
65+
66+ >>> dt1 = DateTime(nz_dst_timestamp)
67+ >>> dt2 = datetime(
68+ dt1.year, dt1.month, dt1.day, dt1.hour, dt1.minute, dt1.second
69+ )
70+ >>> dt1 == dt2
71+ True
72+
73+ .. note:: DateTime.timestamp() will not always equal the passed in
74+ timestamp.
75+ To paraphrase a message from [http://bugs.python.org/msg229393]
76+ "datetime.timestamp is supposed to be inverse of
77+ datetime.fromtimestamp(), but since the later is not monotonic, no such
78+ inverse exists in the strict mathematical sense."
79+
80+ DateTime instances can be converted to datetime instances::
81
82 >>> isinstance(my_dt.datetime, datetime.datetime)
83 True
84@@ -599,7 +648,41 @@
85 """
86 def __init__(self, *args, **kwargs):
87 super(DateTime, self).__init__(*args, **kwargs)
88- self._cached_dt = datetime.fromtimestamp(self[0])
89+ # Using timedelta in this manner is a workaround so that we can support
90+ # timestamps larger than the 32bit time_t limit on 32bit hardware.
91+ # We then apply another workaround where timedelta doesn't apply
92+ # daylight savings, so we need to work out the offsets for the
93+ # localtime manually and apply them to give us the correct local time.
94+ #
95+ # Note. self[0] is a UTC timestamp
96+ EPOCH = datetime(1970, 1, 1, tzinfo=tzutc())
97+ utc_dt = EPOCH + timedelta(seconds=self[0])
98+
99+ local_tzinfo = gettz()
100+
101+ # Get the localtimes timezone offset (known as standard offset) by
102+ # subtracting its dst offset (if any) from its utc offset.
103+ # We apply this to the utc datetime object to get datetime object in
104+ # localtime.
105+ # (We will check (once we have a local datetime) if the time is in dst
106+ # and make that adjustment then.)
107+ utc_offset = local_tzinfo.utcoffset(utc_dt)
108+ dst_offset = local_tzinfo.dst(utc_dt)
109+ standard_offset = utc_offset - dst_offset
110+
111+ # Create a local timezone aware datetime object from the utc_dt
112+ # (i.e. attaching a timezone to it) and apply the standard offset to
113+ # give us the local time.
114+ local_dt = utc_dt.replace(tzinfo=local_tzinfo) + standard_offset
115+
116+ # If the new local time is firmly in std time then the standard offset
117+ # will be 0 (i.e. timedelta(0)).
118+ # If the delta isn't 0 then we need to use the timezone information to
119+ # apply the dst delta to the local time.
120+ if standard_offset != timedelta(0):
121+ local_dt = local_dt + local_tzinfo.dst(local_dt)
122+
123+ self._cached_dt = local_dt
124
125 @property
126 def year(self):
127@@ -627,14 +710,21 @@
128
129 @property
130 def timestamp(self):
131- return self[0]
132+ return self._cached_dt.timestamp()
133
134 @property
135 def datetime(self):
136 return self._cached_dt
137
138 def __eq__(self, other):
139+ # A little 'magic' here, if the datetime object to test against is
140+ # naive, use the tzinfo from the cached datetime (just for the
141+ # comparison)
142 if isinstance(other, datetime):
143+ if other.tzinfo is None:
144+ return other.replace(
145+ tzinfo=self._cached_dt.tzinfo
146+ ) == self._cached_dt
147 return other == self._cached_dt
148 return super(DateTime, self).__eq__(other)
149
150
151=== modified file 'autopilot/tests/functional/fixtures.py'
152--- autopilot/tests/functional/fixtures.py 2014-05-23 13:44:36 +0000
153+++ autopilot/tests/functional/fixtures.py 2014-10-22 20:43:57 +0000
154@@ -25,8 +25,9 @@
155 from shutil import rmtree
156 import tempfile
157 from textwrap import dedent
158+import time
159
160-from fixtures import Fixture
161+from fixtures import EnvironmentVariable, Fixture
162
163
164 logger = logging.getLogger(__name__)
165@@ -166,3 +167,16 @@
166 with open(tmp_file_path, 'w') as desktop_file:
167 desktop_file.write(file_contents)
168 return tmp_file_path
169+
170+
171+class SetTimezone(Fixture):
172+ def __init__(self, timezone):
173+ self._timezone = timezone
174+
175+ def setUp(self):
176+ super().setUp()
177+ # These steps need to happen in the right order otherwise they won't
178+ # get cleaned up properly and we'll be left in an incorrect timezone.
179+ self.addCleanup(time.tzset)
180+ self.useFixture(EnvironmentVariable('TZ', self._timezone))
181+ time.tzset()
182
183=== modified file 'autopilot/tests/functional/test_types.py'
184--- autopilot/tests/functional/test_types.py 2014-05-22 06:41:16 +0000
185+++ autopilot/tests/functional/test_types.py 2014-10-22 20:43:57 +0000
186@@ -1,29 +1,99 @@
187+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
188+#
189+# Autopilot Functional Test Tool
190+# Copyright (C) 2013-2014 Canonical
191+#
192+# This program is free software: you can redistribute it and/or modify
193+# it under the terms of the GNU General Public License as published by
194+# the Free Software Foundation, either version 3 of the License, or
195+# (at your option) any later version.
196+#
197+# This program is distributed in the hope that it will be useful,
198+# but WITHOUT ANY WARRANTY; without even the implied warranty of
199+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
200+# GNU General Public License for more details.
201+#
202+# You should have received a copy of the GNU General Public License
203+# along with this program. If not, see <http://www.gnu.org/licenses/>.
204+#
205
206 from datetime import datetime
207
208 from autopilot.testcase import AutopilotTestCase
209 from autopilot.tests.functional import QmlScriptRunnerMixin
210+from autopilot.tests.functional.fixtures import SetTimezone
211
212 from textwrap import dedent
213
214
215-class TypeTests(AutopilotTestCase, QmlScriptRunnerMixin):
216-
217- def test_date(self):
218- proxy = self.start_qml_script(
219- dedent(
220- """\
221- import QtQuick 2.0
222-
223- Item {
224- objectName: "TestMePlease"
225- property date foo: "2014-01-01"
226+class DateTimeTests(AutopilotTestCase, QmlScriptRunnerMixin):
227+ scenarios = [
228+ ('UTC', dict(
229+ TZ='UTC',
230+ expected_string='2014-09-29T12:00:00',
231+ )),
232+ ('NZ', dict(
233+ TZ='Pacific/Auckland',
234+ expected_string='2014-09-30T01:00:00',
235+ )),
236+ ('US Central', dict(
237+ TZ='US/Central',
238+ expected_string='2014-09-29T07:00:00',
239+ )),
240+ ('US Eastern', dict(
241+ TZ='US/Eastern',
242+ expected_string='2014-09-29T08:00:00',
243+ )),
244+ ('MSK', dict(
245+ TZ='Europe/Moscow',
246+ expected_string='2014-09-29T16:00:00',
247+ )),
248+ ]
249+
250+ def get_test_qml_string(self, date_string):
251+ return dedent("""
252+ import QtQuick 2.0
253+ import QtQml 2.2
254+ Rectangle {
255+ property date testingTime: new Date(%s);
256+ Text {
257+ text: testingTime;
258 }
259- """
260- )
261- )
262- item = proxy.select_single('*', objectName="TestMePlease")
263+ }""" % date_string)
264+
265+ def test_qml_applies_timezone_to_timestamp(self):
266+ """Test that when given a timestamp the datetime displayed has the
267+ timezone applied to it.
268+
269+ QML will apply a timezone calculation to a timestamp (but not a
270+ timestring).
271+
272+ """
273+ self.useFixture(SetTimezone(self.TZ))
274+
275+ qml_script = self.get_test_qml_string('1411992000000')
276+
277+ proxy = self.start_qml_script(qml_script)
278 self.assertEqual(
279- item.foo,
280- datetime(2014, 1, 1, 0, 0, 0)
281+ proxy.select_single('QQuickText').text,
282+ self.expected_string
283 )
284+
285+ def test_timezone_not_applied_to_timestring(self):
286+ """Test that, in all timezones, the literal representation we get in
287+ the proxy object matches the one in the Qml script.
288+
289+ """
290+ self.useFixture(SetTimezone(self.TZ))
291+
292+ qml_script = self.get_test_qml_string("'2014-01-15 12:34:52'")
293+ proxy = self.start_qml_script(qml_script)
294+ date_object = proxy.select_single("QQuickRectangle").testingTime
295+
296+ self.assertEqual(date_object.year, 2014)
297+ self.assertEqual(date_object.month, 1)
298+ self.assertEqual(date_object.day, 15)
299+ self.assertEqual(date_object.hour, 12)
300+ self.assertEqual(date_object.minute, 34)
301+ self.assertEqual(date_object.second, 52)
302+ self.assertEqual(datetime(2014, 1, 15, 12, 34, 52), date_object)
303
304=== modified file 'autopilot/tests/unit/test_test_fixtures.py'
305--- autopilot/tests/unit/test_test_fixtures.py 2014-05-20 08:53:21 +0000
306+++ autopilot/tests/unit/test_test_fixtures.py 2014-10-22 20:43:57 +0000
307@@ -19,6 +19,7 @@
308
309 from autopilot.tests.functional.fixtures import (
310 ExecutableScript,
311+ SetTimezone,
312 TempDesktopFile,
313 )
314
315@@ -213,3 +214,22 @@
316 def test_creates_file_with_execute_bit_set(self):
317 fixture = self.useFixture(ExecutableScript(script=""))
318 self.assertTrue(os.stat(fixture.path).st_mode & stat.S_IXUSR)
319+
320+
321+class SetTimezoneTests(TestCase):
322+
323+ def test_sets_environment_variable_to_timezone(self):
324+ token = self.getUniqueString()
325+
326+ self.useFixture(SetTimezone(token))
327+
328+ self.assertEqual(os.environ.get('TZ'), token)
329+
330+ def test_resets_timezone_back_to_original(self):
331+ original_tz = os.environ.get('TZ', None)
332+ token = self.getUniqueString()
333+
334+ fixture = self.useFixture(SetTimezone(token))
335+ fixture.cleanUp()
336+
337+ self.assertEqual(os.environ.get('TZ', None), original_tz)
338
339=== modified file 'autopilot/tests/unit/test_types.py'
340--- autopilot/tests/unit/test_types.py 2014-07-22 02:39:26 +0000
341+++ autopilot/tests/unit/test_types.py 2014-10-22 20:43:57 +0000
342@@ -18,13 +18,14 @@
343 #
344
345 from datetime import datetime, time
346-from testscenarios import TestWithScenarios
347+from testscenarios import TestWithScenarios, multiply_scenarios
348 from testtools import TestCase
349 from testtools.matchers import Equals, IsInstance, NotEquals, raises
350
351 import dbus
352 from unittest.mock import patch, Mock
353
354+from autopilot.tests.functional.fixtures import SetTimezone
355 from autopilot.introspection.types import (
356 Color,
357 create_value_instance,
358@@ -51,6 +52,8 @@
359 from autopilot.introspection.dbus import DBusIntrospectionObject
360 from autopilot.utilities import compatible_repr
361
362+from dateutil import tz
363+
364
365 class PlainTypeTests(TestWithScenarios, TestCase):
366
367@@ -281,19 +284,32 @@
368 self.assertEqual(repr(c), str(c))
369
370
371-class DateTimeTests(TestCase):
372+def unable_to_handle_timestamp(timestamp):
373+ """Return false if the platform can handle timestamps larger than 32bit
374+ limit.
375+
376+ """
377+ try:
378+ datetime.fromtimestamp(timestamp)
379+ return False
380+ except:
381+ return True
382+
383+
384+class DateTimeCreationTests(TestCase):
385+
386+ timestamp = 1405382400 # No significance, just a timestamp
387
388 def test_can_construct_datetime(self):
389- dt = DateTime(1377209927)
390+ dt = DateTime(self.timestamp)
391 self.assertThat(dt, IsInstance(dbus.Array))
392
393 def test_datetime_has_slice_access(self):
394- dt = DateTime(1377209927)
395-
396- self.assertThat(dt[0], Equals(1377209927))
397+ dt = DateTime(self.timestamp)
398+ self.assertThat(dt[0], Equals(self.timestamp))
399
400 def test_datetime_has_properties(self):
401- dt = DateTime(1377209927)
402+ dt = DateTime(self.timestamp)
403
404 self.assertTrue(hasattr(dt, 'timestamp'))
405 self.assertTrue(hasattr(dt, 'year'))
406@@ -303,58 +319,154 @@
407 self.assertTrue(hasattr(dt, 'minute'))
408 self.assertTrue(hasattr(dt, 'second'))
409
410+ def test_repr(self):
411+ # Use a well known timezone for comparison
412+ self.useFixture(SetTimezone('UTC'))
413+ dt = DateTime(self.timestamp)
414+ observed = repr(dt)
415+
416+ expected = "DateTime({:%Y-%m-%d %H:%M:%S})".format(
417+ datetime.fromtimestamp(self.timestamp)
418+ )
419+ self.assertEqual(expected, observed)
420+
421+ def test_repr_equals_str(self):
422+ dt = DateTime(self.timestamp)
423+ self.assertEqual(repr(dt), str(dt))
424+
425+ def test_can_create_DateTime_using_large_timestamp(self):
426+ """Must be able to create a DateTime object using a timestamp larger
427+ than the 32bit time_t limit.
428+
429+ """
430+ # Use a well known timezone for comparison
431+ self.useFixture(SetTimezone('UTC'))
432+ large_timestamp = 2**32+1
433+ dt = DateTime(large_timestamp)
434+
435+ self.assertEqual(dt.year, 2106)
436+ self.assertEqual(dt.month, 2)
437+ self.assertEqual(dt.day, 7)
438+ self.assertEqual(dt.hour, 6)
439+ self.assertEqual(dt.minute, 28)
440+ self.assertEqual(dt.second, 17)
441+ self.assertEqual(dt.timestamp, large_timestamp)
442+
443+
444+class DateTimeTests(TestWithScenarios, TestCase):
445+
446+ timestamps = [
447+ # This timestamp uncovered an issue during development.
448+ ('Explicit US/Pacific test', dict(
449+ timestamp=1090123200
450+ )),
451+
452+ ('NZ DST example', dict(
453+ timestamp=2047570047
454+ )),
455+
456+ ('Winter', dict(
457+ timestamp=1389744000
458+ )),
459+
460+ ('Summer', dict(
461+ timestamp=1405382400
462+ )),
463+
464+ ('32bit max', dict(
465+ timestamp=2**32+1
466+ )),
467+
468+ ('32bit limit', dict(
469+ timestamp=2983579200
470+ )),
471+
472+ ]
473+
474+ timezones = [
475+ ('UTC', dict(
476+ timezone='UTC'
477+ )),
478+
479+ ('London', dict(
480+ timezone='Europe/London'
481+ )),
482+
483+ ('New Zealand', dict(
484+ timezone='NZ',
485+ )),
486+
487+ ('Pacific', dict(
488+ timezone='US/Pacific'
489+ )),
490+
491+ ('Hongkong', dict(
492+ timezone='Hongkong'
493+ )),
494+
495+ ('Moscow', dict(
496+ timezone='Europe/Moscow'
497+ ))
498+ ]
499+
500+ scenarios = multiply_scenarios(timestamps, timezones)
501+
502+ def skip_if_timestamp_too_large(self, timestamp):
503+ if unable_to_handle_timestamp(self.timestamp):
504+ self.skip("Timestamp to large for platform time_t")
505+
506 def test_datetime_properties_have_correct_values(self):
507- dt = DateTime(1377209927)
508- dt_with_tz = datetime.fromtimestamp(1377209927)
509-
510- self.assertThat(dt.timestamp, Equals(dt_with_tz.timestamp()))
511- self.assertThat(dt.year, Equals(dt_with_tz.year))
512- self.assertThat(dt.month, Equals(dt_with_tz.month))
513- self.assertThat(dt.day, Equals(dt_with_tz.day))
514- self.assertThat(dt.hour, Equals(dt_with_tz.hour))
515- self.assertThat(dt.minute, Equals(dt_with_tz.minute))
516- self.assertThat(dt.second, Equals(dt_with_tz.second))
517+ self.skip_if_timestamp_too_large(self.timestamp)
518+ self.useFixture(SetTimezone(self.timezone))
519+
520+ dt1 = DateTime(self.timestamp)
521+ dt2 = datetime.fromtimestamp(self.timestamp, tz.gettz())
522+
523+ self.assertThat(dt1.year, Equals(dt2.year))
524+ self.assertThat(dt1.month, Equals(dt2.month))
525+ self.assertThat(dt1.day, Equals(dt2.day))
526+ self.assertThat(dt1.hour, Equals(dt2.hour))
527+ self.assertThat(dt1.minute, Equals(dt2.minute))
528+ self.assertThat(dt1.second, Equals(dt2.second))
529+ self.assertThat(dt1.timestamp, Equals(dt2.timestamp()))
530
531 def test_equality_with_datetime(self):
532- dt1 = DateTime(1377209927)
533- dt2 = DateTime(1377209927)
534+ self.skip_if_timestamp_too_large(self.timestamp)
535+ self.useFixture(SetTimezone(self.timezone))
536+
537+ dt1 = DateTime(self.timestamp)
538+ dt2 = datetime(
539+ dt1.year, dt1.month, dt1.day, dt1.hour, dt1.minute, dt1.second
540+ )
541
542 self.assertThat(dt1, Equals(dt2))
543
544 def test_equality_with_list(self):
545- dt1 = DateTime(1377209927)
546- dt2 = [1377209927]
547+ self.skip_if_timestamp_too_large(self.timestamp)
548+ self.useFixture(SetTimezone(self.timezone))
549+
550+ dt1 = DateTime(self.timestamp)
551+ dt2 = [self.timestamp]
552
553 self.assertThat(dt1, Equals(dt2))
554
555- def test_equality_with_datetime_timestamp(self):
556- # DateTime no longer assumes UTC and uses local TZ.
557- dt1 = DateTime(1377209927)
558- dt2 = datetime.fromtimestamp(1377209927)
559- dt3 = datetime.fromtimestamp(1377209928)
560+ def test_equality_with_datetime_object(self):
561+ self.skip_if_timestamp_too_large(self.timestamp)
562+ self.useFixture(SetTimezone(self.timezone))
563+
564+ dt1 = DateTime(self.timestamp)
565+ dt2 = datetime.fromtimestamp(self.timestamp, tz.gettz())
566+ dt3 = datetime.fromtimestamp(self.timestamp + 1, tz.gettz())
567
568 self.assertThat(dt1, Equals(dt2))
569 self.assertThat(dt1, NotEquals(dt3))
570
571 def test_can_convert_to_datetime(self):
572- dt1 = DateTime(1377209927)
573+ self.skip_if_timestamp_too_large(self.timestamp)
574
575+ dt1 = DateTime(self.timestamp)
576 self.assertThat(dt1.datetime, IsInstance(datetime))
577
578- def test_repr(self):
579- expected = repr_type(
580- u"DateTime({:%Y-%m-%d %H:%M:%S})".format(
581- datetime.fromtimestamp(1377209927)
582- )
583- )
584- dt = DateTime(1377209927)
585- observed = repr(dt)
586- self.assertEqual(expected, observed)
587-
588- def test_repr_equals_str(self):
589- dt = DateTime(1377209927)
590- self.assertEqual(repr(dt), str(dt))
591-
592
593 class TimeTests(TestCase):
594
595
596=== modified file 'debian/control'
597--- debian/control 2014-08-05 21:58:41 +0000
598+++ debian/control 2014-10-22 20:43:57 +0000
599@@ -15,6 +15,7 @@
600 libjs-underscore,
601 liblttng-ust-dev,
602 python3-all-dev (>= 3.4),
603+ python3-dateutil,
604 python3-dbus,
605 python3-decorator,
606 python3-evdev,
607@@ -44,6 +45,7 @@
608 ${python3:Depends},
609 gir1.2-ubuntu-app-launch-2 | gir1.2-upstart-app-launch-2,
610 libjs-underscore, libjs-jquery,
611+ python3-dateutil,
612 python3-dbus,
613 python3-decorator,
614 python3-fixtures,
615@@ -135,6 +137,7 @@
616 libautopilot-gtk (>= 1.4),
617 libautopilot-qt (>= 1.4),
618 python3-autopilot,
619+ python3-dateutil,
620 python3-dbus.mainloop.qt,
621 python3-evdev,
622 python3-pyqt4,

Subscribers

People subscribed via source and target branches