date returns "invalid date" for some timezone's DST

Thanks for taking the time to report this bug and helping to make Ubuntu better. We appreciate the difficulties you are facing, but this appears to be a "regular" (non-security) bug. I have unmarked it as a security issue since this bug does not show evidence of allowing attackers to cross privilege boundaries nor directly cause loss of data/privacy. Please feel free to report any other bugs you may find.

If this isn't a security bug on date, it is, at least, a security bug on apt, since, as I said, it makes well configured servers out of date.

Can you please give us:

1. the version of Ubuntu you are running;
2. the output for 'date --version'
3. Your locale (run 'locale', and paste the output here).

Per the output you show I would guess you are in a PT locale, but I do not know if pt_BR or pt_PT.

I have tried your commands with both date 6.10 and 7.2.10, under the en_US and pt_PT locales, and could not reproduce it (for the US DST).

Can you also give us what exactly went wrong on your system(s)? Which component failed, and how did it fail? I cannot see, at least on my systems, any cron entries for root.

1. Running Ubuntu Server 8.04.2
2. date (GNU coreutils) 6.10
3. locale
LANG=pt_BR.UTF-8
LANGUAGE=pt_BR:pt:en
LC_CTYPE="pt_BR.UTF-8"
LC_NUMERIC="pt_BR.UTF-8"
LC_TIME="pt_BR.UTF-8"
LC_COLLATE="pt_BR.UTF-8"
LC_MONETARY="pt_BR.UTF-8"
LC_MESSAGES="pt_BR.UTF-8"
LC_PAPER="pt_BR.UTF-8"
LC_NAME="pt_BR.UTF-8"
LC_ADDRESS="pt_BR.UTF-8"
LC_TELEPHONE="pt_BR.UTF-8"
LC_MEASUREMENT="pt_BR.UTF-8"
LC_IDENTIFICATION="pt_BR.UTF-8"
LC_ALL=

It happens when using BRT timezone.

I found the same bug reported here:
http://lists.gnu.org/archive/html/bug-coreutils/2007-09/msg00176.html

This causes the script /etc/cron.daily/apt fail on line 53:
stamp=$(date --date=$(date -r $stamp --iso-8601) +%s)

In this script, $stamp assumes, among others, the values:
/var/lib/apt/periodic/update-stamp
/var/lib/apt/periodic/upgrade-stamp
/var/lib/apt/periodic/autoclean-stamp

The biggest problem is, if these files where last modified on the dates that triggers the bug, it makes the script exit, without even touching the files again. So, the system is never more updated periodically.

/etc/cron.daily/apt needs to be more robust in case the stamp files have an invalid date. The upstream bug clearly indicates that the developers don't intend to fix this bug, so /etc/cron.daily/apt needs to better handle this condition.

Marking coreutils bug as "Won't Fix" as upstream isn't fixing it because they feel that returning 'Invalid date' is correct behavior.

@Alexandre: thank you for bringing this to our attention. I do not believe 'date' behaviour will be changed (since it is, after all, correct), but an internal discussion with the security folks got us back as a security issue.

Please do keep on bringing us any issues you find.

FYI-- an affected timezone is America/Sao_Paulo (at least a few others in BRT are not affected). With America/Sao_Paulo, have (locale does not matter):
$ date --date=2009-10-18
date: invalid date `2009-10-18'

Here is a possible fix for apt. Basically, if date exits with error when getting the data on the stamp file, remove the stamp file and continue as if the stamp file didn't exist. This should allow apt to recover regardless of DST or date bugs.

I am not sure why I also got coreutils nominated for releases... sorry. I do not know if Dapper/Gutsy are also affected (don't have them).

About the fix, I think $now should also be fixed, as it will be invalid on these dates.

Regarding coreutils, I really can't understand how this can be considered the correct behavior. If it accepts dates formated as "yyyy-mm-dd", it should handle it the best as possible. I wouldn't be surprised to find other scripts that rely on this. And, if you decide this is really correct, there must be some kind of warning somewhere, so that other programmers don't need to predict such strange behavior.

Alexandre: 'now' shouldn't need to be fixed, because date will get the system time and convert it to the proper date. The problem comes with 'stamp' using 'date -r' on a file on the system, which could conceivably have the wrong date. Please correct me if I missed something.

As for coreutils, upstream (ie, not me or Ubuntu) has decided that the time in question and other dates such as 'Feb 30' are wrong because they don't exist and choose to express that as an error condition. As such, we need to adjust our tools/scripts to account for this.

Alexandre: I see the issue with 'now' now. Since we use 'date --iso-8601' and feed that to date, then the hour, minute and second are all zero. My patch doesn't properly address using --iso-8601 for stamp either. I'll send an updated patch in a few minutes.

I agree that other scripts may probably also be hit by this type of error. Nevertheless, 'date' *does* return a non-zero return code if an error if found; *not* checking if a command ended in error is a much more serious programming error than this, ah, "invalid" date returned. With that said, if 'date' did not return a non-zero RC on this... then *this* is certainly a bug, no doubt.

The behaviour of 'date' is well-documented (or, at least, well, documented), and this type of 'invalid' return can happen. We can discuss upstream (in the same mailing list you quoted) -- this would be the best venue. I can see -- and, again, quoting from the email thread you provided -- that a case that 'date' should return the first available time in a date, instead or erroring out if 00:00:00 (or some other time/date/TZ combination) does not exist.

Just be aware that this will be a "short blanket" thing, since (now) the apt cron scripts will work, but somewhere out in the wild world someone will have a script that depends on the day beginning at 00:00:00... and this will now fail!

Here is an updated patch. It should now properly handle --iso-8601 for 'stamp' and 'now'. This patch is against 0.7.20.2ubuntu5 in Jaunty.

Here is a simple test case for the bug. It will return '2' if the stamp file has a bad date, '1' if 'now' is a bad date (this is hardcoded) and '0' if all is well. The code is the same as in the patch, except for the return codes and hardcoded 'now'. Note, you need to have your timezone set to America/San_Paulo.

Here is a tar file which contains the file 'foo', with a date that will trigger the bug. To use with the test case:

$ tar -xvf ./354793.tar
foo
tar: foo: time stamp 2009-10-18 01:00:00 is 16613753.588577226 s in the future
$ ./354793_test.sh ./foo ; echo "$?"
2
$ ./354793_test.sh ./foo ; echo "$?"
date: ./foo: No such file or directory
1

The first run removed the file, and the second checked the hard coded date.

This bug was fixed in the package apt - 0.7.20.2ubuntu6

---------------
apt (0.7.20.2ubuntu6) jaunty; urgency=low

  [ Jamie Strandboge ]
  * apt.cron.daily: catch invalid dates due to DST time changes
    in the stamp files (LP: #354793)

  [ Michael Vogt ]
  * methods/gpgv.cc:
    - properly check for expired and revoked keys (closes: #433091)
      LP: #356012

 -- Michael Vogt <email address hidden> Wed, 08 Apr 2009 22:39:50 +0200

I won't have an upload for Gutsy, because I won't be publishing this until next week, and Gutsy will be EOL.

http://www.ubuntu.com/usn/usn-762-1

same error:
date: invalid date `2009/10/18'

==================================================================================
Linux hp-work 2.6.28-16-generic #55-Ubuntu SMP Tue Oct 20 19:48:32 UTC 2009 x86_64 GNU/Linux
-----------------------------------------------------
date --version
date (GNU coreutils) 6.10
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by David MacKenzie.
---------------------------------------------------------------------------------

I have a problem setting the date on:

gep@gep-work:~$ sudo date 03280243
date: invalid date `03280243'

But:

gep@gep-work:~$ sudo date 03280343
Sun Mar 28 03:43:00 EEST 2010

It is still not working as expected with Lucid.

And it's breaking a script of mine, sending every instance to an endless, CPU eating loop…

It happens on Ubuntu 14.04 too.

# date
Sun Oct 18 02:08:18 BRST 2015

# date --date=2015-10-18
date: invalid date ‘2015-10-18’

# date --date=2015-10-19
Mon Oct 19 00:00:00 BRST 2015

# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.3 LTS
Release: 14.04
Codename: trusty