Merge lp:~richard-wilbur/bzr/1537319-reproducible-builds into lp:bzr

Proposed by Richard Wilbur on 2016-02-01
Status: Work in progress
Proposed branch: lp:~richard-wilbur/bzr/1537319-reproducible-builds
Merge into: lp:bzr
Diff against target: 630 lines (+254/-84)
10 files modified
Makefile (+45/-34)
bzrlib/builtins.py (+2/-2)
bzrlib/cmd_version_info.py (+14/-1)
bzrlib/doc_generate/autodoc_bash_completion.py (+25/-13)
bzrlib/doc_generate/autodoc_man.py (+44/-16)
bzrlib/doc_generate/autodoc_rstx.py (+40/-15)
bzrlib/tests/test_version_info.py (+30/-1)
bzrlib/version_info_formats/__init__.py (+6/-1)
bzrlib/version_info_formats/format_source_date_epoch.py (+45/-0)
doc/en/Makefile (+3/-1)
To merge this branch: bzr merge lp:~richard-wilbur/bzr/1537319-reproducible-builds
Reviewer Review Type Date Requested Status
Vincent Ladeuil 2016-02-01 Pending
Review via email: mp+284564@code.launchpad.net

Description of the change

Changes to support GNU and debian reproducible builds effort: Add functionality to version-info to make populating SOURCE_DATE_EPOCH trivial on a bzr branch. Modify build and packaging code and rules to make bzr deliver reproducible builds for the same build environment.
Fixes lp:1537319

To post a comment you must log in.
Richard Wilbur (richard-wilbur) wrote :

I have nothing against 'bzr version-info --source-date-epoch' but I figured some folks might complain about the length of the required command line so I implemented (and documented) a hard-coded alias 'bzr sde' for that command. This is for those who value compactness over verbosity.

6615. By Richard Wilbur on 2016-02-01

Change makefile rules for reproducible input order. Use SOURCE_DATE_EPOCH for generating timestamps and datestamps, if available. Otherwise forgo timestamps and datestamps. Use SOURCE_DATE_EPOCH to clamp file modification times when generating tarball. Use root user and group as numeric id 0 when generating tarball.

6616. By Richard Wilbur on 2016-02-01

Fix creation of date and time stamps. man pages always have a datestamp in title so, in the absence of SOURCE_DATE_EPOCH, give the impossible date of 1 second before the UNIX epoch.

Vincent Ladeuil (vila) wrote :

Hmm.

This looks like a rabbit hole :-/

Overall, this is going in the right direction but I think this will end up in several MPs.

There is duplicated code you're modifying which is worth refactoring as are the duplicate doc Makefiles.

Not to mention the comments (I feel your pain :-/) you had to modify to maintain consistency.

And we need more tests :) At least a setup where we can build all docs and see the results to start with.

Richard Wilbur (richard-wilbur) wrote :
Download full text (3.6 KiB)

I'm happy to accommodate multiple merge proposals but am in need of counsel on how best to accomplish that. Should I shelve everything except one changeset, polish, push to launchpad, propose the merge, review, merge, and repeat? Using the same branch? Should I break the changes into separate branches?

I see the main changes here as:
revision 6614: Add explicit support to bzr for creating SOURCE_DATE_EPOCH from a bzr branch. (If no version information is available, presently returns 0 => *nix epoch. Would a different value be more suitable?)

revision 6615:
./Makefile: Change make rules to sort the file lists for constant file order in tar file creation. Also use numeric user and group ids, root/root => 0/0 and clamp modification time to SOURCE_DATE_EPOCH when creating tar files. (Set SOURCE_DATE_EPOCH with `bzr sde` if not already set.)

./doc/en/Makefile: Use (or set with `bzr sde`) SOURCE_DATE_EPOCH to create BUILD_DATE for sphinx documentation build.

./bzrlib/doc_generate/autodoc_bash_completion.py: Try to get SOURCE_DATE_EPOCH and, if found, use it to create timestamp and include 'Last commit: %' in output file. Retitled from 'Generation time' to 'Last commit' as it seemed more descriptive.

./bzrlib/doc_generate/autodoc_man.py: Try to get SOURCE_DATE_EPOCH and, if found, use it to create date/time stamps. Split preamble and head so that datestamp and timestamp are only output when SOURCE_DATE_EPOCH is available.

./bzrlib/doc_generate/autodoc_rstx.py: Try to get SOURCE_DATE_EPOCH and, if found, use it to create date/time stamps. Split preamble so that timestamp is output when SOURCE_DATE_EPOCH is available.

revision 6616: Fixed syntax of date/time stamp creation in bzrlib/doc_generate/autodoc_*.py. Changed man head to always use datestamp but in the absence of SOURCE_DATE_EPOCH, use 1 second before the *nix epoch (valid, impossible date).

1. I see the utility of factoring common code out of the bzrlib/doc_generate/autodoc_*.py into bzrlib/doc_generate/__init__.py
If we decide to substitute a flag date/time for missing SOURCE_DATE_EPOCH information, this will greatly simplify the process of factoring out the common code (more will be common), and it will also simplify the code which creates the documentation as there won't be as many conditionally included pieces.

If so, what would be a good, fixed value to use as a flag date/time? I notice that I already return "0" from `bzr sde` if there is no revision information on the branch and use "-1" in the event we are missing SOURCE_DATE_EPOCH in the man documentation generator. I think a case could be made for these being materially similar situations: no revision information available. So I think it would benefit us to use a consistent flag for this situation. I like "-1" as it should never occur as a valid source date. The only issue would possibly occur if SOURCE_DATE_EPOCH were to be treated as an unsigned integer, in which case "0" would be preferrable. I expect "0" is pretty much equally unlikely to be a valid source date. (I suppose in Python we are relatively unlikely to ever find SOURCE_DATE_EPOCH treated as an unsigned integer. If we are interested i...

Read more...

Jelmer Vernooij (jelmer) wrote :

Hi Richard,

Please split this up into separate merge proposals. At least for Debian, most of these changes are unnecessary. See https://tests.reproducible-builds.org/rb-pkg/unstable/amd64/bzr.html

Richard Wilbur (richard-wilbur) wrote :

Separate merge proposals on the same branch or do I need to create separate branches?

Unmerged revisions

6616. By Richard Wilbur on 2016-02-01

Fix creation of date and time stamps. man pages always have a datestamp in title so, in the absence of SOURCE_DATE_EPOCH, give the impossible date of 1 second before the UNIX epoch.

6615. By Richard Wilbur on 2016-02-01

Change makefile rules for reproducible input order. Use SOURCE_DATE_EPOCH for generating timestamps and datestamps, if available. Otherwise forgo timestamps and datestamps. Use SOURCE_DATE_EPOCH to clamp file modification times when generating tarball. Use root user and group as numeric id 0 when generating tarball.

6614. By Richard Wilbur on 2016-02-01

Add source-date-epoch format for version-info command (and alias sde) to support reproducible builds.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'Makefile'
2--- Makefile 2012-03-09 16:48:55 +0000
3+++ Makefile 2016-02-01 16:52:40 +0000
4@@ -1,4 +1,4 @@
5-# Copyright (C) 2005-2011 Canonical Ltd
6+# Copyright (C) 2005-2011, 2016 Canonical Ltd
7 #
8 # This program is free software; you can redistribute it and/or modify
9 # it under the terms of the GNU General Public License as published by
10@@ -28,6 +28,10 @@
11 PLUGIN_TARGET=plugin-release
12 PYTHON_BUILDFLAGS=
13
14+# Shorter replacement for $(sort $(wildcard <arg>)) as $(call sw,<arg>)
15+sw = $(sort $(wildcard $(1)))
16+
17+
18 .PHONY: all clean realclean extensions pyflakes api-docs check-nodocs check
19
20 all: extensions
21@@ -106,10 +110,10 @@
22 ### Man-page Documentation ###
23
24 MAN_DEPENDENCIES = bzrlib/builtins.py \
25- $(wildcard bzrlib/*.py) \
26- $(wildcard bzrlib/*/*.py) \
27+ $(call sw,bzrlib/*.py) \
28+ $(call sw,bzrlib/*/*.py) \
29 tools/generate_docs.py \
30- $(wildcard $(addsuffix /*.txt, bzrlib/help_topics/en))
31+ $(call sw,$(addsuffix /*.txt, bzrlib/help_topics/en))
32
33 MAN_PAGES = man1/bzr.1
34 man1/bzr.1: $(MAN_DEPENDENCIES)
35@@ -145,7 +149,7 @@
36 doc/developers/Makefile \
37 doc/developers/make.bat
38
39-NEWS_FILES = $(wildcard doc/en/release-notes/bzr-*.txt)
40+NEWS_FILES = $(call sw,doc/en/release-notes/bzr-*.txt)
41
42 doc/en/user-reference/index.txt: $(MAN_DEPENDENCIES)
43 $(PYTHON) tools/generate_docs.py -o $@ rstx
44@@ -229,19 +233,19 @@
45 doc/en/tutorials/tutorial.txt \
46 doc/en/tutorials/using_bazaar_with_launchpad.txt \
47 doc/en/tutorials/centralized_workflow.txt \
48- $(wildcard doc/es/tutorials/*.txt) \
49- $(wildcard doc/ru/tutorials/*.txt) \
50+ $(call sw,doc/es/tutorials/*.txt) \
51+ $(call sw,doc/ru/tutorials/*.txt) \
52 doc/ja/tutorials/tutorial.txt \
53 doc/ja/tutorials/using_bazaar_with_launchpad.txt \
54 doc/ja/tutorials/centralized_workflow.txt \
55- $(wildcard doc/*/mini-tutorial/index.txt) \
56- $(wildcard doc/*/user-guide/index-plain.txt) \
57+ $(call sw,doc/*/mini-tutorial/index.txt) \
58+ $(call sw,doc/*/user-guide/index-plain.txt) \
59 doc/en/admin-guide/index-plain.txt \
60- $(wildcard doc/es/guia-usario/*.txt) \
61+ $(call sw,doc/es/guia-usario/*.txt) \
62 $(derived_txt_files) \
63 doc/en/upgrade-guide/index.txt \
64 doc/index.txt \
65- $(wildcard doc/index.*.txt)
66+ $(call sw,doc/index.*.txt)
67 txt_nohtml = \
68 doc/en/user-guide/index.txt \
69 doc/es/user-guide/index.txt \
70@@ -253,16 +257,16 @@
71
72 non_txt_files = \
73 doc/default.css \
74- $(wildcard doc/*/bzr-en-quick-reference.svg) \
75- $(wildcard doc/*/bzr-en-quick-reference.png) \
76- $(wildcard doc/*/bzr-en-quick-reference.pdf) \
77- $(wildcard doc/*/bzr-es-quick-reference.svg) \
78- $(wildcard doc/*/bzr-es-quick-reference.png) \
79- $(wildcard doc/*/bzr-es-quick-reference.pdf) \
80- $(wildcard doc/*/bzr-ru-quick-reference.svg) \
81- $(wildcard doc/*/bzr-ru-quick-reference.png) \
82- $(wildcard doc/*/bzr-ru-quick-reference.pdf) \
83- $(wildcard doc/*/user-guide/images/*.png)
84+ $(call sw,doc/*/bzr-en-quick-reference.svg) \
85+ $(call sw,doc/*/bzr-en-quick-reference.png) \
86+ $(call sw,doc/*/bzr-en-quick-reference.pdf) \
87+ $(call sw,doc/*/bzr-es-quick-reference.svg) \
88+ $(call sw,doc/*/bzr-es-quick-reference.png) \
89+ $(call sw,doc/*/bzr-es-quick-reference.pdf) \
90+ $(call sw,doc/*/bzr-ru-quick-reference.svg) \
91+ $(call sw,doc/*/bzr-ru-quick-reference.png) \
92+ $(call sw,doc/*/bzr-ru-quick-reference.pdf) \
93+ $(call sw,doc/*/user-guide/images/*.png)
94
95 # doc/developers/*.txt files that should *not* be individually
96 # converted to HTML
97@@ -292,20 +296,20 @@
98 doc/developers/status.txt \
99 doc/developers/uncommit.txt
100
101-dev_txt_all = $(wildcard $(addsuffix /*.txt, doc/developers))
102+dev_txt_all = $(call sw,$(addsuffix /*.txt, doc/developers))
103 dev_txt_files = $(filter-out $(dev_txt_nohtml), $(dev_txt_all))
104 dev_htm_files = $(patsubst %.txt, %.html, $(dev_txt_files))
105
106-doc/en/user-guide/index-plain.html: $(wildcard $(addsuffix /*.txt, doc/en/user-guide))
107+doc/en/user-guide/index-plain.html: $(call sw,$(addsuffix /*.txt, doc/en/user-guide))
108 $(rst2html) --stylesheet=../../default.css $(dir $@)index-plain.txt $@
109
110-#doc/es/user-guide/index.html: $(wildcard $(addsuffix /*.txt, doc/es/user-guide))
111-# $(rst2html) --stylesheet=../../default.css $(dir $@)index.txt $@
112-#
113-#doc/ru/user-guide/index.html: $(wildcard $(addsuffix /*.txt, doc/ru/user-guide))
114-# $(rst2html) --stylesheet=../../default.css $(dir $@)index.txt $@
115-#
116-doc/en/admin-guide/index-plain.html: $(wildcard $(addsuffix /*.txt, doc/en/admin-guide))
117+#doc/es/user-guide/index.html: $(call sw,$(addsuffix /*.txt, doc/es/user-guide))
118+# $(rst2html) --stylesheet=../../default.css $(dir $@)index.txt $@
119+#
120+#doc/ru/user-guide/index.html: $(call sw,$(addsuffix /*.txt, doc/ru/user-guide))
121+# $(rst2html) --stylesheet=../../default.css $(dir $@)index.txt $@
122+#
123+doc/en/admin-guide/index-plain.html: $(call sw,$(addsuffix /*.txt, doc/en/admin-guide))
124 $(rst2html) --stylesheet=../../default.css $(dir $@)index-plain.txt $@
125
126 doc/developers/%.html: doc/developers/%.txt
127@@ -323,7 +327,7 @@
128 doc/en/release-notes/NEWS.txt: $(NEWS_FILES) tools/generate_release_notes.py
129 $(PYTHON) tools/generate_release_notes.py "$@" $(NEWS_FILES)
130
131-upgrade_guide_dependencies = $(wildcard $(addsuffix /*.txt, doc/en/upgrade-guide))
132+upgrade_guide_dependencies = $(call sw,$(addsuffix /*.txt, doc/en/upgrade-guide))
133
134 doc/en/upgrade-guide/index.html: $(upgrade_guide_dependencies)
135 $(rst2html) --stylesheet=../../default.css $(dir $@)index.txt $@
136@@ -435,8 +439,9 @@
137 update-pot: po/bzr.pot
138
139 TRANSLATABLE_PYFILES:=$(shell find bzrlib -name '*.py' \
140- | grep -v 'bzrlib/tests/' \
141- | grep -v 'bzrlib/doc' \
142+ | grep -v 'bzrlib/tests/' \
143+ | grep -v 'bzrlib/doc' \
144+ | LC_ALL=C sort \
145 )
146
147 po/bzr.pot: $(PYFILES) $(DOCFILES)
148@@ -453,6 +458,12 @@
149
150 .PHONY: dist check-dist-tarball
151
152+SOURCE_DATE_EPOCH ?= $(shell bzr sde)
153+TAR_OPTS = "--sort=name --owner=root --group=root --numeric-owner"
154+ifdef SOURCE_DATE_EPOCH
155+TAR_OPTS += "--mtime='$SOURCE_DATE_EPOCH' --clamp-mtime"
156+endif
157+
158 # build a distribution source tarball
159 #
160 # this method of copying the pyrex generated files is a bit ugly; it would be
161@@ -467,7 +478,7 @@
162 $(MAKE) && \
163 bzr export $$expdir && \
164 cp bzrlib/*.c bzrlib/*.h $$expdir/bzrlib/. && \
165- tar cfz $$tarball -C $$expbasedir bzr-$$version && \
166+ tar $(TAR_OPTS) cfz $$tarball -C $$expbasedir bzr-$$version && \
167 gpg --detach-sign $$tarball && \
168 rm -rf $$expbasedir
169
170
171=== modified file 'bzrlib/builtins.py'
172--- bzrlib/builtins.py 2013-05-23 10:04:17 +0000
173+++ bzrlib/builtins.py 2016-02-01 16:52:40 +0000
174@@ -1,4 +1,4 @@
175-# Copyright (C) 2005-2012 Canonical Ltd
176+# Copyright (C) 2005-2012, 2016 Canonical Ltd
177 #
178 # This program is free software; you can redistribute it and/or modify
179 # it under the terms of the GNU General Public License as published by
180@@ -6724,7 +6724,7 @@
181 ('cmd_bundle_info', [], 'bzrlib.bundle.commands'),
182 ('cmd_config', [], 'bzrlib.config'),
183 ('cmd_dpush', [], 'bzrlib.foreign'),
184- ('cmd_version_info', [], 'bzrlib.cmd_version_info'),
185+ ('cmd_version_info', ['sde'], 'bzrlib.cmd_version_info'),
186 ('cmd_resolve', ['resolved'], 'bzrlib.conflicts'),
187 ('cmd_conflicts', [], 'bzrlib.conflicts'),
188 ('cmd_ping', [], 'bzrlib.smart.ping'),
189
190=== modified file 'bzrlib/cmd_version_info.py'
191--- bzrlib/cmd_version_info.py 2011-12-30 12:52:54 +0000
192+++ bzrlib/cmd_version_info.py 2016-02-01 16:52:40 +0000
193@@ -1,4 +1,4 @@
194-# Copyright (C) 2005-2011 Canonical Ltd
195+# Copyright (C) 2005-2011, 2016 Canonical Ltd
196 #
197 # This program is free software; you can redistribute it and/or modify
198 # it under the terms of the GNU General Public License as published by
199@@ -71,6 +71,14 @@
200 * {branch_nick} - branch nickname
201 * {clean} - 0 if the source tree contains uncommitted changes,
202 otherwise 1
203+
204+ In order to support reproducible builds, you can set the
205+ SOURCE_DATE_EPOCH to the timestamp of the last commit on a bzr
206+ working tree as follows:
207+
208+ SOURCE_DATE_EPOCH=$(bzr version-info --source-date-epoch)
209+ or using the alias
210+ SOURCE_DATE_EPOCH=$(bzr sde)
211 """
212
213 takes_options = [RegistryOption('format',
214@@ -88,6 +96,7 @@
215 'revision',
216 ]
217 takes_args = ['location?']
218+ aliases = ['sde']
219
220 encoding_type = 'exact'
221
222@@ -96,6 +105,10 @@
223 include_file_revisions=False, template=None,
224 revision=None):
225
226+ if 'sde' == self.invoked_as:
227+ format = \
228+ version_info_formats.format_registry.get(key=
229+ 'source-date-epoch')
230 if revision and len(revision) > 1:
231 raise errors.BzrCommandError(
232 gettext('bzr version-info --revision takes exactly'
233
234=== modified file 'bzrlib/doc_generate/autodoc_bash_completion.py'
235--- bzrlib/doc_generate/autodoc_bash_completion.py 2011-12-19 13:23:58 +0000
236+++ bzrlib/doc_generate/autodoc_bash_completion.py 2016-02-01 16:52:40 +0000
237@@ -1,4 +1,4 @@
238-# Copyright (C) 2005 Canonical Ltd
239+# Copyright (C) 2005, 2016 Canonical Ltd
240
241 # This program is free software; you can redistribute it and/or modify
242 # it under the terms of the GNU General Public License as published by
243@@ -18,7 +18,8 @@
244
245 from __future__ import absolute_import
246
247-import time
248+import datetime
249+import os
250
251 import bzrlib
252 import bzrlib.help
253@@ -30,24 +31,35 @@
254
255
256 def infogen(options, outfile):
257- t = time.time()
258- tt = time.gmtime(t)
259- params = \
260- { "bzrcmd": options.bzr_name,
261- "datestamp": time.strftime("%Y-%m-%d",tt),
262- "timestamp": time.strftime("%Y-%m-%d %H:%M:%S +0000",tt),
263- "version": bzrlib.__version__,
264- }
265+ try:
266+ tt = datetime.datetime.utcfromtimestamp(int(os.environ['SOURCE_DATE_EPOCH']))
267+ params = \
268+ { "bzrcmd": options.bzr_name,
269+ "datestamp": tt.strftime("%Y-%m-%d"),
270+ "timestamp": tt.strftime("%Y-%m-%d %H:%M:%S +0000"),
271+ "version": bzrlib.__version__,
272+ }
273+
274+ except (KeyError, ValueError):
275+ tt = None
276+ params = \
277+ { "bzrcmd": options.bzr_name,
278+ "version": bzrlib.__version__,
279+ }
280
281 outfile.write(preamble % params)
282+ if tt not None:
283+ outfile.write(last_commit % params)
284
285
286 preamble = """\
287-# bash completion functions for for Bazaar (%(bzrcmd)s)
288+# bash completion functions for Bazaar (%(bzrcmd)s)
289 #
290 # Large parts of this file are autogenerated from the internal
291 # Bazaar documentation and data structures.
292+"""
293+
294+last_commit = """\
295 #
296-# Generation time: %(timestamp)s
297+# Last commit: %(timestamp)s
298 """
299-
300
301=== modified file 'bzrlib/doc_generate/autodoc_man.py'
302--- bzrlib/doc_generate/autodoc_man.py 2015-03-14 23:44:01 +0000
303+++ bzrlib/doc_generate/autodoc_man.py 2016-02-01 16:52:40 +0000
304@@ -1,4 +1,4 @@
305-# Copyright (C) 2005-2010 Canonical Ltd
306+# Copyright (C) 2005-2010, 2016 Canonical Ltd
307
308 # This program is free software; you can redistribute it and/or modify
309 # it under the terms of the GNU General Public License as published by
310@@ -15,18 +15,15 @@
311 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
312
313 """man.py - create man page from built-in bzr help and static text
314-
315-TODO:
316- * use usage information instead of simple "bzr foo" in COMMAND OVERVIEW
317- * add command aliases
318 """
319
320 from __future__ import absolute_import
321
322 PLUGINS_TO_DOCUMENT = ["launchpad"]
323
324+import datetime
325+import os
326 import textwrap
327-import time
328
329 import bzrlib
330 import bzrlib.help
331@@ -44,15 +41,35 @@
332
333 def infogen(options, outfile):
334 """Assembles a man page"""
335- t = time.time()
336- tt = time.gmtime(t)
337- params = \
338- { "bzrcmd": options.bzr_name,
339- "datestamp": time.strftime("%Y-%m-%d",tt),
340- "timestamp": time.strftime("%Y-%m-%d %H:%M:%S +0000",tt),
341- "version": bzrlib.__version__,
342- }
343- outfile.write(man_preamble % params)
344+ try:
345+ tt = datetime.datetime.utcfromtimestamp(int(os.environ['SOURCE_DATE_EPOCH']))
346+ params = \
347+ { "bzrcmd": options.bzr_name,
348+ "datestamp": tt.strftime("%Y-%m-%d"),
349+ "timestamp": tt.strftime("%Y-%m-%d %H:%M:%S +0000"),
350+ "version": bzrlib.__version__,
351+ }
352+
353+ except (KeyError, ValueError):
354+ tt = None
355+ ep = datetime.datetime.utcfromtimestamp(-1)
356+ params = \
357+ { "bzrcmd": options.bzr_name,
358+ # Linux man-pages convention: Title includes
359+ # datestamp of last non-trivial revision. In absence
360+ # of SOURCE_DATE_EPOCH use 1 second before UNIX
361+ # epoch (impossible date).
362+ "datestamp": ep.strftime("%Y-%m-%d"),
363+ "version": bzrlib.__version__,
364+ }
365+ # Split part of man_preamble out that uses timestamp to allow
366+ # reproducible builds in the absence of SOURCE_DATE_EPOCH.
367+ outfile.write(man_prepreamble % params)
368+ if tt not None:
369+ # Uses timestamp, if available.
370+ outfile.write(man_last_commit_preamble % params)
371+ outfile.write(man_postpreamble % params)
372+
373 outfile.write(man_escape(man_head % params))
374 outfile.write(man_escape(getcommand_list(params)))
375 outfile.write(man_escape(getcommand_help(params)))
376@@ -182,13 +199,24 @@
377 yield man_escape(desc) + "\n"
378
379
380-man_preamble = """\
381+# Split preamble to conditionally provide last commit time when
382+# SOURCE_DATE_EPOCH provided by build system in order to provide
383+# reproducible builds.
384+man_prepreamble = """\
385 .\\\"Man page for Bazaar (%(bzrcmd)s)
386 .\\\"
387 .\\\" Large parts of this file are autogenerated from the output of
388 .\\\" \"%(bzrcmd)s help commands\"
389 .\\\" \"%(bzrcmd)s help <cmd>\"
390 .\\\"
391+"""
392+
393+man_last_commit_preamble = """\
394+.\\\" Last commit: %(timestamp)s
395+.\\\"
396+"""
397+
398+man_postpreamble = """\
399
400 .ie \\n(.g .ds Aq \\(aq
401 .el .ds Aq '
402
403=== modified file 'bzrlib/doc_generate/autodoc_rstx.py'
404--- bzrlib/doc_generate/autodoc_rstx.py 2015-11-15 02:30:05 +0000
405+++ bzrlib/doc_generate/autodoc_rstx.py 2016-02-01 16:52:40 +0000
406@@ -1,4 +1,4 @@
407-# Copyright (C) 2006-2010 Canonical Ltd
408+# Copyright (C) 2006-2010, 2016 Canonical Ltd
409 #
410 # This program is free software; you can redistribute it and/or modify
411 # it under the terms of the GNU General Public License as published by
412@@ -22,7 +22,8 @@
413
414 from __future__ import absolute_import
415
416-import time
417+import datetime
418+import os
419
420 import bzrlib
421 import bzrlib.help
422@@ -38,20 +39,34 @@
423
424 def infogen(options, outfile):
425 """Create manual in RSTX format"""
426- t = time.time()
427- tt = time.gmtime(t)
428- params = \
429- { "bzrcmd": options.bzr_name,
430- "datestamp": time.strftime("%Y-%m-%d",tt),
431- "timestamp": time.strftime("%Y-%m-%d %H:%M:%S +0000",tt),
432- "version": bzrlib.__version__,
433- }
434+ try:
435+ tt = datetime.datetime.utcfromtimestamp(int(os.environ['SOURCE_DATE_EPOCH']))
436+ params = \
437+ { "bzrcmd": options.bzr_name,
438+ "datestamp": tt.strftime("%Y-%m-%d"),
439+ "timestamp": tt.strftime("%Y-%m-%d %H:%M:%S +0000"),
440+ "version": bzrlib.__version__,
441+ }
442+
443+ except (KeyError, ValueError):
444+ tt = None
445+ params = \
446+ { "bzrcmd": options.bzr_name,
447+ "version": bzrlib.__version__,
448+ }
449 nominated_filename = getattr(options, 'filename', None)
450 if nominated_filename is None:
451 topic_dir = None
452 else:
453 topic_dir = bzrlib.osutils.dirname(nominated_filename)
454- outfile.write(rstx_preamble % params)
455+ # Split part of rstx_preamble out that uses timestamp to allow
456+ # reproducible builds in the absence of SOURCE_DATE_EPOCH.
457+ outfile.write(rstx_prepreamble % params)
458+ if tt not None:
459+ # Uses timestamp, if available.
460+ outfile.write(rstx_last_commit_preamble % params)
461+ outfile.write(rstx_postpreamble % params)
462+
463 outfile.write(rstx_head % params)
464 outfile.write(_get_body(params, topic_dir))
465 outfile.write(rstx_foot % params)
466@@ -139,14 +154,24 @@
467 ##
468 # TEMPLATES
469
470-rstx_preamble = """.. This file is autogenerated from the output of
471+# Split preamble to conditionally provide last commit time when
472+# SOURCE_DATE_EPOCH provided by build system in order to provide
473+# reproducible builds.
474+rstx_prepreamble = """\
475+.. This file is autogenerated from the output of
476 .. %(bzrcmd)s help topics
477 .. %(bzrcmd)s help commands
478 .. %(bzrcmd)s help <cmd>
479 ..
480-
481-"""
482-
483+"""
484+
485+rstx_last_commit_preamble = """\
486+.. Last commit: %(timestamp)s
487+"""
488+
489+rstx_postpreamble = """\
490+
491+"""
492
493 rstx_head = """\
494 #####################
495
496=== modified file 'bzrlib/tests/test_version_info.py'
497--- bzrlib/tests/test_version_info.py 2012-01-03 13:08:44 +0000
498+++ bzrlib/tests/test_version_info.py 2016-02-01 16:52:40 +0000
499@@ -1,4 +1,4 @@
500-# Copyright (C) 2005-2011 Canonical Ltd
501+# Copyright (C) 2005-2011, 2016 Canonical Ltd
502 #
503 # This program is free software; you can redistribute it and/or modify
504 # it under the terms of the GNU General Public License as published by
505@@ -33,6 +33,7 @@
506 from bzrlib.version_info_formats.format_custom import CustomVersionInfoBuilder
507 from bzrlib.version_info_formats.format_rio import RioVersionInfoBuilder
508 from bzrlib.version_info_formats.format_python import PythonVersionInfoBuilder
509+from bzrlib.version_info_formats.format_source_date_epoch import SourceDateEpochVersionInfoBuilder
510
511
512 class VersionInfoTestCase(TestCaseWithTransport):
513@@ -399,6 +400,34 @@
514 self.assertRaises(errors.NoTemplate, builder.generate, sio)
515
516
517+class SourceDateEpochVersionInfoTests(VersionInfoTestCase):
518+
519+ def test_custom_null(self):
520+ """No revision yet on tree."""
521+ sio = StringIO()
522+ wt = self.make_branch_and_tree('branch')
523+ builder = SourceDateEpochVersionInfoBuilder(wt.branch, working_tree=wt)
524+ builder.generate(sio)
525+ self.assertEquals("0", sio.getvalue())
526+
527+ def regen(self, wt):
528+ sio = StringIO()
529+ builder = SourceDateEpochVersionInfoBuilder(wt.branch, working_tree=wt)
530+ builder.generate(sio)
531+ val = sio.getvalue()
532+ return val
533+
534+ def test_source_date_epoch(self):
535+ """Should return seconds from *nix epoch till most recent revision."""
536+ wt = self.create_branch()
537+ # Get timestamp of most recent revision on tree.
538+ br = wt.branch
539+ revision_id = wt.last_revision()
540+ last_rev = br.repository.get_revision(revision_id)
541+ val = self.regen(wt)
542+ self.assertEqual(val, '%0d' % last_rev.timestamp)
543+
544+
545 class TestBuilder(version_info_formats.VersionInfoBuilder):
546 pass
547
548
549=== modified file 'bzrlib/version_info_formats/__init__.py'
550--- bzrlib/version_info_formats/__init__.py 2011-12-30 13:02:27 +0000
551+++ bzrlib/version_info_formats/__init__.py 2016-02-01 16:52:40 +0000
552@@ -1,4 +1,4 @@
553-# Copyright (C) 2005, 2006 Canonical Ltd
554+# Copyright (C) 2005, 2006, 2016 Canonical Ltd
555 #
556 # This program is free software; you can redistribute it and/or modify
557 # it under the terms of the GNU General Public License as published by
558@@ -205,3 +205,8 @@
559 'bzrlib.version_info_formats.format_custom',
560 'CustomVersionInfoBuilder',
561 'Version info in Custom template-based format.')
562+format_registry.register_lazy(
563+ 'source-date-epoch',
564+ 'bzrlib.version_info_formats.format_source_date_epoch',
565+ 'SourceDateEpochVersionInfoBuilder',
566+ 'SOURCE_DATE_EPOCH version info (for reproducible builds).')
567
568=== added file 'bzrlib/version_info_formats/format_source_date_epoch.py'
569--- bzrlib/version_info_formats/format_source_date_epoch.py 1970-01-01 00:00:00 +0000
570+++ bzrlib/version_info_formats/format_source_date_epoch.py 2016-02-01 16:52:40 +0000
571@@ -0,0 +1,45 @@
572+# Copyright (C) 2016 Canonical Ltd
573+#
574+# This program is free software; you can redistribute it and/or modify
575+# it under the terms of the GNU General Public License as published by
576+# the Free Software Foundation; either version 2 of the License, or
577+# (at your option) any later version.
578+#
579+# This program is distributed in the hope that it will be useful,
580+# but WITHOUT ANY WARRANTY; without even the implied warranty of
581+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
582+# GNU General Public License for more details.
583+#
584+# You should have received a copy of the GNU General Public License
585+# along with this program; if not, write to the Free Software
586+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
587+
588+"""A generator which creates the SOURCE_DATE_EPOCH (seconds from the
589+*nix epoch till the most recent revision commit) from the current tree
590+info. This is used to aid the reproducible builds effort by replacing
591+date and time stamps derived from the build date and time with
592+information regarding the last revision of the source code.
593+"""
594+
595+from __future__ import absolute_import
596+
597+from bzrlib.revision import (
598+ NULL_REVISION,
599+ )
600+from bzrlib.version_info_formats import (
601+ VersionInfoBuilder,
602+ )
603+
604+
605+class SourceDateEpochVersionInfoBuilder(VersionInfoBuilder):
606+ """Create a version file which consists of the source epoch."""
607+
608+ def generate(self, to_file):
609+ revision_id = self._get_revision_id()
610+ if revision_id == NULL_REVISION:
611+ sde = 0
612+ else:
613+ rev = self._branch.repository.get_revision(revision_id)
614+ sde = rev.timestamp
615+
616+ to_file.write('%0d' % sde)
617
618=== modified file 'doc/en/Makefile'
619--- doc/en/Makefile 2012-03-09 16:48:55 +0000
620+++ doc/en/Makefile 2016-02-01 16:52:40 +0000
621@@ -2,7 +2,9 @@
622 #
623
624 # You can set these variables from the command line.
625-SPHINXOPTS =
626+SOURCE_DATE_EPOCH ?= $(shell bzr sde)
627+BUILD_DATE=$(shell LC_ALL=C date -u "+%B %d, %Y" -d "@$(SOURCE_DATE_EPOCH)")
628+SPHINXOPTS = "-D today=\"$(BUILD_DATE)\""
629 SPHINXBUILD = sphinx-build
630 PAPER =
631