Merge lp:~xaav/loggerhead/export-tarball into lp:loggerhead

Proposed by xaav
Status: Superseded
Proposed branch: lp:~xaav/loggerhead/export-tarball
Merge into: lp:loggerhead
Diff against target: 373 lines (+161/-13) (has conflicts)
9 files modified
loggerhead/apps/branch.py (+13/-2)
loggerhead/config.py (+3/-0)
loggerhead/controllers/__init__.py (+1/-1)
loggerhead/controllers/download_ui.py (+39/-8)
loggerhead/controllers/revision_ui.py (+19/-0)
loggerhead/exporter.py (+46/-0)
loggerhead/history.py (+5/-1)
loggerhead/templates/revision.pt (+4/-1)
loggerhead/tests/test_controllers.py (+31/-0)
Text conflict in loggerhead/apps/branch.py
Text conflict in loggerhead/controllers/revision_ui.py
Text conflict in loggerhead/tests/test_controllers.py
To merge this branch: bzr merge lp:~xaav/loggerhead/export-tarball
Reviewer Review Type Date Requested Status
Vincent Ladeuil (community) Needs Fixing
Gavin Panella (community) Needs Fixing
Launchpad code reviewers code Pending
Robert Collins Pending
Martin Albisetti Pending
Review via email: mp+63931@code.launchpad.net

This proposal supersedes a proposal from 2011-05-31.

This proposal has been superseded by a proposal from 2011-06-30.

Description of the change

This branch **may** accomplish exporting the tarball using chunked transfer encoding. The code all looks to be correct, but I have not tested it, so I would like your opinion.

Thanks!

To post a comment you must log in.
Revision history for this message
Martin Pool (mbp) wrote : Posted in a previous version of this proposal

Thanks very much, that'd be a really useful feature to have. Thanks
also for making it optional, because probably some installations would
not want it on.

This looks broadly reasonable -- I'm not deeply familiar with
loggerhead -- but I am very curious why you apparently reimplemented
the export-to-tarball feature. I'd rather reuse the bzr code and if
necessary change it to let it be reused here.

When you say "not tested" do you mean you haven't even run it, or only
that you didn't add automatic tests?

Revision history for this message
Robert Collins (lifeless) wrote : Posted in a previous version of this proposal

This looks very interesting.

Three isues:

1)
+class TarExporterFileObject(object):
+
+ def __init__(self):
+ self._buffer = ''
+
+ def write(self, str):
+ self._buffer += str
+
+ def get_buffer(self):
+ buffer = self._buffer
+ self._buffer = ''
+ return buffer

This is going to be somewhat inefficient. Try this:

+class TarExporterFileObject(object):
+
+ def __init__(self):
+ self._buffer = []
+
+ def write(self, str):
+ self._buffer.append(str)
+
+ def get_buffer(self):
+ try:
+ return ''.join(self._buffer)
+ finally:
+ self._buffer = []

2) There are no tests for this. the test suite is still pretty new, but its a good idea to test things - in particular in cases like this we need to be fairly confident it will be incremental and not block on the export - I can imagine the wsgi layer buffering everything, for instance. [in fact, I'll lay odds it will unless we fix a few things].

3) The export function is a copy-paste-tweak of the core from bzrlib. This will lead to bugs as that code base evolves - we should instead get a supported function in bzrlib lib that we can import and use.

I'm putting this back to WIP - but its a great start. Please keep at it and just shout if you need pointers.

review: Needs Fixing
Revision history for this message
xaav (xaav) wrote : Posted in a previous version of this proposal

> 3) The export function is a copy-paste-tweak of the core from bzrlib. This will lead to bugs as
> that code base evolves - we should instead get a supported function in bzrlib lib that we can
> import and use.

Well, there is only one problem with that. According to the WSGI spec, you must return an iterable that will export the blocks. If I were to call the provided function, it would be impossible to break the response into pieces because the provided function would export it all at once. I know that it is a copy and paste tweak, but there is really no way I can inject the 'yield' keyword into the provided function. If you have another suggestion, I would be glad to hear it.

Issue number one I will be glad to fix.

Regarding issue number two, I have not written tests before but I will try my best.

Revision history for this message
Robert Collins (lifeless) wrote : Posted in a previous version of this proposal

On Wed, Jun 1, 2011 at 9:46 AM, Geoff <email address hidden> wrote:
>> 3) The export function is a copy-paste-tweak of the core from bzrlib. This will lead to bugs as
>> that code base evolves - we should instead get a supported function in bzrlib lib that we can
>> import and use.
>
> Well, there is only one problem with that. According to the WSGI spec, you must return an iterable that will export the blocks. If I were to call the provided function, it would be impossible to break the response into pieces because the provided function would export it all at once. I know that it is a copy and paste tweak, but there is really no way I can inject the 'yield' keyword into the provided function. If you have another suggestion, I would be glad to hear it.

Extract the function in bzrlib into two parts - a generator (what you
have here) and a consumer than consumes it all triggering the writes.

Then we can reuse the generator.

Revision history for this message
xaav (xaav) wrote : Posted in a previous version of this proposal

> On Wed, Jun 1, 2011 at 9:46 AM, Geoff <email address hidden> wrote:
> >> 3) The export function is a copy-paste-tweak of the core from bzrlib. This
> will lead to bugs as
> >> that code base evolves - we should instead get a supported function in
> bzrlib lib that we can
> >> import and use.
> >
> > Well, there is only one problem with that. According to the WSGI spec, you
> must return an iterable that will export the blocks. If I were to call the
> provided function, it would be impossible to break the response into pieces
> because the provided function would export it all at once. I know that it is a
> copy and paste tweak, but there is really no way I can inject the 'yield'
> keyword into the provided function. If you have another suggestion, I would be
> glad to hear it.
>
> Extract the function in bzrlib into two parts - a generator (what you
> have here) and a consumer than consumes it all triggering the writes.
>
> Then we can reuse the generator.

Okay, I see what you mean.

Revision history for this message
xaav (xaav) wrote : Posted in a previous version of this proposal

See Bug #791005 for further information on this.

Revision history for this message
xaav (xaav) wrote : Posted in a previous version of this proposal

Okay, I think this should work, but I haven't tested it yet.

Revision history for this message
Martin Pool (mbp) wrote :

Thanks, xaav.

When you say "haven't tested" do you mean just "not written any tests", or "not even been able to run it"?

Revision history for this message
xaav (xaav) wrote :

I haven't written any tests (Sorry, I'll do this ASAP.)
I also have not been able to run it because I'm lazy and it requires too much work to get loggerhead running on W1nd0w$.

lp:~xaav/loggerhead/export-tarball updated
445. By xaav on 2011-06-13

Added tarfile test.

Revision history for this message
xaav (xaav) wrote :

Okay, I've added a simple tarfile test. However, I am not able to run the test to I would appreciate if someone would do that for me and/or try to download a tarball from their browser.

Revision history for this message
Gavin Panella (allenap) wrote :

Once I'd set up a virtualenv with the right prerequisites, I got the
following error when running the test suite:

{{{
Traceback (most recent call last):
  ...
  File ".../loggerhead/tests/test_controllers.py", line 8, in <module>
    from loggerhead.apps.branch import BranchWSGIApp
  File ".../loggerhead/apps/branch.py", line 36, in <module>
    from loggerhead.controllers.download_ui import DownloadUI, DownloadTarballUI
  File ".../loggerhead/controllers/download_ui.py", line 29, in <module>
    from loggerhead.exporter import export_tarball
ImportError: cannot import name export_tarball
}}}

After fixing that I got the following error from
TestDownloadTarballUI.test_download_tarball:

{{{
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/testtools/runtest.py", line 169, in _run_user
    return fn(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 499, in _run_test_method
    return self._get_test_method()()
  File ".../loggerhead/tests/test_controllers.py", line 135, in test_download_tarball
    app = self.setUpLoggerhead()
  File ".../loggerhead/tests/test_simple.py", line 47, in setUpLoggerhead
    branch_app = BranchWSGIApp(self.tree.branch, '', **kw).app
AttributeError: 'TestDownloadTarballUI' object has no attribute 'tree'
}}}

Obviously this needs some work.

We've been talking about taking more of a "patch pilot" approach in
Launchpad. That seems to mean that one of the core team - fwiw, I
would be happy to do it - would actively help getting this landed,
rather than just reviewing it. Would you like that, or would you
prefer to iterate on your own?

review: Needs Fixing
Revision history for this message
xaav (xaav) wrote :

Sorry, I had been gone. I will be sure and look into this right away!

On Thu, Jun 16, 2011 at 8:56 AM, Gavin Panella
<email address hidden>wrote:

> Review: Needs Fixing
> Once I'd set up a virtualenv with the right prerequisites, I got the
> following error when running the test suite:
>
> {{{
> Traceback (most recent call last):
> ...
> File ".../loggerhead/tests/test_controllers.py", line 8, in <module>
> from loggerhead.apps.branch import BranchWSGIApp
> File ".../loggerhead/apps/branch.py", line 36, in <module>
> from loggerhead.controllers.download_ui import DownloadUI,
> DownloadTarballUI
> File ".../loggerhead/controllers/download_ui.py", line 29, in <module>
> from loggerhead.exporter import export_tarball
> ImportError: cannot import name export_tarball
> }}}
>
> After fixing that I got the following error from
> TestDownloadTarballUI.test_download_tarball:
>
> {{{
> Traceback (most recent call last):
> File "/usr/lib/python2.7/dist-packages/testtools/runtest.py", line 169, in
> _run_user
> return fn(*args, **kwargs)
> File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 499,
> in _run_test_method
> return self._get_test_method()()
> File ".../loggerhead/tests/test_controllers.py", line 135, in
> test_download_tarball
> app = self.setUpLoggerhead()
> File ".../loggerhead/tests/test_simple.py", line 47, in setUpLoggerhead
> branch_app = BranchWSGIApp(self.tree.branch, '', **kw).app
> AttributeError: 'TestDownloadTarballUI' object has no attribute 'tree'
> }}}
>
> Obviously this needs some work.
>
> We've been talking about taking more of a "patch pilot" approach in
> Launchpad. That seems to mean that one of the core team - fwiw, I
> would be happy to do it - would actively help getting this landed,
> rather than just reviewing it. Would you like that, or would you
> prefer to iterate on your own?
>
> --
> https://code.launchpad.net/~xaav/loggerhead/export-tarball/+merge/63931
> You are the owner of lp:~xaav/loggerhead/export-tarball.
>

Revision history for this message
xaav (xaav) wrote :

We've been talking about taking more of a "patch pilot" approach in
Launchpad. That seems to mean that one of the core team - fwiw, I
would be happy to do it - would actively help getting this landed,
rather than just reviewing it. Would you like that, or would you
prefer to iterate on your own?

That would be great! Any help would be greatly appreciated.

Revision history for this message
xaav (xaav) wrote :

Okay, I've fixed some stuff (the tests are still broken), but here is what I'm getting:

Traceback (most recent call last):
  File "C:\Python27\lib\site-packages\paste-1.7.5.1-py2.7.egg\paste\httpserver.py", l
ine 1068, in process_request_in_thread
    self.finish_request(request, client_address)
  File "C:\Python27\lib\SocketServer.py", line 323, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Python27\lib\SocketServer.py", line 639, in __init__
    self.handle()
  File "C:\Python27\lib\site-packages\paste-1.7.5.1-py2.7.egg\paste\httpserver.py", l
ine 442, in handle
    BaseHTTPRequestHandler.handle(self)
  File "C:\Python27\lib\BaseHTTPServer.py", line 337, in handle
    self.handle_one_request()
  File "C:\Python27\lib\site-packages\paste-1.7.5.1-py2.7.egg\paste\httpserver.py", l
ine 437, in handle_one_request
    self.wsgi_execute()
  File "C:\Python27\lib\site-packages\paste-1.7.5.1-py2.7.egg\paste\httpserver.py", l
ine 289, in wsgi_execute
    for chunk in result:
  File "C:\Users\Xaav\workspace\loggerhead\loggerhead\exporter.py", line 31, in e
xport_archive
    for _ in get_export_generator(tree=tree, fileobj=fileobj, format=format):
  File "C:\Python27\lib\site-packages\bzrlib\export\__init__.py", line 112, in get_ex
port_generator
    root = get_root_name(dest)
  File "C:\Python27\lib\site-packages\bzrlib\export\__init__.py", line 174, in get_ro
ot_name
    dest = os.path.basename(dest)
  File "C:\Python27\lib\ntpath.py", line 198, in basename
    return split(p)[1]
  File "C:\Python27\lib\ntpath.py", line 170, in split
    d, p = splitdrive(p)
  File "C:\Python27\lib\ntpath.py", line 125, in splitdrive
    if p[1:2] == ':':
TypeError: 'NoneType' object is not subscriptable

Any help would be appreciated.

lp:~xaav/loggerhead/export-tarball updated
446. By xaav on 2011-06-27

Fixed import.

447. By xaav on 2011-06-27

Fixed code issues.

Revision history for this message
Gavin Panella (allenap) wrote :

Cool. I am busy this week, but I might get to it. If not, next week for sure.

Revision history for this message
Vincent Ladeuil (vila) wrote :
Download full text (3.9 KiB)

Hi,

I almost got the test running with some additional fixes
(available at lp:~vila/loggerhead/export-tarball) only to run
into a bug in bzr itself (I think you should be able to fix that
one ;).

Note that your code requires bzr >= 2.4 (launchpad only runs
2.3.3 so far) so we'll need some support from the lp guys to
deploy a more recent version there.

Summary of my fixes:

- you need to create a branch (with some content even) before
  being able to call

         app = self.setUpLoggerhead()

  So I've added a setUp method for your class to do that. You
  probably want to add *more* tests to check that you get a valid
  tarball with the expected content (which an empty branch
  doesn't allow ;).

- you're calling get_export_generator without dest nor root and
  the code in bzrlib defaults to dest to set root. This raises an
  interesing point: which root should be used here (i.e. what do
  we want to prefix all the paths in the archive
  with). <project>-<branch nick>-<revno> may be nice (but ask
  others for feedback too).

- you used '.tar.gz' for the format but bzr expects either 'tgz'
  OR a dest file name to deduce the format from the file
  suffix. I just used 'tgz' there.

With these fixes in place we get:

======================================================================
ERROR: bzrlib.plugins.loggerhead.loggerhead.tests.test_controllers.TestDownloadTarballUI.test_download_tarball
----------------------------------------------------------------------
_StringException: Text attachment: log
------------
0.622 creating repository in file:///tmp/testbzr-KY_qfE.tmp/bzrlib.plugins.loggerhead.loggerhead.tests.test_controllers.TestDownloadTarballUI.test_download_tarball/work/.bzr/.
0.624 creating branch <bzrlib.branch.BzrBranchFormat7 object at 0x22ae990> in file:///tmp/testbzr-KY_qfE.tmp/bzrlib.plugins.loggerhead.loggerhead.tests.test_controllers.TestDownloadTarballUI.test_download_tarball/work/
0.631 trying to create missing lock '/tmp/testbzr-KY_qfE.tmp/bzrlib.plugins.loggerhead.loggerhead.tests.test_controllers.TestDownloadTarballUI.test_download_tarball/work/.bzr/checkout/dirstate'
0.631 opening working tree '/tmp/testbzr-KY_qfE.tmp/bzrlib.plugins.loggerhead.loggerhead.tests.test_controllers.TestDownloadTarballUI.test_download_tarball/work'
0.642 export version <InventoryRevisionTree instance at 29ece90, rev_id='null:'>
0.649 opening working tree '/tmp/testbzr-KY_qfE.tmp'
------------
Text attachment: traceback
------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/testtools/runtest.py", line 169, in _run_user
    return fn(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 499, in _run_test_method
    return self._get_test_method()()
  File "/home/vila/.bazaar/plugins/loggerhead/loggerhead/tests/test_controllers.py", line 140, in test_download_tarball
    res = app.get('/tarball')
  File "/usr/lib/pymodules/python2.7/paste/fixture.py", line 208, in get
    return self.do_request(req, status=status)
  File "/usr/lib/pymodules/python2.7/paste/fixture.py", line 389, in do_request
    **req.environ)
  File "/usr/lib/pymodules/python2.7/paste/wsgilib.py", li...

Read more...

review: Needs Fixing
lp:~xaav/loggerhead/export-tarball updated
448. By xaav <email address hidden> on 2011-06-28

modified bzrignore

449. By xaav <email address hidden> on 2011-06-28

Merged branch

450. By xaav on 2011-06-28

Fixed gzip bug.

451. By xaav <email address hidden> on 2011-07-07

Merged lp:loggerhead

452. By xaav <email address hidden> on 2011-07-07

Fixed extension issue

453. By xaav on 2011-07-10

Fixed buggy merging.

454. By xaav <email address hidden> on 2011-07-10

Fixed buggy merging and removed IDE files

455. By xaav on 2011-07-11

Fixed serve tarballs issue.

456. By xaav on 2011-07-11

Fixed tests.

457. By xaav on 2011-07-11

Added to UI.

458. By xaav on 2011-07-12

UI fixes.

Unmerged revisions

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'loggerhead/apps/branch.py'
2--- loggerhead/apps/branch.py 2011-06-28 16:09:37 +0000
3+++ loggerhead/apps/branch.py 2011-06-28 18:02:56 +0000
4@@ -33,7 +33,7 @@
5 from loggerhead.controllers.atom_ui import AtomUI
6 from loggerhead.controllers.changelog_ui import ChangeLogUI
7 from loggerhead.controllers.diff_ui import DiffUI
8-from loggerhead.controllers.download_ui import DownloadUI
9+from loggerhead.controllers.download_ui import DownloadUI, DownloadTarballUI
10 from loggerhead.controllers.filediff_ui import FileDiffUI
11 from loggerhead.controllers.inventory_ui import InventoryUI
12 from loggerhead.controllers.revision_ui import RevisionUI
13@@ -49,7 +49,7 @@
14
15 def __init__(self, branch, friendly_name=None, config={},
16 graph_cache=None, branch_link=None, is_root=False,
17- served_url=_DEFAULT, use_cdn=False):
18+ served_url=_DEFAULT, use_cdn=False, export_tarballs=True):
19 self.branch = branch
20 self._config = config
21 self.friendly_name = friendly_name
22@@ -61,6 +61,7 @@
23 self.is_root = is_root
24 self.served_url = served_url
25 self.use_cdn = use_cdn
26+ self.export_tarballs = export_tarballs
27
28 def get_history(self):
29 file_cache = None
30@@ -126,6 +127,7 @@
31 'revision': RevisionUI,
32 'search': SearchUI,
33 'view': ViewUI,
34+ 'tarball': DownloadTarballUI,
35 }
36
37 def last_updated(self):
38@@ -176,11 +178,20 @@
39 self.branch.lock_read()
40 try:
41 try:
42+<<<<<<< TREE
43 c = self.lookup_app(environ)
44 return c(environ, start_response)
45+=======
46+ c = cls(self, self.get_history)
47+ to_ret = c(environ, start_response)
48+>>>>>>> MERGE-SOURCE
49 except:
50 environ['exc_info'] = sys.exc_info()
51 environ['branch'] = self
52 raise
53+ if type(to_ret) == type(httpexceptions.HTTPSeeOther('/')):
54+ raise to_ret
55+ else:
56+ return to_ret
57 finally:
58 self.branch.unlock()
59
60=== modified file 'loggerhead/config.py'
61--- loggerhead/config.py 2010-05-12 14:38:05 +0000
62+++ loggerhead/config.py 2011-06-28 18:02:56 +0000
63@@ -36,6 +36,7 @@
64 use_cdn=False,
65 sql_dir=None,
66 allow_writes=False,
67+ export_tarballs=True,
68 )
69 parser.add_option("--user-dirs", action="store_true",
70 help="Serve user directories as ~user.")
71@@ -75,6 +76,8 @@
72 help="The directory to place the SQL cache in")
73 parser.add_option("--allow-writes", action="store_true",
74 help="Allow writing to the Bazaar server.")
75+ parser.add_option("--export-tarballs", action="store_true",
76+ help="Allow exporting revisions to tarballs.")
77 return parser
78
79
80
81=== modified file 'loggerhead/controllers/__init__.py'
82--- loggerhead/controllers/__init__.py 2011-06-28 16:09:37 +0000
83+++ loggerhead/controllers/__init__.py 2011-06-28 18:02:56 +0000
84@@ -21,7 +21,7 @@
85 import simplejson
86 import time
87
88-from paste.httpexceptions import HTTPNotFound
89+from paste.httpexceptions import HTTPNotFound, HTTPSeeOther
90 from paste.request import path_info_pop, parse_querystring
91
92 from loggerhead import util
93
94=== modified file 'loggerhead/controllers/download_ui.py'
95--- loggerhead/controllers/download_ui.py 2010-05-05 19:03:40 +0000
96+++ loggerhead/controllers/download_ui.py 2011-06-28 18:02:56 +0000
97@@ -19,46 +19,51 @@
98
99 import logging
100 import mimetypes
101+import os
102 import urllib
103
104 from paste import httpexceptions
105 from paste.request import path_info_pop
106
107 from loggerhead.controllers import TemplatedBranchView
108+from loggerhead.exporter import export_archive
109
110 log = logging.getLogger("loggerhead.controllers")
111
112
113 class DownloadUI (TemplatedBranchView):
114
115- def __call__(self, environ, start_response):
116- # /download/<rev_id>/<file_id>/[filename]
117-
118- h = self._history
119-
120+ def encode_filename(self, filename):
121+
122+ return urllib.quote(filename.encode('utf-8'))
123+
124+ def get_args(self, environ):
125 args = []
126 while True:
127 arg = path_info_pop(environ)
128 if arg is None:
129 break
130 args.append(arg)
131+ return args
132
133+ def __call__(self, environ, start_response):
134+ # /download/<rev_id>/<file_id>/[filename]
135+ h = self._history
136+ args = self.get_args(environ)
137 if len(args) < 2:
138 raise httpexceptions.HTTPMovedPermanently(
139 self._branch.absolute_url('/changes'))
140-
141 revid = h.fix_revid(args[0])
142 file_id = args[1]
143 path, filename, content = h.get_file(file_id, revid)
144 mime_type, encoding = mimetypes.guess_type(filename)
145 if mime_type is None:
146 mime_type = 'application/octet-stream'
147-
148 self.log.info('/download %s @ %s (%d bytes)',
149 path,
150 h.get_revno(revid),
151 len(content))
152- encoded_filename = urllib.quote(filename.encode('utf-8'))
153+ encoded_filename = self.encode_filename(filename)
154 headers = [
155 ('Content-Type', mime_type),
156 ('Content-Length', str(len(content))),
157@@ -67,3 +72,29 @@
158 ]
159 start_response('200 OK', headers)
160 return [content]
161+
162+
163+class DownloadTarballUI(DownloadUI):
164+
165+ def __call__(self, environ, start_response):
166+ """Stream a tarball from a bazaar branch."""
167+ # Tried to re-use code from downloadui, not very successful
168+ format = "tar"
169+ history = self._history
170+ self.args = self.get_args(environ)
171+ if len(self.args):
172+ revid = history.fix_revid(self.args[0])
173+ else:
174+ revid = self.get_revid()
175+ if self._branch.export_tarballs:
176+ root = 'branch'
177+ encoded_filename = self.encode_filename(root + format)
178+ headers = [
179+ ('Content-Type', 'application/octet-stream'),
180+ ('Content-Disposition',
181+ "attachment; filename*=utf-8''%s" % (encoded_filename,)),
182+ ]
183+ start_response('200 OK', headers)
184+ return export_archive(history, root, revid, format)
185+ else:
186+ raise httpexceptions.HTTPSeeOther('/')
187
188=== modified file 'loggerhead/controllers/revision_ui.py'
189--- loggerhead/controllers/revision_ui.py 2011-06-27 17:11:24 +0000
190+++ loggerhead/controllers/revision_ui.py 2011-06-28 18:02:56 +0000
191@@ -136,9 +136,19 @@
192 self._branch.friendly_name,
193 self._branch.is_root,
194 'changes'))
195+<<<<<<< TREE
196
197 values.update({
198 'history': self._history,
199+=======
200+ can_export = self._branch.export_tarballs
201+ return {
202+ 'branch': self._branch,
203+ 'revid': revid,
204+ 'change': change,
205+ 'file_changes': file_changes,
206+ 'diff_chunks': diff_chunks,
207+>>>>>>> MERGE-SOURCE
208 'link_data': simplejson.dumps(link_data),
209 'json_specific_path': simplejson.dumps(path),
210 'path_to_id': simplejson.dumps(path_to_id),
211@@ -149,6 +159,15 @@
212 'filter_file_id': filter_file_id,
213 'diff_chunks': diff_chunks,
214 'query': query,
215+<<<<<<< TREE
216 'specific_path': path,
217 'start_revid': start_revid,
218 })
219+=======
220+ 'remember': remember,
221+ 'compare_revid': compare_revid,
222+ 'url': self._branch.context_url,
223+ 'directory_breadcrumbs': directory_breadcrumbs,
224+ 'can_export': can_export,
225+ }
226+>>>>>>> MERGE-SOURCE
227
228=== added file 'loggerhead/exporter.py'
229--- loggerhead/exporter.py 1970-01-01 00:00:00 +0000
230+++ loggerhead/exporter.py 2011-06-28 18:02:56 +0000
231@@ -0,0 +1,46 @@
232+# Copyright (C) 2011 Canonical Ltd
233+#
234+# This program is free software; you can redistribute it and/or modify
235+# it under the terms of the GNU General Public License as published by
236+# the Free Software Foundation; either version 2 of the License, or
237+# (at your option) any later version.
238+#
239+# This program is distributed in the hope that it will be useful,
240+# but WITHOUT ANY WARRANTY; without even the implied warranty of
241+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
242+# GNU General Public License for more details.
243+#
244+"""Exports an archive from a bazaar branch"""
245+
246+from bzrlib.export import get_export_generator
247+
248+
249+class ExporterFileObject(object):
250+
251+ def __init__(self):
252+ self._buffer = []
253+
254+ def write(self, s):
255+ self._buffer.append(s)
256+
257+ def get_buffer(self):
258+ try:
259+ return ''.join(self._buffer)
260+ finally:
261+ self._buffer = []
262+
263+
264+def export_archive(history, root, revid, format=".tar.gz"):
265+ """Export tree contents to an archive
266+
267+ :param history: Instance of history to export
268+ :param root: Root location inside the archive.
269+ :param revid: Revision to export
270+ :param format: Format of the archive
271+ """
272+ fileobj = ExporterFileObject()
273+ tree = history._branch.repository.revision_tree(revid)
274+ for _ in get_export_generator(tree=tree, root=root, fileobj=fileobj, format=format):
275+ yield fileobj.get_buffer()
276+ # Might have additonal contents written
277+ yield fileobj.get_buffer()
278
279=== modified file 'loggerhead/history.py'
280--- loggerhead/history.py 2011-03-25 13:09:10 +0000
281+++ loggerhead/history.py 2011-06-28 18:02:56 +0000
282@@ -33,6 +33,7 @@
283 import re
284 import textwrap
285 import threading
286+import tarfile
287
288 from bzrlib import tag
289 import bzrlib.branch
290@@ -44,6 +45,7 @@
291 from loggerhead import search
292 from loggerhead import util
293 from loggerhead.wholehistory import compute_whole_history_data
294+from bzrlib.export.tar_exporter import export_tarball
295
296
297 def is_branch(folder):
298@@ -816,4 +818,6 @@
299 renamed=sorted(reporter.renamed, key=lambda x: x.new_filename),
300 removed=sorted(reporter.removed, key=lambda x: x.filename),
301 modified=sorted(reporter.modified, key=lambda x: x.filename),
302- text_changes=sorted(reporter.text_changes, key=lambda x: x.filename))
303+ text_changes=sorted(reporter.text_changes,
304+ key=lambda x: x.filename))
305+
306
307=== added directory 'loggerhead/static/downloads'
308=== modified file 'loggerhead/templates/revision.pt'
309--- loggerhead/templates/revision.pt 2011-06-28 16:51:55 +0000
310+++ loggerhead/templates/revision.pt 2011-06-28 18:02:56 +0000
311@@ -84,7 +84,10 @@
312 tal:attributes="href python:url(['/diff', change.revno], clear=1)">download diff</a>
313 <a tal:condition="python:compare_revid is not None"
314 tal:attributes="href python:url(['/diff', change.revno, history.get_revno(compare_revid)], clear=1)">download diff</a>
315- </li>
316+ </li>
317+ <li tal:condition="python:can_export">
318+ <a tal:attributes="href python:url(['/tarball', change.revno])">Download tarball</a>
319+ </li>
320 <li id="last"><a tal:attributes="href python:url(['/changes', change.revno]);
321 title string:view history from revision ${change/revno}"
322 tal:content="string:view history from revision ${change/revno}"></a></li>
323
324=== modified file 'loggerhead/tests/test_controllers.py'
325--- loggerhead/tests/test_controllers.py 2011-06-28 16:06:12 +0000
326+++ loggerhead/tests/test_controllers.py 2011-06-28 18:02:56 +0000
327@@ -1,4 +1,14 @@
328+<<<<<<< TREE
329 import simplejson
330+=======
331+from cStringIO import StringIO
332+import logging
333+import tarfile
334+
335+from paste.httpexceptions import HTTPServerError
336+
337+from bzrlib import errors
338+>>>>>>> MERGE-SOURCE
339
340 from loggerhead.apps.branch import BranchWSGIApp
341 from loggerhead.controllers.annotate_ui import AnnotateUI
342@@ -135,6 +145,7 @@
343 kwargs={'file_id': 'file_id'}, headers={})
344 annotated = annotate_info['annotated']
345 self.assertEqual(2, len(annotated))
346+<<<<<<< TREE
347 self.assertEqual('2', annotated[1].change.revno)
348 self.assertEqual('1', annotated[2].change.revno)
349
350@@ -204,3 +215,23 @@
351 revlog_ui = branch_app.lookup_app(env)
352 self.assertOkJsonResponse(revlog_ui, env)
353
354+=======
355+ self.assertEqual('2', annotated[0].change.revno)
356+ self.assertEqual('1', annotated[1].change.revno)
357+
358+
359+class TestDownloadTarballUI(BasicTests):
360+
361+ def setUp(self):
362+ super(TestDownloadTarballUI, self).setUp()
363+ self.createBranch()
364+
365+ def test_download_tarball(self):
366+ app = self.setUpLoggerhead()
367+ res = app.get('/tarball')
368+ f = open('tarball', 'w')
369+ f.write(res)
370+ f.close()
371+ self.failIf(not tarfile.is_tarfile('tarball'))
372+ # Now check the content. TBC
373+>>>>>>> MERGE-SOURCE

Subscribers

People subscribed via source and target branches