AttributeError OOPS in codebrowse (lazy_import not threadsafe)

Bug #702914 reported by Ursula Junque
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Bazaar
Confirmed
High
canonical-bazaar
Launchpad itself
In Progress
Critical
Benji York

Bug Description

As seen on OOPS-1839CBA1:

http://bazaar.launchpad.net/%7Evcs-imports/busybox/main/changes

   Traceback (most recent call last):
  Module launchpad_loggerhead.app, line 340, in wrapped_app
    app_iter = iter(app(environ, wrapped.start_response))
  Module launchpad_loggerhead.debug, line 106, in wrapped_application
    result = application(environ, response_hook)
  Module __main__, line 165, in wrapped
  Module launchpad_loggerhead.debug, line 45, in wrapped
    return app(environ, start_response)
  Module paste.translogger, line 68, in __call__
    return self.application(environ, replacement_start_response)
  Module paste.deploy.config, line 285, in __call__
    return self.app(environ, start_response)
  Module __main__, line 149, in wrapped
  Module paste.wsgilib, line 179, in catch_errors
    app_iter = application(environ, start_response)
  Module launchpad_loggerhead.session, line 53, in __call__
    return self.cookie_handler(environ, start_response)
  Module paste.auth.cookie, line 305, in __call__
    return self.application(environ, response_hook)
  Module launchpad_loggerhead.session, line 80, in _process
    return self.application(environ, response_hook)
  Module paste.httpexceptions, line 636, in __call__
    return self.application(environ, start_response)
  Module launchpad_loggerhead.app, line 219, in __call__
    lp_server.get_url().strip(':/'), branch_url)
  Module lp.codehosting.bzrutils, line 361, in safe_open
    makeURLChecker(allowed_scheme), url, possible_transports)
  Module lp.codehosting.bzrutils, line 333, in checked_open
    return Branch.open(url, possible_transports=possible_transports)
  Module bzrlib.branch, line 168, in open
    return control.open_branch(unsupported=_unsupported)
  Module bzrlib.bzrdir, line 1871, in open_branch
    _found=True, ignore_fallbacks=ignore_fallbacks)
  Module bzrlib.branch, line 2069, in open
    _repository=a_bzrdir.find_repository(),
  Module bzrlib.bzrdir, line 735, in find_repository
    found_repo = self._find_containing(usable_repository)
  Module bzrlib.bzrdir, line 701, in _find_containing
    result, stop = evaluate(found_bzrdir)
  Module bzrlib.bzrdir, line 725, in usable_repository
    repository = found_bzrdir.open_repository()
  Module bzrlib.bzrdir, line 1878, in open_repository
    return format.open(self, _found=True)
  Module bzrlib.repofmt.pack_repo, line 2584, in open
    _serializer=self._serializer)
  Module bzrlib.repofmt.pack_repo, line 2234, in __init__
    use_chk_index=self._format.supports_chks,
  Module bzrlib.repofmt.pack_repo, line 1399, in __init__
    self.revision_index = AggregateIndex(self.reload_pack_names, flush)
  Module bzrlib.repofmt.pack_repo, line 589, in __init__
    flush_func=flush_func)
  Module bzrlib.lazy_import, line 124, in __call__
    _cleanup()
  Module bzrlib.lazy_import, line 97, in _cleanup
    del self._factory
AttributeError: _factory

Related branches

Changed in launchpad:
status: New → Triaged
importance: High → Critical
description: updated
description: updated
tags: added: codebrowse
Martin Pool (mbp)
Changed in bzr:
assignee: nobody → canonical-bazaar (canonical-bazaar)
importance: Undecided → High
status: New → Confirmed
Revision history for this message
Robert Collins (lifeless) wrote :

Looks like a race condition on startup I think - is lazy_import threadsafe?

Revision history for this message
Andrew Bennetts (spiv) wrote : Re: [Bug 702914] Re: AttributeError OOPS in codebrowse

Robert Collins wrote:
> Looks like a race condition on startup I think - is lazy_import
> threadsafe?

I doubt it. Certainly I don't think any effort has gone into trying to
make it threadsafe, so it's safest to assume not.

(Then again, I don't think you can say that import in general is
thread-safe either, it depends on what code is run at import time.
Python guarantees that the import machinery is threadsafe[1]… but
obviously it can't make guarantees about code run within that machinery,
and people do all sorts of strange things at import time.)

-Andrew.

[1] with two caveats: 1. imports shouldn't spawn threads and then wait
    on them, 2. imports must be completed before interpreter shutdown
    starts.
    <http://docs.python.org/library/threading.html#importing-in-threaded-code>

Revision history for this message
John A Meinel (jameinel) wrote : Re: AttributeError OOPS in codebrowse

It is a lazy import race condition. I thought I documented that in a different bug.

It is pretty easy to reproduce, if you do:

bzr serve --http

And then spawn a bunch of concurrent requests against that.

Note that it only happens one time, the first time anything is opened. So *if* you start loggerhead, have 1 successful page request, then none of the others will fail with this error (at least that I've found).

One thought was to force lazy_import to not be lazy, but you can't actually do that in bzrlib, because we use lazy_import to work around circular import issues.

John A Meinel (jameinel)
summary: - AttributeError OOPS in codebrowse
+ AttributeError OOPS in codebrowse (lazy_import not threadsafe)
Gary Poster (gary)
Changed in launchpad:
assignee: nobody → Benji York (benji)
Benji York (benji)
Changed in launchpad:
status: Triaged → In Progress
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.