Archive.getPublishedBinaries (for ddebs) keeps timing out

Bug #1635126 reported by Martin Pitt
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself
Fix Released
Critical
William Grant

Bug Description

ddeb-retriever has a hard time importing zesty -- all zesty packages got published around the same time, so it has to do the ddeb query/download in one batch.

This query (http://bazaar.launchpad.net/~ubuntu-archive/ddeb-retriever/trunk/view/head:/lpinfo.py#L55):

    return archive.getPublishedBinaries(
        order_by_date=True, created_since_date=created_since_date)

keeps failing with:

DEBUG: Got ddeb: glmark2-dbgsym 2014.03+git20150611.fa71af2d-0ubuntu4 in zesty s390x (pocket: Release)
Traceback (most recent call last):
  File "/srv/ddebs.ubuntu.com//ddeb-retriever/ddeb-retriever", line 4, in <module>
    ddeb_retriever.main()
  File "/srv/ddebs.ubuntu.com/ddeb-retriever/ddeb_retriever.py", line 164, in main
    for pub in binary_pubs:
  File "/usr/lib/python2.7/dist-packages/lazr/restfulclient/resource.py", line 794, in __iter__
    unicode(self._root._browser.get(URI(next_link))))
  File "/usr/lib/python2.7/dist-packages/lazr/restfulclient/_browser.py", line 386, in get
    response, content = self._request(url, extra_headers=headers)
  File "/usr/lib/python2.7/dist-packages/lazr/restfulclient/_browser.py", line 376, in _request
    raise error
lazr.restfulclient.errors.ServerError: HTTP Error 503: Service Unavailable
Response headers:
---
-content-encoding: gzip
connection: close
content-length: 9769
content-type: text/html;charset=utf-8
date: Wed, 19 Oct 2016 22:14:01 GMT
retry-after: 900
server: zope.server.http (HTTP)
status: 503
vary: Accept-Encoding
x-lazr-oopsid: OOPS-c468be3f4d79009221aa4efed904acac
x-powered-by: Zope (www.zope.org), Python (www.python.org)
---
Response body:
---
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"
      lang="en" dir="ltr">
  <head>
    <meta charset="UTF-8" />
    <title>Error: Timeout</title>
    <link rel="shortcut icon" href="/@@/launchpad.png" />

  <link type="text/css" rel="stylesheet"
        media="screen, print"
        href="/+icing/rev18232/combo.css" />

  <script type="text/javascript">
    var LP = {
        cache: {},
        links: {}
    };
  </script>

  <script type="text/javascript">var cookie_scope = '; Path=/; Secure; Domain=.launchpad.net';</script>

   <script type="text/javascript"
           src="/+combo/rev18232/?yui/yui/yui-min.js&amp;lp/meta.js&amp;yui/loader/loader-min.js"></script>
   <script type="text/javascript">
        var raw = null;
        if (LP.devmode) {
           raw = 'raw';
        }
        YUI.GlobalConfig = {
            combine: true,
            comboBase: '/+combo/rev18232/?',
            root: 'yui/',
            filter: raw,
            debug: false,
            fetchCSS: false,
            maxURLLength: 2000,
            groups: {
                lp: {
                    combine: true,
                    base: '/+combo/rev18232/?lp/',
                    comboBase: '/+combo/rev18232/?',
                    root: 'lp/',
                    // comes from including lp/meta.js
                    modules: LP_MODULES,
                    fetchCSS: false
                }
            }
        }</script>

  <script type="text/javascript">
      // we need this to create a single YUI instance all events and code
      // talks across. All instances of YUI().use should be based off of
      // LPJS instead.
      LPJS = new YUI();
  </script>

    <script id="base-layout-load-scripts" type="text/javascript">
        //<![CDATA[
        LPJS.use('base', 'node', 'console', 'event',
            'oop', 'lp', 'lp.app.foldables','lp.app.sorttable',
            'lp.app.inlinehelp', 'lp.app.links', 'lp.app.longpoll',
            'lp.bugs.bugtask_index', 'lp.bugs.subscribers',
            'lp.app.ellipsis', 'lp.code.branchmergeproposal.diff',
            'lp.views.global',
             function(Y) {

            Y.on("domready", function () {
                var global_view = new Y.lp.views.Global();
                global_view.render();

                Y.lp.app.sorttable.SortTable.init();
                Y.lp.app.inlinehelp.init_help();
                Y.lp.activate_collapsibles();
                Y.lp.app.foldables.activate();
                Y.lp.app.links.check_valid_lp_links();
                // Longpolling will only start if
                // LP.cache.longpoll is populated.
                // We use Y.later to work around a Safari/Chrome 'feature':
                // The mouse cursor stays 'busy' until all the requests started during
                // page load are finished. Hence we want the long poll request to start
                // right *after* the page has loaded.
                Y.later(0, Y.lp.app.longpoll, Y.lp.app.longpoll.setupLongPollManager);

            });

            Y.on('lp:context:web_link:changed', function(e) {
                  window.location = e.new_value;
            });
        });
        //]]>
    </script>
    <script id="base-helper-functions" type="text/javascript">
         //<![CDATA[
        // This code is pulled from lp.js that needs to be available on every
        // request. Pulling here to get it outside the scope of the YUI block.
        function setFocusByName(name) {
            // Focus the first element matching the given name which can be focused.
            var nodes = document.getElementsByName(name);
            var i, node;
            for (i = 0; i < nodes.length; i++) {
                node = nodes[i];
                if (node.focus) {
                    try {
                        // Trying to focus a hidden element throws an error in IE8.
                        if (node.offsetHeight !== 0) {
                            node.focus();
                        }
                    } catch (e) {
                        LPJS.use('console', function(Y) {
                            Y.log('In setFocusByName(<' +
                                node.tagName + ' type=' + node.type + '>): ' + e);
                        });
                    }
                    break;
                }
            }
        }

        function selectWidget(widget_name, event) {
          if (event && (event.keyCode === 9 || event.keyCode === 13)) {
              // Avoid firing if user is tabbing through or simply pressing
              // enter to submit the form.
              return;
          }
          document.getElementById(widget_name).checked = true;
        }
        //]]>
    </script>

  </head>

  <body id="document" itemscope=""
        itemtype="http://schema.org/WebPage"
        class="tab-unknown
      main_only
      public
      yui3-skin-sam">

    <div class="yui-d0">
      <div id="locationbar" class="login-logout">

<div id="logincontrol"><a href="https://api.launchpad.net/devel/devel/ubuntu/+archive/primary/+login?created_since_date=%222016-10-18T05%3A22%3A43%2B00%3A00%22&amp;memo=86025&amp;order_by_date=true&amp;ws.op=getPublishedBinaries&amp;ws.size=75&amp;ws.start=86025">Log in / Register</a></div>

      </div><!--id="locationbar"-->

      <div id="watermark" class="watermark-apps-portlet">
        <div>
          <img alt="" width="64" height="64" src="/@@/launchpad-logo" />
        </div>
        <div class="wide">
          <h2 id="watermark-heading"><span>Launchpad.net</span></h2>
        </div>

  <!-- Application Menu -->
  <ul class="facetmenu">

      <li class="overview active"><a
    href="https://launchpad.net/ubuntu">Overview</a></li>

      <li class="branches"><a
    href="https://code.launchpad.net/ubuntu">Code</a></li>

      <li class="bugs"><a href="https://bugs.launchpad.net/ubuntu">Bugs</a></li>

      <li class="specifications"><a
    href="https://blueprints.launchpad.net/ubuntu">Blueprints</a></li>

      <li class="translations"><a
    href="https://translations.launchpad.net/ubuntu">Translations</a></li>

      <li class="answers"><a href="https://answers.launchpad.net/ubuntu">Answers</a></li>

  </ul>

      </div>

        <div id="maincontent" class="yui-main">
          <div class="yui-b" dir="ltr">
            <div class="context-publication">

              <div id="registration" class="registering">

              </div>
            </div>

            <div id="request-notifications">

            </div>

              <div class="top-portlet">
      <h1 class="exception">Timeout error</h1>
      <p>
        Sorry, something just went wrong in Launchpad.
      </p>
      <p>
        We&#8217;ve recorded what happened,
        and we&#8217;ll fix it as soon as possible.
        Apologies for the inconvenience.
      </p>
      <p>
        Trying again in a couple of minutes might work.
      </p>
      <p>
        (Error <abbr>ID</abbr>:
        <code class="oopsid">OOPS-c468be3f4d79009221aa4efed904acac</code>)
      </p>

    </div>

          </div><!-- yui-b -->
        </div><!-- yui-main -->

          <!-- yui-b side -->

      <!-- yui-t4 -->

  <div id="footer" class="footer">
    <div class="lp-arcana">
        <div class="lp-branding">
          <a href="https://launchpad.net/"><img src="/@@/launchpad-logo-and-name-hierarchy.png" alt="Launchpad" /></a>
          &nbsp;&bull;&nbsp;
          <a href="https://launchpad.net/+tour">Take the tour</a>
          &nbsp;&bull;&nbsp;
          <a href="https://help.launchpad.net/">Read the guide</a>
          <form id="globalsearch" method="get"
                accept-charset="UTF-8"
                action="https://launchpad.net/+search">
            <input type="search" id="search-text" name="field.text" />
            <input type="image" src="/@@/search" style="vertical-align:5%" alt="Search Launchpad" />
          </form>
        </div>

    </div>

    <div class="colophon">
      &copy; 2004-2016
      <a href="http://canonical.com/">Canonical&nbsp;Ltd.</a>
      &nbsp;&bull;&nbsp;
      <a href="https://launchpad.net/legal">Terms of use</a>
      &nbsp;&bull;&nbsp;
      <a href="/feedback">Contact Launchpad Support</a>

      &nbsp;&bull;&nbsp;
      <a href="http://blog.launchpad.net/">Blog</a>

        &nbsp;&bull;&nbsp;
        <a href="http://www.canonical.com/about-canonical/careers">Careers</a>

      &nbsp;&bull;&nbsp;
      <a href="https://twitter.com/launchpadstatus">System status</a>
      <span id="lp-version">
      &nbsp;&bull;&nbsp;
        r18232

        (<a href="https://dev.launchpad.net/">Get the code!</a>)
      </span>
    </div>
  </div>

    </div><!-- yui-d0-->

  <script id="json-cache-script">LP.cache = {"related_features": {}};</script>

  </body>

  <!--
    Facet name: unknown
    Page type: main_only
    Has global search: True
    Has application tabs: True
    Has side portlets: False

    At least 4 queries/external actions issued in 0.11 seconds OOPS-c468be3f4d79009221aa4efed904acac
    Features: {'app.mainsite_only.canonical_url': None, 'js.yui_version': None, 'visible_render_time': None, 'baselayout.careers_link.disabled': None, 'hard_timeout': None}

    r18232

    -->

</html>

Related branches

Martin Pitt (pitti)
summary: - getPublishedSources() (for ddebs) keeps timing out
+ getPublishedBinaries() (for ddebs) keeps timing out
Colin Watson (cjwatson)
summary: - getPublishedBinaries() (for ddebs) keeps timing out
+ Archive.getPublishedBinaries (for ddebs) keeps timing out
tags: added: lp-soyuz oops trivial
Changed in launchpad:
status: New → Triaged
importance: Undecided → Critical
William Grant (wgrant)
Changed in launchpad:
assignee: nobody → William Grant (wgrant)
status: Triaged → In Progress
Revision history for this message
Launchpad QA Bot (lpqabot) wrote :
tags: added: qa-needstesting
Changed in launchpad:
status: In Progress → Fix Committed
William Grant (wgrant)
tags: added: qa-ok
removed: qa-needstesting
William Grant (wgrant)
Changed in launchpad:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.