Merge lp:~bzoltan/click/vivid-transition_mirrors into lp:ubuntu/vivid-proposed/click

[Zoltan Balogh]

Transition of locally used mirror servers to the chroots.

Transition of locally used mirror servers to the chroots.

[Zoltan Balogh (bzoltan)]

Some explanation on the MR:

I have chosen to use the local ports and archive mirror because other options seems to be less optimal:

a) There is a way to measure bandwidth to/from the apt's mirror support, but that give a geographically close mirror and not a mirror with a fast access

b) Measuring the bandwidth to mirrors and trying to find the most optimal would need way much more code and would be too complex for a fairly simple problem

I decided to use a strict and limited expression to find the locally used mirror. I accept only the /XX.archive.ubuntu.com $series main/ source line. In case I do not find the mirror the code falls back to the central archive what is a safe choice.

All in all, the job here is to make the Chinese app developer's life easy. I am sure that we can make a more generic and smarter solution to support Ubuntu mirrors, but this one I believe is safe and good enough for now. Also I think it is legit idea to use the same mirror in the chroots what is used on the SDK's host environment.

[Rex Tsai (chihchun)]

Feedback and summing-up of irc discussion

Ubuntu users are suggested to use [countrycode].archive.ubuntu.com, but most users will select a random mirror server from the software center config (software-properties) or intstaller. http://i.imgur.com/2EvQIrv.png It ends up the user will not have xx.archive.ubuntu.com in /etc/apt/sources.list as reference.

It would make more sense that user can pick a mirror server during the new `click root` setup
1. A dialogue for user to pick which mirror to use. [countrycode].[ports|archive].ubuntu.com
2. The dialogue should suggest the default or preferred mirror server for the user. software-properties use user's locale to guess where is the most close mirror site.

Please note only [countrycode].[ports|archive].ubuntu.com are guarantee up-to-date, the other mirror can delay for few weeks. So we only use [countrycode].[ports|archive].ubuntu.com for phone developer.

[lp:~bzoltan/click/vivid-transition_mirrors]

66. By Zoltan Balogh on 2015-01-21

Use the local archive mirror and figure out the ports mirror with geoip lookup

67. By Zoltan Balogh on 2015-01-21

remove redundant imports

68. By Zoltan Balogh on 2015-01-21

Use the official archive mirror

69. By Zoltan Balogh on 2015-01-21

Make it flake8 friendly

70. By Zoltan Balogh on 2015-01-22

remove unused local_mirror_server

71. By Zoltan Balogh on 2015-01-23

Handle exception when chroot is created without network and add dependecies

[Michael Vogt (mvo)]

This got merged in http://bazaar.launchpad.net/~click-hackers/click/devel/revision/555

review: Approve

=== modified file 'click/chroot.py'
--- click/chroot.py	2014-11-14 12:29:14 +0000
+++ click/chroot.py	2015-01-23 16:18:15 +0000
@@ -26,6 +26,8 @@
     "ClickChrootDoesNotExistException",
     ]
 
+import urllib
+import urllib.request
 import os
 import pwd
 import re
@@ -34,6 +36,7 @@
 import subprocess
 import sys
 from textwrap import dedent
+from xml.etree import ElementTree
 
 
 framework_base = {
@@ -215,6 +218,21 @@
     pass
 
 
+def get_geoip_country_code_prefix():
+    GEOIP_SERVER = "http://geoip.ubuntu.com/lookup"
+    try:
+        with urllib.request.urlopen(GEOIP_SERVER) as f:
+            xml_data = f.read()
+        try:
+            et = ElementTree.fromstring(xml_data)
+            return et.find("CountryCode").text.lower()+"."
+        except ElementTree.ParseError:
+            pass
+    except:
+        pass
+    return ""
+
+
 class ClickChroot:
 
     DAEMON_POLICY = dedent("""\
@@ -247,11 +265,14 @@
         if chroots_dir is None:
             chroots_dir = "/var/lib/schroot/chroots"
         self.chroots_dir = chroots_dir
+
+        self.country_code = get_geoip_country_code_prefix()
         # this doesn't work because we are running this under sudo
         if 'DEBOOTSTRAP_MIRROR' in os.environ:
             self.archive = os.environ['DEBOOTSTRAP_MIRROR']
         else:
-            self.archive = "http://archive.ubuntu.com/ubuntu"
+            self.archive = "http://%sarchive.ubuntu.com/ubuntu" \
+                           % self.country_code
         if "SUDO_USER" in os.environ:
             self.user = os.environ["SUDO_USER"]
         elif "PKEXEC_UID" in os.environ:
@@ -331,7 +352,8 @@
                         mount=mount))
 
     def _generate_sources(self, series, native_arch, target_arch, components):
-        ports_mirror = "http://ports.ubuntu.com/ubuntu-ports"
+        ports_mirror = "http://%sports.ubuntu.com/ubuntu-ports" \
+            % self.country_code
         pockets = ['%s' % series]
         for pocket in ['updates', 'security']:
             pockets.append('%s-%s' % (series, pocket))
@@ -467,7 +489,8 @@
             proxy = os.environ["http_proxy"]
         if not proxy:
             proxy = subprocess.check_output(
-                'unset x; eval "$(apt-config shell x Acquire::HTTP::Proxy)"; echo "$x"',
+                'unset x; eval "$(apt-config shell x Acquire::HTTP::Proxy)"; \
+                 echo "$x"',
                 shell=True, universal_newlines=True).strip()
         build_pkgs = [
             "build-essential", "fakeroot",

=== modified file 'debian/control'
--- debian/control	2014-11-03 12:49:25 +0000
+++ debian/control	2015-01-23 16:18:15 +0000
@@ -3,7 +3,7 @@
 Priority: optional
 Maintainer: Colin Watson <[e-mail]>
 Standards-Version: 3.9.5
-Build-Depends: debhelper (>= 9~), dh-autoreconf, intltool, python3:any (>= 3.2), python3-all:any, python3-setuptools, python3-apt, python3-debian, python3-gi, python3:any (>= 3.3) | python3-mock, pep8, python3-pep8, pyflakes, python3-sphinx, pkg-config, valac, gobject-introspection (>= 0.6.7), libgirepository1.0-dev (>= 0.6.7), libglib2.0-dev (>= 2.34), gir1.2-glib-2.0, libjson-glib-dev (>= 0.10), libgee-0.8-dev, libpackagekit-glib2-dev (>= 0.7.2), python3-coverage, python3-six, dh-systemd (>= 1.3)
+Build-Depends: debhelper (>= 9~), dh-autoreconf, intltool, python3:any (>= 3.2), python3-all:any, python3-setuptools, python3-apt, python3-debian, python3-gi, python3:any (>= 3.3) | python3-mock, pep8, python3-pep8, pyflakes, python3-sphinx, pkg-config, valac, gobject-introspection (>= 0.6.7), libgirepository1.0-dev (>= 0.6.7), libglib2.0-dev (>= 2.34), gir1.2-glib-2.0, libjson-glib-dev (>= 0.10), libgee-0.8-dev, libpackagekit-glib2-dev (>= 0.7.2), python3-coverage, python3-six, python3-urllib3, dh-systemd (>= 1.3)
 Vcs-Bzr: https://code.launchpad.net/~ubuntu-managed-branches/click/click
 Vcs-Browser: http://bazaar.launchpad.net/~ubuntu-managed-branches/click/click/files
 X-Auto-Uploader: no-rewrite-version
@@ -40,7 +40,7 @@
 Package: python3-click
 Section: python
 Architecture: any
-Depends: ${misc:Depends}, ${python3:Depends}, gir1.2-click-0.4 (= ${binary:Version}), gir1.2-glib-2.0, python3-apt, python3-debian, python3-gi
+Depends: ${misc:Depends}, ${python3:Depends}, gir1.2-click-0.4 (= ${binary:Version}), gir1.2-glib-2.0, python3-apt, python3-debian, python3-gi, python3-urllib3
 Conflicts: python3-click-package
 Replaces: python3-click-package
 Provides: python3-click-package