Merge lp:~cprov/adt-cloud-worker/no-proxy into lp:adt-cloud-worker

[Celso Providelo]

Extending 'no_proxy' testbed setup to allow direct access to 127.0.1.1, internal and novalocal (default PS4 domain). ubuntu.com and lp.net access are still only available via proxy.

Access to .ubuntu.com, .lp.net should continue to happen via http_proxy in adt-cloud, this is different than the original (vm-based) infrastructure.

However, 127.0.1.1, internal and novalocal (default PS4/OS domain) were added to the no_proxy env var, following the original infrastructure configuration.

Testing/probing source are ubuntu-make and ubuntu-developer-tools-center.

All the known-failures (external-access-related ones) remain in the same state (https://docs.google.com/spreadsheets/d/1BblC5MooCac2zcXLZzx2PaWH1BWWc0eYaM9o0TCMz7w/ and https://trello.com/c/zSqZgQZk/67-extra-adjust-adt-cloud-testbed-no-proxy)

[Martin Pitt (pitti)]

Thanks Celso!

review: Approve

[lp:~cprov/adt-cloud-worker/no-proxy]

50. By Celso Providelo on 2015-05-12

typo

51. By Celso Providelo on 2015-05-12

wiki.ubuntu.com will/should not be exposed directly, so squid3 tests will have to cope. 'novalocal' is the default nova domain and should be listed as no_proxy (ubuntu-make and ubuntu-developer-tools-center pass).

52. By Celso Providelo on 2015-05-12

Check for [archive,www].ubuntu.com and *.launchpad.net http[s] access.

53. By Celso Providelo on 2015-05-12

ubuntu.com and launchpad.net should be accessed via proxy.

[Martin Pitt (pitti)]

Thanks muchly, Celso! FWIW, I can't make any statement whether *.ubuntu.com and *.launchpad.net are supposed to be talked to via proxy (as that's not necessary anywhere else in the data center), but I trust you on that.

review: Approve

[Francis Ginther (fginther)]

Approve. Discussed the no_proxy changes with cprov over IRC.

review: Approve

=== modified file 'dep8/tests/test_proxy.py'
--- dep8/tests/test_proxy.py	2015-04-30 12:56:08 +0000
+++ dep8/tests/test_proxy.py	2015-05-12 16:02:38 +0000
@@ -33,6 +33,14 @@
             "Failed to access {}".format(url)
         )
 
+    def test_generic_access(self):
+        # All tests (theoretically) should be able to reach *.launchpad.net
+        # and *.ubuntu.com via http_proxy, for fetching code or conveninent
+        # testing payloads.
+        self.assertAccess('https://launchpad.net/')
+        self.assertAccess('http://archive.ubuntu.com/')
+        self.assertAccess('http://www.ubuntu.com/')
+
     def test_unity_firefox_extension(self):
         # 'unity-firefox-extension' test requires pypi for installing mozmill.
         self.assertAccess('https://pypi.python.org/pypi')
@@ -43,7 +51,7 @@
 
     def test_linux(self):
         # 'linux' test requires git access (tcp:9418) to kernel.ubuntu.com
-        # and a specific github (https) projet for patching.
+        # and a specific github (https) project for patching.
         exit_code = subprocess.call(
             ['git', 'ls-remote',
              'git://kernel.ubuntu.com/ubuntu/kernel-testing.git'],

=== modified file 'uci-nova'
--- uci-nova	2015-05-01 17:49:18 +0000
+++ uci-nova	2015-05-12 16:02:38 +0000
@@ -245,7 +245,7 @@
     # Sets proxy-related environment variables, the local address and the
     # configured archive_host are excluded from http[s] proxying and general
     # http[s] proxying it set according to the user-given --proxy option.
-    environment_setup="archive_host=\`awk '/^deb .*(debian|ubuntu)/ { split(\$2, u, \"/\"); print u[3]; exit }' \"\$root/etc/apt/sources.list\"\`; echo \"no_proxy=\\\"localhost,127.0.0.1,localaddress,.localdomain.com,\${archive_host}\\\"\nhttp_proxy=\\\"${PROXY}\\\"\nhttps_proxy=\\\"${PROXY}\\\"\nHTTP_PROXY=\\\"${PROXY}\\\"\nHTTPS_PROXY=\\\"${PROXY}\\\"\" >> /etc/environment"
+    environment_setup="archive_host=\`awk '/^deb .*(debian|ubuntu)/ { split(\$2, u, \"/\"); print u[3]; exit }' \"\$root/etc/apt/sources.list\"\`; echo \"no_proxy=\\\"127.0.0.1,127.0.1.1,localhost,localdomain,novalocal,internal,\${archive_host}\\\"\nhttp_proxy=\\\"${PROXY}\\\"\nhttps_proxy=\\\"${PROXY}\\\"\nHTTP_PROXY=\\\"${PROXY}\\\"\nHTTPS_PROXY=\\\"${PROXY}\\\"\" >> /etc/environment"
 
     # generate cloud-init user data; mostly for manage_etc_hosts, but also get
     # rid of some unnecessary stuff in the VM