Merge lp:~hopem/charms/trusty/ceph-radosgw/lp1517551 into lp:~openstack-charmers-archive/charms/trusty/ceph-radosgw/next

[Edward Hope-Morley]

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #14365 ceph-radosgw-next for hopem mp277882
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/14365/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #13390 ceph-radosgw-next for hopem mp277882
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/13390/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #8032 ceph-radosgw-next for hopem mp277882
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
Timeout occurred (2700s), printing juju status...environment: osci-sv08
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/13501909/
Build: http://10.245.162.77:8080/job/charm_amulet_test/8032/

[Ryan Beisner (1chb1n)]

Rerunning amulet test...

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #8052 ceph-radosgw-next for hopem mp277882
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/13581819/
Build: http://10.245.162.77:8080/job/charm_amulet_test/8052/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #8056 ceph-radosgw-next for hopem mp277882
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/13584216/
Build: http://10.245.162.77:8080/job/charm_amulet_test/8056/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #8057 ceph-radosgw-next for hopem mp277882
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
Timeout occurred (2700s), printing juju status...environment: osci-sv08
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/13586064/
Build: http://10.245.162.77:8080/job/charm_amulet_test/8057/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #8064 ceph-radosgw-next for hopem mp277882
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/13600378/
Build: http://10.245.162.77:8080/job/charm_amulet_test/8064/

[lp:~hopem/charms/trusty/ceph-radosgw/lp1517551]

53. By Edward Hope-Morley on 2015-12-21

synced /next

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #15769 ceph-radosgw-next for hopem mp277882
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/15769/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #14717 ceph-radosgw-next for hopem mp277882
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/14717/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #8383 ceph-radosgw-next for hopem mp277882
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/8383/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #15485 ceph-radosgw-next for hopem mp277882
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/15485/

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #16584 ceph-radosgw-next for hopem mp277882
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/16584/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #8500 ceph-radosgw-next for hopem mp277882
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/8500/

[Chris MacNaughton (chris.macnaughton)]

Thanks for the help, Edward! It looks like this merge proposal merges the two ports that the RadosGW uses currently, both the web server port and the civetweb port. Generally, when using civitweb, port 80 ends up load balanced between ALL RadosGW instances and port 70 is the port used to directly access the targeted machine. Maybe there should be two settings to direct that?

review: Needs Fixing

[James Page (james-page)]

Chris

I think this update is OK - the way the port funnel works means that the configuration option is applied as the port which should be used to access the service, whether in hamode or node - Ed's change just makes this configurable - so if the normal 80 port is used then haproxy listens on 80, civetweb or apache on 70.

Likewise for 8080 - haproxy on 8080, civetweb on 8070.

As the amulet test is passing, we're not regressing function here, so +1 for me on this MP.

review: Approve

[Chris MacNaughton (chris.macnaughton)]

I'm only concerned that we seem to be merging port 70 and port 80 in this change. It doesn't kill me as accessing port 70 currently is only useful if trying to reduce interference in performance testing and shouldn't be done in a production setting.

[James Page (james-page)]

> On 9 Jan 2016, at 15:32, Chris MacNaughton <email address hidden> wrote:
>
> I'm only concerned that we seem to be merging port 70 and port 80 in this change. It doesn't kill me as accessing port 70 currently is only useful if trying to reduce interference in performance testing and shouldn't be done in a production setting.

Maybe I missed something - I'll take another look

> --
> https://code.launchpad.net/~hopem/charms/trusty/ceph-radosgw/lp1517551/+merge/277882
> You are reviewing the proposed merge of lp:~hopem/charms/trusty/ceph-radosgw/lp1517551 into lp:~openstack-charmers/charms/trusty/ceph-radosgw/next.

[Chris MacNaughton (chris.macnaughton)]

These are the two different ports merging that I see

[lp:~hopem/charms/trusty/ceph-radosgw/lp1517551]

54. By Edward Hope-Morley on 2016-01-11

sync /next

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #17076 ceph-radosgw-next for hopem mp277882
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/17076/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #15951 ceph-radosgw-next for hopem mp277882
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/15951/

[James Page (james-page)]

review: Approve

=== modified file 'config.yaml'
--- config.yaml	2015-12-03 23:02:21 +0000
+++ config.yaml	2016-01-11 12:21:21 +0000
@@ -25,6 +25,11 @@
     description: |
       Key ID to import to the apt keyring to support use with arbitary source
       configuration from outside of Launchpad archives or PPA's.
+  port:
+    type: int
+    default: 80
+    description: |
+      The port that the RADOS Gateway will listen on.
   # Keystone integration
   operator-roles:
     default: "Member,Admin"

=== modified file 'hooks/ceph_radosgw_context.py'
--- hooks/ceph_radosgw_context.py	2016-01-11 11:21:53 +0000
+++ hooks/ceph_radosgw_context.py	2016-01-11 12:21:21 +0000
@@ -22,18 +22,17 @@
 
     def __call__(self):
         ctxt = super(HAProxyContext, self).__call__()
+        port = config('port')
 
         # Apache ports
-        a_cephradosgw_api = determine_apache_port(80,
-                                                  singlenode_mode=True)
+        a_cephradosgw_api = determine_apache_port(port, singlenode_mode=True)
 
         port_mapping = {
-            'cephradosgw-server': [
-                80, a_cephradosgw_api]
+            'cephradosgw-server': [port, a_cephradosgw_api]
         }
 
         ctxt['cephradosgw_bind_port'] = determine_api_port(
-            80,
+            port,
             singlenode_mode=True,
         )
 
@@ -101,6 +100,8 @@
             'use_syslog': str(config('use-syslog')).lower(),
             'embedded_webserver': config('use-embedded-webserver'),
             'loglevel': config('loglevel'),
+            'port': determine_apache_port(config('port'),
+                                          singlenode_mode=True)
         }
 
         certs_path = '/var/lib/ceph/nss'

=== modified file 'hooks/hooks.py'
--- hooks/hooks.py	2015-12-27 00:21:00 +0000
+++ hooks/hooks.py	2016-01-11 12:21:21 +0000
@@ -39,6 +39,9 @@
     lsb_release,
     restart_on_change,
 )
+from charmhelpers.contrib.hahelpers.cluster import (
+    determine_apache_port,
+)
 from utils import (
     render_template,
     enable_pocket,
@@ -65,6 +68,9 @@
 from charmhelpers.contrib.openstack.utils import (
     set_os_workload_status,
 )
+
+APACHE_PORTS_CONF = '/etc/apache2/ports.conf'
+
 hooks = Hooks()
 CONFIGS = register_configs()
 
@@ -137,7 +143,8 @@
 
 def emit_apacheconf():
     apachecontext = {
-        "hostname": unit_get('private-address')
+        "hostname": unit_get('private-address'),
+        "port": determine_apache_port(config('port'), singlenode_mode=True)
     }
     site_conf = '/etc/apache2/sites-available/rgw'
     if is_apache_24():
@@ -164,7 +171,11 @@
 
 
 def apache_ports():
-    shutil.copy('files/ports.conf', '/etc/apache2/ports.conf')
+    portscontext = {
+        "port": determine_apache_port(config('port'), singlenode_mode=True)
+    }
+    with open(APACHE_PORTS_CONF, 'w') as portsconf:
+        portsconf.write(render_template('ports.conf', portscontext))
 
 
 def setup_keystone_certs(unit=None, rid=None):
@@ -294,22 +305,22 @@
 @hooks.hook('gateway-relation-joined')
 def gateway_relation():
     relation_set(hostname=unit_get('private-address'),
-                 port=80)
+                 port=config('port'))
 
 
 def start():
     subprocess.call(['service', 'radosgw', 'start'])
-    open_port(port=80)
+    open_port(port=config('port'))
 
 
 def stop():
     subprocess.call(['service', 'radosgw', 'stop'])
-    open_port(port=80)
+    open_port(port=config('port'))
 
 
 def restart():
     subprocess.call(['service', 'radosgw', 'restart'])
-    open_port(port=80)
+    open_port(port=config('port'))
 
 
 @hooks.hook('identity-service-relation-joined')
@@ -318,7 +329,7 @@
         log('Integration with keystone requires ceph >= 0.55')
         sys.exit(1)
 
-    port = 80
+    port = config('port')
     admin_url = '%s:%i/swift' % (canonical_url(None, ADMIN), port)
     internal_url = '%s:%s/swift/v1' % \
         (canonical_url(None, INTERNAL), port)

=== modified file 'templates/ceph.conf'
--- templates/ceph.conf	2016-01-11 11:21:53 +0000
+++ templates/ceph.conf	2016-01-11 12:21:21 +0000
@@ -18,7 +18,7 @@
 rgw socket path = /tmp/radosgw.sock
 log file = /var/log/ceph/radosgw.log
 {% if embedded_webserver %}
-rgw frontends = civetweb port=70
+rgw frontends = civetweb port={{ port }}
 {% elif disable_100_continue %}
 # Turn off 100-continue optimization as stock mod_fastcgi
 # does not support it

=== renamed file 'files/ports.conf' => 'templates/ports.conf'
--- files/ports.conf	2015-01-15 11:10:34 +0000
+++ templates/ports.conf	2016-01-11 12:21:21 +0000
@@ -1,4 +1,4 @@
-Listen 70
+Listen {{ port }}
 
 <IfModule ssl_module>
 	Listen 443

=== modified file 'templates/rgw'
--- templates/rgw	2015-01-14 16:48:07 +0000
+++ templates/rgw	2016-01-11 12:21:21 +0000
@@ -2,7 +2,7 @@
 	FastCgiExternalServer /var/www/s3gw.fcgi -socket /tmp/radosgw.sock
 </IfModule>
 
-<VirtualHost *:70>
+<VirtualHost *:{{ port }}>
 	ServerName {{ hostname }}
 	ServerAdmin [e-mail]
 	DocumentRoot /var/www

=== modified file 'unit_tests/test_ceph_radosgw_context.py'
--- unit_tests/test_ceph_radosgw_context.py	2015-12-02 11:09:50 +0000
+++ unit_tests/test_ceph_radosgw_context.py	2016-01-11 12:21:21 +0000
@@ -168,6 +168,7 @@
             'old_auth': False,
             'use_syslog': 'false',
             'loglevel': 1,
+            'port': 70
         }
         self.assertEqual(expect, mon_ctxt())
 
@@ -202,6 +203,7 @@
             'old_auth': False,
             'use_syslog': 'false',
             'loglevel': 1,
+            'port': 70
         }
         self.assertEqual(expect, mon_ctxt())
 
@@ -228,5 +230,6 @@
             'old_auth': False,
             'use_syslog': 'false',
             'loglevel': 1,
+            'port': 70
         }
         self.assertEqual(expect, mon_ctxt())

=== modified file 'unit_tests/test_hooks.py'
--- unit_tests/test_hooks.py	2015-12-27 00:21:00 +0000
+++ unit_tests/test_hooks.py	2016-01-11 12:21:21 +0000
@@ -127,6 +127,7 @@
         self.unit_get.return_value = '10.0.0.1'
         apachecontext = {
             "hostname": '10.0.0.1',
+            "port": 70,
         }
         vhost_file = '/etc/apache2/sites-available/rgw.conf'
         with patch_open() as (_open, _file):
@@ -167,6 +168,7 @@
         ]
         self.subprocess.call.assert_has_calls(calls)
 
+    @patch.object(ceph_hooks, 'apache_ports', lambda *args: True)
     @patch.object(ceph_hooks, 'mkdir', lambda *args: None)
     def test_config_changed(self):
         _install_packages = self.patch('install_packages')