Merge lp:~freyes/charms/trusty/nova-cloud-controller/bug-989337 into lp:~openstack-charmers-archive/charms/trusty/nova-cloud-controller/next

Proposed by Felipe Reyes
Status: Merged
Approved by: Edward Hope-Morley
Approved revision: 134
Merged at revision: 129
Proposed branch: lp:~freyes/charms/trusty/nova-cloud-controller/bug-989337
Merge into: lp:~openstack-charmers-archive/charms/trusty/nova-cloud-controller/next
Diff against target: 268 lines (+145/-1)
10 files modified
hooks/nova_cc_context.py (+25/-0)
hooks/nova_cc_hooks.py (+9/-0)
hooks/nova_cc_utils.py (+3/-1)
metadata.yaml (+2/-0)
templates/folsom/nova.conf (+5/-0)
templates/grizzly/nova.conf (+5/-0)
templates/havana/nova.conf (+4/-0)
templates/icehouse/nova.conf (+4/-0)
unit_tests/test_nova_cc_contexts.py (+87/-0)
unit_tests/test_nova_cc_hooks.py (+1/-0)
To merge this branch: bzr merge lp:~freyes/charms/trusty/nova-cloud-controller/bug-989337
Reviewer Review Type Date Requested Status
Jorge Niedbalski (community) Approve
Edward Hope-Morley Pending
James Troup Pending
OpenStack Charmers Pending
Review via email: mp+244857@code.launchpad.net

This proposal supersedes a proposal from 2014-12-15.

Description of the change

This branch extends the charm to install python-memcache, and configure nova.conf adding the key memcached_servers when a relationship with memcached service is established.

If multiple units of memcached are available, all of them are used (memcached_servers = host1:port1,host2:port,host3:port3)

To secure memcached access this relies in the memcached charm ability to use iptables rules to only allow access to related machines[0]

This patch can be tested using the deployer file available at https://code.launchpad.net/~openstack-charm-testers/+junk/nova-memcached

[0] http://bazaar.launchpad.net/~charmers/charms/trusty/memcached/trunk/revision/61

To post a comment you must log in.
Revision history for this message
Ryan Beisner (1chb1n) wrote : Posted in a previous version of this proposal

UOSCI bot says:
charm_lint_check #771 nova-cloud-controller-next for freyes mp239671
    LINT OK: believed to pass, but you should confirm results

LINT Results (max last 4 lines) from
/var/lib/jenkins/workspace/charm_lint_check/make-lint.771:
I: config.yaml: option haproxy-client-timeout has no default value
I: config.yaml: option ssl_cert has no default value
I: config.yaml: option nvp-l3-uuid has no default value
I: config.yaml: option os-internal-network has no default value

Full lint output: http://paste.ubuntu.com/8694351/
Build: http://10.98.191.181:8080/job/charm_lint_check/771/

Revision history for this message
Ryan Beisner (1chb1n) wrote : Posted in a previous version of this proposal

UOSCI bot says:
charm_unit_test #579 nova-cloud-controller-next for freyes mp239671
    UNIT FAIL: unit-test failed

UNIT Results (max last 4 lines) from
/var/lib/jenkins/workspace/charm_unit_test/unit-test.579:
Ran 4 tests in 0.150s

FAILED (errors=2)
make: *** [unit_test] Error 1

Full unit output: http://paste.ubuntu.com/8694352/
Build: http://10.98.191.181:8080/job/charm_unit_test/579/

Revision history for this message
Ryan Beisner (1chb1n) wrote : Posted in a previous version of this proposal

UOSCI bot says:
charm_amulet_test #304 nova-cloud-controller-next for freyes mp239671
    AMULET OK: believed to pass, but you should confirm results

AMULET Results (max last 4 lines) from
/var/lib/jenkins/workspace/charm_amulet_test/make-test.304:
juju-test.conductor DEBUG : Tearing down osci-sv05 juju environment
juju-test.conductor DEBUG : Calling "juju destroy-environment -y osci-sv05"
WARNING cannot delete security group "juju-osci-sv05-0". Used by another environment?
juju-test INFO : Results: 3 passed, 0 failed, 0 errored

Full amulet output: http://paste.ubuntu.com/8694683/
Build: http://10.98.191.181:8080/job/charm_amulet_test/304/

Revision history for this message
Ryan Beisner (1chb1n) wrote : Posted in a previous version of this proposal

UOSCI bot says:
charm_unit_test #582 nova-cloud-controller-next for freyes mp239671
    UNIT FAIL: unit-test failed

UNIT Results (max last 4 lines) from
/var/lib/jenkins/workspace/charm_unit_test/unit-test.582:
Ran 4 tests in 0.153s

FAILED (errors=2)
make: *** [unit_test] Error 1

Full unit output: http://paste.ubuntu.com/8705925/
Build: http://10.98.191.181:8080/job/charm_unit_test/582/

Revision history for this message
Ryan Beisner (1chb1n) wrote : Posted in a previous version of this proposal

UOSCI bot says:
charm_lint_check #774 nova-cloud-controller-next for freyes mp239671
    LINT OK: believed to pass, but you should confirm results

LINT Results (max last 4 lines) from
/var/lib/jenkins/workspace/charm_lint_check/make-lint.774:
I: config.yaml: option haproxy-client-timeout has no default value
I: config.yaml: option ssl_cert has no default value
I: config.yaml: option nvp-l3-uuid has no default value
I: config.yaml: option os-internal-network has no default value

Full lint output: http://paste.ubuntu.com/8705924/
Build: http://10.98.191.181:8080/job/charm_lint_check/774/

Revision history for this message
Ryan Beisner (1chb1n) wrote : Posted in a previous version of this proposal

UOSCI bot says:
charm_amulet_test #307 nova-cloud-controller-next for freyes mp239671
    AMULET FAIL: amulet-test failed

AMULET Results (max last 4 lines) from
/var/lib/jenkins/workspace/charm_amulet_test/make-test.307:
    return self.read(buflen)
  File "/usr/lib/python2.7/ssl.py", line 260, in read
    return self._sslobj.read(len)
socket.error: [Errno 110] Connection timed out

Full amulet output: http://paste.ubuntu.com/8706467/
Build: http://10.98.191.181:8080/job/charm_amulet_test/307/

Revision history for this message
Edward Hope-Morley (hopem) wrote : Posted in a previous version of this proposal

I'm getting a couple of unit test failures here. Looks like a couple of extra mocks needed - see attached u/t output.

review: Needs Fixing
Revision history for this message
Edward Hope-Morley (hopem) wrote : Posted in a previous version of this proposal
Download full text (3.7 KiB)

Hmm can't attach so pasting:

$ make unit_test
Starting unit tests...
nose.plugins.cover: ERROR: Coverage not available: unable to import coverage module
test_instance_console_context_with_memcache (unit_tests.test_nova_cc_contexts.NovaComputeContextTests) ... ok
test_instance_console_context_without_memcache (unit_tests.test_nova_cc_contexts.NovaComputeContextTests) ... ok
Failure: OSError ([Errno 2] No such file or directory) ... ERROR
Failure: OSError ([Errno 2] No such file or directory) ... ERROR

======================================================================
ERROR: Failure: OSError ([Errno 2] No such file or directory)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/home/gizmo/Documents/reviews/nova-cloud-controller/bug-989337/unit_tests/test_nova_cc_hooks.py", line 6, in <module>
    import nova_cc_utils as utils
  File "hooks/nova_cc_utils.py", line 142, in <module>
    nova_cc_context.NeutronPostgresqlDBContext(),
  File "hooks/nova_cc_context.py", line 268, in __init__
    self).__init__(config('neutron-database'))
  File "hooks/charmhelpers/core/hookenv.py", line 44, in wrapper
    res = func(*args, **kwargs)
  File "hooks/charmhelpers/core/hookenv.py", line 281, in config
    config_data = json.loads(subprocess.check_output(config_cmd_line))
  File "/usr/lib/python2.7/subprocess.py", line 566, in check_output
    process = Popen(stdout=PIPE, *popenargs, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

======================================================================
ERROR: Failure: OSError ([Errno 2] No such file or directory)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/home/gizmo/Documents/reviews/nova-cloud-controller/bug-989337/unit_tests/test_nova_cc_utils.py", line 11, in <module>
    import nova_cc_utils as utils
  File "hooks/nova_cc_utils.py", line 142, in <module>
    nova_cc_context.NeutronPostgresqlDBContext(),
  File "hooks/nova_cc_context.py", line 268, in __init__
    self).__init__(config('neutron-database'))
  File "hooks/charmhelpers/core/hookenv.py", line 44, in wrapper
    res = func(*args, **kwa...

Read more...

Revision history for this message
Edward Hope-Morley (hopem) wrote : Posted in a previous version of this proposal

FWIW, other than the above this lgtm

Revision history for this message
Felipe Reyes (freyes) wrote : Posted in a previous version of this proposal

Edward, I fixed the broken tests, the patch was a little bit larger than what I expected, but they are working OK now.

Name Stmts Miss Cover Missing
-----------------------------------------------------
hooks/nova_cc_context 166 112 33% 21-28, 39-41, 48-59, 65-79, 86-104, 116-169, 180-195, 203-204, 208, 212-214, 219, 222-236, 242-256, 273-276, 281-283, 297-299
hooks/nova_cc_hooks 447 146 67% 132-135, 148-149, 173, 184-185, 189, 224, 229-234, 245, 325-328, 336-339, 345-348, 358-374, 383-385, 395-409, 413-422, 508, 518, 522-523, 576, 582-592, 597-608, 618-628, 633-672, 682-697, 705-709, 734-743, 767, 772-780, 806, 861, 865-868
hooks/nova_cc_utils 442 111 75% 296-301, 312-315, 325-326, 382, 384, 428-430, 434, 448-456, 463-468, 472-486, 592-594, 599-602, 607, 611, 635-636, 650-652, 673-674, 680-703, 707-713, 717-723, 729, 735, 742, 753-757, 842, 902-908, 912-914, 918-921, 925-937
-----------------------------------------------------
TOTAL 1055 369 65%
----------------------------------------------------------------------
Ran 99 tests in 7.518s

Revision history for this message
Ryan Beisner (1chb1n) wrote : Posted in a previous version of this proposal

UOSCI bot says:
charm_lint_check #922 nova-cloud-controller-next for freyes mp239671
    LINT OK: passed

LINT Results (max last 5 lines):
  I: config.yaml: option os-admin-network has no default value
  I: config.yaml: option haproxy-client-timeout has no default value
  I: config.yaml: option ssl_cert has no default value
  I: config.yaml: option nvp-l3-uuid has no default value
  I: config.yaml: option os-internal-network has no default value

Full lint test output: http://paste.ubuntu.com/8822840/
Build: http://10.98.191.181:8080/job/charm_lint_check/922/

Revision history for this message
Ryan Beisner (1chb1n) wrote : Posted in a previous version of this proposal

UOSCI bot says:
charm_unit_test #757 nova-cloud-controller-next for freyes mp239671
    UNIT OK: passed

UNIT Results (max last 5 lines):
  hooks/nova_cc_hooks 447 146 67% 132-135, 148-149, 173, 184-185, 189, 224, 229-234, 245, 325-328, 336-339, 345-348, 358-374, 383-385, 395-409, 413-422, 508, 518, 522-523, 576, 582-592, 597-608, 618-628, 633-672, 682-697, 705-709, 734-743, 767, 772-780, 806, 861, 865-868
  hooks/nova_cc_utils 442 111 75% 296-301, 312-315, 325-326, 382, 384, 428-430, 434, 448-456, 463-468, 472-486, 592-594, 599-602, 607, 611, 635-636, 650-652, 673-674, 680-703, 707-713, 717-723, 729, 735, 742, 753-757, 842, 902-908, 912-914, 918-921, 925-937
  TOTAL 1055 369 65%
  Ran 99 tests in 8.707s
  OK

Full unit test output: http://paste.ubuntu.com/8822843/
Build: http://10.98.191.181:8080/job/charm_unit_test/757/

Revision history for this message
Ryan Beisner (1chb1n) wrote : Posted in a previous version of this proposal

UOSCI bot says:
charm_amulet_test #340 nova-cloud-controller-next for freyes mp239671
    AMULET OK: passed

AMULET Results (max last 5 lines):
  juju-test.conductor.15-basic-trusty-icehouse RESULT :
  juju-test.conductor DEBUG : Tearing down osci-sv07 juju environment
  juju-test.conductor DEBUG : Calling "juju destroy-environment -y osci-sv07"
  WARNING cannot delete security group "juju-osci-sv07-0". Used by another environment?
  juju-test INFO : Results: 3 passed, 0 failed, 0 errored

Full amulet test output: http://paste.ubuntu.com/8823194/
Build: http://10.98.191.181:8080/job/charm_amulet_test/340/

Revision history for this message
Ryan Beisner (1chb1n) wrote : Posted in a previous version of this proposal

UOSCI bot says:
charm_lint_check #979 nova-cloud-controller-next for freyes mp241318
    LINT OK: passed

LINT Results (max last 5 lines):
  I: config.yaml: option os-admin-network has no default value
  I: config.yaml: option haproxy-client-timeout has no default value
  I: config.yaml: option ssl_cert has no default value
  I: config.yaml: option nvp-l3-uuid has no default value
  I: config.yaml: option os-internal-network has no default value

Full lint test output: http://paste.ubuntu.com/8925102/
Build: http://10.98.191.181:8080/job/charm_lint_check/979/

Revision history for this message
Ryan Beisner (1chb1n) wrote : Posted in a previous version of this proposal

UOSCI bot says:
charm_unit_test #814 nova-cloud-controller-next for freyes mp241318
    UNIT OK: passed

UNIT Results (max last 5 lines):
  hooks/nova_cc_hooks 447 146 67% 132-135, 148-149, 173, 184-185, 189, 224, 229-234, 245, 325-328, 336-339, 345-348, 358-374, 383-385, 395-409, 413-422, 508, 518, 522-523, 576, 582-592, 597-608, 618-628, 633-672, 682-697, 705-709, 734-743, 767, 772-780, 806, 861, 865-868
  hooks/nova_cc_utils 442 111 75% 296-301, 312-315, 325-326, 382, 384, 428-430, 434, 448-456, 463-468, 472-486, 592-594, 599-602, 607, 611, 635-636, 650-652, 673-674, 680-703, 707-713, 717-723, 729, 735, 742, 753-757, 842, 902-908, 912-914, 918-921, 925-937
  TOTAL 1055 369 65%
  Ran 99 tests in 8.712s
  OK

Full unit test output: http://paste.ubuntu.com/8925116/
Build: http://10.98.191.181:8080/job/charm_unit_test/814/

Revision history for this message
Ryan Beisner (1chb1n) wrote : Posted in a previous version of this proposal

UOSCI bot says:
charm_amulet_test #359 nova-cloud-controller-next for freyes mp241318
    AMULET OK: passed

AMULET Results (max last 5 lines):
  juju-test.conductor.15-basic-trusty-icehouse RESULT :
  juju-test.conductor DEBUG : Tearing down osci-sv07 juju environment
  juju-test.conductor DEBUG : Calling "juju destroy-environment -y osci-sv07"
  WARNING cannot delete security group "juju-osci-sv07-0". Used by another environment?
  juju-test INFO : Results: 3 passed, 0 failed, 0 errored

Full amulet test output: http://paste.ubuntu.com/8925544/
Build: http://10.98.191.181:8080/job/charm_amulet_test/359/

Revision history for this message
Edward Hope-Morley (hopem) wrote : Posted in a previous version of this proposal

Felipe, thanks for working on this it is much needed.

So some initial comments;

  * could you please rename the cache-relation-* to memcache-relation-*
  * is there are reason for replacing config() with hookenv.config()? I know it is not the nicest coding style but it is the same across all charms so to do it differently here would be breaking convention.
  * some more comments inline

review: Needs Fixing
Revision history for this message
Felipe Reyes (freyes) wrote : Posted in a previous version of this proposal

Hi,

> * could you please rename the cache-relation-* to memcache-relation-*
I initially used memcache-relation-* hooks, but they weren't triggered, a quick look to mediawiki charm also uses cache-relation-* for memcache, I'll give it another try to double check this.

> * is there are reason for replacing config() with hookenv.config()? I know
> it is not the nicest coding style but it is the same across all charms so to
> do it differently here would be breaking convention.
I know, but importing config() function made fail the mocking due to the way the modules are imported, if this is a blocker I totally understand and I can revisit this to find a way to make the tests happy.

Best,

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

UOSCI bot says:
charm_lint_check #1151 nova-cloud-controller-next for freyes mp241318
    LINT OK: passed

LINT Results (max last 5 lines):
  I: config.yaml: option os-admin-network has no default value
  I: config.yaml: option haproxy-client-timeout has no default value
  I: config.yaml: option ssl_cert has no default value
  I: config.yaml: option nvp-l3-uuid has no default value
  I: config.yaml: option os-internal-network has no default value

Full lint test output: http://paste.ubuntu.com/9125387/
Build: http://10.98.191.181:8080/job/charm_lint_check/1151/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

UOSCI bot says:
charm_unit_test #985 nova-cloud-controller-next for freyes mp241318
    UNIT OK: passed

UNIT Results (max last 5 lines):
  hooks/nova_cc_hooks 447 146 67% 132-135, 148-149, 173, 184-185, 189, 224, 229-234, 245, 325-328, 336-339, 345-348, 358-374, 383-385, 395-409, 413-422, 508, 518, 522-523, 576, 582-592, 597-608, 618-628, 633-672, 682-697, 705-709, 734-743, 767, 772-780, 806, 861, 865-868
  hooks/nova_cc_utils 442 111 75% 296-301, 312-315, 325-326, 382, 384, 428-430, 434, 448-456, 463-468, 472-486, 592-594, 599-602, 607, 611, 635-636, 650-652, 673-674, 680-703, 707-713, 717-723, 729, 735, 742, 753-757, 842, 902-908, 912-914, 918-921, 925-937
  TOTAL 1054 369 65%
  Ran 99 tests in 9.596s
  OK

Full unit test output: http://paste.ubuntu.com/9125402/
Build: http://10.98.191.181:8080/job/charm_unit_test/985/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

UOSCI bot says:
charm_amulet_test #493 nova-cloud-controller-next for freyes mp241318
    AMULET FAIL: amulet-test failed

AMULET Results (max last 5 lines):
"
  WARNING cannot delete security group "juju-osci-sv07-0". Used by another environment?
  juju-test INFO : Results: 2 passed, 0 failed, 1 errored
  ERROR subprocess encountered error code 124
  make: *** [test] Error 124

Full amulet test output: http://paste.ubuntu.com/9125932/
Build: http://10.98.191.181:8080/job/charm_amulet_test/493/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

UOSCI bot says:
charm_lint_check #1159 nova-cloud-controller-next for freyes mp242384
    LINT OK: passed

LINT Results (max last 5 lines):
  I: config.yaml: option os-admin-network has no default value
  I: config.yaml: option haproxy-client-timeout has no default value
  I: config.yaml: option ssl_cert has no default value
  I: config.yaml: option nvp-l3-uuid has no default value
  I: config.yaml: option os-internal-network has no default value

Full lint test output: http://paste.ubuntu.com/9127053/
Build: http://10.98.191.181:8080/job/charm_lint_check/1159/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

UOSCI bot says:
charm_unit_test #993 nova-cloud-controller-next for freyes mp242384
    UNIT OK: passed

UNIT Results (max last 5 lines):
  hooks/nova_cc_hooks 445 146 67% 132-135, 148-149, 173, 184-185, 189, 221, 226-231, 242, 322-325, 333-336, 342-345, 355-371, 380-382, 392-406, 410-419, 505, 515, 519-520, 573, 579-589, 594-605, 615-625, 630-669, 679-694, 702-706, 731-740, 764, 769-777, 803, 858, 862-865
  hooks/nova_cc_utils 445 112 75% 298-303, 314-317, 327-328, 384, 386, 432-434, 438, 452-460, 467-472, 476-490, 546, 595-597, 602-605, 610, 614, 638-639, 653-655, 676-677, 683-706, 710-716, 720-726, 732, 738, 745, 756-760, 845, 905-911, 915-917, 921-924, 928-940
  TOTAL 1055 370 65%
  Ran 98 tests in 10.450s
  OK

Full unit test output: http://paste.ubuntu.com/9127055/
Build: http://10.98.191.181:8080/job/charm_unit_test/993/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

UOSCI bot says:
charm_amulet_test #501 nova-cloud-controller-next for freyes mp242384
    AMULET FAIL: amulet-test failed

AMULET Results (max last 5 lines):
  subprocess.CalledProcessError: Command '['juju-deployer', '-W', '-L', '-c', '/tmp/amulet-juju-deployer-bskSEW.json', '-e', 'osci-sv05', '-t', '1000', 'osci-sv05']' returned non-zero exit status 1
  WARNING cannot delete security group "juju-osci-sv05-0". Used by another environment?
  juju-test INFO : Results: 0 passed, 2 failed, 1 errored
  ERROR subprocess encountered error code 2
  make: *** [test] Error 2

Full amulet test output: http://paste.ubuntu.com/9127779/
Build: http://10.98.191.181:8080/job/charm_amulet_test/501/

Revision history for this message
Felipe Reyes (freyes) wrote : Posted in a previous version of this proposal

> juju-test.conductor.15-basic-trusty-icehouse RESULT : ✔
> juju-test.conductor DEBUG : Tearing down osci-sv07 juju environment
> juju-test.conductor DEBUG : Calling "juju destroy-environment -y osci-sv07ERROR:root:Make target returned non-zero.
> "
> WARNING cannot delete security group "juju-osci-sv07-0". Used by another environment?
> juju-test INFO : Results: 2 passed, 0 failed, 1 errored
> ERROR subprocess encountered error code 124

Should I do something with this error?

review: Needs Resubmitting
Revision history for this message
Felipe Reyes (freyes) : Posted in a previous version of this proposal
review: Abstain
Revision history for this message
Felipe Reyes (freyes) wrote : Posted in a previous version of this proposal

By the way, this patch can be tested using the bundler file available at https://code.launchpad.net/~openstack-charm-testers/+junk/nova-memcached

Thanks,

Revision history for this message
James Troup (elmo) wrote : Posted in a previous version of this proposal

memcache (by design) has no ACLs and is not safe to deploy without some sort of protection (e.g. local iptables rules or being run on a private network not accessible to anything but trusted machines). How is this MP dealing with that?

review: Needs Information
Revision history for this message
Edward Hope-Morley (hopem) wrote : Posted in a previous version of this proposal
Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

UOSCI bot says:
charm_lint_check #1236 nova-cloud-controller-next for freyes mp242980
    LINT OK: passed

LINT Results (max last 5 lines):
  I: config.yaml: option os-admin-network has no default value
  I: config.yaml: option haproxy-client-timeout has no default value
  I: config.yaml: option ssl_cert has no default value
  I: config.yaml: option nvp-l3-uuid has no default value
  I: config.yaml: option os-internal-network has no default value

Full lint test output: http://paste.ubuntu.com/9256250/
Build: http://10.98.191.181:8080/job/charm_lint_check/1236/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

UOSCI bot says:
charm_unit_test #1070 nova-cloud-controller-next for freyes mp242980
    UNIT OK: passed

UNIT Results (max last 5 lines):
  hooks/nova_cc_hooks 445 146 67% 132-135, 148-149, 173, 184-185, 189, 221, 226-231, 242, 322-325, 333-336, 342-345, 355-371, 380-382, 392-406, 410-419, 505, 515, 519-520, 573, 579-589, 594-605, 615-625, 630-669, 679-694, 702-706, 731-740, 764, 769-777, 803, 858, 862-865
  hooks/nova_cc_utils 445 112 75% 298-303, 314-317, 327-328, 384, 386, 432-434, 438, 452-460, 467-472, 476-490, 546, 595-597, 602-605, 610, 614, 638-639, 653-655, 676-677, 683-706, 710-716, 720-726, 732, 738, 745, 756-760, 845, 905-911, 915-917, 921-924, 928-940
  TOTAL 1057 370 65%
  Ran 99 tests in 10.551s
  OK

Full unit test output: http://paste.ubuntu.com/9256252/
Build: http://10.98.191.181:8080/job/charm_unit_test/1070/

Revision history for this message
Edward Hope-Morley (hopem) wrote : Posted in a previous version of this proposal

We should already be leveraging split network support here e.g. have memcached running on the os-admin-network if defined. That should suffice.

Revision history for this message
Edward Hope-Morley (hopem) wrote : Posted in a previous version of this proposal

See inline comment

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

UOSCI bot says:
charm_amulet_test #539 nova-cloud-controller-next for freyes mp242980
    AMULET OK: passed

AMULET Results (max last 5 lines):
  juju-test.conductor.15-basic-trusty-icehouse RESULT :
  juju-test.conductor DEBUG : Tearing down osci-sv07 juju environment
  juju-test.conductor DEBUG : Calling "juju destroy-environment -y osci-sv07"
  WARNING cannot delete security group "juju-osci-sv07-0". Used by another environment?
  juju-test INFO : Results: 3 passed, 0 failed, 0 errored

Full amulet test output: http://paste.ubuntu.com/9256538/
Build: http://10.98.191.181:8080/job/charm_amulet_test/539/

Revision history for this message
Jorge Niedbalski (niedbalski) wrote : Posted in a previous version of this proposal

Will not be so easy to enable SASL support on the proposal, mostly since the memcache client requires to support it, and the default python-memcache module seems to not support it. We could use python-binary-memcached or pylibmc-sasl and that will require changes on openstack upstream and packages .. I think the best we could do now is make some iptables assertions based on the related ncc hosts ..

Revision history for this message
Felipe Reyes (freyes) wrote : Posted in a previous version of this proposal

@elmo, I'm addressing your concerns about memcached security adding iptables rules to allow access only to related machines, I would like to get your feedback on that MP[0]

[0] https://code.launchpad.net/~freyes/charms/trusty/memcached/python-rewrite/+merge/243407

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_lint_check #202 nova-cloud-controller-next for freyes mp244432
    LINT OK: passed

Build: http://10.230.18.80:8080/job/charm_lint_check/202/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_unit_test #165 nova-cloud-controller-next for freyes mp244432
    UNIT OK: passed

Build: http://10.230.18.80:8080/job/charm_unit_test/165/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_amulet_test #120 nova-cloud-controller-next for freyes mp244432
    AMULET OK: passed

Build: http://10.230.18.80:8080/job/charm_amulet_test/120/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_lint_check #83 nova-cloud-controller-next for freyes mp244432
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/83/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_lint_check #89 nova-cloud-controller-next for freyes mp244432
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/89/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_unit_test #118 nova-cloud-controller-next for freyes mp244432
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/118/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_unit_test #121 nova-cloud-controller-next for freyes mp244432
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/121/

Revision history for this message
Jorge Niedbalski (niedbalski) wrote : Posted in a previous version of this proposal

All LGTM. Unit and Amulet tests passing, the python written memcache charm has already landed. So just a very minor string refactor required.

Thanks Felipe

review: Approve
Revision history for this message
Felipe Reyes (freyes) wrote : Posted in a previous version of this proposal

@niedbalski, updated MP to address your comment.

Thanks for reviewing it :)

Revision history for this message
Edward Hope-Morley (hopem) wrote : Posted in a previous version of this proposal

Felipe, this mostly lgtm but I have a couple of nits. This is top of my list of reviews though!

review: Needs Fixing
Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_lint_check #183 nova-cloud-controller-next for freyes mp244763
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/183/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_unit_test #216 nova-cloud-controller-next for freyes mp244763
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/216/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_amulet_test #236 nova-cloud-controller-next for freyes mp244763
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
  ERROR subprocess encountered error code 1
  make: *** [test] Error 1

Full amulet test output: http://paste.ubuntu.com/9535877/
Build: http://10.245.162.77:8080/job/charm_amulet_test/236/

Revision history for this message
Felipe Reyes (freyes) wrote :

Edward, thanks for reviewing my patch. I pushed this new version that addresses your comments.

Best,

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #189 nova-cloud-controller-next for freyes mp244857
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/189/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #218 nova-cloud-controller-next for freyes mp244857
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/218/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #238 nova-cloud-controller-next for freyes mp244857
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
  ERROR subprocess encountered error code 1
  make: *** [test] Error 1

Full amulet test output: http://paste.ubuntu.com/9540988/
Build: http://10.245.162.77:8080/job/charm_amulet_test/238/

Revision history for this message
Edward Hope-Morley (hopem) wrote :

OK this lgtm +1

Revision history for this message
Jorge Niedbalski (niedbalski) :
review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== added symlink 'hooks/memcache-relation-broken'
=== target is u'nova_cc_hooks.py'
=== added symlink 'hooks/memcache-relation-changed'
=== target is u'nova_cc_hooks.py'
=== added symlink 'hooks/memcache-relation-departed'
=== target is u'nova_cc_hooks.py'
=== added symlink 'hooks/memcache-relation-joined'
=== target is u'nova_cc_hooks.py'
=== modified file 'hooks/nova_cc_context.py'
--- hooks/nova_cc_context.py 2014-12-03 23:49:34 +0000
+++ hooks/nova_cc_context.py 2014-12-16 14:11:00 +0000
@@ -6,6 +6,7 @@
6 ERROR,6 ERROR,
7 unit_get,7 unit_get,
8 related_units,8 related_units,
9 relations_for_id,
9 relation_get,10 relation_get,
10)11)
11from charmhelpers.fetch import (12from charmhelpers.fetch import (
@@ -299,3 +300,27 @@
299 ctxt = super(NovaIPv6Context, self).__call__()300 ctxt = super(NovaIPv6Context, self).__call__()
300 ctxt['use_ipv6'] = config('prefer-ipv6')301 ctxt['use_ipv6'] = config('prefer-ipv6')
301 return ctxt302 return ctxt
303
304
305class InstanceConsoleContext(context.OSContextGenerator):
306 interfaces = []
307
308 def __call__(self):
309 ctxt = {}
310 servers = []
311 try:
312 for rid in relation_ids('memcache'):
313 for rel in relations_for_id(rid):
314 priv_addr = rel['private-address']
315 # format it as IPv6 address if neeeded
316 priv_addr = format_ipv6_addr(priv_addr) or priv_addr
317 servers.append({'private-address': priv_addr,
318 'port': rel['port']})
319 except Exception as ex:
320 log("Couldn't get memcache servers: {}".format(str(ex)),
321 level='WARNING')
322 servers = []
323
324 ctxt['memcached_servers'] = ','.join(
325 ["%s:%s" % (s['private-address'], s['port']) for s in servers])
326 return ctxt
302327
=== modified file 'hooks/nova_cc_hooks.py'
--- hooks/nova_cc_hooks.py 2014-10-23 07:32:45 +0000
+++ hooks/nova_cc_hooks.py 2014-12-16 14:11:00 +0000
@@ -849,6 +849,15 @@
849 quantum_joined(rid=rid)849 quantum_joined(rid=rid)
850850
851851
852@hooks.hook('memcache-relation-joined',
853 'memcache-relation-departed',
854 'memcache-relation-changed',
855 'memcache-relation-broken')
856@restart_on_change(restart_map())
857def memcached_joined():
858 CONFIGS.write(NOVA_CONF)
859
860
852def main():861def main():
853 try:862 try:
854 hooks.execute(sys.argv)863 hooks.execute(sys.argv)
855864
=== modified file 'hooks/nova_cc_utils.py'
--- hooks/nova_cc_utils.py 2014-11-14 09:55:10 +0000
+++ hooks/nova_cc_utils.py 2014-12-16 14:11:00 +0000
@@ -68,6 +68,7 @@
68 'python-psycopg2',68 'python-psycopg2',
69 'python-psutil',69 'python-psutil',
70 'uuid',70 'uuid',
71 'python-memcache',
71]72]
7273
73BASE_SERVICES = [74BASE_SERVICES = [
@@ -123,7 +124,8 @@
123 nova_cc_context.VolumeServiceContext(),124 nova_cc_context.VolumeServiceContext(),
124 nova_cc_context.NovaIPv6Context(),125 nova_cc_context.NovaIPv6Context(),
125 nova_cc_context.NeutronCCContext(),126 nova_cc_context.NeutronCCContext(),
126 nova_cc_context.NovaConfigContext()],127 nova_cc_context.NovaConfigContext(),
128 nova_cc_context.InstanceConsoleContext()],
127 }),129 }),
128 (NOVA_API_PASTE, {130 (NOVA_API_PASTE, {
129 'services': [s for s in BASE_SERVICES if 'api' in s],131 'services': [s for s in BASE_SERVICES if 'api' in s],
130132
=== modified file 'metadata.yaml'
--- metadata.yaml 2014-07-11 09:14:57 +0000
+++ metadata.yaml 2014-12-16 14:11:00 +0000
@@ -40,6 +40,8 @@
40 nova-vmware:40 nova-vmware:
41 interface: nova-vmware41 interface: nova-vmware
42 scope: container42 scope: container
43 memcache:
44 interface: memcache
43peers:45peers:
44 cluster:46 cluster:
45 interface: nova-ha47 interface: nova-ha
4648
=== modified file 'templates/folsom/nova.conf'
--- templates/folsom/nova.conf 2014-04-16 08:25:14 +0000
+++ templates/folsom/nova.conf 2014-12-16 14:11:00 +0000
@@ -21,6 +21,11 @@
21enabled_apis=ec2,osapi_compute,metadata21enabled_apis=ec2,osapi_compute,metadata
22auth_strategy=keystone22auth_strategy=keystone
23compute_driver=libvirt.LibvirtDriver23compute_driver=libvirt.LibvirtDriver
24
25{% if memcached_servers %}
26memcached_servers = {%for s in memcached_servers %}{% if loop.index0 != 0 %},{% endif %}{{s['private-address']}}:{{s['port']}}{% endfor %}
27{% endif %}
28
24{% if keystone_ec2_url -%}29{% if keystone_ec2_url -%}
25keystone_ec2_url = {{ keystone_ec2_url }}30keystone_ec2_url = {{ keystone_ec2_url }}
26{% endif -%}31{% endif -%}
2732
=== modified file 'templates/grizzly/nova.conf'
--- templates/grizzly/nova.conf 2014-03-31 11:56:09 +0000
+++ templates/grizzly/nova.conf 2014-12-16 14:11:00 +0000
@@ -20,6 +20,11 @@
20enabled_apis=ec2,osapi_compute,metadata20enabled_apis=ec2,osapi_compute,metadata
21auth_strategy=keystone21auth_strategy=keystone
22compute_driver=libvirt.LibvirtDriver22compute_driver=libvirt.LibvirtDriver
23
24{% if memcached_servers %}
25memcached_servers = {{ memcached_servers }}
26{% endif %}
27
23{% if keystone_ec2_url -%}28{% if keystone_ec2_url -%}
24keystone_ec2_url = {{ keystone_ec2_url }}29keystone_ec2_url = {{ keystone_ec2_url }}
25{% endif -%}30{% endif -%}
2631
=== modified file 'templates/havana/nova.conf'
--- templates/havana/nova.conf 2014-12-03 23:31:19 +0000
+++ templates/havana/nova.conf 2014-12-16 14:11:00 +0000
@@ -27,6 +27,10 @@
27use_syslog={{ use_syslog }}27use_syslog={{ use_syslog }}
28my_ip = {{ host_ip }}28my_ip = {{ host_ip }}
2929
30{% if memcached_servers %}
31memcached_servers = {{ memcached_servers }}
32{% endif %}
33
30{% if keystone_ec2_url -%}34{% if keystone_ec2_url -%}
31keystone_ec2_url = {{ keystone_ec2_url }}35keystone_ec2_url = {{ keystone_ec2_url }}
32{% endif -%}36{% endif -%}
3337
=== modified file 'templates/icehouse/nova.conf'
--- templates/icehouse/nova.conf 2014-12-03 23:31:19 +0000
+++ templates/icehouse/nova.conf 2014-12-16 14:11:00 +0000
@@ -38,6 +38,10 @@
38use_syslog={{ use_syslog }}38use_syslog={{ use_syslog }}
39my_ip = {{ host_ip }}39my_ip = {{ host_ip }}
4040
41{% if memcached_servers %}
42memcached_servers = {{ memcached_servers }}
43{% endif %}
44
41{% if keystone_ec2_url -%}45{% if keystone_ec2_url -%}
42keystone_ec2_url = {{ keystone_ec2_url }}46keystone_ec2_url = {{ keystone_ec2_url }}
43{% endif -%}47{% endif -%}
4448
=== added file 'unit_tests/test_nova_cc_contexts.py'
--- unit_tests/test_nova_cc_contexts.py 1970-01-01 00:00:00 +0000
+++ unit_tests/test_nova_cc_contexts.py 2014-12-16 14:11:00 +0000
@@ -0,0 +1,87 @@
1from __future__ import print_function
2
3import mock
4
5#####
6# NOTE(freyes): this is a workaround to patch config() function imported by
7# nova_cc_utils before it gets a reference to the actual config() provided by
8# hookenv module.
9from charmhelpers.core import hookenv
10_conf = hookenv.config
11hookenv.config = mock.MagicMock()
12import nova_cc_utils as _utils
13# this assert is a double check + to avoid pep8 warning
14assert _utils.config == hookenv.config
15hookenv.config = _conf
16#####
17
18import nova_cc_context as context
19
20from charmhelpers.contrib.openstack import utils
21
22from test_utils import CharmTestCase
23
24
25TO_PATCH = [
26 'apt_install',
27 'filter_installed_packages',
28 'relation_ids',
29 'relation_get',
30 'related_units',
31 'config',
32 'log',
33 'unit_get',
34 'relations_for_id',
35]
36
37
38def fake_log(msg, level=None):
39 level = level or 'INFO'
40 print('[juju test log (%s)] %s' % (level, msg))
41
42
43class NovaComputeContextTests(CharmTestCase):
44 def setUp(self):
45 super(NovaComputeContextTests, self).setUp(context, TO_PATCH)
46 self.relation_get.side_effect = self.test_relation.get
47 self.config.side_effect = self.test_config.get
48 self.log.side_effect = fake_log
49
50 @mock.patch.object(utils, 'os_release')
51 @mock.patch('charmhelpers.contrib.network.ip.log')
52 def test_instance_console_context_without_memcache(self, os_release, log_):
53 self.unit_get.return_value = '127.0.0.1'
54 self.relation_ids.return_value = 'cache:0'
55 self.related_units.return_value = 'memcached/0'
56 instance_console = context.InstanceConsoleContext()
57 os_release.return_value = 'icehouse'
58 self.assertEqual({'memcached_servers': ''},
59 instance_console())
60
61 @mock.patch.object(utils, 'os_release')
62 @mock.patch('charmhelpers.contrib.network.ip.log')
63 def test_instance_console_context_with_memcache(self, os_release, log_):
64 self.check_instance_console_context_with_memcache(os_release,
65 '127.0.1.1',
66 '127.0.1.1')
67
68 @mock.patch.object(utils, 'os_release')
69 @mock.patch('charmhelpers.contrib.network.ip.log')
70 def test_instance_console_context_with_memcache_ipv6(self, os_release,
71 log_):
72 self.check_instance_console_context_with_memcache(os_release, '::1',
73 '[::1]')
74
75 def check_instance_console_context_with_memcache(self, os_release, ip,
76 formated_ip):
77 memcached_servers = [{'private-address': formated_ip,
78 'port': '11211'}]
79 self.unit_get.return_value = ip
80 self.relation_ids.return_value = ['cache:0']
81 self.relations_for_id.return_value = memcached_servers
82 self.related_units.return_value = 'memcached/0'
83 instance_console = context.InstanceConsoleContext()
84 os_release.return_value = 'icehouse'
85 self.maxDiff = None
86 self.assertEqual({'memcached_servers': "%s:11211" % (formated_ip, )},
87 instance_console())
088
=== modified file 'unit_tests/test_nova_cc_hooks.py'
--- unit_tests/test_nova_cc_hooks.py 2014-10-21 13:18:47 +0000
+++ unit_tests/test_nova_cc_hooks.py 2014-12-16 14:11:00 +0000
@@ -1,6 +1,7 @@
1from mock import MagicMock, patch, call1from mock import MagicMock, patch, call
2from test_utils import CharmTestCase, patch_open2from test_utils import CharmTestCase, patch_open
3import os3import os
4
4with patch('charmhelpers.core.hookenv.config') as config:5with patch('charmhelpers.core.hookenv.config') as config:
5 config.return_value = 'neutron'6 config.return_value = 'neutron'
6 import nova_cc_utils as utils7 import nova_cc_utils as utils

Subscribers

People subscribed via source and target branches