Merge lp:~springfield-team/charms/trusty/nova-cloud-controller/next into lp:~openstack-charmers-archive/charms/trusty/nova-cloud-controller/next

[Jorge Niedbalski]

This merge proposal is prepared to enable N1KV plugin to install with charm logic for nova-cloud-controller.
Similar to ovs and nvp, n1kv (Nexus 1000V) is another layer 2 virtual switch provided by Cisco Nexus 1000V engineering team, which is a host side VM switching fabric building block we call it VEM (Virtual Ethernet Switch). 1+ VEM work in tandem with a VSM (Virtual Switch Module) to form a virtual chassis in a data center.
Please also refer to the VEM/VSM charm and Cisco Nexus1000V related document for its detail.

config.yaml
- added n1kv as a possible value for quantum-plugin
- added n1kv-vsm-username, n1kv-vsm-password, n1kv-vsm-ip to communicate with its supervisor module with (VSM)

hooks/charmhelpers/contrib/openstack/context.py
- Added n1kv related dictionary for context look-up

hooks/charmhelpers/contrib/openstack/neutron.py
- Added n1kv related plugin logic data formation

hooks/nova_cc_hooks.py
- Added copy logic to copy Cisco n1kv template to /root

templates/havana/cisco_plugins.ini
- New file for Cisco Plugin related parameters

templates/havana/neutron.conf
- Added customized system parameters for N1KV plugin to run with

templates/havana/cisco_plugin.ini
- Adding support for PostgreSQL for cisco_plugin.ini

Please refer to old merge proposal if needed:
https://code.launchpad.net/~springfield-team/charms/precise/nova-cloud-controller/trunk/+merge/218172

[Shiv Prasad Rao (shivrao)]

I have submitted changes required for charm-helpers as part of another commit to lp:charm-helpers
https://code.launchpad.net/~springfield-team/charm-helpers/trunk/+merge/234402

[Hua Zhang (zhhuabj)]

some inline comments, thanks.

review: Needs Fixing

[Shiv Prasad Rao (shivrao)]

Accepted the review comments.
Synced with latest revision

[Marga Millet (millet)]

Shiv, there's a typo in the description of the n1kv-restrict-policy-profiles parameter.

Thanks.

[Hua Zhang (zhhuabj)]

looks good for me, thanks for addressing the comments.

review: Approve

[James Page (james-page)]

review: Needs Fixing

[James Page (james-page)]

Please set back to 'Needs review' once updated.

Thanks!

[Shiv Prasad Rao (shivrao)]

Addressed review comments.

1.
I will remove all changes made to render the openrc file. These changes were made to generate the openrc file so that vxlan gateway charm could use it to upload the iso to glance.
As this approach was not acceptable during the vxlan-gateway charm review, we will do with manually uploading the iso to glance (for now atleast).

2. Trimmed down the comments in the cisco_plugins.ini file.

review: Needs Resubmitting

[Ryan Beisner (1chb1n)]

UOSCI bot says:
This MP triggered a test on the Ubuntu OSCI system. Here is a summary of results.

#462 nova-cloud-controller-next for shivrao mp235182
charm_unit_test

This build:
http://10.98.191.181:8080/job/charm_unit_test/462/

MP URL:
https://code.launchpad.net/~springfield-team/charms/trusty/nova-cloud-controller/next/+merge/235182

Proposed branch:
lp:~springfield-team/charms/trusty/nova-cloud-controller/next

Results summary:
    UNIT OK: believed to pass, but you should confirm results

UNIT Results (max last 25 lines) from
/var/lib/jenkins/workspace/charm_unit_test/unit-test.462:
Ensure services with incomplete interfaces are stopped ... ok
Ensure services with complete interfaces are not stopped ... ok
Ensure that if disabled, service guards nothing ... ok
test_ssh_authorized_key_doesnt_exist (unit_tests.test_nova_cc_utils.NovaCCUtilsTests) ... ok
test_ssh_authorized_key_exists (unit_tests.test_nova_cc_utils.NovaCCUtilsTests) ... ok
test_ssh_compute_remove (unit_tests.test_nova_cc_utils.NovaCCUtilsTests) ... ok
test_ssh_directory_for_unit (unit_tests.test_nova_cc_utils.NovaCCUtilsTests) ... ok
test_ssh_known_host_key (unit_tests.test_nova_cc_utils.NovaCCUtilsTests) ... ok
Simulate a call to do_openstack_upgrade() for grizzly->icehouse ... ok
Verify get_step_upgrade_source() for grizzly->icehouse ... ok
Simulate a call to do_openstack_upgrade() for havana->icehouse ... ok
Simulate a call to do_openstack_upgrade() for havana->icehouse api ... ok
Verify get_step_upgrade_source() for havana->icehouse ... ok

Name Stmts Miss Cover Missing
-----------------------------------------------------
hooks/nova_cc_context 139 100 28% 21-28, 39-41, 47-61, 68-86, 98-151, 162-177, 185-186, 190, 194-196, 201, 204-218, 224-238, 255-258, 263-265
hooks/nova_cc_hooks 424 151 64% 131-134, 147-148, 172, 182-190, 221, 226-231, 242, 319-322, 330-333, 339-342, 352-368, 377-379, 389-403, 407-416, 502, 512, 516-517, 570, 576-586, 591-602, 612-622, 627-666, 676-691, 699-703, 716, 726-735, 759, 764-772, 820-823
hooks/nova_cc_utils 442 111 75% 292-297, 308-311, 321-322, 378, 380, 424-426, 430, 444-452, 459-464, 468-482, 588-590, 595-598, 603, 607, 631-632, 646-648, 669-670, 676-699, 703-709, 713-719, 725, 731, 738, 749-753, 838, 898-904, 908-910, 914-917, 921-933
-----------------------------------------------------
TOTAL 1005 362 64%
----------------------------------------------------------------------
Ran 90 tests in 10.250s

OK

Ubuntu OSCI Jenkins is currently in development on a Canonical private network, but we plan to publish results to a public instance soon. Tests are triggered if the proposed branch rev changes, or if the MP is placed into "Needs review" status after being otherwise for >= 1hr. Human review of results is still recommended.
http://10.98.191.181:8080/

[Ryan Beisner (1chb1n)]

UOSCI bot says:
This MP triggered a test on the Ubuntu OSCI system. Here is a summary of results.

#656 nova-cloud-controller-next for shivrao mp235182
charm_lint_check

This build:
http://10.98.191.181:8080/job/charm_lint_check/656/

MP URL:
https://code.launchpad.net/~springfield-team/charms/trusty/nova-cloud-controller/next/+merge/235182

Proposed branch:
lp:~springfield-team/charms/trusty/nova-cloud-controller/next

Results summary:
    LINT OK: believed to pass, but you should confirm results

LINT Results (max last 25 lines) from
/var/lib/jenkins/workspace/charm_lint_check/make-lint.656:
I: relation cloud-controller has no hooks
I: config.yaml: option ssl_key has no default value
I: config.yaml: option nvp-tz-uuid has no default value
I: config.yaml: option vip has no default value
I: config.yaml: option nvp-controllers has no default value
I: config.yaml: option console-access-protocol has no default value
I: config.yaml: option ssl_ca has no default value
I: config.yaml: option n1kv-config-flags has no default value
I: config.yaml: option os-public-network has no default value
W: config.yaml: option n1kv-vsm-ip does not have the keys: default
I: config.yaml: option haproxy-server-timeout has no default value
I: config.yaml: option os-admin-network has no default value
I: config.yaml: option haproxy-client-timeout has no default value
I: config.yaml: option ssl_cert has no default value
I: config.yaml: option nvp-l3-uuid has no default value
I: config.yaml: option os-internal-network has no default value

Ubuntu OSCI Jenkins is currently in development on a Canonical private network, but we plan to publish results to a public instance soon. Tests are triggered if the proposed branch rev changes, or if the MP is placed into "Needs review" status after being otherwise for >= 1hr. Human review of results is still recommended.
http://10.98.191.181:8080/

[Jorge Niedbalski (niedbalski)]

review: Approve

[Ryan Beisner (1chb1n)]

UOSCI bot says:
charm_lint_check #923 nova-cloud-controller-next for niedbalski mp240720
    LINT OK: passed

LINT Results (max last 5 lines):
  I: config.yaml: option os-admin-network has no default value
  I: config.yaml: option haproxy-client-timeout has no default value
  I: config.yaml: option ssl_cert has no default value
  I: config.yaml: option nvp-l3-uuid has no default value
  I: config.yaml: option os-internal-network has no default value

Full lint test output: http://paste.ubuntu.com/8835784/
Build: http://10.98.191.181:8080/job/charm_lint_check/923/

[Ryan Beisner (1chb1n)]

UOSCI bot says:
charm_unit_test #758 nova-cloud-controller-next for niedbalski mp240720
    UNIT OK: passed

UNIT Results (max last 5 lines):
  hooks/nova_cc_hooks 424 151 64% 131-134, 147-148, 172, 182-190, 221, 226-231, 242, 319-322, 330-333, 339-342, 352-368, 377-379, 389-403, 407-416, 502, 512, 516-517, 570, 576-586, 591-602, 612-622, 627-666, 676-691, 699-703, 716, 726-735, 759, 764-772, 820-823
  hooks/nova_cc_utils 442 111 75% 292-297, 308-311, 321-322, 378, 380, 424-426, 430, 444-452, 459-464, 468-482, 588-590, 595-598, 603, 607, 631-632, 646-648, 669-670, 676-699, 703-709, 713-719, 725, 731, 738, 749-753, 838, 898-904, 908-910, 914-917, 921-933
  TOTAL 1005 362 64%
  Ran 90 tests in 8.235s
  OK

Full unit test output: http://paste.ubuntu.com/8835786/
Build: http://10.98.191.181:8080/job/charm_unit_test/758/

[Ryan Beisner (1chb1n)]

UOSCI bot says:
charm_amulet_test #341 nova-cloud-controller-next for niedbalski mp240720
    AMULET OK: passed

AMULET Results (max last 5 lines):
  juju-test.conductor DEBUG : Calling "juju destroy-environment -y osci-sv07"
  WARNING cannot delete security group "juju-osci-sv07-0". Used by another environment?
  WARNING cannot delete security group "juju-osci-sv07". Used by another environment?
  WARNING cannot delete security group "juju-osci-sv07-0". Used by another environment?
  juju-test INFO : Results: 3 passed, 0 failed, 0 errored

Full amulet test output: http://paste.ubuntu.com/8836007/
Build: http://10.98.191.181:8080/job/charm_amulet_test/341/

[Edward Hope-Morley (hopem)]

Lost mostly good. Few inline comments.

review: Needs Fixing

[lp:~springfield-team/charms/trusty/nova-cloud-controller/next]

108. By Shiv Prasad Rao on 2014-11-05

Addressing review comments

1. Added delimiter for cisco options in config.yaml
2. Removed cisco_plugins.ini for havana. Added conditional block in icehouse template

[Shiv Prasad Rao (shivrao)]

Addressed Review comments.
Added delimiter in config.yaml

Added code block for != havana as we dont support releases before that. Anyways, the config should not cause the plugin to error out.

Thanks for the review.

review: Needs Resubmitting

[Ryan Beisner (1chb1n)]

UOSCI bot says:
charm_lint_check #929 nova-cloud-controller-next for niedbalski mp240720
    LINT OK: passed

LINT Results (max last 5 lines):
  I: config.yaml: option os-admin-network has no default value
  I: config.yaml: option haproxy-client-timeout has no default value
  I: config.yaml: option ssl_cert has no default value
  I: config.yaml: option nvp-l3-uuid has no default value
  I: config.yaml: option os-internal-network has no default value

Full lint test output: http://paste.ubuntu.com/8841281/
Build: http://10.98.191.181:8080/job/charm_lint_check/929/

[Ryan Beisner (1chb1n)]

UOSCI bot says:
charm_unit_test #764 nova-cloud-controller-next for niedbalski mp240720
    UNIT OK: passed

UNIT Results (max last 5 lines):
  hooks/nova_cc_hooks 424 151 64% 131-134, 147-148, 172, 182-190, 221, 226-231, 242, 319-322, 330-333, 339-342, 352-368, 377-379, 389-403, 407-416, 502, 512, 516-517, 570, 576-586, 591-602, 612-622, 627-666, 676-691, 699-703, 716, 726-735, 759, 764-772, 820-823
  hooks/nova_cc_utils 442 111 75% 292-297, 308-311, 321-322, 378, 380, 424-426, 430, 444-452, 459-464, 468-482, 588-590, 595-598, 603, 607, 631-632, 646-648, 669-670, 676-699, 703-709, 713-719, 725, 731, 738, 749-753, 838, 898-904, 908-910, 914-917, 921-933
  TOTAL 1005 362 64%
  Ran 90 tests in 8.767s
  OK

Full unit test output: http://paste.ubuntu.com/8841284/
Build: http://10.98.191.181:8080/job/charm_unit_test/764/

[Ryan Beisner (1chb1n)]

UOSCI bot says:
charm_amulet_test #347 nova-cloud-controller-next for niedbalski mp240720
    AMULET OK: passed

AMULET Results (max last 5 lines):
  juju-test.conductor DEBUG : Calling "juju destroy-environment -y osci-sv07"
  WARNING cannot delete security group "juju-osci-sv07-0". Used by another environment?
  WARNING cannot delete security group "juju-osci-sv07". Used by another environment?
  WARNING cannot delete security group "juju-osci-sv07-0". Used by another environment?
  juju-test INFO : Results: 3 passed, 0 failed, 0 errored

Full amulet test output: http://paste.ubuntu.com/8841503/
Build: http://10.98.191.181:8080/job/charm_amulet_test/347/

[Edward Hope-Morley (hopem)]

Ok first an apology. I was incorrect to say that we could meld the cisco_plugins.ini files for havana and Icehouse since if you run Icehouse the template engine will not look further than templates/havana so your last commit was not required (and would break havana n1kv). Now, upon further inspection and discussion there are a few other issues here:

1. neutron-server support in the nova-cloud-controller charm is being deprecated and has been shifted to the neutron-api charm (lp:~openstack-charmers/charms/trusty/neutron-api/next). So we cannot accept new neutron features/functionality for this charm (neutron-api charm already has some features that do not exist here e.g. neutron l2 population support).

2. Also, this patch is adding support for Havana and Icehouse but Havana is no longer supported in the charms (same as earlier versions) - see https://wiki.ubuntu.com/ServerTeam/CloudArchive for more info on what releases are supported. So this patch will need to be constrained to Icehouse and Juno.

So apologies for not pointing this out earlier but I'll have to reject this patch and request it be moved to neutron-api. It would also be nice to have a test deployer (e.g. added to https://code.launchpad.net/~openstack-charm-testers) to give me some idea of how you propose to deploy n1kv so I can try it out (without cisco hardware it should still be possible to deploy this and check that config files have the correct settings etc).

review: Needs Fixing

[Edward Hope-Morley (hopem)]

I have migrated this patchset to the new neutron-api charm:

https://code.launchpad.net/~springfield-team/charms/trusty/neutron-api/n1kv/+merge/242158

I will deprecate this MP but keep for posterity until we merge the new patchset.

Unmerged revisions

108. By Shiv Prasad Rao on 2014-11-05

Addressing review comments

1. Added delimiter for cisco options in config.yaml
2. Removed cisco_plugins.ini for havana. Added conditional block in icehouse template

107. By Shiv Prasad Rao on 2014-10-09

typo

106. By Shiv Prasad Rao on 2014-10-09

Sync with revision 112

105. By Shiv Prasad Rao on 2014-10-09

Addressed Review comments:
1.Removed openrc generation
2.Trimmed down comments from cisco_plugins.ini

104. By Shiv Prasad Rao on 2014-09-18

Fixing a typo

103. By Shiv Prasad Rao on 2014-09-17

Upstream 102

102. By Shiv Prasad Rao on 2014-09-17

Addressing review comments

101. By Shiv Prasad Rao on 2014-09-15

Adding support for Cisco N1kV

=== modified file 'config.yaml'
--- config.yaml	2014-10-22 21:09:02 +0000
+++ config.yaml	2014-11-05 20:35:35 +0000
@@ -292,3 +292,27 @@
       order for this charm to function correctly, the privacy extension must be
       disabled and a non-temporary address must be configured/available on
       your network interface.
+# Cisco N1KV options start  
+  n1kv-vsm-username:
+    type: string
+    default: admin
+    description: Username to connect to Cisco Nexus1000v VSM
+  n1kv-vsm-password:
+    type: string
+    default: abc123
+    description: Password to connect to Cisco Nexus1000v VSM
+  n1kv-vsm-ip:
+    type: string
+    description: IP address to connect to Cisco Nexus1000v VSM
+  n1kv-restrict-policy-profiles:
+    type: boolean
+    default: false
+    description: |
+      All tenants will be able to access all policy profiles if false.
+      If set to true, tenants can access only those policy profiles which are
+      explicitly assigned to them by admin.
+  n1kv-config-flags:
+    default:
+    type: string
+    description: Comma separated list of key=value config flags to be set in cisco_plugins.ini
+# Cisco N1KV options end

=== modified file 'hooks/charmhelpers/contrib/openstack/context.py'
--- hooks/charmhelpers/contrib/openstack/context.py	2014-10-13 16:18:58 +0000
+++ hooks/charmhelpers/contrib/openstack/context.py	2014-11-05 20:35:35 +0000
@@ -700,6 +700,7 @@
                                           self.network_manager)
         n1kv_config = neutron_plugin_attribute(self.plugin, 'config',
                                                self.network_manager)
+        n1kv_user_config_flags = config('n1kv-config-flags')
         n1kv_ctxt = {
             'core_plugin': driver,
             'neutron_plugin': 'n1kv',
@@ -710,9 +711,12 @@
             'vsm_username': config('n1kv-vsm-username'),
             'vsm_password': config('n1kv-vsm-password'),
             'restrict_policy_profiles': config(
-                'n1kv_restrict_policy_profiles'),
+                'n1kv-restrict-policy-profiles'),
         }
-
+        if n1kv_user_config_flags:
+            flags = config_flags_parser(n1kv_user_config_flags)
+            n1kv_ctxt['n1kv_user_config_flags'] = flags
+        
         return n1kv_ctxt
 
     def neutron_ctxt(self):

=== modified file 'templates/havana/neutron.conf'
--- templates/havana/neutron.conf	2014-04-16 08:25:14 +0000
+++ templates/havana/neutron.conf	2014-11-05 20:35:35 +0000
@@ -27,6 +27,11 @@
 
 {% include "parts/rabbitmq" %}
 
+{% if neutron_plugin and neutron_plugin == 'n1kv' -%}
+api_extensions_path = /usr/share/pyshared/neutron/plugins/cisco/extensions
+network_auto_schedule = False
+{% endif %}
+
 [quotas]
 quota_driver = neutron.db.quota_db.DbQuotaDriver
 {% if neutron_security_groups -%}

=== added file 'templates/icehouse/cisco_plugins.ini'
--- templates/icehouse/cisco_plugins.ini	1970-01-01 00:00:00 +0000
+++ templates/icehouse/cisco_plugins.ini	2014-11-05 20:35:35 +0000
@@ -0,0 +1,43 @@
+###############################################################################
+# [ WARNING ]
+# Configuration file maintained by Juju. Local changes may be overwritten.
+###############################################################################
+[cisco_plugins]
+ 
+[cisco]
+ 
+[cisco_n1k]
+integration_bridge = br-int
+default_policy_profile = default-pp
+network_node_policy_profile = default-pp
+{% if openstack_release != 'havana' -%}
+http_timeout = 120
+# (BoolOpt) Specify whether plugin should attempt to synchronize with the VSM
+# when neutron is started.
+# Default value: False, indicating no full sync will be performed.
+#
+enable_sync_on_start = False
+{% endif -%}
+restrict_policy_profiles = {{ restrict_policy_profiles }}
+{% if n1kv_user_config_flags -%}
+{% for key, value in n1kv_user_config_flags.iteritems() -%}
+{{ key }} = {{ value }}
+{% endfor -%}
+{% endif -%}
+
+{% include "parts/section-database" %}
+ 
+[CISCO_PLUGINS]
+vswitch_plugin = neutron.plugins.cisco.n1kv.n1kv_neutron_plugin.N1kvNeutronPluginV2
+ 
+[N1KV:{{ vsm_ip }}]
+password = {{ vsm_password }}
+username = {{ vsm_username }}
+
+[securitygroup]
+{% if neutron_security_groups -%}
+firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
+enable_security_group = True
+{% else -%}
+firewall_driver = neutron.agent.firewall.NoopFirewallDriver
+{% endif -%}

=== modified file 'templates/icehouse/neutron.conf'
--- templates/icehouse/neutron.conf	2014-09-17 08:37:24 +0000
+++ templates/icehouse/neutron.conf	2014-11-05 20:35:35 +0000
@@ -31,6 +31,10 @@
 
 {% include "parts/rabbitmq" %}
 
+{% if neutron_plugin and neutron_plugin == 'n1kv' -%}
+network_auto_schedule = False
+{% endif %}
+
 notify_nova_on_port_status_changes = True
 notify_nova_on_port_data_changes = True
 nova_url = {{ nova_url }}

=== modified file 'templates/icehouse/nova.conf'
--- templates/icehouse/nova.conf	2014-10-07 11:37:20 +0000
+++ templates/icehouse/nova.conf	2014-11-05 20:35:35 +0000
@@ -76,6 +76,15 @@
 {% endif -%}
 {% endif -%}
 
+{% if neutron_plugin and neutron_plugin == 'n1kv' -%}
+libvirt_user_virtio_for_bridges =  True
+security_group_api = {{ network_manager }}
+nova_firewall_driver = nova.virt.firewall.NoopFirewallDriver
+{% if external_network -%}
+default_floating_pool = {{ external_network }}
+{% endif -%}
+{% endif -%}
+
 {% if network_manager_config -%}
 {% for key, value in network_manager_config.iteritems() -%}
 {{ key }} = {{ value }}