Comment 2 for bug 1557223

Revision history for this message
Andres Toomsalu (andres-active) wrote :

Im having similar problem. Deployed Mitaka on Trusty with MaaS 1.94 and Juju 1.25.6.
In the log for nova-api-metadata service when VM boots and requests instance metadata:
--- LOG ---
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler [req-28a7fa71-aa04-4e9e-99c1-6b251a396174 - - - - -] Failed to get metadata for instance id: 7bd7798b-9491-47b4-a972-7d979a26dd20
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler Traceback (most recent call last):
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/nova/api/metadata/handler.py", line 311, in _get_meta_by_instance_id
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler remote_address)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/nova/api/metadata/handler.py", line 106, in get_metadata_by_instance_id
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler data = base.get_metadata_by_instance_id(instance_id, address)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/nova/api/metadata/base.py", line 551, in get_metadata_by_instance_id
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler return InstanceMetadata(instance, address)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/nova/api/metadata/base.py", line 134, in __init__
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler ctxt, instance)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/nova/network/security_group/neutron_driver.py", line 417, in get_instance_security_groups
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler context, servers, detailed)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/nova/network/security_group/neutron_driver.py", line 381, in get_instances_security_groups_bindings
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler ports = self._get_ports_from_server_list(servers, neutron)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/nova/network/security_group/neutron_driver.py", line 340, in _get_ports_from_server_list
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler ports.extend(neutron.list_ports(**search_opts).get('ports'))
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 97, in with_params
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler ret = self.function(instance, *args, **kwargs)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 644, in list_ports
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler **_params)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 373, in list
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler for r in self._pagination(collection, path, **params):
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 388, in _pagination
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler res = self.get(path, params=params)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 358, in get
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler headers=headers, params=params)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 335, in retry_request
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler headers=headers, params=params)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 286, in do_request
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler resp, replybody = self.httpclient.do_request(action, method, body=body)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/neutronclient/client.py", line 305, in do_request
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler self._check_uri_length(url)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/neutronclient/client.py", line 298, in _check_uri_length
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler uri_len = len(self.endpoint_url) + len(url)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/neutronclient/client.py", line 312, in endpoint_url
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler return self.get_endpoint()
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 132, in get_endpoint
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler return self.session.get_endpoint(auth or self.auth, **kwargs)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 661, in get_endpoint
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler return auth.get_endpoint(self, **kwargs)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/base.py", line 210, in get_endpoint
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler service_catalog = self.get_access(session).service_catalog
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/base.py", line 136, in get_access
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler self.auth_ref = self.get_auth_ref(session)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/generic/base.py", line 179, in get_auth_ref
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler self._plugin = self._do_create_plugin(session)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/generic/base.py", line 118, in _do_create_plugin
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler authenticated=False)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/positional/__init__.py", line 94, in inner
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler return func(*args, **kwargs)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/base.py", line 308, in get_discovery
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler authenticated=authenticated)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/positional/__init__.py", line 94, in inner
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler return func(*args, **kwargs)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/keystoneauth1/discover.py", line 141, in __init__
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler authenticated=authenticated)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/positional/__init__.py", line 94, in inner
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler return func(*args, **kwargs)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/keystoneauth1/discover.py", line 40, in get_version_data
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler resp = session.get(url, headers=headers, authenticated=authenticated)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 564, in get
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler return self.request(url, 'GET', **kwargs)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/positional/__init__.py", line 94, in inner
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler return func(*args, **kwargs)
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 394, in request
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler raise exceptions.EndpointNotFound()
2016-08-31 22:59:42.347 24078 ERROR nova.api.metadata.handler EndpointNotFound: Could not find requested endpoint in Service Catalog.
--- LOG ---

nova-api-metadata service nova.conf has:
--- EXAMPLE ---
[neutron]
url=
auth_url=://:
auth_type=password
project_domain_name=default
user_domain_name=default
region=
project_name=
username=
password=
service_metadata_proxy=True
metadata_proxy_shared_secret=xxxxxxx...
--- EXAMPLE ---

While nova-cloud-controller nova.conf has:
--- EXAMPLE ---
[neutron]
url = http://<ip>:9696
auth_strategy = keystone
auth_section = keystone_authtoken
auth_plugin = password
[keystone_authtoken]
auth_uri = http://<ip>:5000
auth_url = http://<ip>:35357
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = services
username = nova
password = <pass>
signing_dir = /var/cache/nova
--- EXAMPLE ---

Merging this back to nova-api-metadata nova.conf seems to cure the problem.