~raharper/cloud-init:fix/threaded-reporter

Last commit made on 2018-08-31
Get this branch:
git clone -b fix/threaded-reporter https://git.launchpad.net/~raharper/cloud-init
Only Ryan Harper can upload to this branch. If you are Ryan Harper please log in for upload directions.

Branch merges

Branch information

Name:
fix/threaded-reporter
Repository:
lp:~raharper/cloud-init

Recent commits

9d9cc39... by Ryan Harper

Add a reporting.flush_events() call to allow main process to synchronize with daemon thread

2bc5b4b... by Ryan Harper

hyperv_reporting_handler: simplify threaded publisher

Switch the implementation to a daemon thread which uses a
blocking get from the Queue. No additional locking or flag checking
is needed since the Queue itself handles acquiring the lock as needed.
cloud-init only has a single producer (the main thread calling publish)
and the consumer will read all events in the queue and write them out.

Using the daemon mode of the thread handles flushing the queue on
main exit in python3; in python2.7 we handle the EOFError that results
when the publish thread calls to get() fails indicating the main thread
has exited.

The result is that the handler is no longer spawing a thread on each
publish event but rather creates a single thread when we start up
the reporter and we remove any additional use of separate locks and
flags as we only have a single Queue object and we're only calling
queue.put() from main thread and queue.get() from consuming thread.

9c35f97... by Pengpeng Sun

VMWare: Fix a network config bug in vm with static IPv4 and no gateway.

The issue is when customize a VM with static IPv4 and without gateway, it
will still extend route list and will loop a gateways list which is None.
This fix is to make sure when no gateway is here, it will not extend route
list.

LP: #1766538

2320c3d... by Andy

logging: Add logging config type hyperv for reporting via Azure KVP

Linux guests can provide information to Hyper-V hosts via KVP.
KVP allows the guests to provide any string key-value-pairs back to the
host's registry. On linux, kvp communication pools are presented as pool
files in /var/lib/hyperv/.kvp_pool_#.

The following reporting configuration can enable this kvp reporting in
addition to default logging if the pool files exist:

reporting:
    logging:
        type: log
    telemetry:
        type: hyperv

dab5908... by Joshua Powers

tests: disable other snap test as well

The snap test requires access to a proxy and currently the integration
tests do not handle this scenario. I am disabling the test until I can
loop back around and fix this.

a03ed8b... by Joshua Powers

tests: disable snap, fix write_files binary

The snap test requires access to a proxy and currently the integration
tests do not handle this scenario. I am disabling the test untill I can
loop back around and fix this.

The write_files test, specifically, the binary test is failing on cosmic
because the "binary" file we were writting was not a complete elf
executable, but we expected 'file' to identify it as such.

The change here is to simply use some 24 bytes of random, non-utf data
and check that file was written correctly via expected checksum.

aaffd59... by Scott Moser

Add datasource Oracle Compute Infrastructure (OCI).

This adds a Oracle specific datasource that functions with OCI.
It is a simplified version of the OpenStack metadata server
with support for vendor-data.

It does not support the OCI-C (classic) platform.

Also here is a move of BrokenMetadata to common 'sources'
as this was the third occurrence of that class.

47548df... by Chad Smith

azure: allow azure to generate network configuration from IMDS per boot.

Azure datasource now queries IMDS metadata service for network
configuration at link local address
http://169.254.169.254/metadata/instance?api-version=2017-12-01. The
azure metadata service presents a list of macs and allocated ip addresses
associated with this instance. Azure will now also regenerate network
configuration on every boot because it subscribes to EventType.BOOT
maintenance events as well as the 'first boot'
EventType.BOOT_NEW_INSTANCE.

For testing add azure-imds --kind to cloud-init devel net_convert tool
for debugging IMDS metadata.

Also refactor _get_data into 3 discrete methods:
  - is_platform_viable: check quickly whether the datasource is
    potentially compatible with the platform on which is is running
  - crawl_metadata: walk all potential metadata candidates, returning a
    structured dict of all metadata and userdata. Raise InvalidMetaData on
    error.
  - _get_data: call crawl_metadata and process results or error. Cache
    instance data on class attributes: metadata, userdata_raw etc.

51f49dc... by Louis Bouchard <email address hidden>

Scaleway: Add network configuration to the DataSource

DEP_NETWORK is removed since the network_config must
run at each boot. New EventType.BOOT event is used
for that.

Network is brought up early to fetch the metadata which
is required to configure the network (ipv4 and/or v6).

Adds unittests for the following and fixes test_common for
LOCAL and NETWORK sets.

f624927... by Wesley Gao

docs: Fix example cloud-init analyze command to match output.

Fix a typo in in documentation that showed 'analyze blame' but
intended to show 'analyze dump'.