Merge ~ahosmanmsft/cloud-init:azurecloudtests into cloud-init:master

Proposed by Ahmed
Status: Merged
Approved by: Chad Smith
Approved revision: 6d682c1aefccfc919c723cee537c0e056790662e
Merge reported by: Chad Smith
Merged at revision: aa3e4961ceae5a5c5b5cf13221b5f6721991fe75
Proposed branch: ~ahosmanmsft/cloud-init:azurecloudtests
Merge into: cloud-init:master
Diff against target: 911 lines (+760/-2)
15 files modified
.pylintrc (+1/-1)
doc/rtd/topics/tests.rst (+52/-0)
integration-requirements.txt (+9/-0)
tests/cloud_tests/platforms.yaml (+6/-0)
tests/cloud_tests/platforms/__init__.py (+2/-0)
tests/cloud_tests/platforms/azurecloud/__init__.py (+0/-0)
tests/cloud_tests/platforms/azurecloud/image.py (+108/-0)
tests/cloud_tests/platforms/azurecloud/instance.py (+243/-0)
tests/cloud_tests/platforms/azurecloud/platform.py (+232/-0)
tests/cloud_tests/platforms/azurecloud/regions.json (+42/-0)
tests/cloud_tests/platforms/azurecloud/snapshot.py (+58/-0)
tests/cloud_tests/platforms/ec2/image.py (+1/-0)
tests/cloud_tests/platforms/ec2/platform.py (+2/-1)
tests/cloud_tests/releases.yaml (+2/-0)
tox.ini (+2/-0)
Reviewer Review Type Date Requested Status
Server Team CI bot continuous-integration Needs Fixing
Chad Smith Approve
Review via email: mp+372957@code.launchpad.net

Commit message

cloud_tests: add azure platform support to integration tests

Added Azure to cloud tests supporting upstream integration testing.

Implement the inherited platform classes, Azure configurations
to release/platform, and docs on how to run Azure CI.

To post a comment you must log in.
Revision history for this message
Chad Smith (chad.smith) wrote :

This looks really good Ahmed!!!! Thanks for putting this together. I'm marking Needs fixing on a couple of suggestions, please feel free to comment one way or another with your thoughts and mark this review back to "Needs review" at the top.

I tried testing a bit with the following and ran into some issues

.tox/citest/bin/python -m tests.cloud_tests run --verbose --os-name bionic --preserve-data --data-dir ./data --preserve-instance --platform=azurecloud --test modules/write_file

# Suggestions for Azure platform CI additions related to https://code.launchpad.net/~ahosmanmsft/cloud-init/+git/cloud-init/+merge/372957

# 1. Can we source a credentials file instead of os.environ? It's tough to pass environment variables into tox when running tox -e cittest without checking that into our codebase.

# 2.
# Add a regions.json lookup because the mappings of Azure locations to simplestreams 'regions' is not equivalent.
# Using Azure's location['name'] is something like eastus2, but simplestreams region is either us-east-2 or "East US 2"
# Azure's location["displayName"] seems to map to simplestream's CamelCase region names. Without a proper translation to simplestreams region names we can't find any images for our img filter.

diff included here

https://pastebin.ubuntu.com/p/dDJ5k8yZc9/

review: Needs Fixing
Revision history for this message
Chad Smith (chad.smith) wrote :

Hi Ahmed, I'm setting this branch to 'Work in progress' at the top of the proposal until you get a chance to come back with a followup approach.

When it is ready for re-review, please set the branch "Status:" to "Needs review"
Thank you.

Revision history for this message
Chad Smith (chad.smith) :
Revision history for this message
Chad Smith (chad.smith) wrote :

Ahmed, thanks for continued work on this. Can you also add a short platform doc for Azure that describes the configuration steps needed in order to properly create a the service credentials needed by the Azure cloudtests in doc/rtd/topics/tests.rst.

The current example for Ec2 is https://cloudinit.readthedocs.io/en/latest/topics/tests.html#ec2

Revision history for this message
Chad Smith (chad.smith) wrote :

thanks for this, Ahmed. A couple of touch ups so far to get things functional I think then I'll take it for another spin. Please mark needs review when you've addressed the fixes.

review: Needs Fixing
Revision history for this message
Chad Smith (chad.smith) wrote :

Hi Ahmed, I'm still getting traces from your code. please make sure to fix the tracebacks you encounter when running this single integration test and set the branch back to "Needs review" once it is working order.

Here's a simple test that should succeed without tracebacks:

tox -e citest -- run --verbose --os-name xenial --test modules/write_files --data-dir results --preserve-data --platform azurecloud

I've made a suggestion for to file fix, but didn't dig into the other tracebacks

thanks again for working through these.

review: Needs Fixing
Revision history for this message
Chad Smith (chad.smith) wrote :

Thanks for the push here Ahmed, it looks like you've resolved tracebacks now and the credentials.json is driving the initial creation of nics, ips resources groups etc very well.

Two issues seem to still exist when I run
ox -e citest -- run --verbose --os-name bionic --test modules/write_files --data-dir results --preserve-data --platform azurecloud

 the images you attempt to create are not showing up in the resource group per your debug message:
DEBUG - image not found, launching instance with base image

And the ssh key imported doesn't seem to match results/cloud_init_rsa.pub because I can't ssh -i results/cloud_init_rsa ubuntu@<vm_ip> to the instance under test (it's prompting me for a password instead of accepting the login attempt with my private key)

you might be able to provide --preserve-instance to the tox command to make sure the instance isn't torn down so you can debug what cloud-init actually tried to do on that system via console access.

I'm still seeing in ability to ssh to the vms which is preventing the tests from completing. Looks like you are almost there with this branch. Please resolve the connectivity issues

Revision history for this message
Chad Smith (chad.smith) wrote :

Thanks for bumping a couple of additional fixes here. I'm still hitting tracebacks
during teardown

2019-11-01 23:01:28,275 - tests.cloud_tests - ERROR - stage: collect test data for bionic encountered error: 'StorageManagementClient' object has no attribute 'storage_manager'
2019-11-01 23:01:28,276 - tests.cloud_tests - ERROR - traceback:
  File "/home/ubuntu/cloud-init/tests/cloud_tests/stage.py", line 97, in run_stage
    (call_res, call_failed) = call()
  File "/home/ubuntu/cloud-init/tests/cloud_tests/collect.py", line 124, in collect_test_data
    collect_console(instance, test_output_dir)
  File "/home/ubuntu/cloud-init/tests/cloud_tests/collect.py", line 48, in collect_console
    data = instance.console_log()
  File "/home/ubuntu/cloud-init/tests/cloud_tests/platforms/azurecloud/instance.py", line 174, in console_log
    storage_keys = self.platform.storage_client.storage_manager\

2019-11-01 23:01:28,276 - tests.cloud_tests - DEBUG - destroying image Ubuntu_DAILY_BUILD-bionic-18_04-LTS-amd64-server-20191030-en-us-30GB

Revision history for this message
Chad Smith (chad.smith) wrote :

I *think* this is the last round from me for this iteration. Thanks a lot Ahmed. Minor inline comments as I'm awaiting a full test run on Bionic :)

review: Approve
Revision history for this message
Chad Smith (chad.smith) wrote :

A few more nits

Revision history for this message
Chad Smith (chad.smith) :
Revision history for this message
Ahmed (ahosmanmsft) wrote :

This is the expected behavior:

Create all the resources (resource group, vnet, subnet etc.). Then use streams to get the image_id, parse the image_id to launch vm with base image(meaning this image doesn't exist in the resource group). Then clean the vm and make an image from that. Now we have a clean image with the same image_id. A VM is launched to run tests with that clean image using the image_id (now that the image exists). After all the tests are run and the console log is successfully retrieved all the resources are shutdown.

Revision history for this message
Ahmed (ahosmanmsft) :
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:42978056dbe2daa5ac2b92121428f739ca34cc80
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1284/
Executed test runs:
    FAILED: Checkout

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1284//rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:42978056dbe2daa5ac2b92121428f739ca34cc80
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1285/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1285//rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:eb65ed5303861c859df41f4389d3222efc15c0f3
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1286/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1286//rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Chad Smith (chad.smith) wrote :

Hi Ahmed,

pylint still doesn't quite like your branch.
We'll need to resolve the "tox -r -e pylint" errors before we can land it.

^[[6~^[[6~^[[6~^[[6~************* Module tests.cloud_tests.platforms.azurecloud.instance
tests/cloud_tests/platforms/azurecloud/instance.py:12: [E0401(import-error), ] Unable to import 'azure.storage.blob'
tests/cloud_tests/platforms/azurecloud/instance.py:13: [E0401(import-error), ] Unable to import 'msrestazure.azure_exceptions'
tests/cloud_tests/platforms/azurecloud/instance.py:161: [W1201(logging-not-lazy), AzureCloudInstance.shutdown] Specify string format arguments as logging function parameters
************* Module tests.cloud_tests.platforms.azurecloud.platform
tests/cloud_tests/platforms/azurecloud/platform.py:11: [E0611(no-name-in-module), ] No name 'common' in module 'azure'
tests/cloud_tests/platforms/azurecloud/platform.py:11: [E0401(import-error), ] Unable to import 'azure.common.credentials'
tests/cloud_tests/platforms/azurecloud/platform.py:12: [E0611(no-name-in-module), ] No name 'mgmt' in module 'azure'
tests/cloud_tests/platforms/azurecloud/platform.py:12: [E0401(import-error), ] Unable to import 'azure.mgmt.resource'
tests/cloud_tests/platforms/azurecloud/platform.py:13: [E0611(no-name-in-module), ] No name 'mgmt' in module 'azure'
tests/cloud_tests/platforms/azurecloud/platform.py:13: [E0401(import-error), ] Unable to import 'azure.mgmt.network'
tests/cloud_tests/platforms/azurecloud/platform.py:14: [E0611(no-name-in-module), ] No name 'mgmt' in module 'azure'
tests/cloud_tests/platforms/azurecloud/platform.py:14: [E0401(import-error), ] Unable to import 'azure.mgmt.compute'
tests/cloud_tests/platforms/azurecloud/platform.py:15: [E0611(no-name-in-module), ] No name 'mgmt' in module 'azure'
tests/cloud_tests/platforms/azurecloud/platform.py:15: [E0401(import-error), ] Unable to import 'azure.mgmt.storage'
tests/cloud_tests/platforms/azurecloud/platform.py:16: [E0401(import-error), ] Unable to import 'msrestazure.azure_exceptions'
tests/cloud_tests/platforms/azurecloud/platform.py:101: [W1201(logging-not-lazy), AzureCloudPlatform.get_image] Specify string format arguments as logging function parameters
tests/cloud_tests/platforms/azurecloud/platform.py:117: [W1201(logging-not-lazy), AzureCloudPlatform.azure_location_to_simplestreams_region] Specify string format arguments as logging function parameters

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:8529cba90e1da1cdf7f1de4e2256cebd12070f34
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1288/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1288//rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:57c275a763a59aa82ffd999771ccfc0c8d5d1f09
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1289/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1289//rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Chad Smith (chad.smith) wrote :

After a bit of back and forth to make pylint happy, we have a success
https://travis-ci.org/blackboxsw/cloud-init/jobs/616949469

Will land this branch in github now thank you Ahmed

Revision history for this message
Chad Smith (chad.smith) wrote :

Waiting on CLA

Revision history for this message
Chad Smith (chad.smith) wrote :

Ahmed, confirmed with our Legal department that individual microsoft employees should sign the CLA if they can https://ubuntu.com/legal/contributors/agreement

Revision history for this message
Ahmed (ahosmanmsft) wrote :

> Ahmed, confirmed with our Legal department that individual microsoft employees
> should sign the CLA if they can
> https://ubuntu.com/legal/contributors/agreement

I just signed as CI, what are the next steps?

Revision history for this message
Chad Smith (chad.smith) wrote :

This merge has landed in commit aa3e4961 to cloud-init branch master.

To view that commit see the following URL:
https://github.com/canonical/cloud-init/commit/aa3e4961

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/.pylintrc b/.pylintrc
2index 365c8c8..c83546a 100644
3--- a/.pylintrc
4+++ b/.pylintrc
5@@ -62,7 +62,7 @@ ignored-modules=
6 # for classes with dynamically set attributes). This supports the use of
7 # qualified names.
8 # argparse.Namespace from https://github.com/PyCQA/pylint/issues/2413
9-ignored-classes=argparse.Namespace,optparse.Values,thread._local
10+ignored-classes=argparse.Namespace,optparse.Values,thread._local,ImageManager,ContainerManager
11
12 # List of members which are set dynamically and missed by pylint inference
13 # system, and so shouldn't trigger E1101 when accessed. Python regular
14diff --git a/doc/rtd/topics/tests.rst b/doc/rtd/topics/tests.rst
15index a2c703a..3b27f80 100644
16--- a/doc/rtd/topics/tests.rst
17+++ b/doc/rtd/topics/tests.rst
18@@ -423,6 +423,58 @@ generated when running ``aws configure``:
19 region = us-west-2
20
21
22+Azure Cloud
23+-----------
24+
25+To run on Azure Cloud platform users login with Service Principal and export
26+credentials file. Region is defaulted and can be set in ``tests/cloud_tests/platforms.yaml``.
27+The Service Principal credentials are the standard authentication for Azure SDK
28+to interact with Azure Services:
29+
30+Create Service Principal account or login
31+
32+.. code-block:: shell-session
33+
34+ $ az ad sp create-for-rbac --name "APP_ID" --password "STRONG-SECRET-PASSWORD"
35+
36+.. code-block:: shell-session
37+
38+ $ az login --service-principal --username "APP_ID" --password "STRONG-SECRET-PASSWORD"
39+
40+Export credentials
41+
42+.. code-block:: shell-session
43+
44+ $ az ad sp create-for-rbac --sdk-auth > $HOME/.azure/credentials.json
45+
46+.. code-block:: json
47+
48+ {
49+ "clientId": "<Service principal ID>",
50+ "clientSecret": "<Service principal secret/password>",
51+ "subscriptionId": "<Subscription associated with the service principal>",
52+ "tenantId": "<The service principal's tenant>",
53+ "activeDirectoryEndpointUrl": "https://login.microsoftonline.com",
54+ "resourceManagerEndpointUrl": "https://management.azure.com/",
55+ "activeDirectoryGraphResourceId": "https://graph.windows.net/",
56+ "sqlManagementEndpointUrl": "https://management.core.windows.net:8443/",
57+ "galleryEndpointUrl": "https://gallery.azure.com/",
58+ "managementEndpointUrl": "https://management.core.windows.net/"
59+ }
60+
61+Set region in platforms.yaml
62+
63+.. code-block:: yaml
64+ :emphasize-lines: 3
65+
66+ azurecloud:
67+ enabled: true
68+ region: West US 2
69+ vm_size: Standard_DS1_v2
70+ storage_sku: standard_lrs
71+ tag: ci
72+
73+
74 Architecture
75 ============
76
77diff --git a/integration-requirements.txt b/integration-requirements.txt
78index fe5ad45..897d611 100644
79--- a/integration-requirements.txt
80+++ b/integration-requirements.txt
81@@ -20,3 +20,12 @@ git+https://github.com/lxc/pylxd.git@4b8ab1802f9aee4eb29cf7b119dae0aa47150779
82
83 # finds latest image information
84 git+https://git.launchpad.net/simplestreams
85+
86+# azure backend
87+azure-storage==0.36.0
88+msrestazure==0.6.1
89+azure-common==1.1.23
90+azure-mgmt-compute==7.0.0
91+azure-mgmt-network==5.0.0
92+azure-mgmt-resource==4.0.0
93+azure-mgmt-storage==6.0.0
94diff --git a/tests/cloud_tests/platforms.yaml b/tests/cloud_tests/platforms.yaml
95index 652a705..eaaa0a7 100644
96--- a/tests/cloud_tests/platforms.yaml
97+++ b/tests/cloud_tests/platforms.yaml
98@@ -67,5 +67,11 @@ platforms:
99 nocloud-kvm:
100 enabled: true
101 cache_mode: cache=none,aio=native
102+ azurecloud:
103+ enabled: true
104+ region: West US 2
105+ vm_size: Standard_DS1_v2
106+ storage_sku: standard_lrs
107+ tag: ci
108
109 # vi: ts=4 expandtab
110diff --git a/tests/cloud_tests/platforms/__init__.py b/tests/cloud_tests/platforms/__init__.py
111index a01e51a..6a410b8 100644
112--- a/tests/cloud_tests/platforms/__init__.py
113+++ b/tests/cloud_tests/platforms/__init__.py
114@@ -5,11 +5,13 @@
115 from .ec2 import platform as ec2
116 from .lxd import platform as lxd
117 from .nocloudkvm import platform as nocloudkvm
118+from .azurecloud import platform as azurecloud
119
120 PLATFORMS = {
121 'ec2': ec2.EC2Platform,
122 'nocloud-kvm': nocloudkvm.NoCloudKVMPlatform,
123 'lxd': lxd.LXDPlatform,
124+ 'azurecloud': azurecloud.AzureCloudPlatform,
125 }
126
127
128diff --git a/tests/cloud_tests/platforms/azurecloud/__init__.py b/tests/cloud_tests/platforms/azurecloud/__init__.py
129new file mode 100644
130index 0000000..e69de29
131--- /dev/null
132+++ b/tests/cloud_tests/platforms/azurecloud/__init__.py
133diff --git a/tests/cloud_tests/platforms/azurecloud/image.py b/tests/cloud_tests/platforms/azurecloud/image.py
134new file mode 100644
135index 0000000..96a946f
136--- /dev/null
137+++ b/tests/cloud_tests/platforms/azurecloud/image.py
138@@ -0,0 +1,108 @@
139+# This file is part of cloud-init. See LICENSE file for license information.
140+
141+"""Azure Cloud image Base class."""
142+
143+from tests.cloud_tests import LOG
144+
145+from ..images import Image
146+from .snapshot import AzureCloudSnapshot
147+
148+
149+class AzureCloudImage(Image):
150+ """Azure Cloud backed image."""
151+
152+ platform_name = 'azurecloud'
153+
154+ def __init__(self, platform, config, image_id):
155+ """Set up image.
156+
157+ @param platform: platform object
158+ @param config: image configuration
159+ @param image_id: image id used to boot instance
160+ """
161+ super(AzureCloudImage, self).__init__(platform, config)
162+ self.image_id = image_id
163+ self._img_instance = None
164+
165+ @property
166+ def _instance(self):
167+ """Internal use only, returns a running instance"""
168+ LOG.debug('creating instance')
169+ if not self._img_instance:
170+ self._img_instance = self.platform.create_instance(
171+ self.properties, self.config, self.features,
172+ self.image_id, user_data=None)
173+ return self._img_instance
174+
175+ def destroy(self):
176+ """Delete the instance used to create a custom image."""
177+ LOG.debug('deleting VM that was used to create image')
178+ if self._img_instance:
179+ LOG.debug('Deleting instance %s', self._img_instance.name)
180+ delete_vm = self.platform.compute_client.virtual_machines.delete(
181+ self.platform.resource_group.name, self.image_id)
182+ delete_vm.wait()
183+
184+ super(AzureCloudImage, self).destroy()
185+
186+ def _execute(self, *args, **kwargs):
187+ """Execute command in image, modifying image."""
188+ LOG.debug('executing commands on image')
189+ self._instance.start()
190+ return self._instance._execute(*args, **kwargs)
191+
192+ def push_file(self, local_path, remote_path):
193+ """Copy file at 'local_path' to instance at 'remote_path'."""
194+ LOG.debug('pushing file to image')
195+ return self._instance.push_file(local_path, remote_path)
196+
197+ def run_script(self, *args, **kwargs):
198+ """Run script in image, modifying image.
199+
200+ @return_value: script output
201+ """
202+ LOG.debug('running script on image')
203+ self._instance.start()
204+ return self._instance.run_script(*args, **kwargs)
205+
206+ def snapshot(self):
207+ """ Create snapshot (image) of instance, wait until done.
208+
209+ If no instance has been booted, base image is returned.
210+ Otherwise runs the clean script, deallocates, generalizes
211+ and creates custom image from instance.
212+ """
213+ LOG.debug('creating image from VM')
214+ if not self._img_instance:
215+ return AzureCloudSnapshot(self.platform, self.properties,
216+ self.config, self.features,
217+ self.image_id, delete_on_destroy=False)
218+
219+ if self.config.get('boot_clean_script'):
220+ self._img_instance.run_script(self.config.get('boot_clean_script'))
221+
222+ deallocate = self.platform.compute_client.virtual_machines.deallocate(
223+ self.platform.resource_group.name, self.image_id)
224+ deallocate.wait()
225+
226+ self.platform.compute_client.virtual_machines.generalize(
227+ self.platform.resource_group.name, self.image_id)
228+
229+ image_params = {
230+ "location": self.platform.location,
231+ "properties": {
232+ "sourceVirtualMachine": {
233+ "id": self._img_instance.instance.id
234+ }
235+ }
236+ }
237+ self.platform.compute_client.images.create_or_update(
238+ self.platform.resource_group.name, self.image_id,
239+ image_params)
240+
241+ self.destroy()
242+
243+ return AzureCloudSnapshot(self.platform, self.properties, self.config,
244+ self.features, self.image_id)
245+
246+# vi: ts=4 expandtab
247diff --git a/tests/cloud_tests/platforms/azurecloud/instance.py b/tests/cloud_tests/platforms/azurecloud/instance.py
248new file mode 100644
249index 0000000..3d77a1a
250--- /dev/null
251+++ b/tests/cloud_tests/platforms/azurecloud/instance.py
252@@ -0,0 +1,243 @@
253+# This file is part of cloud-init. See LICENSE file for license information.
254+
255+"""Base Azure Cloud instance."""
256+
257+from datetime import datetime, timedelta
258+from urllib.parse import urlparse
259+from time import sleep
260+import traceback
261+import os
262+
263+
264+# pylint: disable=no-name-in-module
265+from azure.storage.blob import BlockBlobService, BlobPermissions
266+from msrestazure.azure_exceptions import CloudError
267+
268+from tests.cloud_tests import LOG
269+
270+from ..instances import Instance
271+
272+
273+class AzureCloudInstance(Instance):
274+ """Azure Cloud backed instance."""
275+
276+ platform_name = 'azurecloud'
277+
278+ def __init__(self, platform, properties, config,
279+ features, image_id, user_data=None):
280+ """Set up instance.
281+
282+ @param platform: platform object
283+ @param properties: dictionary of properties
284+ @param config: dictionary of configuration values
285+ @param features: dictionary of supported feature flags
286+ @param image_id: image to find and/or use
287+ @param user_data: test user-data to pass to instance
288+ """
289+ super(AzureCloudInstance, self).__init__(
290+ platform, image_id, properties, config, features)
291+
292+ self.ssh_port = 22
293+ self.ssh_ip = None
294+ self.instance = None
295+ self.image_id = image_id
296+ self.user_data = user_data
297+ self.ssh_key_file = os.path.join(
298+ platform.config['data_dir'], platform.config['private_key'])
299+ self.ssh_pubkey_file = os.path.join(
300+ platform.config['data_dir'], platform.config['public_key'])
301+ self.blob_client, self.container, self.blob = None, None, None
302+
303+ def start(self, wait=True, wait_for_cloud_init=False):
304+ """Start instance with the platforms NIC."""
305+ if self.instance:
306+ return
307+ data = self.image_id.split('-')
308+ release, support = data[2].replace('_', '.'), data[3]
309+ sku = '%s-%s' % (release, support) if support == 'LTS' else release
310+ image_resource_id = '/subscriptions/%s' \
311+ '/resourceGroups/%s' \
312+ '/providers/Microsoft.Compute/images/%s' % (
313+ self.platform.subscription_id,
314+ self.platform.resource_group.name,
315+ self.image_id)
316+ storage_uri = "http://%s.blob.core.windows.net" \
317+ % self.platform.storage.name
318+ with open(self.ssh_pubkey_file, 'r') as key:
319+ ssh_pub_keydata = key.read()
320+
321+ image_exists = False
322+ try:
323+ LOG.debug('finding image in resource group using image_id')
324+ self.platform.compute_client.images.get(
325+ self.platform.resource_group.name,
326+ self.image_id
327+ )
328+ image_exists = True
329+ LOG.debug('image found, launching instance')
330+ except CloudError:
331+ LOG.debug(
332+ 'image not found, launching instance with base image')
333+ pass
334+
335+ vm_params = {
336+ 'location': self.platform.location,
337+ 'os_profile': {
338+ 'computer_name': 'CI',
339+ 'admin_username': self.ssh_username,
340+ "customData": self.user_data,
341+ "linuxConfiguration": {
342+ "disable_password_authentication": True,
343+ "ssh": {
344+ "public_keys": [{
345+ "path": "/home/%s/.ssh/authorized_keys" %
346+ self.ssh_username,
347+ "keyData": ssh_pub_keydata
348+ }]
349+ }
350+ }
351+ },
352+ "diagnosticsProfile": {
353+ "bootDiagnostics": {
354+ "storageUri": storage_uri,
355+ "enabled": True
356+ }
357+ },
358+ 'hardware_profile': {
359+ 'vm_size': self.platform.vm_size
360+ },
361+ 'storage_profile': {
362+ 'image_reference': {
363+ 'id': image_resource_id
364+ } if image_exists else {
365+ 'publisher': 'Canonical',
366+ 'offer': 'UbuntuServer',
367+ 'sku': sku,
368+ 'version': 'latest'
369+ }
370+ },
371+ 'network_profile': {
372+ 'network_interfaces': [{
373+ 'id': self.platform.nic.id
374+ }]
375+ },
376+ 'tags': {
377+ 'Name': self.platform.tag,
378+ }
379+ }
380+
381+ try:
382+ self.instance = self.platform.compute_client.virtual_machines.\
383+ create_or_update(self.platform.resource_group.name,
384+ self.image_id, vm_params)
385+ except CloudError:
386+ raise RuntimeError('failed creating instance:\n{}'.format(
387+ traceback.format_exc()))
388+
389+ if wait:
390+ self.instance.wait()
391+ self.ssh_ip = self.platform.network_client.\
392+ public_ip_addresses.get(
393+ self.platform.resource_group.name,
394+ self.platform.public_ip.name
395+ ).ip_address
396+ self._wait_for_system(wait_for_cloud_init)
397+
398+ self.instance = self.instance.result()
399+ self.blob_client, self.container, self.blob =\
400+ self._get_blob_client()
401+
402+ def shutdown(self, wait=True):
403+ """Finds console log then stopping/deallocates VM"""
404+ LOG.debug('waiting on console log before stopping')
405+ attempts, exists = 5, False
406+ while not exists and attempts:
407+ try:
408+ attempts -= 1
409+ exists = self.blob_client.get_blob_to_bytes(
410+ self.container, self.blob)
411+ LOG.debug('found console log')
412+ except Exception as e:
413+ if attempts:
414+ LOG.debug('Unable to find console log, '
415+ '%s attempts remaining', attempts)
416+ sleep(15)
417+ else:
418+ LOG.warning('Could not find console log: %s', e)
419+ pass
420+
421+ LOG.debug('stopping instance %s', self.image_id)
422+ vm_deallocate = \
423+ self.platform.compute_client.virtual_machines.deallocate(
424+ self.platform.resource_group.name, self.image_id)
425+ if wait:
426+ vm_deallocate.wait()
427+
428+ def destroy(self):
429+ """Delete VM and close all connections"""
430+ if self.instance:
431+ LOG.debug('destroying instance: %s', self.image_id)
432+ vm_delete = self.platform.compute_client.virtual_machines.delete(
433+ self.platform.resource_group.name, self.image_id)
434+ vm_delete.wait()
435+
436+ self._ssh_close()
437+
438+ super(AzureCloudInstance, self).destroy()
439+
440+ def _execute(self, command, stdin=None, env=None):
441+ """Execute command on instance."""
442+ env_args = []
443+ if env:
444+ env_args = ['env'] + ["%s=%s" for k, v in env.items()]
445+
446+ return self._ssh(['sudo'] + env_args + list(command), stdin=stdin)
447+
448+ def _get_blob_client(self):
449+ """
450+ Use VM details to retrieve container and blob name.
451+ Then Create blob service client for sas token to
452+ retrieve console log.
453+
454+ :return: blob service, container name, blob name
455+ """
456+ LOG.debug('creating blob service for console log')
457+ storage = self.platform.storage_client.storage_accounts.get_properties(
458+ self.platform.resource_group.name, self.platform.storage.name)
459+
460+ keys = self.platform.storage_client.storage_accounts.list_keys(
461+ self.platform.resource_group.name, self.platform.storage.name
462+ ).keys[0].value
463+
464+ virtual_machine = self.platform.compute_client.virtual_machines.get(
465+ self.platform.resource_group.name, self.instance.name,
466+ expand='instanceView')
467+
468+ blob_uri = virtual_machine.instance_view.boot_diagnostics.\
469+ serial_console_log_blob_uri
470+
471+ container, blob = urlparse(blob_uri).path.split('/')[-2:]
472+
473+ blob_client = BlockBlobService(
474+ account_name=storage.name,
475+ account_key=keys)
476+
477+ sas = blob_client.generate_blob_shared_access_signature(
478+ container_name=container, blob_name=blob, protocol='https',
479+ expiry=datetime.utcnow() + timedelta(hours=1),
480+ permission=BlobPermissions.READ)
481+
482+ blob_client = BlockBlobService(
483+ account_name=storage.name,
484+ sas_token=sas)
485+
486+ return blob_client, container, blob
487+
488+ def console_log(self):
489+ """Instance console.
490+
491+ @return_value: bytes of this instance’s console
492+ """
493+ boot_diagnostics = self.blob_client.get_blob_to_bytes(
494+ self.container, self.blob)
495+ return boot_diagnostics.content
496diff --git a/tests/cloud_tests/platforms/azurecloud/platform.py b/tests/cloud_tests/platforms/azurecloud/platform.py
497new file mode 100644
498index 0000000..77f159e
499--- /dev/null
500+++ b/tests/cloud_tests/platforms/azurecloud/platform.py
501@@ -0,0 +1,232 @@
502+# This file is part of cloud-init. See LICENSE file for license information.
503+
504+"""Base Azure Cloud class."""
505+
506+import os
507+import base64
508+import traceback
509+from datetime import datetime
510+from tests.cloud_tests import LOG
511+
512+# pylint: disable=no-name-in-module
513+from azure.common.credentials import ServicePrincipalCredentials
514+# pylint: disable=no-name-in-module
515+from azure.mgmt.resource import ResourceManagementClient
516+# pylint: disable=no-name-in-module
517+from azure.mgmt.network import NetworkManagementClient
518+# pylint: disable=no-name-in-module
519+from azure.mgmt.compute import ComputeManagementClient
520+# pylint: disable=no-name-in-module
521+from azure.mgmt.storage import StorageManagementClient
522+from msrestazure.azure_exceptions import CloudError
523+
524+from .image import AzureCloudImage
525+from .instance import AzureCloudInstance
526+from ..platforms import Platform
527+
528+from cloudinit import util as c_util
529+
530+
531+class AzureCloudPlatform(Platform):
532+ """Azure Cloud test platforms."""
533+
534+ platform_name = 'azurecloud'
535+
536+ def __init__(self, config):
537+ """Set up platform."""
538+ super(AzureCloudPlatform, self).__init__(config)
539+ self.tag = '%s-%s' % (
540+ config['tag'], datetime.now().strftime('%Y%m%d%H%M%S'))
541+ self.storage_sku = config['storage_sku']
542+ self.vm_size = config['vm_size']
543+ self.location = config['region']
544+
545+ try:
546+ self.credentials, self.subscription_id = self._get_credentials()
547+
548+ self.resource_client = ResourceManagementClient(
549+ self.credentials, self.subscription_id)
550+ self.compute_client = ComputeManagementClient(
551+ self.credentials, self.subscription_id)
552+ self.network_client = NetworkManagementClient(
553+ self.credentials, self.subscription_id)
554+ self.storage_client = StorageManagementClient(
555+ self.credentials, self.subscription_id)
556+
557+ self.resource_group = self._create_resource_group()
558+ self.public_ip = self._create_public_ip_address()
559+ self.storage = self._create_storage_account(config)
560+ self.vnet = self._create_vnet()
561+ self.subnet = self._create_subnet()
562+ self.nic = self._create_nic()
563+ except CloudError:
564+ raise RuntimeError('failed creating a resource:\n{}'.format(
565+ traceback.format_exc()))
566+
567+ def create_instance(self, properties, config, features,
568+ image_id, user_data=None):
569+ """Create an instance
570+
571+ @param properties: image properties
572+ @param config: image configuration
573+ @param features: image features
574+ @param image_id: string of image id
575+ @param user_data: test user-data to pass to instance
576+ @return_value: cloud_tests.instances instance
577+ """
578+ user_data = str(base64.b64encode(
579+ user_data.encode('utf-8')), 'utf-8')
580+
581+ return AzureCloudInstance(self, properties, config, features,
582+ image_id, user_data)
583+
584+ def get_image(self, img_conf):
585+ """Get image using specified image configuration.
586+
587+ @param img_conf: configuration for image
588+ @return_value: cloud_tests.images instance
589+ """
590+ ss_region = self.azure_location_to_simplestreams_region()
591+
592+ filters = [
593+ 'arch=%s' % 'amd64',
594+ 'endpoint=https://management.core.windows.net/',
595+ 'region=%s' % ss_region,
596+ 'release=%s' % img_conf['release']
597+ ]
598+
599+ LOG.debug('finding image using streams')
600+ image = self._query_streams(img_conf, filters)
601+
602+ try:
603+ image_id = image['id']
604+ LOG.debug('found image: %s', image_id)
605+ if image_id.find('__') > 0:
606+ image_id = image_id.split('__')[1]
607+ LOG.debug('image_id shortened to %s', image_id)
608+ except KeyError:
609+ raise RuntimeError('no images found for %s' % img_conf['release'])
610+
611+ return AzureCloudImage(self, img_conf, image_id)
612+
613+ def destroy(self):
614+ """Delete all resources in resource group."""
615+ LOG.debug("Deleting resource group: %s", self.resource_group.name)
616+ delete = self.resource_client.resource_groups.delete(
617+ self.resource_group.name)
618+ delete.wait()
619+
620+ def azure_location_to_simplestreams_region(self):
621+ """Convert location to simplestreams region"""
622+ location = self.location.lower().replace(' ', '')
623+ LOG.debug('finding location %s using simple streams', location)
624+ regions_file = os.path.join(
625+ os.path.dirname(os.path.abspath(__file__)), 'regions.json')
626+ region_simplestreams_map = c_util.load_json(
627+ c_util.load_file(regions_file))
628+ return region_simplestreams_map.get(location, location)
629+
630+ def _get_credentials(self):
631+ """Get credentials from environment"""
632+ LOG.debug('getting credentials from environment')
633+ cred_file = os.path.expanduser('~/.azure/credentials.json')
634+ try:
635+ azure_creds = c_util.load_json(
636+ c_util.load_file(cred_file))
637+ subscription_id = azure_creds['subscriptionId']
638+ credentials = ServicePrincipalCredentials(
639+ client_id=azure_creds['clientId'],
640+ secret=azure_creds['clientSecret'],
641+ tenant=azure_creds['tenantId'])
642+ return credentials, subscription_id
643+ except KeyError:
644+ raise RuntimeError('Please configure Azure service principal'
645+ ' credentials in %s' % cred_file)
646+
647+ def _create_resource_group(self):
648+ """Create resource group"""
649+ LOG.debug('creating resource group')
650+ resource_group_name = self.tag
651+ resource_group_params = {
652+ 'location': self.location
653+ }
654+ resource_group = self.resource_client.resource_groups.create_or_update(
655+ resource_group_name, resource_group_params)
656+ return resource_group
657+
658+ def _create_storage_account(self, config):
659+ LOG.debug('creating storage account')
660+ storage_account_name = 'storage%s' % datetime.now().\
661+ strftime('%Y%m%d%H%M%S')
662+ storage_params = {
663+ 'sku': {
664+ 'name': config['storage_sku']
665+ },
666+ 'kind': "Storage",
667+ 'location': self.location
668+ }
669+ storage_account = self.storage_client.storage_accounts.create(
670+ self.resource_group.name, storage_account_name, storage_params)
671+ return storage_account.result()
672+
673+ def _create_public_ip_address(self):
674+ """Create public ip address"""
675+ LOG.debug('creating public ip address')
676+ public_ip_name = '%s-ip' % self.resource_group.name
677+ public_ip_params = {
678+ 'location': self.location,
679+ 'public_ip_allocation_method': 'Dynamic'
680+ }
681+ ip = self.network_client.public_ip_addresses.create_or_update(
682+ self.resource_group.name, public_ip_name, public_ip_params)
683+ return ip.result()
684+
685+ def _create_vnet(self):
686+ """create virtual network"""
687+ LOG.debug('creating vnet')
688+ vnet_name = '%s-vnet' % self.resource_group.name
689+ vnet_params = {
690+ 'location': self.location,
691+ 'address_space': {
692+ 'address_prefixes': ['10.0.0.0/16']
693+ }
694+ }
695+ vnet = self.network_client.virtual_networks.create_or_update(
696+ self.resource_group.name, vnet_name, vnet_params)
697+ return vnet.result()
698+
699+ def _create_subnet(self):
700+ """create sub-network"""
701+ LOG.debug('creating subnet')
702+ subnet_name = '%s-subnet' % self.resource_group.name
703+ subnet_params = {
704+ 'address_prefix': '10.0.0.0/24'
705+ }
706+ subnet = self.network_client.subnets.create_or_update(
707+ self.resource_group.name, self.vnet.name,
708+ subnet_name, subnet_params)
709+ return subnet.result()
710+
711+ def _create_nic(self):
712+ """Create network interface controller"""
713+ LOG.debug('creating nic')
714+ nic_name = '%s-nic' % self.resource_group.name
715+ nic_params = {
716+ 'location': self.location,
717+ 'ip_configurations': [{
718+ 'name': 'ipconfig',
719+ 'subnet': {
720+ 'id': self.subnet.id
721+ },
722+ 'publicIpAddress': {
723+ 'id': "/subscriptions/%s"
724+ "/resourceGroups/%s/providers/Microsoft.Network"
725+ "/publicIPAddresses/%s" % (
726+ self.subscription_id, self.resource_group.name,
727+ self.public_ip.name),
728+ }
729+ }]
730+ }
731+ nic = self.network_client.network_interfaces.create_or_update(
732+ self.resource_group.name, nic_name, nic_params)
733+ return nic.result()
734diff --git a/tests/cloud_tests/platforms/azurecloud/regions.json b/tests/cloud_tests/platforms/azurecloud/regions.json
735new file mode 100644
736index 0000000..c1b4da2
737--- /dev/null
738+++ b/tests/cloud_tests/platforms/azurecloud/regions.json
739@@ -0,0 +1,42 @@
740+{
741+ "eastasia": "East Asia",
742+ "southeastasia": "Southeast Asia",
743+ "centralus": "Central US",
744+ "eastus": "East US",
745+ "eastus2": "East US 2",
746+ "westus": "West US",
747+ "northcentralus": "North Central US",
748+ "southcentralus": "South Central US",
749+ "northeurope": "North Europe",
750+ "westeurope": "West Europe",
751+ "japanwest": "Japan West",
752+ "japaneast": "Japan East",
753+ "brazilsouth": "Brazil South",
754+ "australiaeast": "Australia East",
755+ "australiasoutheast": "Australia Southeast",
756+ "southindia": "South India",
757+ "centralindia": "Central India",
758+ "westindia": "West India",
759+ "canadacentral": "Canada Central",
760+ "canadaeast": "Canada East",
761+ "uksouth": "UK South",
762+ "ukwest": "UK West",
763+ "westcentralus": "West Central US",
764+ "westus2": "West US 2",
765+ "koreacentral": "Korea Central",
766+ "koreasouth": "Korea South",
767+ "francecentral": "France Central",
768+ "francesouth": "France South",
769+ "australiacentral": "Australia Central",
770+ "australiacentral2": "Australia Central 2",
771+ "uaecentral": "UAE Central",
772+ "uaenorth": "UAE North",
773+ "southafricanorth": "South Africa North",
774+ "southafricawest": "South Africa West",
775+ "switzerlandnorth": "Switzerland North",
776+ "switzerlandwest": "Switzerland West",
777+ "germanynorth": "Germany North",
778+ "germanywestcentral": "Germany West Central",
779+ "norwaywest": "Norway West",
780+ "norwayeast": "Norway East"
781+}
782diff --git a/tests/cloud_tests/platforms/azurecloud/snapshot.py b/tests/cloud_tests/platforms/azurecloud/snapshot.py
783new file mode 100644
784index 0000000..580cc59
785--- /dev/null
786+++ b/tests/cloud_tests/platforms/azurecloud/snapshot.py
787@@ -0,0 +1,58 @@
788+# This file is part of cloud-init. See LICENSE file for license information.
789+
790+"""Base Azure Cloud snapshot."""
791+
792+from ..snapshots import Snapshot
793+
794+from tests.cloud_tests import LOG
795+
796+
797+class AzureCloudSnapshot(Snapshot):
798+ """Azure Cloud image copy backed snapshot."""
799+
800+ platform_name = 'azurecloud'
801+
802+ def __init__(self, platform, properties, config, features, image_id,
803+ delete_on_destroy=True):
804+ """Set up snapshot.
805+
806+ @param platform: platform object
807+ @param properties: image properties
808+ @param config: image config
809+ @param features: supported feature flags
810+ """
811+ super(AzureCloudSnapshot, self).__init__(
812+ platform, properties, config, features)
813+
814+ self.image_id = image_id
815+ self.delete_on_destroy = delete_on_destroy
816+
817+ def launch(self, user_data, meta_data=None, block=True, start=True,
818+ use_desc=None):
819+ """Launch instance.
820+
821+ @param user_data: user-data for the instance
822+ @param meta_data: meta_data for the instance
823+ @param block: wait until instance is created
824+ @param start: start instance and wait until fully started
825+ @param use_desc: description of snapshot instance use
826+ @return_value: an Instance
827+ """
828+ if meta_data is not None:
829+ raise ValueError("metadata not supported on Azure Cloud tests")
830+
831+ instance = self.platform.create_instance(
832+ self.properties, self.config, self.features,
833+ self.image_id, user_data)
834+
835+ return instance
836+
837+ def destroy(self):
838+ """Clean up snapshot data."""
839+ LOG.debug('destroying image %s', self.image_id)
840+ if self.delete_on_destroy:
841+ self.platform.compute_client.images.delete(
842+ self.platform.resource_group.name,
843+ self.image_id)
844+
845+# vi: ts=4 expandtab
846diff --git a/tests/cloud_tests/platforms/ec2/image.py b/tests/cloud_tests/platforms/ec2/image.py
847index 7bedf59..d7b2c90 100644
848--- a/tests/cloud_tests/platforms/ec2/image.py
849+++ b/tests/cloud_tests/platforms/ec2/image.py
850@@ -4,6 +4,7 @@
851
852 from ..images import Image
853 from .snapshot import EC2Snapshot
854+
855 from tests.cloud_tests import LOG
856
857
858diff --git a/tests/cloud_tests/platforms/ec2/platform.py b/tests/cloud_tests/platforms/ec2/platform.py
859index f188c27..7a3d0fe 100644
860--- a/tests/cloud_tests/platforms/ec2/platform.py
861+++ b/tests/cloud_tests/platforms/ec2/platform.py
862@@ -135,6 +135,7 @@ class EC2Platform(Platform):
863 def _create_internet_gateway(self):
864 """Create Internet Gateway and assign to VPC."""
865 LOG.debug('creating internet gateway')
866+ # pylint: disable=no-member
867 internet_gateway = self.ec2_resource.create_internet_gateway()
868 internet_gateway.attach_to_vpc(VpcId=self.vpc.id)
869 self._tag_resource(internet_gateway)
870@@ -190,7 +191,7 @@ class EC2Platform(Platform):
871 """Setup AWS EC2 VPC or return existing VPC."""
872 LOG.debug('creating new vpc')
873 try:
874- vpc = self.ec2_resource.create_vpc(
875+ vpc = self.ec2_resource.create_vpc( # pylint: disable=no-member
876 CidrBlock=self.ipv4_cidr,
877 AmazonProvidedIpv6CidrBlock=True)
878 except botocore.exceptions.ClientError as e:
879diff --git a/tests/cloud_tests/releases.yaml b/tests/cloud_tests/releases.yaml
880index 924ad95..7ddc5b8 100644
881--- a/tests/cloud_tests/releases.yaml
882+++ b/tests/cloud_tests/releases.yaml
883@@ -55,6 +55,8 @@ default_release_config:
884 # cloud-init, so must pull cloud-init in from repo using
885 # setup_image.upgrade
886 upgrade: true
887+ azurecloud:
888+ boot_timeout: 300
889
890 features:
891 # all currently supported feature flags
892diff --git a/tox.ini b/tox.ini
893index f5baf32..042346b 100644
894--- a/tox.ini
895+++ b/tox.ini
896@@ -24,6 +24,7 @@ deps =
897 pylint==2.3.1
898 # test-requirements because unit tests are now present in cloudinit tree
899 -r{toxinidir}/test-requirements.txt
900+ -r{toxinidir}/integration-requirements.txt
901 commands = {envpython} -m pylint {posargs:cloudinit tests tools}
902
903 [testenv:py3]
904@@ -135,6 +136,7 @@ deps =
905 pylint
906 # test-requirements
907 -r{toxinidir}/test-requirements.txt
908+ -r{toxinidir}/integration-requirements.txt
909
910 [testenv:citest]
911 basepython = python3

Subscribers

People subscribed via source and target branches