Merge ~chad.smith/cloud-init:feature/1797480-azure-support-gen2-instances into cloud-init:master

[Chad Smith]

azure: add udev rules to create cloud-init Gen2 disk name symlinks

Cloud-init delivers udev rules on Azure to create the following symlinks:
- /dev/disk/cloud/azure_root
- /dev/disk/cloud/azure_root-part#
- /dev/disk/cloud/azure_resource
- /dev/disk/cloud/azure_resource-part#

Cloud-init cc_disk_setup expects presence of these dev links in order to
setup the mounted ephemeral disks. Gen1 instances udev rules match based
only a DEVICE_ID attribute that no longer exists on Gen2 instances.
Supplement existing Gen1 rules with matches on specitic SCSI target/lun
path 0:0:0 and 0:0:1 and generate links for azure_root and azure_resource
respectively.

LP: #1797480

[Chad Smith (chad.smith)]

To test:
Gen2 instances are only on eastus region or westeurope I believe

# Spawn gen1 & gen2, install this version of cloud-init and run cloud-init clean --reboot --logs
# Ensure proper /dev/disk/cloud symllinks exist and no Tracebacks in /var/log/cloud-init.log

make deb

# Gen1
az vm create --name=cosmic-gen1 --image=Canonical:UbuntuServer:18.10-DAILY:latest --admin-username=root -g srugroup1 --admin-username=ubuntu --custom-data ../sethostname.yaml

vm_ip=`az vm list-ip-addresses --name cosmic-gen1 | jq -r '.[] | .virtualMachine.network.publicIpAddresses[].ipAddress'`
scp cloud-init*deb ubuntu@$vm_ip:

ssh ubuntu@$vm_ip grep Trace /var/log/cloud-init.log;
echo '--- Expect gen1 cloud disk dev links for root and resource + partiions';
ssh ubuntu@$vm_ip ls -l /dev/disk/cloud;
ssh ubuntu@$vm_ip 'sudo dpkg -i ./cloud-init*deb';
ssh ubuntu@$vm_ip 'sudo cloud-init clean --logs --reboot'
echo '--- Expect same gen1 cloud disk dev links for root and resource + partiions';
ssh ubuntu@$vm_ip ls -l /dev/disk/cloud;
echo '--- Expect no Traceback";
ubuntu@$vm_ip grep Trace /var/log/cloud-init.log; # Expect no trace

# Gen2
az vm create --name=cosmic-gen2 --image=Canonical:UbuntuServer:18.10-DAILY:latest --admin-username=root -g srugroup1 --admin-username=ubuntu --custom-data ../sethostname.yaml --size Standard_DC2s -l eastus

vm_ip=`az vm list-ip-addresses --name cosmic-gen1 | jq -r '.[] | .virtualMachine.network.publicIpAddresses[].ipAddress'`
make deb

echo '--- Expect Traceback on Gen2 with stock cloud-init";
ssh ubuntu@$vm_ip grep Trace -A 8 /var/log/cloud-init.log;
echo '--- Expect no Gen2 cloud disk dev links for root and resource + partiions';
ssh ubuntu@$vm_ip ls -l /dev/disk/cloud;
ssh ubuntu@$vm_ip 'sudo dpkg -i ./cloud-init*deb';
ssh ubuntu@$vm_ip 'sudo cloud-init clean --logs --reboot'
echo '--- Expect Gen2 cloud disk dev links for root and resource + partiions';
ssh ubuntu@$vm_ip ls -l /dev/disk/cloud;
echo '--- Expect no Traceback after upgrade";
ubuntu@$vm_ip grep Trace /var/log/cloud-init.log; # Expect no trace

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:bbaeef570d9ae9146e2abe9c8f76813c3d402cb7
https://jenkins.ubuntu.com/server/job/cloud-init-ci/423/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/423/rebuild

review: Approve (continuous-integration)

[Paul Meyer (paul-meyer)]

Checking on the lun numbers

[Paul Meyer (paul-meyer)]

> Checking on the lun numbers

I can confirm the LUN numbers are correct (OS/root=0, resource=1) and hard coded.

review: Approve

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:93e9602e94b3d7e954c3a12db52fe8cc954c1f84
https://jenkins.ubuntu.com/server/job/cloud-init-ci/425/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/425/rebuild

review: Approve (continuous-integration)

diff --git a/udev/66-azure-ephemeral.rules b/udev/66-azure-ephemeral.rules
index b9c5c3e..3032f7e 100644
--- a/udev/66-azure-ephemeral.rules
+++ b/udev/66-azure-ephemeral.rules
@@ -4,10 +4,26 @@ SUBSYSTEM!="block", GOTO="cloud_init_end"
 ATTRS{ID_VENDOR}!="Msft", GOTO="cloud_init_end"
 ATTRS{ID_MODEL}!="Virtual_Disk", GOTO="cloud_init_end"
 
-# Root has a GUID of 0000 as the second value
+# Root has a GUID of 0000 as the second value on Gen1 instances
 # The resource/resource has GUID of 0001 as the second value
 ATTRS{device_id}=="?00000000-0000-*", ENV{fabric_name}="azure_root", GOTO="ci_azure_names"
 ATTRS{device_id}=="?00000000-0001-*", ENV{fabric_name}="azure_resource", GOTO="ci_azure_names"
+
+# Azure well known SCSI controllers on Gen2 instances
+ATTRS{device_id}=="{f8b3781a-1e82-4818-a1c3-63d806ec15bb}", ENV{fabric_scsi_controller}="scsi0", GOTO="azure_datadisk"
+# Do not create symlinks for scsi[1-3] or unmatched device_ids
+ATTRS{device_id}=="{f8b3781b-1e82-4818-a1c3-63d806ec15bb}", ENV{fabric_scsi_controller}="scsi1", GOTO="cloud_init_end"
+ATTRS{device_id}=="{f8b3781c-1e82-4818-a1c3-63d806ec15bb}", ENV{fabric_scsi_controller}="scsi2", GOTO="cloud_init_end"
+ATTRS{device_id}=="{f8b3781d-1e82-4818-a1c3-63d806ec15bb}", ENV{fabric_scsi_controller}="scsi3", GOTO="cloud_init_end"
+GOTO="cloud_init_end"
+
+# Map scsi#/lun# fabric_name to azure_root|resource on Gen2 instances
+LABEL="azure_datadisk"
+ENV{DEVTYPE}=="partition", PROGRAM="/bin/sh -c 'readlink /sys/class/block/%k/../device|cut -d: -f4'", ENV{fabric_name}="$env{fabric_scsi_controller}/lun$result"
+ENV{DEVTYPE}=="disk", PROGRAM="/bin/sh -c 'readlink /sys/class/block/%k/device|cut -d: -f4'", ENV{fabric_name}="$env{fabric_scsi_controller}/lun$result"
+
+ENV{fabric_name}=="scsi0/lun0", ENV{fabric_name}="azure_root", GOTO="ci_azure_names"
+ENV{fabric_name}=="scsi0/lun1", ENV{fabric_name}="azure_resource", GOTO="ci_azure_names"
 GOTO="cloud_init_end"
 
 # Create the symlinks