New Amazon AWS C5 instances are not recognised as a VM

Bug #1742531 reported by David Coronel
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Landscape Client
Fix Released
High
Simon Poirier
landscape-client (Ubuntu)
Fix Released
Medium
Eric Desrochers
Trusty
Fix Released
Medium
Eric Desrochers
Xenial
Fix Released
Medium
Eric Desrochers
Artful
Fix Released
Medium
Eric Desrochers
Bionic
Fix Released
Medium
Eric Desrochers

Bug Description

[Impact]

The new Amazon AWS C5 instance type is not a recognised VM type in Landscape and thus won't allow use of a Virtual Guest asset to register with.

The C5 instance type is a new type that uses a KVM-family hypervisor instead of Xen (displaying "Amazon EC2" in sys_vendor which landscape-client doesn't recognise, thus cannot associate/map to kvm)

"AWS has revealed it has created a new hypervisor based on KVM, not the Xen hypervisor on which it has relied for years."

From https://www.theregister.co.uk/2017/11/07/aws_writes_new_kvm_based_hypervisor_to_make_its_cloud_go_faster

[Test Case]

Scenario #1
 * Deploy an AWS C5 instance
 * Look sys_vendor
   $ cat /sys/class/dmi/id/sys_vendor
   Amazon EC2

 * Install landscape-client
 * Register the client to the desired landscape-server
 * Verify "VM Type" to be "kvm" under the computer info in landscape-server.

Scenario #2
 * Deploy an AWS non KVM/non C5 instance (Example : t2.medium)
 * Look sys_vendor
   $ cat /sys/class/dmi/id/sys_vendor
   Xen

 * Install landscape-client
 * Register the client to the desired landscape-server
 * Verify "VM Type" to still be "xen" under the computer info in landscape-server.

[Regression Potential]

 * No regression anticipated, the fix is only adding a new entry mapping to recognize C5 instance (Amazon EC2)

[Other Info]

 * Upstream commit:
   https://github.com/CanonicalLtd/landscape-client/pull/22/files

[Original Description]
The new Amazon AWS C5 instance type is not a recognised VM type in Landscape and thus won't allow use of a Virtual Guest asset to register with.

The C5 instance type is a new type that uses a KVM-family hypervisor instead of Xen:

"AWS has revealed it has created a new hypervisor based on KVM, not the Xen hypervisor on which it has relied for years."

From https://www.theregister.co.uk/2017/11/07/aws_writes_new_kvm_based_hypervisor_to_make_its_cloud_go_faster

============

ubuntu@ip-172-31-54-123:~$ cat /sys/class/dmi/id/sys_vendor
Amazon EC2

============

dmidecode output:

# dmidecode 3.0
Getting SMBIOS data from sysfs.
SMBIOS 2.7 present.
5 structures occupying 233 bytes.
Table at 0x000F9010.

Handle 0x0000, DMI type 0, 24 bytes
BIOS Information
 Vendor: Amazon EC2
 Version: 1.0
 Release Date: 10/16/2017
 Address: 0xF0000
 Runtime Size: 64 kB
 ROM Size: 64 kB
 Characteristics:
  PCI is supported
  EDD is supported
  ACPI is supported
  System is a virtual machine
 BIOS Revision: 1.0

Handle 0x0001, DMI type 1, 27 bytes
System Information
 Manufacturer: Amazon EC2
 Product Name: c5.large
 Version: Not Specified
 Serial Number: ec23cdb5-a248-6935-b9f6-de3a34334018
 UUID: EC23CDB5-A248-6935-B9F6-DE3A34334018
 Wake-up Type: Power Switch
 SKU Number: Not Specified
 Family: Not Specified

Handle 0x0002, DMI type 2, 15 bytes
Base Board Information
 Manufacturer: Amazon EC2
 Product Name: Not Specified
 Version: Not Specified
 Serial Number: Not Specified
 Asset Tag: i-09898bff0b7b55c35
 Features: None
 Location In Chassis: Not Specified
 Chassis Handle: 0x0003
 Type: Other
 Contained Object Handles: 0

Handle 0x0003, DMI type 3, 21 bytes
Chassis Information
 Manufacturer: Amazon EC2
 Type: Other
 Lock: Not Present
 Version: Not Specified
 Serial Number: Not Specified
 Asset Tag: Amazon EC2
 Boot-up State: Safe
 Power Supply State: Safe
 Thermal State: Safe
 Security Status: None
 OEM Information: 0x00000000
 Height: Unspecified
 Number Of Power Cords: 1
 Contained Elements: 0

Handle 0x0004, DMI type 127, 4 bytes
End Of Table

============

contents of /proc/cpuinfo :

processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 85
model name : Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz
stepping : 3
microcode : 0x100013e
cpu MHz : 3000.000
cache size : 25344 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 1
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f rdseed adx smap clflushopt clwb avx512cd xsaveopt xsavec xgetbv1 ida arat
bugs :
bogomips : 6000.00
clflush size : 64
cache_alignment : 64
address sizes : 46 bits physical, 48 bits virtual
power management:

processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 85
model name : Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz
stepping : 3
microcode : 0x100013e
cpu MHz : 3000.000
cache size : 25344 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 1
apicid : 1
initial apicid : 1
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f rdseed adx smap clflushopt clwb avx512cd xsaveopt xsavec xgetbv1 ida arat
bugs :
bogomips : 6000.00
clflush size : 64
cache_alignment : 64
address sizes : 46 bits physical, 48 bits virtual
power management:

David Britton (dpb)
Changed in landscape:
status: New → Confirmed
importance: Undecided → High
assignee: nobody → Simon Poirier (simpoir)
Revision history for this message
Eric Desrochers (slashd) wrote :

I wonder if this is just matter of adding a new entry in content_vendor_map specifically for "Amazon EC2" since it is not listed as a vendor in landscape yet.

# landscape/lib/vm_info.py
def _get_vm_by_vendor(sys_vendor_path):
    """Return the VM type string (possibly empty) based on the vendor."""
    vendor = read_file(sys_vendor_path).lower()
    # Use lower-key string for vendors, since we do case-insentive match.
    content_vendors_map = (
        ("bochs", "kvm"),
        ("google", "gce"),
        ("innotek", "virtualbox"),
        ("microsoft", "hyperv"),
        ("openstack", "kvm"),
+ ("amazon ec2", "kvm"),
        ("qemu", "kvm"),
        ("vmware", "vmware"))
    for name, vm_type in content_vendors_map:
        if name in vendor:
            return vm_type

    return ""

We have tested using "virt-what" just for comparison and it displays "kvm" inside a C5 instance as $cpuid == KVMKVMKVM. (cpuid get populated via command "virt-what-cpuid-helper" which return "KVMKVMKVM")

if ! "$skip_qemu_kvm"; then
 if [ "$cpuid" = "KVMKVMKVM" ]; then
 echo kvm

I'll let Simon judge the best way to approach this.

- Eric

Simon Poirier (simpoir)
Changed in landscape:
status: Confirmed → In Progress
Revision history for this message
Eric Desrochers (slashd) wrote :
Simon Poirier (simpoir)
information type: Proprietary → Public
Changed in landscape-client:
status: New → In Progress
importance: Undecided → High
assignee: nobody → Simon Poirier (simpoir)
no longer affects: landscape
Simon Poirier (simpoir)
Changed in landscape-client:
status: In Progress → Fix Committed
Eric Desrochers (slashd)
description: updated
Eric Desrochers (slashd)
Changed in landscape-client (Ubuntu Trusty):
status: New → Confirmed
Changed in landscape-client (Ubuntu Xenial):
status: New → Confirmed
Changed in landscape-client (Ubuntu Zesty):
status: New → Confirmed
Changed in landscape-client (Ubuntu Artful):
status: New → Confirmed
Changed in landscape-client (Ubuntu Bionic):
status: New → Confirmed
Changed in landscape-client (Ubuntu Trusty):
importance: Undecided → Medium
Changed in landscape-client (Ubuntu Xenial):
importance: Undecided → Medium
Changed in landscape-client (Ubuntu Zesty):
importance: Undecided → Medium
Changed in landscape-client (Ubuntu Artful):
importance: Undecided → Medium
Changed in landscape-client (Ubuntu Bionic):
importance: Undecided → Medium
description: updated
Eric Desrochers (slashd)
tags: added: sts
Revision history for this message
Simon Poirier (simpoir) wrote :

I should point out DigitalOcean instances started to inhibit the same behaviour and will also need to be identified in a similar fashion.

Simon Poirier (simpoir)
description: updated
Eric Desrochers (slashd)
Changed in landscape-client (Ubuntu Bionic):
assignee: nobody → Eric Desrochers (slashd)
Changed in landscape-client (Ubuntu Artful):
assignee: nobody → Eric Desrochers (slashd)
Changed in landscape-client (Ubuntu Zesty):
assignee: nobody → Eric Desrochers (slashd)
Changed in landscape-client (Ubuntu Xenial):
assignee: nobody → Eric Desrochers (slashd)
Changed in landscape-client (Ubuntu Trusty):
assignee: nobody → Eric Desrochers (slashd)
tags: added: sts-sponsor-slashd
Revision history for this message
Eric Desrochers (slashd) wrote :

I have reported a separate bug for digitcalocean :
https://bugs.launchpad.net/bugs/1743232

- Eric

Changed in landscape-client (Ubuntu Bionic):
status: Confirmed → In Progress
description: updated
Revision history for this message
Eric Desrochers (slashd) wrote :

"lp1742531-lp1743232-bionic.debdiff"

Revision history for this message
Eric Desrochers (slashd) wrote :

lp1742531-lp1743232-bionic_V2.debdiff

Revision history for this message
David Coronel (davecore) wrote :

I tested Eric's fix on Bionic on t2 and c5 instances:

==============

AWS t2 instance with community bionic AMI ubuntu/images-testing/hvm-ssd/ubuntu-bionic-daily-amd64-server-20180101 (ami-c55c16bf)

ubuntu@ip-172-31-48-172:~$ cat /sys/class/dmi/id/sys_vendor
Xen

ubuntu@ip-172-31-48-172:~$ dpkg -l | grep -i landscape-client
ii landscape-client 16.03-0ubuntu5 amd64 The Landscape administration system client

ubuntu@ip-172-31-48-172:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu Bionic Beaver (development branch)
Release: 18.04
Codename: bionic

Computer information in Landscape:
-Distribution: Ubuntu Bionic Beaver (development branch) (bionic)
-VM Type: xen
-Product identifier: HVM domU
-Instance type: t2.small
-AMI ID: ami-c55c16bf

==============

AWS c5 instance with community bionic AMI ubuntu/images-testing/hvm-ssd/ubuntu-bionic-daily-amd64-server-20180101 (ami-c55c16bf)

ubuntu@ip-172-31-72-153:~$ cat /sys/class/dmi/id/sys_vendor
Amazon EC2

ubuntu@ip-172-31-72-153:~$ dpkg -l | grep -i landscape-client
ii landscape-client 16.03-0ubuntu5 amd64 The Landscape administration system client

ubuntu@ip-172-31-72-153:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu Bionic Beaver (development branch)
Release: 18.04
Codename: bionic

Computer information in Landscape:
-Distribution: Ubuntu Bionic Beaver (development branch) (bionic)
-VM Type: kvm
-Product identifier: c5.large
-Instance type: c5.large
-AMI ID: ami-c55c16bf

tags: added: patch
Changed in landscape-client:
status: Fix Committed → In Progress
status: In Progress → Fix Committed
Revision history for this message
Simon Poirier (simpoir) wrote :
Revision history for this message
Simon Poirier (simpoir) wrote :
Revision history for this message
Simon Poirier (simpoir) wrote :
Revision history for this message
Simon Poirier (simpoir) wrote :
Eric Desrochers (slashd)
Changed in landscape-client (Ubuntu Bionic):
status: In Progress → Fix Committed
Revision history for this message
Eric Desrochers (slashd) wrote :

I'll sponsor Simon's debdiff as soon as landscape-client is consider valid for bionic.

For now it is stuck in bionic-proposed, waiting for the non-i386/amd64 builds to be back operational.

# Ubuntu excuses page
landscape-client (16.03-0ubuntu4 to 16.03-0ubuntu5)
Maintainer: Ubuntu Developers
0 days old
missing build on arm64: landscape-client, landscape-common (from 16.03-0ubuntu4)
missing build on armhf: landscape-client, landscape-common (from 16.03-0ubuntu4)
missing build on ppc64el: landscape-client, landscape-common (from 16.03-0ubuntu4)
missing build on s390x: landscape-client, landscape-common (from 16.03-0ubuntu4)
Not considered

- Eric

Changed in landscape-client:
milestone: none → 18.01
Eric Desrochers (slashd)
no longer affects: landscape-client (Ubuntu Zesty)
Eric Desrochers (slashd)
Changed in landscape-client (Ubuntu Artful):
status: Confirmed → In Progress
Changed in landscape-client (Ubuntu Xenial):
status: Confirmed → In Progress
Changed in landscape-client (Ubuntu Trusty):
status: Confirmed → In Progress
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package landscape-client - 16.03-0ubuntu5

---------------
landscape-client (16.03-0ubuntu5) bionic; urgency=medium

  * d/p/set-vm-info-to-kvm-for-aws-C5-instances.patch:
    Sets vm_info to kvm for new AWS EC2 C5 instances. (LP: #1742531)

  * d/p/set-vm-info-to-kvm-for-digitalocean-instances.patch:
    Sets vm_info to kvm for digitalocean instances. (LP: #1743232)

 -- Eric Desrochers <email address hidden> Sun, 14 Jan 2018 11:05:38 -0500

Changed in landscape-client (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Eric Desrochers (slashd) wrote :

lds-client has been uploaded for TXA. It is now waiting on SRU verification team to approve the uploads.

- Eric

tags: added: sts-sponsor-slashd-done
removed: sts-sponsor-slashd
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello David, or anyone else affected,

Accepted landscape-client into artful-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/landscape-client/16.03-0ubuntu3.17.10.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-artful to verification-done-artful. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-artful. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in landscape-client (Ubuntu Artful):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-artful
Changed in landscape-client (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed-xenial
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Hello David, or anyone else affected,

Accepted landscape-client into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/landscape-client/16.03-0ubuntu2.16.04.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Hello David, or anyone else affected,

Accepted landscape-client into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/landscape-client/14.12-0ubuntu6.14.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-trusty to verification-done-trusty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-trusty. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in landscape-client (Ubuntu Trusty):
status: In Progress → Fix Committed
tags: added: verification-needed-trusty
Revision history for this message
David Coronel (davecore) wrote :

I tested that the new landscape-client package installs on AWS C5 instances with trusty, xenial and artful and registers in Landscape with the right information. Attaching screenshots.

Revision history for this message
David Coronel (davecore) wrote :
Revision history for this message
David Coronel (davecore) wrote :
Revision history for this message
David Coronel (davecore) wrote :
Revision history for this message
Eric Desrochers (slashd) wrote :

Note that the same exact test has been also performed on "non-C5" instance.

For this particular bug we used "t2.medium" instances on T/X/A. We confirmed it still work as expected for non-C5 instances for every affected stable releases.

To summarize :
* C5 instance -> Display KVM in Landscape server as it should.
* Non C5 instance -> Still display XEN in Landscape-server as it should.

- Eric

tags: added: verification-done verification-done-artful verification-done-trusty verification-done-xenial
removed: verification-needed verification-needed-artful verification-needed-trusty verification-needed-xenial
Eric Desrochers (slashd)
description: updated
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for landscape-client has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package landscape-client - 16.03-0ubuntu2.16.04.3

---------------
landscape-client (16.03-0ubuntu2.16.04.3) xenial; urgency=medium

  * d/p/set-vm-info-to-kvm-for-aws-C5-instances.patch:
    Sets vm_info to kvm for new AWS EC2 C5 instances. (LP: #1742531)

  * d/p/set-vm-info-to-kvm-for-digitalocean-instances.patch:
    Sets vm_info to kvm for digitalocean instances. (LP: #1743232)

 -- Eric Desrochers <email address hidden> Tue, 23 Jan 2018 10:53:42 -0500

Changed in landscape-client (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package landscape-client - 14.12-0ubuntu6.14.04.2

---------------
landscape-client (14.12-0ubuntu6.14.04.2) trusty; urgency=medium

  * d/p/set-vm-info-to-kvm-for-aws-C5-instances.patch:
    Sets vm_info to kvm for new AWS EC2 C5 instances. (LP: #1742531)

  * d/p/set-vm-info-to-kvm-for-digitalocean-instances.patch:
    Sets vm_info to kvm for digitalocean instances. (LP: #1743232)

 -- Eric Desrochers <email address hidden> Tue, 23 Jan 2018 11:32:00 -0500

Changed in landscape-client (Ubuntu Trusty):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package landscape-client - 16.03-0ubuntu3.17.10.2

---------------
landscape-client (16.03-0ubuntu3.17.10.2) artful; urgency=medium

  * d/p/set-vm-info-to-kvm-for-aws-C5-instances.patch:
    Sets vm_info to kvm for new AWS EC2 C5 instances. (LP: #1742531)

  * d/p/set-vm-info-to-kvm-for-digitalocean-instances.patch:
    Sets vm_info to kvm for digitalocean instances. (LP: #1743232)

 -- Eric Desrochers <email address hidden> Tue, 23 Jan 2018 10:44:56 -0500

Changed in landscape-client (Ubuntu Artful):
status: Fix Committed → Fix Released
Changed in landscape-client:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.