Merge lp:~blake-rouse/maas/fix-power8 into lp:~maas-committers/maas/trunk

[Blake Rouse]

Fix storage configuration so it adds a prep partition at the beginning of the disk for deploying PowerNV. Fix PowerNV and PowerKVM to not create the EFI partition when the storage layout is generated.

[Gavin Panella (allenap)]

The abundant special-casing puts this on the slope towards spaghetti, but additional tests and documentation (chiefly in the form of comments) will alleviate that and make it easier to unravel later on.

I have several questions but no reservations other than the above, so Needs Information.

review: Needs Information

[Blake Rouse (blake-rouse)]

Thanks for the review.

"slope towards spaghetti", totally agree! But this is one branch that could not be avoided, due to the current state of MAAS in the release process. Because of Django 1.8 and migrations we cannot release any new migrations in MAAS 1.9 and this branch has a requirement to be in MAAS 1.9. The only benefit we get out of this is we get to hide this stupid partition from the user. This is also fallout on not having every architecture in our CI. If this would have existed in our CI we would have caught this very early in the MAAS 1.9 cycle and I would not be doing it now and it could have been done properly and not as some magic if case throughout the storage generation.

See inline comments for responses. Also you reviewed an earlier version of the branch, some things have already changed.

[Gavin Panella (allenap)]

review: Approve

[Blake Rouse (blake-rouse)]

I am waiting to land this once curtin is fixed to work with the way we are creating partitions.

Once this bug is resolved and I can QA this I will land this and backport to 1.9.

https://bugs.launchpad.net/curtin/+bug/1543263

[MAAS Lander (maas-lander)]

The attempt to merge lp:~blake-rouse/maas/fix-power8 into lp:maas failed. Below is the output from the failed tests.

Get:1 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial InRelease [95.8 kB]
Hit:2 http://security.ubuntu.com/ubuntu xenial-security InRelease
Hit:3 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-updates InRelease
Hit:4 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-backports InRelease
Get:5 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial/main Sources [1,108 kB]
Get:6 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial/universe Sources [7,717 kB]
Get:7 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial/main amd64 Packages [1,442 kB]
Get:8 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial/main Translation-en [844 kB]
Get:9 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial/universe amd64 Packages [7,352 kB]
Get:10 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial/universe Translation-en [4,889 kB]
Fetched 23.4 MB in 34s (688 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind bind9 bind9utils build-essential bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm postgresql pxelinux python3-apt python3-bson python3-convoy python3-coverage python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-mock python3-netaddr python3-netifaces python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-seamicroclient python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
apache2 is already the newest version (2.4.18-1ubuntu1).
archdetect-deb is already the newest version (1.114ubuntu2).
authbind is already the newest version (2.1.1+nmu1).
bind9 is already the newest version (1:9.10.3.dfsg.P2-4).
bind9utils is already the newest version (1:9.10.3.dfsg.P2-4).
build-essential is already the newest version (12.1ubuntu2).
curl is already the newest version (7.47.0-1ubuntu2).
debhelper is already the newest version (9.20160115ubuntu2).
dh-apport is already the newest version (2.20-0ubuntu3).
dh-systemd is already the newest version (1.28ubuntu2)....

=== modified file 'docs/changelog.rst'
--- docs/changelog.rst	2016-02-17 16:12:30 +0000
+++ docs/changelog.rst	2016-02-24 16:12:28 +0000
@@ -3,6 +3,26 @@
 =========
 
 
+1.9.1
+=====
+
+See https://launchpad.net/maas/+milestone/1.9.1 for full details.
+
+Bug Fix Update
+--------------
+
+#1523779    Fix grub-install error on deploying power8 machines.
+
+#1526542    Skip block devices with duplicate serial numbers to fix multipath issue.
+
+#1536754    Upgrade from 1.8 to 1.9 lost connected macs in all but one network.
+
+#1532262    Fix failure to power query requests for SM15K servers.
+
+#1484696    Fix bug in apache2 maas config where it will reuse websocket connections
+            to work around a bug in apache2 itself.
+
+
 1.9.0
 =====
 

=== modified file 'src/maasserver/models/node.py'
--- src/maasserver/models/node.py	2016-02-19 01:46:23 +0000
+++ src/maasserver/models/node.py	2016-02-24 16:12:28 +0000
@@ -158,7 +158,7 @@
 
 # Holds the known `bios_boot_methods`. If `bios_boot_method` is not in this
 # list then it will fallback to `DEFAULT_BIOS_BOOT_METHOD`.
-KNOWN_BIOS_BOOT_METHODS = ["pxe", "uefi"]
+KNOWN_BIOS_BOOT_METHODS = ["pxe", "uefi", "powernv", "powerkvm"]
 
 # Default `bios_boot_method`. See `KNOWN_BIOS_BOOT_METHOD` above for usage.
 DEFAULT_BIOS_BOOT_METHOD = "pxe"

=== modified file 'src/maasserver/models/partition.py'
--- src/maasserver/models/partition.py	2015-12-01 18:12:59 +0000
+++ src/maasserver/models/partition.py	2016-02-24 16:12:28 +0000
@@ -180,7 +180,17 @@
         partitions_in_table = sorted(partitions_in_table, key=attrgetter('id'))
         idx = partitions_in_table.index(self)
         if self.partition_table.table_type == PARTITION_TABLE_TYPE.GPT:
-            return idx + 1
+            # ppc64el machines get part1 skipped when this partition is on
+            # the boot disk. This is because the prep partition is part1 and
+            # is added when the preseed for storage is generated.
+            node = self.get_node()
+            arch, _ = node.split_arch()
+            boot_disk = node.get_boot_disk()
+            if (arch == "ppc64el" and
+                    self.partition_table.block_device.id == boot_disk.id):
+                return idx + 2
+            else:
+                return idx + 1
         elif self.partition_table.table_type == PARTITION_TABLE_TYPE.MBR:
             # If more than 4 partitions then the 4th partition number is
             # skipped because that is used for the extended partition.

=== modified file 'src/maasserver/models/partitiontable.py'
--- src/maasserver/models/partitiontable.py	2015-12-01 18:12:59 +0000
+++ src/maasserver/models/partitiontable.py	2016-02-24 16:12:28 +0000
@@ -40,6 +40,13 @@
 PARTITION_TABLE_EXTRA_SPACE = (
     INITIAL_PARTITION_OFFSET + END_OF_PARTITION_TABLE_SPACE)
 
+# The amount of space required to be reserved for the prep partition. Prep
+# partition is required by all ppc64el architectures. Because of the way the
+# system boots it requires that a 8MiB prep partition exist with grub installed
+# on that partition. Without this partition the installation of grub will fail
+# on ppc64el and will fail to boot.
+PREP_PARTITION_SIZE = 8 * 1024 * 1024  # 8MiB
+
 
 class PartitionTable(CleanSave, TimestampedModel):
     """A partition table on a block device.
@@ -64,7 +71,7 @@
     def get_size(self):
         """Total usable size of partition table."""
         return round_size_to_nearest_block(
-            self.block_device.size - PARTITION_TABLE_EXTRA_SPACE,
+            self.block_device.size - self.get_overhead_size(),
             PARTITION_ALIGNMENT_SIZE,
             False)
 
@@ -72,6 +79,15 @@
         """Block size of partition table."""
         return self.block_device.block_size
 
+    def get_overhead_size(self):
+        """Return the total amount of extra space this partition table
+        requires."""
+        extra_space = PARTITION_TABLE_EXTRA_SPACE
+        node_arch, _ = self.block_device.node.split_arch()
+        if node_arch == "ppc64el":
+            extra_space += PREP_PARTITION_SIZE
+        return extra_space
+
     def get_used_size(self, ignore_partitions=[]):
         """Return the used size of partitions on the table."""
         ignore_ids = [
@@ -84,7 +100,7 @@
         if used_size is None:
             used_size = 0
         # The extra space taken by the partition table header is used space.
-        return used_size + PARTITION_TABLE_EXTRA_SPACE
+        return used_size + self.get_overhead_size()
 
     def get_available_size(self, ignore_partitions=[]):
         """Return the remaining size available for partitions."""
@@ -133,8 +149,8 @@
             # placed on the boot disk.
             if boot_disk is not None and self.block_device.id == boot_disk.id:
                 bios_boot_method = node.get_bios_boot_method()
-                if bios_boot_method == "uefi":
-                    # UEFI must always use a GPT table.
+                if bios_boot_method in ["uefi", "powernv", "powerkvm"]:
+                    # UEFI, PowerNV, or PowerKVM must always use a GPT table.
                     if not self.table_type:
                         self.table_type = PARTITION_TABLE_TYPE.GPT
                     elif self.table_type != PARTITION_TABLE_TYPE.GPT:

=== modified file 'src/maasserver/models/tests/test_node.py'
--- src/maasserver/models/tests/test_node.py	2016-02-19 01:46:23 +0000
+++ src/maasserver/models/tests/test_node.py	2016-02-24 16:12:28 +0000
@@ -390,6 +390,14 @@
         node = factory.make_Node(bios_boot_method="uefi")
         self.assertEqual("uefi", node.get_bios_boot_method())
 
+    def test_get_bios_boot_method_returns_powernv(self):
+        node = factory.make_Node(bios_boot_method="powernv")
+        self.assertEqual("powernv", node.get_bios_boot_method())
+
+    def test_get_bios_boot_method_returns_powerkvm(self):
+        node = factory.make_Node(bios_boot_method="powerkvm")
+        self.assertEqual("powerkvm", node.get_bios_boot_method())
+
     def test_get_bios_boot_method_fallback_to_pxe(self):
         node = factory.make_Node(bios_boot_method=factory.make_name("boot"))
         self.assertEqual("pxe", node.get_bios_boot_method())

=== modified file 'src/maasserver/models/tests/test_partition.py'
--- src/maasserver/models/tests/test_partition.py	2015-12-01 18:12:59 +0000
+++ src/maasserver/models/tests/test_partition.py	2016-02-24 16:12:28 +0000
@@ -22,7 +22,10 @@
     Partition,
     PARTITION_ALIGNMENT_SIZE,
 )
-from maasserver.models.partitiontable import PARTITION_TABLE_EXTRA_SPACE
+from maasserver.models.partitiontable import (
+    PARTITION_TABLE_EXTRA_SPACE,
+    PREP_PARTITION_SIZE,
+)
 from maasserver.testing.factory import factory
 from maasserver.testing.orm import reload_object
 from maasserver.testing.testcase import MAASServerTestCase
@@ -302,6 +305,27 @@
             self.expectThat(idx, Equals(partition.get_partition_number()))
             idx += 1
 
+    def test_get_partition_number_returns_starting_at_2_for_ppc64el(self):
+        node = factory.make_Node(
+            architecture="ppc64el/generic", bios_boot_method="uefi")
+        block_device = factory.make_PhysicalBlockDevice(
+            node=node,
+            size=(
+                (MIN_PARTITION_SIZE * 4) + PARTITION_TABLE_EXTRA_SPACE +
+                PREP_PARTITION_SIZE))
+        node.boot_disk = block_device
+        node.save()
+        partition_table = factory.make_PartitionTable(
+            block_device=block_device, table_type=PARTITION_TABLE_TYPE.GPT)
+        partitions = [
+            partition_table.add_partition(size=MIN_BLOCK_DEVICE_SIZE)
+            for _ in range(4)
+        ]
+        idx = 2
+        for partition in partitions:
+            self.expectThat(idx, Equals(partition.get_partition_number()))
+            idx += 1
+
     def test_get_partition_number_returns_correct_numbering_for_mbr(self):
         block_device = factory.make_PhysicalBlockDevice(
             size=(MIN_BLOCK_DEVICE_SIZE * 6) + PARTITION_TABLE_EXTRA_SPACE)

=== modified file 'src/maasserver/models/tests/test_partitiontable.py'
--- src/maasserver/models/tests/test_partitiontable.py	2015-12-01 18:12:59 +0000
+++ src/maasserver/models/tests/test_partitiontable.py	2016-02-24 16:12:28 +0000
@@ -19,7 +19,10 @@
     MIN_PARTITION_SIZE,
     PARTITION_ALIGNMENT_SIZE,
 )
-from maasserver.models.partitiontable import PARTITION_TABLE_EXTRA_SPACE
+from maasserver.models.partitiontable import (
+    PARTITION_TABLE_EXTRA_SPACE,
+    PREP_PARTITION_SIZE,
+)
 from maasserver.testing.factory import factory
 from maasserver.testing.testcase import MAASServerTestCase
 from maasserver.utils.converters import round_size_to_nearest_block
@@ -43,6 +46,19 @@
                 False),
             partition_table.get_size())
 
+    def test_get_size_returns_block_device_size_minus_ppc64el(self):
+        node = factory.make_Node(architecture="ppc64el/generic")
+        block_device = factory.make_PhysicalBlockDevice(node=node)
+        partition_table = factory.make_PartitionTable(
+            block_device=block_device)
+        self.assertEqual(
+            round_size_to_nearest_block(
+                partition_table.block_device.size -
+                PARTITION_TABLE_EXTRA_SPACE - PREP_PARTITION_SIZE,
+                PARTITION_ALIGNMENT_SIZE,
+                False),
+            partition_table.get_size())
+
     def test_get_block_size_returns_block_device_block_size(self):
         partition_table = factory.make_PartitionTable()
         self.assertEqual(
@@ -114,6 +130,24 @@
         self.assertRaises(
             ValidationError, partition_table.add_partition)
 
+    def test_get_overhead_size(self):
+        node = factory.make_Node(bios_boot_method="pxe")
+        block_device = factory.make_PhysicalBlockDevice(node=node)
+        partition_table = factory.make_PartitionTable(
+            block_device=block_device)
+        self.assertEquals(
+            PARTITION_TABLE_EXTRA_SPACE,
+            partition_table.get_overhead_size())
+
+    def test_get_overhead_size_for_ppc64el(self):
+        node = factory.make_Node(architecture="ppc64el/generic")
+        block_device = factory.make_PhysicalBlockDevice(node=node)
+        partition_table = factory.make_PartitionTable(
+            block_device=block_device)
+        self.assertEquals(
+            PARTITION_TABLE_EXTRA_SPACE + PREP_PARTITION_SIZE,
+            partition_table.get_overhead_size())
+
     def test_get_available_size(self):
         block_size = 4096
         device = factory.make_BlockDevice(
@@ -153,6 +187,20 @@
         partition_table = factory.make_PartitionTable(block_device=boot_disk)
         self.assertEqual(PARTITION_TABLE_TYPE.GPT, partition_table.table_type)
 
+    def test_save_sets_table_type_to_gpt_for_powernv_boot(self):
+        node = factory.make_Node(
+            with_boot_disk=False, bios_boot_method="powernv")
+        boot_disk = factory.make_PhysicalBlockDevice(node=node)
+        partition_table = factory.make_PartitionTable(block_device=boot_disk)
+        self.assertEqual(PARTITION_TABLE_TYPE.GPT, partition_table.table_type)
+
+    def test_save_sets_table_type_to_gpt_for_powerkvm_boot(self):
+        node = factory.make_Node(
+            with_boot_disk=False, bios_boot_method="powerkvm")
+        boot_disk = factory.make_PhysicalBlockDevice(node=node)
+        partition_table = factory.make_PartitionTable(block_device=boot_disk)
+        self.assertEqual(PARTITION_TABLE_TYPE.GPT, partition_table.table_type)
+
     def test_save_sets_table_type_to_gpt_for_none_boot_disk(self):
         node = factory.make_Node(with_boot_disk=False, bios_boot_method="pxe")
         factory.make_PhysicalBlockDevice(node=node)

=== modified file 'src/maasserver/preseed_storage.py'
--- src/maasserver/preseed_storage.py	2016-02-23 12:13:09 +0000
+++ src/maasserver/preseed_storage.py	2016-02-24 16:12:28 +0000
@@ -14,7 +14,10 @@
     FILESYSTEM_TYPE,
     PARTITION_TABLE_TYPE,
 )
-from maasserver.models.partitiontable import INITIAL_PARTITION_OFFSET
+from maasserver.models.partitiontable import (
+    INITIAL_PARTITION_OFFSET,
+    PREP_PARTITION_SIZE,
+)
 from maasserver.models.physicalblockdevice import PhysicalBlockDevice
 from maasserver.models.virtualblockdevice import VirtualBlockDevice
 import yaml
@@ -26,6 +29,7 @@
     def __init__(self, node):
         self.node = node
         self.boot_disk = node.get_boot_disk()
+        self.boot_disk_first_partition = None
         self.operations = {
             "disk": [],
             "partition": [],
@@ -103,6 +107,13 @@
                 raise ValueError("Unknown block device instance: %s" % (
                     block_device.__class__.__name__))
 
+    def _requires_prep_partition(self, block_device):
+        """Return True if block device requires the prep partition."""
+        arch, _ = self.node.split_arch()
+        return (
+            self.boot_disk.id == block_device.id and
+            arch == "ppc64el")
+
     def _add_partition_operations(self):
         """Add all the partition operations.
 
@@ -110,9 +121,16 @@
         attached to the node.
         """
         for block_device in self.node.blockdevice_set.order_by('id'):
+            requires_prep = self._requires_prep_partition(block_device)
             partition_table = block_device.get_partitiontable()
             if partition_table is not None:
-                for partition in partition_table.partitions.order_by('id'):
+                partitions = list(partition_table.partitions.order_by('id'))
+                for idx, partition in enumerate(partitions):
+                    # If this is the last partition and prep partition is
+                    # required then set boot_disk_first_partition so extra
+                    # space can be removed.
+                    if requires_prep and idx == 0:
+                        self.boot_disk_first_partition = partition
                     self.operations["partition"].append(partition)
 
     def _add_format_and_mount_operations(self):
@@ -172,21 +190,35 @@
 
         # Set the partition table type if a partition table exists or if this
         # is the boot disk.
+        add_prep_partition = False
         partition_table = block_device.get_partitiontable()
         if partition_table is not None:
             disk_operation["ptable"] = self._get_ptable_type(
                 partition_table)
         elif block_device.id == self.boot_disk.id:
-            if self.node.get_bios_boot_method() == "uefi":
+            bios_boot_method = self.node.get_bios_boot_method()
+            node_arch, _ = self.node.split_arch()
+            if bios_boot_method in [
+                    "uefi", "powernv", "powerkvm"]:
                 disk_operation["ptable"] = "gpt"
+                if node_arch == "ppc64el":
+                    add_prep_partition = True
             else:
                 disk_operation["ptable"] = "msdos"
 
-        # Set this disk to be the grub device if its the boot disk.
-        if self.boot_disk == block_device:
+        # Set this disk to be the grub device if it's the boot disk and doesn't
+        # require a prep partition. When a prep partition is required grub
+        # must be installed on that partition and not in the partition header
+        # of that disk.
+        requires_prep = self._requires_prep_partition(block_device)
+        if self.boot_disk.id == block_device.id and not requires_prep:
             disk_operation["grub_device"] = True
         self.storage_config.append(disk_operation)
 
+        # Add the prep partition at the end of the disk when it is required.
+        if add_prep_partition:
+            self._generate_prep_partition(block_device.get_name())
+
     def _get_ptable_type(self, partition_table):
         """Return the value for the "ptable" entry in the physical operation.
         """
@@ -199,12 +231,38 @@
                 "Unknown partition table type: %s" % (
                     partition_table.table_type))
 
+    def _generate_prep_partition(self, device_name):
+        """Generate the prep partition at the beginning of the block device."""
+        prep_part_name = "%s-part1" % (device_name)
+        partition_operation = {
+            "id": prep_part_name,
+            "name": prep_part_name,
+            "type": "partition",
+            "number": 1,
+            "offset": "%dB" % INITIAL_PARTITION_OFFSET,
+            "size": "%dB" % PREP_PARTITION_SIZE,
+            "device": device_name,
+            "wipe": "zero",
+            "flag": "prep",
+            "grub_device": True,
+        }
+        self.storage_config.append(partition_operation)
+
     def _generate_partition_operations(self):
         """Generate all partition operations."""
         for partition in self.operations["partition"]:
-            self._generate_partition_operation(partition)
+            if partition == self.boot_disk_first_partition:
+                # This is the first partition in the boot disk and add prep
+                # partition at the beginning of the partition table.
+                device_name = partition.partition_table.block_device.get_name()
+                self._generate_prep_partition(device_name)
+                self._generate_partition_operation(
+                    partition, include_initial=False)
+            else:
+                self._generate_partition_operation(
+                    partition, include_initial=True)
 
-    def _generate_partition_operation(self, partition):
+    def _generate_partition_operation(self, partition, include_initial):
         """Generate partition operation for `partition` and place in
         `storage_config`."""
         partition_table = partition.partition_table
@@ -221,7 +279,7 @@
             "wipe": "superblock",
         }
         # First partition always sets the initial offset.
-        if partition_number == 1:
+        if partition_number == 1 and include_initial:
             partition_operation["offset"] = "%sB" % INITIAL_PARTITION_OFFSET
         if partition.bootable:
             partition_operation["flag"] = "boot"

=== modified file 'src/maasserver/static/js/angular/controllers/node_details_storage.js'
--- src/maasserver/static/js/angular/controllers/node_details_storage.js	2016-02-02 14:20:45 +0000
+++ src/maasserver/static/js/angular/controllers/node_details_storage.js	2016-02-24 16:12:28 +0000
@@ -51,6 +51,7 @@
         var END_OF_PARTITION_TABLE_SPACE = 1024 * 1024;
         var PARTITION_TABLE_EXTRA_SPACE = INITIAL_PARTITION_OFFSET +
             END_OF_PARTITION_TABLE_SPACE;
+        var PREP_PARTITION_SIZE = 8 * 1024 * 1024;
 
         // From models/partition.py - must be kept in sync.
         var PARTITION_ALIGNMENT_SIZE = 4 * 1024 * 1024;
@@ -855,6 +856,11 @@
                 || disk.original.partition_table_type === "") {
                 // Disk has no partition table, so reserve space for it.
                 space_to_reserve = PARTITION_TABLE_EXTRA_SPACE;
+                // ppc64el node requires that space be saved for the prep
+                // partition.
+                if($scope.node.architecture.indexOf("ppc64el") === 0) {
+                    space_to_reserve += PREP_PARTITION_SIZE;
+                }
             }
             return ConverterService.roundByBlockSize(
                 disk.original.available_size - space_to_reserve,
@@ -1130,6 +1136,11 @@
             if(disk.original.partition_table_type === "mbr" &&
                 length > 2) {
                 return disk.name + "-part" + (length + 2);
+            } else if($scope.node.architecture.indexOf("ppc64el") === 0 &&
+                disk.original.is_boot) {
+                // Boot disk on ppc64el machines skip the first partition as
+                // its reserved for the prep partition.
+                return disk.name + "-part" + (length + 2);
             } else {
                 return disk.name + "-part" + (length + 1);
             }

=== modified file 'src/maasserver/static/js/angular/controllers/tests/test_node_details_storage.js'
--- src/maasserver/static/js/angular/controllers/tests/test_node_details_storage.js	2016-02-02 14:20:45 +0000
+++ src/maasserver/static/js/angular/controllers/tests/test_node_details_storage.js	2016-02-24 16:12:28 +0000
@@ -101,6 +101,7 @@
     var node, updateNodeSpy, canEditSpy;
     beforeEach(function() {
         node = {
+            architecture: "amd64/generic",
             disks: []
         };
         updateNodeSpy = jasmine.createSpy("updateNode");
@@ -1455,6 +1456,25 @@
                 expect($scope.canAddPartition(disk)).toBe(false);
             });
 
+        it("returns false if available_size is less than partition size " +
+           "when node is ppc64el architecture",
+            function() {
+                var controller = makeController();
+                var disk = {
+                    type: "physical",
+                    fstype: "",
+                    original: {
+                        partition_table_type: null,
+                        available_size: (2.5 * 1024 * 1024) + (8 * 1024 * 1024),
+                        block_size: 1024
+                    }
+                };
+                node.architecture = "ppc64el/generic";
+                spyOn($scope, "isAllStorageDisabled").and.returnValue(false);
+                $scope.isSuperUser = function() { return true; };
+                expect($scope.canAddPartition(disk)).toBe(false);
+            });
+
         it("returns false if not super user", function() {
             var controller = makeController();
             var disk = {
@@ -2350,6 +2370,37 @@
             expect($scope.getAddPartitionName(disk)).toBe(name + "-part3");
         });
 
+        it("returns disk.name with -part2 for ppc64el", function() {
+            node.architecture = "ppc64el/generic";
+            var controller = makeController();
+            var name = makeName("sda");
+            var disk = {
+                name: name,
+                original: {
+                    is_boot: true,
+                    partition_table_type: "gpt"
+                }
+            };
+
+            expect($scope.getAddPartitionName(disk)).toBe(name + "-part2");
+        });
+
+        it("returns disk.name with -part4 for ppc64el", function() {
+            node.architecture = "ppc64el/generic";
+            var controller = makeController();
+            var name = makeName("sda");
+            var disk = {
+                name: name,
+                original: {
+                    is_boot: true,
+                    partition_table_type: "gpt",
+                    partitions: [{}, {}]
+                }
+            };
+
+            expect($scope.getAddPartitionName(disk)).toBe(name + "-part4");
+        });
+
         it("returns disk.name with -part3 for MBR", function() {
             var controller = makeController();
             var name = makeName("sda");

=== modified file 'src/maasserver/storage_layouts.py'
--- src/maasserver/storage_layouts.py	2015-12-01 18:12:59 +0000
+++ src/maasserver/storage_layouts.py	2016-02-24 16:12:28 +0000
@@ -169,7 +169,12 @@
         from maasserver.models.partitiontable import PartitionTable
         boot_partition_table = PartitionTable.objects.create(
             block_device=self.boot_disk)
-        if boot_partition_table.table_type == PARTITION_TABLE_TYPE.GPT:
+        bios_boot_method = self.node.get_bios_boot_method()
+        node_arch, _ = self.node.split_arch()
+        if (boot_partition_table.table_type == PARTITION_TABLE_TYPE.GPT and
+                bios_boot_method == "uefi" and node_arch != "ppc64el"):
+            # Add EFI partition only if booting UEFI and not a ppc64el
+            # architecture.
             efi_partition = boot_partition_table.add_partition(
                 size=EFI_PARTITION_SIZE, bootable=True)
             Filesystem.objects.create(

=== modified file 'src/maasserver/tests/test_preseed_storage.py'
--- src/maasserver/tests/test_preseed_storage.py	2016-02-23 12:13:09 +0000
+++ src/maasserver/tests/test_preseed_storage.py	2016-02-24 16:12:28 +0000
@@ -20,7 +20,10 @@
     RAID,
     VolumeGroup,
 )
-from maasserver.models.partitiontable import PARTITION_TABLE_EXTRA_SPACE
+from maasserver.models.partitiontable import (
+    PARTITION_TABLE_EXTRA_SPACE,
+    PREP_PARTITION_SIZE,
+)
 from maasserver.preseed_storage import compose_curtin_storage_config
 from maasserver.testing.factory import factory
 from maasserver.testing.testcase import MAASServerTestCase
@@ -770,6 +773,217 @@
         self.assertStorageConfig(self.STORAGE_CONFIG, config)
 
 
+class TestSimplePower8Layout(MAASServerTestCase, AssertStorageConfigMixin):
+
+    STORAGE_CONFIG = dedent("""\
+        config:
+          - id: sda
+            name: sda
+            type: disk
+            wipe: superblock
+            ptable: gpt
+            model: QEMU HARDDISK
+            serial: QM00001
+          - id: sda-part1
+            name: sda-part1
+            type: partition
+            number: 1
+            offset: 4194304B
+            size: 8388608B
+            device: sda
+            wipe: zero
+            flag: prep
+            grub_device: True
+          - id: sda-part2
+            name: sda-part2
+            type: partition
+            number: 2
+            uuid: f74ff260-2a5b-4a36-b1b8-37f746b946bf
+            size: 8573157376B
+            wipe: superblock
+            device: sda
+          - id: sda-part2_format
+            type: format
+            fstype: ext4
+            label: root
+            uuid: 90a69b22-e281-4c5b-8df9-b09514f27ba1
+            volume: sda-part2
+          - id: sda-part2_mount
+            type: mount
+            path: /
+            device: sda-part2_format
+        """)
+
+    def test__renders_expected_output(self):
+        node = factory.make_Node(
+            status=NODE_STATUS.ALLOCATED, architecture="ppc64el/generic",
+            bios_boot_method="uefi", with_boot_disk=False)
+        boot_disk = factory.make_PhysicalBlockDevice(
+            node=node, size=8 * 1024 ** 3, name="sda",
+            model="QEMU HARDDISK", serial="QM00001")  # 8 GiB
+        partition_table = factory.make_PartitionTable(
+            table_type=PARTITION_TABLE_TYPE.GPT, block_device=boot_disk)
+        root_partition = factory.make_Partition(
+            partition_table=partition_table,
+            uuid="f74ff260-2a5b-4a36-b1b8-37f746b946bf",
+            size=(
+                (8 * 1024 ** 3) - PARTITION_TABLE_EXTRA_SPACE -
+                PREP_PARTITION_SIZE),
+            bootable=False)
+        factory.make_Filesystem(
+            partition=root_partition, fstype=FILESYSTEM_TYPE.EXT4,
+            uuid="90a69b22-e281-4c5b-8df9-b09514f27ba1", label="root",
+            mount_point="/", mount_options=None)
+        node._create_acquired_filesystems()
+        config = compose_curtin_storage_config(node)
+        self.assertStorageConfig(self.STORAGE_CONFIG, config)
+
+
+class TestPower8ExtraSpaceLayout(
+        MAASServerTestCase, AssertStorageConfigMixin):
+
+    STORAGE_CONFIG = dedent("""\
+        config:
+          - id: sda
+            name: sda
+            type: disk
+            wipe: superblock
+            ptable: gpt
+            model: QEMU HARDDISK
+            serial: QM00001
+          - id: sda-part1
+            name: sda-part1
+            type: partition
+            number: 1
+            offset: 4194304B
+            size: 8388608B
+            device: sda
+            wipe: zero
+            flag: prep
+            grub_device: True
+          - id: sda-part2
+            name: sda-part2
+            type: partition
+            number: 2
+            uuid: f74ff260-2a5b-4a36-b1b8-37f746b946bf
+            size: 7507804160B
+            wipe: superblock
+            device: sda
+          - id: sda-part2_format
+            type: format
+            fstype: ext4
+            label: root
+            uuid: 90a69b22-e281-4c5b-8df9-b09514f27ba1
+            volume: sda-part2
+          - id: sda-part2_mount
+            type: mount
+            path: /
+            device: sda-part2_format
+        """)
+
+    def test__renders_expected_output(self):
+        node = factory.make_Node(
+            status=NODE_STATUS.ALLOCATED, architecture="ppc64el/generic",
+            bios_boot_method="uefi", with_boot_disk=False)
+        boot_disk = factory.make_PhysicalBlockDevice(
+            node=node, size=8 * 1024 ** 3, name="sda",
+            model="QEMU HARDDISK", serial="QM00001")  # 8 GiB
+        partition_table = factory.make_PartitionTable(
+            table_type=PARTITION_TABLE_TYPE.GPT, block_device=boot_disk)
+        root_partition = factory.make_Partition(
+            partition_table=partition_table,
+            uuid="f74ff260-2a5b-4a36-b1b8-37f746b946bf",
+            size=(7 * 1024 ** 3) - PARTITION_TABLE_EXTRA_SPACE,
+            bootable=False)
+        factory.make_Filesystem(
+            partition=root_partition, fstype=FILESYSTEM_TYPE.EXT4,
+            uuid="90a69b22-e281-4c5b-8df9-b09514f27ba1", label="root",
+            mount_point="/", mount_options=None)
+        node._create_acquired_filesystems()
+        config = compose_curtin_storage_config(node)
+        self.assertStorageConfig(self.STORAGE_CONFIG, config)
+
+
+class TestPower8NoPartitionTableLayout(
+        MAASServerTestCase, AssertStorageConfigMixin):
+
+    STORAGE_CONFIG = dedent("""\
+        config:
+          - id: sda
+            name: sda
+            type: disk
+            wipe: superblock
+            ptable: gpt
+            model: QEMU HARDDISK
+            serial: QM00001
+          - id: sdb
+            name: sdb
+            type: disk
+            wipe: superblock
+            ptable: gpt
+            model: QEMU HARDDISK
+            serial: QM00002
+          - id: sdb-part1
+            name: sdb-part1
+            type: partition
+            number: 1
+            offset: 4194304B
+            size: 8388608B
+            device: sdb
+            wipe: zero
+            flag: prep
+            grub_device: True
+          - id: sda-part1
+            name: sda-part1
+            type: partition
+            number: 1
+            uuid: f74ff260-2a5b-4a36-b1b8-37f746b946bf
+            offset: 4194304B
+            size: 8573157376B
+            wipe: superblock
+            device: sda
+          - id: sda-part1_format
+            type: format
+            fstype: ext4
+            label: root
+            uuid: 90a69b22-e281-4c5b-8df9-b09514f27ba1
+            volume: sda-part1
+          - id: sda-part1_mount
+            type: mount
+            path: /
+            device: sda-part1_format
+        """)
+
+    def test__renders_expected_output(self):
+        node = factory.make_Node(
+            status=NODE_STATUS.ALLOCATED, architecture="ppc64el/generic",
+            bios_boot_method="uefi", with_boot_disk=False)
+        root_disk = factory.make_PhysicalBlockDevice(
+            node=node, size=8 * 1024 ** 3, name="sda",
+            model="QEMU HARDDISK", serial="QM00001")  # 8 GiB
+        partition_table = factory.make_PartitionTable(
+            table_type=PARTITION_TABLE_TYPE.GPT, block_device=root_disk)
+        boot_disk = factory.make_PhysicalBlockDevice(
+            node=node, size=8 * 1024 ** 3, name="sdb",
+            model="QEMU HARDDISK", serial="QM00002")  # 8 GiB
+        node.boot_disk = boot_disk
+        node.save()
+        root_partition = factory.make_Partition(
+            partition_table=partition_table,
+            uuid="f74ff260-2a5b-4a36-b1b8-37f746b946bf",
+            size=(
+                (8 * 1024 ** 3) - PARTITION_TABLE_EXTRA_SPACE -
+                PREP_PARTITION_SIZE),
+            bootable=False)
+        factory.make_Filesystem(
+            partition=root_partition, fstype=FILESYSTEM_TYPE.EXT4,
+            uuid="90a69b22-e281-4c5b-8df9-b09514f27ba1", label="root",
+            mount_point="/", mount_options=None)
+        node._create_acquired_filesystems()
+        config = compose_curtin_storage_config(node)
+        self.assertStorageConfig(self.STORAGE_CONFIG, config)
+
+
 def shuffled(things):
     things = list(things)
     random.shuffle(things)

=== modified file 'src/maasserver/tests/test_storage_layouts.py'
--- src/maasserver/tests/test_storage_layouts.py	2016-01-21 17:03:28 +0000
+++ src/maasserver/tests/test_storage_layouts.py	2016-02-24 16:12:28 +0000
@@ -20,7 +20,10 @@
     MAX_PARTITION_SIZE_FOR_MBR,
     PARTITION_ALIGNMENT_SIZE,
 )
-from maasserver.models.partitiontable import PARTITION_TABLE_EXTRA_SPACE
+from maasserver.models.partitiontable import (
+    PARTITION_TABLE_EXTRA_SPACE,
+    PREP_PARTITION_SIZE,
+)
 from maasserver.storage_layouts import (
     BcacheStorageLayout,
     BcacheStorageLayoutBase,
@@ -54,6 +57,20 @@
     return factory.make_Node(*args, **kwargs)
 
 
+def make_ppc64el_Node_with_powernv_boot_method(*args, **kwargs):
+    kwargs['bios_boot_method'] = "powernv"
+    kwargs['with_boot_disk'] = False
+    kwargs['architecture'] = "ppc64el/generic"
+    return factory.make_Node(*args, **kwargs)
+
+
+def make_ppc64el_Node_with_uefi_boot_method(*args, **kwargs):
+    kwargs['bios_boot_method'] = "powerkvm"
+    kwargs['with_boot_disk'] = False
+    kwargs['architecture'] = "ppc64el/generic"
+    return factory.make_Node(*args, **kwargs)
+
+
 class TestFormHelpers(MAASServerTestCase):
 
     def test_get_storage_layout_choices(self):
@@ -505,6 +522,66 @@
                 mount_point="/",
                 ))
 
+    def test__creates_layout_for_powernv(self):
+        node = make_ppc64el_Node_with_powernv_boot_method()
+        boot_disk = factory.make_PhysicalBlockDevice(
+            node=node, size=LARGE_BLOCK_DEVICE)
+        layout = FlatStorageLayout(node)
+        layout.configure()
+
+        # Validate partition table.
+        partition_table = boot_disk.get_partitiontable()
+        self.assertEqual(PARTITION_TABLE_TYPE.GPT, partition_table.table_type)
+
+        # Validate root partition.
+        partitions = partition_table.partitions.order_by('id').all()
+        root_partition = partitions[0]
+        self.assertIsNotNone(root_partition)
+        self.assertEqual(
+            round_size_to_nearest_block(
+                boot_disk.size - PARTITION_TABLE_EXTRA_SPACE -
+                PREP_PARTITION_SIZE,
+                PARTITION_ALIGNMENT_SIZE,
+                False),
+            root_partition.size)
+        self.assertThat(
+            root_partition.get_effective_filesystem(),
+            MatchesStructure.byEquality(
+                fstype=FILESYSTEM_TYPE.EXT4,
+                label="root",
+                mount_point="/",
+                ))
+
+    def test__creates_layout_for_powerkvm(self):
+        node = make_ppc64el_Node_with_uefi_boot_method()
+        boot_disk = factory.make_PhysicalBlockDevice(
+            node=node, size=LARGE_BLOCK_DEVICE)
+        layout = FlatStorageLayout(node)
+        layout.configure()
+
+        # Validate partition table.
+        partition_table = boot_disk.get_partitiontable()
+        self.assertEqual(PARTITION_TABLE_TYPE.GPT, partition_table.table_type)
+
+        # Validate root partition.
+        partitions = partition_table.partitions.order_by('id').all()
+        root_partition = partitions[0]
+        self.assertIsNotNone(root_partition)
+        self.assertEqual(
+            round_size_to_nearest_block(
+                boot_disk.size - PARTITION_TABLE_EXTRA_SPACE -
+                PREP_PARTITION_SIZE,
+                PARTITION_ALIGNMENT_SIZE,
+                False),
+            root_partition.size)
+        self.assertThat(
+            root_partition.get_effective_filesystem(),
+            MatchesStructure.byEquality(
+                fstype=FILESYSTEM_TYPE.EXT4,
+                label="root",
+                mount_point="/",
+                ))
+
     def test__creates_layout_with_uefi_defaults(self):
         node = make_Node_with_uefi_boot_method()
         boot_disk = factory.make_PhysicalBlockDevice(

=== modified file 'src/provisioningserver/boot/powerkvm.py'
--- src/provisioningserver/boot/powerkvm.py	2015-12-01 18:12:59 +0000
+++ src/provisioningserver/boot/powerkvm.py	2016-02-24 16:12:28 +0000
@@ -33,7 +33,7 @@
 class PowerKVMBootMethod(BootMethod):
 
     name = "powerkvm"
-    bios_boot_method = "pxe"
+    bios_boot_method = "powerkvm"
     template_subdir = None
     bootloader_path = "bootppc64.bin"
     bootloader_arches = ['ppc64el']

=== modified file 'src/provisioningserver/boot/powernv.py'
--- src/provisioningserver/boot/powernv.py	2015-12-01 18:12:59 +0000
+++ src/provisioningserver/boot/powernv.py	2016-02-24 16:12:28 +0000
@@ -61,7 +61,7 @@
 class PowerNVBootMethod(BootMethod):
 
     name = "powernv"
-    bios_boot_method = "pxe"
+    bios_boot_method = "powernv"
     template_subdir = "pxe"
     bootloader_path = "pxelinux.0"
     arch_octet = "00:0E"