Unable to live-migrate : Disk of instance is too large

Are you using OpenStackClient for calling the live-migration action ? Which exact CLI call do you do ?

I just used 'live migration' button of dashboard with 'disk over commit' and 'block migration' options.@Sylvain Bauza (sylvain-bauza)

Fix proposed to branch: master
Review: https://review.openstack.org/524681

Reviewed: https://review.openstack.org/524681
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=e097c001c8e11110efe8879da57264fcb7bdfdf2
Submitter: Zuul
Branch: master

commit e097c001c8e11110efe8879da57264fcb7bdfdf2
Author: Matthew Booth <email address hidden>
Date: Fri Dec 1 16:39:49 2017 +0000

    Fix disk size during live migration with disk over-commit

    Prior to microversion 2.25, the migration api supported a
    'disk_over_commit' parameter, which indicated that we should do disk
    size check on the destination host taking into account disk
    over-commit. Versions since 2.25 no longer have this parameter, but we
    continue to support the older microversion.

    This disk size check was broken when disk over commit was in use on
    the host. In LibvirtDriver._assert_dest_node_has_enough_disk() we
    correctly calculate the required space using allocated disk size
    rather than virtual disk size when doing an over-committed check.
    However, we were checking against 'disk_available_least' as reported
    by the destination host. This value is: the amount of disk space which
    the host would have if all instances fully allocated their storage. On
    an over-committed host it will therefore be negative, despite there
    actually being space on the destination host.

    The check we actually want to do for disk over commit is: does the
    target host have enough free space to take the allocated size of this
    instance's disks? We leave checking over-allocation ratios to the
    scheduler. Note that if we use disk_available_least here and the
    destination host is over-allocated, this will always fail because free
    space will be negative, even though we're explicitly ok with that.

    Using disk_available_least would make sense for the non-overcommit
    case, where the test would be: would the target host have enough free
    space for this instance if the instance fully allocated all its
    storage, and everything else on the target also fully allocated its
    storage? As noted, we no longer actually run that test, though.

    We fix the issue for legacy microversions by fixing the destination
    host's reported disk space according to the given disk_over_commit
    parameter.

    Change-Id: I8a705114d47384fcd00955d4a4f204072fed57c2
    Resolves-bug: #1708572

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/530743

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/530744

How to get this patch in Newton?

When I launched an instance using a volume stored on IP-SAN, 'Disk of instance is too large' exception was raised, and the amount of disk space available was also a negative number. It seems that all allocated IP-SAN volumes used to create instances are taken into account when computing the free space on the target host.

The fix in https://review.openstack.org/524681 does an over-committed check, but I'm not sure it would help if we launch an instance from a Cinder storage backend.

Change abandoned by Matt Riedemann (<email address hidden>) on branch: stable/pike
Review: https://review.openstack.org/530743
Reason: I'm just going to abandon. If anyone cares, restore and sort this out.

Change abandoned by Matt Riedemann (<email address hidden>) on branch: stable/ocata
Review: https://review.openstack.org/530744
Reason: I'm just going to abandon. If anyone cares, restore and sort this out.

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/631372

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/631376

Change abandoned by Zhang Hua (<email address hidden>) on branch: stable/pike
Review: https://review.openstack.org/631372

Reviewed: https://review.openstack.org/530743
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=1a4debb3dd2726299bbee0c07b7fc052ac72555b
Submitter: Zuul
Branch: stable/pike

commit 1a4debb3dd2726299bbee0c07b7fc052ac72555b
Author: Matthew Booth <email address hidden>
Date: Fri Dec 1 16:39:49 2017 +0000

    Fix disk size during live migration with disk over-commit

    Prior to microversion 2.25, the migration api supported a
    'disk_over_commit' parameter, which indicated that we should do disk
    size check on the destination host taking into account disk
    over-commit. Versions since 2.25 no longer have this parameter, but we
    continue to support the older microversion.

    This disk size check was broken when disk over commit was in use on
    the host. In LibvirtDriver._assert_dest_node_has_enough_disk() we
    correctly calculate the required space using allocated disk size
    rather than virtual disk size when doing an over-committed check.
    However, we were checking against 'disk_available_least' as reported
    by the destination host. This value is: the amount of disk space which
    the host would have if all instances fully allocated their storage. On
    an over-committed host it will therefore be negative, despite there
    actually being space on the destination host.

    The check we actually want to do for disk over commit is: does the
    target host have enough free space to take the allocated size of this
    instance's disks? We leave checking over-allocation ratios to the
    scheduler. Note that if we use disk_available_least here and the
    destination host is over-allocated, this will always fail because free
    space will be negative, even though we're explicitly ok with that.

    Using disk_available_least would make sense for the non-overcommit
    case, where the test would be: would the target host have enough free
    space for this instance if the instance fully allocated all its
    storage, and everything else on the target also fully allocated its
    storage? As noted, we no longer actually run that test, though.

    We fix the issue for legacy microversions by fixing the destination
    host's reported disk space according to the given disk_over_commit
    parameter.

    Change-Id: I8a705114d47384fcd00955d4a4f204072fed57c2
    Resolves-bug: #1708572
    (cherry picked from commit e097c001c8e11110efe8879da57264fcb7bdfdf2)

Reviewed: https://review.openstack.org/530744
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=e2d2c3e8b14a5461c95014be3accb1e40f3d1a0b
Submitter: Zuul
Branch: stable/ocata

commit e2d2c3e8b14a5461c95014be3accb1e40f3d1a0b
Author: Matthew Booth <email address hidden>
Date: Fri Dec 1 16:39:49 2017 +0000

    Fix disk size during live migration with disk over-commit

    Prior to microversion 2.25, the migration api supported a
    'disk_over_commit' parameter, which indicated that we should do disk
    size check on the destination host taking into account disk
    over-commit. Versions since 2.25 no longer have this parameter, but we
    continue to support the older microversion.

    This disk size check was broken when disk over commit was in use on
    the host. In LibvirtDriver._assert_dest_node_has_enough_disk() we
    correctly calculate the required space using allocated disk size
    rather than virtual disk size when doing an over-committed check.
    However, we were checking against 'disk_available_least' as reported
    by the destination host. This value is: the amount of disk space which
    the host would have if all instances fully allocated their storage. On
    an over-committed host it will therefore be negative, despite there
    actually being space on the destination host.

    The check we actually want to do for disk over commit is: does the
    target host have enough free space to take the allocated size of this
    instance's disks? We leave checking over-allocation ratios to the
    scheduler. Note that if we use disk_available_least here and the
    destination host is over-allocated, this will always fail because free
    space will be negative, even though we're explicitly ok with that.

    Using disk_available_least would make sense for the non-overcommit
    case, where the test would be: would the target host have enough free
    space for this instance if the instance fully allocated all its
    storage, and everything else on the target also fully allocated its
    storage? As noted, we no longer actually run that test, though.

    We fix the issue for legacy microversions by fixing the destination
    host's reported disk space according to the given disk_over_commit
    parameter.

    Change-Id: I8a705114d47384fcd00955d4a4f204072fed57c2
    Resolves-bug: #1708572
    (cherry picked from commit e097c001c8e11110efe8879da57264fcb7bdfdf2)

Hello lanrui280, or anyone else affected,

Accepted nova into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/nova/2:13.1.4-0ubuntu4.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello lanrui280, or anyone else affected,

Accepted nova into mitaka-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:mitaka-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-mitaka-needed to verification-mitaka-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-mitaka-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Tagged xenial and mitaka as verified in line with https://bugs.launchpad.net/nova/+bug/1744079.

The verification of the Stable Release Update for nova has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package nova - 2:13.1.4-0ubuntu4.4

---------------
nova (2:13.1.4-0ubuntu4.4) xenial; urgency=medium

  * Refix disk size during live migration with disk over-commit
    - (LP: #1708572) and (LP: #1744079)
    - d/p/0001-Fix-disk-size-during-live-migration-with-disk-over-c.patch
    - d/p/0002-Refix-disk-size-during-live-migration-with-disk-over.patch

 -- Zhang Hua <email address hidden> Tue, 02 Apr 2019 18:48:16 +0800

The verification of the Stable Release Update for nova has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package nova - 2:13.1.4-0ubuntu4.4~cloud0
---------------

 nova (2:13.1.4-0ubuntu4.4~cloud0) trusty-mitaka; urgency=medium
 .
   * New update for the Ubuntu Cloud Archive.
 .
 nova (2:13.1.4-0ubuntu4.4) xenial; urgency=medium
 .
   * Refix disk size during live migration with disk over-commit
     - (LP: #1708572) and (LP: #1744079)
     - d/p/0001-Fix-disk-size-during-live-migration-with-disk-over-c.patch
     - d/p/0002-Refix-disk-size-during-live-migration-with-disk-over.patch