[2.4,b1] images never finish syncing to rack controller

I can confirm this is the case, although it doesn't happen always. In our case this is what happens:

1. Install & configure MAAS
2. Add arm64 or ppc64 boot resources
3. Add centos boot resources
4. MAAS imports all images in the region
5. MAAS imports /some/ images in the rack, but the last boot resources added never get into the rack
6. The images page reports "that its waiting for rack controllers to finish syncing" for the centos boot resource.

We're still seeing this on

maas_2.4.0~beta2-6783-g2739ac4-0ubuntu1~18.04.1~20180402~ubuntu18.04.1

escalated to critical as it blocks all 2.4 deployments

Is this still the case where it stays over 30+ mins without the rack controllers syncing?

So I just did a quick test in a container:

1. sudo apt-get install maas
2. sudo maas init
3. waiting for images to sync while tailing logs
4. Saw this:

==> /var/log/maas/maas.log <==
Apr 3 18:26:17 maas-bionic maas.bootresources: [info] Finished importing of boot images from 1 source(s).

==> /var/log/maas/regiond.log <==
2018-04-03 18:26:17 maasserver.clusterrpc.boot_images: [info] Rack controller (fbcqp7) did not import boot resources; it is not connected to the region at this time.

5. Went to the UI to check region/rack connection, showed as everything connected.
6. After a few mins, the rack controller disconnected, the UI 'Controllers' page went blank.
7. While 6 happened, I saw this:

2018-04-03 18:26:28 regiond: [info] 127.0.0.1 GET /MAAS/rpc/ HTTP/1.1 --> 200 OK (referrer: -; agent: provisioningserver.rpc.clusterservice.ClusterClientService)
2018-04-03 18:26:59 regiond: [info] 127.0.0.1 GET /MAAS/rpc/ HTTP/1.1 --> 200 OK (referrer: -; agent: provisioningserver.rpc.clusterservice.ClusterClientService)
2018-04-03 18:27:29 regiond: [info] 127.0.0.1 GET /MAAS/rpc/ HTTP/1.1 --> 200 OK (referrer: -; agent: provisioningserver.rpc.clusterservice.ClusterClientService)

==> /var/log/maas/maas.log <==
Apr 3 18:27:59 maas-bionic maas.import-images: [info] Starting rack boot image import
Apr 3 18:27:59 maas-bionic maas.import-images: [info] Downloading image descriptions from http://localhost:5240/MAAS/images-stream/streams/v1/index.json

Yes, the latest run is still sitting out of sync after 3+ hours.

On Tue, Apr 3, 2018 at 1:25 PM, Andres Rodriguez
<email address hidden> wrote:
> Is this still the case where it stays over 30+ mins without the rack
> controllers syncing?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1754493
>
> Title:
> [2.4,b1] images never finish syncing to rack controller
>
> Status in MAAS:
> New
>
> Bug description:
> On a single node region/rack controller setup, images never finish
> syncing to the rack controller. The rack controller has stayed in out-
> of-sync status for over 30 minutes.
>
> In the UI, I only have 16.04/amd64 selected in the UI, but those don't
> even show up on the rack-controller list-boot-images output; instead,
> i386 and ppc64el show up.
>
> http://paste.ubuntu.com/p/gSZBKP2Qb5/
>
> This is with 2.4.0~alpha1-6573-g12ee2331b-0ubuntu1
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/maas/+bug/1754493/+subscriptions

Sorry, this isn't "Fix Released" - we're still seeing the exact same behavior we reported.

Marking this bug report as fix released again, as the issue is a different, non-related issue to this one, even though it displays the same symptoms.

I've filed this new one:

https://bugs.launchpad.net/maas/+bug/1760958

I'm on MAAS version: 2.5.3 (7533-g65952b418-0ubuntu1~18.04.1) and uploaded a 5G Windows image which seemed to finish, but it's not sync'ing with rackd.

* Windows 2016 amd64 5.0 GB Waiting for rack controller(s) to sync

It's been sitting like this for over 24 hours.

Same issue I am seeing , trying to upload the windows image and syncing is not happening .