Author: Scott Moser
Revision Date: 2018-06-21 14:40:35 UTC

simplify a bit and add a test.

Author: Scott Moser
Revision Date: 2018-06-08 17:29:50 UTC

mfdiff has moved to git.

mfdiff has moved its revision control to git.
Its new upstream home is:
  https://code.launchpad.net/~cloud-images-release-managers/cloud-images/+git/mfdiff

To clone, either:
   git clone https://git.launchpad.net/~cloud-images-release-managers/cloud-images/+git/mfdiff
or
   git clone ssh://git.launchpad.net/~cloud-images-release-managers/cloud-images/+git/mfdiff

Author: Scott Moser
Revision Date: 2018-03-29 15:57:10 UTC

capital unreleased

Author: Scott Moser
Revision Date: 2018-02-13 18:20:48 UTC

much simpler installation of required packages.

Just let apt figure it out.

Author: Scott Moser
Revision Date: 2018-02-06 14:40:50 UTC

Do not install cloud-initramfs-dyn-netconf in releases newer than xenial.

The cloud-initramfs-dyn-netconf was being added to the maas image.
It was only absolutely necessary in <xenial, but is left as present
in xenial as images still use the /etc/network/interfaces link.
See change="symlink ENI for cloud-initramfs-dyn-netconf".

It is not needed in or used without ifupdown, and ifupdown is not present
in artful or bionic.

Author: Scott Moser
Revision Date: 2018-02-05 20:49:41 UTC

use the already defined dsst_squash filename

Author: Scott Moser
Revision Date: 2017-02-10 20:36:43 UTC

releasing package cloud-utils version 0.30-0ubuntu2

Author: Scott Moser
Revision Date: 2018-01-04 18:45:50 UTC

mount-image-callback: run blockdev and udevadm settle on nbd devices.

When we connect nbd devices, ensure that the kernel re-reads the partition
table information. This ensures that the partition devices (nbd0p1, nbd0p2)
are correct when used later.

Also required then is a udevadm settle.

On xenial, the blockdev is needed, on newer releases some udev magic or
systemd handles it.

Author: Scott Moser
Revision Date: 2017-12-20 17:33:03 UTC

README: Mention move of revision control to git.

curtin development has moved its revision control to git.
It is available at
  https://code.launchpad.net/curtin

Clone with
  git clone https://git.launchpad.net/curtin
or
  git clone git+ssh://git.launchpad.net/curtin

For more information see
  http://curtin.readthedocs.io/en/latest/topics/development.html

Author: Scott Moser
Revision Date: 2017-09-12 14:16:06 UTC

move the bus or index to the -drive...even though it doesnt seem right

Author: Scott Moser
Revision Date: 2017-04-27 17:50:14 UTC

early changes to get keystone v3 auth.

Author: Scott Moser
Revision Date: 2017-04-30 22:08:51 UTC

add releases. support lxd/qemu dual openstack.

Author: Scott Moser
Revision Date: 2017-04-27 17:50:14 UTC

early changes to get keystone v3 auth.

Author: Scott Moser
Revision Date: 2017-04-18 17:13:02 UTC

update to function for for 2.2 maas.

Author: Scott Moser
Revision Date: 2017-03-06 17:22:07 UTC

releasing package software-properties version 0.96.24.11

Author: Scott Moser
Revision Date: 2017-03-06 16:57:32 UTC

debian/tests/run-tests: use gpg rather than gpg1, and show errors

Change 'gpg1 -k' to 'gpg --list-keys'.
gpg1 is not the default gpg for newer ubuntu.

Capturing of output was causing very silent failure. Now, show output and
exit code on failure, and a message indicating why it was used.

Author: Scott Moser
Revision Date: 2017-02-22 16:29:33 UTC

debian/rules: record debian package version in installed package.
(LP: #1666986)

Author: Scott Moser
Revision Date: 2017-02-22 16:27:44 UTC

debian/rules: record debian package version in installed package.
(LP: #1666986)

Author: Scott Moser
Revision Date: 2017-02-10 20:36:43 UTC

releasing package cloud-utils version 0.30-0ubuntu2

Author: Scott Moser
Revision Date: 2017-01-18 17:39:34 UTC

* debian/new-upstream-snapshot: change to not use bzr merge-upstream.
* New upstream snapshot.
  - pep8: fix pep8 errors found with 'make pep8' on zesty.
  - Workaround failures caused by gpg2 daemons left running in chroot.
    (LP: #1645680)
  - Install u-boot-tools when running on a system with u-boot. (LP: #1640519)
  - block: fix partition kname for raid devices (LP: #1641661)
  - Fix up tox errors that slipped in and new pycodestyle 2.1.0 complaints.
  - vmtests: adjust vmtest image sync metadata filenames
  - vmtests: Add centos support
  - Disable WilyTestRaid5Bcache vmtest
  - tools/xkvm: fix --netdev=<bridge>
  - bytes2human: fix for values larger than 32 bit int on 32 bit python2.

Author: Scott Moser
Revision Date: 2017-01-18 16:01:35 UTC

* debian/new-upstream-snapshot: change to not use bzr merge-upstream.
* New upstream snapshot.
  - revert: Test Workaround: skip XenialTestNvme for a short time.
  - Test Workaround: skip XenialTestNvme for a short time.
  - pep8: fix pep8 errors found with 'make pep8' on zesty.
  - Workaround failures caused by gpg2 daemons left running in chroot.
    (LP: #1645680)
  - Install u-boot-tools when running on a system with u-boot. (LP: #1640519)
  - block: fix partition kname for raid devices (LP: #1641661)
  - Fix up tox errors that slipped in and new pycodestyle 2.1.0 complaints.
  - vmtests: adjust vmtest image sync metadata filenames
  - vmtests: Add centos support
  - Disable WilyTestRaid5Bcache vmtest
  - tools/xkvm: fix --netdev=<bridge>
  - bytes2human: fix for values larger than 32 bit int on 32 bit python2.

Author: Scott Moser
Revision Date: 2016-12-22 19:09:24 UTC

Add and use local keyring for verification of apt data.

A transition is being made from signing with the dsa1024 'Archive'
signing key below to the newer rsa4096 Archive key.
In zesty, the package installed keyring no longer contains the
older keys.

Without this, if running on zesty, then mfdiff will fail to download
changelogs (using apt.cache), as it will not trust the data.

The solution here is to check in the gpg keyring that was provided
by the Ubuntu 16.04 (xenial) package ubuntu-keyring, and to use that
for verification.

$ gpg keyring.gpg

pub dsa1024 2004-09-12 [SC]
      630239CC130E1A7FD81A27B140976EAF437D05B5
uid Ubuntu Archive Automatic Signing Key <ftpmaster@ubuntu.com>
sub elg2048 2004-09-12 [E]
pub dsa1024 2004-12-30 [SC]
      C5986B4F1257FFA86632CBA746181433FBB75451
uid Ubuntu CD Image Automatic Signing Key <cdimage@ubuntu.com>
pub rsa4096 2012-05-11 [SC]
      790BC7277767219C42C86F933B4FE6ACC0B21F32
uid Ubuntu Archive Automatic Signing Key (2012) <ftpmaster@ubuntu.com>
pub rsa4096 2012-05-11 [SC]
      843938DF228D22F7B3742BC0D94AA3F0EFE21092
uid Ubuntu CD Image Automatic Signing Key (2012) <cdimage@ubuntu.com>

Author: Scott Moser
Revision Date: 2016-12-01 22:15:32 UTC

releasing package curtin version 0.1.0~bzr434-0ubuntu1

Author: Scott Moser
Revision Date: 2016-11-28 15:15:22 UTC

Attempt diagnosis of c-i hang by running nose with -v.

C-i is hanging on tox runs. Once today [1] and once
https://jenkins.ubuntu.com/server/view/
    Simplestreams/job/simplestreams-tot/13/nodes=amd64/console

and once last week on ppc64
    Simplestreams/job/simplestreams-tot/12/nodes=ppc64el/console

Trying to figure out what test is hanging.

Author: LaMont Jones
Revision Date: 2016-10-04 20:52:53 UTC

Add IPV6 support to cloud-initramfs-dyn-netconf. (LP: #1621615)

Author: Scott Moser
Revision Date: 2016-10-26 20:22:56 UTC

add ec2-disk2ebs

this is mostly copied from ec2-image2ebs.
The 2 differences are:
 a.) it does not do any of the modifications to the image.
     It just does a 'dd'.
 b.) it assumes you're on the instance that you're working with.
     ec2-image2ebs ssh's to the other system. This just executes
     locally.

Author: Scott Moser
Revision Date: 2016-10-24 18:10:27 UTC

fix state of directory, that was just wrong.

Author: Scott Moser
Revision Date: 2016-10-03 16:09:25 UTC

modprobe by filesystem name rather than module name

previously we modprobed 'overlay' and assumed/hoped that would
give us 'overlay' filesystem support. Instead, the kernel modules
provide a fs-overlay alias that will allow us to directly ask for
the module that provides the 'overlay' filesystem.

Author: Scott Moser
Revision Date: 2016-10-03 16:06:12 UTC

load the module by fs-<filesystem>

this is a better path to loading a kernel module since
we're askign for a specific filesystem support, this will load
that filesystem rather than assuming that 'overlay' module
provides the 'overlay' filesystem.

Author: Scott Moser
Revision Date: 2016-09-29 18:55:32 UTC

releasing package curtin version 0.1.0~bzr425-0ubuntu1

Author: Scott Moser
Revision Date: 2016-09-22 20:48:25 UTC

adding code i was playing with.

The intent of this would be for curtin to do something like the main
here. Instead of invoking the shell reboot function, it would
search for a parent that it liked (such as one named cloud-init).
If there was no good looking parent, then it would use itself.

So basically:
 - select a process P to wait until it is gone.
 - fork
 - exit from main process
 - second process polls for process P to have left the building
 - second process (conditionally) invokes the reboot.

Author: Scott Moser
Revision Date: 2016-08-05 15:37:33 UTC

re-add atomic_helper

Author: Scott Moser
Revision Date: 2016-08-01 00:43:57 UTC

boots with localds now. cleanups still needed.

Author: Scott Moser
Revision Date: 2016-07-26 13:43:17 UTC

releasing package smello version 0.4~ppa1

Author: Scott Moser
Revision Date: 2016-07-25 18:06:53 UTC

add a comment

Author: Scott Moser
Revision Date: 2016-07-25 15:22:39 UTC

add test case covering not all nics have 'gateways' entry.

also for brevity import convert_smartos_network_data as convert_net.

Author: Scott Moser
Revision Date: 2016-07-19 19:04:04 UTC

releasing package curtin version 0.1.0~bzr399-0ubuntu1~14.04.1

Author: Scott Moser
Revision Date: 2016-07-19 15:12:39 UTC

whitespace

Author: Scott Moser
Revision Date: 2016-07-15 14:09:44 UTC

bin/encrypt-pass: add tool for converting plaintext to encrypted

Author: Scott Moser
Revision Date: 2016-06-27 20:29:10 UTC

releasing package cloud-init version 0.7.7~bzr1246-0ubuntu1

Author: Scott Moser
Revision Date: 2016-07-12 19:31:23 UTC

comment update

Author: Scott Moser
Revision Date: 2016-06-29 15:05:48 UTC

include mptscsih kernel module in x86 arch for vmware networking.

This is reported to fix vmware networking for at least some cases.

Author: Scott Moser
Revision Date: 2016-06-22 20:08:38 UTC

releasing package cloud-init version 0.7.7~bzr1245-0ubuntu1~16.04.1

Author: Ryan Harper
Revision Date: 2016-06-03 15:22:35 UTC

Add RDO LinuxBridge ConfigDrive

Author: Scott Moser
Revision Date: 2016-05-20 13:47:10 UTC

backdoor-image: prefer mount-image-callback

basically decprecate usage of mount-callback-umount .
mount-image-callback is preferable as
 a.) it knows about fmt= flag to qemu-nbd that is necessary
 b.) it is in distro

Author: Scott Moser
Revision Date: 2016-05-17 15:45:49 UTC

add maas.io to the list of no_proxies

Author: Scott Moser
Revision Date: 2016-05-04 19:25:34 UTC

fix test and add comments

Author: Scott Moser
Revision Date: 2016-04-28 19:48:07 UTC

move openssh-server to server seed

Added here as a recommends so someone could remove it without removing their
ubuntu-server package.

Author: Scott Moser
Revision Date: 2016-04-28 17:51:22 UTC

closer to removal of cloud-image seed

 - make cloud-image seed include ubuntu-server metapackage
 - move pollinate from cloud-image to server seed
 - remove all packages that are in ubuntu-server
 - add bug comments for the remaining packages (cloud-init, openssh-server)

Author: Scott Moser
Revision Date: 2016-03-23 13:44:03 UTC

releasing package simplestreams version 0.1.0~bzr426-0ubuntu1

Author: Scott Moser
Revision Date: 2016-03-22 08:08:51 UTC

commit data source transfer stuff

the goal here is to make a v2 datasources that are mroe aware of
network_config and can return a 'transfer' mode, taht basically
says "Yes, I'm the datasource you're looking for, but I need some network".

I walked through the DS and looked which ones could benefit from this.
The first ones to look at would be:
ConfigDrive: claim, hostname, network
NoCloud : claim, hostname, network
MAAS : claim, network (disabled)
Azure : claim, hostname

                  definitive hostname network
== Pure Local ==
ConfigDrive: YES yes yes
CloudSigma: YES yes no
NoCloud YES yes yes
OpenNebula Weak yes yes
OVF Weak yes yes
SmartOS YES yes no
AltCloud YES no no

== Hybrid ==
Azure: YES yes no
BigStep: YES no could[partner mod]

== Pure Network ==
CloudStack: NO no no
Ec2 NO no yes[network]
GCE NO no no?
MAAS YES no no
OpenStack NO no no
DigitalOcean NO no no

Author: Scott Moser
Revision Date: 2016-03-21 21:09:25 UTC

add check_instance_id checker support

now we call check_instance_id on the restored-from-cache datasource.
This allows it to make a quick response as to whether or not it
is stale.

Add the default implementation (which returns false).
Also add Nocloud iplmementation which checks kernel command line
and files for iid (does not check seed).

Author: Scott Moser
Revision Date: 2016-03-18 16:24:56 UTC

dl-update change defaults to only trusty, wily, xenial. allow args.

Author: Scott Moser
Revision Date: 2016-03-14 16:55:34 UTC

cloud-localds: support --vendor-data, network-config and 'tar'
output. This is in support of the work going into cloud-init
under LP: #1549403.

Author: Scott Moser
Revision Date: 2016-03-02 21:38:59 UTC

add 'aliases' data as a comma separated list to products.

Bug: https://bugs.launchpad.net/cpc-core/+bug/1550421
This branch introduces a new "aliases" field as a comma separated list
of alias in a string json entry.

Its content is as follow:
 - release codename
 - the first letter of the release codename
 - the version number
 - the aliases "lts" and "default" if the current active LTS
 - "devel" if the current development release

Author: Scott Moser
Revision Date: 2016-03-02 16:56:17 UTC

hackily fix writing of multipath bindings file

this is just a hack/quick fix, but one that does work for me.

I also have another that uses dpkg --compare-versions
to compare the versions.

Author: Scott Moser
Revision Date: 2016-03-01 02:38:46 UTC

initial commit

Author: Scott Moser
Revision Date: 2016-02-12 17:13:45 UTC

do not proxy to launchpad.net

After adding lp:simplestreams to the tox environment, tox began failing.
pip install would error like:
   Downloading/unpacking bzr+lp:simplestreams
     Checking out lp:simplestreams to /tmp/pip-q94lu3i6-build
   bzr: ERROR: Transport error: Server refuses to fulfill the request
       (403 Forbidden) for https://xmlrpc.launchpad.net/bazaar/

this was because for whatever reason, squid.internal does not
want to take https traffic destined for launchpad.

Author: Scott Moser
Revision Date: 2016-02-04 13:33:45 UTC

desktop: add ppp, pppconfig, pppoeconf

Add ppp, pppconfig and pppoeconf to the desktop seed as recommends.
This is removed from the standard seed, so to be included needs adding here.

Author: Scott Moser
Revision Date: 2016-02-03 17:30:23 UTC

standard: remove ppp, pppconfig, pppoeconf from standard seed.

I'm adding these to the server-ship so they stay in main,
and get them via ISO to a user that would need them to get to
the network.

Author: Scott Moser
Revision Date: 2016-02-01 09:18:28 UTC

Enable better support for multi-hypervisor regions in OpenStack

1) Determine correct disk format.
2) Set hypervisor_type on images based on ftype field.

Author: Scott Moser
Revision Date: 2016-01-27 18:17:45 UTC

releasing package cloud-utils version 0.27-0ubuntu22

Author: Scott Moser
Revision Date: 2016-01-26 19:07:51 UTC

another fix for logic around smt disabling

Author: Scott Moser
Revision Date: 2016-01-19 20:35:37 UTC

supported-misc-servers: drop clvm

clvm was added prior to 14.04 with "until we migrate HA stack away from it".
That migration should have happened by now.

Author: Scott Moser
Revision Date: 2015-12-18 20:53:06 UTC

fix logic error in kernel installation

If a kernel package was available but not installed
the code would use the fallback path, rather than installing it.

This fixes the logic error and uses 'has_pkg_available'.

Author: Scott Moser
Revision Date: 2015-12-14 18:45:42 UTC

add sbuild-cloud-tools and support running that.

the change here reads 'release' from anything after first -, allowing for
the '-' that exists in cloud-tools.

Author: Scott Moser
Revision Date: 2015-12-09 18:02:23 UTC

remove copied / bad code

Author: Scott Moser
Revision Date: 2015-11-18 18:50:20 UTC

fix version number from - to ~

Author: Scott Moser
Revision Date: 2015-10-20 16:18:36 UTC

add random_seed dict

Author: Scott Moser
Revision Date: 2015-10-07 20:57:03 UTC

Drop the source entries also.

Author: Scott Moser
Revision Date: 2015-10-06 13:27:30 UTC

better doc, add support for OVERLAY_DISK being persistent rather than ram

Author: Scott Moser
Revision Date: 2015-09-24 18:42:24 UTC

support for storing keyrings and reading of centos cloudimg .asc files

A couple things here
== pubkey data stored in bzr ==
the tools/pubkeys-to-keyring comes from ubuntu-cloudimage-keyring package.
  that then allows you to store ascii armorred files
  in bzr and generate the keyring to verify against from those.

  You can then do:
  ./tools/pubkeys-to-keyring centos.gpg pubkeys/centos/*.gpg

  And use '--keyring=./cents.gpg' to gpg.

  pubkeys/centos/README.txt, describes where these keys came from
  and why we should trust them.

== Usage of the code to verify ==
We should probably move the get_file_sums_list() from meph2.netinst
to util.py, but the following illustrates its use. It uses gpgv to
verify the data against the keyring created above.

  from meph2 import netinst
  keyring="./centos.gpg"
  burl="http://cloud.centos.org/centos/7/images/"
  centos_suminfo= (("sha256", "sha256sum.txt.asc", True),)
  print(
      netinst.get_file_sums_list(
          burl, keyring=keyring, suminfo=centos_suminfo))

Author: Scott Moser
Revision Date: 2015-09-17 20:18:34 UTC

drop landscape-common from server seed.

There is no reason to be found for this package to be listed in the
server seed, as landscape-common is not much use without landscape-client.
landscape-client is the only rdepends of landscape-common.

landscape-client is still listed in the server-ship seed, so this
should have no affect on the overal support of the package, only
its presense in default 'server' task installs.

Author: Scott Moser
Revision Date: 2015-09-10 16:49:33 UTC

remove use of 'decode_binary'

there is no 'decode_binary' here.
the return values from subp in python2 are strings.

Author: Scott Moser
Revision Date: 2015-07-27 20:34:09 UTC

add rsyslog tests

reasonable test of reworked rsyslog module

Author: Scott Moser
Revision Date: 2015-07-20 18:02:08 UTC

fix late_command to echo to set CONSOLES

Author: Scott Moser
Revision Date: 2015-07-20 13:36:04 UTC

maas-proxy: sync apt specific squid config with squid-deb-proxy
(LP: #1474417)

Author: Scott Moser
Revision Date: 2015-07-15 16:14:39 UTC

re-order for a bit of clarity

Author: Scott Moser
Revision Date: 2015-07-15 15:57:40 UTC

maas-proxy: sync apt specific squid config with squid-deb-proxy
(LP: #1474417)

Author: Scott Moser
Revision Date: 2015-06-22 15:02:43 UTC

remove 1463043 as it is not part of backport (did not exist ever on trusty)

Author: Scott Moser
Revision Date: 2015-05-07 03:56:23 UTC

attempt at handling cciss!c0d0

Author: Scott Moser
Revision Date: 2015-04-22 19:10:27 UTC

update kernel

Author: Scott Moser
Revision Date: 2015-04-21 20:54:37 UTC

some cleanups

Author: Scott Moser
Revision Date: 2015-04-18 01:31:57 UTC

releasing package uvtool version 0~bzr99-0ubuntu2~snap1

Author: Scott Moser
Revision Date: 2015-04-10 15:23:28 UTC

initial commit

Author: Scott Moser
Revision Date: 2015-03-27 21:17:16 UTC

initial stab at installing from urls

still to do here would be to ake render_snap realize http:
and then to download that.

Author: Scott Moser
Revision Date: 2015-03-11 13:03:19 UTC

check 'echo' for failure

Author: Scott Moser
Revision Date: 2015-02-27 21:21:47 UTC

refresh patch

Author: Scott Moser
Revision Date: 2015-02-27 21:21:47 UTC

refresh patch

Author: Scott Moser
Revision Date: 2015-02-27 17:14:53 UTC

add ipxe binary

ipxe replaced gpxe. use the ipxe binary now.

Author: Scott Moser
Revision Date: 2015-02-11 19:04:39 UTC

sync again

Author: Scott Moser
Revision Date: 2015-01-12 15:26:46 UTC

update usage

Author: Scott Moser
Revision Date: 2014-12-12 17:25:46 UTC

fix bad shell syntax on snappy mirror

Author: Scott Moser
Revision Date: 2014-12-08 22:04:16 UTC

catch some failures

Author: Scott Moser
Revision Date: 2014-12-08 20:47:37 UTC

explicitly disable ssh password auth in default config

In cloud images, sshd's PasswordAuthentication is not enabled.
So having the default be "no change" is ok there.

But in snappy, there is a user with known password, and only:
  #PasswordAuthentication yes
cloud-images have
  PasswordAuthentication no

Its better to expect the user to turn it on.

Also, the fix here is to allow setting to 'unchanged' explicitly
to make no change.

Author: Scott Moser
Revision Date: 2014-12-03 13:58:20 UTC

add manifest

Author: Scott Moser
Revision Date: 2014-11-20 04:32:11 UTC

call the wrapper from systemd by wrapper name

Author: Scott Moser
Revision Date: 2014-10-28 01:00:22 UTC

releasing package curtin version 0.1.0~bzr195-0ubuntu1~14.10.1