cloud-init ca-certs leaves a blank line in /etc/ca-certificates.conf

Bug #1077020 reported by Luis Arias
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init
Fix Released
High
Scott Moser
cloud-init (Ubuntu)
Fix Released
High
Scott Moser
Precise
Fix Released
High
Unassigned
Quantal
Fix Released
High
Scott Moser
Raring
Fix Released
High
Scott Moser

Bug Description

== Begin SRU Information ==
[Impact]
 * a documented feature of cloud-init, for adding ca-certificates does not function as it should. Instead, certificates added in this manner simply are ignored. This is because apparently, a line directly following a blank line in /etc/ca-certificates.conf is ignored.

[Test Case]
  - start a cloud instance with no user-data
  - add content below to /etc/cloud/cloud.cfg.d/99-local-certs.cfg
  - run the ca-certs code through cloud-init single
    you will see output from update-ca-certificates indicating no
    new certificates were added
    $ sudo cloud-init single --name=ca_certs --frequency=always
    Cloud-init v. 0.7 running 'single' at Sun, 02 Dec 2012 02:23:21 +0000. Up 2429.68 seconds.
    Updating certificates in /etc/ssl/certs... 0 added, 0 removed; done.
    Running hooks in /etc/ca-certificates/update.d....done.

    # this can be fixed by removing the blank line and re-running
    # update-ca-certificates
    Also, note that the following has no output:
    $ ls -l /usr/lib/ssl/certs/ | grep -i cloud
  - edit /etc/ca-certificates.conf, remove cloud-init added entry
    $ sed -i '/cloud-init-ca-certs.crt/d'
  - upgrade cloud-init, re-run the ca_certs
    $ sudo dpkg -i cloud-init_all.deb
    $ sudo cloud-init single --name=ca_certs --frequency=always

    This time, you will see output containing: "1 added, 0 removed; done."
    Also, (trimmed output), you will see:
    $ ls -l /usr/lib/ssl/certs/ | grep -i cloud
    lrwxrwxrwx b1d2b355.0 -> cloud-init-ca-certs.pem
    lrwxrwxrwx cbbf81bb.0 -> cloud-init-ca-certs.pem
    lrwxrwxrwx cloud-init-ca-certs.pem -> /usr/share/ca-certificates/cloud-init-ca-certs.crt

[Regression Potential]
 * regression potential is low. It could break the ca_certs module further, but the module is not functional as it is. Tracebacks are caught when modules are executed, so there is really no potential for further harm.

== End SRU Information ==

Using a cloud-init yaml file adding a certificate like this:

# BEGIN /etc/cloud/cloud.cfg.d/99-local-certs.cfg
ca-certs:
  # If present, the 'trusted' parameter should contain a certificate (or list
  # of certificates) to add to the system as trusted CA certificates.
  # Pay close attention to the YAML multiline list syntax. The example shown
  # here is for a list of multiline certificates.
  # - Amazon RDS SSL Certificate (http://s3.amazonaws.com/rds-downloads/mysql-ssl-ca-cert.pem)
  trusted:
    - |
      -----BEGIN CERTIFICATE-----
      MIIDQzCCAqygAwIBAgIJAOd1tlfiGoEoMA0GCSqGSIb3DQEBBQUAMHUxCzAJBgNV
      BAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdTZWF0dGxlMRMw
      EQYDVQQKEwpBbWF6b24uY29tMQwwCgYDVQQLEwNSRFMxHDAaBgNVBAMTE2F3cy5h
      bWF6b24uY29tL3Jkcy8wHhcNMTAwNDA1MjI0NDMxWhcNMTUwNDA0MjI0NDMxWjB1
      MQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHU2Vh
      dHRsZTETMBEGA1UEChMKQW1hem9uLmNvbTEMMAoGA1UECxMDUkRTMRwwGgYDVQQD
      ExNhd3MuYW1hem9uLmNvbS9yZHMvMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKB
      gQDKhXGU7tizxUR5WaFoMTFcxNxa05PEjZaIOEN5ctkWrqYSRov0/nOMoZjqk8bC
      med9vPFoQGD0OTakPs0jVe3wwmR735hyVwmKIPPsGlaBYj1O6llIpZeQVyupNx56
      UzqtiLaDzh1KcmfqP3qP2dInzBfJQKjiRudo1FWnpPt33QIDAQABo4HaMIHXMB0G
      A1UdDgQWBBT/H3x+cqSkR/ePSIinPtc4yWKe3DCBpwYDVR0jBIGfMIGcgBT/H3x+
      cqSkR/ePSIinPtc4yWKe3KF5pHcwdTELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldh
      c2hpbmd0b24xEDAOBgNVBAcTB1NlYXR0bGUxEzARBgNVBAoTCkFtYXpvbi5jb20x
      DDAKBgNVBAsTA1JEUzEcMBoGA1UEAxMTYXdzLmFtYXpvbi5jb20vcmRzL4IJAOd1
      tlfiGoEoMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADgYEAvguZy/BDT66x
      GfgnJlyQwnFSeVLQm9u/FIvz4huGjbq9dqnD6h/Gm56QPFdyMEyDiZWaqY6V08lY
      LTBNb4kcIc9/6pc0/ojKciP5QJRm6OiZ4vgG05nF4fYjhU7WClUx7cxq1fKjNc2J
      UCmmYqgiVkAGWRETVo+byOSDZ4swb10=
      -----END CERTIFICATE-----
# END /etc/cloud/cloud.cfg.d/99-local-certs.cfg

The certificate is added to the /etc/ca-certificates.conf file but there is a blank line between the previous content and the line added by cloud-init. In this situation running update-ca-certificates doesn't take the cloud-init certificates into account. Removing the blank line and running update-ca-certificates again fixes the issue.

ProblemType: Bug
DistroRelease: Ubuntu 12.04
Package: cloud-init 0.6.3-0ubuntu1.1
ProcVersionSignature: User Name 3.2.0-31.50-virtual 3.2.28
Uname: Linux 3.2.0-31-virtual x86_64
ApportVersion: 2.0.1-0ubuntu14
Architecture: amd64
Date: Fri Nov 9 15:01:03 2012
Ec2AMI: ami-3d4ff254
Ec2AMIManifest: (unknown)
Ec2AvailabilityZone: us-east-1d
Ec2InstanceType: m1.medium
Ec2Kernel: aki-825ea7eb
Ec2Ramdisk: unavailable
PackageArchitecture: all
ProcEnviron:
 TERM=screen
 PATH=(custom, user)
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: cloud-init
UpgradeStatus: No upgrade log present (probably fresh install)

Related branches

Revision history for this message
Luis Arias (kaaloo) wrote :
Revision history for this message
James Page (james-page) wrote :

Using the cloud-config provided by the bug reported I was able to confirm this issue on a recent 12.04 image.

Changed in cloud-init (Ubuntu):
status: New → Confirmed
importance: Undecided → High
Revision history for this message
James Page (james-page) wrote :

Also confirmed on quantal release images.

Scott Moser (smoser)
Changed in cloud-init:
status: New → Triaged
importance: Undecided → High
milestone: none → 0.7.2
Changed in cloud-init (Ubuntu Quantal):
status: New → Triaged
Changed in cloud-init (Ubuntu Precise):
status: New → Triaged
importance: Undecided → High
Changed in cloud-init (Ubuntu Quantal):
importance: Undecided → High
Scott Moser (smoser)
description: updated
Revision history for this message
Scott Moser (smoser) wrote :

fixed in revno 744

Changed in cloud-init:
assignee: nobody → Scott Moser (smoser)
status: Triaged → Fix Committed
Scott Moser (smoser)
Changed in cloud-init (Ubuntu Quantal):
status: Triaged → In Progress
assignee: nobody → Scott Moser (smoser)
todaioan (alan-ar06)
Changed in cloud-init (Ubuntu Precise):
status: Triaged → Fix Committed
Changed in cloud-init (Ubuntu Quantal):
status: In Progress → Opinion
Scott Moser (smoser)
Changed in cloud-init (Ubuntu Quantal):
status: Opinion → In Progress
Changed in cloud-init (Ubuntu Precise):
status: Fix Committed → Triaged
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.1-0ubuntu3

---------------
cloud-init (0.7.1-0ubuntu3) raring; urgency=low

  * cherry pick relevant patches from trunk up to revision 745
  * fix writing of sudoers when suders rule is a string rather than
    an array (LP: #1079002)
  * add trailing slash to sudoers files that are written
  * fix resizefs module when 'noblock' was provided (LP: #1080985)
  * make sure there is no blank line before cloud-init entry in
    there are no blank lines in /etc/ca-certificates.conf (LP: #1077020)
 -- Scott Moser <email address hidden> Mon, 03 Dec 2012 21:45:48 -0500

Changed in cloud-init (Ubuntu Raring):
status: Confirmed → Fix Released
Scott Moser (smoser)
Changed in cloud-init (Ubuntu Raring):
assignee: nobody → Scott Moser (smoser)
Revision history for this message
todaioan (alan-ar06) wrote :

ok

Changed in cloud-init (Ubuntu Precise):
assignee: nobody → todaioan (alan-ar06)
status: Triaged → Fix Released
Revision history for this message
Scott Moser (smoser) wrote :

Why did you mark this fix-released in precise? The upload went to raring to fix it.

Changed in cloud-init (Ubuntu Precise):
status: Fix Released → Triaged
Revision history for this message
Clint Byrum (clint-fewbar) wrote : Please test proposed package

Hello Luis, or anyone else affected,

Accepted cloud-init into quantal-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/cloud-init/0.7.0-0ubuntu2.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cloud-init (Ubuntu Quantal):
status: In Progress → Fix Committed
tags: added: verification-needed
Revision history for this message
Adolfo Jayme Barrientos (fitojb) wrote :

Hi Luis, does the update in quantal-proposed fix the bug for you?

Changed in cloud-init (Ubuntu Precise):
assignee: todaioan (alan-ar06) → nobody
Revision history for this message
Clint Byrum (clint-fewbar) wrote :

Hello Luis, or anyone else affected,

Accepted cloud-init into quantal-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/cloud-init/0.7.0-0ubuntu2.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Scott Moser (smoser)
tags: added: verification-done
removed: verification-needed
Revision history for this message
Luis Arias (kaaloo) wrote :

Hi,

I have been trying to run a test against the package in quantal-proposed this morning but it is not clear to me how to get cloud-init to use the package in proposed. Adding quantal-proposed as a package source is not working out because it comes too late in the process. The package in proposed would have to be installed before cloud-init kicks in somehow. If you have some tips in this area I can give it another try. Thanks!

Luis

Revision history for this message
Scott Moser (smoser) wrote :

Luis, for testing with -proposed, I used the attached following cloud-config file passed in as user-data.
It adds -proposed and moves the ca-cert config module to run later in the process so the update has already occurred.

Just for reference, I launched instance like this:
$ EC2PRE=euca- ubuntu-ec2-run quantal daily --dry-run --user-data-file=/tmp/my.ud --key=brickies
# us-east-1/ebs/ubuntu-quantal-daily-amd64-server-20130109
euca-run-instances --user-data-file=/tmp/my.ud --key=brickies --instance-type=t1.micro ami-ed21a884

Then, in console output and /var/log/cloud-init-output.log I see:

Cloud-init v. 0.7 running 'modules:final' at Wed, 09 Jan 2013 14:26:23 +0000. Up 58.53 seconds.
Updating certificates in /etc/ssl/certs... Error opening Certificate cert-ec2.pem
140485162526368:error:02001002:system library:fopen:No such file or directory:bss_file.c:398:fopen('cert-ec2.pem','r')
140485162526368:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:400:
unable to load certificate
WARNING: cert-ec2.pem does not contain a certificate or CRL: skipping
1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d....done.

The noise about invalid certificicate is due to bug 1085537.

Then, inside the instance:
$ ls -l /usr/lib/ssl/certs/ | grep -i cloud
lrwxrwxrwx 1 root root 23 Jan 9 14:26 b1d2b355.0 -> cloud-init-ca-certs.pem
lrwxrwxrwx 1 root root 23 Jan 9 14:26 cbbf81bb.0 -> cloud-init-ca-certs.pem
lrwxrwxrwx 1 root root 50 Jan 9 14:26 cloud-init-ca-certs.pem -> /usr/share/ca-certificates/cloud-init-ca-certs.crt

Revision history for this message
Scott Moser (smoser) wrote :
Revision history for this message
Luis Arias (kaaloo) wrote :

Nice ! Didn't know you could control cloud-init with such precision. I'll give it a try on my setup and post back.

Revision history for this message
Luis Arias (kaaloo) wrote :

Worked great. I can confirm the package in quantal proposed fixed this issue. Thanks for the fix guys!

Revision history for this message
Scott Moser (smoser) wrote :

I've committed changes for this in a precise branch at lp:~smoser/ubuntu/precise/cloud-init/sru . I have a ppa build of that at https://launchpad.net/~smoser/+archive/cloud-init-test/ . Any testing on that would be appreciated.

The plan is to move SRU this as soon as the current SRU moves to -updates.

Revision history for this message
Colin Watson (cjwatson) wrote : Update Released

The verification of this Stable Release Update has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.0-0ubuntu2.2

---------------
cloud-init (0.7.0-0ubuntu2.2) quantal-proposed; urgency=low

  * debian/patches/lp-1090482-fix-cloud-config-mirrors.patch:
    fix issue with cloud-config data in user-data providing mirror
    info (LP: #1073077)

cloud-init (0.7.0-0ubuntu2.1) quantal-proposed; urgency=low

  * debian/patches/lp-1073077-zsh-workaround-for-locale_warn.patch: avoid
    warning when user's shell is zsh (LP: #1073077)
  * debian/patches/lp-1077700-config-drive-fix-ssh-authorized-keys.patch:
    fix bug in config-drive-v2 usage of authoried keys (LP: #1077700)
  * debian/patches/lp-1080985-fix-resize-root-noblock.patch:
    fix 'resize_root: noblock' (LP: #1080985)
  * debian/patches/lp-1076811-fix-userdata-update-to-distro-config.patch:
    fix updates to distro config via user-data. (LP: #1076811)
  * debian/patches/lp-1066115-install-landscape-if-needed.patch:
    fix permissions on landscape config, and ensure landscape client
    is installed if landscape config is given. (LP: #1066115)
  * debian/patches/lp-1070345-restart-landscape-if-needed.patch:
    restart the landscape-client if changes to config were made. (LP: #1070345)
  * debian/patches/lp-1077020-fix-ca-certificates-blanklines.patch: fix
    adding of empty lines in ca-certificates file (LP: #1077020)
 -- Scott Moser <email address hidden> Mon, 17 Dec 2012 10:15:03 -0500

Changed in cloud-init (Ubuntu Quantal):
status: Fix Committed → Fix Released
Revision history for this message
Clint Byrum (clint-fewbar) wrote : Please test proposed package

Hello Luis, or anyone else affected,

Accepted cloud-init into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/cloud-init/0.6.3-0ubuntu1.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cloud-init (Ubuntu Precise):
status: Triaged → Fix Committed
tags: removed: verification-done
tags: added: verification-needed
Revision history for this message
Scott Moser (smoser) wrote :

Verification done using latest released 12.04 cloud-image booting it under kvm in raring.

$ sudo apt-get install genisoimage -y
$ bzr branch lp:~smoser/+junk/backdoor-image ./bi

$ imgurl="http://cloud-images.ubuntu.com/releases/precise/release-20130204/ubuntu-12.04-server-cloudimg-amd64-disk1.img"
$ deburl="https://launchpad.net/ubuntu/+source/cloud-init/0.6.3-0ubuntu1.5/+build/4312778/+files/cloud-init_0.6.3-0ubuntu1.5_all.deb"

$ wget $imgurl -O precise-amd64.img.dist
$ wget $deburl -O cloud-init.deb

$ qemu-img convert -O qcow2 precise-amd64.img.dist disk1.img.dist
$ qemu-img create -f qcow2 -b disk1.img.dist patched.img.dist

## patch the patched.img.dist with new cloud-init
$ sudo ./bi/mount-callback-umount patched.img.dist -- \
    sh -ec 'mp=$MOUNTPOINT; cp cloud-init.deb $mp/tmp &&
            LANG=C chroot $mp dpkg -i /tmp/cloud-init.deb ;
            rm $mp/tmp/cloud-init.deb' --

$ qemu-img create -f qcow2 -b patched.img.dist patched.img

$ cat my-user-data
#cloud-config
password: passw0rd
chpasswd: { expire: False }
ssh_pwauth: True
output: {all: '| tee -a /var/log/cloud-init-output.log'}
ca-certs:
  ... ### content from above example 99-local-certs.cfg ###

$ echo "instance-id: $(uuidgen || echo i-abcdefg)" > my-meta-data
$ cloud-localds my-seed.img my-user-data my-meta-data

$ kvm -m 512 -drive file=patched.img,if=virtio -cdrom my-seed.img -curses

## now, logged in as ubuntu:passw0rd
% grep cloud-init.*found.data.source /var/log/cloud-init.log
found data source: DataSourceNoCloud [seed=/dev/sr0]
% grep -i cert /var/log/cloud-init-output.log
Updating certificates in /etc/ssl/certs... 1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d....done.
% ls -l /usr/lib/ssl/certs/ | grep -i cloud
lrwxrwxrwx 1 root root 23 Feb 19 22:04 b1d2b355.0 -> cloud-init-ca-certs.pem
lrwxrwxrwx 1 root root 23 Feb 19 22:04 cbbf81bb.0 -> cloud-init-ca-certs.pem
lrwxrwxrwx 1 root root 50 Feb 19 22:04 cloud-init-ca-certs.pem ->
    /usr/share/ca-certificates/cloud-init-ca-certs.crt

tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.6.3-0ubuntu1.5

---------------
cloud-init (0.6.3-0ubuntu1.5) precise-proposed; urgency=low

  * debian/update-grub-legacy-ec2: consider kernels bootable on ec2
    that are named -generic, in addition to -virtual. This fixes a problem
    where the kernels installed by linux-lts-quantal were not added to
    /boot/grub/menu.lst (LP: #1005551)
  * debian/patches/lp-1077020-fix-ca-certificates-blanklines.patch: fix
    adding of empty lines in ca-certificates file (LP: #1077020)
  * debian/patches/lp-1031065-nonet-not-start-networking.patch: do not 'start
    networking' in cloud-init-nonet upstart job. Doing so can cause networking
    to be started earlier than it should be. Instead, add a
    cloud-init-container job that runs only in a container and emits
    net-device-added (LP: #1031065).
  * debian/patches/lp-1037567-add-config-drive-v2-support.conf:
    backport support for config-drive-v2 which is part of Openstack Nova in
    Folsom and later. (LP: #1037567) (LP: #1100545)
 -- Scott Moser <email address hidden> Wed, 16 Jan 2013 19:37:57 -0500

Changed in cloud-init (Ubuntu Precise):
status: Fix Committed → Fix Released
Revision history for this message
Scott Moser (smoser) wrote :

fixed in 0.7.2

Changed in cloud-init:
milestone: 0.7.2 → none
status: Fix Committed → Fix Released
Revision history for this message
James Falcon (falcojr) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.