Upstart early job logging causes boot failure for systems with no initramfs (error is "No available ptys")

michael@panther:~$ apt-cache policy upstart
upstart:
  Installed: 1.4-0ubuntu8
  Candidate: 1.4-0ubuntu8
  Version table:
 *** 1.4-0ubuntu8 0
        500 http://archive.ubuntu.com/ubuntu/ precise/main amd64 Packages
        100 /var/lib/dpkg/status
michael@panther:~$ ls -l /etc/init/
total 256
-rw-r--r-- 1 root root 320 Sep 1 10:13 acpid.conf
-rw-r--r-- 1 root root 268 Mar 1 2011 alsa-restore.conf
-rw-r--r-- 1 root root 267 Feb 21 2011 alsa-store.conf
-rw-r--r-- 1 root root 385 Feb 9 02:59 cgroup-lite.conf
-rw-r--r-- 1 root root 266 Feb 8 18:13 console.conf
-rw-r--r-- 1 root root 509 Jan 24 2011 console-setup.conf
-rw-r--r-- 1 root root 1122 Feb 8 18:13 container-detect.conf
-rw-r--r-- 1 root root 356 Aug 12 2010 control-alt-delete.conf
-rw-r--r-- 1 root root 297 Aug 24 2010 cron.conf
-rw-r--r-- 1 root root 1089 Oct 26 02:13 cryptdisks-enable.conf
-rw-r--r-- 1 root root 435 Oct 26 02:13 cryptdisks-udev.conf
-rw-r--r-- 1 root root 510 Jan 10 04:28 dbus.conf
-rw-r--r-- 1 root root 273 Feb 24 2010 dmesg.conf
-rw-r--r-- 1 root root 784 Feb 18 2010 ecryptfs-utils-restore.conf
-rw-r--r-- 1 root root 782 Feb 18 2010 ecryptfs-utils-save.conf
-rw-r--r-- 1 root root 1345 Oct 27 00:14 failsafe.conf
-rw-r--r-- 1 root root 265 Feb 17 10:21 flush-early-job-log.conf
-rw-r--r-- 1 root root 317 Jul 5 2010 hostname.conf
-rw-r--r-- 1 root root 557 Mar 22 2010 hwclock.conf
-rw-r--r-- 1 root root 444 Mar 22 2010 hwclock-save.conf
-rw-r--r-- 1 root root 2382 Feb 8 11:20 libvirt-bin.conf
-rw-r--r-- 1 root root 367 Oct 1 2010 module-init-tools.conf
-rw-r--r-- 1 root root 943 Sep 28 2010 mountall.conf
-rw-r--r-- 1 root root 349 Sep 28 2010 mountall-net.conf
-rw-r--r-- 1 root root 261 Sep 28 2010 mountall-reboot.conf
-rw-r--r-- 1 root root 1201 Sep 28 2010 mountall-shell.conf
-rw-r--r-- 1 root root 405 Feb 15 12:27 mounted-debugfs.conf
-rw-r--r-- 1 root root 550 Jul 15 2011 mounted-dev.conf
-rw-r--r-- 1 root root 480 Mar 7 2011 mounted-proc.conf
-rw-r--r-- 1 root root 610 Jul 15 2011 mounted-run.conf
-rw-r--r-- 1 root root 1663 Feb 22 2011 mounted-tmp.conf
-rw-r--r-- 1 root root 903 Jul 15 2011 mounted-var.conf
-rw-r--r-- 1 root root 373 Jan 12 07:48 networking.conf
-rw-r--r-- 1 root root 803 Jan 12 07:58 network-interface.conf
-rw-r--r-- 1 root root 523 Feb 12 15:11 network-interface-container.conf
-rw-r--r-- 1 root root 1603 Jan 12 07:48 network-interface-security.conf
-rw-r--r-- 1 root root 971 Sep 28 00:22 plymouth.conf
-rw-r--r-- 1 root root 326 Mar 26 2010 plymouth-log.conf
-rw-r--r-- 1 root root 899 Mar 18 2011 plymouth-splash.conf
-rw-r--r-- 1 root root 800 Sep 28 00:22 plymouth-stop.conf
-rw-r--r-- 1 root root 367 Feb 11 2011 plymouth-upstart-bridge.conf
-rw-r--r-- 1 root root 363 Nov 17 11:37 procps.conf
-rw-r--r-- 1 root root 454 Oct 3 21:12 rc.conf
-rw-r--r-- 1 root root 705 Oct 3 21:09 rcS.conf
-rw-r--r-- 1 root root 1543 Oct 3 21:12 rc-sysinit.conf
-rw-r--r-- 1 root root 426 Jan 4 15:38 rsyslog.conf
-rw-r--r-- 1 root root 683 Apr 14 2011 screen-cleanup.conf
-rw-r--r-- 1 root root 348 Feb 8 18:13 tty1.conf
-rw-r--r-- 1 root root 333 Feb 8 18:13 tty2.conf
-rw-r--r-- 1 root roo...

Thanks for reporting this bug and helping to make Ubuntu better.

Can you confirm the following:

- The /dev/ptmx device exists and has permissions 0666.
- Your kernel version (cat /proc/version). Are you running a custom kernel without pty support?

Hi All, I'm affected this bug too. I use custom kernel with following options.

CONFIG_UNIX98_PTYS=y
CONFIG_LEGACY_PTYS=y

It may be important that my kernel has no ram disk (initrd).

Distro kernel boots without problem.

Status changed to 'Confirmed' because the bug affects multiple users.

@Lukas: can you confirm the perms are correct on /dev/ptmx? Also, see comment #3 on bug 936293.

This bu affects me too.

Issue comes just after a fresh net-install of Ubuntu Precise Pangolin PowerPC.

Since is a fresh install and i even doesn't get the login terminal, don't know how to check /dev/ptmx perms.

I'm running a custom 2.6.38.8 kernel, no rd and i do have support for PTYs, also multiple instances of pts.

@Daniele: add "init=/bin/sh" to your grub command line and boot. This will drop you to a shell. You can then inspect permissions with "ls -l /dev/ptmx".

"ls -l /dev/ptmx" returns the following:

crw-rw-rw- 1 root root 5, 2 Nov 15 00:01 /dev/ptmx

Hope this helps.

And, how i could add --no-log parameter to the kernel in order to have a temporary workaround to the issue?
My environment passes parameters to the kernel via "kboot.conf". I have no bootloader installed, kernel also is loaded via kboot.conf.
I can also add that with Oneiric I never had this issue and Ubuntu was working just fine.

Sounds like this might be a "chicken-and-egg" issue where Upstart wants a pty, but pty support isn't yet available (/dev/pts not mounted) as the mountall job hasn't yet run. This being a side-effect of having no initramfs since /dev/pts is normally mounted from there.

Then, what we can do to troubleshoot this?

@Michael: please can you confirm you too are using a custom kernel?

Yup- I do. Same thing with no initramfs or initrd.

On Mon, Feb 20, 2012 at 2:26 PM, James Hunt <email address hidden>wrote:

> @Michael: please can you confirm you too are using a custom kernel?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/936667
>
> Title:
> init(4): no available ptys stops boot
>
> Status in “upstart” package in Ubuntu:
> Confirmed
>
> Bug description:
> When booting 12.04 after a recent update, I see "init(4): no available
> ptys" being displayed faster than a machine gun. When I add --no-log
> as a kernel option and I'm able to boot just fine.
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/upstart/+bug/936667/+subscriptions
>

The best current solution for this scenario is to disable job logging if pty setup fails (as a result of /dev/pts not being mounted for example).

Please could those affected by this bug test Upstart version 1.4-0ubuntu9 in the PPA below which includes such a fix. Note - has not yet built: code should be available by Sat 25th (see PPA for details):

  https://launchpad.net/~jamesodhunt/+archive/upstart-job-logging

With upstart I can boot on 3.2.0-17.

Seems to be working for me. I ended up putting the --no-log in grub conf.
Took it out to test and it works.

On Wed, Feb 29, 2012 at 9:36 AM, KrzysiekT <email address hidden>wrote:

> With upstart I can boot on 3.2.0-17.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/936667
>
> Title:
> Upstart early job logging causes boot failure for systems with no
> initramfs (error is "No available ptys")
>
> Status in “upstart” package in Ubuntu:
> Confirmed
>
> Bug description:
> When booting 12.04 after a recent update, I see "init(4): no available
> ptys" being displayed faster than a machine gun. When I add --no-log
> as a kernel option and I'm able to boot just fine.
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/upstart/+bug/936667/+subscriptions
>

@James: I hit this issue when upgrading to 1.4-0ubuntu8. I added your PPA repository and tried to build your version 1.4-0ubuntu9 on my pandaboard (armhf architecture) but I got the following errors:

BAD: wrong content in file 0x407e9298 (output), expected 'hello world
' got 'hello world'
 at tests/test_job_process.c:1673 (test_run).
/bin/bash: line 5: 21763 Aborted ${dir}$tst
FAIL: test_job_process
....
...ensure remainder of log written when file deleted with uid 0
BAD: wrong value for output, got unexpected (nil)
 at tests/test_log.c:832 (test_log_new).
/bin/bash: line 5: 22431 Aborted ${dir}$tst
FAIL: test_log
...
BAD: wrong value for block file, got unexpected NULL
 at tests/test_conf.c:285 (test_source_reload_job_dir).
/bin/bash: line 5: 22782 Aborted ${dir}$tst
FAIL: test_conf
....
===============================================
3 of 14 tests failed
Please report to <email address hidden>
===============================================
make[4]: *** [check-TESTS] Error 1
make[4]: Leaving directory `/home/juan/upstart-1.4/init'
make[3]: *** [check-am] Error 2
make[3]: Leaving directory `/home/juan/upstart-1.4/init'
make[2]: *** [check] Error 2
make[2]: Leaving directory `/home/juan/upstart-1.4/init'
make[1]: *** [check-recursive] Error 1
make[1]: Leaving directory `/home/juan/upstart-1.4'
dh_auto_test: make -j1 check returned exit code 2
make: *** [build] Error 29
dpkg-buildpackage: error: debian/rules build gave error exit status 2

Booting Linux 3.3.0-rc5 (vanilla, no initramfs) with upstart-1.4-0ubuntu9 (from James' upstart-job-logging PPA) went fine, even w/o the "--no-log" kernel option. However, at the end of the booting process, syslog says:

 kernel: [ 17.913723] init: No available ptys
 kernel: [ 17.913826] init: Failed to create pty - disabling logging
 kernel: [ 17.915596] init: Temporary process spawn error: No such file or directory
 kernel: [ 17.929426] init: No available ptys
 kernel: [ 17.929531] init: Failed to create pty - disabling logging
 kernel: [ 17.931403] init: Temporary process spawn error: No such file or directory
 kernel: [ 17.939599] init: No available ptys
 kernel: [ 17.939687] init: Failed to create pty - disabling logging
 kernel: [ 17.947603] init: Temporary process spawn error: No such file or directory
 kernel: [ 17.976511] init: plymouth-stop pre-start process (605) terminated with status 1

The "Failed to create pty - disabling logging" message is also printed on the local console and repeated every now and then. Maybe this (debugging?) information can be omitted in the final version?

$ ls -l /dev/ptmx
crw-rw-rw- 1 root tty 5, 2 Mar 7 01:34 /dev/ptmx

This bug or similar affects me. Installed from ubuntu-12.04-beta1-desktop-amd64.iso downloaded 3/3/2012.

Adding --no-log to grub : startup logs on screen to about point that " init: no available ptys " message looped and then hangs ( I presume the " no log" is effective, but not producing progress in startup.)

The /boot/grub/ directory has only a blacklist file, otherwise empty. Is this because it has not yet been set up for initrm?

After recent update problem appeared again, I can not boot.

So where should I put "--no-log" to boot precise?

The --no-log will be a kernel command line option.

If you're on x86 or amd64, you would stop the Grub boot, and manually edit the kernel cmdline to add the --no-log. Then, once you're up and running, you would edit /etc/default/grub changing GRUB_CMDLINE_LINUX, to affect both normal and recovery modes. Once you've done that, you run 'update-grub' and you should be good to go.

Ref: https://help.ubuntu.com/community/Grub2

Affected me with Nemo kernel on Nokia N900. Trying to figure out if -- are needed for no-log with U-Boot bootloader.

This bug was fixed in the package upstart - 1.5-0ubuntu1

---------------
upstart (1.5-0ubuntu1) precise; urgency=low

  * FFe. LP: #962124.
  * New upstream 1.5 release.
    - fix boot failure when /dev/pts is not mounted before upstart starts.
      LP: #936667.
    - fix a file descriptor leak on job restarts. LP: #940290.
  * debian/conf/failsafe.conf: Added missing "emits" which breaks
    'initctl check-config'.
  * debian/manpages/upstart-events.7: Added 'failsafe-boot' and
    'recovery' events.
 -- James Hunt <email address hidden> Thu, 22 Mar 2012 13:50:31 +0000

I'm still getting this error when booting 12.04 as a domU with the 1.5-0ubuntu1 Upstart package installed. The system does boot, and I am able to get to my console. However, the hvc0 console is initially flooded with these messages.

Any ideas?

-Tim

I just have upgraded from 11.10 to 12.04 and am getting these messages during boot.

As I believe this issue is entirely separate from the bug this fixed, I opened a new bug report relating to the errors I am seeing on the hvc0 console as a guest in a Xen environment:

- https://bugs.launchpad.net/ubuntu/+source/upstart/+bug/980917

-Tim

Now on Lubuntu 12.04 update not available, and the error still persisit !!!
The server on "https://code.launchpad.net/~ubuntu-branches/ubuntu/precise/upstart/precise" give me HTTP ERROR 404!!!

Sorry for another post, I use upstart version 1.5-0ubuntu7 on Lubuntu 12.04 x64, and the update server, linked upper, give me HTTP 404 on synaptic or on apt-get update!
I make every wrong?
I now will probe the --no-log flag to the kernel in grub.conf, this will probably works for me too.
Thanks!

Hi Turbeki,

I'm new to Ubuntu. I just to know the procedure how and where we can update / add the kernel option "--no-log" in the system for me to address the issue on "Failed to create pty - disabling logging'.

Hope you can guide me how to do it.

Thanks in advance.

Fox example, open a console/terminal, and write this line, and then press Enter:
sudo pico /boot/grub/grub.cfg
You can replace pico word with an other simple text editor(gedit).
Please enter your root/admin password, and then, please press Enter key, then locate with the keyboard arrows similar to this, and not the recovery copy of this, or an older version of this, with older version number! Verify, that the line begins with menuentry, what contains later in the line, and so locate the correct!
menuentry 'Ubuntu, Linux 3.2.0-24-generic verzióval' --class ubuntu --class gnu-linux --class gnu --class os {
 recordfail
 gfxmode $linux_gfx_mode
 insmod gzio
 insmod part_msdos
 insmod ext2
 set root='(hd0,msdos5)'
 search --no-floppy --fs-uuid --set=root f10afaa4-bb63-4de3-90d4-2cf3bae096c1
 linux /boot/vmlinuz-3.2.0-24-generic root=/dev/sda5 rootfstype=ext4 ro crashkernel=384M-2G:64M,2G-:128M quiet slient plymouth.enable=0
}
Write to the end of the line begins with linux, the --no-log flag, like this:
 linux /boot/vmlinuz-3.2.0-24-generic root=/dev/sda5 rootfstype=ext4 ro crashkernel=384M-2G:64M,2G-:128M quiet slient plymouth.enable=0 --no-log
Then press in pico(if You use this) Control+O, and then Control+X key combinations, then type into the console: exit, and then please press Enter!
Or in other text editor(that run with root/admin rights), please save the file, then close the editor!
Then please reboot Your computer, to verify!
Please don't modify other characters, if You write a mistake, please don't save with Control+O, or with other text editor, then close the text editor(the pico can close with Control+X), and don't save(in pico, please say No(the first character of Your languages No word), then restart from the console command line!
If you save a mistake, your computer will probably not boot at next time!
But, if your PC install kernel updates, this file will rebuilt, and this workaround must be reapplied, and also must reapply the disable of initrd, but, if You don't use initrd, i haven't a permanent workaround, because i'm also new to Ubuntu.
Sorry for my bad English!

Hi
I was running a Ubuntu 12.04 x86 server until this evening. It is running kernel 3.2.0-32-generic-pae. When I went to see the PPA ppa:jamesodhunt/upstart-job-logging mentioned above, I issued the add-apt-repository command only to realize that since the system hasn't booted, it is offline and has no Internet access.
Is there another way to fix the issue? Student work (185 students) is suffering due to this downtime and hence an early response would be very highly appreciated by everyone down here.
Thanks and best regards
Saqib

This bug was fixed prior to the release of 12.04. Your problem seems to be unrelated to the subject of this bug report.