lp:~vorlon/upstart/lp.926473

Branch merges

Propose for merging

No branches dependent on this one.

Ready for review for merging into lp:~jamesodhunt/upstart/early-job-log-flush

James Hunt: Pending requested 2012-02-15

Diff: 49 lines (+25/-0)

2 files modified

ChangeLog (+7/-0)
init/tests/test_job_process.c (+18/-0)

api

Related source package recipes

No recipes using this branch. (?)

Create packaging recipe

Related snap packages

Related bugs

Bug #926473: upstart test suite assumes no inherited fds

Medium

Fix Released

Link a bug report

Related blueprints

1359. By Steve Langasek on 2012-02-15

don't let leaked file descriptors from the environment cause the test suite to
fail

1358. By James Hunt on 2012-02-14

* dbus/com.ubuntu.Upstart.xml:
  - added 'FlushEarlyJobLog' method.
* init/control.c:
  - control_flush_early_job_log(): New function to flush early job log.
* init/job_process.c:
  - job_process_terminated(): Call log_handle_unflushed() to potentially
    add log object to unflushed list (the early job log) in certain
    scenarios.
* init/log.c:
  - log_flushed: bool indicating successful flush of early job log.
  - log_unflushed_files: the "early job log" list.
  - log_new(): Call log_unflushed_init() and initialize new log members.
  - log_flush():
    - Only call log_read_watch() conditionally now.
    - Update for new log_file_open() return value.
  - log_io_reader(): Update for new log_file_open() return value.
  - log_io_error_handler(): Set remote_closed for the benefit of
    log_flushed() (to avoid flushing multiple times).
  - log_file_open: Now saves errno value from open(2).
  - log_read_watch(): Removed log->unflushed->len assert since it was
    erroneous: even if unflushed data exists, it will be written in
    order when log_io_reader() calls log_file_write().
  - log_unflushed_init(): New function to initialise the
    log_unflushed_files list.
  - log_handle_unflushed(): New function that potentially adds log
    object to the log_unflushed_files list to allow the data to be
    flushed _after_ the parent object has been destroyed.
  - log_clear_unflushed(): New function to clear the
    log_unflushed_files list by attempting to flush the data to disk.
* init/log.h:
  - Added new Log members: detached, remote_closed and open_errno.
  - Updated documentation.
  - Added prototypes for new functions: log_handle_unflushed(),
    log_clear_unflushed() and log_unflushed_init().
* init/tests/test_job_process.c:
  - test_run():
    - Call log_unflushed_init().
    - Corrected grammar in error messages for "ensure sane fds" tests.
    - "with single line command writing fast and exiting": Call
      nih_child_add_watch().
  - test_spawn():
    - Call log_unflushed_init().
    - "simple test": More memorable log file name, added call
      to log_handle_unflushed(), added missing nih_free().
    - "with multi-line script and 'console log'": More memorable log
      file name.
    - "with single-line script and 'console log'": More memorable log file
      name.
    - "read single null bytes with 'console log'": More memorable log
      file name, call log_handle_unflushed(), added missing nih_free().
* init/tests/test_log.c:
  - Comments.
  - test_log_new(): Call log_unflushed_init().
    - "object checks with uid 0": Add extra checks.
    - "writing a new log file with uid 0": Call log_handle_unflushed()
      and associated checks.
    - New test "ensure logger flushes cached data on request"
    - "ensure log written when directory created accessible with uid 0":
      - Call log_handle_unflushed().
* util/initctl.c:
  - flush_early_job_log_action(): New function to flush the early job
    log.
  - commands: Added new command flush-early-job-log.
* util/man/initctl.8: Updated for new flush-early-job-log command.
* util/tests/test_initctl.c:
  - STOP_UPSTART(): Check return from kill(2).
  - test_show_config(): Adding missing rmdir(2).
  - test_check_config(): Adding missing rmdir(2).
  - test_flush_early_job_log(): New function embodying new test
    "with job ending before log disk writeable".

1357. By James Hunt on 2012-02-13

* init/log.c:
  - log_flush(): Comments.
  - log_read_watch():
    - Improved comments.
    - Don't re-attempt read on EAGAIN/EWOULDBLOCK since those errors are
      nominally impossible due to the remote end of the pty already
      having ended (LP: #926468).
    - Added a debug-mode warning when 'badly-behaved' application
      detected that leaks fds to its children, as this is generally a bug.
* init/tests/test_job_process.c:
  - test_run():
    - "with single line command writing fast and exiting": added
      waitpid() to ensure no EAGAIN.
  - test_spawn():
    - "with no such file, no shell and console log": added
      waitpid() to ensure no EAGAIN.

1356. By James Hunt on 2012-02-03

* init/job_process.c: job_process_spawn():
  - Set close-on-exec for pty_master.
  - Ensure stdio buffers flushed prior to forking to ensure no data
    leakage to child (should init be run with '--debug', or the tests
    be run with redirected output, for example).
  - Free log object if child process fails to exec(3) rather than just
    closing fd. This ensures io watch is removed correctly. (LP: #922754)
  - Only need to remap pty_master if CONSOLE_LOG in operation.
* init/tests/test_job_process:
  - fd_valid(): New helper function to determine if specified fd is valid.
  - child(): Added new 'TEST_FDS' test to ensure no fd leakage to child
    processes.
  - test_run():
    - New tests:
      - "ensure sane fds with no console, no script"
      - "ensure sane fds with no console, and script"
      - "ensure sane fds with console log, no script"
      - "ensure sane fds with console log, and script"
      - "with single-line command running an invalid command, then a 1-line post-stop script"
      - "with single-line command running an invalid command, then a 2-line post-stop script"
      - "with single-line command running an invalid command, then a post-stop command"
      - "with single-line command running an invalid command, then an invalid post-stop command"
      - "with single-line command running a valid command, then a 1-line invalid post-stop command"
    - Test "with single-line command running an invalid command":
      - now diverts stderr output for less chatty test-run experience.
      - Improved checking.
    - Test "with setuid me" now diverts stderr output for less chatty
      test-run experience.
  - test_spawn():
    - New tests:
      - "with no such file, no shell and console log"
      - "ensure sane fds with no console"
      - "ensure sane fds with console log"
* init/tests/test_log.c: test_log_destroy():
  - New test "ensure watch freed when log destroyed"

1355. By James Hunt on 2012-01-27

* init/tests/test_job_process.c: test_run(): Fixed typo in
          test "with setuid me" where uid value was being set to gid.

1354. By James Hunt on 2012-01-26

* init/job_process.c: job_process_terminated(): Free log to ensure data
  written as soon as _any_ process ends (consider respawn jobs).
* init/log.c:
  - log_destroy():
    - Improved documentation.
    - Now calls new function log_flush().
  - log_flush(): New function to ensure no lingering buffered job data
    remains. Now considers EBADF (LP: #912558).
  - log_io_reader():
    - Added missing assert for @len.
    - Simplified ENOSPC handling.
    - Ensure log->io set to NULL to allow other routines to detect it
      really has gone.
  - log_file_write(): Added @len checks.
  - log_read_watch(): New function to drain data from a watch descriptor
    (which also must consider EBADF).
* init/log.h: Added define for LOG_READ_SIZE.
* init/tests/test_job_process.c:
  - test_run():
    - Added some extra pointer checks.
    - Free class *before* checking file to ensure destructor invoked at
      correct point.
    - Added test "with single-line command running an invalid command"
      (for scenario bug 912558 exposed).
    - Added test "with single-line command writing fast and exiting".
* init/tests/test_log.c: Changed all tests to use openpty(3) rather than
  pipe(2) for semantic parity with actual code.
* util/tests/test_user_sessions.sh:
  - ensure_no_output(): Now calls check_job_output() and delete_job() to
    simplify logic.
  - delete_job(): Call get_job_file() rather than doing it long-hand.
  - check_job_output(): New function.
  - start_job(): Added allow_failure parameter.
  - test_ensure_no_unexpected_output(): New test
    "ensure command job does not create log file with invalid command".

1353. By James Hunt on 2012-01-05

* init/man/init.5: Explain that all job processes affected
  by 'setuid' and 'setgid' stanzas.

1352. By James Hunt on 2011-12-22

* init/tests/test_job_process.c: test_run():
  - Changed timeout for test feature "ensure that no log
  file written for CONSOLE_NONE".

1351. By James Hunt on 2011-12-22

init/job_process.c: job_process_spawn():
- Set child handler to default rather than explicit ignore
  to avoid test failures in environments that disallow
  ignoring SIGCHLD.

1350. By James Hunt on 2011-12-15

* init/tests/test_job_process.c: Add missing include for
  fnmatch.h.