lp:debian/crash

Created by James Westby and last modified
Get this branch:
bzr branch lp:debian/crash
Members of Ubuntu branches can upload to this branch. Log in for directions.

Related bugs

Related blueprints

Branch information

Owner:
Ubuntu branches
Status:
Development

Recent revisions

48. By Troy Heber

* Autopkgtest fix from Martin Pitt <email address hidden> (Closes: #756969):
 - Drop sudo calls; they potentially hang forever asking for a password or
   might get denied, destroy the environment, and are not necessary as the
   test already has "needs-root".
 - Fix invalid indentation of here-doc
 - Suppress expected stderr output of apt-key (which causes a test failure)
 - Drop "set -x", as that causes stderr output and the called commands are
   already verbose enough.
 - Drop -security pocket for Ubuntu's ddebs.u.c., as that does not exist.

* Fix for two minor issues with the "net" command. Without the patch, the
  "net -a" option appends its correct output with the command's "Usage:"
  message; and if either the "net -x" or "net -d" options are used without
  also specifying "-s" or "-S", the error message would indicate "net:
  illegal flag: 800000" or "net: illegal flag: 1000000" instead of showing
  the command's "Usage:" message.

* If the kernel (live or dumpfile) has the TAINT_LIVEPATCH bit set, or if
  the Red Hat "kpatch" module is installed, the tag "[LIVEPATCH]" will be
  displayed next to the kernel name in the initial system banner and by the
  "sys" command. This new tag replaces the "[KPATCH]" tag that was
  introduced in crash-7.0.7.

* Addressed three Coverity Scan complaints in vmware_vmss.c:

  50:leaked_storage: Variable "fp" going out of scope leaks the
     storage it points to.
  53:leaked_storage: Variable "fp" going out of scope leaks the
     storage it points to.
 256:warning: Use of memory after it is freed

* Remove the LKCD-only "propeller spinner" seen when a dumpfile read
  requires more than 2048 page header accesses. This was put in place
  because of the non-random-access design of LKCD dumpfiles. Without the
  patch, the spinner display is intermingled with command output, which
  complicates the parsing of the output.

* Fix to support the Linux version increment from 3 to 4. Without the
  patch, both dumpfile and live sessions fail during initialization, issuing
  the message "WARNING: kernel version inconsistency between vmlinux and
  dumpfile" or "WARNING: kernel version inconsistency between vmlinux and
  live memory", followed by the nonsensical fatal error message "crash:
  incompatible arguments: vmlinux is not SMP -- vmcore is SMP" or "crash:
  incompatible arguments: vmlinux is not SMP -- live system is SMP". To
  prevent unexpected kernel version bumps in the future, support has been
  added for version 5.

* Add support for more than 16TB of physical memory space in the SADUMP
  dumpfile format. Without the patch, there is a limitation caused by
  several 32-bit members of dump_header structure, in particular the
  max_mapnr member, which overflows if the dumpfile contains more than 16TB
  of physical memory space. The header_version member of the dump_header
  structure has been increased from 0 to 1 in this extended new format, and
  the new 64-bit members will be used.

* Fix for command lines that are redirected to a pipe. Without the patch,
  if an external piped-to command contains a quoted string that includes a
  "|" character, the command fails with the message "crash: pipe operation
  failed".

* Fix for insecure temporary file usage in _rl_tropen() as reported by
  readline library CVE-2014-2524.

* When the gdb-<version>.patch file has changed and a rebuild is done from
  within a previously-existing build tree, the "patch -N" option is used to
  ignore patches that have been previously applied; this patch also applies
  the "patch -r-" option to prevent unnecessary .rej files from being
  created.

* Fix to account for Xen hypervisor's "domain" structure member name change
  from "is_paused_by_controller" to "controller_pause_count". Without the
  patch, in Xen 4.2.5 and later, the crash session fails during
  initialization with the error message 'crash: invalid structure member
  offset: domain_is_paused_by_controller".

* During initialization, reject useless ARM64 "(A)" and "(a)" absolute
  symbols that are below the text region. Without the patch, several
  recently-introduced absolute symbols have been introduced into the kernel,
  which will be displayed by "sym -l" prior to the first kernel virtual
  address symbol, and will show up in command output where memory values are
  translated into kernel symbol references.

* Fix for ARM64 kernels to account for changes in the virtual memory layout
  introduced in Linux 3.17. The vmalloc region end address, and the vmemmap
  start and end addresses are now calculated at kernel build time, because
  they depend upon the size of a struct page. Accordingly, the crash
  utility needs to calculate those three address values dynamically, after
  the embedded gdb module has initialized. Without the patch, reads of page
  structures return invalid data due to incorrect virtual-to-physical
  translations of memory in the vmemmap range. This in turn causes commands
  that require page structure contents to fail or show invalid data, such as
  "kmem -p", "kmem -[sS]", and the "kmem -[fF]" options.

* Fix to support ELF vmcore dumpfiles whose PT_LOAD file offset values of
  their respective memory segments are not laid out sequentially from low to
  high in the dumpfile. This has only been seen in ELF dumpfiles created by
  VMware's "vmss2core -M" facility. Without the patch, the crash session
  may fail during initialization, either with the message "cannot malloc ELF
  header buffer", or "crash: <dumpfile>: not a supported file format".

* Enhancement to the support of VMware .vmss suspended state dumpfiles.
  There may be holes in the memory address saved for PCI, etc. In such
  cases, the memory dump is divided into regions. With this patch, up to 3
  memory regions are supported.

* Fortified the error handling of task gathering from the pid_hash[] chains
  during session initialization. If a chain has been corrupted, the patch
  prevents the sequence from entering an infinite loop, and the error
  messages associated with corrupt/invalid chains have been updated to
  report the pid_hash[] index number.

* Implemented a new STRDUPBUF() utility that will duplicate an existing
  string into a buffer allocated with GETBUF(). As is the case with any
  buffer allocated with GETBUF(), it is only meant to exist during the
  life-span of the current command. If it is not explicitly freed via
  FREEBUF(), then it will be freed automatically prior to the next command.

* Implemented a new fill_struct_member_data() function that gathers a bundle
  of data that describes a structure member. The function receives a
  pointer to a struct_member_data structure, in which the caller has
  initialized the "structure" and "member" name pointers:

  struct struct_member_data {
          char *structure;
          char *member;
          long type;
          long unsigned_type;
          long length;
          long offset;
          long bitpos;
          long bitsize;
  };

  A gdb "printm" command is crafted using those two fields, and the output
  of the command is used to initialize the remaining six fields. Adapted
  from Qiao Nuohan's "pstruct" extension module.

* Implemented a new "runq -c cpu(s)" option to display the run queue data of
  specified cpus. It can be used in conjunction with all runq command
  options. The cpus must be specified in a comma- and/or dash-separated
  list; for examples, "3", "1,8,9", "1-23", or "1,8-15".

* Build extension modules that utilize the generic extensions/Makefile
with -g. In addition, build the snap.c extension module with -g.

* Several fixes, updates, and enhancements for 32-bit MIPS support:
  (1) The MIPS general purpose registers in the elf_gregset_t
      don't start at index 0 but at index 6.
  (2) Adjust for the kernel's pt_regs structure changes between
      kernel versions. For example, fields are inserted into the
      middle based on build time options, and the amount of padding
      at the head of the structure was changed relatively recently.
      To handle this, split the structure definition into two parts
      and get the offsets of these two parts dynamically.
  (3) Do not display each parsed kernel symbol during initialization
      when invoked with "crash -d8".
  (4) Add support for loading raw MIPS ramdump dumpfiles.
  (5) Add support for compressed kdump dumpfiles.

* Fix for a typo in "help foreach", and a fix for a spelling error in "help
  input".

* Fix for "and and" and "the the" typos in the README file.

* Fix to address the Xen 4.5.0 hypervisor symbol name change from "dom0" to
  "hardware_domain". Without the patch, the crash session fails with the
  error message "crash: cannot resolve: dom0".

* Fix for a regression in crash-7.1.0 that causes failures when the "crash
  -t" option is run on a live system, and when analyzing remote Linux
  kernels. Without the patch, "crash -t" on a live system fails with the
  message "crash: cannot open remote memory source: /dev/mem", and attempts
  to analyze a Linux kernel remotely just shows the kernel timestamp and
  exits immediately.

* Speed up the session invocation time of "flattened" format dumpfiles
  created by the makedumpfile(8) facility. When sorting the blocks of
  memory by their intended ELF or compressed kdump file offsets, the patch
  replaces the bubble-sort method that is currently used with an insertion
  sort method.

* Remove the non-existent "-L" option from the "ps" command's mutually-
  exclusive options error message.

* Fix for the "irq", "mount", "kmem -p" and "kmem -v" commands when they are
  used in an input file. If more than one of any of those four commands are
  used in an input file, the output of the second and subsequent command
  instances will not display their respective command headers.

* Implemented a new "kmem -m" option that is similar to "kmem -p", but it
  allows the user to specify the page struct members to be displayed. The
  option takes a comma-separated list of one or more page struct members,
  which will be displayed following the page structure address. The "flags"
  member will always be expressed in hexadecimal format, and the "_count"
  and "_mapcount" members will always be expressed in decimal format.
  Otherwise, all other members will be displayed in hexadecimal format
  unless the current output radix is 10 and the member is a signed/unsigned
  integer. Members that are data structures may be specified by the data
  structure's member name, or expanded to specify a member of that data
  structure. For example, "-m lru" refers to a list_head data structure, in
  which case both the list_head.next and list_head.prev pointer values will
  be displayed; if "-m lru.next" is specified, just the list_head.next value
  will be displayed.

* Support enhancement for the 32-bit MIPS architecture that retrieves the
  per-cpu registers from the NT_PRSTATUS notes stored in the header of
  compressed kdump dumpfiles.

* Fix to remove an invalid warning message on ARM64 if a crash session is
  invoked with the "-d<number>" debug flag. Without the patch, the invalid
  message is "WARNING: SPARSEMEM_EX: questionable section values".

* Remove the leftover ".constructor" build file in the extensions
  subdirectory when "make extensions" is complete, and update the top-level
  .gitignore file to ignore post-build extensions subdirectory files.

* Fix for a segmentation violation generated by the "help -[n|D]" options on
  ARM64 compressed kdumps.

* Additional output for the "help [-D|-n]" options on ARM64. For ELF kdump
  vmcores and compressed kdumps, the elf_prstatus structure in each
  NT_PRSTATUS note will be translated.

* The "help -r" option has been extended to dump the ARM64 registers stored
  in each per-cpu NT_PRSTATUS note in compressed kdump and ELF kdump
  dumpfiles.

* Fix for the ARM64 page size determination on Linux 4.1 and later kernels.
  Without the patch, the crash session fails during initialization with the
  message "crash: invalid/unsupported page size: 98304" on kernels with 64K
  pages. On kernels with 4K pages, the message is "crash:
  invalid/unsupported page size: 6144". In addition, the "-p <page-size>"
  command line override option had no effect on ARM64; that has been fixed
  as well.

* Fix for the DATE display in the initial system banner and by the "sys"
  command to account for the Linux 3.17 change that moved the "timekeeper"
  symbol and structure into a containing tk_core structure; the
  "shadow_timekeeper" timekeeper will be used as an alternative. Without
  the patch, the DATE shows something within a few hours of the Linux epoch,
  such as "Wed Dec 31 18:00:00 1969".

* Fixes for the translation of ARM64 PTEs, as displayed by the "vm -p" and
  "vtop" commands. Without the patch, if "vm -p" references a swapped-out
  page on Linux 4.0 and later kernels, the SWAP location may indicate
  "(unknown swap location)", and will show an invalid OFFSET value; on Linux
  3.13 and later kernels, running "vtop" on a user virtual address
  incorrectly translates the PTE contents of swapped out pages by showing a
  PHYSICAL address and FLAGS translation instead of the SWAP device and
  OFFSET. It is possible that there may be PTE bit translation errors on
  other kernel versions; the patch addresses the changes in ARM64 PTE bit
  definitions made in Linux 3.11, 3.13, and 4.0 kernels.

* Enhanced the "struct.member" display capability of the "struct", "union",
  "task", "list" and "tree" commands. If a specified structure member
  contains an embedded structure, the output may be restricted to just the
  embedded structure by expressing the .member argument as "member.member".
  If a specified structure member is an array, the output may be restricted
  to a single array element by expressing the .member argument as
  "member[index]". Furthermore, these embedded member specifications may
  extend beyond one level deep, for example, by expressing the member
  argument as "member.member.member", or "member[index].member".

* Fix for any command that passes strings to gdb for evaluation, where the
  string contains a parentheses-within-parentheses expression along with a
  ">" or ">>" operator inside the outermost set of parentheses. Without the
  patch, a command such as the following fails like so:

  crash> p ((1+1) >> 1)
  p: gdb request failed: p ((1+1)
  crash>

* Fix for the handling of ARM64 kernel module per-cpu symbols. Without the
  patch, if the debuginfo data of an ARM64 kernel module that contains a
  per-cpu section is loaded by "mod -s <module>" or "mod -S", commands such
  as "bt" or "sym" may incorrectly translate the module's virtual addresses
  to symbol names.

* Fix to support the Linux version increment from 3 to 4. (Closes: #699367)
  Without the patch, both dumpfile and live sessions fail during
  initialization, issuing the message "WARNING: kernel version inconsistency
  between vmlinux and dumpfile" or "WARNING: kernel version inconsistency
  between vmlinux and live memory", followed by the nonsensical fatal error
  message "crash: incompatible arguments: vmlinux is not SMP -- vmcore is
  SMP" or "crash: incompatible arguments: vmlinux is not SMP -- live system
  is SMP"

47. By Troy Heber

* Fix for the handling of 32-bit ELF xendump dumpfiles if the guest was
  configured with more than 4GB of memory. Without the patch, the crash
  session may fail during initialization with the error message "crash:
  vmlinux and <dumpfile> do not match!".
* Fix for file-handling errors when a compressed vmlinux.debug file is
  followed by a vmlinux file on the crash command line. When the crash
  session ends, two errors will occur: (1) the vmlinux file will be deleted
  (2) the temporary uncompressed version of the vmlinux.debug file will
  remain in /var/tmp This problem also occurs in the highly unlikely case
  where a compressed vmlinux file is followed by a vmlinux.debug file on the
  command line, and the uncompressed temporary version of the vmlinux file
  is larger than the vmlinux.debug file. In that case: (1) the
  vmlinux.debug file will be deleted (2) the temporary uncompressed version
  of the vmlinux file will remain in /var/tmp
* Fix for the "search -t" option if the system has 2064 or more tasks.
  Without the patch, the command fails with a dump of the crash utility
  memory allocation statistics, ending with "search: cannot allocate any
  more memory!".
* Fix for the "mod -S" command to find the debuginfo data for Red Hat
  "kpatch" modules. Without the patch, the command would display "mod:
  cannot find or load object file for <kpatch-module> module".
* Deprecated the "mount -f" option for Linux 3.13 and later kernels
  containing commit eee5cc2702929fd41cce28058dc6d6717f723f87, which removed
  the super_block.s_files list_head member and the open files list that it
  contained. Without the patch, the command option fails with the error
  message "mount: invalid structure member offset: super_block_s_files"
* If a compressed kdump is damaged/truncated such that the bitmap data in
  the dumpfile header is not contained within the file, attempts to analyze
  it with a vmlinux file, or using the "crash --osrelease" or "crash --log"
  options with just the vmcore, will result in the crash utility spinning
  forever, endlessly performing reads of 0 bytes from the file without
  recognizing the EOF condition.
* Fix for an ARM64 compilation failure of the embedded gdb file
  "aarch-linux-nat.c" in the Fedora fc21 rawhide environment, which uses
  glibc-headers-2.19.90-24.fc21.
* Document the reason behind the deprecation of the "mount -f" option for
  Linux 3.13 and later kernels if the option is attempted, and in the "help
  mount" output, similar to the deprecated "mount -d" option.
* During initialization, reject useless ARM64 "(A)" absolute symbols that
  begin with "__crc_". Without the patch, several thousand of them may be
  displayed by "sym -l" prior to the first kernel virtual address symbol.
* When running against an ARM64 dumpfile created with the "snap.so"
  extension module, do not attempt to read the crash_notes. Since the
  dumpfile was taken while running on a live system, the crash_notes, if
  configured into the kernel, would not contain valid data. Without the
  patch, the message "WARNING: could not retrieve crash_notes" is displayed
  during session initialization.
* Determine the various ARM64 kernel virtual address ranges using the
  kernel's VA_BITS value. It currently is hardwired in the kernel to one of
  two values depending upon whether 4K or 64K pages are configured.
  However, there are plans to support 16K paqes, to make VA_BITS a
  configurable value, and to make the number of page-table levels
  configurable. Towards that end, the crash utility has been changed to
  determine the VA_BITS value based upon known kernel virtual addresses, and
  to then calculate the relevant kernel virtual address ranges on that value
  instead of hardwiring them based upon the page size.
* Enhancement to the "kmem -S" option for Linux 3.2 and later kernels
  configured with CONFIG_SLUB to display the address of each per-cpu
  kmem_cache_cpu address and the contents of its per-cpu partial list.
* If an ARM or ARM64 dumpfile does not contain the register sets of the
  active tasks in the kernel's per-cpu crash_notes, there is an
  initialization-time warning message indicating "could not retrieve
  crash_notes". It has been changed to a more meaningful warning message
  indicating "cannot retrieve registers for active tasks".
* Implement support for ARM and ARM64 raw RAM dumpfiles. One or more
  "ramdump" files may be entered on the crash command line in an ordered
  pair format consisting of the RAM dump filename and the starting physical
  address expressed in hexadecimal, connected with an ampersand:

  $ crash vmlinux ramdump@address [ramdump@address]

  A temporary ELF header will be created in /var/tmp, and the combination of
  the header and the ramdump file(s) will be handled like a normal ELF
  vmcore. The ELF header will only exist during the crash session. If
  desired, an optional "-o <filename>" may be entered to create a permanent
  ELF vmcore file from the ramdump file(s).
* Fix for the "help -[nD]" ELF header translation to recognize the EM_ARM
  and EM_AARCH values as "e_machine" types, and ELFOSABI_LINUX as an
  "e_ident[EI_OSABI]" type. Without the patch, the e_machine translation
  would show "40 (unsupported)" for 32-bit ARM, or "183 (unsupported)" on
  ARM64; and the ELFOSABI_LINUX type would be translated as "3 (?)".
* Re-run a command in the history list by entering an "!" followed by the
  number identifying the command. However, unlike the similar "r"
  pseudo-command, if the number is a command name in the user's PATH,
  maintain the current behavior and execute that command.
* Fix to recognize that the live system "crash.ko" memory driver may be
  compressed and named "crash.ko.xz". Without the patch, the driver is not
  recognized and loaded, and as a result the /dev/mem driver and/or
  /proc/kcore will be tried as the live memory source.
* On a live system during session initialization, delay the first read error
  message (typically when reading the "cpu_possible_mask") until it is
  confirmed that all of the following are true: (1) /dev/crash does not
  exist, and (2) /dev/mem is restricted via CONFIG_STRICT_DEVMEM, and (3)
  /proc/kcore cannot be read/accessed. The "kernel may be configured with
  CONFIG_STRICT_DEVMEM" and the "trying /proc/kcore as an alternative"
  messages will still be displayed when appropriate. The read error message
  be displayed only if all three live memory read options fail.
* Fortify the validity verification of the data structures traversed by the
  "kmem [-sS]" options for kernels configured with CONFIG_SLUB. Without the
  patch, the contents of several structure members are not validated, and
  may generate bogus or never-ending output, typically seen when running the
  commands on a "live dump" where the dumpfile was taken while the kernel
  was still running. The patch aborts the relevant parts of per-kmem_cache
  output when invalid data is encountered or if an object list contains
  duplicate entries, and error messages have been enhanced to more
  accurately describe the issues encountered.
* Implement support for the ppc64le PPC64 little-endian architecture. Since
  this required a large number of patches to be applied to
  architecture-neutral files in the gdb-7.6 tree, the changes are only
  applied if the host build system is a ppc64le.
* Fix for SMP active task register-gathering from "kvmdump" dumpfiles that
  were created with a cpu version id of 12 or greater that contain
  additional XSAVE related fields in their cpu device headers. Without the
  patch, active tasks running on cpus above 0 may have truncated backtraces.
* Maintain backwards-compatibility for "kvmdump" dumpfiles that were created
  by older development versions of KVM tools in which the cpu version id was
  12, but the cpu device headers did not contain the additional XSAVE
  related fields.
* Address a "ps" command performance degradation that was introduced by a
  crash-7.0.4 patch which added per-thread task_struct.rss_stat page counts
  to the task's mm_struct.rss_stat page counts in order to show an
  accurate/synchronized RSS value. Without the patch, the "ps" command
  performance would degrade as the number of tasks increased, most notably
  when there were thousands of tasks.

46. By Troy Heber

* Export the static ELF and compressed kdump vmcoreinfo_read_string()
  functions from netdump.c and kdump.c via a new read_vmcoreinfo() method in
  the global program_context structure. The function
  get_log_from_vmcoreinfo() will access vmcoreinfo data via the new pointer
  instead of requiring its callers to pass pointers to their
  dumpfile-specific function.
* Linux 3.15 and later kernels configured with CONFIG_RANDOMIZE_BASE can be
  now be readily identified because of new kernel symbols that have been
  added. For those kernels, the new "--kaslr=<offset>" and/or
  "--kaslr=auto" options are not necessary for ELF or compressed kdump
  vmcores, or for live systems that have /proc/kallsyms showing the
  relocated symbol values. A new KASLR initialization function called
  kaslr_init() is now called by symtab_init() prior to the initial
  symbol-sorting operation. If kaslr_init() determines that KASLR may be in
  effect, it will trigger a search for the relevant vmlinux symbols during
  the sorting operation, which in turn will cause the relocation value to be
  automatically calculated.
* Implemented a new "bt -c cpu(s)" option to display the backtrace of the
  active task on one or more cpus. The cpus must be specified in a comma-
  and/or dash-separated list; for examples ""3", "1,8,9", "1-23", or
  "1,8,9-14". Similar to "bt -a", the option is only applicable with crash
  dumps.
* Fix for Linux 3.11 and later ARM kernels, in which all non-panicking cpus
  offline themselves during a kdump procedure. This causes an invalid cpu
  count determination during crash session initialization from an ARM
  vmcore. The patch utilizes the cpu count found in the cpu_active_map if
  it is greater than the count in the cpu_online_map. In addition, the
  maximum NR_CPUS value for the ARM architecture has been raised from 4 to
  32.
* Fix for the X86_64 "bt" command on Linux 3.3 and later kernels to properly
  display exception frame register contents on NMI stacks. Kernel commit
  3f3c8b8c4b2a34776c3470142a7c8baafcda6eb0 added 12 more values to the NMI
  exception stack to handle nested NMIs caused by page faults or breakpoints
  that could occur while handling an NMI exception.
* Kernel commit 28696f434fef0efa97534b59986ad33b9c4df7f8 changed the stack
  layout again, swapping the location of the "saved" and "copied" registers.
  This can be detected automatically, because the "copied" registers contain
  either a copy of the "saved" registers, or point to "repeat_nmi". So, if
  "repeat_nmi" is found as the return address, assume that this is the old
  layout, and adjust the stack pointer again. Without the patch, incorrect
  register values are displayed in the exception frame dump in the NMI stack
  backtrace.
* Fix for the built-in "g" alias, which apparently has not worked correctly
  since crash-5.1.4. Without the patch, if the "g" alias and the first
  argument are separated by one space, then the first first character of
  that argument would get stripped prior to being passed to the embedded gdb
  module.
* Removed the BASELEVEL_REVISION string from defs.h, which serves no purpose
  since the deprecation of the remote daemon, and typically has been out of
  sync with the crash version.
* Fix for the "p", "irq", "struct", "union" and "*" commands if a cpu
  specification contains an invalid cpu number. Without the patch, a
  segmentation violation may be generated.
* Implemented a new capability for the "ptov" command that takes a per-cpu
  offset and cpu specification argument and translates it into the kernel
  virtual addresses for the cpus specified.
* Implemented a new "ps -m" option that is a similar, complementary option
  to "ps -l", but which translates the task timestamp value from a decimal
  or hexadecimal nanoseconds value into a more human-readable string
  consisting of the number of days, hours, minutes, seconds and milliseconds
  that have elapsed since the task started executing on a cpu. More
  accurately described, it is the time difference between the timestamp
  copied from the per-cpu runqueue clock when the task last started
  executing compared to the most current value of the per-cpu runqueue
  clock.
* In addition, a new "ps -C <cpu-specifier>" option has been added that can
  only be used with "ps -l" and "ps -m", which sorts the global task list
  into per-cpu blocks; the cpu-specifier uses the standard comma or dash
  separated list, expressed as "-C 1,3,5", "-C 1-3", "-C 1,3,5-7,10", or
  "-Call" or "-Ca" for all cpus.
* Implemented a new "runq -m" option that is a simliar, complementary option
  to "runq -t", but which displays the amount of time that the active task
  on each cpu has been running, expressed in a format consisting of days,
  hours, minutes, seconds and milliseconds.
* Implemented a new "kmem -h" option that displays the address of each
  hugepage hstate array entry, its hugepage size, its free and total counts,
  and name string.
* Implemented a new "ps -S" option that displays a summary consisting of the
  number of tasks in a task state.
* Fix for the "arguments-input-file" feature to protect against a called
  command modifying an argument string. For example, the "struct" command
  modifies "-l struct_name.member" argument strings, and so without the
  patch, all iterative calls after the first one will fail.
* Fix failure to build from source when compiling the crash utility with
  gcc-4.9 (closes: #746835)
* Fix for displaying enum values that are greater than 32-bits in size.
  Without the patch, the upper 32-bits are clipped off and displayed as
  integer-sized value.
* If the kernel (live or dumpfile) has the "kpatch" module installed, the
  tag "[KPATCH]" will be displayed next to the kernel name in the initial
  system banner and by the "sys" command.
* Fix for the "DEBUG KERNEL:" display in the initial system banner and by
  the "sys" command when using a System.map file with a Linux 3.0 and later
  debug kernel. Without the patch, the kernel version is not displayed in
  parentheses following the debug kernel name.
* If the gdb-<version>.patch file has changed and a rebuild is being done
  from within a previously-existing build tree, "patch -N" the gdb sources,
  and start the rebuild from the gdb-<version> directory instead of the
  gdb-<version>/gdb directory.
* Fix to prevent a possible segmentation violation generated by the "runq
  -g" command when run on a very active live system due to an active task on
  a cpu exiting while the command is running.
* Fix for the "runq -g" command on Linux 3.15 and later kernels, where the
  cgroup_name() function now utilizes kernfs_name(). Without the patch, the
  command fails with the error message "runq: invalid structure member
  offset: cgroup_dentry".
* Fix for the "extend" command when running with an x86_64 crash binary that
  was built with "make target=ARM64" in order to analyze ARM64 dumpfiles on
  an x86_64 host. Without the patch, if the extend command is used with an
  extension module built in the same manner, it fails with the message
  "extend: <module>.so: not an ELF format object file".
* Introduce support for 32-bit ARM kernels that are configured with
  CONFIG_ARM_LPAE. The patch implements the virtual-to-physical address
  translation of 64-bit PTEs used by ARM LPAE kernels.

45. By Troy Heber

* Fix for custom X86_64 kernels that change the declaration of the
  context_switch() function so that it is not an inline function. Without
  the patch, the message "crash: cannot determine thread return address" is
  displayed during invocation, and backtraces of blocked tasks may have
  missing or invalid frames.
* Fix to prevent a possible invocation-time error on Linux 3.7 and later
  kernels configured with CONFIG_SLAB, running against vmcore files filtered
  with the makedumpfile(8) facility. Without the patch, the message "crash:
  page excluded: kernel virtual address: <address> type: kmem_cache buffer"
  is immediately followed by the message "crash: unable to initialize kmem
  slab cache subsystem". Because of a kernel data structure name change
  from "cache_cache" to "kmem_cache_boot", the crash utility failed to
  properly downsize the stored size of the kernel's kmem_cache data
  structure from the size indicated by the vmlinux debuginfo data. This in
  turn could lead to reading beyond the end of a kmem_cache data structure
  into a page of memory that had been excluded from the vmcore. The fix was
  also applied to kernels configured with CONFIG_SLUB.
* Added a new "--kaslr <offset>" command line option for X86_64 kernels that
  are configured with CONFIG_RANDOMIZE_BASE. The offset value must be equal
  to the difference between the symbol values compiled into the vmlinux file
  and their relocated KASLR values.
* Added a new "--kaslr=auto" command line option for X86_64 kernels that
  that are configured with CONFIG_RANDOMIZE_BASE. When set to "auto", the
  KASLR relocation value will be determined automatically by comparing the
  "_stext" symbol value compiled into the vmlinux file with the _stext
  symbol value stored in kdump vmcoreinfo data; on live systems the
  comparison will be made with the "_stext" symbol value that is found in
  /proc/kallsyms.
* Enable kernel text line number capability for the "dis -l", "bt -l", "sys
  -c", and "sym" commands for kernels that are configured with
  CONFIG_RANDOMIZE_BASE.
* Fix for the "crash --log vmcore" command to account for the kernel data
  structure and VMCOREINFO string name changes from "log" to "printk_log" in
  Linux 3.11-rc4 and later kernels. Without the patch, the command fails
  with the error message "crash: VMCOREINFO: no log buffer data".
* Adjustment to the internal symbol-handling to prevent the usage of kernel
  system call alias/wrapper names, for examples, "SyS_read" and
  "compat_SyS_futex" instead of "sys_read" and "compat_sys_futex". Without
  the patch, commands such as "dis", "sym <address>", and "sys -c" display
  the alias/wrapper name instead of the real system call name in Linux 3.10
  and later kernels.
* Increase the internal hash queue head count from 128 to 32768. The hash
  queue is used for gathering and verifying lists, and the original count of
  128 may be overwhelmed if a list is extremely large. For example, on a
  256GB system with 192GB of free pages, the "kmem -f" command takes hours
  to complete; with this patch, the time is reduced to a few minutes. In
  addition, a new command line option "--hash <count>" has been added to
  allow a user to override the default hash queue head count of 32768.
* Fix for the "kmem -F" display and the "kmem -f <address>" or "kmem
  <address>" options. Without the patch, "kmem -F" does not display the
  first page in a list of free page blocks on its own line, but rather at
  the end of the previous line that shows the area number, block size, and
  free_area struct address that the page is linked to. Due to this error,
  both "kmem -f <address>" and "kmem -f address>" would not find the
  associated page or page block if it happened to be the first page or page
  block in the list.
* Created a new feature for the internal do_list() function if it is
  necessary to immediately perform a function for each entry in a list while
  the list is being traversed. A callback function, and an option callback
  data pointer, can be registered in the list_data structure. The address
  of each entry in the list along with the optional callback data pointer
  will be passed to the callback function. If desired, the callback
  function may also dictate that do_list() should stop the list traversal
  and return immediately to its caller.
* Made the "kmem -f <address>" and "kmem <address>" options more efficient
  by using the new do_list() callback function feature above as well as
  restricting the search to only the NUMA node that contains the address.
* If the first assembly language instruction in an X86_64 function is "nopl
  0x0(%rax,%rax,1)" or "data32 data32 data32 xchg %ax,%ax", which are
  generated when the ftrace facility is configured, the X86_64 "dis" command
  will append "[FTRACE NOP]" to the line.
* Correction for the "crash -h" and crash.8 man page documentation of the
  "--machdep phys_base=<physical-address>" command line option. In both
  places the parameter mistakenly indicated "physbase".
* If a host build system does not have /usr/bin/wget installed, and the
  crash package is built from a directory that was git-cloned from
  github.com/crash-utility/crash.git, the error message has been clarified
  to indicate "/usr/bin/wget is required to download gdb-7.6.tar.gz".
  Without the patch, the message indicates "tar (child): gdb-7.6.tar.gz:
  Cannot open: No such file or directory".
* Updated the ARM64 implementation to support Linux 3.13 and later kernels
  that expand to a 42-bit address space when 64K pages are configured. This
  is also the first crash version that has been tested on a live ARM64
  system with 4K pages, where it cleanly make it to the "crash>" prompt.
  However, it should be noted that some commands (most notably "bt") still
  do not work as of yet.
* Document the "--machdep phys_offset=<physical-address>" command line
  option for the ARM64 architecture in the crash.8 man page and the "crash
  -h" output.
* Fix for KVM dumpfiles created with "virsh dump --memory-only" if an X86_64
  kernel was loaded with a non-zero "phys_base". Without the patch, the
  crash session fails with the warning message "WARNING: cannot read
  linux_banner string" followed by the fatal error message "crash: vmlinux
  and <dumpfile name> do not match!".
* Initial working implementation of the basic ARM64 "bt" command, with
  several command options still under development. In-kernel exception
  frames are only dumped if the exception handler function is contained
  within the symbol boundaries from "__exception_text_start" to
  "__exception_text_end"; when ARM64 kdump is eventually implemented,
  further exception-related work will be resumed.
* Cleaned up the exception frame displays of 64-bit in-kernel and both
  32-bit and 64-bit user-mode exceptions.
* Implemented support for the ARM64 "bt -e" option.
* Implemented support for the ARM64 "bt -l" option.
* Update for the X86_64 "bt -l" option such that it also displays the
  available file and line number information for functions indicated as the
  "exception RIP" in kernel exception frames. The line number information
  will follow the exception frame register dump.
* Fix for the ARM64 virtual-to-physical translation of vmemmap page
  structure addresses for kernels configured with 4K pages. Without the
  patch, any command that required the contents of a page structure would
  fail with a readmem error.
* Added support for the ARM64 architecture in the extensions/snap.c
  extension module. Also fixed the progress percentage display to correct
  for systems which have non-zero starting physical addresses.
* Implemented support for the ARM64 "bt -f" and "bt -F[F]" options.
* Increase the ARM64 PTRS_PER_PGD_L2_64K from 1024 to 9182 to account for
  the Linux 3.13 increase of the ARM64 virtual address space size from 39 to
  42 bits when 64K pages are configured. Without the patch, the warning
  message "WARNING: cannot access vmalloc'd module memory" is displayed
  during session initialization.
* Fix for the "vm -p" option on ARM64 so that file-backed pages are properly
  translated to the filename and offset. Without the patch, file-backed
  pages are erroneously shown as being backed on a swap device.
* Increment maximum ARM64 physical address from 40 to 48 bits to match
  upstream kernel commit 87366d8cf7b3f6dc34633938aa8766e5a390ce33.
* Fix for a segmentation violation generated by the "crash -g vmlinux"
  command on ARM64.
* Fix for the ARM64 "vtop <address>" command on kernels configured with 64K
  pages if the address argument is located in the kernel logical memory map
  region, which uses 512MB hugepage mappings. Without the patch, the
  verbose page table walk mistakenly continues to the PTE level.
* Fix for ARM64 /proc/kcore support. Without the patch, the crash session
  fails with the warning message "WARNING: cannot read linux_banner string"
  followed by the fatal error message "crash: vmlinux and <dumpfile name> do
  not match!". At this point in time, the kernel requires a patch to the
  ARM64 kern_addr_valid() function to properly allow memory to be read from
  the kernel logical memory map region.

44. By Troy Heber

* New upstream version 7.0.5
* Fix for the "runq -g" option for kernels that are configured with
  CONFIG_FAIR_GROUP_SCHED, but not CONFIG_CFS_BANDWIDTH. Without the patch,
  the command fails with the message "runq: invalid structure member offset:
  cfs_rq_throttled".
* Add support for Xen PVH guest types introduced in Xen 4.4. Without the
  patch, running against a Xen 4.4 hypervisor binary would fail during
  session initialization with the error message "crash: invalid structure
  member offset: domain_is_hvm". In addition, the PVH guest type is being
  registered internally as an HVM guest type, the debug "help -X ofs"
  command's display of the domain_domain_flags offset has been fixed to show
  it in decimal, and the setting of the internal dc->domain_flags has been
  fixed to contain all flags set, not just the first one found.
* Fix for the "kmem -S" command on Linux 3.1 and later kernels that are
  configured with CONFIG_SLUB. Because the the page structure's inuse and
  objects fields used by SLUB were changed from discrete u16 types to
  bit-fields within an unsigned int, the display of per-node partial slab
  statistics are incorrect. Without the patch, the TOTAL and ALLOCATED
  values are incorrectly shown as equal values, and therefore the FREE value
  is always zero.
* Fix for the "kmem -S" command for kernels that are configured with
  CONFIG_SLUB. Eash per-cpu slab object dump may show incorrect ALLOCATED
  and FREE values; and as seen on Linux 3.5 and later kernels, the TOTAL
  value and the number of individual objects dumped may also be incorrect
  (too small).
* When executing the commands from an input file specified by the "-i
  <file>" command line option, or when accepting input from a file as a set
  of commands or as a set of command arguments using the "<" redirection
  character, unconditionally cease the operation if CTRL-c is entered.
  Without the patch, depending upon the command that was running when the
  SIGINT was received, the operation may continue uninterruptibly until the
  file contents are consumed.
* Enhanced the "bt -F" option such that if "-F" is entered twice, and if the
  stack frame contents reference a slab cache object, both the slab cache
  name and the stack contents will be displayed within brackets.
* Enhanced the "rd -S" option such that if "-S" is entered twice, and if the
  memory contents reference a slab cache object, both the slab cache name
  and the memory contents will be displayed within brackets.
* Fix for the X86_64 "bt" command to prevent an unwarranted message
  indicating "WARNING: possibly bogus exception frame" generated from a
  blocked kernel thread that was in the process of exec'ing a user process
  via the call_usermodehelper() facility.
* Fix for the X86_64 "bt" command to more correctly determine the function
  frame that called into an interrupted function. Without the patch, the
  first frame just above an IRQ exception frame register dump may show an
  invalid/stale function.
* Fix for the X86_64 "bt" command if a page fault exception was generated by
  the invalid contents of the RIP register. Without the patch, the
  exception frame register dump is not displayed above the "page_fault"
  stack frame; and in a related issue, the "bt -e" option will not find and
  display the exception frame.
* When invoking a crash session with a compressed vmlinux file, make the
  same host-machine/vmlinux endian verification that is done with
  uncompressed vmlinx files.
* Reduce the number of CTRL-c entries required to unconditionally terminate
  any manually-entered command from three to one.
* Fix for the X86_64 "bt" command if an async page fault exception occurred
  in a KVM guest running a Linux 2.6.38 or later kernel. Without the patch,
  the exception frame register dump is not displayed above the
  "async_page_fault" stack frame.

43. By Troy Heber

Add autopkgtest and try to keep the Debian and Ubuntu crash packages in
sync (closes: #721095)

42. By Troy Heber

Fix from Louis Bouchard <email address hidden> to resolve FTBS on
i386

41. By Troy Heber

* Fix for the ARM architecture if the backtrace unwind information cannot be
  gathered during session initialization. Without the patch, the two
  unwind-related warning messages indicating "WARNING: UNWIND: failed to
  gather unwind_table list" and "WARNING: UNWIND: failed to initialize
  module unwind tables" are followed by the fatal error message "crash:
  cannot hash task_struct entries".

* Fix for the "help -[Dn]" dumpfile information display of the GUID EFI
  table in the header of SADUMP dumpfiles. Without the patch, only 33 of
  the 36 bytes in the table are translated.

* Fix for the determination of the kernel NR_CPUS configurable for Linux 3.8
  and later kernels that are configured with CONFIG_SLAB. Without the
  patch, the kernel's compiled-in NR_CPUS value was incorrectly calculated
  to be the sum of the kernel's NR_CPUS and MAX_NUMNODES configurables.

* In the next release of makedumpfile, the status field of the dumpfile
  header of compressed kdumps will show the compression type that was
  utilized. The "help -[Dn]" output has been updated to display that
  information.

* For kernels configured with CONFIG_SLAB in which an array_cache pointer
  referenced by a kmem_cache structure is invalid, the individual cache(s)
  will be marked as invalid. During session initialization, the message
  "crash: kmem_cache: <cache-address>: invalid array_cache pointer" will be
  displayed, and during runtime, attempts to access the cache(s) will result
  in a message indicating that the cache is "[INVALID/CORRPUTED]". Without
  the patch, the message "crash: unable to initialize kmem slab cache
  subsystem" is displayed during session initialization, and run-time
  commands that attempt to access the kmem slab cache subsystem fail with
  the error message "kmem cache slab subsystem not available".

* Fix for the "kmem -[sS] <slab-object-address>" option in Linux 3.6 and
  later kernels configured with CONFIG_SLAB. Without the patch, the command
  fails with the message "kmem: address is not allocated in slab subsystem:
  <slab-object-address>. This also causes the "kmem <slab-object-address>"
  command to (quietly) fail to determine that the address is a slab object.

* Fix for the "bt" command if a kernel __init text address is encountered.
  Without the patch, and depending upon the reallocation of the __init text
  memory, a bogus framesize may be calculated, or more likely, in a
  compressed kdump, a warning message indicating "bt: page excluded: kernel
  virtual address: <address> type: gdb_readmem_callback" will be displayed
  following the frame data.

* Update for determining whether an S390X PTE contains a swap entry in Linux
  3.12 and later kernels.

* Resurrected the translation and display of the page.flags bits by the
  "kmem -p" command on Linux 2.6.26 and later kernels whose vmlinux
  debuginfo data contains either the "pageflags" enumerator or the
  "pageflag_names" array of trace_print_flags structures. If they are not
  available, just the page.flags value is printed in hexadecimal, as has
  been done since Linux 2.4.9.

* Fix for the "bt" command when used with vmcore files that were created
  with the recently-introduced "virsh dump --memory-only", which dumps KVM
  guests into an ELF vmcore similar to those created by the kdump facility.
  Without the patch, a faulty backtrace for the panic task may be generated
  due to the use of incorrect starting RSP/RIP registers; this happens
  because (unlike kdump) the non-panicking cpus are offlined prior to the
  dumpfile being created, which in turn leads to the use of the wrong
  NT_PRSTATUS note.

* Fix for the CPU number display on systems with 255 or more cpus during the
  initial banner, by the "set" command, the "ps" command, and by all
  commands that display the per-task header consisting of the task address,
  pid, cpu and command name. Without the patch, for cpu 255, the "sys"
  command displays "NO_PROC_ID", and the other commands would show a "-" for
  the cpu number; for cpu numbers greater than 255, garbage values would be
  displayed in the cpu number field.

* Implemented support for compressed kdump header version 6, in which
  makedumpfile(8) adds new fields in the kdump_sub_header to support large
  memory systems with pfn values that are larger than 32-bits. Without the
  patch, if the system contains physical memory located in high memory such
  that its maximum pfn value is overflows the 32-bit "max_mapnr" field in
  the header, the crash session will fail with the error message "crash:
  vmlinux and vmcore do not match!".

* Fix for the "net -s" command on Linux 3.8 and later kernels. Without the
  patch, the command fails with the message "net: invalid structure member
  offset: inet_opt_daddr".

* Fix a build failure in a native ARM64 environment due to obsolete LKCD
  dumpfile headers.

* Implementation of a new "per-cpu object" as an argument format that can be
  passed to the "p", "struct", "union" or "*" commands. The format is
  expressed as either <per-cpu symbol>:<cpu-specifier> or as <per-cpu
  offset>:<cpu-specifier>, where the per-cpu symbol or per-cpu offset must
  precede a colon, and where the <cpu-identifier> follows the colon.
  Without the patch, per-cpu symbols are only accepted by the "p" command,
  and the data type and the resolved kernel virtual address for each per-cpu
  instance are displayed shown. With this patch, a colon and a
  cpu-specifier may be appended to the symbol name, and the the contents of
  the symbol on each cpu that is specified will be displayed by the "p"
  command. For the "struct/union/*" commands, an argument may be specified
  using either a per-cpu offset value or per-cpu symbol name followed by a
  colon and cpu-specifier, and the contents of each structure/union on each
  specified cpu will be displayed.

  Fixed several minor flaws that were detected by a Coverity Scan.

40. By Troy Heber

* Added "bison" to the BuildRequires line of the crash.spec file.
         Without the patch, the build of the embedded gdb-7.6 module will fail
         unless either /usr/bin/bison or /usr/bin/yacc are available. The
         failure will result in a stream of error messages from different
         files that indicate:

           multiple definition of 'main'
           undefined reference to 'c_parse_escape'
           undefined reference to 'ada_parse'
           undefined reference to 'ada_error'
           undefined reference to 'c_parse'
           undefined reference to 'c_error'
           undefined reference to 'cp_demangled_name_to_comp'
           undefined reference to 'cp_demangled_name_parse_free'
           undefined reference to 'cp_comp_to_string'
           undefined reference to 'cp_new_demangle_parse_info'

         and the build fails like so:

           collect2: ld returned 1 exit status
           make[4]: *** [gdb] Error 1
           crash build failed

         If building with rpmbuild, the new BuildRequires "bison" entry will
         prevent the build from initiating unless the bison package has been
         installed. If building with the tar.gz file, the build attempt will
         proceed and fail unless either the bison or byacc (Berkeley Yacc)
         package is installed.

* Fix the S390X initialization sequence on kernels that are configured
         with CONFIG_STRICT_DEVMEM to automatically try /proc/kcore if:

          (1) the /dev/crash driver is not available, and
          (2) the initial /dev/mem access fails.

         Without the patch, if /dev/mem is selected as the memory source and
         it is restricted, the crash session will fail during initialization
         with the error message "crash: read error: kernel virtual address:
         <address> type: cpu_possible_mask".

* When checking whether a argument on the crash command line is a dumpfile
  that may be in makedumpfile's "flattened" format, do not bother checking
  character device files.

* Fix for the PPC64 virtual-to-physical virtual address translation
  mechanism for vmalloc and user-space virtual addresses on Linux 3.10 and
  later kernels. Without the patch, the message "WARNING: cannot access
  vmalloc'd module memory" is displayed during initialization, and during
  the crash session, if a command attempts to translate or read a vmalloc or
  user-space virtual address, it will fail.

* Clean up all files that emit "warning: format not a string literal and no
  format arguments" when compiled with -Wformat-security warning option.
  All instances of fprintf, sprintf and snprintf using the format
  "fprintf(fp, buf)" are replaced with "fprintf(fp, "%s", buf)". Also, the
  -Wformat-security warning option has been added to the option list used
  when compiling with "make warn".

* Fix a build failure when compiling with very old gcc-3.4.6 version on a
  2.6.9-based RHEL4 IA64 host. The bfd library in gdb-7.6 is compiled with
  the -Werror option, and it fails with the message "elflink.c:4733:
  warning: 'idx' might be used uninitialized in this function".

* Fix a build failure when compiling with very old gcc-3.4.6 version on a
  2.6.9-based RHEL4 S390 or S390X hosts. The embedded gdb-7.6 fails to
  compile with the error message "s390-nat.c:364: error: storage size of
  'iov' isn't known".

* Fix to properly store two-digit kernel version numbers. (closes: #717036)

* Fix to provide hugepage address translation for the "vtop" command on the
  PPC64 architecture.

* Fix for the "log" command to account for the kernel data structure name
  change from "log" to "printk_log" in Linux 3.11-rc4 and later kernels.
  Without the patch, the message "WARNING: log buf data structure(s) have
  changed" will be displayed during initialization and by the "log" command.

* Fix to add a linefeed after the description of the "kmem -I" option in the
  "help kmem" output, which was recently added in crash-7.0.0.

* Document the "-s" command line option in the "crash -h|--help" output and
  in the crash.8 man page to also indicate that runtime command scrolling is
  turned off by default.

* Fix for the "irq -d" option on 2.6.25 and later X86_64 kernels to display
  the Intel interrupt descriptor table contents. Without the patch, those
  kernel versions would display "irq: -d option not supported or applicable
  on this architecture or kernel".

* Fix for the "kmem -[sS]" options on Linux 3.11-rc1 and later kernels that
  are configured with CONFIG_SLAB. Without the patch, the command fails
  with the error message "kmem: invalid structure member offset:

* Fix for the "kmem <address>" and the "bt -F" options on Linux 3.8 and
  later kernels that are configured with CONFIG_SLUB. Without the patch,
  the command would fail with the error message "kmem: invalid structure
  member offset: page_slab".

* Fix misspellings in the "bt" and "search" help page output.

* Fix for the determination of the base of the kernel's unity-mapped virtual
  address region on recent ARM kernels whose "_stext" variable address has
  changed from 0xc0008000 to 0xc0100000. Without the patch, the crash
  session fails during invocation with the error message "crash: vmlinux and
  vmcore do not match!".

* When printing data structures, prevent the embedded gdb from symbolically
  translating pointers that are not kernel virtual addresses. Kernel or
  module symbols that are not virtual addresses can be mistaken for virtual
  addresses, leading to NULL pointers being invalidly translated into a
  symbol name from the vmlinux or module object file. For example, in
  X86_64 kernels, NULL pointers are translated into the symbol
  "irq_stack_union", whose value is not a virtual address, but rather a
  per-cpu offset value of 0.

* Fix for the "kmem -s <address>" or "kmem <address>" options on Linux 3.11
  and later kernels configured with CONFIG_SLAB. Without the patch, both
  commands fail with the error message "kmem: cannot resolve cache_cache".

* Fix to prevent the "bt" command from generating a segmentation violation
  in a case where the per-cpu "current_task" variable and the runqueue's
  "curr" variable did not agree, and the panic task had overflowed its
  kernel stack. This led to the selection of the a starting RSP address
  which belonged to the other task; without the patch, the command generated
  a segmentation violation after printing the first frame of the backtrace.

39. By Troy Heber

* Updated format-strings patch to cover additional architectures
* Requested update to packages-arch-specific to enable s390x and armel
  (#717283)

Branch metadata

Branch format:
Branch format 7
Repository format:
Bazaar repository format 2a (needs bzr 1.16 or later)
Stacked on:
lp:debian/squeeze/crash
This branch contains Public information 
Everyone can see this information.

Subscribers