KeyError in remove_recursive_id during commit

Same problem here after deletion of some images (using rm .*png) and adding some new files (bzr add):

bzr: ERROR: exceptions.KeyError: 'add_func_en.png-20091219153325-gsid8xpcjd8bbp7z-150'

Traceback (most recent call last):
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 842, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 1037, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 654, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.5/site-packages/bzrlib/builtins.py", line 3058, in run
    exclude=safe_relpath_files(tree, exclude))
  File "/usr/lib/python2.5/site-packages/bzrlib/decorators.py", line 192, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/workingtree_4.py", line 197, in commit
    result = WorkingTree3.commit(self, message, revprops, *args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/decorators.py", line 192, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/mutabletree.py", line 229, in commit
    *args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/commit.py", line 401, in commit
    self.work_tree.unversion(self.deleted_ids)
  File "/usr/lib/python2.5/site-packages/bzrlib/mutabletree.py", line 53, in tree_write_locked
    return unbound(self, *args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/workingtree_4.py", line 1239, in unversion
    self._inventory.remove_recursive_id(file_id)
  File "/usr/lib/python2.5/site-packages/bzrlib/inventory.py", line 1424, in remove_recursive_id
    to_find_delete = [self._byid[file_id]]
KeyError: 'add_func_en.png-20091219153325-gsid8xpcjd8bbp7z-150'

bzr 2.0.2 on python 2.5.4 (Linux-2.6.30-2-amd64-x86_64-with-debian-squeeze-sid)
arguments: ['/usr/bin/bzr', 'commit', '-F', 'change.log']
encoding: 'UTF-8', fsenc: 'UTF-8', lang: 'de_DE.UTF-8'
plugins:
  bzr_difftools /home/user/.bazaar/plugins/bzr_difftools [unknown]
  bzrtools /usr/lib/python2.5/site-packages/bzrlib/plugins/bzrtools [2.0.1]
  dbus /usr/lib/python2.5/site-packages/bzrlib/plugins/dbus [0.1.0dev]
  extmerge /home/user/.bazaar/plugins/extmerge [unknown]
  gtk /usr/lib/python2.5/site-packages/bzrlib/plugins/gtk [0.97.0.final]
  launchpad /usr/lib/python2.5/site-packages/bzrlib/plugins/launchpad [2.0.2]
  loom /usr/lib/python2.5/site-packages/bzrlib/plugins/loom [1.4.0dev]
  netrc_credential_store /usr/lib/python2.5/site-packages/bzrlib/plugins/netrc_credential_store [2.0.2]
  search /usr/lib/python2.5/site-packages/bzrlib/plugins/search [1.7.0dev]
  stats /usr/lib/python2.5/site-packages/bzrlib/plugins/stats [unknown]
  upload /usr/lib/python2.5/site-packages/bzrlib/plugins/upload [1.0.0dev]

Once again, using bzr 2.0.3 on Debian testing.
Cannot reproduce this error.
log attached.

2010/1/17 Thomas 'thorn' Hornik <email address hidden>:
> Once again, using bzr 2.0.3 on Debian testing.
> Cannot reproduce this error.
> log attached.

Do you mean that you think it's fixed in 2.0.3, or it's just hard to
reproduce consistently in any version?
--
Martin <http://launchpad.net/~mbp/>

> Do you mean that you think it's fixed in 2.0.3, or it's just hard to
> reproduce consistently in any version?

This bug appeared (randomly?) both in 2.0.2 and 2.0.3. Most the time there are no problems committing changes (even with removed files). But sometimes this bug appears.

same crash here

ubuntu amd64 with bzr 2.0.2 and an alias set to "commit --strict"

removed a folder of files, the commit went fine for a handful of them... but on another one it crashes (i'll attach the crashlog)

after the crash it says that the working tree is out of date (i haven't yet tried to see the log, update the tree or retry the commit)

judging from the stacktrace it seems this bug is quite common... shouldn't some of these (or this one) be marked as duplicates?

https://bugs.launchpad.net/bzr/+bug/435000
https://bugs.launchpad.net/bzr/+bug/494221
https://bugs.launchpad.net/bzr/+bug/391580

On 23 February 2010 07:36, Dario Bertini <email address hidden> wrote:
> same crash here
>
> ubuntu amd64 with bzr 2.0.2 and an alias set to "commit --strict"
>
> removed a folder of files, the commit went fine for a handful of them...
> but on another one it crashes (i'll attach the crashlog)
>
> after the crash it says that the working tree is out of date (i haven't
> yet tried to see the log, update the tree or retry the commit)
>
> judging from the stacktrace it seems this bug is quite common...
> shouldn't some of these (or this one) be marked as duplicates?
>
> https://bugs.launchpad.net/bzr/+bug/435000
> https://bugs.launchpad.net/bzr/+bug/494221
> https://bugs.launchpad.net/bzr/+bug/391580

Yes, thanks for finding them.

--
Martin <http://launchpad.net/~mbp/>

This is not exactly building confidence- bzr now aborts in this way for *every* remove of a tree structure. I added a trivial script to illustrate the problem:
jal@odeon:~$ ./trivial
Created a standalone tree (format: 2a)
adding a
adding a/b
adding a/b/c
adding a/b/c/d
adding a/b/c/d/file
Committing to: /home/jal/test/a/
added a
added a/b
added a/b/c
added a/b/c/d
added a/b/c/d/file
Missing checkcommit.jar, not checking commits.
Committed revision 1.
Committing to: /home/jal/test/a/
missing a/b
modified a/b
missing a/b/c
modified a/b/c
missing a/b/c/d
modified a/b/c/d
missing a/b/c/d/file
modified a/b/c/d/file
Missing checkcommit.jar, not checking commits.
bzr: ERROR: exceptions.KeyError: 'c-20100223092848-lv5tl8m8u8au91dd-3'

*** Bazaar has encountered an internal error. This probably indicates a
    bug in Bazaar. You can help us fix it by filing a bug report at
        https://bugs.launchpad.net/bzr/+filebug
    attaching the crash file
        /home/jal/.cache/crash/bzr-20100223092848-27410.crash
    and including a description of the problem.

    The crash file is plain text and you can inspect or edit it to remove
    private information.

I've just tried Fritz Jahlvingh's test script on my Ubuntu 8.04 amd64 machine :

Bazaar (bzr) 2.0.4
  Python interpreter: /usr/bin/python 2.5.2
  Python standard library: /usr/lib/python2.5
  Platform: Linux-2.6.24-27-generic-x86_64-with-debian-lenny-sid
  bzrlib: /usr/lib/python2.5/site-packages/bzrlib
  Bazaar configuration: /home/home/.bazaar
  Bazaar log file: /home/home/.bzr.log

Sure enough it fails as advertised:

bzr: ERROR: exceptions.KeyError: 'c-20100223095810-vsbchpz1r6kuxe3x-3'

But If I alter line 11:

- rm -r a/b
+ bzr rm a/b

then i no longer get the bug.

FWIW

Pete

Feeling brave today.
I just installed the latest update from Canonical:

Bazaar (bzr) 2.1.0
  Python interpreter: /usr/bin/python 2.5.2
  Python standard library: /usr/lib/python2.5
  Platform: Linux-2.6.24-27-generic-x86_64-with-debian-lenny-sid
  bzrlib: /usr/lib/python2.5/site-packages/bzrlib
  Bazaar configuration: /home/home/.bazaar
  Bazaar log file: /home/home/.bzr.log

Same behaviour as above. Using rm causes a crash, using bzr rm is OK.

Pete

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Frits Jalvingh wrote:
> This is not exactly building confidence- bzr now aborts in this way for *every* remove of a tree structure. I added a trivial script to illustrate the problem:
> jal@odeon:~$ ./trivial
> Created a standalone tree (format: 2a)
> adding a
> adding a/b
> adding a/b/c
> adding a/b/c/d
> adding a/b/c/d/file
> Committing to: /home/jal/test/a/
> added a
> added a/b
> added a/b/c
> added a/b/c/d
> added a/b/c/d/file
> Missing checkcommit.jar, not checking commits.
> Committed revision 1.
> Committing to: /home/jal/test/a/
> missing a/b
> modified a/b
> missing a/b/c
> modified a/b/c
> missing a/b/c/d
> modified a/b/c/d
> missing a/b/c/d/file
> modified a/b/c/d/file
> Missing checkcommit.jar, not checking commits.
> bzr: ERROR: exceptions.KeyError: 'c-20100223092848-lv5tl8m8u8au91dd-3'
>

I've been seeing it more as well. just doing 'bzr rm' before 'bzr
commit' works fine. Also, the commit does succeed, and 'bzr update' then
brings you into the correct state.

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkuH5D8ACgkQJdeBCYSNAAN9WwCgl3ghBq3x9jNS9A/DgRaE8aVk
Ox4AoIpja5SxFbI8KhVZrQeTbJCUagAG
=hkJG
-----END PGP SIGNATURE-----

By itself when the bug occurs simply doing a bzr update usually is all that is needed to get things working again.

My point is more that regardless of ways around this, having such a simple operation abort in this way is not exactly confidence building in the product.

the attached script does not crash bzr for me (using bzr.dev)

Bug 705802 shows people are still hitting this in Lucid, so we could possibly use it to drive an SRU.

Accepted bzr into lucid-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Per the guidelines attached to the bzr MicroReleaseException, I have installed the package from lucid-proposed in a chroot, and run the full testsuite - passed successfully.

Assuming this is fixed in maverick and above, closing floating task.

Thanks Max!

This bug was fixed in the package bzr - 2.1.4-0ubuntu1

---------------
bzr (2.1.4-0ubuntu1) lucid-proposed; urgency=low

  * Update watch file to use 2.1 series.
  * New upstream release.
   + Fix file descriptors leaks in dirstate compiled extension. LP: #583486
   + Refuse to stack on repositories in incompatible formats. LP: #562380
   + Don't delete nested trees/repos. LP: #572098
   + Fix 'bzr switch' crash when a 'ConfigurableFileMerger' is used. LP: #559436
   + Fix compatibility with older smart servers. LP: #528041
   + Fix symlinks addition. LP: #192859
   + Properly unversion children of unversioned directories. LP: #494221
   + Lock configuration files in '~/.bazaar' for updates. LP: #525571
   + Fix 'bzr commit <symlink>'. LP: #128562
   + Fix `lp:` urls when behind an http proxy. LP: #558343
   + Stop using edge.launchpad.net. LP: #583667
 -- Max Bowsher <email address hidden> Tue, 17 May 2011 09:54:17 +0100