tox / run-tests do not run from normal distribution

Problem appears to be in dist/makedist

$ bzr branch lp:duplicity -r 1165 dup_rev
Branched 1165 revisions.
$ cd dup_rev/testing/
$ ./run-tests
GLOB sdist-make: /home/aaron/Programming/duplicity/dup_rev/setup.py
Traceback (most recent call last):
  File "/usr/local/bin/tox", line 9, in <module>
    load_entry_point('tox==1.7.2', 'console_scripts', 'tox')()
  File "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py", line 26, in main
    retcode = Session(config).runcommand()
  File "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py", line 310, in runcommand
    return self.subcommand_test()
  File "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py", line 442, in subcommand_test
    sdist_path = self.sdist()
  File "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py", line 418, in sdist
    sdist_path = self._makesdist()
  File "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py", line 334, in _makesdist
    return self.config.distdir.listdir()[0]
IndexError: list index out of range
$ cd ..
$ bzr revert -r 1164 dist/makedist
 M dist/makedist
$ cd testing/
$ ./run-tests
GLOB sdist-make: /home/aaron/Programming/duplicity/dup_rev/setup.py
py26 create: /home/aaron/Programming/duplicity/dup_rev/.tox/py26
py26 installdeps: mock==0.7.2, pexpect==2.4, ptyprocess==0.4, python==2.6, unittest2
py26 inst: /home/aaron/Programming/duplicity/dup_rev/.tox/dist/duplicity-0.tar.gz
py26 runtests: PYTHONHASHSEED='1488455712'
py26 runtests: commands[0] | /home/aaron/Programming/duplicity/dup_rev/.tox/py26/bin/python /home/aaron/Programming/duplicity/dup_rev/setup.py test
running test
creating build
creating build/lib.linux-x86_64-2.6
creating build/lib.linux-x86_64-2.6/duplicity
copying duplicity/backend.py -> build/lib.linux-x86_64-2.6/duplicity
copying duplicity/dup_time.py -> build/lib.linux-x86_64-2.6/duplicity
copying duplicity/cached_ops.py -> build/lib.linux-x86_64-2.6/duplicity
copying duplicity/collections.py -> build/lib.linux-x86_64-2.6/duplicity
copying duplicity/librsync.py -> build/lib.linux-x86_64-2.6/duplicity
[... works as expected ...]

Not sure what your environment is, but 'tox -e py27' runs fine for me. I
run the entire tox suite before any commit/push.

On Sun, Feb 21, 2016 at 9:55 AM, Aaron Whitehouse <
<email address hidden>> wrote:

> Public bug reported:
>
> I just tried pulling down the latest code and did a ./run-test, but
> received an error "IndexError: list index out of range".
>
> Through a binary search, I found that the error was introduced in rev 1165:
>
> http://bazaar.launchpad.net/~duplicity-team/duplicity/0.7-series/revision/1165
>
> If I pull that down and do ./run-tests, I receive the following error
> (whereas 1164 works as expected):
> $ ./run-tests
> GLOB sdist-make: /home/aaron/Programming/duplicity/dup_rev/setup.py
> Traceback (most recent call last):
> File "/usr/local/bin/tox", line 9, in <module>
> load_entry_point('tox==1.7.2', 'console_scripts', 'tox')()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 26, in main
> retcode = Session(config).runcommand()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 310, in runcommand
> return self.subcommand_test()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 442, in subcommand_test
> sdist_path = self.sdist()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 418, in sdist
> sdist_path = self._makesdist()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 334, in _makesdist
> return self.config.distdir.listdir()[0]
> IndexError: list index out of range
>
> Can other people run the tests properly, or has this been broken for
> everyone from 1165?
>
> ** Affects: duplicity
> Importance: Undecided
> Status: New
>
> --
> You received this bug notification because you are subscribed to
> Duplicity.
> https://bugs.launchpad.net/bugs/1548080
>
> Title:
> Revision 1165 creates "IndexError: list index out of range" on testing
> /run-tests
>
> Status in Duplicity:
> New
>
> Bug description:
> I just tried pulling down the latest code and did a ./run-test, but
> received an error "IndexError: list index out of range".
>
> Through a binary search, I found that the error was introduced in rev
> 1165:
>
> http://bazaar.launchpad.net/~duplicity-team/duplicity/0.7-series/revision/1165
>
> If I pull that down and do ./run-tests, I receive the following error
> (whereas 1164 works as expected):
> $ ./run-tests
> GLOB sdist-make: /home/aaron/Programming/duplicity/dup_rev/setup.py
> Traceback (most recent call last):
> File "/usr/local/bin/tox", line 9, in <module>
> load_entry_point('tox==1.7.2', 'console_scripts', 'tox')()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 26, in main
> retcode = Session(config).runcommand()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 310, in runcommand
> return self.subcommand_test()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py"...

What environment are you running under? I'm not able to replicate.

On Sun, Feb 21, 2016 at 10:14 AM, Aaron Whitehouse <
<email address hidden>> wrote:

> Problem appears to be in dist/makedist
>
> $ bzr branch lp:duplicity -r 1165 dup_rev
> Branched 1165 revisions.
> $ cd dup_rev/testing/
> $ ./run-tests
> GLOB sdist-make: /home/aaron/Programming/duplicity/dup_rev/setup.py
> Traceback (most recent call last):
> File "/usr/local/bin/tox", line 9, in <module>
> load_entry_point('tox==1.7.2', 'console_scripts', 'tox')()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 26, in main
> retcode = Session(config).runcommand()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 310, in runcommand
> return self.subcommand_test()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 442, in subcommand_test
> sdist_path = self.sdist()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 418, in sdist
> sdist_path = self._makesdist()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 334, in _makesdist
> return self.config.distdir.listdir()[0]
> IndexError: list index out of range
> $ cd ..
> $ bzr revert -r 1164 dist/makedist
> M dist/makedist
> $ cd testing/
> $ ./run-tests
> GLOB sdist-make: /home/aaron/Programming/duplicity/dup_rev/setup.py
> py26 create: /home/aaron/Programming/duplicity/dup_rev/.tox/py26
> py26 installdeps: mock==0.7.2, pexpect==2.4, ptyprocess==0.4, python==2.6,
> unittest2
> py26 inst:
> /home/aaron/Programming/duplicity/dup_rev/.tox/dist/duplicity-0.tar.gz
> py26 runtests: PYTHONHASHSEED='1488455712'
> py26 runtests: commands[0] |
> /home/aaron/Programming/duplicity/dup_rev/.tox/py26/bin/python
> /home/aaron/Programming/duplicity/dup_rev/setup.py test
> running test
> creating build
> creating build/lib.linux-x86_64-2.6
> creating build/lib.linux-x86_64-2.6/duplicity
> copying duplicity/backend.py -> build/lib.linux-x86_64-2.6/duplicity
> copying duplicity/dup_time.py -> build/lib.linux-x86_64-2.6/duplicity
> copying duplicity/cached_ops.py -> build/lib.linux-x86_64-2.6/duplicity
> copying duplicity/collections.py -> build/lib.linux-x86_64-2.6/duplicity
> copying duplicity/librsync.py -> build/lib.linux-x86_64-2.6/duplicity
> [... works as expected ...]
>
> --
> You received this bug notification because you are subscribed to
> Duplicity.
> https://bugs.launchpad.net/bugs/1548080
>
> Title:
> Revision 1165 creates "IndexError: list index out of range" on testing
> /run-tests
>
> Status in Duplicity:
> New
>
> Bug description:
> I just tried pulling down the latest code and did a ./run-test, but
> received an error "IndexError: list index out of range".
>
> Through a binary search, I found that the error was introduced in rev
> 1165:
>
> http://bazaar.launchpad.net/~duplicity-team/duplicity/0.7-series/revision/1165
>
> If I pull that down and do ./run-tests, I receive the following error
> (whereas 1164 works as expected):
> $ ./run-tests
> GLOB sdist-make: /home...

The attached patch (which simply reverts a small section of code and the associated import statement to what was in 1164) makes rev 1165 (and the current trunk) run tests without the IndexError.

I haven't put this through as a merge request because I haven't looked into what the reverted code did -- at this stage I just wanted to isolate the problem.

$ tox --version
1.7.2 imported from /usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/__init__.pyc

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 15.10
Release: 15.10
Codename: wily

makedist has nothing to do with running duplicity, just the build. The
changes you made took out the change I made to get everything from bzr (or
git) for the release. The original makedist assumed (bad!) that the
directory you were in matched the repo. A distribution should get the code
for the release from the repo, not from a local dir.

It might suggest a path problem on your end where something is not being
resolved properly.

On Sun, Feb 21, 2016 at 10:51 AM, Aaron Whitehouse <
<email address hidden>> wrote:

> $ tox --version
> 1.7.2 imported from
> /usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/__init__.pyc
>
> $ lsb_release -a
> No LSB modules are available.
> Distributor ID: Ubuntu
> Description: Ubuntu 15.10
> Release: 15.10
> Codename: wily
>
> --
> You received this bug notification because you are subscribed to
> Duplicity.
> https://bugs.launchpad.net/bugs/1548080
>
> Title:
> Revision 1165 creates "IndexError: list index out of range" on testing
> /run-tests
>
> Status in Duplicity:
> New
>
> Bug description:
> I just tried pulling down the latest code and did a ./run-test, but
> received an error "IndexError: list index out of range".
>
> Through a binary search, I found that the error was introduced in rev
> 1165:
>
> http://bazaar.launchpad.net/~duplicity-team/duplicity/0.7-series/revision/1165
>
> If I pull that down and do ./run-tests, I receive the following error
> (whereas 1164 works as expected):
> $ ./run-tests
> GLOB sdist-make: /home/aaron/Programming/duplicity/dup_rev/setup.py
> Traceback (most recent call last):
> File "/usr/local/bin/tox", line 9, in <module>
> load_entry_point('tox==1.7.2', 'console_scripts', 'tox')()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 26, in main
> retcode = Session(config).runcommand()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 310, in runcommand
> return self.subcommand_test()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 442, in subcommand_test
> sdist_path = self.sdist()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 418, in sdist
> sdist_path = self._makesdist()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 334, in _makesdist
> return self.config.distdir.listdir()[0]
> IndexError: list index out of range
>
> Can other people run the tests properly, or has this been broken for
> everyone from 1165?
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/duplicity/+bug/1548080/+subscriptions
>

I really got sideways on this one! The problem is not in makedist at all.
Going all the way back to 0.7.05 shows the same problem:

ken@dione:/tmp$ tar xzf
~/workspace/duplicity-releases/duplicity-0.7.05.tar.gz
ken@dione:/tmp$ cd duplicity-0.7.05/
ken@dione:/tmp/duplicity-0.7.05$ tox
GLOB sdist-make: /private/tmp/duplicity-0.7.05/setup.py
Traceback (most recent call last):
  File "/usr/local/bin/tox", line 11, in <module>
    sys.exit(cmdline())
  File "/usr/local/lib/python2.7/site-packages/tox/session.py", line 39, in
main
    retcode = Session(config).runcommand()
  File "/usr/local/lib/python2.7/site-packages/tox/session.py", line 375,
in runcommand
    return self.subcommand_test()
  File "/usr/local/lib/python2.7/site-packages/tox/session.py", line 520,
in subcommand_test
    path = self.get_installpkg_path()
  File "/usr/local/lib/python2.7/site-packages/tox/session.py", line 496,
in get_installpkg_path
    path = self._makesdist()
  File "/usr/local/lib/python2.7/site-packages/tox/session.py", line 399,
in _makesdist
    return self.config.distdir.listdir()[0]
IndexError: list index out of range

Not sure what's going on with this. Going to gen a clean VM and try it
after 'setup.py install'.

On Sun, Feb 21, 2016 at 11:17 AM, Kenneth Loafman <email address hidden>
wrote:

> makedist has nothing to do with running duplicity, just the build. The
> changes you made took out the change I made to get everything from bzr (or
> git) for the release. The original makedist assumed (bad!) that the
> directory you were in matched the repo. A distribution should get the code
> for the release from the repo, not from a local dir.
>
> It might suggest a path problem on your end where something is not being
> resolved properly.
>
> On Sun, Feb 21, 2016 at 10:51 AM, Aaron Whitehouse <
> <email address hidden>> wrote:
>
>> $ tox --version
>> 1.7.2 imported from
>> /usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/__init__.pyc
>>
>> $ lsb_release -a
>> No LSB modules are available.
>> Distributor ID: Ubuntu
>> Description: Ubuntu 15.10
>> Release: 15.10
>> Codename: wily
>>
>> --
>> You received this bug notification because you are subscribed to
>> Duplicity.
>> https://bugs.launchpad.net/bugs/1548080
>>
>> Title:
>> Revision 1165 creates "IndexError: list index out of range" on testing
>> /run-tests
>>
>> Status in Duplicity:
>> New
>>
>> Bug description:
>> I just tried pulling down the latest code and did a ./run-test, but
>> received an error "IndexError: list index out of range".
>>
>> Through a binary search, I found that the error was introduced in rev
>> 1165:
>>
>> http://bazaar.launchpad.net/~duplicity-team/duplicity/0.7-series/revision/1165
>>
>> If I pull that down and do ./run-tests, I receive the following error
>> (whereas 1164 works as expected):
>> $ ./run-tests
>> GLOB sdist-make: /home/aaron/Programming/duplicity/dup_rev/setup.py
>> Traceback (most recent call last):
>> File "/usr/local/bin/tox", line 9, in <module>
>> load_entry_point('tox==1.7.2', 'console_scripts', 'tox')()
>> File
>> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py...

Kind of a Catch-22 here. Both versions assume that project repos exist,
the older one to get the list of files, the new one to get the files
themselves. Problem is that in the tgz case, no project has been set up,
so no repo is available. Makedist will need to change to know the default
repo location. Tox uses setup.py which uses makedist, so I'm guessing we
need to check the environment to see if a project repo exists and use the
tgz install if not. Twitchy!

On Mon, Feb 22, 2016 at 10:02 AM, Kenneth Loafman <email address hidden>
wrote:

> I really got sideways on this one! The problem is not in makedist at
> all. Going all the way back to 0.7.05 shows the same problem:
>
> ken@dione:/tmp$ tar xzf
> ~/workspace/duplicity-releases/duplicity-0.7.05.tar.gz
> ken@dione:/tmp$ cd duplicity-0.7.05/
> ken@dione:/tmp/duplicity-0.7.05$ tox
> GLOB sdist-make: /private/tmp/duplicity-0.7.05/setup.py
> Traceback (most recent call last):
> File "/usr/local/bin/tox", line 11, in <module>
> sys.exit(cmdline())
> File "/usr/local/lib/python2.7/site-packages/tox/session.py", line 39,
> in main
> retcode = Session(config).runcommand()
> File "/usr/local/lib/python2.7/site-packages/tox/session.py", line 375,
> in runcommand
> return self.subcommand_test()
> File "/usr/local/lib/python2.7/site-packages/tox/session.py", line 520,
> in subcommand_test
> path = self.get_installpkg_path()
> File "/usr/local/lib/python2.7/site-packages/tox/session.py", line 496,
> in get_installpkg_path
> path = self._makesdist()
> File "/usr/local/lib/python2.7/site-packages/tox/session.py", line 399,
> in _makesdist
> return self.config.distdir.listdir()[0]
> IndexError: list index out of range
>
> Not sure what's going on with this. Going to gen a clean VM and try it
> after 'setup.py install'.
>
>
> On Sun, Feb 21, 2016 at 11:17 AM, Kenneth Loafman <email address hidden>
> wrote:
>
>> makedist has nothing to do with running duplicity, just the build. The
>> changes you made took out the change I made to get everything from bzr (or
>> git) for the release. The original makedist assumed (bad!) that the
>> directory you were in matched the repo. A distribution should get the code
>> for the release from the repo, not from a local dir.
>>
>> It might suggest a path problem on your end where something is not being
>> resolved properly.
>>
>> On Sun, Feb 21, 2016 at 10:51 AM, Aaron Whitehouse <
>> <email address hidden>> wrote:
>>
>>> $ tox --version
>>> 1.7.2 imported from
>>> /usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/__init__.pyc
>>>
>>> $ lsb_release -a
>>> No LSB modules are available.
>>> Distributor ID: Ubuntu
>>> Description: Ubuntu 15.10
>>> Release: 15.10
>>> Codename: wily
>>>
>>> --
>>> You received this bug notification because you are subscribed to
>>> Duplicity.
>>> https://bugs.launchpad.net/bugs/1548080
>>>
>>> Title:
>>> Revision 1165 creates "IndexError: list index out of range" on testing
>>> /run-tests
>>>
>>> Status in Duplicity:
>>> New
>>>
>>> Bug description:
>>> I just tried pulling down the latest code and did a ./run-test, but
>>> received an error "IndexError: lis...

For what it's worth, I thought I'd try with a recent version of tox, given that mine was quite old (though we probably do need to run versions that are as old as the oldest Ubuntu build server to ensure we aren't using new features).

It doesn't seem to change anything from my original report:
$ tox --version
2.3.1 imported from /usr/local/lib/python2.7/dist-packages/tox/__init__.pyc

Still the same problem:
$ ./run-tests
GLOB sdist-make: /home/aaron/Programming/duplicity/latest_dup/setup.py
Traceback (most recent call last):
  File "/usr/local/bin/tox", line 11, in <module>
    sys.exit(cmdline())
  File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 39, in main
    retcode = Session(config).runcommand()
  File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 375, in runcommand
    return self.subcommand_test()
  File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 520, in subcommand_test
    path = self.get_installpkg_path()
  File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 496, in get_installpkg_path
    path = self._makesdist()
  File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 399, in _makesdist
    return self.config.distdir.listdir()[0]
IndexError: list index out of range

and the patch fixes it:
$ patch dist/makedist ~/Downloads/Temp/makedist_reversion.diff
patching file dist/makedist

Then run-tests works as expected.

I haven't yet reviewed this enough to completely follow your comments, Kenneth, but thought I'd provide this update to rule out tox as the issue. From your playing with this, have you figured out what should fix this as a workaround? Given I tend to code TDD, this is stopping me from doing some of the coding I had planned.

When you run "./run-tests", all it does is run "tox" directly. If you have
checked this out from a repo it will work, otherwise not. The reason is
that tox uses "setup.py sdist" to make a source tarball and build a
virtualenv from that the old version uses "bzr ls" to list the files needed
and the new version uses "bzr export" to export the files needed. I wanted
the files to always come from the repo, so made the change to export
directly. Both create an empty tarball without a project repo, so for me
at least, it fails on both versions.

This setup.py was written several years ago and does not really conform to
current best practices. I'm thinking of allowing setup to operate with or
without a project repo and build a list using the setup options now
available. Without a repo, it would still test the existing code, but the
results would be "dirty" since we could not validate their provance. But,
it would allow the tarball install to run tox without a problem.

On Sat, Feb 27, 2016 at 6:53 AM, Aaron Whitehouse <
<email address hidden>> wrote:

> For what it's worth, I thought I'd try with a recent version of tox,
> given that mine was quite old (though we probably do need to run
> versions that are as old as the oldest Ubuntu build server to ensure we
> aren't using new features).
>
> It doesn't seem to change anything from my original report:
> $ tox --version
> 2.3.1 imported from /usr/local/lib/python2.7/dist-packages/tox/__init__.pyc
>
> Still the same problem:
> $ ./run-tests
> GLOB sdist-make: /home/aaron/Programming/duplicity/latest_dup/setup.py
> Traceback (most recent call last):
> File "/usr/local/bin/tox", line 11, in <module>
> sys.exit(cmdline())
> File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 39,
> in main
> retcode = Session(config).runcommand()
> File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 375,
> in runcommand
> return self.subcommand_test()
> File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 520,
> in subcommand_test
> path = self.get_installpkg_path()
> File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 496,
> in get_installpkg_path
> path = self._makesdist()
> File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 399,
> in _makesdist
> return self.config.distdir.listdir()[0]
> IndexError: list index out of range
>
> and the patch fixes it:
> $ patch dist/makedist ~/Downloads/Temp/makedist_reversion.diff
> patching file dist/makedist
>
> Then run-tests works as expected.
>
> I haven't yet reviewed this enough to completely follow your comments,
> Kenneth, but thought I'd provide this update to rule out tox as the
> issue. From your playing with this, have you figured out what should fix
> this as a workaround? Given I tend to code TDD, this is stopping me from
> doing some of the coding I had planned.
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1548080
>
> Title:
> tox / run-tests do not run from normal distribution
>
> Status in Duplicity:
> In Progress
>
> Bug description:
> I just tried pulling down the latest code...

> If you have checked this out from a repo it will work, otherwise not.
I may be misunderstanding you, Kenneth, but to clarify I am running the tests in a version of duplicity checked out of the repo and it isn't working:

$ bzr branch lp:duplicity latest_duplicity
Branched 1194 revisions.
$ cd latest_duplicity/
$ cd testing/
$ ./run-testsGLOB sdist-make: /[...]/latest_duplicity/setup.py
Traceback (most recent call last):
  File "/usr/local/bin/tox", line 11, in <module>
    sys.exit(cmdline())
  File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 39, in main
    retcode = Session(config).runcommand()
  File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 375, in runcommand
    return self.subcommand_test()
  File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 520, in subcommand_test
    path = self.get_installpkg_path()
  File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 496, in get_installpkg_path
    path = self._makesdist()
  File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 399, in _makesdist
    return self.config.distdir.listdir()[0]
IndexError: list index out of range

Take out the "cd testing/" and run it from the root as "testing/run-tests".

Guess we need a "cd .." in run-tests...

On Sat, Feb 27, 2016 at 12:50 PM, Aaron Whitehouse <
<email address hidden>> wrote:

> > If you have checked this out from a repo it will work, otherwise not.
> I may be misunderstanding you, Kenneth, but to clarify I am running the
> tests in a version of duplicity checked out of the repo and it isn't
> working:
>
> $ bzr branch lp:duplicity latest_duplicity
> Branched 1194 revisions.
> $ cd latest_duplicity/
> $ cd testing/
> $ ./run-testsGLOB sdist-make: /[...]/latest_duplicity/setup.py
> Traceback (most recent call last):
> File "/usr/local/bin/tox", line 11, in <module>
> sys.exit(cmdline())
> File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 39,
> in main
> retcode = Session(config).runcommand()
> File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 375,
> in runcommand
> return self.subcommand_test()
> File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 520,
> in subcommand_test
> path = self.get_installpkg_path()
> File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 496,
> in get_installpkg_path
> path = self._makesdist()
> File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 399,
> in _makesdist
> return self.config.distdir.listdir()[0]
> IndexError: list index out of range
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1548080
>
> Title:
> tox / run-tests do not run from normal distribution
>
> Status in Duplicity:
> In Progress
>
> Bug description:
> I just tried pulling down the latest code and did a ./run-test, but
> received an error "IndexError: list index out of range".
>
> Through a binary search, I found that the error was introduced in rev
> 1165:
>
> http://bazaar.launchpad.net/~duplicity-team/duplicity/0.7-series/revision/1165
>
> If I pull that down and do ./run-tests, I receive the following error
> (whereas 1164 works as expected):
> $ ./run-tests
> GLOB sdist-make: /home/aaron/Programming/duplicity/dup_rev/setup.py
> Traceback (most recent call last):
> File "/usr/local/bin/tox", line 9, in <module>
> load_entry_point('tox==1.7.2', 'console_scripts', 'tox')()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 26, in main
> retcode = Session(config).runcommand()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 310, in runcommand
> return self.subcommand_test()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 442, in subcommand_test
> sdist_path = self.sdist()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 418, in sdist
> sdist_path = self._makesdist()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 334, in _makesdist
> return self.config.distdir.listdir()[0]
> IndexError: list index out of range
>
> Can other people run the t...

That didn't work for me:

$ bzr branch lp:duplicity latest_duplicity
Branched 1194 revisions.
$ cd latest_duplicity/
$ testing/run-tests
GLOB sdist-make: /[...]/latest_duplicity/setup.py
Traceback (most recent call last):
  File "/usr/local/bin/tox", line 11, in <module>
    sys.exit(cmdline())
  File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 39, in main
    retcode = Session(config).runcommand()
  File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 375, in runcommand
    return self.subcommand_test()
  File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 520, in subcommand_test
    path = self.get_installpkg_path()
  File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 496, in get_installpkg_path
    path = self._makesdist()
  File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 399, in _makesdist
    return self.config.distdir.listdir()[0]
IndexError: list index out of range

Again, patching makedist with my diff above means that running from the repo root with "testing/run-tests" runs the tests as expected.

(Also, if we are changing which directory one needs to execute run-tests from, we need to change README-REPO, as that specifically says one needs to change into the testing directory.)

Yes on the 'cd testing', we'd need to document that. But that's not really
the problem.

OK, so we have some answer variability here:

On Ubuntu 14.04.4:
  - ./run-tests succeeds
  - testing/run-tests succeeds

On Mac OS X 10.11.3:
  - ./run-tests fails
  - testing/run-tests succeeds

On your system (what?):
  - ./run-tests fails
  - testing/run-tests fails

The only difference I can see is that the first two tests start with py26
and the last with py27, but if I modify run-tests to run 'tox -e py27', I
get the same results.

It has to be in the environment somewhere.

Bazaar (bzr) 2.7.0dev1
Python 2.7.6

Try this:
bzr export duplicity-0.7.06.tar.gz
from latest_duplicity/

On Sun, Feb 28, 2016 at 5:16 AM, Aaron Whitehouse <
<email address hidden>> wrote:

> That didn't work for me:
>
> $ bzr branch lp:duplicity latest_duplicity
> Branched 1194 revisions.
> $ cd latest_duplicity/
> $ testing/run-tests
> GLOB sdist-make: /[...]/latest_duplicity/setup.py
> Traceback (most recent call last):
> File "/usr/local/bin/tox", line 11, in <module>
> sys.exit(cmdline())
> File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 39,
> in main
> retcode = Session(config).runcommand()
> File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 375,
> in runcommand
> return self.subcommand_test()
> File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 520,
> in subcommand_test
> path = self.get_installpkg_path()
> File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 496,
> in get_installpkg_path
> path = self._makesdist()
> File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 399,
> in _makesdist
> return self.config.distdir.listdir()[0]
> IndexError: list index out of range
>
> Again, patching makedist with my diff above means that running from the
> repo root with "testing/run-tests" runs the tests as expected.
>
> (Also, if we are changing which directory one needs to execute run-tests
> from, we need to change README-REPO, as that specifically says one needs
> to change into the testing directory.)
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1548080
>
> Title:
> tox / run-tests do not run from normal distribution
>
> Status in Duplicity:
> In Progress
>
> Bug description:
> I just tried pulling down the latest code and did a ./run-test, but
> received an error "IndexError: list index out of range".
>
> Through a binary search, I found that the error was introduced in rev
> 1165:
>
> http://bazaar.launchpad.net/~duplicity-team/duplicity/0.7-series/revision/1165
>
> If I pull that down and do ./run-tests, I receive the following error
> (whereas 1164 works as expected):
> $ ./run-tests
> GLOB sdist-make: /home/aaron/Programming/duplicity/dup_rev/setup.py
> Traceback (most recent call last):
> File "/usr/local/bin/tox", line 9, in <module>
> load_entry_point('tox==1.7.2', 'console_scripts', 'tox')()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 26, in main
> retcode = Session(config).runcommand()
> File
...

I see the problem, and it is with the environment.

latest_duplicity/testing $ bzr export foo.tar.gz yields a tar.gz file of
size 368,889 -- just the testing dir
latest_duplicity $ bzr export foo.tar.gz yields a tar.gz file of size
1,037,248 -- the whole package

On Sun, Feb 28, 2016 at 6:52 AM, Kenneth Loafman <email address hidden>
wrote:

> Yes on the 'cd testing', we'd need to document that. But that's not
> really the problem.
>
> OK, so we have some answer variability here:
>
> On Ubuntu 14.04.4:
> - ./run-tests succeeds
> - testing/run-tests succeeds
>
> On Mac OS X 10.11.3:
> - ./run-tests fails
> - testing/run-tests succeeds
>
> On your system (what?):
> - ./run-tests fails
> - testing/run-tests fails
>
> The only difference I can see is that the first two tests start with py26
> and the last with py27, but if I modify run-tests to run 'tox -e py27', I
> get the same results.
>
> It has to be in the environment somewhere.
>
> Bazaar (bzr) 2.7.0dev1
> Python 2.7.6
>
> Try this:
> bzr export duplicity-0.7.06.tar.gz
> from latest_duplicity/
>
>
> On Sun, Feb 28, 2016 at 5:16 AM, Aaron Whitehouse <
> <email address hidden>> wrote:
>
>> That didn't work for me:
>>
>> $ bzr branch lp:duplicity latest_duplicity
>> Branched 1194 revisions.
>> $ cd latest_duplicity/
>> $ testing/run-tests
>> GLOB sdist-make: /[...]/latest_duplicity/setup.py
>> Traceback (most recent call last):
>> File "/usr/local/bin/tox", line 11, in <module>
>> sys.exit(cmdline())
>> File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 39,
>> in main
>> retcode = Session(config).runcommand()
>> File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 375,
>> in runcommand
>> return self.subcommand_test()
>> File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 520,
>> in subcommand_test
>> path = self.get_installpkg_path()
>> File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 496,
>> in get_installpkg_path
>> path = self._makesdist()
>> File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 399,
>> in _makesdist
>> return self.config.distdir.listdir()[0]
>> IndexError: list index out of range
>>
>> Again, patching makedist with my diff above means that running from the
>> repo root with "testing/run-tests" runs the tests as expected.
>>
>> (Also, if we are changing which directory one needs to execute run-tests
>> from, we need to change README-REPO, as that specifically says one needs
>> to change into the testing directory.)
>>
>> --
>> You received this bug notification because you are a bug assignee.
>> https://bugs.launchpad.net/bugs/1548080
>>
>> Title:
>> tox / run-tests do not run from normal distribution
>>
>> Status in Duplicity:
>> In Progress
>>
>> Bug description:
>> I just tried pulling down the latest code and did a ./run-test, but
>> received an error "IndexError: list index out of range".
>>
>> Through a binary search, I found that the error was introduced in rev
>> 1165:
>>
>> http://bazaar.launchpad.net/~duplicity-team/duplicity/0.7-series/revision/1165
>>
>> If I pull that down and do ./run-tests, I receive the following err...

Thanks Kenneth.

My system is Ubuntu 15.10 with Python 2.7.10

bzr export duplicity-0.7.06.tar.gz from the repo root gives a file that is 1037628 in size.
bzr export duplicity-0.7.06.tar.gz from the testing folder gives a file that is 368929 in size.

But running run-tests (or just "tox") from either of those folders fails with the IndexError.

I'll just revert the changes for now and move makedist into setup.py
eventually. Some changes around versioning will have to be made, but
otherwise it's something that needs to happen.

On Sun, Feb 28, 2016 at 11:33 AM, Aaron Whitehouse <
<email address hidden>> wrote:

> Thanks Kenneth.
>
> My system is Ubuntu 15.10 with Python 2.7.10
>
> bzr export duplicity-0.7.06.tar.gz from the repo root gives a file that is
> 1037628 in size.
> bzr export duplicity-0.7.06.tar.gz from the testing folder gives a file
> that is 368929 in size.
>
> But running run-tests (or just "tox") from either of those folders fails
> with the IndexError.
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1548080
>
> Title:
> tox / run-tests do not run from normal distribution
>
> Status in Duplicity:
> In Progress
>
> Bug description:
> I just tried pulling down the latest code and did a ./run-test, but
> received an error "IndexError: list index out of range".
>
> Through a binary search, I found that the error was introduced in rev
> 1165:
>
> http://bazaar.launchpad.net/~duplicity-team/duplicity/0.7-series/revision/1165
>
> If I pull that down and do ./run-tests, I receive the following error
> (whereas 1164 works as expected):
> $ ./run-tests
> GLOB sdist-make: /home/aaron/Programming/duplicity/dup_rev/setup.py
> Traceback (most recent call last):
> File "/usr/local/bin/tox", line 9, in <module>
> load_entry_point('tox==1.7.2', 'console_scripts', 'tox')()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 26, in main
> retcode = Session(config).runcommand()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 310, in runcommand
> return self.subcommand_test()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 442, in subcommand_test
> sdist_path = self.sdist()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 418, in sdist
> sdist_path = self._makesdist()
> File
> "/usr/local/lib/python2.7/dist-packages/tox-1.7.2-py2.7.egg/tox/_cmdline.py",
> line 334, in _makesdist
> return self.config.distdir.listdir()[0]
> IndexError: list index out of range
>
> Can other people run the tests properly, or has this been broken for
> everyone from 1165?
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/duplicity/+bug/1548080/+subscriptions
>

Changes reverted. Not sure what is going on with Ubuntu 15.10, but it's not worth chasing now.

This seems to be fixed, thanks Kenneth.

Ken,

What did you mean in Comment #17 by:
> I'll just revert the changes for now and move makedist into setup.py
> eventually. Some changes around versioning will have to be made, but
> otherwise it's something that needs to happen.

We seem to have hit this problem again in:
https://code.launchpad.net/~mgorse/duplicity/0.8-series/+merge/359864

This is not an area that I know a lot about, but I have also hit challenges with our versioning when building snaps from the code (snapcraft only works after our code has been through makedist).

Do you know what it is that is causing the problems here?

Aaron,

I really detest version files, so I made it so that makedist searches for $VERSION and changes it to the version specified in the command line. The makedist output is the only 'versioned' copy in existence. So far that only causes a minor warning when tox runs from the trunk.

...Ken