adt-run does not work with autopilot on emulator

Adding emulator environment details:

current build number: 90
device name: generic_x86
channel: ubuntu-touch/devel-proposed
alias: ubuntu-touch/vivid-proposed
last update: 2015-02-02 03:18:30
version version: 90
version ubuntu: 20150202
version device: 20150129
version custom: 20150202

Hm, I'm on almost the same version:

current build number: 92
device name: generic_x86
channel: ubuntu-touch/devel-proposed
alias: ubuntu-touch/vivid-proposed
last update: 2015-02-05 03:25:51

I ran the tests like this:
bzr branch lp:ubuntu-sanity-tests
adt-run --setup-commands 'mount -o remount,ro /' -B ubuntu-sanity-tests// --- ssh -s adb

but they fail with something completely different:

adt-run [08:00:10]: test sanity: [-----------------------
The ubuntuuitoolkit.emulators module is deprecated. Import the autopilot helpers from the top-level ubuntuuitoolkit module.
Loading tests from: /tmp/adt-run.Zfp5X5/deps/usr/lib/python3/dist-packages

Traceback (most recent call last):
  File "/usr/lib/python3.4/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.4/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmp/adt-run.Zfp5X5/deps/usr/lib/python3/dist-packages/autopilot/run.py", line 748, in <module>
    main()
  File "/tmp/adt-run.Zfp5X5/deps/usr/lib/python3/dist-packages/autopilot/run.py", line 742, in main
    test_app.run()
  File "/tmp/adt-run.Zfp5X5/deps/usr/lib/python3/dist-packages/autopilot/run.py", line 628, in run
    action()
  File "/tmp/adt-run.Zfp5X5/deps/usr/lib/python3/dist-packages/autopilot/run.py", line 694, in run_tests
    test_result = test_suite.run(result)
  File "/usr/lib/python3.4/unittest/suite.py", line 122, in run
    test(result)
  File "/usr/lib/python3.4/unittest/case.py", line 625, in __call__
    return self.run(*args, **kwds)
  File "/tmp/adt-run.Zfp5X5/deps/usr/lib/python3/dist-packages/testscenarios/testcase.py", line 65, in run
    return super(WithScenarios, self).run(result)
  File "/tmp/adt-run.Zfp5X5/deps/usr/lib/python3/dist-packages/testtools/testcase.py", line 585, in run
    return self.__RunTest(self, self.exception_handlers).run(result)
  File "/tmp/adt-run.Zfp5X5/deps/usr/lib/python3/dist-packages/autopilot/testcase.py", line 132, in run
    return super().run(*args, **kwargs)
  File "/tmp/adt-run.Zfp5X5/deps/usr/lib/python3/dist-packages/testtools/runtest.py", line 74, in run
    return self._run_one(actual_result)
  File "/tmp/adt-run.Zfp5X5/deps/usr/lib/python3/dist-packages/testtools/runtest.py", line 88, in _run_one
    return self._run_prepared_result(ExtendedToOriginalDecorator(result))
  File "/tmp/adt-run.Zfp5X5/deps/usr/lib/python3/dist-packages/testtools/runtest.py", line 107, in _run_prepared_result
    handler(self.case, self.result, e)
  File "/tmp/adt-run.Zfp5X5/deps/usr/lib/python3/dist-packages/testtools/testcase.py", line 550, in _report_error
    result.addError(self, details=self.getDetails())
  File "/tmp/adt-run.Zfp5X5/deps/usr/lib/python3/dist-packages/testtools/testresult/real.py", line 1121, in addError
    return self.decorated.addError(test, err)
  File "/tmp/adt-run.Zfp5X5/deps/usr/lib/python3/dist-packages/autopilot/testresult.py", line 71, in addError
    return super(type(self), self).addError(test, err, details)
  File "/tmp/adt-run.Zfp5X5/deps/usr/lib/python3/dist-packages/testtools/testresult/real.py", line 1543, in addError
    return self.decorated.addError(test, err, details=details)
  File "/t...

When I run them on an r/w image so that the test dependencies actually get installed with apt, this now fails with

adt-run [08:20:02]: test sanity: [-----------------------
Traceback (most recent call last):
  File "/tmp/adt-run.qsFTLT/build.sGK/real-tree/debian/tests/sanity", line 23, in <module>
    from ubuntu_sanity_tests import sanity
ImportError: No module named 'ubuntu_sanity_tests'

because it doesn't set PYTHONPATH=. anywhere, and the binary is in debian/tests/. That's another reason why they currently can't work runnning from the tree instead of from the package. I fixed that with http://paste.ubuntu.com/10068247/ and now the tests actually do run. So I recommend applying this patch to lp:ubuntu-sanity-tests. The tests fail with

adt-run [08:28:52]: test sanity: [-----------------------
The ubuntuuitoolkit.emulators module is deprecated. Import the autopilot helpers from the top-level ubuntuuitoolkit module.
Loading tests from: /usr/lib/python3/dist-packages

Loading tests from: /tmp/adt-run.SVTNuP/build.0XB/real-tree

Sorry, try again.
Sorry, try again.
Sorry, try again.
sudo: 3 incorrect password attempts
adt-run [08:31:09]: test sanity: -----------------------]
adt-run [08:31:10]: test sanity: - - - - - - - - - - results - - - - - - - - - -
sanity FAIL non-zero exit status 1

But I did see quite some windows flying around; not sure what the sudo thingy is, presumably the config.py device_password key doesn't default to 0000?

But that still leaves me with not being able to reproduce your original crash. Can you please tell me the precise steps and commands that you did to get the /dev/uinput issue?

Indeed I don't get the weird error message from comment 2 with chmod 666 /dev/uinput. Can you confirm that this works for you on a fresh emulator?

  adt-run --setup-commands 'mount -o remount,ro /; chmod 666 /dev/uinput' -B ubuntu-sanity-tests// --- ssh -s adb

This unpacks the test deps and starts running the tests here, and just fails due to the sudo password problem.

This should be fixed with http://anonscm.debian.org/cgit/autopkgtest/autopkgtest.git/commit/?id=18a4492f0bc , it works for me with that. Can you confirm? You can either download http://anonscm.debian.org/cgit/autopkgtest/autopkgtest.git/plain/runner/adt-run and install it in /usr/bin/, or checkout git and call ./run-from-checkout instead of adt-run.

This bug was fixed in the package autopkgtest - 3.9.5git1

---------------
autopkgtest (3.9.5git1) vivid; urgency=medium

  * Skip test_tmp_install_imagemagick test case if python3-wand is not
    available.
 -- Martin Pitt <email address hidden> Wed, 11 Feb 2015 07:10:51 +0100