Merge lp:~nskaggs/reminders-app/autopkgtest into lp:reminders-app

[Nicholas Skaggs]

Add autopkgtest runner support

Add autopkgtest runner support

[Nicholas Skaggs (nskaggs)]

Unsure why this is failing; I get

badpkg: got 3 lines of results from extract where 5 expected
adt-run [14:29:56]: ERROR: erroneous package: got 3 lines of results from extract where 5 expected

[David Planella (dpm)]

Just out of interest, why is the syntax:

"depends": ["python3-dbus,python3-dbusmock,python3-fixtures,python3-oauthlib,python3-requests-oauthlib"]

instead of:

"depends": ["python3-dbus", "python3-dbusmock", "python3-fixtures", "python3-oauthlib", "python3-requests-oauthlib"]

review: Needs Information

[Nicholas Skaggs (nskaggs)]

because I did it wrong. thanks dpm :-)

[Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot)]

PASSED: Continuous integration, rev:183
http://91.189.93.70:8080/job/reminders-app-ci/462/
Executed test runs:
    SUCCESS: http://91.189.93.70:8080/job/generic-mediumtests-utopic/1036
        deb: http://91.189.93.70:8080/job/generic-mediumtests-utopic/1036/artifact/work/output/*zip*/output.zip
    SUCCESS: http://91.189.93.70:8080/job/reminders-app-utopic-amd64-ci/158

Click here to trigger a rebuild:
http://91.189.93.70:8080/job/reminders-app-ci/462/rebuild

review: Approve (continuous-integration)

[Nicholas Skaggs (nskaggs)]

So, you'll need to build a local click and then pass that to adt-run in order for this to run.

However, it never seems to execute tests, sitting for hours instead doing nothing, until I break the process and end it.

Loading tests from: /tmp/adt-run.Itowjr/tree/tests/autopilot

Tests running...
^Cadt-run [14:24:41]: ERROR: unexpected error:
Traceback (most recent call last):
  File "/usr/bin/adt-run", line 1981, in main
    process_actions()
  File "/usr/bin/adt-run", line 1944, in process_actions
    run_tests(act.tests, act.tests_tree)
  File "/usr/bin/adt-run", line 1378, in run_tests
    testbed.run_test(tree, t)
  File "/usr/bin/adt-run", line 1202, in run_test
    rc = self.execute(test_argv, kind='test')[0]
  File "/usr/bin/adt-run", line 835, in execute
    (out, err) = proc.communicate()
  File "/usr/lib/python3.4/subprocess.py", line 951, in communicate
    self.wait()
  File "/usr/lib/python3.4/subprocess.py", line 1565, in wait
    (pid, sts) = self._try_wait(0)
  File "/usr/lib/python3.4/subprocess.py", line 1513, in _try_wait
    (pid, sts) = _eintr_retry_call(os.waitpid, self.pid, wait_flags)
  File "/usr/lib/python3.4/subprocess.py", line 491, in _eintr_retry_call
    return func(*args)
KeyboardInterrupt
adt-run: DBG: testbed stop
adt-run: DBG: testbed close, scratch=/tmp/adt-run.Itowjr
adt-run: DBG: sending command to testbed: close
adt-run: DBG: bomb unexpected eof from the testbed
adt-run: DBG: testbed stop
adt-run [14:24:42]: ERROR:
adt-run: error cleaning up:

adt-run [14:24:42]: ERROR: testbed failed: unexpected eof from the testbed

[Nicholas Skaggs (nskaggs)]

review: Needs Fixing

[Nicholas Skaggs (nskaggs)]

Scratch that, it's something with my local setup. This should be able to merge.

review: Approve

[Francis Ginther (fginther)]

I created a click package from this branch (merged with trunk) here:
https://jenkins.qa.ubuntu.com/job/generic-click-builder-trusty-armhf/141/

Whenever I attempt to run the tests with adt, it just hangs. Before this MP the tests would fail to run due to missing dependencies.

When I try to run the app from the apps scope, it opens but tells me to add an account (I don't have one). Do the tests rely on an account being configured?

[Nicholas Skaggs (nskaggs)]

Francis, it appears like indeed it wasn't just me and my setup then; adt-run appears to setup everything but then the tests never run for me. Sounds like the same for you. Jibel was thinking it might be a unity crash or something else. I'll try looking at it further.

The tests don't require an account; they setup and utilize test accounts during the tests.

review: Needs Fixing

[Nicholas Skaggs (nskaggs)]

For the curious, here's an example run:

http://paste.ubuntu.com/7815272/

[Martin Pitt (pitti)]

When running autopilot with -v (you can add it to "autopilot_module" and use --override-control, or run under ADT_AUTOPILOT_MODULE="-v reminders" (but *shht*, that's a temporary hack), I get:

adt-run [16:55:04]: test autopilot: [-----------------------
14:55:08.333 INFO run:229 - Autopilot Source Version: 1.5.0
Autopilot Package Version: 1.5.0+14.10.20140613-0ubuntu1
14:55:08.449 WARNING emulators:26 - The ubuntuuitoolkit.emulators module is deprecated. Import the autopilot helpers from the top-level ubuntuuitoolkit module.
14:55:20.196 INFO testresult:46 - OK: reminders.tests.test_evernote.EvernoteTestCase.test_expunge_notebook_must_permanently_remove_it
14:55:31.256 INFO testresult:46 - OK: reminders.tests.test_evernote.EvernoteTestCase.test_expunge_notebook_by_name_must_permanently_remove_it
14:55:31.381 INFO globals:59 - ************************************************************
14:55:31.383 INFO globals:60 - Starting test reminders.tests.test_credentials.EvernoteCredentialsTestCase.test_add_evernote_account_must_set_provider
14:55:31.387 WARNING testcase:112 - No tracing available - install the python-autopilot-trace package!
Loading tests from: /tmp/adt-run.2V6juQ/tree/tests/autopilot

Tests running...
14:55:31.650 WARNING testcase:153 - Process manager backend unavailable, application snapshot support disabled.
14:55:31.702 INFO logging:45 - str: Set the value of an initctl environment variable. Arguments ('/tmp/tmpzuqxlzjk/.config',). Keyword arguments: {}.
14:55:31.753 INFO logging:45 - str: Set the value of an initctl environment variable. Arguments ('/tmp/tmpzuqxlzjk',). Keyword arguments: {}.
14:55:31.771 DEBUG __init__:131 - Patched home to fake home directory /tmp/tmpzuqxlzjk
14:55:31.798 DEBUG credentials:59 - Add an evernote account.
14:55:31.804 DEBUG credentials:83 - Creating the Evernote account.
14:55:32.007 DEBUG credentials:111 - Setting credentials to account.
14:55:32.044 DEBUG credentials:122 - Processing session.

So autopilot is running, but there's nothing visible on the screen, and there's no running "reminders" app. I wonder if that's due to the broken .desktop file? It has "Exec=reminders" which most likely works for a deb, but not for a click? There is no "reminders" executable, but then again I don't quite understand how ubuntu-app-launch really works (it doesn't seem to use the .desktop files, even though it creates symlinks to them).

[Martin Pitt (pitti)]

When I run the test directly on the phone:

dpkg-deb -x ubuntu-ui-toolkit-autopilot_0.1.49+14.10.20140715-0ubuntu1_armhf.deb /tmp/x
PYTHONPATH=.:/tmp/x/usr/lib/python python3 -m autopilot.run run -v reminders

I get

15:12:31.398 DEBUG credentials:83 - Creating the Evernote account.
15:12:31.576 DEBUG credentials:111 - Setting credentials to account.
15:12:31.604 DEBUG credentials:122 - Processing session.
Traceback (most recent call last):
  File "/tmp/tests/autopilot/reminders/credentials.py", line 133, in _process_session
    session = Signon.AuthSession.new(identity, method)
TypeError: Argument 1 does not allow None as a value

I didn't see that in my previous run with launching autopilot from "outside". This might be a side effect of using phablet-shell?

[Martin Pitt (pitti)]

I now painstakingly got phablet-test-run to run (manually unpacked all dependencies into ~/autopilot), and I get exactly the same hang/result.

[Nicholas Skaggs (nskaggs)]

For the traceback:

  File "/tmp/tests/autopilot/reminders/credentials.py", line 133, in _process_session
    session = Signon.AuthSession.new(identity, method)
TypeError: Argument 1 does not allow None as a value

We should catch this error and be more explicit about it. I believe this is simply an issue of our fake account not being created properly.

Why this is magically occurring again is unknown.

[Martin Pitt (pitti)]

> Unsure why this is failing; I get
>
> badpkg: got 3 lines of results from extract where 5 expected
> adt-run [14:29:56]: ERROR: erroneous package: got 3 lines of results from
> extract where 5 expected

I bet you forgot --click. I filed bug 1346388, this is a really bad error message.

[Nicholas Skaggs (nskaggs)]

For the traceback:

  File "/tmp/tests/autopilot/reminders/credentials.py", line 133, in _process_session
    session = Signon.AuthSession.new(identity, method)
TypeError: Argument 1 does not allow None as a value

We should catch this error and be more explicit about it. I believe this is simply an issue of our fake account not being created properly.

Why this is magically occurring again is unknown.

[Nicholas Skaggs (nskaggs)]

I've opened this bug for the issue. This MP however should be considered mergable.

https://bugs.launchpad.net/reminders-app/+bug/1347905

[David Planella (dpm)]

Top-approving after a call with Nick yesterday.

review: Approve

=== modified file 'manifest.json'
--- manifest.json	2014-07-14 07:38:42 +0000
+++ manifest.json	2014-07-16 15:30:22 +0000
@@ -17,7 +17,10 @@
         "vcs-bzr": "lp:reminders-app",
         "vcs-bzr-revno": "@BZR_REVNO@"
     },
-    "x-test": {
-        "autopilot": "reminders"
-    }
-}
\ No newline at end of file
+   "x-test": {
+           "autopilot": {
+                "autopilot_module": "@AUTOPILOT_DIR@",
+                "depends": ["python3-dbus", "python3-dbusmock", "python3-fixtures", "python3-oauthlib", "python3-requests-oauthlib"]
+            }
+   }
+}