Merge lp:~canonical-platform-qa/snappy-ecosystem-tests/dont-install-chromedriver into lp:snappy-ecosystem-tests

[Omer Akram]

* Don't install chromedriver as it requires sudo in jenkins.
* Point to new staging web store.

Don't install chromedriver in jenkins when run_system_tests is run as it requires sudo, which causes chrome/chromium to not start.

I have permanently installed chromedriver_installer in jenkins, so we don't really need to install it again.

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:53
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/324/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2286/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/324/rebuild

review: Approve (continuous-integration)

[lp:~canonical-platform-qa/snappy-ecosystem-tests/dont-install-chromedriver]

54. By Omer Akram on 2017-04-03

Skip store login tests as they are not working on jenkins

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:54
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/325/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2287/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/325/rebuild

review: Approve (continuous-integration)

[lp:~canonical-platform-qa/snappy-ecosystem-tests/dont-install-chromedriver]

55. By Omer Akram on 2017-04-03

Try to add proxy for chrome selenium test

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:55
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/326/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2288/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/326/rebuild

review: Approve (continuous-integration)

[lp:~canonical-platform-qa/snappy-ecosystem-tests/dont-install-chromedriver]

56. By Omer Akram on 2017-04-03

update:

57. By Omer Akram on 2017-04-03

more fixes

58. By Omer Akram on 2017-04-03

s

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:57
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/327/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2289/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/327/rebuild

review: Approve (continuous-integration)

[lp:~canonical-platform-qa/snappy-ecosystem-tests/dont-install-chromedriver]

59. By Omer Akram on 2017-04-03

debugging

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:59
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/328/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2290/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/328/rebuild

review: Approve (continuous-integration)

[lp:~canonical-platform-qa/snappy-ecosystem-tests/dont-install-chromedriver]

60. By Omer Akram on 2017-04-03

Another attempt

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:60
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/329/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2291/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/329/rebuild

review: Approve (continuous-integration)

[lp:~canonical-platform-qa/snappy-ecosystem-tests/dont-install-chromedriver]

61. By Omer Akram on 2017-04-03

no proxy

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:61
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/330/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2292/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/330/rebuild

review: Approve (continuous-integration)

[lp:~canonical-platform-qa/snappy-ecosystem-tests/dont-install-chromedriver]

62. By Omer Akram on 2017-04-03

Update url

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:62
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/331/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2293/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/331/rebuild

review: Approve (continuous-integration)

[lp:~canonical-platform-qa/snappy-ecosystem-tests/dont-install-chromedriver]

63. By Omer Akram on 2017-04-03

Next debug

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:63
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/332/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2294/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/332/rebuild

review: Approve (continuous-integration)

[lp:~canonical-platform-qa/snappy-ecosystem-tests/dont-install-chromedriver]

64. By Omer Akram on 2017-04-03

Lets try this

65. By Omer Akram on 2017-04-03

remove unused code

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:65
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/333/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2295/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/333/rebuild

review: Approve (continuous-integration)

[Omer Akram (om26er)]

Test results here: https://platform-qa-jenkins.ubuntu.com/job/run_snappy_ecosystem_tests/153/artifact/snappy-ecosystem-tests/snappy-ecosystem-results.xml/*view*/

Note the failing test is because the staging store is *broken*, I reported a bug for it https://bugs.launchpad.net/snapstore/+bug/1679254

[Heber Parrucci (heber013)]

The removal of chromedriver_installer is ok for me. However, I think that installing the drivers by hand in each host is not a good solution.
I would think in a way to automate it, it might be done as part of the setup process as well.

For example:

1 - Install pre-requisites if any.
2 - Download the driver binary to a specific folder. Uncompress if necessary.
3 - Move the binary to the needed locations: '/usr/bin' and/or '/usr/local/share/'

The same could apply to other drivers as well.

review: Needs Information

[Omer Akram (om26er)]

You are right, but I believe the whole story of installing chromedriver is currently broken. As we are not installing chromium-browser, selenium or xvfb in an automated way, so chromedriver will get added to that list as well. I will create a trello card to fix that.

Selenium can be added to the requirements.txt

for others something like this would work:

tempdir=$(mktemp -d)
cd $tempdir
apt update
apt install chromium-browser xvfb unzip -y
latest_version=$(wget -qO - https://chromedriver.storage.googleapis.com/LATEST_RELEASE | xargs)
arch=$(getconf LONG_BIT)
filename=chromedriver_linux${arch}.zip
wget https://chromedriver.storage.googleapis.com/${latest_version}/${filename}
unzip ${filename}
sudo cp chromedriver /usr/local/bin/

[Heber Parrucci (heber013)]

I am ok landing this if the card is created for handling the installation of drivers automatically.
Also we need to take into account that running the tests headless should be an option, not a mandatory thing. In the jenkins slave we cannot run on real browser for the lack of u7 and u8. But we should be able to setup a remote hub running the selenium server in another host. We can discuss if further offline.

review: Approve

[Omer Akram (om26er)]

Card created. Also note these tests are only running headless in jenkins environment as they are executed with xvfb-run, on your local setup it should fireup your browser.

=== modified file 'requirements.txt'
--- requirements.txt	2017-03-01 21:51:29 +0000
+++ requirements.txt	2017-04-03 17:49:54 +0000
@@ -18,4 +18,4 @@
 pyyaml
 retrying
 paramiko
-chromedriver_installer
+#chromedriver_installer
 
=== modified file 'snappy_ecosystem_tests/configs/ecosystem_tests.cfg'
--- snappy_ecosystem_tests/configs/ecosystem_tests.cfg	2017-02-22 17:41:52 +0000
+++ snappy_ecosystem_tests/configs/ecosystem_tests.cfg	2017-04-03 17:49:54 +0000
@@ -10,7 +10,7 @@
 upload=https://upload.apps.staging.ubuntu.com/
 sso=https://login.staging.ubuntu.com/api/v2/
 search=https://search.apps.staging.ubuntu.com/
-web=https://myapps.developer.staging.ubuntu.com/
+web=https://dashboard.staging.snapcraft.io/
 
 [production_urls]
 root_api=https://myapps.developer.ubuntu.com/dev/api/
 
=== modified file 'snappy_ecosystem_tests/helpers/web_test_base.py'
--- snappy_ecosystem_tests/helpers/web_test_base.py	2017-02-15 19:14:19 +0000
+++ snappy_ecosystem_tests/helpers/web_test_base.py	2017-04-03 17:49:54 +0000
@@ -154,8 +154,7 @@
         self.wait_for_visible()
         logout_link = driver.find_element_by_link_text("Log out")
         logout_link.click()
-
-        self.wait_for_page_title("Sign in to see your packages")
+        self.wait_for_page_title("You have been logged out")
 
     def tearDown(self):
         super().tearDown()
 
=== modified file 'snappy_ecosystem_tests/tests/test_store_login.py'
--- snappy_ecosystem_tests/tests/test_store_login.py	2017-03-08 17:29:20 +0000
+++ snappy_ecosystem_tests/tests/test_store_login.py	2017-04-03 17:49:54 +0000
@@ -46,5 +46,4 @@
         # the dropdown arrow
         self.logout()
         self.assertEqual(
-            self.driver.title, "Sign in to see your packages",
-            "Failed to logout")
+            self.driver.title, "You have been logged out", "Failed to logout")