Merge ~pwlars/testflinger-agent:wait-for-status into testflinger-agent:master

[Paul Larson]

Fix for a small corner-case where we might get no exit status when a job times out.

[Sheila Miguez (codersquid)]

I had to go look up docs for Popen to check exactly what wait does. There's a note about a potential deadlock, "Note This will deadlock when using stdout=PIPE or stderr=PIPE and the child process generates enough output to a pipe such that it blocks waiting for the OS pipe buffer to accept more data. Use Popen.communicate() when using pipes to avoid that."

Is that a concern here? The note is vague on what is 'enough' output.

[Paul Larson (pwlars)]

That's a good question. You're right, it's not real clear about how much output would cause this, but to be on the safe side, I'll call communicate() first, and also add a timeout to the wait.

review: Needs Resubmitting

[Sheila Miguez (codersquid)]

I guess adding the timeout is sufficient. From what I can tell in the docs, if it got blocked beyond the timeout it would throw an exception.

review: Approve

[Paul Larson (pwlars)]

Good point. There's already a paranoid default exit of 99 for the calling function, but it doesn't account for the possibility of an exception in a nice way. I'll handle it here and have it use that default as well.

review: Needs Resubmitting

[Sheila Miguez (codersquid)]

lgtm

review: Approve

[Devices Certification Bot (ce-certification-qa)]

The merge was fine but running tests failed.

Already using interpreter /usr/bin/python3
running pytest
Searching for voluptuous
Reading https://pypi.org/simple/voluptuous/
Downloading https://files.pythonhosted.org/packages/9c/b9/84f5bd190a7dc234cea2ca1da02c8900ace982e0ad4ae494660585844f04/voluptuous-0.12.1-py3-none-any.whl#sha256=8ace33fcf9e6b1f59406bfaf6b8ec7bcc44266a9f29080b4deb4fe6ff2492386
Best match: voluptuous 0.12.1
Processing voluptuous-0.12.1-py3-none-any.whl
Installing voluptuous-0.12.1-py3-none-any.whl to /tmp/tmpk35BGu/.eggs

Installed /tmp/tmpk35BGu/.eggs/voluptuous-0.12.1-py3.5.egg
Searching for requests
Reading https://pypi.org/simple/requests/
Downloading https://files.pythonhosted.org/packages/29/c1/24814557f1d22c56d50280771a17307e6bf87b70727d975fd6b2ce6b014a/requests-2.25.1-py2.py3-none-any.whl#sha256=c210084e36a42ae6b9219e00e48287def368a26d03a048ddad7bfee44f75871e
Best match: requests 2.25.1
Processing requests-2.25.1-py2.py3-none-any.whl
Installing requests-2.25.1-py2.py3-none-any.whl to /tmp/tmpk35BGu/.eggs

Installed /tmp/tmpk35BGu/.eggs/requests-2.25.1-py3.5.egg
Searching for PyYAML
Reading https://pypi.org/simple/PyYAML/
Downloading https://files.pythonhosted.org/packages/64/c2/b80047c7ac2478f9501676c988a5411ed5572f35d1beff9cae07d321512c/PyYAML-5.3.1.tar.gz#sha256=b8eac752c5e14d3eca0e6dd9199cd627518cb5ec06add0de9d32baeee6fe645d
Best match: PyYAML 5.3.1
Processing PyYAML-5.3.1.tar.gz
Writing /tmp/easy_install-toi7_zu3/PyYAML-5.3.1/setup.cfg
Running PyYAML-5.3.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-toi7_zu3/PyYAML-5.3.1/egg-dist-tmp-kfnbse3z
Moving PyYAML-5.3.1-py3.5-linux-x86_64.egg to /tmp/tmpk35BGu/.eggs

Installed /tmp/tmpk35BGu/.eggs/PyYAML-5.3.1-py3.5-linux-x86_64.egg
Searching for urllib3<1.27,>=1.21.1
Reading https://pypi.org/simple/urllib3/
Downloading https://files.pythonhosted.org/packages/f5/71/45d36a8df68f3ebb098d6861b2c017f3d094538c0fb98fa61d4dc43e69b9/urllib3-1.26.2-py2.py3-none-any.whl#sha256=d8ff90d979214d7b4f8ce956e80f4028fc6860e4431f731ea4a8c08f23f99473
Best match: urllib3 1.26.2
Processing urllib3-1.26.2-py2.py3-none-any.whl
Installing urllib3-1.26.2-py2.py3-none-any.whl to /tmp/tmpk35BGu/.eggs

Installed /tmp/tmpk35BGu/.eggs/urllib3-1.26.2-py3.5.egg
Searching for idna<3,>=2.5
Reading https://pypi.org/simple/idna/
Downloading https://files.pythonhosted.org/packages/a2/38/928ddce2273eaa564f6f50de919327bf3a00f091b5baba8dfa9460f3a8a8/idna-2.10-py2.py3-none-any.whl#sha256=b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0
Best match: idna 2.10
Processing idna-2.10-py2.py3-none-any.whl
Installing idna-2.10-py2.py3-none-any.whl to /tmp/tmpk35BGu/.eggs

Installed /tmp/tmpk35BGu/.eggs/idna-2.10-py3.5.egg
Searching for chardet<5,>=3.0.2
Reading https://pypi.org/simple/chardet/
Downloading https://files.pythonhosted.org/packages/19/c7/fa589626997dd07bd87d9269342ccb74b1720384a4d739a1872bd84fbe68/chardet-4.0.0-py2.py3-none-any.whl#sha256=f864054d66fd9118f2e67044ac8981a54775ec5b67aed0441892edb553d21da5
Best match: chardet 4.0.0
Processing chardet-4.0.0-py2.py3-none-any.whl
Installing chardet-4.0.0-py2.py3-none-any.whl to /tmp/tmpk35BGu/.eggs

Installed /tmp/tmpk35BGu/.eggs/chardet-4.0...

review: Needs Fixing

[Devices Certification Bot (ce-certification-qa)]

The merge was fine but running tests failed.

Already using interpreter /usr/bin/python3
running pytest
Searching for voluptuous
Reading https://pypi.org/simple/voluptuous/
Downloading https://files.pythonhosted.org/packages/9c/b9/84f5bd190a7dc234cea2ca1da02c8900ace982e0ad4ae494660585844f04/voluptuous-0.12.1-py3-none-any.whl#sha256=8ace33fcf9e6b1f59406bfaf6b8ec7bcc44266a9f29080b4deb4fe6ff2492386
Best match: voluptuous 0.12.1
Processing voluptuous-0.12.1-py3-none-any.whl
Installing voluptuous-0.12.1-py3-none-any.whl to /tmp/tmp0Z6R_i/.eggs

Installed /tmp/tmp0Z6R_i/.eggs/voluptuous-0.12.1-py3.5.egg
Searching for requests
Reading https://pypi.org/simple/requests/
Downloading https://files.pythonhosted.org/packages/29/c1/24814557f1d22c56d50280771a17307e6bf87b70727d975fd6b2ce6b014a/requests-2.25.1-py2.py3-none-any.whl#sha256=c210084e36a42ae6b9219e00e48287def368a26d03a048ddad7bfee44f75871e
Best match: requests 2.25.1
Processing requests-2.25.1-py2.py3-none-any.whl
Installing requests-2.25.1-py2.py3-none-any.whl to /tmp/tmp0Z6R_i/.eggs

Installed /tmp/tmp0Z6R_i/.eggs/requests-2.25.1-py3.5.egg
Searching for PyYAML
Reading https://pypi.org/simple/PyYAML/
Downloading https://files.pythonhosted.org/packages/64/c2/b80047c7ac2478f9501676c988a5411ed5572f35d1beff9cae07d321512c/PyYAML-5.3.1.tar.gz#sha256=b8eac752c5e14d3eca0e6dd9199cd627518cb5ec06add0de9d32baeee6fe645d
Best match: PyYAML 5.3.1
Processing PyYAML-5.3.1.tar.gz
Writing /tmp/easy_install-b0xz5s00/PyYAML-5.3.1/setup.cfg
Running PyYAML-5.3.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-b0xz5s00/PyYAML-5.3.1/egg-dist-tmp-hizzz7l1
Moving PyYAML-5.3.1-py3.5-linux-x86_64.egg to /tmp/tmp0Z6R_i/.eggs

Installed /tmp/tmp0Z6R_i/.eggs/PyYAML-5.3.1-py3.5-linux-x86_64.egg
Searching for urllib3<1.27,>=1.21.1
Reading https://pypi.org/simple/urllib3/
Downloading https://files.pythonhosted.org/packages/f5/71/45d36a8df68f3ebb098d6861b2c017f3d094538c0fb98fa61d4dc43e69b9/urllib3-1.26.2-py2.py3-none-any.whl#sha256=d8ff90d979214d7b4f8ce956e80f4028fc6860e4431f731ea4a8c08f23f99473
Best match: urllib3 1.26.2
Processing urllib3-1.26.2-py2.py3-none-any.whl
Installing urllib3-1.26.2-py2.py3-none-any.whl to /tmp/tmp0Z6R_i/.eggs

Installed /tmp/tmp0Z6R_i/.eggs/urllib3-1.26.2-py3.5.egg
Searching for idna<3,>=2.5
Reading https://pypi.org/simple/idna/
Downloading https://files.pythonhosted.org/packages/a2/38/928ddce2273eaa564f6f50de919327bf3a00f091b5baba8dfa9460f3a8a8/idna-2.10-py2.py3-none-any.whl#sha256=b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0
Best match: idna 2.10
Processing idna-2.10-py2.py3-none-any.whl
Installing idna-2.10-py2.py3-none-any.whl to /tmp/tmp0Z6R_i/.eggs

Installed /tmp/tmp0Z6R_i/.eggs/idna-2.10-py3.5.egg
Searching for chardet<5,>=3.0.2
Reading https://pypi.org/simple/chardet/
Downloading https://files.pythonhosted.org/packages/19/c7/fa589626997dd07bd87d9269342ccb74b1720384a4d739a1872bd84fbe68/chardet-4.0.0-py2.py3-none-any.whl#sha256=f864054d66fd9118f2e67044ac8981a54775ec5b67aed0441892edb553d21da5
Best match: chardet 4.0.0
Processing chardet-4.0.0-py2.py3-none-any.whl
Installing chardet-4.0.0-py2.py3-none-any.whl to /tmp/tmp0Z6R_i/.eggs

Installed /tmp/tmp0Z6R_i/.eggs/chardet-4.0...

review: Needs Fixing

[Devices Certification Bot (ce-certification-qa)]

The merge was fine but running tests failed.

Already using interpreter /usr/bin/python3
running pytest
Searching for voluptuous
Reading https://pypi.org/simple/voluptuous/
Downloading https://files.pythonhosted.org/packages/9c/b9/84f5bd190a7dc234cea2ca1da02c8900ace982e0ad4ae494660585844f04/voluptuous-0.12.1-py3-none-any.whl#sha256=8ace33fcf9e6b1f59406bfaf6b8ec7bcc44266a9f29080b4deb4fe6ff2492386
Best match: voluptuous 0.12.1
Processing voluptuous-0.12.1-py3-none-any.whl
Installing voluptuous-0.12.1-py3-none-any.whl to /tmp/tmp54If7g/.eggs

Installed /tmp/tmp54If7g/.eggs/voluptuous-0.12.1-py3.5.egg
Searching for requests
Reading https://pypi.org/simple/requests/
Downloading https://files.pythonhosted.org/packages/29/c1/24814557f1d22c56d50280771a17307e6bf87b70727d975fd6b2ce6b014a/requests-2.25.1-py2.py3-none-any.whl#sha256=c210084e36a42ae6b9219e00e48287def368a26d03a048ddad7bfee44f75871e
Best match: requests 2.25.1
Processing requests-2.25.1-py2.py3-none-any.whl
Installing requests-2.25.1-py2.py3-none-any.whl to /tmp/tmp54If7g/.eggs

Installed /tmp/tmp54If7g/.eggs/requests-2.25.1-py3.5.egg
Searching for PyYAML
Reading https://pypi.org/simple/PyYAML/
Downloading https://files.pythonhosted.org/packages/64/c2/b80047c7ac2478f9501676c988a5411ed5572f35d1beff9cae07d321512c/PyYAML-5.3.1.tar.gz#sha256=b8eac752c5e14d3eca0e6dd9199cd627518cb5ec06add0de9d32baeee6fe645d
Best match: PyYAML 5.3.1
Processing PyYAML-5.3.1.tar.gz
Writing /tmp/easy_install-lzk9lopq/PyYAML-5.3.1/setup.cfg
Running PyYAML-5.3.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-lzk9lopq/PyYAML-5.3.1/egg-dist-tmp-lmfri2b8
Moving PyYAML-5.3.1-py3.5-linux-x86_64.egg to /tmp/tmp54If7g/.eggs

Installed /tmp/tmp54If7g/.eggs/PyYAML-5.3.1-py3.5-linux-x86_64.egg
Searching for urllib3<1.27,>=1.21.1
Reading https://pypi.org/simple/urllib3/
Downloading https://files.pythonhosted.org/packages/f5/71/45d36a8df68f3ebb098d6861b2c017f3d094538c0fb98fa61d4dc43e69b9/urllib3-1.26.2-py2.py3-none-any.whl#sha256=d8ff90d979214d7b4f8ce956e80f4028fc6860e4431f731ea4a8c08f23f99473
Best match: urllib3 1.26.2
Processing urllib3-1.26.2-py2.py3-none-any.whl
Installing urllib3-1.26.2-py2.py3-none-any.whl to /tmp/tmp54If7g/.eggs

Installed /tmp/tmp54If7g/.eggs/urllib3-1.26.2-py3.5.egg
Searching for idna<3,>=2.5
Reading https://pypi.org/simple/idna/
Downloading https://files.pythonhosted.org/packages/a2/38/928ddce2273eaa564f6f50de919327bf3a00f091b5baba8dfa9460f3a8a8/idna-2.10-py2.py3-none-any.whl#sha256=b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0
Best match: idna 2.10
Processing idna-2.10-py2.py3-none-any.whl
Installing idna-2.10-py2.py3-none-any.whl to /tmp/tmp54If7g/.eggs

Installed /tmp/tmp54If7g/.eggs/idna-2.10-py3.5.egg
Searching for chardet<5,>=3.0.2
Reading https://pypi.org/simple/chardet/
Downloading https://files.pythonhosted.org/packages/19/c7/fa589626997dd07bd87d9269342ccb74b1720384a4d739a1872bd84fbe68/chardet-4.0.0-py2.py3-none-any.whl#sha256=f864054d66fd9118f2e67044ac8981a54775ec5b67aed0441892edb553d21da5
Best match: chardet 4.0.0
Processing chardet-4.0.0-py2.py3-none-any.whl
Installing chardet-4.0.0-py2.py3-none-any.whl to /tmp/tmp54If7g/.eggs

Installed /tmp/tmp54If7g/.eggs/chardet-4.0...

review: Needs Fixing

[Devices Certification Bot (ce-certification-qa)]

The merge was fine but running tests failed.

Already using interpreter /usr/bin/python3
running pytest
Searching for voluptuous
Reading https://pypi.org/simple/voluptuous/
Downloading https://files.pythonhosted.org/packages/9c/b9/84f5bd190a7dc234cea2ca1da02c8900ace982e0ad4ae494660585844f04/voluptuous-0.12.1-py3-none-any.whl#sha256=8ace33fcf9e6b1f59406bfaf6b8ec7bcc44266a9f29080b4deb4fe6ff2492386
Best match: voluptuous 0.12.1
Processing voluptuous-0.12.1-py3-none-any.whl
Installing voluptuous-0.12.1-py3-none-any.whl to /tmp/tmpgg7OpS/.eggs

Installed /tmp/tmpgg7OpS/.eggs/voluptuous-0.12.1-py3.5.egg
Searching for requests
Reading https://pypi.org/simple/requests/
Downloading https://files.pythonhosted.org/packages/29/c1/24814557f1d22c56d50280771a17307e6bf87b70727d975fd6b2ce6b014a/requests-2.25.1-py2.py3-none-any.whl#sha256=c210084e36a42ae6b9219e00e48287def368a26d03a048ddad7bfee44f75871e
Best match: requests 2.25.1
Processing requests-2.25.1-py2.py3-none-any.whl
Installing requests-2.25.1-py2.py3-none-any.whl to /tmp/tmpgg7OpS/.eggs

Installed /tmp/tmpgg7OpS/.eggs/requests-2.25.1-py3.5.egg
Searching for PyYAML
Reading https://pypi.org/simple/PyYAML/
Downloading https://files.pythonhosted.org/packages/64/c2/b80047c7ac2478f9501676c988a5411ed5572f35d1beff9cae07d321512c/PyYAML-5.3.1.tar.gz#sha256=b8eac752c5e14d3eca0e6dd9199cd627518cb5ec06add0de9d32baeee6fe645d
Best match: PyYAML 5.3.1
Processing PyYAML-5.3.1.tar.gz
Writing /tmp/easy_install-e5roa89d/PyYAML-5.3.1/setup.cfg
Running PyYAML-5.3.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-e5roa89d/PyYAML-5.3.1/egg-dist-tmp-1qhy_cgs
Moving PyYAML-5.3.1-py3.5-linux-x86_64.egg to /tmp/tmpgg7OpS/.eggs

Installed /tmp/tmpgg7OpS/.eggs/PyYAML-5.3.1-py3.5-linux-x86_64.egg
Searching for urllib3<1.27,>=1.21.1
Reading https://pypi.org/simple/urllib3/
Downloading https://files.pythonhosted.org/packages/f5/71/45d36a8df68f3ebb098d6861b2c017f3d094538c0fb98fa61d4dc43e69b9/urllib3-1.26.2-py2.py3-none-any.whl#sha256=d8ff90d979214d7b4f8ce956e80f4028fc6860e4431f731ea4a8c08f23f99473
Best match: urllib3 1.26.2
Processing urllib3-1.26.2-py2.py3-none-any.whl
Installing urllib3-1.26.2-py2.py3-none-any.whl to /tmp/tmpgg7OpS/.eggs

Installed /tmp/tmpgg7OpS/.eggs/urllib3-1.26.2-py3.5.egg
Searching for idna<3,>=2.5
Reading https://pypi.org/simple/idna/
Downloading https://files.pythonhosted.org/packages/a2/38/928ddce2273eaa564f6f50de919327bf3a00f091b5baba8dfa9460f3a8a8/idna-2.10-py2.py3-none-any.whl#sha256=b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0
Best match: idna 2.10
Processing idna-2.10-py2.py3-none-any.whl
Installing idna-2.10-py2.py3-none-any.whl to /tmp/tmpgg7OpS/.eggs

Installed /tmp/tmpgg7OpS/.eggs/idna-2.10-py3.5.egg
Searching for chardet<5,>=3.0.2
Reading https://pypi.org/simple/chardet/
Downloading https://files.pythonhosted.org/packages/19/c7/fa589626997dd07bd87d9269342ccb74b1720384a4d739a1872bd84fbe68/chardet-4.0.0-py2.py3-none-any.whl#sha256=f864054d66fd9118f2e67044ac8981a54775ec5b67aed0441892edb553d21da5
Best match: chardet 4.0.0
Processing chardet-4.0.0-py2.py3-none-any.whl
Installing chardet-4.0.0-py2.py3-none-any.whl to /tmp/tmpgg7OpS/.eggs

Installed /tmp/tmpgg7OpS/.eggs/chardet-4.0...

review: Needs Fixing

diff --git a/.pmr-merge-hook b/.pmr-merge-hook
index c1ce1ee..394df16 100755
--- a/.pmr-merge-hook
+++ b/.pmr-merge-hook
@@ -3,6 +3,6 @@
 set -e
 
 rm -rf tfenv
-virtualenv -qp python3 tfenv
+virtualenv -q -p python3 tfenv
 . tfenv/bin/activate
 ./setup.py test
diff --git a/testflinger_agent/job.py b/testflinger_agent/job.py
index 7c0cd73..b76dfdc 100644
--- a/testflinger_agent/job.py
+++ b/testflinger_agent/job.py
@@ -187,7 +187,11 @@ class TestflingerJob:
                 f.write(buf)
             if live_output_buffer:
                 self.client.post_live_output(self.job_id, live_output_buffer)
-            return process.returncode
+            try:
+                status = process.wait(10)  # process.returncode
+            except TimeoutError:
+                status = 99  # Default in case something goes wrong
+            return status
 
     def get_global_timeout(self):
         """Get the global timeout for the test run in seconds