Undercloud ERROR instances cause subsequent OSCI jobs to fail
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| OpenStack Charm Test Infra |
Fix Released
|
High
|
Ryan Beisner | ||
Bug Description
Undercloud ERROR instances cause subsequent OSCI jobs to fail Edit
There are a number of undercloud and/or tooling failure conditions which cause OSCI deploy tests to fail, resulting in false failure reports for subsequent CI jobs.
Two of those conditions specifically identified and addressed in this bug are:
#1 Undercloud produces and ERROR instance and the destroy + cleanup logic fails to remove the ERROR instance, resulting in subsequent jobs on that slave also failing.
#2 Tooling and/or undercloud fail to issue or receive a `nova delete` type of operation for an instance in a Juju environment / model, resulting in an ACTIVE (non-ERROR) instance remaining in the tenant.
In both of these cases, OSCI can detect the condition and resolve the situation so that subsequent jobs/builds that execute on the same slave are not affected.
Related branches
- Andrew McLeod (community): Approve
-
Diff: 836 lines (+196/-227)19 files modified.bzrignore (+1/-0)
common/osci_utils.py (+4/-4)
job-parts/build_deploy_test.sh (+1/-1)
job-parts/build_deployer_quickcheck.sh (+1/-1)
job-parts/build_git_deploy_test.sh (+1/-1)
job-parts/build_juju_deploy_single_charm.sh (+2/-2)
job-parts/build_mojo_runner.sh (+9/-5)
job-parts/build_osi.sh (+1/-1)
job-parts/deprecated/Z_DEPRECATE_build_mojo_runner_baremetal.sh (+0/-69)
job-parts/deprecated/part_juju_cleanup.sh (+0/-22)
job-parts/juju1_bighammer_destroy.sh (+32/-27)
job-parts/juju1_deploy_ok.sh (+45/-35)
job-parts/osci_job_common.sh (+33/-16)
job-parts/osci_mojo_common.sh (+21/-35)
job-parts/trigger_by_mojo_group.sh (+6/-0)
populate/mappings.yaml (+9/-7)
tools/juju_status_health_check.py (+1/-1)
tools/requirements.txt (+10/-0)
tools/tox.ini (+19/-0)
| Ryan Beisner (1chb1n) wrote | #2 |
| Changed in charm-test-infra: | |
| assignee: | nobody → Ryan Beisner (1chb1n) |
| importance: | Undecided → High |
| status: | New → Incomplete |
| status: | Incomplete → Confirmed |
| status: | Confirmed → In Progress |
| Changed in charm-test-infra: | |
| milestone: | none → 17.05 |
| summary: |
- Undercloud periodically produces ERROR instances, OSCI jobs fail + Undercloud ERROR instances cause subsequent OSCI jobs to fail |
| description: | updated |
| Changed in charm-test-infra: | |
| status: | In Progress → Fix Released |
FYI, the linked branch also addresses https:/