Merge ~bladernr/plainbox-provider-checkbox:add-syslog-calltrace-check into plainbox-provider-checkbox:master

[Jeff Lane ]

Add new job to miscellanea to check syslog for call traces. Intended to be the last job run after testing just as a marker to say "Hey, something caused an oops!"

[Maciej Kisielewski (kissiel)]

+0.99. manage.py will complain (or advice) to use `flags: preserve-locale`.
Landable, just consider adding that notorious flag ;).

review: Approve

[Jeff Lane  (bladernr)]

Flag added, and added an attachment job too...

[Jeff Lane  (bladernr)]

Can you doublecheck that the attachment job makes sense?

review: Needs Resubmitting

[Maciej Kisielewski (kissiel)]

I don't like the idea of having two jobs doing the same thing, really.
If you insist on having two, I recommend adding 'depends: ' for the attachment one.

review: Needs Fixing

[Jeff Lane  (bladernr)]

On Tue, Aug 22, 2017 at 1:13 PM, Maciej Kisielewski
<email address hidden> wrote:
> Review: Needs Fixing
>
> I don't like the idea of having two jobs doing the same thing, really.

In a perfect world, I would just have the one job, and the attachment
job would be much more simple, and only run if the parent job fails.
Unfortunately, checkbox doesn't have that ability.

> If you insist on having two, I recommend adding 'depends: ' for the attachment one.

If I add a depends, the attachment job will only run when the check
job passes. And the check job passes when there are no Call Trace
dumps present. So that would have the opposite effect of what is
desired here (that the syslog be attached when the Call Trace check
fails.

Unfortunately, I can't use just the attachment job, because it will
never output into the results summary (e.g. there will never be a line
item in the test results that says "attach-syslog: FAIL: Call Trace
dumps detected, syslog has been attached."

I also don't want it to attach syslog needlessly on every submission.
This was a compromise.

Likewise, I can't change call-trace-check to PASS if call traces are
detected, because call traces in syslog are a fail condition, not a
success condition. That just creates a false positive (or is that a
positive negative?).

And I don't want to just cat syslog in call-trace-check itself,
polluting the results table with potentially hundreds or thousands of
lines of text that are better off as a physical attachment.

That said, this bug would address the issue nicely:
https://bugs.launchpad.net/checkbox-ng/+bug/1712352

> --
> https://code.launchpad.net/~bladernr/plainbox-provider-checkbox/+git/plainbox-provider-checkbox/+merge/329229
> You are the owner of ~bladernr/plainbox-provider-checkbox:add-syslog-calltrace-check.
>
> Launchpad-Message-Rationale: Owner
> Launchpad-Message-For: bladernr
> Launchpad-Notification-Type: code-review
> Launchpad-Branch: ~bladernr/plainbox-provider-checkbox/+git/plainbox-provider-checkbox:add-syslog-calltrace-check
> Launchpad-Project: plainbox-provider-checkbox

--
"Entropy isn't what it used to be."

Jeff Lane -
Server Certification Lead, Warrior Poet, Biker, Lover of Pie
Phone: [phone]
Ubuntu Ham: W4KDH Freenode IRC: bladernr or bladernr_
gpg: 1024D/3A14B2DD 8C88 B076 0DD7 B404 1417 C466 4ABD 3635 3A14 B2DD

[Maciej Kisielewski (kissiel)]

> On Tue, Aug 22, 2017 at 1:13 PM, Maciej Kisielewski
> <email address hidden> wrote:
> > Review: Needs Fixing
> >
> > I don't like the idea of having two jobs doing the same thing, really.
>
> In a perfect world, I would just have the one job, and the attachment
> job would be much more simple, and only run if the parent job fails.
> Unfortunately, checkbox doesn't have that ability.
>
> > If you insist on having two, I recommend adding 'depends: ' for the
> attachment one.
>
> If I add a depends, the attachment job will only run when the check
> job passes. And the check job passes when there are no Call Trace
> dumps present. So that would have the opposite effect of what is
> desired here (that the syslog be attached when the Call Trace check
> fails.
>
> Unfortunately, I can't use just the attachment job, because it will
> never output into the results summary (e.g. there will never be a line
> item in the test results that says "attach-syslog: FAIL: Call Trace
> dumps detected, syslog has been attached."
>
> I also don't want it to attach syslog needlessly on every submission.
> This was a compromise.
>
> Likewise, I can't change call-trace-check to PASS if call traces are
> detected, because call traces in syslog are a fail condition, not a
> success condition. That just creates a false positive (or is that a
> positive negative?).
>
> And I don't want to just cat syslog in call-trace-check itself,
> polluting the results table with potentially hundreds or thousands of
> lines of text that are better off as a physical attachment.
>
> That said, this bug would address the issue nicely:
> https://bugs.launchpad.net/checkbox-ng/+bug/1712352
>
>
>
>
> > --
> > https://code.launchpad.net/~bladernr/plainbox-provider-checkbox/+git
> /plainbox-provider-checkbox/+merge/329229
> > You are the owner of ~bladernr/plainbox-provider-checkbox:add-syslog-
> calltrace-check.
> >
> > Launchpad-Message-Rationale: Owner
> > Launchpad-Message-For: bladernr
> > Launchpad-Notification-Type: code-review
> > Launchpad-Branch: ~bladernr/plainbox-provider-checkbox/+git/plainbox-
> provider-checkbox:add-syslog-calltrace-check
> > Launchpad-Project: plainbox-provider-checkbox
>
>
>
> --
> "Entropy isn't what it used to be."
>
> Jeff Lane -
> Server Certification Lead, Warrior Poet, Biker, Lover of Pie
> Phone: [phone]
> Ubuntu Ham: W4KDH Freenode IRC: bladernr or bladernr_
> gpg: 1024D/3A14B2DD 8C88 B076 0DD7 B404 1417 C466 4ABD 3635 3A14 B2DD

Woderful explanation, thank you.
I changed my mind. +1

review: Approve

diff --git a/jobs/miscellanea.txt.in b/jobs/miscellanea.txt.in
index 6dec07d..c9434c1 100644
--- a/jobs/miscellanea.txt.in
+++ b/jobs/miscellanea.txt.in
@@ -354,3 +354,21 @@ user: root
 command:
  SOSFILE=`ls -t $PLAINBOX_SESSION_SHARE/sosreport*xz | head -1`; [ -e ${SOSFILE} ] && base64 $SOSFILE
 _summary: Attach the baseline sosreport file
+
+plugin: shell
+category_id: com.canonical.plainbox::miscellanea
+estimated_duration: 0.5
+id: miscellanea/call-trace-check
+command: if [ -n "$(grep "Call Trace:" /var/log/syslog)" ]; then echo "Call Trace detected in syslog"; false; else true; fi
+_summary: Check syslog for call traces
+_description: Checks syslog for call traces after testing is complete.
+flags: preserve-locale
+
+plugin: attachment
+category_id: com.canonical.plainbox::miscellanea
+estimated_duration: 0.5
+id: miscellanea/attach-syslog
+command: if [ -n "$(grep "Call Trace:" /var/log/syslog)" ]; then cat /var/log/syslog; fi
+_summary: Attach syslog if Call Trace appears
+_description: Attaches a copy of syslog if call traces are present after testing is complete.
+flags: preserve-locale