Code review comment for lp:~al-maisan/launchpad/duration-485524

Jonathan Lange wrote:
> Review: Needs Information
> Hello Muharem,
>
> I've tried to find you at UDS, but in any case it might be good to
> talk about this so that folks at home can play along too.
Very true.

> On IRC earlier, we talked a little about possibly moving this column
> to the Job table -- something that I thought was a sensible sounding
> idea but wanted a use case. Here, you move the estimated_duration
> column to the BuildQueue instead.
Hmm .. I was suggesting to move it to the Job table but others (bigjools,
mwhudson) were not unsure whether an estimated job duration and/or
dispatch time is something that applies to any infrastructure that
utilises the Job table or to the (Soyuz) build farm only.
Due to that we thought it's best to contain the change to the
(Soyuz) build farm. Since the `BuildQueue` table is used by all build
farm jobs it was the natural place to put that property.

> This, as I understand it, makes the estimated duration a property of a
> build a property of how it is scheduled, rather than an intrinsic part
> of the definition of the task. However, I feel my understanding is
> uncertain.
This is about the estimation of job dispatch times only. The question
this answers is: "when will my waiting job start executing on a build
farm machine"?

> In your original comment, you say that this patch is supposed to make
> it possible to get job dispatch times irrespective of job type.
> However, given that you've almost certainly thought about this more
> than I have, I'd really appreciate it if you could add some
> information about how this achieves that goal, why it's the best
> solution, what trade-offs this makes (if any), and maybe give me a
> hint as to why the column placed onto the Build table in the first
> place.
Right. Putting the 'estimated_build_duration' on the `Build` table was a
design mistake that was exposed by the necessity to support the
estimation of job dispatch times irrespective of job type i.e. for
source package builds, recipe builds etc.

The job dispatch times for a given build farm job BFJ are calculated
based on 2 ingredients: the

  1 - sum of the estimated durations of the jobs that are ahead of BFJ in
      the queue.
  2 - time span until the next build machine becomes available
      (= min(estimated_duration - (now - job.date_started)) across the
       build machine pool that could run BFJ)

As you can see having a somewhat realistic 'estimated_duration' is
crucial to the quality of the job dispatch time estimation.

Also, in order to be able to formulate less than nightmarish database
queries for the 2 ingredients above we need the 'estimated_duration' to
be in some common place for all build farm job types and that would be
the `BuildQueue` table in this case.

I hope this helps.

Best regards

--
Muharem Hrnjadovic <email address hidden>
Public key id : B2BBFCFC
Key fingerprint : A5A3 CC67 2B87 D641 103F 5602 219F 6B60 B2BB FCFC