diff options
| author | Aleksandr Nogikh <nogikh@google.com> | 2023-07-12 20:32:59 +0200 |
|---|---|---|
| committer | Aleksandr Nogikh <nogikh@google.com> | 2023-07-14 12:36:52 +0000 |
| commit | 35d9ecc508aef508b67ee7986a7abb0864e74f8e (patch) | |
| tree | 5b044a39374c6e3b7c91d14974b7abe1268d62b0 /syz-ci/jobs.go | |
| parent | d624500f3877323fae8eb084872c5ef9a8ce3ef9 (diff) | |
pkg/bisect: estimate confidence in the result
Estimate reproducer's flakiness more carefully, as it can help us get
better results.
During config minimization, do not let reproducibility drop too low.
For each test() run, estimate our confidence in the result. For now,
only consider the false negative case: if we got no crashes, it's likely
that the bug is there, but the reproducer was not lucky enough. Given an
estimate of reproduction likelihood, we can easily calculate the chance
of an invalid result: (1-probability)^runs.
Combine all individual test() confidences into Result.Confidence. If the
resulting Confidence is too low, ignore the bisection result.
Diffstat (limited to 'syz-ci/jobs.go')
| -rw-r--r-- | syz-ci/jobs.go | 4 |
1 files changed, 4 insertions, 0 deletions
diff --git a/syz-ci/jobs.go b/syz-ci/jobs.go index 13abcbf2b..ee62e7e03 100644 --- a/syz-ci/jobs.go +++ b/syz-ci/jobs.go @@ -541,6 +541,10 @@ func (jp *JobProcessor) bisect(job *Job, mgrcfg *mgrconfig.Config) error { if res.IsRelease { resp.Flags |= dashapi.BisectResultRelease } + const confidenceCutOff = 0.5 + if res.Confidence < confidenceCutOff { + resp.Flags |= dashapi.BisectResultIgnore + } ignoredCommits := []string{ // Commit "usb: gadget: add raw-gadget interface" adds a kernel interface for // triggering USB bugs, which ends up being the guilty commit during bisection |