aboutsummaryrefslogtreecommitdiffstats
path: root/pkg/bisect/bisect_test.go
Commit message (Collapse)AuthorAgeFilesLines
* pkg/report: use crash.KASANUnknown instead of crash.KASANOtherTaras Madan2025-07-031-12/+12
| | | | | | What we need is the category for "matched unknown KASAN bug". This king on bugs should be recategorised. The final goal is to keep this category empty.
* pkg/report: split crash.KASAN into partsTaras Madan2025-07-031-12/+12
| | | | We want to prioritize KASAN bugs differently.
* all: remove loop variables scopingTaras Madan2025-02-171-4/+0
|
* pkg/bisect: ignore irrelevant lost connection crashesAleksandr Nogikh2024-12-031-11/+31
| | | | | | | These have been the cause of too many invalid bisection results recently. This is the first step towards the more universal approach of #5414.
* pkg/bisect: recognize lost connection errorsAleksandr Nogikh2024-12-031-0/+28
| | | | Only select them as relevant if there are not other crash types.
* pkg/vcs: change HeadCommit to CommitDmitry Vyukov2024-10-151-1/+1
| | | | | | | Currently we have HeadCommit function that returns info about the HEAD commit. Change it to a more flexible Commit function that can return info about any commit. This will be used in future changes.
* pkg/build: use the build environment in clean() callsFlorent Revest2024-10-141-0/+4
| | | | | | This unifies the build() and clean() interfaces such that if a custom compiler or make binary is provided in the manager or bisection config, they can be taken into account by the clean() interface.
* pkg/build: handle OOM-killed build errorSabyrzhan Tasbolatov2024-09-231-1/+1
| | | | | | | Handle SIGKILL (exit code = 137) on osutil.Run() during Linux kernel image building and return build.InfraError without reporting. Fixes: https://github.com/google/syzkaller/issues/5317
* pkg/bisect: set a lower bound for BisectBad verdictAleksandr Nogikh2024-07-151-9/+19
| | | | | | | | | The "1 crashed, 9 OK" cases are a frequent reason of invalid bisection results on syzbot. Let's define a cutoff for a BisectBad verdict and use it to prevent such obvious outliers. We cannot safely declare such results as BisectGood either, so let's return BisectSkip in this case.
* pkg/bisect: test a subset of releasesAleksandr Nogikh2023-08-251-3/+59
| | | | | | | | | | For older bugs (or for bugs on stable trees), our cause bisection strategy times out while trying to iterate over all reachable tags. Try to be smarter and only take a subset of them, thus limiting the time we spend detecting the bug-free release. Closes #3376.
* pkg/bisect: test merge base for cross-tree bisectionsAleksandr Nogikh2023-08-241-0/+25
| | | | | | | | | | | | | | | | | Consider the following situation: we're fix bisecting an lts bug on the mainline tree. Reproducer works on lts HEAD, but does not crash the mainline. If the bug was not introduced in the mainline, but it's a purely lts bug, bisection would currently end up pointing to the mainline HEAD (in #4123, we assume env.commit to be a commit on the tree we're currently bisecting, otherwise we'd always hit the same merge base commit). Let's first determine 100% correct bisection range on the mainline tree by testing that both the HEAD does not crash (we already do) and by testing that the new starting commit (=merge base) can be crashed by the reproducer.
* pkg/bisect: start cause bisections from any commitsAleksandr Nogikh2023-08-221-2/+13
| | | | | | | | For cause bisections, don't require Kernel.Commit to be reachable from Kernel.Branch. We can start cause bisections from any existing commit. This should help with linux-next bisections, where no older HEADs are ever reachable from each new `master`.
* pkg/bisect: make fix-inconclusive test more stableAleksandr Nogikh2023-08-111-5/+5
| | | | | | | | | | Merge history is quite complicated in the 500-600 range. Depending on randomly selected `git bisect skip` values, output might be different. If 500 is randomly selected, it will not be among skipped values. If 650 is selected first, it excludes 500. Use the 600-700 range instead. It's stable.
* pkg/bisect: consider bug's existenceAleksandr Nogikh2023-08-111-0/+22
| | | | | | | | | | | | If there's a merge from a branch that was based on a much older revision, it's likely that it does not YET contain the bug. If we don't consider this possibility, we're likely to get invalid fix bisection results for bugs that existed only for a short period of time. For every fix bisection step, determine first whether the guilty revision is reachable from it. Cache resuls to speed up processing. See #4117.
* pkg/bisect: refactor testsAleksandr Nogikh2023-08-111-34/+59
| | | | | | | | | Currently we assume that a bug is present in all revisions before the culprit one (or, alternatively, in all revisions after it). The reality is more complicated, so let's split culprit into introduced and fixCommit. Use commit titles instead of numbers as it provides more flexibility.
* pkg/bisect: estimate confidence in the resultAleksandr Nogikh2023-07-141-2/+16
| | | | | | | | | | | | | | | | Estimate reproducer's flakiness more carefully, as it can help us get better results. During config minimization, do not let reproducibility drop too low. For each test() run, estimate our confidence in the result. For now, only consider the false negative case: if we got no crashes, it's likely that the bug is there, but the reproducer was not lucky enough. Given an estimate of reproduction likelihood, we can easily calculate the chance of an invalid result: (1-probability)^runs. Combine all individual test() confidences into Result.Confidence. If the resulting Confidence is too low, ignore the bisection result.
* pkg/bisect: fix SYZFATAL bisectionsAleksandr Nogikh2023-07-071-3/+15
| | | | | After the previous change, pkg/bisect is unable to bisect SYZFATAL errors. Fix the bug in the logic and add a test.
* pkg/report: ignore transient crashesAleksandr Nogikh2023-07-061-5/+29
| | | | | | | | | There are cases when a crash, even though it didn't occur during boot and image testing phases, still should not be counted during bisection. Otherwise we risk diverting the whole process in the wrong direction. One of such problem classes is getting SYZFATAL errors when we are not bisecting a SYZFATAL crash.
* pkg/bisect: remember the most frequent report typesAleksandr Nogikh2023-07-051-0/+83
| | | | | | | | | This will later help in adjusting kernel configuration. Also, adjust how test() determines the crash report: pick the one with the most frequent type. Earlier we were taking the last crash report, but this seems to have a bias towards the crashes that take longer to appear (e.g. rcu stalls).
* pkg/bisect: change verdict calculation rulesAleksandr Nogikh2023-06-291-0/+110
| | | | | | | | | 1) For the good verdict, demand a certain minimum number of good runs to be present. 2) For the bad verdict, demand that at least 50% of runs did not fail with boot/image test error. Refactor the code and add tests.
* pkg/bisect: support bisections on other treesAleksandr Nogikh2023-06-291-12/+41
| | | | | | | | | | | | | | | | | | | The current code only supports fix/cause bisections when the known bad commit is reachable from Kernel.Repo/Kernel.Branch. Add a CrossTree parameter to pkg/bisect. If it's set to true and we're doing a fix bisection, the bisection algorithm first operates with the original commit message (i.e. checks that it indeed crashes the kernel and performs config minimization), but the actual bisection starts from the merge base of Commit and Branch. We could have calculated the merge base outside of pkg/bisect and just started the algorithm from that merge base, but there's a problem: there's no guarantee that the kernel will build/boot with a syzbot config at the merge base. So we take the commit known to work well and then assume that the bug is also present on the merge base commit. If it were not present, we wouldn't have found a fix commit from Branch anyway.
* pkg/bisect: abort on infrastructure errorsAleksandr Nogikh2023-05-091-8/+50
| | | | | | There's not much sense to continue the bisection if something in the testing infrastructure got broken. Indicate it with the InfraError error.
* pkg/bisect: mark jobs with untestable history as failedSpace Meyer2023-05-051-11/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Background info: When a bisection is started, we only know the kernel commit on which syzkaller encountered the crash. Before the actual bisection begins, bisect() needs to find a good commit. Then we can bisect between them. For fix bisections bisect() tests HEAD. If HEAD doesn't have the bug the fixing commit must be somewhere in the middle. For cause bisection we test a number of old releases, starting with the newest release. Both this commit range search and the actual bisection later use test() to build the kernel and run the reproducer. During actual bisections we invoke test() for every step. If any test() invocation returns a non nil error, the bisection it was called from is aborted. Any non-fatal errors should be signaled via the testResult returned from test(). For this reason a build/boot failure does not return an error. Instead testResult.verdict will be vcs.BisectSkip. The Problem: Given the following call stack: bisect() -> commitRange() -> commitRangeFor{Fix,Cause}() -> test() Previously we reported fix bisections, where HEAD was build broken and cause bisections where all tested releases were build broken as inconclusive. This is confusing for users. For fix bisections it looks like the fixing commit is either the commit from the original crash or HEAD. For cause bisections it looks like the breaking commit is either the original commit or the commit of the oldest tested release. Neither is correct. For fix bisections we see this, when the HEAD of a tested branch is build broken. When this happens all attempted fix bisections will get nonsense results. For cause bisections we see this, when changes to the bisection compilers or test rootfs cause us to not be able to build or boot very old kernels. These inconclusive bisection results will not be retried automatically and we don't have an easy way to clear them. The Solution: For fix bisections: Retry the bisection later. If HEAD is completely broken we will know, because fuzzing will stop. For cause bisections: Mark the bisection as failed. The result is unlikely to change in the future without intervention by the syzbot admins. Users won't bother looking at those bisections and the dashboard already has code to mass-retry failed bisections. - In test(): Populate testResult.rep with a meaningful report before returning, after build/boot failures. - In bisect(): Explicitly check the testResult.verdict of the last commit tested in commitRange(), instead of using testResult.rep==nil as an oracle for aborting the bisection. - In bisect(): Don't return an inconclusive result when build/boot failures prevent us from finding a commit range to bisect between.
* pkg/instance: move BuildKernel() args to structSpace Meyer2023-01-091-4/+3
|
* syz-ci/jobs: use linker supplied in syz-ci configSpace Meyer2023-01-091-1/+1
| | | | | | Previously we only used the linter from the syz-ci config when building the kernel for regular fuzzing. We were missing some plumbing to have this setting reach patch testing and bisection jobs.
* pkg/bisect: try to reidentify commit rebased after crashSpace Meyer2022-10-061-37/+51
| | | | | | | | | | | | | | When bisecting a breaking commit, syzkaller starts the bisection from the commit recorded in the last crash for the given bug. Previously the bisection was aborted should the commit no longer exist in the repo. Now we try to reidentify the breaking commit. For git pretty much the best we can do is to search a commit reachable from HEAD with the same title. Other VCS systems might have something better. Syzkaller will still first validate that the start commit is indeed broken in the way it expects. This prevents syzkaller from getting confused should we accidentally pick a completely unrelated commit.
* syz-ci: remember syzkaller builds logsAleksandr Nogikh2022-06-281-2/+2
| | | | | This might help get more insight into patch testing failures, especially for old bugs.
* all: remember console output for all patch testsAleksandr Nogikh2022-06-081-14/+15
| | | | | | Currently syzbot only saves a log if there was a build/test error. Closes #3185
* all: use `t.TempDir` to create temporary test directoryEng Zer Jun2022-03-281-17/+4
| | | | | | | | | This commit replaces all `ioutil.TempDir` with `t.TempDir` in tests. The directory created by `t.TempDir` is automatically removed when the test and all its subtests complete. Reference: https://pkg.go.dev/testing#T.TempDir Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* all: capture compiler id during the build processAleksandr Nogikh2021-07-201-5/+7
| | | | | | | | | Default compilers are specified in the OS- and platform-dependent logic. It is more convenient to extract info about them during the kernel build itself, rather than during the manager object initialization. Apply the necessary changes throughout the code that is involved in building the kernels and processing information about this process.
* pkg/kconfig: store minimization resultsJouni Hogander2020-12-101-4/+2
| | | | | Store config options identified using DebugTracer. Also change bisection and configuration minimization code to use new DebugTracer.
* pkg/mgrconfig: add prog&sys.targets targets to ConfigDmitry Vyukov2020-11-301-1/+1
| | | | | | These are widely used with the config as the refactoring shows. This removes a bunch of unnecessary code. Also fixes a number of bugs where we confused Arch with VMArch.
* pkg/mgrconfig: move derived fields into separate structDmitry Vyukov2020-11-301-4/+6
| | | | | | Users should not be concerned with the internal derived fields. Move all derived fields into a separate struct before adding more. This leaves config.go as a better documentation for end users.
* pkg/bisect: add flaky flagJouni Hogander2020-11-171-3/+21
| | | | | On first test (original commit) bump up number of tests always. Also bump up number of tests if observing more good results than bad.
* sys/targets: add OS/Arch name constsDmitry Vyukov2020-10-261-3/+4
| | | | | | | | | | | | We use strings to identify OS/Arch. These strings are duplicated throughout the code base massively. golangci-lint points to possiblity of typos and duplication. We already had to define these names in pkg/csource and disable checking for prog package. A future change triggers such warnings in another package. Add OS/Arch name consts to sys/targets so that they can be used to refer to OS/Arch. Use the consts everywhere.
* pkg/bisect: switch to kconfig.MinimizeDmitry Vyukov2020-10-211-10/+1
| | | | | | | | | | | | | | | Use the new kconfig.Minimize for config minization instead of the config-bisect.pl script. This is mostly just deleting code. Also update tests: - minimization is now supposed to test the baseline config (update "testos" stub accordingly) - minimization is not supposed to return a config that does not build (a reasonable config minimization procedure can't arrive to such config), remove test that tests this Update #2171
* pkg/vcs: add repo OptPrecious and OptDontSandbox optionsDmitry Vyukov2020-10-211-1/+1
| | | | | | | | | | The pkg/vcs code assumed that we fully manage the repo within an autonomous program. In particular it tried to repair any errors by dropping and re-creating the repo. This does not work well for command-line tools that work with a user-provided repo. Add OptPrecious for such uses. Update #2171
* pkg/bisect: add ccache optionJouni Hogander2020-09-211-1/+1
| | | | | | Add option to use ccache in kernel builds. Signed-off-by: Jouni Hogander <jouni.hoegander@partner.bmw.de>
* all: fix log/error text starting with capital letterDmitry Vyukov2020-07-051-1/+1
| | | | Update #1876
* pkg/bisect: fix noop change detection with config minimizationDmitry Vyukov2020-07-021-2/+16
| | | | | | Config minimization did not update test results for the starting commit. Test result for the starting commit still refered to the original config, not the minimized config that was actually used during bisection.
* pkg/bisect: improve config minimization testsDmitry Vyukov2020-07-021-3/+12
| | | | | | | Change a test to actually produce a new config as it is supposed to do during config minimization and check the resulting config. Add a regression test for the bug where we committed a broken config.
* pkg/bisect: test resulting configDmitry Vyukov2020-07-021-33/+47
| | | | | | One of main outputs of the config minimization is the new config, but tests don't do any single check of it. Check the resulting config (is it correct in all cases?).
* pkg/bisect: don't ignore errors during config minimizationDmitry Vyukov2020-07-021-5/+3
| | | | | | | Ignoring errors is bad. It leads to silent failures. If there is some particular error condition that is expected to happen and we want to ignore, we need to ignore that one only rather than all potenital errors in the whole process.
* pkg/bisect: minor style fix upsDmitry Vyukov2020-07-021-3/+2
|
* pkg/bisect: fix crash when all releases are brokenDmitry Vyukov2020-07-021-0/+8
| | | | Currently we crash with nil deref in this special case.
* pkg/bisect: Implement config bisectionJouni Hogander2020-07-021-9/+72
| | | | | | | | | Implement Linux kernel configuration bisection. Use bisected minimalistic configuration in commit bisection. Utilizes config_bisect.pl script from Linux kernel tree in bisection. Modify syz-bisect to read in kernel.baseline_config. This is used as a "good" configuration when bisection is run.
* .golangci.yml: enable funlen checkerDmitry Vyukov2020-06-051-173/+174
| | | | Checks for too long functions (based on lines and statements).
* pkg/bisect: speed up testsDmitry Vyukov2020-05-181-56/+86
| | | | | | Creating new repos takes majority of test time. Reuse them across tests. With GOMAXPROCS=2 (CI) this speeds up tests from 33 sec to 7 secs.
* pkg/bisect: always test parent commitDmitry Vyukov2019-12-101-3/+2
| | | | Fixes #1527
* pkg/bisect: add test for #1527Dmitry Vyukov2019-12-101-2/+24
| | | | Update #1527
generated by ggit (джигит) mrf-deployment (git 2.46.0) at 2026-03-12 00:27:55 +0000