| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| | |
|
| |
|
|
| |
probability coverage
|
| |
|
|
| |
Any is the preferred over interface{} now in Go.
|
| |
|
|
|
|
|
| |
Introduce a KFuzzTest mode for the fuzzer so that a smaller number of
recommended calls can be used if we are fuzzing KFuzzTest targets.
Signed-off-by: Ethan Graham <ethangraham@google.com>
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
Track the right moment to start bug reproductions more exactly:
1) Either once 90% of the corpus is triaged (*).
2) Or once we are past 50% of the time dedicated for fuzzing.
Whatever happens earlier.
(*) The last percents are usually quite slow and they bring much less
covered PCs that all the previous ones.
|
| |
|
|
|
| |
If the overflows happen often, it's bad.
Add visibility into this.
|
| |
|
|
|
| |
When the option is set, more time is spent on "exec fuzz" and less time
is spent minimizing the programs.
|
| |
|
|
| |
This will enable its reuse.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently we kill hanged processes and consider the corresponding test finished.
We don't kill/wait for the actual test subprocess (we don't know its pid to kill,
and waiting will presumably hang). This has 2 problems:
1. If the hanged process causes "task hung" report, we can't reproduce it,
since the test finished too long ago (manager thinks its finished and
discards the request).
2. The test process still consumed per-pid resources.
Explicitly detect and handle such cases:
Manager keeps these hanged tests forever,
and we assign a new proc id for future processes
(don't reuse the hanged one).
|
| |
|
|
| |
Instead of printing the full program, enable per-job logs.
|
| |
|
|
| |
This will let us understand better what exactly the fuzzer was doing.
|
| | |
|
| |
|
|
| |
Distribute triage requests to different VMs.
|
| |
|
|
|
|
| |
Tune number of deflake/minimize runs in snapshot more.
Presumably snapshot mode must be more stable and should
require fewer runs.
|
| |
|
|
|
|
|
|
| |
Go package names should generally be singular form:
https://go.dev/blog/package-names
https://rakyll.org/style-packages
https://groups.google.com/g/golang-nuts/c/buBwLar1gNw
|
| |
|
|
|
|
|
|
|
| |
We are getting too many generated candidates, the fuzzer may not keep up
with them at all (hints jobs keep growing infinitely). If a hint indeed came
from the input w/o transformation, then we should guess it on the first
attempt (or at least after few attempts). If it did not come from the input,
or came with a non-trivial transformation, then any number of attempts won't
help. So limit the total number of attempts (until the next restart).
|
| |
|
|
|
|
|
| |
Signal rotation is intended to make the fuzzer re-discover flaky coverage
in non flaky way. However, taking into accout that we get effectively
the same effect after each manager restart, and that the fuzzer is overloaded
with triage/smash jobs, it does not look to be worth it.
|
| |
|
|
|
|
|
|
|
| |
1. Run only 25 mutations during smash.
2. Run collide during normal fuzzing rather than during smashing.
3. Run hints and fault injection before random mutations
and order hints/fault injection jobs.
4. Random mutations still run round-robin w/o ordering
to give better diversity.
|
| |
|
|
|
|
|
| |
Move all syz-fuzzer logic into syz-executor and remove syz-fuzzer.
Also restore syz-runtest functionality in the manager.
Update #4917 (sets most signal handlers to SIG_IGN)
|
| |
|
|
|
|
|
| |
During deflake/minimization we have 1 target call that should be handled specially:
during triage we want all signal, during minimization we want new+target signal.
For the rest of the calls we can do normal handling: collect new signal
and start triage if we see any new signal.
|
| |
|
|
| |
It can be large and we need just errno. Store just it.
|
| |
|
|
|
|
|
|
| |
Currently we always retry corpus candidates 2 times.
Retry up to 3 times if we did not get new signal.
This should both reduce total number of runs (if we get new signal
on the first run) and reduce signal loss after restart
(since we try up to 3 times instead of 2).
|
| |
|
|
|
|
|
|
|
| |
The next commit will add another Candidate flag.
Candidate flags duplicate progTypes enum, so to avoid conversions
of one to another use progTypes in Candidate struct directly.
Rename progTypes to progFlags since multiple can be set,
so this is effectively flags rather than a single type.
|
| |
|
|
|
|
|
|
| |
Add corpus triage mode and support it in testbed.
This is useful to benchmark just the triage phase
w/o any subsequent fuzzing. First, fuzzing is more random.
Second, if triage duration is different in different versions,
then they will do different amount of fuzzing in fixed testbed time.
|
| |
|
|
| |
Switch to flatrpc.ExecOpts.
|
| |
|
|
|
|
| |
Switch to flatrpc.ProgInfo.
Note: this disables syz-runtest and syz-verifier.
|
| |
|
|
|
|
| |
Flatrpc flags are passed in RPC execution requests,
so to avoid conversions and duplicate set of flags
use flatrpc flags in pkg/ipc directly.
|
| |
|
|
|
|
| |
We don't need the full priority queue functionality anymore. For our
purposes it's enough to only enforce the order between the elements of
different sub-queues.
|
| |
|
|
| |
For now, only ProgTypes is enough.
|
| |
|
|
| |
There's no need in duplicating the signal, coverage, hints flags.
|
| |
|
|
|
|
|
|
|
|
|
| |
Instead of relying on a fuzzer-internal priority queue, utilize
stackable layers of request-generating steps.
Move the functionality to a separate pkg/fuzzer/queue package.
The pkg/fuzzer/queue package can be reused to add extra processing
layers on top of the fuzzing and to combine machine checking and fuzzing
execution pipelines.
|
| |
|
|
|
| |
Remove unused fields.
Check that we don't need to triage program earlier.
|
| |
|
|
|
|
| |
Now that manager sends ipc.ExecOpts to the fuzzer,
there is no point in having Signal type in rpctype.
It belongs to pkg/fuzzer.
|
| |
|
|
|
|
|
|
|
|
|
| |
This fixes 2 issues:
1. We still want to get new coverage for syscalls during minimization.
We run lots of new programs, and some of them can give new coverage.
2. The signal filter should apply only to the target syscall.
Other syscalls probably can't even reach any of that code.
So add SignalFilterCall field and combine new and filtered signal
for that call. Other calls just collect new coverage as usual.
|
| | |
|
| |
|
|
|
|
|
| |
Add ability for each package to create and export own stats.
Each stat is self-contained, describes how it should be presented,
and there is not need to copy them from one package to another.
Stats also keep historical data and allow building graphs over time.
|
| |
|
|
|
|
|
| |
Currently we throw away saturated calls only after triage/minimization.
Triage/minimization is unnecessary for saturated calls,
we already know we will throw them away later.
Don't send saturated calls for triage/minimization.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
The fuzzer may become too busy doing potentially very long hint jobs,
while we want it to also keep exploring other parts of the input space.
If there are only smash and hint jobs left, ignore them for 33% of
executions.
Reduce the number of smash iterations:
1) If new coverage is found, we will likely repeat the smash job with a
similar program.
2) We mostly do the same during exec fuzz anyway.
|
| |
|
|
|
|
|
| |
Once in 15 minutes, drop 1000 elements of the pure max signal (that is,
max signal minus corpus signal).
It seems to have a positive effect on the total fuzzing performance.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Instead of doing fuzzing in parallel in running VM, make all decisions
in the host syz-manager process.
Instantiate and keep a fuzzer.Fuzzer object in syz-manager and update
the RPC between syz-manager and syz-fuzzer to exchange exact programs to
execute and their resulting signal and coverage.
To optimize the networking traffic, exchange mostly only the difference
between the known max signal and the detected signal.
|
| |
|
|
| |
It does not affect fuzzing.
|
| |
|
|
| |
We'll be tracking requests in syz-manager.
|
| |
|
|
|
|
| |
We are already trying to filter out any flaky coverage at that stage.
Any new signal that we got by re-running the same program can be
automatically treated as flaky as well.
|
| | |
|
| |
|
|
|
| |
Stats() seems to be a more sound choice since these structures include
multiple data points.
|
| |
|
|
|
|
|
|
| |
pkg/fuzzer and syz-manager have a common corpus functionality that can
be well be unified.
Create a separate pkg/corpus package that would be used by both of them.
It will simplify further work of moving pkg/fuzzer to the host.
|
|
|
This is the first step for #1541.
Move the fuzzing engine that used to be interleaved with other syz-fuzzer
code into a separate package.
For now, the algorithm is more or less the same as it was, the only
difference is that a pkg/fuzzer instance scales to the available
computing power.
Add an executor-based test that performs real fuzzing.
|
