| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
| |
probability coverage
|
| |
|
|
|
|
|
| |
Introduce a KFuzzTest mode for the fuzzer so that a smaller number of
recommended calls can be used if we are fuzzing KFuzzTest targets.
Signed-off-by: Ethan Graham <ethangraham@google.com>
|
| |
|
|
|
|
|
|
|
|
| |
The Clone() we do at the beginning of the handleCall() function is
unnecessary and may even confuse the people wo read and modify the code
(see #5878).
For all the jobs we start below, we anyway do another Clone() of the
program. The corpus also does not assume that the programs in it will be
modified elsewhere.
|
| | |
|
| |
|
|
|
| |
When the option is set, more time is spent on "exec fuzz" and less time
is spent minimizing the programs.
|
| |
|
|
| |
This will let us gain even more insight into what the fuzzer is doing.
|
| |
|
|
|
|
| |
Specify the exact call that is being minimized.
Print new signal delta if it's small (these are most suspicious kinds of
triage jobs).
|
| |
|
|
| |
Instead of printing the full program, enable per-job logs.
|
| |
|
|
| |
This will let us understand better what exactly the fuzzer was doing.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
All callers shouldn't control lots of internal details of minimization
(if we have more params, that's just more variations to test,
and we don't have more, params is just a more convoluted way to say
if we minimize for corpus or a crash).
2 bools also allow to express 4 options, but only 3 make sense.
Also when I see MinimizeParams{} in the code, it's unclear what it means.
Replace params with mode.
And potentially "crash" minimization is not "light", it's just different.
E.g. we can simplify int arguments for reproducers (esp in snapshot mode),
but we don't need that for corpus.
|
| |
|
|
| |
Distribute triage requests to different VMs.
|
| |
|
|
|
|
| |
Tune number of deflake/minimize runs in snapshot more.
Presumably snapshot mode must be more stable and should
require fewer runs.
|
| |
|
|
|
|
|
|
|
| |
We are getting too many generated candidates, the fuzzer may not keep up
with them at all (hints jobs keep growing infinitely). If a hint indeed came
from the input w/o transformation, then we should guess it on the first
attempt (or at least after few attempts). If it did not come from the input,
or came with a non-trivial transformation, then any number of attempts won't
help. So limit the total number of attempts (until the next restart).
|
| |
|
|
|
| |
Use 2/6 criteria during corpus triage.
See the large added comment for details.
|
| |
|
|
|
| |
If we have found new signal for several calls of the same programs,
triage them in paralllel.
|
| |
|
|
|
|
| |
During deflake(), we look at the unions of the signal from multiple
iterations. Let's do the same in minimize(), it would be more
consistent.
|
| |
|
|
|
|
| |
Hints can produce insane amounts of candidates (up to 30K).
So run deflaking one more time, even if it reduces amount
of candidates by few percents, it's still profitable.
|
| |
|
|
|
|
|
|
|
| |
1. Run only 25 mutations during smash.
2. Run collide during normal fuzzing rather than during smashing.
3. Run hints and fault injection before random mutations
and order hints/fault injection jobs.
4. Random mutations still run round-robin w/o ordering
to give better diversity.
|
| |
|
|
|
|
|
| |
Move all syz-fuzzer logic into syz-executor and remove syz-fuzzer.
Also restore syz-runtest functionality in the manager.
Update #4917 (sets most signal handlers to SIG_IGN)
|
| |
|
|
|
| |
Let exec smash describe purely the mutation part of the smash job.
Introduce a separate stat for fault injection executions.
|
| |
|
|
|
|
|
| |
During deflake/minimization we have 1 target call that should be handled specially:
during triage we want all signal, during minimization we want new+target signal.
For the rest of the calls we can do normal handling: collect new signal
and start triage if we see any new signal.
|
| |
|
|
| |
It can be large and we need just errno. Store just it.
|
| |
|
|
| |
Update the comment based on our new understanding.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
1. Use the initial signal as the first attempt,
this should reduce total number of runs by 1 in common case.
2. Update max signal on each attempt.
If should eliminate additional runs later.
3. Update the signal we are chasing after each attempt.
It's possible that we won't get any of the orignal new signal,
but instead will get some other stable new signal.
|
| |
|
|
|
|
|
|
|
| |
The next commit will add another Candidate flag.
Candidate flags duplicate progTypes enum, so to avoid conversions
of one to another use progTypes in Candidate struct directly.
Rename progTypes to progFlags since multiple can be set,
so this is effectively flags rather than a single type.
|
| |
|
|
| |
Add an explicit parameter to only run call removal.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Taken some arm64 devices for example:
kaslr_offset is diff at bits 12-40, and kernel modules are loaded at 2GB space,
so we have `ffffffd342e10000 T _stext` where uppper 32bit is ffffffd3. However,
if we check modules range, the 1st module is loaded at 0xffffffd2eeb2a000,
while the last module is loaded at 0xffffffd2f42c4000.
We can see the upper 32bits are diff for core kernel and modules.
If we use current 32bits for covered PC, we will get wrong module address
recovered.
So we need to move to 64bit cover and signal:
- change cover/sig to 64bit to fit for syz-executor change
- remove kernel upper base logic as kernel upper base is not a constant when
kaslr enabled for core kernel and modules.
- remove unused pcBase
|
| |
|
|
|
|
| |
Switch to flatrpc.ProgInfo.
Note: this disables syz-runtest and syz-verifier.
|
| |
|
|
|
|
| |
Flatrpc flags are passed in RPC execution requests,
so to avoid conversions and duplicate set of flags
use flatrpc flags in pkg/ipc directly.
|
| |
|
|
| |
For now, only ProgTypes is enough.
|
| |
|
|
| |
There's no need in duplicating the signal, coverage, hints flags.
|
| |
|
|
|
|
|
|
|
|
| |
Mark some requests as Important. The Retry() layer will give them one
more chance even if they were not executed due to a VM crash.
For now, the only important requests are related to triage, candidates
and pkg/vminfo tests.
Add tests for retry.go.
|
| |
|
|
|
|
| |
Make Result statuses more elaborate.
Instead of retrying inputs directly in rpc.go, extract this logic to a
separate entity in pkg/fuzzer/queue.
|
| |
|
|
|
|
|
|
|
|
|
| |
Instead of relying on a fuzzer-internal priority queue, utilize
stackable layers of request-generating steps.
Move the functionality to a separate pkg/fuzzer/queue package.
The pkg/fuzzer/queue package can be reused to add extra processing
layers on top of the fuzzing and to combine machine checking and fuzzing
execution pipelines.
|
| |
|
|
|
|
| |
Do two exec hints to only leave stable comparison argument pairs.
In local experiments, it allows to reduce their count by 30-40% (on
average).
|
| |
|
|
|
| |
Remove unused fields.
Check that we don't need to triage program earlier.
|
| |
|
|
|
|
| |
Now that manager sends ipc.ExecOpts to the fuzzer,
there is no point in having Signal type in rpctype.
It belongs to pkg/fuzzer.
|
| |
|
|
|
|
|
|
|
|
|
| |
This fixes 2 issues:
1. We still want to get new coverage for syscalls during minimization.
We run lots of new programs, and some of them can give new coverage.
2. The signal filter should apply only to the target syscall.
Other syscalls probably can't even reach any of that code.
So add SignalFilterCall field and combine new and filtered signal
for that call. Other calls just collect new coverage as usual.
|
| |
|
|
| |
We used to send the non-modified program for execution.
|
| |
|
|
|
|
| |
We don't want to reach just any stable signal, we know the specific new
signal that we target. The previous approach might have reduced the
efficiency of the new deflake() approach.
|
| |
|
|
|
|
|
| |
Add ability for each package to create and export own stats.
Each stat is self-contained, describes how it should be presented,
and there is not need to copy them from one package to another.
Stats also keep historical data and allow building graphs over time.
|
| |
|
|
|
| |
Demand that at least 3 out of 5 runs share common signal.
Exit early if it's not feasible.
|
| |
|
|
|
|
|
| |
Currently we throw away saturated calls only after triage/minimization.
Triage/minimization is unnecessary for saturated calls,
we already know we will throw them away later.
Don't send saturated calls for triage/minimization.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
The fuzzer may become too busy doing potentially very long hint jobs,
while we want it to also keep exploring other parts of the input space.
If there are only smash and hint jobs left, ignore them for 33% of
executions.
Reduce the number of smash iterations:
1) If new coverage is found, we will likely repeat the smash job with a
similar program.
2) We mostly do the same during exec fuzz anyway.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Instead of doing fuzzing in parallel in running VM, make all decisions
in the host syz-manager process.
Instantiate and keep a fuzzer.Fuzzer object in syz-manager and update
the RPC between syz-manager and syz-fuzzer to exchange exact programs to
execute and their resulting signal and coverage.
To optimize the networking traffic, exchange mostly only the difference
between the known max signal and the detected signal.
|
| |
|
|
|
|
| |
We are already trying to filter out any flaky coverage at that stage.
Any new signal that we got by re-running the same program can be
automatically treated as flaky as well.
|
| |
|
|
|
|
|
|
| |
pkg/fuzzer and syz-manager have a common corpus functionality that can
be well be unified.
Create a separate pkg/corpus package that would be used by both of them.
It will simplify further work of moving pkg/fuzzer to the host.
|
| |
|
|
|
| |
The call may potentially generate a very large number of possible
mutations. Add a way to abort the process.
|
|
|
This is the first step for #1541.
Move the fuzzing engine that used to be interleaved with other syz-fuzzer
code into a separate package.
For now, the algorithm is more or less the same as it was, the only
difference is that a pkg/fuzzer instance scales to the available
computing power.
Add an executor-based test that performs real fuzzing.
|
