| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead of generating Go files with descriptions
serialize them as gob and compress with flate.
This significantly reduces build time, go vet time,
and solves scalability problems with some static analysis tools.
Reference times (all after rm -rf ~/.cache/go-build) before:
TIME="%e %P %M" time go install ./syz-manager
48.29 577% 4824820
TIME="%e %P %M" time go test -c ./prog
56.28 380% 6973292
After:
TIME="%e %P %M" time go install ./syz-manager
22.81 865% 859788
TIME="%e %P %M" time go test -c ./prog
12.74 565% 267760
syz-manager size before/after: 194712597 -> 83418407
-57% even provided we now embed all descriptions
instead of just a single arch.
Deflate/decoding time for a single Linux arch is ~330ms.
Fixes #5542
|
| |
|
|
|
|
|
|
|
| |
Disable the syscall in descriptions rather than in the code.
This makes it more visible for users that it's disabled,
and makes it less special (will not need to move this logic to host).
Also change the condition in syz-sysgen to be more precise,
otherwise syz_execute_func becomes unused function.
|
| | |
|
| |
|
|
|
|
| |
Otherwise we may end up corrupting device memory.
See https://groups.google.com/g/syzkaller-netbsd-bugs/c/Iy8-NZ_M9Ug/m/5jKKfncsAQAJ
|
| |
|
|
| |
sys/netbsd: removing comments
|
| |
|
| |
sys/netbsd: fixing descriptions
|
| |
|
|
|
|
| |
Commit empty files into generated packages,
so that the tree is buildable even w/o generated files
and Go tools continue to work.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* sys/netbsd: adding descriptions for device drivers & minor updation of syscall flags
sys/netbsd: reordering rlimit_type
sys/netbsd: reordering flags for stat(2) and mount(2)
sys/netbsd: adding i2c and wscons device drivers
sys/netbsd: adding desciptions for device drivers ioctls
sys/netbsd: minor update in dev_wscons.txt
sys/netbsd: updating descriptions of device driver files
sys/netbsd: adding copyright statement to dev files
sys/netbsd: minor changes in device driver files
sys/netbsd: ioctl descriptions for /dev/tprof
sys/netbsd: adding flags for tprof device driver
* sys/netbsd: updating .const files
|
| |
|
|
|
|
|
|
|
|
| |
If a resource is never used as an input, it is not useful.
It's effectively the same as using an integer.
Detect such cases, they are quite confusing.
Fix all existing errors in descriptions.
This uncovered some interesting bugs as well,
e.g. use of a completely unrelated fd subtype after copy-paste
(while the resource that was supposed to be used there is completely unused).
|
| |
|
|
|
|
|
|
|
| |
* sys/netbsd: Adding syscalls related to mount(2),uname(2) and sockets
* sys/netbsd: Adding missing flags for fstatat(2)
* sys/netbsd: reordering rlimit_type
* sys/netbsd: reordering flags for stat(2) and mount(2)
|
| |
|
|
|
|
|
|
|
| |
* sys/netbsd: adding ksem related syscalls
* sys/netbsd: adding missing header files to lwp.txt
* sys/netbsd: updating descriptions for ksem family syscalls
* sys/netbsd: updating descriptions for ksem family syscalls
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* sys/netbsd: adding filesystem and communication syscalls
* sys/netbsd: add fix for struct sockaddr_storage and profil(2)
* sys/netbsd: add common ioctl(2) commands
* sys/netbsd: resolving conflicts
* sys/netbsd: adding stat() and statfs() related syscalls
* sys/netbsd: adding missing flags for getfsstat(2)
Co-authored-by: Siddharth M <siddharth.muralee@gmail.com>
|
| |
|
|
|
|
|
|
|
|
|
| |
* sys/netbsd: adding filesystem and communication syscalls
* sys/netbsd: add fix for struct sockaddr_storage and profil(2)
* sys/netbsd: add common ioctl(2) commands
* sys/netbsd: resolving conflicts
Co-authored-by: Siddharth M <siddharth.muralee@gmail.com>
|
| |
|
|
|
|
|
|
|
| |
* sys/netbsd: increase syscall support
* sys/netbsd: increase syscall support fixed
* sys/netbsd: fixed mknod issue
* sys/netbsd: add fix for getdents, futimes and setpriority
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Checking in the generated descriptions files makes few things simpler,
but causes pain for pull requests: (1) PRs that touch descriptions
_always_ conflict, (2) PRs are large and harder to review,
(3) people sometimes forget to add auto-generated files.
The proposed way does not require us to hardcode lots of dependencies
in the Makefile (which is nice) and seem to work.
Let's see how it works.
The main contributor-visible consequence is that the auto-generated
files do not need to be checked-in now.
Credit for figuring the Makefile magic goes to @melver.
Fixes #1291
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add prog.Ref Type that serves as a proxy for real types
and allows to deduplicate Types in generated descriptions.
The Ref type is effectively an index in an array of types.
Just before serialization pkg/compiler replaces real types
with the Ref types and prepares corresponding array of real types.
When a Target is registered in prog package, we do the opposite
operation and replace Ref's with the corresponding real types.
This brings improvements across the board:
compiler memory consumption is reduced by 15%,
test building time by 25%, descriptions size by 33%.
Before:
$ du -h sys/linux/gen
54M sys/linux/gen
$ time GOMAXPROCS=1 go test -p=1 -c ./prog
real 0m54.200s
real 0m53.883s
$ time GOMAXPROCS=1 go install -p=1 ./tools/syz-execprog
real 0m27.911s
real 0m27.767s
$ TIME="%e %P %M" GOMAXPROCS=1 time go tool compile ./sys/linux/gen
20.59 100% 3200016
20.97 100% 3445976
20.25 100% 3209684
After:
$ du -h sys/linux/gen
36M sys/linux/gen
$ time GOMAXPROCS=1 go test -p=1 -c ./prog
real 0m42.290s
real 0m43.230s
$ time GOMAXPROCS=1 go install -p=1 ./tools/syz-execprog
real 0m24.337s
real 0m24.727s
$ TIME="%e %P %M" GOMAXPROCS=1 time go tool compile ./sys/linux/gen
19.11 100% 2764952
19.66 100% 2787624
19.35 100% 2749376
Update #1580
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Introduce common infrastructure for describing and parsing attribute
instead of custom per-attribute code scattered across several locations.
Change align attribute syntax from the weird align_N to align[N].
This also allows to use literal constants as N.
Introduce notion of builtin constants.
Currently we have only PTR_SIZE, which is needed to replace
align_ptr with align[PTR_SIZE].
|
| |
|
|
|
|
|
|
|
|
|
| |
Surround the main data mapping with PROT_NONE pages to make virtual address layout more consistent
across different configurations (static/non-static build) and C repros.
One observed case before: executor had a mapping above the data mapping (output region),
while C repros did not have that mapping above, as the result in one case VMA had next link,
while in the other it didn't and it caused a bug to not reproduce with the C repro.
The bug that reproduces only with the mapping above:
https://lkml.org/lkml/2020/4/17/819
|
| |
|
|
|
|
|
|
|
|
|
| |
Make MakeMmap return more than 1 call.
This is a preparation for future changes.
Also remove addr/size as they are effectively
always the same and can be inferred from the target
(will also conflict with the future changes).
Also rename to MakeDataMmap to better represent
the new purpose: it's just some arbitrary mmap,
but rather mapping of the data segment.
|
| |
|
|
|
|
|
| |
Turns out the mmap protection get out of sync
between executor and C reproducers.
C reproducers missed PROT_EXEC.
Add PROT_EXEC for linux, freebsd and akaros.
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
We will need a wrapper for target.SanitizeCall that will do more
than just calling the target-provided function. To avoid confusion
and potential mistakes, give the target function and prog function
different names. Prog package will continue to call this "sanitize",
which will include target's "neutralize" + more.
Also refactor API a bit: we need a helper function that sanitizes
the whole program because that's needed most of the time.
Fixes #477
Fixes #502
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If we have:
ioctl(fd fd, cmd int32)
ioctl$FOO(fd fd, cmd const[FOO])
Currently we assume that cmd size in ioctl$FOO is sizeof(void*).
However, we know that in ioctl it's specified as int32,
so we can infer that the actual syscall size is 4.
This massively reduces sizes of socket/setsockopt/getsockopt/ioctl
and some other syscalls, which is good because we now use physical
size in mutation/hints and some other places.
This will also enable not morphing ioctl's into other ioctl's.
Update #477
Update #502
|
| | |
|
| |
|
|
|
|
|
| |
If we are going to write all values, don't write field names.
This only increases size of generated files.
The change reduces size of generated files by 5.8%
(62870496-59410354=3460142 bytes saved).
|
| |
|
|
|
|
|
| |
They can't be a bitmask. This fixes important cases
of "0, 1" and "0, 1, 2" flags. Fix some descriptions
that added 0 to bitmasks explicitly (we should do it
automatically instead).
|
| |
|
|
|
|
| |
Will simplify runtime analysis of flags.
Also just no reason to make it more deterministic
and avoid unnecessary diffs in future if values are reordered.
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
Signed-off-by: Paul Chaignon <paul.chaignon@orange.com>
|
| |
|
|
|
|
|
|
|
|
|
|
| |
* Initial Commit
* working build of network packet fuzzing
* Add missed csource file
* pkg/csource: fix build
* executor/common_bsd.h: Add comment stating reason for ifconfig create
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
* sys/netbsd: Adding more syscalls
* Added lwp create structures
* Added a few lwp flags
* completed _lwp* syscalls
* add clone syscall
* fix errors, recheck arguments
|
| | |
|
| |
|
|
|
|
|
|
|
| |
Using a build tag to exclude files for golangci-lint
reduces memory consumption (it does not parse them).
The naive attempt with skip-dirs did not work.
So add codeanalysis build tag and use it in auto-generated files.
Update #977
|
| |
|
|
| |
Change the generated format for len type to support multiple path elements.
|
| |
|
|
|
|
|
|
|
|
|
| |
Ptr type has special handling of direction (pointers are always input).
But buffer type missed this special case all the time.
Make buffer less special by aliasing to the ptr[array[int8]] type.
As the result buffer type can't have optional trailing "opt" attribute
because we don't have such support for templates yet.
Change such cases to use ptr type directly.
Fixes #1097
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
We don't specify trailing unused args for some syscalls
(e.g. ioctl that does not use its arg).
Executor always filled tailing unsed args with 0's
but pkg/csource didn't. Some such syscalls actually
check that the unsed arg is 0 and as the result failed with C repro.
We could statically check and eliminate all such cases,
but it turns out the warning fires in 1500+ cases:
https://gist.githubusercontent.com/dvyukov/e59ba1d9a211ee32fa0ba94fab86a943/raw/a3ace5a63f7281f0298f51ea9842ead1e4713418/gistfile1.txt
So instead fill such args with 0's in pkg/csource too.
|
| | |
|