| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead of generating Go files with descriptions
serialize them as gob and compress with flate.
This significantly reduces build time, go vet time,
and solves scalability problems with some static analysis tools.
Reference times (all after rm -rf ~/.cache/go-build) before:
TIME="%e %P %M" time go install ./syz-manager
48.29 577% 4824820
TIME="%e %P %M" time go test -c ./prog
56.28 380% 6973292
After:
TIME="%e %P %M" time go install ./syz-manager
22.81 865% 859788
TIME="%e %P %M" time go test -c ./prog
12.74 565% 267760
syz-manager size before/after: 194712597 -> 83418407
-57% even provided we now embed all descriptions
instead of just a single arch.
Deflate/decoding time for a single Linux arch is ~330ms.
Fixes #5542
|
| |
|
|
|
| |
Create a constructor for the prog.Call type. It allows to reduce
the duplication of code now and during further changes.
|
| |
|
|
|
|
| |
Commit empty files into generated packages,
so that the tree is buildable even w/o generated files
and Go tools continue to work.
|
| |
|
|
|
|
|
|
|
|
| |
If a resource is never used as an input, it is not useful.
It's effectively the same as using an integer.
Detect such cases, they are quite confusing.
Fix all existing errors in descriptions.
This uncovered some interesting bugs as well,
e.g. use of a completely unrelated fd subtype after copy-paste
(while the resource that was supposed to be used there is completely unused).
|
| |
|
|
|
| |
Update the copyright checking script and more files
for the standard convention of marking auto-generated files.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Remvoe FieldName from Type and add a separate Field type
that holds field name. Use Field for struct fields, union options
and syscalls arguments, only these really have names.
Reduces size of sys/linux/gen/amd64.go from 5665583 to 5201321 (-8.2%).
Allows to not create new type for squashed any pointer.
But main advantages will follow, e.g. removing StructDesc,
using TypeRef in Arg, etc.
Update #1580
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Having Dir is Type is handy, but forces us to duplicate lots of types.
E.g. if a struct is referenced as both in and out, then we need to
have 2 copies and 2 copies of structs/types it includes.
If also prevents us from having the struct type as struct identity
(because we can have up to 3 of them).
Revert to the old way we used to do it: propagate Dir as we walk
syscall arguments. This moves lots of dir passing from pkg/compiler
to prog package.
Now Arg contains the dir, so once we build the tree, we can use dirs
as before.
Reduces size of sys/linux/gen/amd64.go from 6058336 to 5661150 (-6.6%).
Update #1580
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Checking in the generated descriptions files makes few things simpler,
but causes pain for pull requests: (1) PRs that touch descriptions
_always_ conflict, (2) PRs are large and harder to review,
(3) people sometimes forget to add auto-generated files.
The proposed way does not require us to hardcode lots of dependencies
in the Makefile (which is nice) and seem to work.
Let's see how it works.
The main contributor-visible consequence is that the auto-generated
files do not need to be checked-in now.
Credit for figuring the Makefile magic goes to @melver.
Fixes #1291
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add prog.Ref Type that serves as a proxy for real types
and allows to deduplicate Types in generated descriptions.
The Ref type is effectively an index in an array of types.
Just before serialization pkg/compiler replaces real types
with the Ref types and prepares corresponding array of real types.
When a Target is registered in prog package, we do the opposite
operation and replace Ref's with the corresponding real types.
This brings improvements across the board:
compiler memory consumption is reduced by 15%,
test building time by 25%, descriptions size by 33%.
Before:
$ du -h sys/linux/gen
54M sys/linux/gen
$ time GOMAXPROCS=1 go test -p=1 -c ./prog
real 0m54.200s
real 0m53.883s
$ time GOMAXPROCS=1 go install -p=1 ./tools/syz-execprog
real 0m27.911s
real 0m27.767s
$ TIME="%e %P %M" GOMAXPROCS=1 time go tool compile ./sys/linux/gen
20.59 100% 3200016
20.97 100% 3445976
20.25 100% 3209684
After:
$ du -h sys/linux/gen
36M sys/linux/gen
$ time GOMAXPROCS=1 go test -p=1 -c ./prog
real 0m42.290s
real 0m43.230s
$ time GOMAXPROCS=1 go install -p=1 ./tools/syz-execprog
real 0m24.337s
real 0m24.727s
$ TIME="%e %P %M" GOMAXPROCS=1 time go tool compile ./sys/linux/gen
19.11 100% 2764952
19.66 100% 2787624
19.35 100% 2749376
Update #1580
|
| |
|
|
|
|
|
|
|
|
|
|
| |
pkg/compiler/compiler.go:182: line is 125 characters
func (comp *compiler) parseAttrs(descs map[string]*attrDesc, parent ast.Node, attrs []*ast.Type) (res map[*attrDesc]uint64) {
sys/targets/common.go:47:21: unnecessary conversion
makeMmap(^uint64(target.PageSize)+1, target.PageSize, 0),
^
sys/targets/common.go:61: File is not `gofmt`-ed with `-s`
&prog.Call{
sys/windows/init.go:35: File is not `gofmt`-ed with `-s`
&prog.Call{
|
| |
|
|
|
|
|
|
|
|
|
| |
Make MakeMmap return more than 1 call.
This is a preparation for future changes.
Also remove addr/size as they are effectively
always the same and can be inferred from the target
(will also conflict with the future changes).
Also rename to MakeDataMmap to better represent
the new purpose: it's just some arbitrary mmap,
but rather mapping of the data segment.
|
| |
|
|
| |
Fixes #1604
|
| | |
|
| |
|
|
|
|
|
| |
If we are going to write all values, don't write field names.
This only increases size of generated files.
The change reduces size of generated files by 5.8%
(62870496-59410354=3460142 bytes saved).
|
| |
|
|
|
|
| |
Will simplify runtime analysis of flags.
Also just no reason to make it more deterministic
and avoid unnecessary diffs in future if values are reordered.
|
| | |
|
| |
|
|
| |
Signed-off-by: Paul Chaignon <paul.chaignon@orange.com>
|
| |
|
|
|
|
|
|
|
| |
Using a build tag to exclude files for golangci-lint
reduces memory consumption (it does not parse them).
The naive attempt with skip-dirs did not work.
So add codeanalysis build tag and use it in auto-generated files.
Update #977
|
| |
|
|
| |
Change the generated format for len type to support multiple path elements.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently when we get target consts with target.ConstMap["name"]
during target initialization, we just get 0 for missing consts.
This is error-prone as we can mis-type a const, or a const may
be undefined only on some archs (as we have common unix code
shared between several OSes).
Check that all the consts are actually defined.
The check detects several violations, to fix them:
1. move mremap to linux as it's only defined on linux
2. move S_IFMT to openbsd, as it's only defined and used on openbsd
3. define missing MAP_ANONYMOUS for freebsd and netbsd
4. fix extract for netbsd
|
| |
|
|
|
|
| |
The function executes random code.
Update #310
|
| |
|
|
|
|
|
|
|
|
| |
Currently target binaries contain support for all OS/arch combinations.
However, obviously a fuchsia target binary won't test windows.
For target binaries we need support only for a single target
(with the exception of 386/arm target in amd64/arm64 binaries).
So compile in only _the_ target into target binaries.
This reduces akaros/amd64 fuzzer binary from 33 to 7 MB
and execprog from 28 to 2 MB.
|
| |
|
|
|
| |
Error on unused structs/unions/resources/flags.
Finds tons of bugs.
|
| | |
|
| |
|
|
|
|
|
|
|
| |
Move generated files to gen subdir. This allows to:
1. Rebuild init.go without rebuilding generated code.
2. Excluding generated files from gometalinter checking.
This makes faster and consume less memory.
Update #538
|
| |
|
|
| |
Update #538
|
| |
|
|
|
|
| |
IDs change whenever a call is added or removed,
this leads to large diffs unnecessarly.
Assign IDs dynamically.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
1. mmap all memory always, without explicit mmap calls in the program.
This makes lots of things much easier and removes lots of code.
Makes mmap not a special syscall and allows to fuzz without mmap enabled.
2. Change address assignment algorithm.
Current algorithm allocates unmapped addresses too frequently
and allows collisions between arguments of a single syscall.
The new algorithm analyzes actual allocations in the program
and places new arguments at unused locations.
|
| |
|
|
|
|
| |
"void": type with static size 0
mostly useful inside of templates and varlen unions
can't be syscall argument
|
| | |
|
| |
|
