aboutsummaryrefslogtreecommitdiffstats
path: root/sys/test/gen
Commit message (Collapse)AuthorAgeFilesLines
* sys/syz-sysgen: serialize descriptions as gob and embedDmitry Vyukov2025-01-231-6/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Instead of generating Go files with descriptions serialize them as gob and compress with flate. This significantly reduces build time, go vet time, and solves scalability problems with some static analysis tools. Reference times (all after rm -rf ~/.cache/go-build) before: TIME="%e %P %M" time go install ./syz-manager 48.29 577% 4824820 TIME="%e %P %M" time go test -c ./prog 56.28 380% 6973292 After: TIME="%e %P %M" time go install ./syz-manager 22.81 865% 859788 TIME="%e %P %M" time go test -c ./prog 12.74 565% 267760 syz-manager size before/after: 194712597 -> 83418407 -57% even provided we now embed all descriptions instead of just a single arch. Deflate/decoding time for a single Linux arch is ~330ms. Fixes #5542
* sys: commit empty source filesDmitry Vyukov2021-04-211-0/+6
| | | | | | Commit empty files into generated packages, so that the tree is buildable even w/o generated files and Go tools continue to work.
* Makefile: generate descriptions on-the-flyDmitry Vyukov2020-04-305-8380/+0
| | | | | | | | | | | | | | | | | | Checking in the generated descriptions files makes few things simpler, but causes pain for pull requests: (1) PRs that touch descriptions _always_ conflict, (2) PRs are large and harder to review, (3) people sometimes forget to add auto-generated files. The proposed way does not require us to hardcode lots of dependencies in the Makefile (which is nice) and seem to work. Let's see how it works. The main contributor-visible consequence is that the auto-generated files do not need to be checked-in now. Credit for figuring the Makefile magic goes to @melver. Fixes #1291
* pkg/compiler: deduplicate Types in descriptionsDmitry Vyukov2020-04-264-3237/+5812
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add prog.Ref Type that serves as a proxy for real types and allows to deduplicate Types in generated descriptions. The Ref type is effectively an index in an array of types. Just before serialization pkg/compiler replaces real types with the Ref types and prepares corresponding array of real types. When a Target is registered in prog package, we do the opposite operation and replace Ref's with the corresponding real types. This brings improvements across the board: compiler memory consumption is reduced by 15%, test building time by 25%, descriptions size by 33%. Before: $ du -h sys/linux/gen 54M sys/linux/gen $ time GOMAXPROCS=1 go test -p=1 -c ./prog real 0m54.200s real 0m53.883s $ time GOMAXPROCS=1 go install -p=1 ./tools/syz-execprog real 0m27.911s real 0m27.767s $ TIME="%e %P %M" GOMAXPROCS=1 time go tool compile ./sys/linux/gen 20.59 100% 3200016 20.97 100% 3445976 20.25 100% 3209684 After: $ du -h sys/linux/gen 36M sys/linux/gen $ time GOMAXPROCS=1 go test -p=1 -c ./prog real 0m42.290s real 0m43.230s $ time GOMAXPROCS=1 go install -p=1 ./tools/syz-execprog real 0m24.337s real 0m24.727s $ TIME="%e %P %M" GOMAXPROCS=1 time go tool compile ./sys/linux/gen 19.11 100% 2764952 19.66 100% 2787624 19.35 100% 2749376 Update #1580
* prog: add ignore_return and breaks_returns call attribtuesDmitry Vyukov2020-04-194-20/+12
| | | | | We had these hard-coded for fuchsia and linux accordingly. Replace with call attributes.
* pkg/compiler: check that flags values fit into base typeDmitry Vyukov2020-03-174-12/+12
| | | | | | | | flags[foo, int8] foo = 0x12345678 is always an error, detect these cases. Found some bugs in mptcp, packet sockets, kvm.
* pkg/compiler: check that const values fit into base typeDmitry Vyukov2020-03-174-8/+8
| | | | | const[0x12345678, int8] is always an error, detect these cases. Found some bugs in mptcp, socket proto and fuchsia fidl descriptions.
* pkg/compiler: calculate more precise sizes for argumentsDmitry Vyukov2020-03-174-80/+80
| | | | | | | | | | | | | | | | | | | | If we have: ioctl(fd fd, cmd int32) ioctl$FOO(fd fd, cmd const[FOO]) Currently we assume that cmd size in ioctl$FOO is sizeof(void*). However, we know that in ioctl it's specified as int32, so we can infer that the actual syscall size is 4. This massively reduces sizes of socket/setsockopt/getsockopt/ioctl and some other syscalls, which is good because we now use physical size in mutation/hints and some other places. This will also enable not morphing ioctl's into other ioctl's. Update #477 Update #502
* pkg/compiler: ensure consistency of syscall argument typesDmitry Vyukov2020-03-174-188/+236
| | | | | | | | | | | | | | | | | | Ensure that we don't have conflicting sizes for the same argument of the same syscall, e.g.: foo$1(a int16) foo$2(a int32) This is useful for several reasons: - we will be able avoid morphing syscalls into other syscalls - we will be able to figure out more precise sizes for args (lots of them are implicitly intptr, which is the largest type on most important arches) - found few bugs in linux descriptions Update #477 Update #502
* pkg/compiler: don't specify syscall consts for test OSDmitry Vyukov2020-03-173-6/+2925
| | | | This is just tedious. Fabricate them on the fly.
* prog: remove unused ResourceDesc.TypeDmitry Vyukov2020-01-261-8/+8
|
* pkg/serializer: do not write field names if it won't save spaceDmitry Vyukov2020-01-264-630/+630
| | | | | | | If we are going to write all values, don't write field names. This only increases size of generated files. The change reduces size of generated files by 5.8% (62870496-59410354=3460142 bytes saved).
* pkg/compiler: don't mark flags with 0 as bitmaskDmitry Vyukov2020-01-184-12/+12
| | | | | | | They can't be a bitmask. This fixes important cases of "0, 1" and "0, 1, 2" flags. Fix some descriptions that added 0 to bitmasks explicitly (we should do it automatically instead).
* prog: don't add fallback coverage after prctlDmitry Vyukov2020-01-151-1/+3
| | | | The same reason as with seccomp.
* sys/linux: fix 2 netlink data layout bugsDmitry Vyukov2020-01-074-7/+319
| | | | | | | | | | | 1. Turns out that NLA_F_NESTED is actually used and checked (nla_parse_nested checks it, while nla_parse_nested_deprecated does not). Similarly, ipset extensively checks NLA_F_NET_BYTEORDER. So we need these bits. 2. nla_len must not account for the trailing alighnment padding. This means we set wrong len for payloads that are not multiple of 4 (int8/int16/strings/arrays/some structs/etc).
* pkg/compiler: fix another bitfield layout bugDmitry Vyukov2020-01-074-18/+81
| | | | See the added test for details.
* prog: fix tests for string enforcementDmitry Vyukov2020-01-051-3/+3
| | | | | | | | String value enforcement broke a number of tests where we use different values. Be more string as to what string values we use in tests. Required to add tmpfs descriptions to test syz_mount_image. Also special-casing AF_ALG algorithms as these are auto-generated.
* prog: don't mutate strings with enumerated valuesDmitry Vyukov2020-01-051-1/+7
| | | | | | | | | | Strings with enumerated values are frequently file names or have complete enumeration of relevant values. Mutating complete enumeration if not very profitable. Mutating file names leads to escaping paths and fuzzer messing with things it is not supposed to mess with as in: r0 = openat$apparmor_task_exec(0xffffffffffffff9c, &(0x7f0000000440)='/proc/self//exe\x00', 0x3, 0x0)
* sys/linux: fix int64 alignment on 386Dmitry Vyukov2019-12-231-12/+11
| | | | | | | | | Turns out int64 alignment is 4 on 386... But on arm it's still 8. Another amusing finding thanks to syz-check. Update #590
* prog: don't fail decoding on non-default out argsDmitry Vyukov2019-12-211-1/+4
| | | | | | | We get them in cross-compilation test where an out const arg has different values in different archs. No reason to fail deserialization in that case, replace with default arg instead.
* pkg/compiler: fix bitfield layout bugDmitry Vyukov2019-12-204-448/+370
| | | | | | Fixes #1542 Found thanks to syz-check. Update #590
* sys/test: and another set of bitfield testsDmitry Vyukov2019-12-204-4/+540
| | | | | | Just trying to get my head around it (and fix this in tests). Update #1542
* sys/test: and another bitfield testDmitry Vyukov2019-12-194-4/+52
| | | | | | Just trying to get my head around it (and fix this in tests). Update #1542
* prog: refactor bitfields representationDmitry Vyukov2019-12-194-187/+187
| | | | | | | | All callers of BitfieldMiddle just want static size (0 for middle). Make it so: Size for middle bitfields just returns 0. Removes lots of if's. Introduce Type.UnitSize, which now holds the underlying type for bitfields. This will be needed to fix #1542 b/c even if UnitSize=4 for last bitfield Size can be anywhere from 0 to 4 (not necessary equal to UnitSize due to overlapping).
* sys/test: add few other tests for tricky bitfieldsDmitry Vyukov2019-12-184-4/+204
| | | | Update #1542
* sys/test: add more tests for bitfieldsDmitry Vyukov2019-12-134-4/+296
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add tests for issue #1542 The correct results are obtained with the following program: struct foo { unsigned char f0; unsigned int f1:4; unsigned short f2:4; }; struct bar { char f0; struct foo foo; }; int main() { struct bar y; memset(&y, 0, sizeof(y)); y.f0 = 0x12; y.foo.f0 = 0x34; y.foo.f1 = 0x56; y.foo.f2 = 0x78; int i; for (i = 0; i < sizeof(y); i++) printf("%02x", ((unsigned char*)&y)[i]); printf("\n"); }
* pkg/compiler: define fileoff templatePaul Chaignon2019-11-011-17/+17
| | | | Signed-off-by: Paul Chaignon <paul.chaignon@orange.com>
* prog: mutate length of output buffersVeronica Radu2019-10-101-1/+7
| | | | Update #480
* prog: implemented argument and call prioritiesVeronica Radu2019-09-041-1/+7
|
* prog: add special mutation for binary flagsVeronica Radu2019-08-091-1/+39
|
* prog: don't minimize ProcType to 0Dmitry Vyukov2019-07-261-1/+6
| | | | | | | | Default value for ProcType is 0 (same for all PID's). Usually 0 either does not make sense at all or make different PIDs collide (since we use ProcType to separate value ranges for different PIDs). So don't change ProcType to 0 unless the type is explicitly marked as opt (in that case we will also generate 0 anyway).
* pkg/compiler: add offsetof typeDmitry Vyukov2019-05-161-1/+25
| | | | | | Similar to C offsetof gives offset of a field from the beginning of the parent struct. We have several TODOs in descriptions asking for this.
* pkg/compiler: work around 0-array-size errors due to missing constsDmitry Vyukov2019-05-151-3/+7
| | | | | | | A const can be used as array size. Then if the const is not present on all arches, compiler will produce an error about 0-sized-array. There is no easy way to work around this for a user. Use value of 1 for missing consts. It's just a bit safer.
* .golangci.yml: add codeanalysis build tagDmitry Vyukov2019-05-154-4/+8
| | | | | | | | | Using a build tag to exclude files for golangci-lint reduces memory consumption (it does not parse them). The naive attempt with skip-dirs did not work. So add codeanalysis build tag and use it in auto-generated files. Update #977
* pkg/compiler: allow to refer to syscall arguments in len pathsDmitry Vyukov2019-05-141-2/+4
| | | | This allows to use len[syscall:arg] expressions.
* prog: implement complex len target supportDmitry Vyukov2019-05-141-4/+33
| | | | | This actually implements support for complex len targets during program generation and mutation.
* pkg/compiler: generate complex len targetsDmitry Vyukov2019-05-144-102/+106
| | | | Change the generated format for len type to support multiple path elements.
* sys/test: remove a leftover fileDmitry Vyukov2019-05-141-865/+0
|
* pkg/compiler: make buffer alias to ptr[array[int8]]Dmitry Vyukov2019-04-011-2/+2
| | | | | | | | | | | Ptr type has special handling of direction (pointers are always input). But buffer type missed this special case all the time. Make buffer less special by aliasing to the ptr[array[int8]] type. As the result buffer type can't have optional trailing "opt" attribute because we don't have such support for templates yet. Change such cases to use ptr type directly. Fixes #1097
* sys/test/test: add tests for test exiting in the middle of executionDmitry Vyukov2019-01-314-4/+16
|
* pkg/csource: use 0 for missing syscall argsDmitry Vyukov2018-12-274-98/+170
| | | | | | | | | | | | | | We don't specify trailing unused args for some syscalls (e.g. ioctl that does not use its arg). Executor always filled tailing unsed args with 0's but pkg/csource didn't. Some such syscalls actually check that the unsed arg is 0 and as the result failed with C repro. We could statically check and eliminate all such cases, but it turns out the warning fires in 1500+ cases: https://gist.githubusercontent.com/dvyukov/e59ba1d9a211ee32fa0ba94fab86a943/raw/a3ace5a63f7281f0298f51ea9842ead1e4713418/gistfile1.txt So instead fill such args with 0's in pkg/csource too.
* prog, pkg/csource: more readable serialization for stringsDmitry Vyukov2018-12-154-12/+30
| | | | | | | Always serialize strings in readable format (non-hex). Serialize binary data in readable format in more cases. Fixes #792
* prog: support AUTO args in programsDmitry Vyukov2018-12-101-1/+12
| | | | | | | | | | | | | | | AUTO arguments can be used for: - consts - lens - pointers For const's and len's AUTO is replaced with the natural value, addresses for AUTO pointers are allocated linearly. This greatly simplifies writing test programs by hand as most of the time we want these natural values. Update tests to use AUTO.
* executor: fix handling of big-endian bitfieldsDmitry Vyukov2018-12-084-6/+147
| | | | | Currently we apply big-endian-ness and bitfield-ness in the wrong order in copyin. This leads to totally bogus result. Fix this.
* sys/linux: add syz_execute_funcDmitry Vyukov2018-08-304-8/+20
| | | | | | The function executes random code. Update #310
* prog: don't add fallback signal after seccompDmitry Vyukov2018-08-191-2/+9
| | | | | seccomp filter can produce arbitrary errno values for subsequent syscalls. Don't trust anything afterwards.
* sys/test: add more testsDmitry Vyukov2018-08-034-7/+98
| | | | | | | | | | | | | | Add syz_errno syscall which sets errno to the argument, and add a test with different errno values. This mostly tests the testing infrastructure itself. Add syz_compare syscall which compare two blobs, this can be used for testing of argument memory layout. Implement syz_mmap and fix Makefile to allow building syz-execprog for test OS. Useful for debugging. Update #603
* Makefile: don't compile all targets into target binariesDmitry Vyukov2018-08-025-8/+27
| | | | | | | | | | Currently target binaries contain support for all OS/arch combinations. However, obviously a fuchsia target binary won't test windows. For target binaries we need support only for a single target (with the exception of 386/arm target in amd64/arm64 binaries). So compile in only _the_ target into target binaries. This reduces akaros/amd64 fuzzer binary from 33 to 7 MB and execprog from 28 to 2 MB.
* executor: overhaulDmitry Vyukov2018-07-244-85/+180
| | | | | | | | | | | | | | | | | Make as much code as possible shared between all OSes. In particular main is now common across all OSes. Make more code shared between executor and csource (in particular, loop function and threaded execution logic). Also make loop and threaded logic shared across all OSes. Make more posix/unix code shared across OSes (e.g. signal handling, pthread creation, etc). Plus other changes along similar lines. Also support test OS in executor (based on portable posix) and add 4 arches that cover all execution modes (fork server/no fork server, shmem/no shmem). This change paves way for testing of executor code and allows to preserve consistency across OSes and executor/csource.
* prog, pkg/compiler: support fmt typeDmitry Vyukov2018-07-082-24/+70
| | | | | fmt type allows to convert intergers and resources to string representation.
generated by ggit (джигит) mrf-deployment (git 2.46.0) at 2026-03-12 03:34:23 +0000