aboutsummaryrefslogtreecommitdiffstats
path: root/pkg/compiler/gen.go
Commit message (Collapse)AuthorAgeFilesLines
* pkg/compiler: fix struct layout bugDmitry Vyukov2025-01-201-0/+13
| | | | | | | | | | | | | | | Currently we have a bug in struct layout that affects some corner cases that involve recursive structs. The result of this bug is that we use wrong alignment 1 (not yet calculated) for some structs when calculating layout of other structs. The root cause of this bug is that we calculate struct alignment too early in typeStruct.Gen when structs are not yet laid out. For this reason we moved struct size calculation to the later phase (after compiler.layoutStruct). Move alignment calculation from typeStruct.Gen to compiler.layoutStruct to fix this.
* all: use min/max functionsDmitry Vyukov2025-01-171-9/+3
| | | | They are shorter, more readable, and don't require temp vars.
* pkg/compiler: add automatic metaDmitry Vyukov2024-12-111-8/+11
| | | | | | Mark the whole file with "meta automatic" instead of marking each syscall. This reduces size of descriptions + allows to do special things with the whole file (e.g. we already treat auto consts specially).
* pkg/compiler: handle string syscall attributesFlorent Revest2024-12-091-3/+12
|
* pkg/compiler: allow recursion via arraysDmitry Vyukov2024-11-181-1/+1
| | | | | Permit structs to recursively contain itself in arrays. This is needed for netlink. Amusingly several netlink policies contain itself.
* all: support || operator in syzlang if conditionJiao, Joey2024-11-131-0/+1
| | | | | | | | | | | ex. f3 field has logic or operator in if condition: conditional_struct { mask int32 f1 field1 (if[value[mask] & FIELD_FLAG1]) f2 int64 (if[value[mask] & FIELD_FLAG2]) f3 int64 (if[value[mask] == FIELD_FLAG1 || value[mask] == FIELD_FLAG2]) } [packed]
* pkg/mgrconfig, prog, tools: allow automatically generated or manually ↵Pimyn Girgis2024-08-121-3/+12
| | | | | | written descriptions or both Add "Auto" type and allow to choose descriptions mode in configurations. Defaults to using manual only.
* prog: fix panic during squashingDmitry Vyukov2024-05-031-0/+8
| | | | | | | | | Netbsd syzbot instance crashes trying to squash a pointer. Pointers must not be squashed. This happens because of recursive ucontext_t type that contains a pointer to itself. When we assign SquashableElem recursive struct types may not be fully generated yet, and ForeachArgType won't observe all types. Assign SquashableElem after all types are fully generated.
* prog: handle multiple matching union fieldsAleksandr Nogikh2024-02-191-0/+5
| | | | | | | | If conditions of several union fields are satisfied, select one randomly. This would be a more logical semantics. When conditional struct fields are translated to unions, negate the condition for the union alternative.
* pkg/compiler: support if[expr] attributesAleksandr Nogikh2024-02-191-12/+129
| | | | | | | | | | | | | | | | | | | The expression may either include integers/consts or reference other fields in the structure via value[field1:field2:field3]. The fields on this path must all belong to structures and must not have any if conditions themselves. For unions, mandate that the last field has no conditions (it will be the default one). For structs, convert conditional fields into fields of a union type of the following form: anonymous_union [ value T (if[expression]) void void ]
* prog: pass ctx by pointer to ForeachType callbackDmitry Vyukov2022-01-111-2/+2
| | | | | This will allow callbacks to stop iteration early by setting ctx.Stop flag (as it works for ForeachArg).
* pkg/compiler: require stricter resource constructorsDmitry Vyukov2022-01-111-1/+2
| | | | | | | | | | | | | Don't consider syscalls that return resources in unions/arrays as constructors. Unions and arrays are problematic because we don't have directed generation in prog.randGen.createResource() and can fail to generate a syscall that returns a particular resource (generate a wrong union option that does not contain the necessary resource). This leads to the following panics: panic: failed to create a resource ifindex with ioctl$sock_SIOCGIFCONF Require each resource to have a constructor syscall that returns the resource outside of unions/arrays.
* pkg/compiler: prohibit use of direction attribute on union fieldsDmitry Vyukov2022-01-111-2/+2
| | | | | | Direction attributes on unions work in a confusing way and don't do what users may think they do. Now we have out_overlay attribute for structs that allows to have overlapping input and output fields.
* pkg/compiler: add out_overlay field attributeDmitry Vyukov2022-01-111-12/+45
|
* pkg/compiler: refactor codeDmitry Vyukov2022-01-111-3/+7
| | | | | Slightly refactor code in preparation for future changes. No functional changes intended.
* pkg, prog: add per-field direction attributeNecip Fazil Yildiran2020-08-131-2/+20
|
* prog/types.go: add Type.Alignment() and TypeCommon.TypeAlignAlbert van der Linde2020-07-141-53/+1
| | | | | Type.Alignment() can be used to obtain byte alignment for correctly allocating aligned memory for the Type.
* prog: remove StructDescDmitry Vyukov2020-05-031-170/+80
| | | | | | | | | | | | Remove StructDesc, KeyedStruct, StructKey and all associated logic/complexity in prog and pkg/compiler. We can now handle recursion more generically with the Ref type, and Dir/FieldName are not a part of the type anymore. This makes StructType/UnionType simpler and more natural. Reduces size of sys/linux/gen/amd64.go from 5201321 to 4180861 (-20%). Update #1580
* prog: introduce Field typeDmitry Vyukov2020-05-021-29/+34
| | | | | | | | | | | | | Remvoe FieldName from Type and add a separate Field type that holds field name. Use Field for struct fields, union options and syscalls arguments, only these really have names. Reduces size of sys/linux/gen/amd64.go from 5665583 to 5201321 (-8.2%). Allows to not create new type for squashed any pointer. But main advantages will follow, e.g. removing StructDesc, using TypeRef in Arg, etc. Update #1580
* prog: rename {PtrType,ArrayType}.Type to ElemDmitry Vyukov2020-05-011-8/+8
| | | | | | | Name "Type" is confusing when referring to pointer/array element type. Frequently there are too many Type/typ/typ1/t and typ.Type is not very informative. It _is_ a type, but what's usually more relevant is that it's an _element_ type. Let's leave type checking to compiler and give it a more meaningful name.
* prog: remove Dir from TypeDmitry Vyukov2020-05-011-20/+16
| | | | | | | | | | | | | | | | | | Having Dir is Type is handy, but forces us to duplicate lots of types. E.g. if a struct is referenced as both in and out, then we need to have 2 copies and 2 copies of structs/types it includes. If also prevents us from having the struct type as struct identity (because we can have up to 3 of them). Revert to the old way we used to do it: propagate Dir as we walk syscall arguments. This moves lots of dir passing from pkg/compiler to prog package. Now Arg contains the dir, so once we build the tree, we can use dirs as before. Reduces size of sys/linux/gen/amd64.go from 6058336 to 5661150 (-6.6%). Update #1580
* pkg/compiler: deduplicate Types in descriptionsDmitry Vyukov2020-04-261-0/+69
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add prog.Ref Type that serves as a proxy for real types and allows to deduplicate Types in generated descriptions. The Ref type is effectively an index in an array of types. Just before serialization pkg/compiler replaces real types with the Ref types and prepares corresponding array of real types. When a Target is registered in prog package, we do the opposite operation and replace Ref's with the corresponding real types. This brings improvements across the board: compiler memory consumption is reduced by 15%, test building time by 25%, descriptions size by 33%. Before: $ du -h sys/linux/gen 54M sys/linux/gen $ time GOMAXPROCS=1 go test -p=1 -c ./prog real 0m54.200s real 0m53.883s $ time GOMAXPROCS=1 go install -p=1 ./tools/syz-execprog real 0m27.911s real 0m27.767s $ TIME="%e %P %M" GOMAXPROCS=1 time go tool compile ./sys/linux/gen 20.59 100% 3200016 20.97 100% 3445976 20.25 100% 3209684 After: $ du -h sys/linux/gen 36M sys/linux/gen $ time GOMAXPROCS=1 go test -p=1 -c ./prog real 0m42.290s real 0m43.230s $ time GOMAXPROCS=1 go install -p=1 ./tools/syz-execprog real 0m24.337s real 0m24.727s $ TIME="%e %P %M" GOMAXPROCS=1 time go tool compile ./sys/linux/gen 19.11 100% 2764952 19.66 100% 2787624 19.35 100% 2749376 Update #1580
* pkg/compiler: simplify sort predicateDmitry Vyukov2020-04-251-4/+4
|
* prog: introduce call attributesDmitry Vyukov2020-04-191-0/+12
| | | | | | Add common infrastructure for syscall attributes. Add few attributes we want, but they are not implemented for now (don't affect behavior, this will follow).
* pkg/compiler: refactor attribute handlingDmitry Vyukov2020-04-191-22/+26
| | | | | | | | | | | | Introduce common infrastructure for describing and parsing attribute instead of custom per-attribute code scattered across several locations. Change align attribute syntax from the weird align_N to align[N]. This also allows to use literal constants as N. Introduce notion of builtin constants. Currently we have only PTR_SIZE, which is needed to replace align_ptr with align[PTR_SIZE].
* pkg/compiler: check that const values fit into base typeDmitry Vyukov2020-03-171-42/+24
| | | | | const[0x12345678, int8] is always an error, detect these cases. Found some bugs in mptcp, socket proto and fuchsia fidl descriptions.
* pkg/compiler: calculate more precise sizes for argumentsDmitry Vyukov2020-03-171-4/+31
| | | | | | | | | | | | | | | | | | | | If we have: ioctl(fd fd, cmd int32) ioctl$FOO(fd fd, cmd const[FOO]) Currently we assume that cmd size in ioctl$FOO is sizeof(void*). However, we know that in ioctl it's specified as int32, so we can infer that the actual syscall size is 4. This massively reduces sizes of socket/setsockopt/getsockopt/ioctl and some other syscalls, which is good because we now use physical size in mutation/hints and some other places. This will also enable not morphing ioctl's into other ioctl's. Update #477 Update #502
* pkg/compiler: add tests for generation phaseDmitry Vyukov2020-03-171-3/+3
| | | | | | Add errors3.txt with tests for errors that are produced during generation phase. Refactor tests to reduce duplication. Tidy struct/union size errors: better locations and make testable.
* pkg/compiler: ensure consistency of syscall argument typesDmitry Vyukov2020-03-171-7/+50
| | | | | | | | | | | | | | | | | | Ensure that we don't have conflicting sizes for the same argument of the same syscall, e.g.: foo$1(a int16) foo$2(a int32) This is useful for several reasons: - we will be able avoid morphing syscalls into other syscalls - we will be able to figure out more precise sizes for args (lots of them are implicitly intptr, which is the largest type on most important arches) - found few bugs in linux descriptions Update #477 Update #502
* prog: remove unused ResourceDesc.TypeDmitry Vyukov2020-01-261-3/+0
|
* pkg/compiler: fix another bitfield layout bugDmitry Vyukov2020-01-071-0/+10
| | | | See the added test for details.
* sys/linux: fix int64 alignment on 386Dmitry Vyukov2019-12-231-6/+8
| | | | | | | | | Turns out int64 alignment is 4 on 386... But on arm it's still 8. Another amusing finding thanks to syz-check. Update #590
* pkg/compiler: fix bitfield layout bugDmitry Vyukov2019-12-201-51/+105
| | | | | | Fixes #1542 Found thanks to syz-check. Update #590
* pkg/compiler: fix alignment of string-formatted valuesDmitry Vyukov2019-12-201-0/+7
| | | | | | We used size as alignment, this is very wrong. Found thanks to syz-check. Update #590
* pkg/compiler: refactor struct layoutDmitry Vyukov2019-12-201-61/+47
| | | | | | | | | Combine markBitfields and addAlignment functions. Fixing #1542 will require doing both at the same time, they are not really independent. Also remove the special case for packed structs, pad them as part of the common procedure. No functional changes.
* prog: refactor bitfields representationDmitry Vyukov2019-12-191-17/+17
| | | | | | | | All callers of BitfieldMiddle just want static size (0 for middle). Make it so: Size for middle bitfields just returns 0. Removes lots of if's. Introduce Type.UnitSize, which now holds the underlying type for bitfields. This will be needed to fix #1542 b/c even if UnitSize=4 for last bitfield Size can be anywhere from 0 to 4 (not necessary equal to UnitSize due to overlapping).
* pkg/compiler: fix incorrect alignment calculation for paddingDmitry Vyukov2019-12-181-5/+7
| | | | | | | | | | | | | | We assumed that for ConstType alignment is equal to size, which is perfectly reasonable for normal int8/16/32/64/ptr. However, padding is also represented by ConstType of arbitrary size, so if we added 157 bytes of padding that becomes alignment of the padding field and as the result of the whole struct. This affects very few structs, but quite radically and quite important structs. Discovered thanks to syz-check. Update #590
* pkg/csource: use 0 for missing syscall argsDmitry Vyukov2018-12-271-7/+16
| | | | | | | | | | | | | | We don't specify trailing unused args for some syscalls (e.g. ioctl that does not use its arg). Executor always filled tailing unsed args with 0's but pkg/csource didn't. Some such syscalls actually check that the unsed arg is 0 and as the result failed with C repro. We could statically check and eliminate all such cases, but it turns out the warning fires in 1500+ cases: https://gist.githubusercontent.com/dvyukov/e59ba1d9a211ee32fa0ba94fab86a943/raw/a3ace5a63f7281f0298f51ea9842ead1e4713418/gistfile1.txt So instead fill such args with 0's in pkg/csource too.
* pkg/compiler: refactor structGenDmitry Vyukov2018-08-021-62/+75
| | | | | | Still too complex. Split more. Update #538
* pkg/compiler: refactor genStructDescsDmitry Vyukov2018-07-311-116/+125
| | | | | | Reduce cyclomatic complexity of genStructDescs. Update #538
* prog, pkg/compiler: support fmt typeDmitry Vyukov2018-07-081-1/+5
| | | | | fmt type allows to convert intergers and resources to string representation.
* pkg/compiler: support size attribute for unionsDmitry Vyukov2018-03-051-3/+13
|
* pkg/compiler: add size attribute for structsDmitry Vyukov2018-03-051-2/+14
| | | | The size attribute allows to pad a struct up to the specified size.
* pkg/compiler: fix alignment corner caseDmitry Vyukov2018-02-251-1/+3
| | | | | Fix alignemnt calculation for packed structs with alignment and bitfields. Amusingly this affected only a single real struct -- ipv6_fragment_ext_header.
* pkg/compiler: don't assign call IDs staticallyDmitry Vyukov2018-02-251-3/+0
| | | | | | IDs change whenever a call is added or removed, this leads to large diffs unnecessarly. Assign IDs dynamically.
* pkg/compiler: fix corner case in alignment calculationDmitry Vyukov2018-02-191-1/+1
|
* pkg/compiler: support void typeDmitry Vyukov2018-01-131-23/+5
| | | | | | "void": type with static size 0 mostly useful inside of templates and varlen unions can't be syscall argument
* pkg/compiler: support type templatesDmitry Vyukov2018-01-131-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Netlink descriptions contain tons of code duplication, and need much more for proper descriptions. Introduce type templates to simplify writing such descriptions and remove code duplication. Note: type templates are experimental, have poor error handling and are subject to change. Type templates can be declared as follows: ``` type buffer[DIR] ptr[DIR, array[int8]] type fileoff[BASE] BASE type nlattr[TYPE, PAYLOAD] { nla_len len[parent, int16] nla_type const[TYPE, int16] payload PAYLOAD } [align_4] ``` and later used as follows: ``` syscall(a buffer[in], b fileoff[int64], c ptr[in, nlattr[FOO, int32]]) ```
* pkg/compiler: don't genererate missing syscallsDmitry Vyukov2017-09-151-8/+11
| | | | | | | | We used to generate them only because manager had no idea what arch it is testing. So syscalls numbers had to match between all arches. This is not needed anymore. Also don't generate unreferenced structs/resources.
* prog, sys: move types to progDmitry Vyukov2017-09-051-56/+56
| | | | | | | | | | | Large overhaul moves syscalls and arg types from sys to prog. Sys package now depends on prog and contains only generated descriptions of syscalls. Introduce prog.Target type that encapsulates all targer properties, like syscall list, ptr/page size, etc. Also moves OS-dependent pieces like mmap call generation from prog to sys. Update #191
generated by ggit (джигит) mrf-deployment (git 2.46.0) at 2026-03-11 21:22:08 +0000