From e0740dc9fe2c4e5f15f1b155859cce2dc93024cb Mon Sep 17 00:00:00 2001 From: Dmitry Vyukov Date: Sun, 18 Oct 2020 18:37:26 +0200 Subject: dashboard/config/linux: update README for syz-kconf Update #2171 --- dashboard/config/linux/README.md | 116 +++++++++++++++++++++++++++++++++++---- 1 file changed, 105 insertions(+), 11 deletions(-) diff --git a/dashboard/config/linux/README.md b/dashboard/config/linux/README.md index 237c15df6..8b6f846fa 100644 --- a/dashboard/config/linux/README.md +++ b/dashboard/config/linux/README.md @@ -1,14 +1,108 @@ -This dir contains kernel configs used by [syzbot](/docs/syzbot.md). +This dir contains Linux kernel configs used by [syzbot](/docs/syzbot.md). -# Updating Linux configs +Configs are auto-generated by [syz-kconf](/tools/syz-kconf/kconf.go) utility from declaration file [main.yml](main.yml) +and config [fragments](bits/). [main.yml](main.yml) specifies list of target config files (instances) and their +`features`, as well as order of inclusion of config fragments and their constraints. Config fragment is used for +an instance if all fragment constraints are present in instance features and all negative constraints (`-foo`) are not +present in instance features. -To update the main linux config [upstream-kasan.config](./upstream-kasan.config) used by `syzbot`: +For example, the following declaration file: +``` +instances: + - instance-a: [x86_64, gcc, upstream] + - instance-b: [x86_64, gcc, upstream, foo] + - instance-c: [arm64, clang, stable] -1. Check out latest `linux-next` (we tend to use `linux-next` as it contains the most latest features in a single tree). -2. Copy the config into kernel tree as `.config`. -3. Run `make olddefconfig`. -4. Make any additional changes to the config you want to do (e.g. enable/disable some configs with `make menuconfig`). -5. Build and boot the kernel. Ensure there are no bugs during boot (in particular, `WARNING`'s and `LOCKDEP` reports). -6. Copy back the [custom configs](https://github.com/google/syzkaller/blob/1f448cd62db290246f8793128f85bd84aaa7a59d/dashboard/config/upstream-kasan.config#L6-L14) into the `.config` (see comments there). -7. For compiler you need to use a recent gcc (8+). Some of the debugging configs may be disabled due to old/different compiler, in particular `CONFIG_KCOV_ENABLE_COMPARISONS`. You may also restore the [compiler info part](https://github.com/google/syzkaller/blob/1f448cd62db290246f8793128f85bd84aaa7a59d/dashboard/config/upstream-kasan.config#L16-L20) just to reduce the diff size. -8. Copy the config back as `upstream-kasan.config` and send a PR. It will be deployed to `syzbot` within a day after merging. +includes: + - always.yml: [] + - only-upstream.yml: [upstream] + - only-clang-arm64.yml: [clang, arm64] + - only-upstream-not-foo.yml: [upstream, -foo] +``` +specifies 3 target configs (instances) and built from 4 config fragments. Here fragment `always.yml` is included first +and for all instances. Fragment `only-upstream.yml` is included next for instances that have `upstream` feature listed +(`instance-a` and `instance-b`). Fragment `only-clang-arm64.yml` is included next for instances that have both +`clang` and `arm64` listed (`instance-c`). Finally, fragment `only-upstream-not-foo.yml` is included for instances +that have `upstream` features but don't have `foo` feature listed (`instance-a`). + +Config fragments can specify several aspects of the target config file. First, they can specify the kernel repository +and revision that will be used for config generation using the following syntax. There must be only one repository +specification for each instance. +``` +kernel: + repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git + tag: v5.8-rc7 +``` + +Then, config fragments can specify kernel `make` commands that will be invoked to create the base config. +For example: +``` +shell: + - make x86_64_defconfig + - make kvm_guest.config +``` + +Shell commands can have constraints similar to includes, for example: +``` +shell: + - make defconfig: [-nodefconfig] +``` + +Then, fragments can specify text for verbatim inclusion at the end of the resulting config. `syz-kconf` does not do +anything with these string. For example: +``` +verbatim: | + CONFIG_THIS_CONFIG_DOES_NOT_EXIST_YET=y +``` + +Finally, fragments can specify individual config options that need to be enabled (or disabled) in the resulting config. +For example: +``` +config: + - CONFIG_WILL_BE_ENABLED + - CONFIG_WILL_BE_DISABLED: n + - CONFIG_WITH_INTEGER_VALUE: 10 + - CONFIG_WITH_STRING_VALUE: "foo" +``` + +Config options can have constraints for more precise control over target instances +(constraints can be combined with values): +``` +config: + - CONFIG_ONLY_FOR_UPSTREAM: [upstream] + - CONFIG_STABLE_AND_ARM64: [stable, arm64] + - CONFIG_NOT_CLANG: [-clang] + - CONFIG_DISABLED_FOR_GCC: [n, -gcc] + - CONFIG_STRING_FOO: ["value", foo] +``` + +Some features/constraints are defined by `syz-kconf` itself or have special meaning: + - `override` on a config allows to override a previous definition of the same config (otherwise it results in an error) + - `optional` on a config disables checking that the resulting kernel config file has the specified value for this + config (otherwise it results in an error) + - `weak` means both `override` and `optional` + - `clang`/`gcc` instance feature controls what compiler is used for config generation + - kernel arch name (`x86_64`, `arm64`, etc) instance feature controls architecture of the kernel config + - kernel release tag is automatically added to instance features and allows to specify a minimum kernel version + for a config (e.g. `v5.5` for a config added in that kernel release) or a maximum kernel version (e.g. `-v5.5` for + a config removed in that kernel release) + +`syz-kconf` generates 2 kernel config files for each instance: one `normal` (or full) and one `baseline`. +The `baseline` config has `-base` suffix and has `baseline` feature added. The `baseline` config is meant to not +contain majority of subsystem configs enabled (should just boot and have debugging configs enabled), however, +that is fully controlled by config fragments based on the presence of the `baseline` feature. `baseline` configs +are used in config minization procedure. + +The only exception to the declarative nature of the process is USB configs. They are added procedurally, see +`addUSBConfigs` function for details. + +While `syz-kconf` switches between `gcc` and `clang` compilers automatically, the compilers themselves are not +part of `syz-kconf` and are supposed to be installed on the system. `gcc` should be version 9+ and `clang` +should be 11+. + +To update kernel configs: + - change config fragments as necessary (e.g. add additional configs to [subsystems.yml](bits/subsystems.yml) along + with minimal kernel version) + - run `make configs SOURCEDIR=/path/to/existing/linux/checkout` (note: it will be mrproper-ed) + - check in config fragments and changed kernel configs and send a PR + - changes will be deployed to `syzbot` within a day after merging -- cgit mrf-deployment