dashboard/config/linux: update README for syz-kconf

Update #2171

1 files changed, 105 insertions, 11 deletions

diff --git a/dashboard/config/linux/README.md b/dashboard/config/linux/README.md
index 237c15df6..8b6f846fa 100644
--- a/dashboard/config/linux/README.md
+++ b/dashboard/config/linux/README.md
@@ -1,14 +1,108 @@
-This dir contains kernel configs used by [syzbot](/docs/syzbot.md).
+This dir contains Linux kernel configs used by [syzbot](/docs/syzbot.md).
 
-# Updating Linux configs
+Configs are auto-generated by [syz-kconf](/tools/syz-kconf/kconf.go) utility from declaration file [main.yml](main.yml)
+and config [fragments](bits/). [main.yml](main.yml) specifies list of target config files (instances) and their
+`features`, as well as order of inclusion of config fragments and their constraints. Config fragment is used for
+an instance if all fragment constraints are present in instance features and all negative constraints (`-foo`) are not
+present in instance features.
 
-To update the main linux config [upstream-kasan.config](./upstream-kasan.config) used by `syzbot`:
+For example, the following declaration file:
+```
+instances:
+ - instance-a:	[x86_64, gcc, upstream]
+ - instance-b:	[x86_64, gcc, upstream, foo]
+ - instance-c:	[arm64, clang, stable]
 
-1. Check out latest `linux-next` (we tend to use `linux-next` as it contains the most latest features in a single tree).
-2. Copy the config into kernel tree as `.config`.
-3. Run `make olddefconfig`.
-4. Make any additional changes to the config you want to do (e.g. enable/disable some configs with `make menuconfig`).
-5. Build and boot the kernel. Ensure there are no bugs during boot (in particular, `WARNING`'s and `LOCKDEP` reports).
-6. Copy back the [custom configs](https://github.com/google/syzkaller/blob/1f448cd62db290246f8793128f85bd84aaa7a59d/dashboard/config/upstream-kasan.config#L6-L14) into the `.config` (see comments there).
-7. For compiler you need to use a recent gcc (8+). Some of the debugging configs may be disabled due to old/different compiler, in particular `CONFIG_KCOV_ENABLE_COMPARISONS`. You may also restore the [compiler info part](https://github.com/google/syzkaller/blob/1f448cd62db290246f8793128f85bd84aaa7a59d/dashboard/config/upstream-kasan.config#L16-L20) just to reduce the diff size.
-8. Copy the config back as `upstream-kasan.config` and send a PR. It will be deployed to `syzbot` within a day after merging.
+includes:
+ - always.yml: []
+ - only-upstream.yml: [upstream]
+ - only-clang-arm64.yml: [clang, arm64]
+ - only-upstream-not-foo.yml: [upstream, -foo]
+```
+specifies 3 target configs (instances) and built from 4 config fragments. Here fragment `always.yml` is included first
+and for all instances. Fragment `only-upstream.yml` is included next for instances that have `upstream` feature listed
+(`instance-a` and `instance-b`). Fragment `only-clang-arm64.yml` is included next for instances that have both
+`clang` and `arm64` listed (`instance-c`). Finally, fragment `only-upstream-not-foo.yml` is included for instances
+that have `upstream` features but don't have `foo` feature listed (`instance-a`).
+
+Config fragments can specify several aspects of the target config file. First, they can specify the kernel repository
+and revision that will be used for config generation using the following syntax. There must be only one repository
+specification for each instance.
+```
+kernel:
+ repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+ tag: v5.8-rc7
+```
+
+Then, config fragments can specify kernel `make` commands that will be invoked to create the base config.
+For example:
+```
+shell:
+ - make x86_64_defconfig
+ - make kvm_guest.config
+```
+
+Shell commands can have constraints similar to includes, for example:
+```
+shell:
+ - make defconfig: [-nodefconfig]
+```
+
+Then, fragments can specify text for verbatim inclusion at the end of the resulting config. `syz-kconf` does not do
+anything with these string. For example:
+```
+verbatim: |
+ CONFIG_THIS_CONFIG_DOES_NOT_EXIST_YET=y
+```
+
+Finally, fragments can specify individual config options that need to be enabled (or disabled) in the resulting config.
+For example:
+```
+config:
+ - CONFIG_WILL_BE_ENABLED
+ - CONFIG_WILL_BE_DISABLED: n
+ - CONFIG_WITH_INTEGER_VALUE: 10
+ - CONFIG_WITH_STRING_VALUE: "foo"
+```
+
+Config options can have constraints for more precise control over target instances
+(constraints can be combined with values):
+```
+config:
+ - CONFIG_ONLY_FOR_UPSTREAM: [upstream]
+ - CONFIG_STABLE_AND_ARM64: [stable, arm64]
+ - CONFIG_NOT_CLANG: [-clang]
+ - CONFIG_DISABLED_FOR_GCC: [n, -gcc]
+ - CONFIG_STRING_FOO: ["value", foo]
+```
+
+Some features/constraints are defined by `syz-kconf` itself or have special meaning:
+ - `override` on a config allows to override a previous definition of the same config (otherwise it results in an error)
+ - `optional` on a config disables checking that the resulting kernel config file has the specified value for this
+   config (otherwise it results in an error)
+ - `weak` means both `override` and `optional`
+ - `clang`/`gcc` instance feature controls what compiler is used for config generation
+ - kernel arch name (`x86_64`, `arm64`, etc) instance feature controls architecture of the kernel config
+ - kernel release tag is automatically added to instance features and allows to specify a minimum kernel version
+   for a config (e.g. `v5.5` for a config added in that kernel release) or a maximum kernel version (e.g. `-v5.5` for
+   a config removed in that kernel release)
+
+`syz-kconf` generates 2 kernel config files for each instance: one `normal` (or full) and one `baseline`.
+The `baseline` config has `-base` suffix and has `baseline` feature added. The `baseline` config is meant to not
+contain majority of subsystem configs enabled (should just boot and have debugging configs enabled), however,
+that is fully controlled by config fragments based on the presence of the `baseline` feature. `baseline` configs
+are used in config minization procedure.
+
+The only exception to the declarative nature of the process is USB configs. They are added procedurally, see
+`addUSBConfigs` function for details.
+
+While `syz-kconf` switches between `gcc` and `clang` compilers automatically, the compilers themselves are not
+part of `syz-kconf` and are supposed to be installed on the system. `gcc` should be version 9+ and `clang`
+should be 11+.
+
+To update kernel configs:
+ - change config fragments as necessary (e.g. add additional configs to [subsystems.yml](bits/subsystems.yml) along
+   with minimal kernel version)
+ - run `make configs SOURCEDIR=/path/to/existing/linux/checkout` (note: it will be mrproper-ed)
+ - check in config fragments and changed kernel configs and send a PR
+ - changes will be deployed to `syzbot` within a day after merging