aboutsummaryrefslogtreecommitdiffstats
path: root/dashboard/dashapi/dashapi.go
Commit message (Collapse)AuthorAgeFilesLines
* syz-manager: support oauth when calling syz-hubGreg Steuck2021-07-301-4/+4
| | | | Permit empty hub_key to indicate oauth.
* pkg/auth: create explicit JWT token cacheGreg Steuck2021-07-301-27/+11
| | | | This will be easier to reuse as it is no longer http-specific.
* pkg/auth: move jwt to auth to be shared by syz-hub clientsGreg Steuck2021-07-301-6/+28
|
* dashboard/dashapi: implement client-side JWT token authorizationGreg Steuck2021-07-141-0/+9
|
* dashboard/app: server-side support for OAuth /api authenticationGreg Steuck2021-07-141-0/+4
|
* dashboard/dashapi: return error from dashapi.NewGreg Steuck2021-07-121-3/+3
| | | | | This will allow it to make a call to meta-server and abort early when authentication by bearer token is impossible.
* dashboard/app: save suppressed reportsDmitry Vyukov2021-02-211-0/+2
| | | | | | | Save suppressed reports into a special single bucked "suppressed report". This will allow some control and monitoring over frequency and will allow to see samples. Fixes #1941
* syz-manager: don't reproduce leaks again and againDmitry Vyukov2021-02-201-3/+4
| | | | Fixes #1956
* dashboard/dashapi: add alternative titles for crashesDmitry Vyukov2021-01-171-0/+1
| | | | Update #1575
* dashboard/app: add unfix commandDmitry Vyukov2020-10-011-12/+13
| | | | | Add "#syz unfix" command that undoes "#syz fix" effects and resets any existing fixing commits.
* dashboard/app: return BugReport from load_bug APIDmitry Vyukov2020-09-301-13/+2
| | | | | | | | Return BugReport instead of LoadBugResp from load_bug API request. LoadBugResp is some limited subset of info in BugReport (mostly), it's useful to have just one type to describe bug with all needed info. Update #2144
* dashboard/app: add more info to BugReportDmitry Vyukov2020-09-301-0/+7
| | | | | | | | | | | | | Add the following fields to BugReport: - bug status - build id - build time - syzkaller commit - crash time - repro opts Will be required to export info to KCIDB. Update #2144
* dashboard/app: send machine information to dashboardCheng-Min Chiang2020-09-141-0/+3
| | | | | | | | | | | | - Change syz-manager so that it will send machine info the first time a crash occurs. - Add a field in entities.Crash to store machine info. - Add a field in dashapi.BugReport to store machine info. - Change the HTML template and struct uiCrash to display machine info. - Add a test to make sure that the link to machine info appears on the webpage. Update #466
* dashboard/dashapi: create RecipientsPedro Lopes2020-07-311-9/+36
| | | | | | | Create struct Recipients to store a slice structs (RecipientInfo) of email, default name, and if the user should be added to To or Cc when sending the email. After this commit Commit::Cc and Crash::Maintainers will become deprecated.
* all: use more respective languageDmitry Vyukov2020-06-111-4/+4
| | | | | | | | Some terms are normalised on the technical level but may be oppressive on a societal level. Replace them with more technically neutral terms. See the following doc for a longer version: https://tools.ietf.org/id/draft-knodel-terminology-00.html
* dashboard, syz-ci: add blacklist for bisection resultsAndrey Konovalov2020-06-031-3/+4
| | | | | | | Currently only ignores the commit that adds the Raw Gadget interface. Requested here: https://groups.google.com/g/syzkaller-bugs/c/sZUeGC3Fh-o/m/t_5cKPrMAQAJ
* dashboard/app: add notion of per-repo build maintainersDmitry Vyukov2020-05-091-0/+1
| | | | | | | | | For context see: https://groups.google.com/d/msg/syzkaller/9HxkC_hRP4M/MzFl4laADgAJ Now if, say, linux-next, is build/boot-broken we can CC maintainers of the tree. Fixes #1667
* dashboard/app: collect BB coverage stats from managersDmitry Vyukov2020-05-081-6/+9
| | | | Fixes #1698
* syz-ci: allow enabling bisect cause and fix jobs separatelyDmitry Vyukov2019-11-271-2/+7
| | | | | Some backport-only kernels may only be interested in fix bisections. Allow enabling these separately.
* dashboard/app: don't report bisections pointing to release commitsDmitry Vyukov2019-11-271-2/+3
| | | | | | | | | | | They should have been detected by "same binary" logic. But the problem is that we may use different compilers for different commits and they switch exactly at release commits. So we can build the release with a differnet compiler than the rest of commits and then obviously it won't be "same binary". Detect release commits separately. Update #1271
* pkg/bisect: detect wrong bisectionsDmitry Vyukov2019-11-071-0/+8
| | | | | | | | Detect bisection to merge commits and to commits that don't affect kernel binary (comments, other arches, whitespaces, etc). Such bisections are not reported in emails (but shown on web). Update #1271
* tools/syz-reprolist: add utility to extract list of reproducersDmitry Vyukov2019-10-081-0/+32
|
* dashboard/app: allow reporting of BisectFix resultsZubin Mithra2019-08-301-0/+1
| | | | | | | | | | | | | * Modify mail_bisect_result.txt to allow for sending fix bisection results. * Modify BisectResult to have a Fix field; introduce selectBisect for use within the template for choosing between BisectCause/BisectFix fields. * Modify bisectFromJob() to return BisectResult with Fix field set if relevant. * Modify the tests inside bisect_test.go to account for bisect fix related reporting emails. * Modify incomingMail() to ignore any emails from syzbot itself.
* dashboard/app: properly handle build failuresDmitry Vyukov2019-03-251-15/+16
| | | | | | | | | Separate kernel and syzkaller build failures. Fix logic to understand when a build is fixed: look if kernel/syzkaller commit changes to understand if it's a new good build or re-upload of an old build. Fixes #1014
* syz-ci: fix filling of kernel commit in bisection jobsDmitry Vyukov2019-03-221-12/+14
| | | | | | | | | There is a bit of a mess: dashboard expects the start commit in build info, but syz-ci sends the resulting cause commit. Moreover for inconclusive bisection the commit is not filled at all. Fill start commit in build info on start. Update #501
* dashboard/app: add uncc commandDmitry Vyukov2019-03-221-0/+1
| | | | | | | | | Add "#syz uncc" command as a safety handle. The command allows sender to unsubscribe from all future communication on the bug. Linus mentioned possibility of saying "I'm not the right person for this report" in the context of bug reminders: https://groups.google.com/d/msg/syzkaller/zYlQ-b-QPHQ/AJzpeObcBAAJ
* dashboard/app: request test/bisect jobs separatelyDmitry Vyukov2019-03-201-3/+3
| | | | | | | | Allow separate sets of managers for patch testing and for bisection. This makes things more flexible on syz-ci deployment side. Remove previous hacks for bisection deployment. Update #501
* dashboard/app: report bisection results to external reportingDmitry Vyukov2019-03-191-1/+2
| | | | Update #501
* dashboard/app, syz-ci: bisection supportDmitry Vyukov2019-03-171-8/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds bulk of support for bisection to dashboard/app and syz-ci: - APIs to send bisection jobs and accept results - syz-ci logic to execute bisection jobs - formatting of emails with results - showing of results on dashboard Some difficulties we have to overcome: - since linux is frequently build/boot broken, lots of bisections are inconclusive, need to present such results too - git bisect is poorly suitable for automation, have to resort to output parsing (is output stable?) - git bisect turns out to fail (exit with non-0 status) when bisection is inconclusive (multiple potential cause commits) - older syzkaller revisions can't be built with newer (broken) kernel header, e.g.: ebtables.h:197:19: error: invalid conversion from ‘void*’ to ‘ebt_entry_target*’ - newer compilers produce more warnings and break old syzkaller builds, e.g.: kvm.S.h:6:12: error: ‘kvm_asm64_vm86’ defined but not used [-Werror=unused-const-variable=] - figuring relevant emails to CC from a commit is non-trivial: besides commit author, there can be some emails in commit tags, or not, which tags to use is an interesting question (some may include irrelevant emails) we can also run get_maintainers.pl on the commit, but this can produce too wide list if commit touches lots of files, it can also produce too small list, and then we need to resort to blame - for inconclusive bisection we probably don't need to include emails referenced in the commits (there can be too many of these commits) - need to be careful to exclude own syzbot email from commit CC list, now syzbot emails are referenced in some commits (Reported-by/Tested-by/etc) (can cause some kind of infinite recursion) - lots of commits reference stable mailing list, we should not include it in CC because it's referenced for backports rather then bug reports - since we add new Bug entity fields which we use in queries, whole datastore need to be upgrades to add the new field to index - we must not discard the crash that was used for bisection (treat it as a reported crash) - bisection results need 2 forms of reports: one when we add bisection results to already reported bug another when we report a bug first time with bisection results - when reporting a bug with bisection results we need to use the crash that was used for bisection - some fraction of bisections will probably fail with various errors and we will need some mechanism to retry bisection after the root cause is resolved this is not implemented yet - linux-next is problematic for 2 reasons: fix bisection can't possibly run on linux-next as commits are not reachable from HEAD lots of commits are missing in linux-next (even in linux-next-history) e.g. we have some c63e9e91a254a52 which is now missing in linux-next/linux-next-history - older kernels can't be build with fresh gcc/binutils/perl/make/glibc for now we have to stop at v3.9 (this only requires switching gcc several times along the way) - kernels past v4.11 do not build with gcc 7 and 8 (undefined reference to `____ilog2_NaN') - v4.1 and back have only compiler-gcc5.h - v3.17 and back have only compiler-gcc4.h - v3.6 and back do not have make olddefconfig - compat socket calls can't be bisected past "x86/entry/syscalls: Wire up 32-bit direct socket calls" (v4.10) because of https://syzkaller.appspot.com/bug?id=b5b150e322d5f48c869bcf1528cdbee08d1421cb - v2.6.28 and below does not work with modern make: *** mixed implicit and normal rules: deprecated syntax - v3.8 build fails: Can't use 'defined(@array)' (Maybe you should just omit the defined()?) at kernel/timeconst.pl line 373. kernel/Makefile:134: recipe for target 'kernel/timeconst.h' failed - make 3.81 works for v2.6.28. 3.81 almost works with current HEAD, you need to run make twice because first run spuriously fails with: - v2.6.28 with gcc-4.9.4 broken with: include/linux/kvm.h:240:9: error: duplicate member ‘padding’ - but even defconfig fails: VDSO arch/x86/vdso/vdso.so.dbg gcc: error: elf_x86_64: No such file or directory gcc: error: unrecognized command line option ‘-m’ It seems that we also need old binutils. - for v3.8 and below we need perl-5.14.4. Unfortunately this or any manually built perl doesn't work for later kernels: Can't locate strict.pm in @INC - kernels starting from 4.14 and older are boot broken: https://lkml.org/lkml/2018/9/7/648 - kernels older than 4.12 are broken during netdev setup (fixed by commit 675c8da049fd6556eb2d6cdd745fe812752f07a8) Update #501
* dashboard/app: make dashapi.BugReport more self-containedDmitry Vyukov2019-03-171-6/+17
| | | | | | | Extend BugReport with few fields that are needed by all reportings anyway. This allows to not create and fill an almost identical object to pass to template. Update #501
* dashboard/app: implement bug notifications/actionsDmitry Vyukov2019-02-221-9/+60
| | | | | | | | | | | | | | | | Currently dashboard can only report new bugs and add reproducers to already reported bugs. This change adds infrastructure for the dashboard to actively act on existing bugs in different ways. 4 new notifications (actions) added: - dashboard can auto-upstream bugs from moderation after an embargo period - dashboard can auto-upstream bugs if reporting criteria changes (e.g. it reported a bug into moderation because there was no repro, but then repro appears and the bug is automatically sent upstream) - dashboard detects when a fixing commit does not appear in any tested trees for too long and sends a notification about this - dashboard detects stale bugs (last happened monts ago, no repro, no activity) and auto-invalidates them This will also be useful to send pings for old bugs and do other automation.
* dashboard/app: poll commits infoDmitry Vyukov2019-02-171-4/+35
| | | | | | | | | | | | | | | This implements 2 features: - syz-ci polls a set of additional repos to discover fixing commits sooner (e.g. it can now discover a fixing commit in netfilter tree before it reaches any of the tested trees). - syz-ci uploads info about commits to dashboard. For example, a user marks a bug as fixed by commit "foo: bar". syz-ci will find this commit in the main namespace repo and upload commmit hash/date/author to dashboard. This in turn allows to show links to fixing commits. Fixes #691 Fixes #610
* dashboard/dashapi: remove unused typeDmitry Vyukov2019-02-141-5/+0
| | | | | Stray change in the previous commit. This unbreaks CI build.
* dashboard/app: remove unused functionDmitry Vyukov2019-02-131-0/+5
|
* dashboard/app: refactor testsDmitry Vyukov2018-06-111-29/+88
| | | | | | | Use standard dashboard client provided by dashapi package in tests. Switch everything to use the client instead of API method. Fixes #606
* gometalinter: enable package comment checkingDmitry Vyukov2018-05-031-1/+1
| | | | Update #538
* dashboard/app: use links instead of attachments in emailsDmitry Vyukov2018-03-251-0/+7
| | | | | As per discussion at: https://groups.google.com/d/msg/syzkaller/zYlQ-b-QPHQ/AJzpeObcBAAJ
* all: fix gometalinter warningsDmitry Vyukov2018-03-081-1/+1
| | | | Fix typos, non-canonical code, remove dead code, etc.
* dashboard/app: switch API to passing args as form-encoded valuesDmitry Vyukov2018-03-081-25/+10
| | | | This is slightly more secure and does not pollute URLs.
* dashboard/app: improve report formatDmitry Vyukov2018-01-171-0/+2
| | | | | | | | | - show kernel repo alias - show commit title/date - say how many times the crash happened/where - some minor tweaks Fixes #468
* dashboard/app: collect more info for better reportsDmitry Vyukov2018-01-171-35/+40
| | | | | | | | | | | | Collect kernel build commit title/date. Add support for kernel repo aliases (to be able to say linux-next instead of full git repo address). Collect on what managers a bug happened. Reuse Crash.ReportLen as generic crash reporting priority. Make it possible to prioritize reporting of particular kernel repos and arches. Fixes #473
* dashboard/app: remember what crashes we reported for what bugsDmitry Vyukov2018-01-161-0/+2
| | | | | | | | We currently can silently switch crashes when report to the next reporting, or test a patch using a repro from a different crash. Remember what crash we reported for a bug and use it in both cases.
* dashboard/app: extract fixing tags from commitsDmitry Vyukov2017-12-271-4/+13
| | | | | | | | | | | | | Support the new scheme of associating fixing commits with bugs. Now we provide a tag along the lines of: Reported-by: <syzbot+a4a91f6fc35e102@syzkaller.appspotmail.com> The tag is supposed to be added to the commit. Then we parse commit logs and extract these tags. The final part on the dashboard is not ready yet, but syz-ci should already parse and send the tags.
* dashboard/app: add API for polling for closed bugsDmitry Vyukov2017-12-191-2/+10
| | | | | | External reporting may need to know when dashboard is not interested in bugs anymore. Add API that returns list of bugs dashboard considers closed.
* syz-manager: send public web addr to dashboardDmitry Vyukov2017-12-041-0/+2
|
* dashboard/app: add manager monitoringDmitry Vyukov2017-12-011-0/+18
| | | | | | | | Make it possible to monitor health and operation of all managers from dashboard. 1. Notify dashboard about internal syz-ci errors (currently we don't know when/if they happen). 2. Send statistics from managers to dashboard.
* syz-ci: test images before using themDmitry Vyukov2017-11-301-0/+9
| | | | | | | Boot and minimally test images before declaring them as good and switching to using them. If image build/boot/test fails, upload report about this to dashboard.
* dashboard/app: support corrupted reportsDmitry Vyukov2017-11-221-2/+3
|
* syz-manager: send corrupted flag to dashboardDmitry Vyukov2017-11-211-0/+1
|
* dashboard/app: test fix patches by requestDmitry Vyukov2017-11-161-0/+51
| | | | | | | Add initial support for testing fix patches. A user on mailing list can request patch testing with: and attach a patch.