diff options
| author | Mara Mihali <maramihali@google.com> | 2021-08-05 16:56:46 +0000 |
|---|---|---|
| committer | maramihali <maramihali@google.com> | 2021-08-06 12:06:44 +0300 |
| commit | f9e341e30b4f3faa468a0b885775a4fbf7825016 (patch) | |
| tree | c4166bb88581d4fd8106a2a84e302b66776de344 /syz-verifier/stats_test.go | |
| parent | 2f5370993e30b561da92f1ca5f0abd19271b7bd0 (diff) | |
pkg/rpctype, syz-runner, syz-verifier: add reruns to syz-verifier architecture
When a mismatch is found in the results returned for a program, the program will be rerun on all the kernels to ensure
the mismatch is not flaky (i.e. it didn't occur because of some background activity or external state and will always
be returned when running the program). If the same mismatch occurs in all reruns, syz-verifier creates a report for
the program, otherwise it discards the program as being flaky
Diffstat (limited to 'syz-verifier/stats_test.go')
| -rw-r--r-- | syz-verifier/stats_test.go | 8 |
1 files changed, 6 insertions, 2 deletions
diff --git a/syz-verifier/stats_test.go b/syz-verifier/stats_test.go index 81dc1ef2e..38f4f4661 100644 --- a/syz-verifier/stats_test.go +++ b/syz-verifier/stats_test.go @@ -11,8 +11,10 @@ import ( func dummyStats() *Stats { return &Stats{ - Progs: 24, - TotalMismatches: 10, + TotalProgs: 24, + TotalMismatches: 10, + FlakyProgs: 4, + MismatchingProgs: 6, Calls: map[string]*CallStats{ "foo": {"foo", 2, 8, map[ReturnState]bool{ returnState(1, 7): true, @@ -69,6 +71,8 @@ func TestReportGlobalStats(t *testing.T) { "total number of mismatches / total number of calls "+ "executed: 10 / 20 (50.00 %)\n\n"+ "programs / minute: 2.40\n\n"+ + "true mismatching programs: 6 / total number of programs: 24 (25.00 %)\n"+ + "flaky programs: 4 / total number of programs: 24 (16.67 %)\n\n"+ "statistics for bar:\n"+ "\t↳ mismatches of bar / occurrences of bar: 5 / 6 (83.33 %)\n"+ "\t↳ mismatches of bar / total number of mismatches: 5 / 10 (50.00 %)\n"+ |