/
githubmirror
/
libmicrohttpd
Обзор
Документация
Войти
/
githubmirror
/
libmicrohttpd
Код
Запросы
0
Пакеты
0
Релизы
0
Аналитика
Безопасность
master
src/fuzz/corpus/README
330 строк
17 KB
Christian Grothoff
distill the second fuzzing campaign into the seed corpus
29 июл 2026, 20:31
Не верифицирован
29 июл 2026, 20:31
641d2e9
Код
Авторство
О чём код?
Seed corpus for the src/fuzz harnesses ====================================== `<harness>-NN.bin` is the built-in seed corpus of that harness, dumped to disk. Regenerate at any time with ./fuzz_<harness> --write-corpus=<this directory> or, from the build tree, make -C src/fuzz refresh-corpus Replay everything (this is what a CI regression run should do): make -C src/fuzz check-corpus # or, per harness: ./fuzz_request --corpus-dir=src/fuzz/corpus Files belonging to another harness are simply uninteresting inputs for the harness that reads them, so pointing every harness at the whole directory is fine and gives some extra cross-pollination. The set is kept minimal: every seed here adds coverage that no other seed provides. After changing a seed table, re-check that with <target> -merge=1 <empty dir> <this directory> from an OSS-Fuzz style build (contrib/oss-fuzz/build.sh); a seed the merge does not copy across is redundant and should be deleted from the table rather than left in the corpus. Note that fuzz_request is mildly non-deterministic -- MHD timestamps its nonces and its connection timeouts -- so a single merge decision worth ~0.1% of the regions is noise; only drop a seed that several independent merges agree on. fuzz_request seeds ------------------ 00 content-length-body body oracle, Content-Length 01 chunked-with-extensions body oracle, chunk extensions -> chunk-extension CRLF bug 02 chunked-split body oracle, chunk boundary split across two send() calls 03 small-pool-trailing-query-arg 128..512 byte connection pool, no header lines, "?novalue" -> read-buffer shift-back bug 04 small-pool-trailing-query-arg-2 same with "?a=1&b" 05 digest-unknown-algorithm algorithm=BOGUS -> MHD_DIGEST_AUTH_ALGO3_INVALID MHD_PANIC() 06 digest-overlong-response 401 challenge, then a replay on a fresh connection with the harvested nonce and a 128 hex digit response= -> stack buffer overflow 07 digest-userhash userhash=true with a 128 char username 08 basic-auth Authorization: Basic 09 multipart-post chunked multipart/form-data 10 urlencoded-post application/x-www-form-urlencoded 11 folded-header obs-fold continuation line 12 pipelined two requests in one segment Seeds 00-12 leave the API-selection block (bytes 4-9, see section 2.2.1 of ../README) zero and only drive the request parser. Seeds 13-39 each switch on one area of it; without them those entry points are reachable only by guessing six configuration bytes. 13 ext-callback-chunked MHD_SIZE_UNKNOWN content reader, response headers and a trailer 14 ext-callback-error known-length reader that fails part way through 15 ext-fd-response MHD_create_response_from_fd() 16 ext-fd-at-offset-response ..._from_fd_at_offset() 17 ext-pipe-response ..._from_pipe() 18 ext-iovec-response ..._from_iovec() 19 ext-empty-response-hdrapi ..._empty() plus get/del header and MHD_set_response_options() 20 ext-digest-check-digest3 pre-computed userdigest path 21 ext-digest-v1-wrappers MHD_digest_auth_check() 22 ext-digest-request-info MHD_digest_auth_get_request_info3() 23 ext-external-event-loop MHD_get_fdset2()/run_from_select2() plus the timeout accessors 24 ext-external-event-loop-v1 the v1 fdset/run_from_select pair 25 ext-connection-api every introspection accessor 26 ext-suspend-resume suspend, resume from the pump loop 27 ext-suspend-two-connections two connections parked at once 28 ext-upgrade 101 + MHD_upgrade_action() 29 ext-buffer-persistent buffer/MHD_RESPMEM_PERSISTENT 30 ext-buffer-free-callback ..._with_free_callback() 31 ext-from-data MHD_create_response_from_data() 32 ext-fd64-response ..._from_fd64() 33 ext-basic-auth-v1 MHD_basic_auth_get_username_password() 34 ext-basic-auth-challenge MHD_queue_basic_auth_fail_response() 35 ext-basic-auth-challenge-utf8 ..._response3() with UTF-8 charset 36 ext-digest-check-digest-v1 MHD_digest_auth_check_digest() 37 ext-digest-check-digest2 ..._digest2() (never with MHD_DIGEST_ALG_AUTO, see K8) 38 ext-digest-get-username-v1 MHD_digest_auth_get_username() 39 ext-digest-get-username3 ..._username3() fuzz_str seeds -------------- Cover MHD_hex_to_bin (including the 128 character input that digestauth.c used to allow into a 32 byte buffer), MHD_bin_to_hex[_z], the percent-decoders (strict, lenient, in place), MHD_str_unquote, MHD_str_quote, MHD_base64_to_bin_n, MHD_str[x]_to_uint64_n_ and the token helpers. fuzz_auth_header seeds ---------------------- Well-formed and broken Digest parameter lists (unknown algorithm, extended `username*` notation, unterminated quoted strings, empty parameter list, over-long values) plus Basic `token68` variants, and -- with bit 0x02 of byte 0 -- the connection-less digest helpers at several algorithms and several deliberately undersized output buffers. fuzz_postprocessor seeds ------------------------ urlencoded bodies with broken percent escapes, multipart bodies with ordinary, degenerate ("-") and quote-containing boundaries, a multipart Content-Type without a boundary, and a non-POST Content-Type. distilled/ ---------- The edge-minimal residue of the fuzzing campaigns: 2562 inputs, 240 KB, named `<harness>-dNNN.bin` and routed by that prefix exactly like the seeds above. Files are numbered in the order the cover picked them, so `-d000` is the highest-gain input of its harness. Unlike `<harness>-NN.bin` these are *not* generated from a built-in table, so `refresh-corpus` neither writes nor clobbers them; and unlike `known-findings/` they are not hand-edited and carry no individual meaning. Do not document them one by one -- the set is only ever regenerated wholesale. Provenance: an 8 hour, 12 core, 1.62 billion execution campaign (ASan+UBSan, libFuzzer, hourly restarts) starting from the seed corpus above. The campaign corpus was 18 226 inputs / 12 MB, which is too much to carry in a source tree, so it was reduced by greedy set cover over *edges* rather than by `-merge=1`, which minimises for features and keeps roughly seven times as many files: <target> -merge=1 -merge_control_file=mcf.txt <empty dir> <corpus> writes a `COV` line per input, and the cover is computed from those offline. Because these harnesses are mildly non-deterministic, the cover is built only from edges that two independent merges agree an input reproduces -- the same "several independent merges" rule stated at the top of this file. The `fuzz_eventloop` and `fuzz_tls` inputs come from a second, 1 hour / 4 core / 17.0 million execution campaign (same sanitizers, same engine, `--enable-https` so that `fuzz_tls` could be built at all -- note that `contrib/oss-fuzz/build.sh` configures `--disable-https`, which is why `fuzz_tls` had no distilled inputs before and started that campaign from its seed table alone). It ran after the PSK scenario and the queue-and-stop scenario were added to those two harnesses, and its starting corpus was this directory plus the first campaign's residue, so what it contributed is beyond what 8 core-hours per harness had already reached: `fuzz_tls` 6355 -> 6844 edges, `fuzz_eventloop` 7406 -> 7655. The other six harnesses were not re-run; their inputs are unchanged. A `COV` line holds only the edges its input was *first* to reach in that merge, so what a cover has to pay for depends on what it is credited with already having. Credit it with everything else that stays in the tree -- both seed sets and the distilled inputs of the harnesses not being regenerated -- because `check-corpus` replays all three directories through every harness, and an input is worth carrying only if it adds an edge none of them reach. Doing that costs nothing and is worth roughly a third of the files: the `fuzz_tls` cover needed 490 edges and 167 inputs against the whole tree, against 1580 edges and 282 inputs when credited with the seed corpus alone -- identical coverage either way, because the other harnesses' inputs were already reaching those 1090 edges. Compute every harness's cover against the same base so that no two of them depend on each other. **An engine-produced corpus is not pristine, and this directory has to be filtered before it can be committed.** `fuzz_request` inputs carry a ground-truth body declaration in their `op == 1` segment, and the body oracle is armed only for a `fuzz_pristine` input precisely because "random mutations invalidate such declarations" (`fuzz_common.h`). libFuzzer mutates that segment independently of the wire segments, so a campaign corpus is full of inputs whose declaration no longer describes their own request -- 383 of 6884 here, 5.6%. Under libFuzzer that is harmless: `-DFUZZ_NO_MAIN` compiles out every assignment to `fuzz_pristine`, so the oracle never arms and the campaign never notices. But `--corpus-dir` sets `fuzz_pristine = 1` for *every* file it reads, so replaying such an input reports a "request body desync" against a declaration that was never true -- a false positive, and one that looks exactly like a request-smuggling finding. Committing the raw cover therefore broke `make check-corpus` with 42 such reports. The fix is to sweep every candidate with `--file=` first and exclude the ones that fire *before* computing the cover, which is what produced the set in this directory; all 2562 files replay clean through all eight harnesses. Anyone replaying a downloaded ClusterFuzz corpus locally will hit the same thing, and it is not a bug in MHD. Sweep every candidate anyway, but expect this particular exclusion only for `fuzz_request`: it is the sole reader of `fuzz_pristine`, so it is the only harness that can report a desync against its own stale declaration. The second campaign's 15 554 candidates produced no rejects at all for that reason. The set reaches ~99% of the first campaign's edges, not 100%: 3763/3798 for fuzz_request, and exactly 100% for the four direct-API harnesses, which are deterministic. Two thirds of that shortfall is the exclusion above -- 64 edges were reached only by inputs with a stale declaration -- and the rest is edges no single input reproduces reliably. Forcing the latter in by adding every input that covers a rare edge was tried and moved the number by one, so that part is inherent to minimising rather than a fixable omission. Trading 1% of edges for 86% fewer files is the intended bargain. Do not chase it. The second campaign lost nothing: `fuzz_eventloop` ends at 7655 of the 7656 edges its campaign corpus reaches, and `fuzz_tls` at 6844 against that corpus' 6770 -- above it, because a `fuzz_tls` cover measured inside the tree also gets the other harnesses' inputs, which are uninteresting to it but not inert. Both harnesses are deterministic enough here that all three `-runs=0` measurements agreed to within two edges. To regenerate after a long campaign: keep the campaign corpus, sweep it for oracle-firing inputs, run two independent merges over what is left, recompute the cover against everything that stays in the tree, and replace the affected harnesses' files wholesale. Check the result the same way it was checked here -- replay both the campaign corpus and the distilled set with `-runs=0` three times each and compare the `INITED cov:` figures, because a single measurement of either is worth a few edges of noise -- and finish with `make check-corpus`. Measure *every* harness before and after, not just the ones being regenerated: dropping an input removes it from the other seven as well. Replacing the 320 first-campaign `fuzz_eventloop` inputs with 248 new ones moved the other six harnesses by at most three edges, which is noise, but that is a thing to confirm rather than assume. Note also that `make check-corpus` is a weaker gate than it looks on a tree configured without `--enable-asserts`, where `mhd_assert()` compiles away entirely; the campaign build has them, so inputs that survived the merges have already been through the stronger one. known-findings/ --------------- Reproducers for the issues that this suite found in MHD itself; see section 6 of ../README, which records the status of each. All of them are fixed on master, so all of them replay clean, and `check-corpus` replays this directory too. A file is named `K<n>-<harness>-<what>.bin` for the harness that found it; the ones without a harness in the name predate that convention and are all fuzz_request inputs. `contrib/oss-fuzz/make_seed_corpus.sh` routes each into the seed corpus of its own harness on that basis. They are kept as a separate, explicitly named set rather than being folded into the main corpus because the files there are *generated*: a hand-written input would be clobbered or renumbered by the next `refresh-corpus`. Replay one with ./fuzz_request --file=src/fuzz/corpus/known-findings/K1-digest-empty-realm.bin K1-digest-empty-realm.bin digestauth.c is_param_equal() mhd_assert (0 != param->value.len) K2-chunkext-no-space.bin connection.c handle_recv_no_space() K3-chunkext-stop-with-error.bin connection.c transmit_error_response_len() K4a-wsp-first-header.bin connection.c get_req_header(), CLIENT_DISCIPLINE_LVL <= -1 K4b-empty-header-name.bin connection.c get_req_header(), CLIENT_DISCIPLINE_LVL <= -2 K5-bare-cr-keep.bin connection.c get_req_header(), CLIENT_DISCIPLINE_LVL == -3 K6-nonce-length-collision.bin digestauth.c check_nonce_nc() mhd_assert (0 == nn->nonce[noncelen]) K7-upgrade-after-must-close.bin connection.c build_header_response() mhd_assert (upgrade -> MUST_UPGRADE); a single request carrying both Content-Length and chunked Transfer-Encoding, then an upgrade K9-fuzz_postprocessor- postprocessor.c post_process_multipart() nested-boundary-leak.bin leaked the previous nested boundary on every extra nested multipart/mixed part K14-fuzz_eventloop- daemon.c call_handlers() asserted a force-close-not-closed.bin post-condition that MHD_connection_handle_read() does not guarantee when the read buffer is full *** still open, see ../../../patches *** Only fires on an --enable-asserts *ASan* build: the redzones change the pool arithmetic that reaches it. K15-fuzz_eventloop- daemon.c close_all_connections() stop-with-queued-connection.bin asserted that only a daemon with an internal polling thread can have connections queued by MHD_add_connection(), while the code that fills that list queues for any thread-safe daemon. An external event loop that adds a connection and stops before the next MHD_run() aborts on an --enable-asserts build. These files are *not* regenerated by `--write-corpus`; they are edited by hand. When the input format changes they have to be migrated, and the migration has to be checked -- replay each one with `--verbose` before and after and confirm the daemon, handler, body and challenge counts are unchanged. The last such change made the API-selection block unconditional, which is six zero bytes inserted at offset 4 for K1-K6. K8 has no reproducer here: its trigger is an argument the application chooses, not network input, so the harness never performs it. See section 6 of ../README for how to reach it by hand. A reproducer added while its finding is still open will make `make check-corpus` fail, which is intended -- that is what a regression test for an unfixed bug does. `contrib/oss-fuzz/make_seed_corpus.sh` skips such a file so that ClusterFuzz does not spend every run rediscovering it; the rule it uses is whether `patches/$ID.diff` exists.