/
githubmirror
/
libmicrohttpd
Обзор
Документация
Войти
/
githubmirror
/
libmicrohttpd
Код
Запросы
0
Пакеты
0
Релизы
0
Аналитика
Безопасность
v1.0.8
src/testzzuf/README
307 строк
14 KB
Christian Grothoff
expand test suite, add CI/CD logic as done in other GNUnet projects
28 июл 2026, 13:36
Не верифицирован
28 июл 2026, 13:36
e34baa3
Код
Авторство
О чём код?
GNU libmicrohttpd -- fuzzing testsuite (src/testzzuf) ===================================================== The testcases in this directory require zzuf, and most of them also require socat. zzuf randomly mangles the TCP byte stream that reaches MHD. The goal is to expose problems in MHD's error handling by introducing random syntax errors into otherwise valid HTTP traffic. There are two modes of operation: * direct mode (zzuf_test_runner.sh) zzuf runs the test binary itself and intercepts the data that MHD reads from its accepted sockets. Only possible when MHD uses accept() (not accept4()) and when no sanitizer is active, because zzuf and the sanitizers both intercept the standard library. * socat mode (zzuf_socat_test_runner.sh) zzuf runs a socat relay that listens on 127.0.0.121:10121 and forwards the mangled stream to MHD on 127.0.0.1:4010. The test binary (and therefore MHD) runs outside of zzuf, so this mode works with sanitizers and with accept4(). configure selects this mode automatically when it is required (FORCE_USE_ZZUF_SOCAT). Because the byte stream is corrupted at random, the testcases do NOT check that whatever the client receives is what was expected. A testcase "fails" when MHD crashes, aborts, hangs, reports an internal inconsistency through its callbacks, or when a sanitizer fires. What is being tested -------------------- Every test starts a series of MHD daemons -- one per polling mode that the current build supports (internal select(), internal select() with thread-per-connection, internal poll(), internal poll() with thread-per-connection, internal epoll, external polling, external polling without thread safety) -- and runs a client against each of them. Clients come in two flavours: * libcurl clients, for everything libcurl can express; * a small raw socket client (mhd_zzuf_common.c), for everything libcurl cannot express: chunk extensions, request pipelining, header-less requests and hand-written "Authorization:" headers. Both flavours use 127.0.0.101 as their source address, while MHD listens on 127.0.0.1 (and socat on 127.0.0.121). zzuf is told to fuzz only the traffic of the server side address, so the data the *clients* read is left alone. The daemon option matrix ------------------------ Historically all fuzzing tests here used essentially one set of daemon options. Several MHD code paths are only reachable with other options, most importantly the read-buffer "shift back" in get_req_headers(), which is only executed when the connection memory pool is smaller than MHD_BUF_INC_SIZE (1500 bytes). mhd_zzuf_common.c therefore defines a matrix of option profiles: name mem_limit conn_limit timeout discipline ---------------- --------- ---------- ------- ------------------ plain - - 2 0 pool-128 128 - 2 0 pool-256-strict 256 - 2 1 pool-512-lax 512 - 2 -1 pool-1024-conn 1024 4 2 2 pool-1400-hard 1400 2 1 3 pool-768-loose 768 - 1 -3 legacy-strict 192 - 2 1 (STRICT_FOR_CLIENT) legacy-lax 320 - 2 -1 (STRICT_FOR_CLIENT) "discipline" is MHD_OPTION_CLIENT_DISCIPLINE_LVL, except for the two "legacy" profiles which use the older MHD_OPTION_STRICT_FOR_CLIENT so that the option translation code is covered as well. Every daemon that a test starts uses the next profile of the matrix, so a single run of one test program sweeps the whole matrix. MHD_OPTION_SERVER_INSANITY is deliberately *not* used: in this version of MHD the only value of "enum MHD_DisableSanityCheck" is MHD_DSC_SANE (0), so the option cannot actually disable any check. The test programs ----------------- All programs decide what to do from their own name (argv[0]). A "10" in the name selects HTTP/1.0, a "_close" suffix makes MHD close the connection after every reply, and a "_hostile" marker enables the option matrix described above. test_get, test_get_chunked, test_post, test_post_form, test_put, test_put_chunked, test_put_large, test_get_long_uri, test_get_long_header (+ "_close" and "10" variants) The original tests: well-formed libcurl traffic, single set of daemon options. test_get_hostile, test_get_chunked_hostile, test_post_hostile, test_put_chunked_hostile, test_put_large_hostile The very same client/server exchanges, but sweeping the daemon option matrix. test_get_args (+ "_close", "10") GET requests with hostile query strings: arguments without '=', empty names, empty values, trailing '&', percent-encoded and invalid percent-encoded names, duplicated names and a very long tail argument. All daemons of this test use a connection memory pool smaller than MHD_BUF_INC_SIZE. In addition to the libcurl requests, the raw client sends *header-less* HTTP/1.0 requests: libcurl always sends at least a "Host:" header, so with libcurl alone the last parsed element of a request header is always a header and never a query argument. test_get_chunked_ext, test_put_chunked_ext (+ "_close") Chunked request bodies whose chunk-size lines carry chunk extensions: "name=value", flags without a value, quoted-string values containing ';', several extensions on one line, extensions on the terminating "0" chunk, extensions after bad whitespace, an extension line longer than the whole memory pool, and trailers (footers) after the last chunk. The body is always split across two send() calls so that MHD sees partial chunk-size lines. Every third iteration appends a pipelined follow-up request, so a chunk-extension line that is not consumed correctly shows up as a request boundary de-synchronisation. test_get_chunked_ext additionally makes MHD answer with a chunked reply that carries a footer. These tests also run a small deterministic *self-check*, see below. test_pipelined (+ "_close") Two to five complete requests written with a single send() call, mixing GET/HEAD/POST/PUT, bodies with Content-Length, chunked bodies, and a leading empty line. Every other iteration the batch is split in the middle of a request. test_digest_auth (+ "_close") [only if ENABLE_DAUTH] libcurl performs a complete Digest handshake, so zzuf's bit flips land inside real "WWW-Authenticate:" and "Authorization:" headers. The raw client additionally sends hand-crafted "Authorization: Digest" headers with an unknown "algorithm" token, an empty "algorithm", a "-sess" algorithm, an over-long "response" (quoted and unquoted), an over-long hex userhash, an over-long username, a missing "response", garbage in "nc", an unterminated quoted string and a bare "Digest" scheme without parameters. The variants that have to survive MHD's nonce check are built from a nonce that the raw client first obtains from MHD's own 401 reply. test_basic_auth (+ "_close") [only if ENABLE_BAUTH] libcurl performs a Basic handshake; the raw client adds truncated Base64, characters outside the Base64 alphabet, payloads without the ':' separator, payloads decoding to binary zeros, padding in the middle, an empty value, a 4 KiB payload, whitespace inside the payload and an unknown authentication scheme. Automake conditionals used by this directory: FORCE_USE_ZZUF_SOCAT, USE_COVERAGE, VHEAVY_TESTS (already used before) ENABLE_DAUTH, ENABLE_BAUTH (new here) All of them are defined in configure.ac. The directory as a whole is only built when RUN_ZZUF_TESTS is true (see src/Makefile.am), which in turn requires --enable-heavy-tests and a usable libcurl. The deterministic self-check ---------------------------- Bit flipping can show that a parser crashes, but it cannot show that a parser produces the *wrong result*. test_get_chunked_ext and test_put_chunked_ext therefore also verify, once per run, that a chunked body with chunk extensions is decoded byte-exactly and answered with "200". That verification obviously must not be fuzzed. In socat mode zzuf only fuzzes the socat relay, so MHD's own port is a clean channel: the self-check connects to MHD directly, bypassing socat. In direct mode every byte that MHD accepts is fuzzed and no clean channel exists, so the self-check is silently skipped there. Since socat mode is mandatory whenever MHD is built with sanitizers, the sanitizer workflow below always includes the self-check. Running under the sanitizers ---------------------------- ./configure --enable-heavy-tests --enable-asserts \ --enable-sanitizers=address,undefined make make -C src/testzzuf check Enabling sanitizers automatically forces the socat mode (see configure.ac), because zzuf and the sanitizers cannot both intercept the standard library calls. The Makefile appends ASAN_OPTIONS=abort_on_error=1:detect_leaks=0 UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 to whatever configure has already put into ASAN_OPTIONS / UBSAN_OPTIONS. Aborting on the first error is essential: otherwise the input that produced the error is lost. Leak detection is off because MHD legitimately abandons half-processed requests when the input is broken, and because the test process may be killed by zzuf. The values can be overridden per run: make ZZUF_ASAN_OPTIONS=abort_on_error=1:detect_leaks=1 check Seeds, replaying and sweeping ----------------------------- A single seed (the default is 0): make ZZUF_SEED=1234 check A range of seeds; zzuf starts one child per seed and stops at the first crash: make ZZUF_SEED_START=0 ZZUF_SEED_STOP=500 check ZZUF_SEED_START takes precedence over ZZUF_SEED. A large range takes a correspondingly long time, so for CI it is usually better to sweep a moderate range per job and to give different jobs different ranges. Both runner scripts print a prominent, easy to grep block when a test fails: ############################################################### ## FUZZING TEST FAILED ## test : ./test_get_args ## exit code : 134 ## seed(s) used: 0 ## FAILING SEED(S): 17 ## Replay a single failing seed with: ## make ZZUF_SEED=17 TESTS=test_get_args check ############################################################### In direct mode the failing seeds are extracted from zzuf's own per-child messages ("zzuf[s=17,r=...]: signal 6 (SIGABRT)"). In socat mode the test program runs outside of zzuf, so the runner reports the exit code (or the signal) of the *server side* together with the seed the run was started with. Other useful knobs: make ZZUF_FLAGS='--ratio=0.01:0.1' check extra zzuf flags make SOCAT_FLAGS='-v' check extra socat flags make TESTS='test_get_args test_pipelined' check run a subset ./configure --enable-heavy-tests=full 200 instead of 10 client iterations per daemon What zzuf-style fuzzing fundamentally cannot find ------------------------------------------------- This is important, because three of the four remotely triggerable bugs that were fixed on 2026-07-27 had been sitting in code that this directory nominally "covered". zzuf flips bits in an otherwise valid, libcurl-generated byte stream. That gives a *uniformly random* perturbation of a *structurally valid* input. Consequently: * It cannot invent structure that the generator never produces. libcurl never emits chunk extensions, never pipelines, never sends a request without a "Host:" header, and never sends an "Authorization:" header unless the test asks it to. No amount of bit flipping turns a libcurl request into a request with a chunk extension: the fuzzer would have to insert bytes, not change them, and it would have to keep the surrounding framing intact. * It essentially never produces a *semantically hostile but syntactically valid* token. "algorithm=BOGUS-ALGO" or a "response=" with 128 hex digits are perfectly well-formed HTTP; they are simply values a well-behaved client never sends. Random bit flips overwhelmingly produce inputs that MHD rejects in the first few hundred bytes of parsing, so the deep parsers are never reached with values that are valid enough to get there. Reaching the over-long "response" check even requires a *server generated* nonce, which a random mutation can never produce. * It only explores the option space that the tests configure. The read-buffer shift-back arithmetic simply does not run unless the connection memory pool is below 1500 bytes. * It only detects crashes and hangs. A parser that silently accepts a malformed chunk-extension line, or that mis-frames a request, looks exactly like a healthy parser to this suite unless the resulting memory access happens to be invalid. The additions described above buy back a good deal of this: the raw client provides the structures libcurl cannot express, the option matrix provides the configurations, and the deterministic self-check provides an oracle for "wrong result" instead of only "crash". But they do it by hand-writing interesting inputs, which does not generalise. The complementary approach is a structure-aware harness, which lives in src/fuzz/. There the HTTP request is *generated* from the fuzzer's random bytes (so the fuzzer can produce any header, any chunk extension, any authentication parameter, in any combination) and the fuzzer is coverage-guided, so it can learn its way into the deep parsers instead of stumbling into them. Use src/testzzuf for "does MHD survive a hostile network", and src/fuzz for "does MHD parse hostile HTTP correctly".