/
nv-lang
/
nova
Обзор
Документация
Войти
/
nv-lang
/
nova
Код
Запросы
0
Задачи
Вики
Пакеты
0
Релизы
0
Аналитика
main
compiler-codegen/src/codegen/split_tu.rs
1 453 строки
70 KB
Evgeniy Golovin
fix(209 Ф.3): duplicate-symbol const-value в multi-TU split — определение в 1 part + extern в common.h
15 июл 2026, 10:58
15 июл 2026, 10:58
13c0396
Код
Авторство
О чём код?
// SPDX-License-Identifier: MIT OR Apache-2.0 //! Plan 209 Ф.1 (A2): post-finalize multi-TU splitter. //! //! Takes the single finalized `.c` string `emit_c.rs::emit_module` produces //! today and, when multi-TU is requested (A4 gate: `NOVA_MULTI_TU` on AND CU //! over threshold), rewrites it into one `_common.h` (declarations only) + N //! `_partK.c` translation units (definitions), each `#include`-ing the //! common header. This is a PURE TEXT transform — no AST/type information — //! because A1 (`CEmitter::top_level_storage`/`top_level_storage_inline`) //! already promoted every top-level `static` definition that must be //! callable across parts to external linkage, and codegen's mangle scheme //! (D381 collision-aware) already guarantees CU-wide unique names. //! //! Design rationale: docs/plans/209-recon-notes.md §5 (segmentator) + §2-4 //! (what goes where). Default path (`emit_module` with multi-TU disabled or //! CU under threshold) NEVER calls this module — zero risk to the //! byte-identical single-`.c` output. //! //! ## Algorithm //! //! 1. **Segment** the finalized text into an ordered, CONTIGUOUS list of //! top-level raw units (concatenating all unit texts reproduces the //! input exactly) — see `segment_top_level`. A unit is one of: //! - a single preprocessor directive line (`#include`/`#define`/`#pragma`/ //! `#undef`), ending at its (possibly backslash-continued) newline; //! - an ATOMIC `#if`/`#ifdef`/`#ifndef` … `#endif` block (nested //! `#if*`/`#endif` tracked so an inner conditional doesn't truncate the //! outer one) — kept whole because splitting `#ifdef X … #else … #endif` //! across different output files would unbalance the preprocessor; //! - a normal top-level C construct — from the end of the previous unit //! through the next depth-0 `;`, OR (for a function body) through the //! matching depth-0 `}` when the text just before the opening `{` ends //! with `)` (a function signature) — determined by //! `looks_like_fn_signature`. Depth tracking is brace-only and skips //! string/char literals and `//`/`/* */` comments so literal braces //! inside them never desync the scan. //! 2. **Classify** each unit (`classify_unit`): `#include`/`#define`/other //! bare directives and typedefs/decl-only constructs → `_common.h`; //! function-body definitions and global-with-initializer definitions → //! a `_partK.c` (round-robin by accumulated byte size, `threshold_bytes` //! per part); a small **known-macro table** (`NOVA_BENCH_STATE_DEFINE` //! and friends — opaque macro-invocation statements that actually expand //! to global storage, recon-notes §5) are treated as definitions. //! 3. **Atomic conditional blocks** containing a definition anywhere inside //! are NOT split: the whole block goes verbatim into one part (still //! correctly conditionally compiled), AND a declaration-only MIRROR of //! the same block (directives preserved, inner definitions rewritten to //! prototypes/`extern` decls) is emitted into `_common.h` — see //! `mirror_cond_block_as_decl`. //! 4. **Dedup**: a decl-only unit (plain forward declaration) whose //! extracted name matches a definition unit found ANYWHERE in the output //! is dropped — the authoritative declaration for `_common.h` is instead //! AUTO-GENERATED from the definition itself (`decl_from_fn_def` / //! `extern_from_global_def`). This means A1 does not need to have found //! every historical forward-decl call site in emit_c.rs: a leftover //! `static`-prefixed forward decl is simply superseded here, not //! concatenated (which would otherwise conflict with the promoted //! external definition within the SAME part — a loud compile error, not //! silent corruption, if some future A1 site is ever missed). //! 5. **Assemble**: `_common.h` = include-guard + the effect-count comment //! (verbatim first line of the input, always) + all common-bound units //! (original relative order) + all auto-generated prototypes/externs. //! `_partK.c` = `#include "<cu>_common.h"` + its assigned definitions //! (original relative order preserved within a part). use std::collections::HashSet; /// Known macro-invocation STATEMENTS (not `#define`s themselves — plain /// `IDENT;` top-level statements) that expand to actual global storage /// definitions (see `compiler-codegen/nova_rt/bench.h`). If treated as an /// ordinary opaque declaration they would end up duplicated into every /// part via `_common.h` inclusion → multiple-definition link error. Recon /// notes §5 calls these out explicitly as needing table-driven handling. const KNOWN_PART_ONLY_MACRO_STATEMENTS: &[&str] = &[ "NOVA_BENCH_STATE_DEFINE", "NOVA_BENCH_HEAP_SAMPLER_THREAD_DEFINE", ]; /// Result of `split_tu`: one `_common.h` body + N part bodies (`_part0.c /// .. _partK.c`, in order). Callers (A4 / Ф.2 toolchain) decide file names /// and actual disk layout; this module only produces text. #[derive(Debug, Clone, PartialEq, Eq)] pub struct SplitResult { pub common_h: String, pub parts: Vec<String>, } /// A classified top-level raw unit, in original order. #[derive(Debug, Clone, PartialEq, Eq)] enum UnitKind { /// `#include`, `#define`, `#pragma`, `#undef`, or a bare conditional /// block whose every inner unit is header-safe (typedef/decl-only) — /// emitted verbatim into `_common.h`, in original relative order. HeaderVerbatim, /// Function-body definition. Payload: the name used for dedup + /// auto-generated prototype. FnDef { name: String, proto: String }, /// Top-level object with an initializer (`TYPE NAME = ...;`). Payload: /// auto-generated `extern` line for `_common.h`. GlobalDef { name: String, extern_decl: String }, /// An atomic `#if.../#endif` block that contains a definition inside — /// kept whole, routed to a single part; `header_mirror` is the /// declaration-only replacement emitted into `_common.h`. CondBlockWithDef { header_mirror: String }, /// Decl-only forward declaration/prototype (ends `;`, no body). Kept /// only if no matching definition supersedes it (dedup in `split_tu`). DeclOnly { name: Option<String> }, /// Known macro-invocation statement that expands to global storage /// (`NOVA_BENCH_STATE_DEFINE;` and friends) — always part-bound, never /// deduped/declared (there is nothing to declare; it's a fixed, /// single-occurrence macro call). KnownPartOnlyMacro, } /// Segments `src` into a contiguous list of raw top-level unit slices /// (concatenating every returned slice reproduces `src` exactly). Handles: /// string/char literals, `//` and `/* */` comments (braces/semicolons /// inside them never affect depth tracking), single-line preprocessor /// directives (with backslash-continuation), and atomic `#if*`/`#endif` /// blocks (nesting tracked so an inner conditional's `#endif` doesn't /// terminate the outer one). fn segment_top_level(src: &str) -> Vec<&str> { let bytes = src.as_bytes(); let n = bytes.len(); let mut out = Vec::new(); let mut unit_start = 0usize; let mut i = 0usize; #[derive(PartialEq)] enum Mode { Normal, Str, Char, LineComment, BlockComment } let mut mode = Mode::Normal; let mut depth: i32 = 0; // Set at the moment depth 0->1 happens (the outermost '{' of this // unit): does the text right before it look like a fn signature // (`...) {`)? If so, the unit ends the instant depth returns to 0 // (no trailing `;` expected — that's how a C function definition // looks). Otherwise (struct/enum typedef, initializer) we keep // scanning for the following top-level `;`. let mut awaiting_semi_after_brace = false; while i < n { // At the start of a fresh unit, check for a preprocessor directive // line or an atomic conditional block before falling into the // generic brace/semicolon scan. if depth == 0 && i == unit_start { let mut k = i; // Plan 209 Ф.2 finding: skip blank lines too (`\n`/`\r`), not // just spaces/tabs — a directive is virtually ALWAYS preceded // by at least one blank line in this codebase's generated // output (`self.line("")` between constructs). Skipping only // ` `/`\t` left `bytes[k]` pointing at `\n` (not `#`) whenever // a unit started with such a blank line, so this whole // cond-block/directive detection silently never fired for it — // the directive then fell through to the generic brace/`;` // scanner below, which cut a multi-line `#ifdef ... #else ... // #endif` block into THREE separate mis-segmented units at each // `;` (observed: `_nova_handler_Random`'s per-E TLS cond-block // split apart, corrupting both its `_common.h` mirror — a // definition without `extern` — and the part — a `#ifdef` // with no matching `#else`/`#endif`, "unterminated conditional // directive" at the C compiler). while k < n && matches!(bytes[k], b' ' | b'\t' | b'\n' | b'\r') { k += 1; } if k < n && bytes[k] == b'#' { let directive_end = k; let is_cond_open = src[directive_end..].starts_with("#if"); if is_cond_open { let end = scan_atomic_cond_block(src, directive_end); out.push(&src[unit_start..end]); unit_start = end; i = end; continue; } else { let end = scan_directive_line(src, directive_end); out.push(&src[unit_start..end]); unit_start = end; i = end; continue; } } } let c = bytes[i]; match mode { Mode::LineComment => { if c == b'\n' { mode = Mode::Normal; } i += 1; } Mode::BlockComment => { if c == b'*' && i + 1 < n && bytes[i + 1] == b'/' { mode = Mode::Normal; i += 2; } else { i += 1; } } Mode::Str => { if c == b'\\' && i + 1 < n { i += 2; } else { if c == b'"' { mode = Mode::Normal; } i += 1; } } Mode::Char => { if c == b'\\' && i + 1 < n { i += 2; } else { if c == b'\'' { mode = Mode::Normal; } i += 1; } } Mode::Normal => { if c == b'/' && i + 1 < n && bytes[i + 1] == b'/' { mode = Mode::LineComment; i += 2; } else if c == b'/' && i + 1 < n && bytes[i + 1] == b'*' { mode = Mode::BlockComment; i += 2; } else if c == b'"' { mode = Mode::Str; i += 1; } else if c == b'\'' { mode = Mode::Char; i += 1; } else if c == b'{' { if depth == 0 { awaiting_semi_after_brace = looks_like_fn_signature(&src[unit_start..i]); } depth += 1; i += 1; } else if c == b'}' { depth -= 1; i += 1; if depth <= 0 { depth = 0; if awaiting_semi_after_brace { out.push(&src[unit_start..i]); unit_start = i; } // else: struct/enum typedef or initializer — keep // scanning for the terminating top-level `;`. } } else if c == b';' && depth == 0 { i += 1; out.push(&src[unit_start..i]); unit_start = i; } else { i += 1; } } } } if unit_start < n { // A trailing whitespace-only remainder (e.g. the final newline // after the CU's last statement) is not a semantic unit of its // own — fold it into the previous unit so unit counts reflect // actual top-level constructs, not incidental EOF whitespace. if src[unit_start..n].trim().is_empty() { if let Some(last) = out.pop() { let start = last.as_ptr() as usize - src.as_ptr() as usize; out.push(&src[start..n]); } else { out.push(&src[unit_start..n]); } } else { out.push(&src[unit_start..n]); } } out } /// Does the (trimmed, trailing-whitespace-stripped) text immediately before /// a depth-0 `{` look like a C function signature (`RET NAME(PARAMS)`)? The /// generated code's only other depth-0-brace constructs are struct/union/ /// enum typedefs and initializers, none of which end their pre-brace text /// with `)`. fn looks_like_fn_signature(pre: &str) -> bool { pre.trim_end().ends_with(')') } /// Scans a single preprocessor directive line starting at `start` (the `#` /// byte), honoring backslash-newline continuation. Returns the offset just /// past the directive's final newline (or end-of-input). fn scan_directive_line(src: &str, start: usize) -> usize { let bytes = src.as_bytes(); let n = bytes.len(); let mut e = start; loop { match src[e..].find('\n') { None => return n, Some(rel) => { let nl = e + rel; let mut back = nl; while back > e && bytes[back - 1] == b'\r' { back -= 1; } if back > e && bytes[back - 1] == b'\\' { e = nl + 1; // continuation — keep scanning } else { return nl + 1; } } } } } /// Scans an atomic `#if`/`#ifdef`/`#ifndef` … `#endif` block starting at /// `start` (the opening `#`), tracking nested `#if*`/`#endif` balance (an /// inner conditional's `#endif` does not close the outer one). Returns the /// offset just past the matching `#endif`'s newline. Never looks inside /// string/char literals or comments for `#` (none of this codebase's /// generated conditionals contain those, and a stray `#` inside a string /// on its own line would not be at start-of-line after whitespace-skip /// anyway, per the caller's `i == unit_start` + leading-whitespace check). fn scan_atomic_cond_block(src: &str, start: usize) -> usize { let n = src.len(); let mut depth = 0i32; let mut pos = start; loop { let line_end = scan_directive_line(src, pos); let line = &src[pos..line_end]; let trimmed = line.trim_start(); if trimmed.starts_with("#if") { depth += 1; } else if trimmed.starts_with("#endif") { depth -= 1; if depth == 0 { return line_end; } } if line_end >= n { return n; } pos = line_end; } } /// One structural piece of an atomic `#if.../#endif` block, at THIS /// block's own nesting level (a nested conditional's directive lines are /// NOT split out here — they stay inside a `Content` piece and are only /// discovered when that piece is itself re-segmented). enum CondPiece<'a> { /// One of this block's own `#if*`/`#elif`/`#else`/`#endif` lines. Directive(&'a str), /// Text between two of this block's own directive lines (may itself /// contain nested nested conditionals/definitions/decls). Content(&'a str), } /// Splits an atomic `#if.../#endif` block into its own directive lines and /// the content spans between them. Byte-contiguous (every byte of `block` /// appears in exactly one piece, in order). fn split_cond_block_pieces(block: &str) -> Vec<CondPiece<'_>> { let n = block.len(); let mut pieces = Vec::new(); let mut depth = 0i32; let mut pos = 0usize; let mut content_start = 0usize; loop { let line_end = scan_directive_line(block, pos); let line = &block[pos..line_end]; let trimmed = line.trim_start(); let is_this_level_directive = if trimmed.starts_with("#if") { depth += 1; depth == 1 } else if trimmed.starts_with("#endif") { let was_top = depth == 1; depth -= 1; was_top } else { (trimmed.starts_with("#else") || trimmed.starts_with("#elif")) && depth == 1 }; if is_this_level_directive { if pos > content_start { pieces.push(CondPiece::Content(&block[content_start..pos])); } pieces.push(CondPiece::Directive(line)); content_start = line_end; } if line_end >= n { break; } pos = line_end; } if content_start < n { pieces.push(CondPiece::Content(&block[content_start..n])); } pieces } /// Extracts the trailing identifier name from a declarator prefix (text up /// to — but not including — a delimiter like `(` or `=`). Handles trailing /// `*`/whitespace between the name and the delimiter. Returns `None` if no /// identifier-shaped token is found (defensive — should not happen for /// this codebase's machine-generated declarations). fn trailing_identifier(prefix: &str) -> Option<String> { let bytes = prefix.as_bytes(); let mut end = bytes.len(); while end > 0 { let c = bytes[end - 1]; if c == b' ' || c == b'\t' || c == b'\n' || c == b'\r' || c == b'*' { end -= 1; } else { break; } } let mut start = end; while start > 0 { let c = bytes[start - 1]; if c.is_ascii_alphanumeric() || c == b'_' { start -= 1; } else { break; } } if start == end { return None; } let ident = &prefix[start..end]; if ident.chars().next().map_or(false, |c| c.is_ascii_digit()) { return None; // starts with a digit — not an identifier } Some(ident.to_string()) } /// For a function-body definition unit's text, extract (name, prototype). /// `prototype` is the signature (everything up to the first depth-0 `{`) /// trimmed and terminated with `;` — a valid forward declaration for /// `_common.h` regardless of which part the body ends up in. /// /// Plan 209 Ф.2 finding: `unit` includes any leading doc-comment attached to /// the definition — a comment referencing a parenthesized call in prose /// (e.g. `/* ...assigned to _nova_supervisor_decide_fn in main()... */`, /// or a doc-example like `` `x.method(args)` ``) contains its OWN `(` /// BEFORE the real signature's — a plain `sig.find('(')` (the original /// implementation) matches THAT one first and misnames the unit (observed: /// `_nova_supervisor_decide_impl`'s unit misclassified as name `"main"`, /// colliding with the real `int main(...)` and tripping the A3 uniqueness /// invariant). `find_first_real_paren` skips comments/strings/chars first /// (mirrors `find_top_level_open_brace`'s mode-tracking exactly), so it /// lands on the signature's own `(` regardless of what the comment above /// it says. /// Does this function-signature prefix (text up to, not including, the /// opening `{`) declare the function `inline` (`inline ...` or /// `static inline ...`, the only two forms this codebase emits — see /// `CEmitter::top_level_storage_inline`'s two permanent exceptions, /// `nova_typeid_user_name` and `_nova_throw_typed_<m>`, which stay /// `static inline` under every flag combination)? Skips a leading doc /// comment first (mirrors `decl_from_uninitialized_global`'s /// `skip_leading_trivia` use, same rationale: `find_first_real_paren`-style /// comment-blindness bugs already bit this file twice, see Ф.2 findings /// above). fn sig_has_inline_keyword(sig: &str) -> bool { let core = skip_leading_trivia(sig); core.starts_with("inline ") || core.starts_with("static inline ") } fn decl_from_fn_def(unit: &str) -> Option<(String, String)> { let brace = find_top_level_open_brace(unit)?; let sig = unit[..brace].trim_end(); if !sig.ends_with(')') { return None; } // must look like a fn signature let paren = find_first_real_paren(sig)?; let name = trailing_identifier(&sig[..paren])?; Some((name, format!("{};", sig))) } /// Find the first `(` that is NOT inside a `//`/`/* */` comment or a /// string/char literal — mirrors `find_top_level_open_brace`'s mode- /// tracking, scanning for `(` instead of `{`. See `decl_from_fn_def` doc /// for why a naive `str::find('(')` is unsound here. fn find_first_real_paren(s: &str) -> Option<usize> { let bytes = s.as_bytes(); let n = bytes.len(); let mut i = 0usize; #[derive(PartialEq)] enum Mode { Normal, Str, Char, LineComment, BlockComment } let mut mode = Mode::Normal; while i < n { let c = bytes[i]; match mode { Mode::LineComment => { if c == b'\n' { mode = Mode::Normal; } i += 1; } Mode::BlockComment => { if c == b'*' && i + 1 < n && bytes[i + 1] == b'/' { mode = Mode::Normal; i += 2; } else { i += 1; } } Mode::Str => { if c == b'\\' && i + 1 < n { i += 2; } else { if c == b'"' { mode = Mode::Normal; } i += 1; } } Mode::Char => { if c == b'\\' && i + 1 < n { i += 2; } else { if c == b'\'' { mode = Mode::Normal; } i += 1; } } Mode::Normal => { if c == b'/' && i + 1 < n && bytes[i + 1] == b'/' { mode = Mode::LineComment; i += 2; } else if c == b'/' && i + 1 < n && bytes[i + 1] == b'*' { mode = Mode::BlockComment; i += 2; } else if c == b'"' { mode = Mode::Str; i += 1; } else if c == b'\'' { mode = Mode::Char; i += 1; } else if c == b'(' { return Some(i); } else { i += 1; } } } } None } fn find_top_level_open_brace(s: &str) -> Option<usize> { // The first depth-0 '{' — mirrors the classification already performed // by segment_top_level (this unit was already confirmed to be a fn-def // there), so a naive scan (skipping strings/comments) is sufficient // and always finds it before any nesting. let bytes = s.as_bytes(); let n = bytes.len(); let mut i = 0usize; #[derive(PartialEq)] enum Mode { Normal, Str, Char, LineComment, BlockComment } let mut mode = Mode::Normal; while i < n { let c = bytes[i]; match mode { Mode::LineComment => { if c == b'\n' { mode = Mode::Normal; } i += 1; } Mode::BlockComment => { if c == b'*' && i + 1 < n && bytes[i + 1] == b'/' { mode = Mode::Normal; i += 2; } else { i += 1; } } Mode::Str => { if c == b'\\' && i + 1 < n { i += 2; } else { if c == b'"' { mode = Mode::Normal; } i += 1; } } Mode::Char => { if c == b'\\' && i + 1 < n { i += 2; } else { if c == b'\'' { mode = Mode::Normal; } i += 1; } } Mode::Normal => { if c == b'/' && i + 1 < n && bytes[i + 1] == b'/' { mode = Mode::LineComment; i += 2; } else if c == b'/' && i + 1 < n && bytes[i + 1] == b'*' { mode = Mode::BlockComment; i += 2; } else if c == b'"' { mode = Mode::Str; i += 1; } else if c == b'\'' { mode = Mode::Char; i += 1; } else if c == b'{' { return Some(i); } else { i += 1; } } } } None } /// Finds the byte offset of the top-level (depth-0, outside strings/ /// comments) `=` sign in a global-with-initializer unit, if any. fn find_top_level_eq(s: &str) -> Option<usize> { let bytes = s.as_bytes(); let n = bytes.len(); let mut i = 0usize; let mut depth = 0i32; #[derive(PartialEq)] enum Mode { Normal, Str, Char, LineComment, BlockComment } let mut mode = Mode::Normal; while i < n { let c = bytes[i]; match mode { Mode::LineComment => { if c == b'\n' { mode = Mode::Normal; } i += 1; } Mode::BlockComment => { if c == b'*' && i + 1 < n && bytes[i + 1] == b'/' { mode = Mode::Normal; i += 2; } else { i += 1; } } Mode::Str => { if c == b'\\' && i + 1 < n { i += 2; } else { if c == b'"' { mode = Mode::Normal; } i += 1; } } Mode::Char => { if c == b'\\' && i + 1 < n { i += 2; } else { if c == b'\'' { mode = Mode::Normal; } i += 1; } } Mode::Normal => { if c == b'/' && i + 1 < n && bytes[i + 1] == b'/' { mode = Mode::LineComment; i += 2; } else if c == b'/' && i + 1 < n && bytes[i + 1] == b'*' { mode = Mode::BlockComment; i += 2; } else if c == b'"' { mode = Mode::Str; i += 1; } else if c == b'\'' { mode = Mode::Char; i += 1; } else if c == b'(' || c == b'[' || c == b'{' { depth += 1; i += 1; } else if c == b')' || c == b']' || c == b'}' { depth -= 1; i += 1; } else if c == b'=' && depth == 0 { // Reject `==`, `!=`, `<=`, `>=` (not relevant at true // top level for a declarator, but defensive) and `=` // used as part of `+=` etc (also not expected here). let prev = if i > 0 { bytes[i - 1] } else { 0 }; let next = if i + 1 < n { bytes[i + 1] } else { 0 }; if next != b'=' && prev != b'!' && prev != b'<' && prev != b'>' && prev != b'=' { return Some(i); } i += 1; } else { i += 1; } } } } None } /// For a top-level `TYPE NAME = INIT;` unit, extract (name, extern decl). /// /// Plan 209 Ф.2 finding: an ARRAY-typed global (`TYPE NAME[] = INIT;` or /// `TYPE NAME[N] = INIT;` — e.g. the interned string literal byte buffers, /// `static const uint8_t _nova_strlit_<hash>_buf[] = "...";`) has its LHS /// end in `]`, not the identifier itself — `trailing_identifier` (which /// only skips trailing whitespace/`*`) then finds no identifier-shaped /// token and returns `None`, so this whole function returned `None` too. /// The unit then fell through to `DeclOnly { name: None }` in /// `classify_unit` — UNNAMED means the A3 dedup pass can never supersede /// it, so it was kept VERBATIM (not `extern`-ified) in `_common.h`; every /// part `#include`ing that header then got its own copy of a Plan 209 /// promoted (non-`static`) definition — `lld-link: error: duplicate /// symbol` for every such array global. Strip the trailing `[...]` /// bracket group(s) first so `trailing_identifier` sees the true name. fn decl_from_global_def(unit: &str) -> Option<(String, String)> { let eq = find_top_level_eq(unit)?; let lhs = unit[..eq].trim_end(); let name = trailing_identifier(strip_trailing_array_brackets(lhs))?; Some((name, format!("extern {};", lhs))) } /// Strips one or more trailing balanced `[...]` bracket groups (e.g. `[]`, /// `[16]`, or multi-dimensional `[4][8]`) plus any whitespace between them, /// so a caller can find the identifier that precedes an array declarator's /// brackets. Returns the input trimmed-of-trailing-whitespace unchanged if /// it doesn't end in `]`, or if the brackets are unbalanced (defensive — /// should not happen for this codebase's generated declarators). fn strip_trailing_array_brackets(s: &str) -> &str { let mut cur = s.trim_end(); loop { if !cur.ends_with(']') { return cur; } let bytes = cur.as_bytes(); let mut depth = 0i32; let mut idx = cur.len(); let mut balanced = false; while idx > 0 { idx -= 1; match bytes[idx] { b']' => depth += 1, b'[' => { depth -= 1; if depth == 0 { balanced = true; break; } } _ => {} } } if !balanced { return cur; // unbalanced brackets — give up, defensive only } cur = cur[..idx].trim_end(); } } /// For a top-level, brace-free, `=`-free unit ending in `;` — i.e. a plain /// C declarator statement (`TYPE NAME;`) — extract (name, extern decl) IF /// this is a tentative-definition GLOBAL OBJECT (uninitialized file-scope /// storage: `nova_int _nova_const_ZERO_value;`, emitted by /// `CEmitter::emit_lazy_const` for every lazy `ro`/module const — the /// combined `nova_consts_init()` assigns it later, at runtime, not via a /// C initializer). In standard C this has NO `extern`/`static` keyword and /// no initializer, which makes it a **tentative definition**: under Clang's /// (and lld-link's) default `-fno-common`, every translation unit that /// contains one becomes a strong definition of that symbol. `_common.h` is /// `#include`d by EVERY part, so if this shape is (as before this fix) kept /// verbatim in the header, every part's `.c` gets its OWN copy of the /// definition — `lld-link: error: duplicate symbol` at link time (Plan 209 /// Ф.3 finding: `_nova_const_ZERO_value`, `_nova_const_lower_map_value`, /// `_nova_const_collate_single_map_value`, ... — every lazy const in any CU /// that splits into >1 part). Must be treated exactly like an initialized /// global (recon-notes §4's "глобал с ОПРЕДЕЛЕНИЕМ" invariant): definition /// in exactly ONE part, `extern` declaration in `_common.h`. /// /// Deliberately excluded (return `None`, fall through to plain `DeclOnly`, /// kept verbatim/duplicate-safe in the header as before): /// - `typedef ...;` (type forward-decls, e.g. `typedef struct Nova_X Nova_X;` /// or the braceless `typedef int64_t Nova_X;` empty-sum shape) — these are /// type declarations, not object definitions; duplicating them across /// every part (they all `#include` the same header) is exactly what a /// header is for. /// - anything already spelled `extern ...;` — already an explicit /// declaration of storage defined elsewhere; safe to duplicate verbatim. /// - anything containing a top-level `(` — a function prototype (handled by /// the caller's existing paren-based `DeclOnly` name extraction), not an /// object declarator. fn decl_from_uninitialized_global(unit: &str) -> Option<(String, String)> { let trimmed = unit.trim(); let body = trimmed.strip_suffix(';')?.trim_end(); if body.is_empty() { return None; } // Plan 209 Ф.3 (found while verifying against real generated output, // `e2e_collate.nv`): a leading DOC COMMENT (e.g. the `/* Plan 36: ... // */` attached to `typedef int64_t Nova_RawMem;`) means a naive // `trimmed.starts_with("typedef ")` misses it — the comment sits before // the keyword — and this function wrongly classified the typedef as a // tentative-definition global, prefixing the WHOLE unit (comment AND // typedef) with `extern `: nonsense C (`extern /* ... */\ntypedef ...`) // that fails to compile ("cannot combine with previous 'extern' // declaration specifier" / spurious redefinition once the malformed // line desyncs the parser). Skip leading `//`/`/* */` trivia (mirrors // `find_first_real_paren`'s mode-tracking) before checking for the // `typedef`/`extern` keywords that must exempt this unit. let core = skip_leading_trivia(body); if core.starts_with("typedef ") || core.starts_with("typedef\t") { return None; } if core.starts_with("extern ") || core.starts_with("extern\t") { return None; } if core.contains('(') || core.contains('{') || core.contains('}') { return None; } if core.trim().is_empty() { return None; } let stripped = strip_trailing_array_brackets(core); let name = trailing_identifier(stripped)?; // Defensive: the declarator must have at least one token before the // name (a bare identifier with no type at all is not a C declaration // this codebase ever emits standalone at top level) — reject rather // than mis-promote something unrecognized. let before_name = &stripped[..stripped.len() - name.len()]; if before_name.trim().is_empty() { return None; } Some((name, format!("extern {};", body))) } /// Skips leading whitespace and `//`/`/* */` comments (repeatedly, so /// `/* a */ // b\n TYPE` correctly lands on `TYPE`), returning the /// remaining suffix. Used by `decl_from_uninitialized_global` to look past /// a leading doc-comment before checking for the `typedef`/`extern` /// keywords that exempt a unit from tentative-definition promotion. fn skip_leading_trivia(s: &str) -> &str { let mut cur = s; loop { let trimmed = cur.trim_start(); if trimmed.starts_with("//") { cur = match trimmed.find('\n') { Some(nl) => &trimmed[nl + 1..], None => "", }; } else if trimmed.starts_with("/*") { cur = match trimmed[2..].find("*/") { Some(end) => &trimmed[2 + end + 2..], None => "", // unterminated — defensive, shouldn't happen }; } else { return trimmed; } } } /// Is this unit (trimmed) exactly one of the known part-only macro /// invocation statements (see `KNOWN_PART_ONLY_MACRO_STATEMENTS`)? fn is_known_part_only_macro(unit: &str) -> bool { let t = unit.trim(); let t = t.strip_suffix(';').unwrap_or(t).trim_end(); KNOWN_PART_ONLY_MACRO_STATEMENTS.iter().any(|m| *m == t) } /// Classifies one raw top-level unit (already segmented by /// `segment_top_level`). fn classify_unit(unit: &str) -> UnitKind { let trimmed = unit.trim_start(); if trimmed.starts_with('#') { // Directive or atomic conditional block. If it's a conditional // block, check whether it contains a definition anywhere inside; // if so it must be part-bound (with a header mirror), otherwise // it's header-safe verbatim. if trimmed.starts_with("#if") { if cond_block_contains_definition(unit) { return UnitKind::CondBlockWithDef { header_mirror: mirror_cond_block_as_decl(unit), }; } } return UnitKind::HeaderVerbatim; } if is_known_part_only_macro(unit) { return UnitKind::KnownPartOnlyMacro; } // Function-body definition? if let Some(brace) = find_top_level_open_brace(unit) { let sig = unit[..brace].trim_end(); if sig.ends_with(')') { // Plan 209 Ф.3 finding: an `inline`/`static inline` function // DEFINITION (the two permanent exceptions the Ф.1 design // deliberately keeps `static inline` under ANY flag — // `nova_typeid_user_name`, per-E throw fast-path // `_nova_throw_typed_<m>` — plus, defensively, any future site // shaped the same way) must NOT be split into a bare prototype // (→ `_common.h`) + body (→ one part) like an ordinary `FnDef`. // `inline` semantics require the FULL definition to be visible // in every TU that calls it; a `static`-linkage function with // only a prototype visible has NO definition in that TU at all // — real failure observed: `_common.h` got // `static inline nova_unit _nova_throw_typed_WorkErr1(...);` // (prototype only), the body landed in exactly one part (this // is exactly the ordinary `FnDef` split, which is CORRECT for a // plain external definition but wrong here), and every OTHER // part calling it hit `lld-link: error: undefined symbol` (a // `static`-linkage callee with no body in that TU — the linker // can't reach across TUs for `static`). Fix: keep the WHOLE // unit (signature + body) verbatim in `_common.h` instead — // safe to duplicate per-TU exactly like any other // `static inline` header helper. if sig_has_inline_keyword(sig) { return UnitKind::HeaderVerbatim; } if let Some((name, proto)) = decl_from_fn_def(unit) { return UnitKind::FnDef { name, proto }; } } // Has a brace but isn't a fn signature (struct/union/enum typedef, // or a global with a brace-initializer) — decl-only (typedef) or // global-with-initializer; disambiguate via top-level `=`. if find_top_level_eq(&unit[..brace]).is_some() { if let Some((name, ext)) = decl_from_global_def(unit) { return UnitKind::GlobalDef { name, extern_decl: ext }; } } return UnitKind::HeaderVerbatim; // typedef struct {...} Name; and friends } // No braces at all: plain declaration, or a global with a scalar // initializer (`TYPE NAME = value;`), or an opaque statement. if find_top_level_eq(unit).is_some() { if let Some((name, ext)) = decl_from_global_def(unit) { return UnitKind::GlobalDef { name, extern_decl: ext }; } } // Plan 209 Ф.3: no `=`, no braces, no top-level `(` — a bare `TYPE // NAME;` declarator. Unless it's `typedef .../extern ...` (a real // declaration, safe to duplicate across parts via the header), this is // an uninitialized tentative-definition global (lazy-const storage // cells from `emit_lazy_const`) and MUST be single-part + `extern`'d, // exactly like an initialized global — see `decl_from_uninitialized_global`. if let Some((name, ext)) = decl_from_uninitialized_global(unit) { return UnitKind::GlobalDef { name, extern_decl: ext }; } // Decl-only (prototype, typedef, extern decl, `#`-free pragma-like // statement). Try to extract a name (for prototypes ending `NAME(...);`) // for dedup purposes; None is fine (kept verbatim, never deduped). let name = unit.rfind(')').and_then(|close_paren| { // Find the matching '(' for this ')' by scanning backward with a // simple paren counter (defensive against nested parens in params). let bytes = unit.as_bytes(); let mut depth = 0i32; let mut idx = close_paren; loop { let c = bytes[idx]; if c == b')' { depth += 1; } if c == b'(' { depth -= 1; if depth == 0 { break; } } if idx == 0 { return None; } idx -= 1; } trailing_identifier(&unit[..idx]) }); UnitKind::DeclOnly { name } } /// Does an atomic `#if.../#endif` block contain, anywhere inside (at any /// nesting depth), a unit that would classify as a definition? Used to /// decide whether the whole block must be part-bound. Walks the block's /// OWN directive lines via `split_cond_block_pieces` and re-segments each /// `Content` span (the directive lines themselves never classify as /// definitions; a nested conditional lives entirely inside a `Content` /// span and is discovered there). fn cond_block_contains_definition(block: &str) -> bool { for piece in split_cond_block_pieces(block) { let content = match piece { CondPiece::Content(c) => c, CondPiece::Directive(_) => continue }; for inner in segment_top_level(content) { match classify_unit(inner) { UnitKind::FnDef { .. } | UnitKind::GlobalDef { .. } | UnitKind::KnownPartOnlyMacro => return true, UnitKind::CondBlockWithDef { .. } => return true, _ => {} } } } false } /// Builds the `_common.h` declaration-only mirror of an atomic conditional /// block that contains a definition: this block's own directive lines are /// preserved verbatim; each inner definition unit (in a `Content` span) is /// replaced by its prototype/extern-decl; inner decl-only/header units are /// kept verbatim. fn mirror_cond_block_as_decl(block: &str) -> String { let mut out = String::new(); for piece in split_cond_block_pieces(block) { let content = match piece { CondPiece::Directive(d) => { out.push_str(d); continue; } CondPiece::Content(c) => c, }; for inner in segment_top_level(content) { match classify_unit(inner) { UnitKind::FnDef { proto, .. } => { out.push_str(&proto); out.push('\n'); } UnitKind::GlobalDef { extern_decl, .. } => { out.push_str(&extern_decl); out.push('\n'); } UnitKind::CondBlockWithDef { header_mirror } => { out.push_str(&header_mirror); } UnitKind::KnownPartOnlyMacro => { // Opaque macro invocation that expands to storage — cannot // safely mirror as a declaration; omit from the header. The // part carries the real (whole, conditional) definition; // nothing else in the CU can reference its internals // directly by name (it's a fixed, single-occurrence bench // scaffolding statement, not a Nova-visible symbol). } _ => { out.push_str(inner); } } } } out } /// Effect-count marker recon-notes require as `_common.h` line 1 (build /// layer reads it — Ф.2 concern, but the invariant is produced here). fn extract_effect_count_line(src: &str) -> (&str, &str) { if src.starts_with("/* nova-effect-count:") { if let Some(nl) = src.find('\n') { return (&src[..nl + 1], &src[nl + 1..]); } } ("", src) } /// Plan 209 Ф.1 (A2): splits a finalized single-`.c` string into one /// `_common.h` + N `_partK.c` bodies. `cu_name` seeds the include-guard /// macro and the `#include "<cu_name>_common.h"` line every part gets; /// `threshold_bytes` is the approximate per-part byte budget (a single /// definition is never split across parts, so a part may slightly exceed /// the threshold if one definition is larger than it). /// /// Callers (A4) MUST NOT invoke this when multi-TU is disabled — this /// function has no "identity" fast path and always re-renders the input /// (whitespace-for-whitespace unchanged for the pieces it keeps, but /// reorganized), so calling it unconditionally would break the Plan 209 /// byte-identical-default guarantee. pub fn split_tu(finalized: &str, cu_name: &str, threshold_bytes: usize) -> Result<SplitResult, String> { let (effect_count_line, rest) = extract_effect_count_line(finalized); let guard = format!("NOVA_{}_COMMON_H", sanitize_guard(cu_name)); let mut common_h = String::new(); common_h.push_str(effect_count_line); common_h.push_str(&format!("#ifndef {}\n#define {}\n", guard, guard)); // Collect (unit text, kind) for every unit, then a name -> exists-as- // definition set for the dedup pass. let raw_units = segment_top_level(rest); let mut kinds: Vec<UnitKind> = Vec::with_capacity(raw_units.len()); let mut defined_names: HashSet<String> = HashSet::new(); // Plan 209 Ф.1 (A3): CU-wide uniqueness invariant — every promoted // top-level definition name must appear EXACTLY ONCE in the whole // output. A duplicate here means the mangle scheme's CU-uniqueness // guarantee (D381 collision-aware mangle, recon-notes §2) was somehow // violated for this symbol — promoting `static` to external in that // case would silently multiply-define it across parts (or, worse, // collide two UNRELATED definitions under one name). Fail LOUD here // instead of letting the linker (or, worse, nothing) discover it. let mut dup_names: Vec<String> = Vec::new(); for u in &raw_units { let k = classify_unit(u); if let UnitKind::FnDef { name, .. } | UnitKind::GlobalDef { name, .. } = &k { if !defined_names.insert(name.clone()) { dup_names.push(name.clone()); } } kinds.push(k); } if !dup_names.is_empty() { dup_names.sort(); dup_names.dedup(); return Err(format!( "split_tu: CU-wide top-level symbol uniqueness invariant violated (Plan 209 A3) — \ duplicate definition name(s), promoting `static`->external would multiply-define \ or collide across parts: {}", dup_names.join(", ") )); } let mut parts: Vec<String> = vec![String::new()]; let mut part_sizes: Vec<usize> = vec![0]; let push_to_part = |text: &str, parts: &mut Vec<String>, part_sizes: &mut Vec<usize>| { let cur = part_sizes.len() - 1; if part_sizes[cur] > 0 && part_sizes[cur] + text.len() > threshold_bytes { parts.push(String::new()); part_sizes.push(0); } let cur = parts.len() - 1; parts[cur].push_str(text); part_sizes[cur] += text.len(); }; for (unit, kind) in raw_units.iter().zip(kinds.into_iter()) { match kind { UnitKind::HeaderVerbatim => { common_h.push_str(unit); } UnitKind::DeclOnly { name } => { // Dedup: drop if a definition with the same name exists // anywhere — the authoritative decl is auto-generated from // that definition instead (see below). let superseded = name.as_deref().map_or(false, |n| defined_names.contains(n)); if !superseded { common_h.push_str(unit); } } UnitKind::FnDef { proto, .. } => { common_h.push_str(&proto); common_h.push('\n'); push_to_part(unit, &mut parts, &mut part_sizes); } UnitKind::GlobalDef { extern_decl, .. } => { common_h.push_str(&extern_decl); common_h.push('\n'); push_to_part(unit, &mut parts, &mut part_sizes); } UnitKind::CondBlockWithDef { header_mirror } => { common_h.push_str(&header_mirror); push_to_part(unit, &mut parts, &mut part_sizes); } UnitKind::KnownPartOnlyMacro => { push_to_part(unit, &mut parts, &mut part_sizes); } } } common_h.push_str(&format!("#endif /* {} */\n", guard)); let include_line = format!("#include \"{}_common.h\"\n", cu_name); let parts: Vec<String> = parts.into_iter() .map(|body| format!("{}{}", include_line, body)) .collect(); // Plan 209 Ф.1 (A3), second half of the invariant: every decl-only unit // we DROPPED as superseded must have had its authoritative replacement // actually emitted into `_common.h` (a prototype/extern derived from // the matching definition) — i.e. no call site loses its declaration. // By construction every `FnDef`/`GlobalDef` unconditionally pushes its // `proto`/`extern_decl` into `common_h` above, so this holds trivially; // assert it defensively in case that invariant is ever weakened. for name in &defined_names { debug_assert!( common_h.contains(name.as_str()), "split_tu (A3): definition `{}` has no corresponding declaration text in _common.h", name ); } Ok(SplitResult { common_h, parts }) } fn sanitize_guard(cu_name: &str) -> String { cu_name.chars() .map(|c| if c.is_ascii_alphanumeric() { c.to_ascii_uppercase() } else { '_' }) .collect() } #[cfg(test)] mod tests { use super::*; fn assert_contiguous(src: &str) { let units = segment_top_level(src); let joined: String = units.concat(); assert_eq!(joined, src, "segments must reproduce the input exactly"); } #[test] fn segment_reproduces_input_simple() { let src = "typedef int Foo;\nint bar(void) { return 1; }\nint g = 5;\n"; assert_contiguous(src); let units = segment_top_level(src); assert_eq!(units.len(), 3); } #[test] fn segment_handles_strings_and_comments_with_braces() { let src = "const char* s = \"a{b}c\"; // comment with { brace\nint f(void) { /* { nested-looking } */ return 0; }\n"; assert_contiguous(src); let units = segment_top_level(src); assert_eq!(units.len(), 2); assert!(units[0].contains("a{b}c")); } #[test] fn classify_fn_def_extracts_name_and_proto() { let unit = "int foo(int x, char* y) {\n return x;\n}"; match classify_unit(unit) { UnitKind::FnDef { name, proto } => { assert_eq!(name, "foo"); assert_eq!(proto, "int foo(int x, char* y);"); } other => panic!("expected FnDef, got {:?}", other), } } #[test] fn classify_global_def_extracts_name_and_extern() { // Plan 209 Ф.3: no initializer -> a C tentative definition, no // top-level '='. This shape (declared-but-uninitialized global, // real example: `CEmitter::emit_lazy_const`'s // `nova_int _nova_const_X_value;` storage cell, assigned later at // runtime by `nova_consts_init()`) must be treated as a definition // needing a single home + `extern` in `_common.h`, matching // recon-notes §4's "глобал с ОПРЕДЕЛЕНИЕМ" invariant — same as an // initialized global. Before this fix it fell through to // `DeclOnly { name: None }`, which is unnamed (A3 dedup can never // supersede it) and so was kept VERBATIM in `_common.h`; every part // `#include`ing that header got its own copy of the definition -> // `lld-link: error: duplicate symbol` (observed for real lazy // consts: `_nova_const_ZERO_value`, `_nova_const_lower_map_value`, // `_nova_const_collate_single_map_value`, ...). let unit = "nova_int _nova_const_X_value;"; match classify_unit(unit) { UnitKind::GlobalDef { name, extern_decl } => { assert_eq!(name, "_nova_const_X_value"); assert_eq!(extern_decl, "extern nova_int _nova_const_X_value;"); } other => panic!("expected GlobalDef, got {:?}", other), } let unit2 = "NovaVtable_Fail_X* _nova_handler_Fail_X = NULL;"; match classify_unit(unit2) { UnitKind::GlobalDef { name, extern_decl } => { assert_eq!(name, "_nova_handler_Fail_X"); assert_eq!(extern_decl, "extern NovaVtable_Fail_X* _nova_handler_Fail_X;"); } other => panic!("expected GlobalDef, got {:?}", other), } } #[test] fn classify_static_inline_fn_def_stays_header_verbatim_whole() { // Plan 209 Ф.3 regression (real generated `.c`, `app_effect_basic_t8_1.nv`): // the two permanent `static inline` exceptions // (`nova_typeid_user_name`, per-E throw fast-path // `_nova_throw_typed_<m>`) are function DEFINITIONS with a body — // ordinary `FnDef` classification would split them into a bare // prototype (`_common.h`) + body (one part), which is correct for a // plain external definition but WRONG for `inline`: a `static` // function with only a prototype visible in a TU has no body THERE // to call, so every part other than the one holding the body hit // `lld-link: error: undefined symbol`. The whole unit (signature + // body) must stay verbatim in the header instead — safe to // duplicate per-TU exactly like any other `static inline` helper. let unit = "static inline nova_unit _nova_throw_typed_WorkErr1(Nova_WorkErr1* payload) {\n return NOVA_UNIT;\n}\n"; match classify_unit(unit) { UnitKind::HeaderVerbatim => {} other => panic!("expected HeaderVerbatim (whole inline def kept together), got {:?}", other), } let r = split_tu(unit, "cu", 1 << 20).expect("split_tu should succeed"); assert!(r.common_h.contains("static inline nova_unit _nova_throw_typed_WorkErr1")); assert!(r.common_h.contains("return NOVA_UNIT;"), "the header must carry the FULL body, not just a prototype"); assert!(r.parts.iter().all(|p| !p.contains("_nova_throw_typed_WorkErr1")), "an inline def must not ALSO be duplicated into a part"); } #[test] fn classify_typedef_with_leading_comment_is_not_promoted_to_global_def() { // Plan 209 Ф.3 regression (real generated `.c`, `e2e_collate.nv`): // `typedef int64_t Nova_RawMem;` preceded by a doc comment (`/* // Plan 36: forward decls для user types — нужны для NovaOpt_<T> */`) // is a braceless, `=`-free, paren-free unit — exactly the shape // `decl_from_uninitialized_global` targets. A naive // `trimmed.starts_with("typedef ")` check misses the keyword // because the comment comes first, so this unit was wrongly // promoted to `GlobalDef` with `extern ` prefixed onto the WHOLE // unit (comment included) — malformed C // (`extern /* ... */\ntypedef int64_t Nova_RawMem;`) that failed to // compile ("cannot combine with previous 'extern' declaration // specifier"). Must stay a plain (unpromoted) declaration — kept // verbatim in the header, safe to duplicate across every part — // NOT `GlobalDef` (which would prepend `extern `). let unit = "/* Plan 36: forward decls для user types — нужны для NovaOpt_<T> */\ntypedef int64_t Nova_RawMem;\n"; match classify_unit(unit) { UnitKind::DeclOnly { .. } => {} other => panic!("expected DeclOnly (unpromoted typedef), got {:?}", other), } } #[test] fn classify_struct_typedef_with_braces_is_header_verbatim() { let unit = "typedef struct NovaOpt_X { int tag; nova_int value; } NovaOpt_X;\n"; match classify_unit(unit) { UnitKind::HeaderVerbatim => {} other => panic!("expected HeaderVerbatim, got {:?}", other), } } #[test] fn classify_global_with_brace_initializer_is_global_def() { let unit = "static const NovaTypeInfo NOVA_TYPEINFO_X = { 42, \"X\" };\n"; // Note: at this point `static` would already have been stripped by // A1 when multi-TU is on; classify_unit doesn't care either way — // it just needs a top-level '=' before the first '{'. match classify_unit(unit) { UnitKind::GlobalDef { name, extern_decl } => { assert_eq!(name, "NOVA_TYPEINFO_X"); assert!(extern_decl.starts_with("extern ")); assert!(extern_decl.contains("NOVA_TYPEINFO_X")); assert!(!extern_decl.contains('{')); } other => panic!("expected GlobalDef, got {:?}", other), } } #[test] fn known_macro_statement_is_part_only() { assert!(is_known_part_only_macro("NOVA_BENCH_STATE_DEFINE;\n")); assert!(is_known_part_only_macro(" NOVA_BENCH_HEAP_SAMPLER_THREAD_DEFINE\n")); assert!(!is_known_part_only_macro("int foo(void);\n")); } #[test] fn cond_block_without_definition_is_header_verbatim() { let block = "#ifdef _MSC_VER\ntypedef int Foo;\n#else\ntypedef long Foo;\n#endif\n"; match classify_unit(block) { UnitKind::HeaderVerbatim => {} other => panic!("expected HeaderVerbatim, got {:?}", other), } } #[test] fn cond_block_with_definition_mirrors_and_stays_atomic() { let block = "#ifdef _MSC_VER\n__declspec(thread) NovaVtable_Fail_X* _nova_handler_Fail_X = NULL;\n#else\n__thread NovaVtable_Fail_X* _nova_handler_Fail_X = NULL;\n#endif\n"; match classify_unit(block) { UnitKind::CondBlockWithDef { header_mirror } => { assert!(header_mirror.contains("#ifdef _MSC_VER")); assert!(header_mirror.contains("#else")); assert!(header_mirror.contains("#endif")); assert!(header_mirror.contains("extern __declspec(thread) NovaVtable_Fail_X* _nova_handler_Fail_X;")); assert!(header_mirror.contains("extern __thread NovaVtable_Fail_X* _nova_handler_Fail_X;")); assert!(!header_mirror.contains("NULL")); } other => panic!("expected CondBlockWithDef, got {:?}", other), } } #[test] fn nested_cond_block_endif_does_not_close_outer() { let src = "#ifdef A\n#ifdef B\nint x(void) { return 1; }\n#endif\n#endif\nint y(void) { return 2; }\n"; let units = segment_top_level(src); // First unit = the whole outer #ifdef..#endif (nested included); // second unit = `int y(void) { return 2; }`. assert_eq!(units.len(), 2); assert!(units[0].starts_with("#ifdef A")); assert!(units[0].trim_end().ends_with("#endif")); assert_eq!(units[0].matches("#endif").count(), 2); assert!(units[1].contains("int y")); } #[test] fn split_tu_default_shape_common_h_and_one_part() { let src = "/* nova-effect-count: 3 */\n#include \"nova_rt/nova_rt.h\"\ntypedef int Foo;\nint foo(void) { return 1; }\nint bar(void) { return foo(); }\n"; let r = split_tu(src, "cu", 1 << 20).expect("split_tu should succeed for these fixtures"); assert!(r.common_h.starts_with("/* nova-effect-count: 3 */\n")); assert!(r.common_h.contains("#include \"nova_rt/nova_rt.h\"")); assert!(r.common_h.contains("typedef int Foo;")); assert!(r.common_h.contains("int foo(void);")); assert!(r.common_h.contains("int bar(void);")); assert_eq!(r.parts.len(), 1); assert!(r.parts[0].starts_with("#include \"cu_common.h\"\n")); assert!(r.parts[0].contains("int foo(void) { return 1; }")); assert!(r.parts[0].contains("int bar(void) { return foo(); }")); } #[test] fn split_tu_dedup_drops_stale_forward_decl() { // A leftover (unpromoted, or just historical) forward decl for // `foo` should be dropped in favor of the auto-generated prototype // from the actual definition — no duplicate/conflicting proto. let src = "int foo(void);\nint foo(void) { return 1; }\n"; let r = split_tu(src, "cu", 1 << 20).expect("split_tu should succeed for these fixtures"); assert_eq!(r.common_h.matches("int foo(void);").count(), 1); } #[test] fn split_tu_round_robins_by_byte_threshold() { let mut src = String::new(); for i in 0..10 { src.push_str(&format!("int f{}(void) {{ return {}; }}\n", i, i)); } // Each fn body is ~30 bytes; threshold small enough to force >1 part. let r = split_tu(&src, "cu", 80).expect("split_tu should succeed for these fixtures"); assert!(r.parts.len() > 1, "expected multiple parts, got {}", r.parts.len()); // Every definition must appear in exactly one part. for i in 0..10 { let marker = format!("return {}; }}", i); let count = r.parts.iter().filter(|p| p.contains(&marker)).count(); assert_eq!(count, 1, "f{} must appear in exactly one part", i); } } #[test] fn split_tu_never_splits_a_single_definition_across_parts() { let src = "int big(void) {\n int a = 1;\n int b = 2;\n return a + b;\n}\n"; let r = split_tu(src, "cu", 4).expect("split_tu should succeed for these fixtures"); // absurdly small threshold assert_eq!(r.parts.len(), 1); assert!(r.parts[0].contains("return a + b;")); } #[test] fn split_tu_known_macro_goes_to_part_not_header() { let src = "NOVA_BENCH_STATE_DEFINE;\nint main_impl(void) { return 0; }\n"; let r = split_tu(src, "cu", 1 << 20).expect("split_tu should succeed for these fixtures"); assert!(!r.common_h.contains("NOVA_BENCH_STATE_DEFINE")); let found = r.parts.iter().any(|p| p.contains("NOVA_BENCH_STATE_DEFINE")); assert!(found, "NOVA_BENCH_STATE_DEFINE must land in a part"); } #[test] fn split_tu_cond_block_with_def_goes_to_one_part_with_mirror_in_header() { let src = "#ifdef _MSC_VER\n__declspec(thread) NovaVtable_Fail_X* _nova_handler_Fail_X = NULL;\n#else\n__thread NovaVtable_Fail_X* _nova_handler_Fail_X = NULL;\n#endif\nnova_unit throw_it(void) { return NOVA_UNIT; }\n"; let r = split_tu(src, "cu", 1 << 20).expect("split_tu should succeed for these fixtures"); assert!(r.common_h.contains("extern __thread NovaVtable_Fail_X* _nova_handler_Fail_X;")); assert!(!r.common_h.contains("= NULL")); let parts_with_hit: Vec<usize> = r.parts.iter() .map(|p| p.matches("_nova_handler_Fail_X = NULL").count()) .enumerate() .filter(|(_, c)| *c > 0) .map(|(i, _)| i) .collect(); assert_eq!(parts_with_hit.len(), 1, "both branches must land in the SAME single part"); let total_hits: usize = r.parts.iter() .map(|p| p.matches("_nova_handler_Fail_X = NULL").count()) .sum(); assert_eq!(total_hits, 2, "both #ifdef branches of the definition travel together into one part"); } #[test] fn split_tu_global_with_initializer_gets_extern_and_single_definition() { let src = "nova_int _nova_const_FOO_value;\nvoid nova_consts_init(void) { _nova_const_FOO_value = 42; }\n"; let r = split_tu(src, "cu", 1 << 20).expect("split_tu should succeed for these fixtures"); // Plan 209 Ф.3: the uninitialized tentative-definition global must // be `extern`'d in the header (NOT kept verbatim — verbatim is what // produced the `lld-link: error: duplicate symbol` for every lazy // const once a CU split into >1 part, see // `classify_global_def_extracts_name_and_extern` for the full // mechanism doc). assert!(r.common_h.contains("extern nova_int _nova_const_FOO_value;")); assert_eq!(r.common_h.matches("nova_int _nova_const_FOO_value;").count(), 1, "the header must contain ONLY the extern-decl form, not also a bare verbatim tentative definition"); assert!(r.common_h.contains("void nova_consts_init(void);")); let def_occurrences: usize = r.parts.iter() .map(|p| p.matches("nova_int _nova_const_FOO_value;").count()) .sum(); assert_eq!(def_occurrences, 1, "the const storage definition must appear in exactly one part"); } #[test] fn split_tu_lazy_const_storage_single_part_no_duplicate_across_multiple_parts() { // Plan 209 Ф.3 end-to-end regression: real generated shape (multiple // `emit_lazy_const` storage cells + `nova_consts_init` + several // functions big enough to be forced into DIFFERENT parts by a tiny // threshold) — the const storage definitions must land in exactly // one part each and be `extern`'d (never bare-duplicated) in the // header, regardless of how many parts the CU splits into. let filler_a = "a".repeat(60); let filler_b = "b".repeat(60); let src = format!( "nova_int _nova_const_ZERO_value;\n\ nova_int _nova_const_SECOND_value;\n\ void nova_consts_init(void) {{ _nova_const_ZERO_value = 0; _nova_const_SECOND_value = 1; }}\n\ int use_a(void) {{ /* {filler_a} */ return (int)_nova_const_ZERO_value; }}\n\ int use_b(void) {{ /* {filler_b} */ return (int)_nova_const_SECOND_value; }}\n" ); let r = split_tu(&src, "cu", 48).expect("split_tu should succeed"); assert!(r.parts.len() >= 2, "fixture must actually exercise >1 part: {:?}", r.parts); for sym in ["_nova_const_ZERO_value", "_nova_const_SECOND_value"] { assert!(r.common_h.contains(&format!("extern nova_int {};", sym))); let def_occurrences: usize = r.parts.iter() .map(|p| p.matches(&format!("nova_int {};", sym)).count()) .sum(); assert_eq!(def_occurrences, 1, "{} definition must appear in exactly one part (no duplicate symbol)", sym); } } #[test] fn split_tu_a3_rejects_duplicate_top_level_definition_names() { // Two distinct function BODIES sharing one name — the exact shape // A3's uniqueness invariant exists to catch (a missed/incorrect // mangle would let this happen for real; here we just fabricate it // directly to exercise the guard). let src = "int dup(void) { return 1; }\nint dup(void) { return 2; }\n"; let err = split_tu(src, "cu", 1 << 20).expect_err("duplicate definition names must be rejected"); assert!(err.contains("dup"), "error should name the offending symbol: {}", err); } // ---- Plan 209 Ф.2 regressions (found while verifying the toolchain // end-to-end — real generated `.c`, not the synthetic fixtures above, // tripped all three) ---- #[test] fn classify_fn_def_name_ignores_parens_in_leading_comment() { // A doc-comment mentioning a parenthesized call in prose (real // example: "...assigned to _nova_supervisor_decide_fn in main()...") // has ITS OWN '(' before the real signature's. A naive // `sig.find('(')` matched that one first and misnamed the unit // "main" — colliding with the CU's real `int main(...)` and // tripping the A3 uniqueness guard on a perfectly valid program. let unit = "/* see _nova_other_fn in main(). */\nstatic int _nova_supervisor_decide_impl(void* ctx) {\n return 0;\n}"; match classify_unit(unit) { UnitKind::FnDef { name, proto } => { assert_eq!(name, "_nova_supervisor_decide_impl"); assert!(proto.ends_with("_nova_supervisor_decide_impl(void* ctx);")); } other => panic!("expected FnDef named _nova_supervisor_decide_impl, got {:?}", other), } } #[test] fn segment_top_level_recognizes_cond_block_after_blank_lines() { // Every real unit boundary in generated code is preceded by a // blank line (`self.line("")` between constructs). The leading- // whitespace skip before the cond-block/directive check only // skipped ' '/'\t' (not '\n'), so `bytes[k]` still pointed at '\n' // (not '#') and this whole detection silently never fired — a // multi-line `#ifdef ... #else ... #endif` got cut into THREE // mis-segmented pieces at each ';' instead of staying one atomic // unit (observed as "unterminated conditional directive" from the // C compiler once split across parts). let src = "int a(void) { return 1; }\n\n#ifdef _MSC_VER\nint x = 1;\n#else\nint x = 2;\n#endif\n\nint b(void) { return 2; }\n"; assert_contiguous(src); let units = segment_top_level(src); // Exactly 3 units: `a`, the WHOLE cond-block (one unit), `b`. assert_eq!(units.len(), 3, "cond-block must stay ONE unit even preceded by a blank line: {:?}", units); let cond_unit = units[1]; assert!(cond_unit.trim_start().starts_with("#ifdef")); assert!(cond_unit.contains("#else") && cond_unit.contains("#endif"), "the atomic cond-block unit must carry its OWN #else/#endif: {:?}", cond_unit); match classify_unit(cond_unit) { UnitKind::CondBlockWithDef { .. } => {} other => panic!("expected CondBlockWithDef, got {:?}", other), } } #[test] fn classify_array_global_def_strips_brackets_for_name() { // An array-typed global (real example: the interned string literal // byte buffer `static const uint8_t _nova_strlit_<hash>_buf[] = // "...";`) has its LHS end in ']', not the identifier — // `trailing_identifier` (which only skips trailing whitespace/'*') // found no identifier there and returned `None`, so the unit fell // through to `DeclOnly { name: None }`. Unnamed means A3 dedup can // never supersede it, so a SECOND occurrence of the identical text // elsewhere (or, for Ф.2, the header-mirror path) surfaced as // `lld-link: error: duplicate symbol` once the definition (kept // verbatim, not `extern`-ified) got `#include`d by every part. let unit = "const uint8_t _nova_strlit_deadbeef_buf[] = \"hi\";"; match classify_unit(unit) { UnitKind::GlobalDef { name, extern_decl } => { assert_eq!(name, "_nova_strlit_deadbeef_buf"); assert_eq!(extern_decl, "extern const uint8_t _nova_strlit_deadbeef_buf[];"); } other => panic!("expected GlobalDef named _nova_strlit_deadbeef_buf, got {:?}", other), } // Multi-dimensional array shape too (defensive — not observed in // practice, but the bracket-stripper must handle it uniformly). let unit2 = "int grid[4][8] = {0};"; match classify_unit(unit2) { UnitKind::GlobalDef { name, .. } => assert_eq!(name, "grid"), other => panic!("expected GlobalDef named grid, got {:?}", other), } } #[test] fn split_tu_array_global_dedups_and_does_not_duplicate_across_parts() { // End-to-end (not just classify_unit): an array global referenced // from TWO definitions big enough to land in different parts must // appear as a real definition in EXACTLY ONE part, `extern`-ified // (only) in `_common.h` — never verbatim-duplicated into the // header, which is what produced the Ф.2 link-time "duplicate // symbol" failure. let filler_a = "b".repeat(40); let filler_b = "c".repeat(40); let src = format!( "const uint8_t _nova_strlit_x_buf[] = \"hi\";\n\ int use_a(void) {{ /* {filler_a} */ return (int)_nova_strlit_x_buf[0]; }}\n\ int use_b(void) {{ /* {filler_b} */ return (int)_nova_strlit_x_buf[0]; }}\n" ); // Tiny per-part threshold forces `use_a`/`use_b` into separate parts. let r = split_tu(&src, "cu", 32).expect("split_tu should succeed"); assert!(r.parts.len() >= 2, "fixture must actually exercise >1 part: {:?}", r.parts); let def_occurrences: usize = r.parts.iter() .map(|p| p.matches("_nova_strlit_x_buf[] = \"hi\"").count()) .sum(); assert_eq!(def_occurrences, 1, "the array global's definition must appear in exactly one part"); assert!(r.common_h.contains("extern const uint8_t _nova_strlit_x_buf[];")); assert!(!r.common_h.contains("\"hi\""), "the header mirror must never carry the initializer"); } }