Muundo — Design Decisions¶
Deliberate "no"s — things that look like missing features but are refusals on purpose. Each one has a specific reason static analysis cannot do better without either executing user code or building a full language-semantics engine, which Muundo declines to add.
These are not TODOs or deferred work. Do not silently patch them with heuristics. If a real project hits one and needs it fixed, open an issue with a repro so we can implement the proper thing rather than a lossy approximation.
Import resolver — static-only limitations¶
core/src/parser/import_resolver.rs resolves imports per-language at a purely
static level. The following behaviors are out of scope by design.
TypeScript / JavaScript¶
- Wildcard collisions: when multiple
pathsentries match the same target (rare), we try each in order and stop at the first file-system hit. We do not emulate TS's full preference ordering. node_modules/ third-party packages: intentionally out of scope, not a deferral. Structural rules reason about user code; resolving intonode_moduleswould bloat the graph with tens of thousands of library entities for zero value to those rules. Call edges already carry the raw callee name (express.Router,hono.get), which is sufficient for pattern-based rules.extendspackaged in node_modules (e.g."extends": "@tsconfig/node18/tsconfig.json"): we only follow relative/absoluteextendspaths. Package-resolved extends would require traversing node_modules, which we decline per the point above.
Python¶
- Dynamic
__init__.pyre-exports (__all__computed at runtime,globals().update(...), star-imports from namespace packages): impossible to follow statically. We resolvefrom pkg.submod import Xdirectly topkg/submod.py, which works for the vast majority of real code. Star- imports that rely on runtime package structure are not traced. .pthfiles and editable installs: not read. If a project puts its package root behind an editable install only, the resolver will miss it until apyproject.toml/setup.pysurfaces the layout.- Implicit relative imports (Python 2 style): not supported — Python 3
mandates explicit relative (
from .foo import bar), which we handle.
Rust¶
pub usechain depth = 1: we follow one hop throughlib.rs/mod.rs. Deeper hub-chains (mod.rsre-exporting from anothermod.rs) are not chased, to avoid runaway traversal on generated code.- Glob re-exports (
pub use foo::*): not chased. The set of names is only knowable after resolving every item infoo, which inverts the resolver's direction.pub use foo::Baris handled.
Kotlin grammar — depending on tree-sitter-kotlin-ng¶
Kotlin support (core/src/parser/kotlin.rs) depends on the
tree-sitter-kotlin-ng grammar crate. There is no official Kotlin
tree-sitter grammar (JetBrains ships none), so this is the de-facto
standard: maintained by amaanq under the tree-sitter-grammars org
(a core tree-sitter maintainer), MIT, ~1M downloads, 100+ reverse-deps.
It uses the modern tree-sitter-language 0.1 binding, so it builds
against our pinned tree-sitter 0.25 — the ABI blocker that previously
kept Kotlin out (old tree-sitter-kotlin pinned <0.23) is gone.
Accepted risk (low–moderate): single-maintainer/community governance
(not the official tree-sitter/ org), and a future grammar major could
rename node kinds and break parser/kotlin.rs.
Containment: version pinned in Cargo.toml and lock-verified at build
(MUUNDO_LOCK_TS_KOTLIN + verify_versions), so upgrades are deliberate.
parser/kotlin.rs is isolated behind our own node-kind handling; if the
grammar breaks or stalls, Kotlin extraction can be disabled by dropping the
Language::Kotlin dispatch arm without touching any other language. The
failure mode is graceful — unparseable .kt constructs yield ERROR nodes
(fewer entities), and Jagora testmap falls back to heuristic COVERS; no
crash, no cross-language impact.
Analysis resource budgets — bounded by default, env-overridable¶
Analysis caps its workload on purpose so a hostile or accidentally huge tree
cannot exhaust memory. collect_source_files() checks file metadata before
reading/parsing: a file larger than max_file_size is skipped (recorded in
skipped_files with an explicit reason and the walk continues), while breaching
max_total_bytes or max_file_count is a hard Error::LimitExceeded —
the server maps it to 413 Payload Too Large, not a generic 500. Parsing
retains decoded source_text only (the raw source_bytes are dropped after the
tree-sitter parse; the BLAKE3 hash is computed before the decode).
The limits live on AnalyzeOptions with named-const defaults in
core/src/analyzer.rs; the server reads the env overrides at startup.
Per-caller path (Python binding / CLI). The env vars above are read by the
server/CLI startup, not by every caller. The Python MuundoAnalyzer
constructor takes the limits as optional keyword args —
MuundoAnalyzer(root, languages=…, ts_stateflow_strategy=…, max_total_bytes=…,
max_file_count=…, max_file_size=…) — each defaulting to the engine const. A host
raises the ceiling for a large monorepo, or lowers it to fail fast in CI. (Before
this, the binding hardcoded AnalyzeOptions::default(), so it silently ignored
MUUNDO_MAX_TOTAL_BYTES / MUUNDO_MAX_FILE_COUNT and a >512 MiB repo just
errored with no recourse from Python.) A byte/file-count breach stays a hard
LimitExceeded on this path too — a global resource ceiling truncates to a
non-deterministic surviving subset, so it is refused (fail-closed), never served
as a partial report + PartialAnalysisNote. Only localized, bounded
partiality (state-flow-pairs truncation, doc-coverage degeneracy, a malformed
tsconfig, a filesystem walk error) degrades-with-note.
| Env var | Default const | Default value |
|---|---|---|
MUUNDO_MAX_FILE_SIZE |
DEFAULT_MAX_FILE_SIZE |
2 MiB |
MUUNDO_MAX_TOTAL_BYTES |
DEFAULT_MAX_TOTAL_BYTES |
512 MiB |
MUUNDO_MAX_FILE_COUNT |
DEFAULT_MAX_FILE_COUNT |
50 000 |
MUUNDO_ANALYZE_MAX_CONCURRENT |
— | 4 (concurrent /analyze requests) |
MUUNDO_ANALYZE_TIMEOUT_SECS |
— | 60 (per-request deadline) |
MUUNDO_MAX_SERVER_BYTES |
DEFAULT_MAX_SERVER_BYTES |
4 GiB (server-wide memory budget) |
MUUNDO_RSS_AMPLIFICATION |
DEFAULT_RSS_AMPLIFICATION |
160 (retained bytes / source byte) |
Server memory admission — weighted, not just a request count¶
max_total_bytes bounds one request's source bytes, but tree-sitter trees plus
the extracted entity/dependency/call-graph tables amplify that many-fold. Measured
peak RSS (core/benches/peak_rss.rs, VmHWM via /proc/self/status) on dense
synthetic input (one call-heavy function per few lines — a near-worst case):
| source | entities | call edges | peak RSS | amplification |
|---|---|---|---|---|
| 1.9 MiB | 32 800 | 32 000 | 298 MiB | ~155× |
| 3.9 MiB | 65 600 | 64 000 | 594 MiB | ~154× |
| 3.9 MiB × 4 concurrent | — | — | 2 370 MiB | ~linear in concurrency |
So a fixed request-count semaphore alone is unsafe: 4 concurrent 512 MiB inputs
would retain ~4 × 80 GiB. The server therefore adds weighted memory admission
(server/src/main.rs): a semaphore holding MUUNDO_MAX_SERVER_BYTES worth of
MiB permits, from which each request reserves
ceil(max_total_bytes × MUUNDO_RSS_AMPLIFICATION / MiB). The permit is moved into
the blocking closure, so a detached (timed-out) analysis keeps holding its share
until it truly winds down. This bounds the sum of concurrent retained
footprints to the global budget regardless of the count cap.
The memory permit covers analysis and JSON serialization (peak = report + encoding), both of which run inside the blocking task — and it is released at blocking-thread exit. It does NOT extend through transmission: once the report is spooled to its temp file, the in-memory report is dropped and RAM holds at most one streamed chunk, so keeping the weighted permit through streaming would reserve memory the response no longer uses (starving admission for no benefit). The spooled bytes are guarded instead by the separate aggregate spool budget (previous section), whose reservation lives inside the response body's stream state and is released when the body is fully sent, times out, or the client disconnects.
To keep a single request safe too, the per-request max_total_bytes is reduced
at startup to MUUNDO_MAX_SERVER_BYTES / amplification / max_concurrent when the
configured cap exceeds it (logged as a warning). Safe defaults — 4 GiB budget,
160× amplification, 4-way concurrency — yield an effective source cap of
≈ 6.4 MiB, so four concurrent worst-case analyses fit ~4 GiB. Operators on larger
hosts raise MUUNDO_MAX_SERVER_BYTES (and may then raise MUUNDO_MAX_TOTAL_BYTES);
those who have measured a gentler amplification on their corpus lower
MUUNDO_RSS_AMPLIFICATION to admit larger inputs. The arithmetic
(resolve_memory_admission) is unit-tested for the fit-within-budget and
never-below-1 invariants; the semaphore behaviour (exactly budget/weight
concurrent holders) has its own test.
Spool admission — aggregate temporary-disk budget for streamed responses¶
Each /analyze response is serialized incrementally to a private temp file
(the spool) and streamed from it in bounded chunks. MUUNDO_MAX_ENCODED_REPORT_BYTES
caps ONE response, but N concurrent responses could each claim that allowance —
and the temp filesystem is often tmpfs, i.e. RAM. The server therefore adds a
server-wide spool budget (MUUNDO_MAX_SPOOL_BYTES, default = half the
server memory budget, resolved and clamped at startup — see below): a CAS-guarded
byte pool from which every response's CountingWriter
reserves as the encoder grows the file (never after the fact), released in
full when the spooled response is dropped — fully sent, timed out, client
disconnected, or failed mid-encode. Exhaustion aborts the serialization with a
retryable 503 ("report spool capacity unavailable"), the same capacity class
as the admission gates, never a partial spool. Concurrency behaviour (at most
capacity / size concurrent spools, exact release, no overshoot even under
racing reservations) is unit-tested.
Startup resolution — the spool must not silently defeat the memory ceiling.
A fixed spool default (say, 8 GiB) is dangerous precisely because the temp
filesystem is often tmpfs: those bytes are RAM the weighted memory semaphore does
not account for (the analysis permit is released the moment the report spools
to disk — see the previous section). An 8 GiB spool on a 4 GiB-budget box could
therefore push real process RAM to ~12 GiB while every in-process accounting
number stayed "within budget". resolve_spool_budget (unit-tested, pure) closes
this at startup:
- The default is a fraction of the server budget, not a constant:
MUUNDO_MAX_SERVER_BYTES / SPOOL_BUDGET_SERVER_DIVISOR(÷2 → half). It scales with the operator's configured budget instead of dwarfing it. - A memory-backed TMPDIR shares one combined ceiling with analysis. When
statfsreports tmpfs/ramfs, the spool is capped atserver_budget / 2and main() subtracts the resolved spool from the analysis budget fed toresolve_memory_admission. Soanalysis_RAM + spool_RAM ≤ MUUNDO_MAX_SERVER_BYTESholds by construction. On a real-disk TMPDIR the spool is separate storage and the full server budget stays available to analysis. - The value is clamped to the temp filesystem's real free space (
statvfs, 80% headroom), so a full spool can never ENOSPC the temp dir — RAM or disk.
Clamps only ever lower the value and are logged with the reason. The capacity
probe is best-effort: a probe failure (or a non-Unix target) skips the
filesystem clamp but keeps the server-fraction default. The startup arithmetic
and the combined analysis + spool ≤ budget invariant are unit-tested alongside
resolve_memory_admission.
Entity identity — overloaded methods stay merged (semantics belong to the consumer)¶
qualified_name is file::Scope::…::name, built from purely syntactic
facts (tree-sitter AST). It distinguishes entities by lexical scope (nested
functions/classes no longer collide — see the scope-chain work), but two
overloaded methods in one class — same name, different signatures — share
one qualified_name and are therefore MERGED into a single node (one metrics
entry, one call-graph node).
We deliberately do not discriminate overloads inside muundo. The reason is a boundary decision, not a limitation to "fix later":
- Muundo is a syntactic extractor. It does the ~90 %: every entity with its
line_number/end_line, and (cheaply, if a consumer needs it) aparam_count. These are all facts the AST gives directly. - Overload disambiguation is a semantic concern. Routing a call
foo(x)tofoo(int)vsfoo(String)requires knowing the static type ofx— i.e. a per-language type-checker (arity only covers the different-arity subset). That knowledge lives in the consuming application (e.g. Jagora), not in a tree-sitter extractor. - Forcing it into muundo would be net-negative. Making overload
qualified_names distinct is not a local change: the mapmetrics_by_entityis keyed byqualified_name, so distinct keys immediately break call-edge resolution (by_name["foo"]becomes ambiguous → the edge is lost) unless arity/arg-count is also threaded through the caller attribution, the resolver, and a newEntityfield — a change comparable in scope to the whole scope-chain refactor, for a rare case, and one that still can't resolve same-arity/different-type overloads without types.
The contract, therefore: muundo surfaces the raw syntactic facts (entity
qualified_name, line_number, end_line, and — when the need is real — an
additive param_count); a consumer that cares about overloads groups the
same-qualified_name methods and disambiguates them for its purpose, where
the type/context knowledge already exists. Muundo does not change its identity,
resolution, or metric keying to carry semantics it cannot soundly compute.
No is_sanitizer flag — sanitiser knowledge belongs to the taint-tracker¶
The same boundary retired a field that used to violate it. muundo once emitted
Entity.is_sanitizer: bool, set by matching the entity name against a list of
sanitiser verbs. This was a semantic judgment wearing a syntactic costume,
and it was wrong on two counts:
- muundo cannot justify it. Whether a function actually clears taint — and
for which sink family (
html.escapeneutralises an XSS context but not a SQL sink;normpathneeds a following containment check) — is domain knowledge applied where taint flows: at the call site, in the consumer's taint tracker. muundo classifies definitions by name and has no call-site type or sink context, so a userdef cleanwas indistinguishable frombleach.clean. - A context-free bool, read downstream, suppresses findings it can't warrant. A consumer that trusts the flag clears taint the flag can't justify — a false negative, the dangerous direction.
The sink-aware value already lives in the consumer (jagora's _SANITIZER_CAPS,
a name → {sink families} map keyed on the callee tail), which satisfies the
"carry the sink context" guarantee at the layer that has the context. muundo's
bool was therefore redundant and unsound, so it is gone — muundo emits
only the syntactic facts (entities, calls, edges). Recognising a sanitiser, and
scoping what it clears, is the consumer's job. Same rule as overloads: muundo
does not carry semantics it cannot soundly compute.