Architecture overview

Module map, transports, and the zim/ package layout.

Notation: examples on this page use Python pseudo-call syntax for tool calls. The MCP wire format is JSON-RPC; your client handles the framing. Tool names match the 8-tool advanced surface (zim_query, zim_search, zim_get, zim_get_section, zim_browse, zim_metadata, zim_links, zim_health).

Source of truth: openzim_mcp/. CONTRIBUTING.md’s Project Structure section sketches the on-disk layout — this page explains why it’s organized that way.

High-level layers#

┌──────────────────────────────────────────────────────────────────┐
│                      MCP Client Layer                            │
│         (Claude Desktop, Claude Code, Inspector, custom)         │
└─────────────────────┬───────────────────────┬────────────────────┘
                      │ stdio                 │ streamable HTTP / SSE
┌─────────────────────▼───────────────────────▼────────────────────┐
│                     OpenZIM MCP Server                           │
│  ┌────────────┐ ┌──────────────┐ ┌────────────┐ ┌────────────┐   │
│  │   Server   │ │  HTTP/SSE    │ │  Subscrip- │ │  Security  │   │
│  │   core     │ │  transport   │ │  tions     │ │   layer    │   │
│  │ (server.py)│ │(http_app.py) │ │(subscrip…) │ │(security)  │   │
│  └────────────┘ └──────────────┘ └────────────┘ └────────────┘   │
└─────────────────────┬────────────────────────────────────────────┘
                      │
┌─────────────────────▼────────────────────────────────────────────┐
│              Tool Surface (advanced=8, simple=1)                 │
│  tools/zim_{query,search,get,get_section,browse,                 │
│             metadata,links,health}.py, resource_tools, prompts   │
└─────────────────────┬────────────────────────────────────────────┘
                      │
┌─────────────────────▼────────────────────────────────────────────┐
│                       Business Logic                             │
│  ┌────────────┐ ┌──────────────┐ ┌──────────────────────────┐    │
│  │   Cache    │ │   Content    │ │   ZIM operations         │    │
│  │  (LRU+TTL) │ │  processor   │ │   (zim/ package, mixin)  │    │
│  │ (cache.py) │ │              │ │                          │    │
│  └────────────┘ └──────────────┘ └──────────────────────────┘    │
│  ┌────────────┐ ┌──────────────┐ ┌──────────────────────────┐    │
│  │   Async    │ │ Rate limiter │ │ Intent parser / simple   │    │
│  │ operations │ │              │ │ tools (NL routing)       │    │
│  └────────────┘ └──────────────┘ └──────────────────────────┘    │
└─────────────────────┬────────────────────────────────────────────┘
                      │
┌─────────────────────▼────────────────────────────────────────────┐
│                    Data Access Layer                             │
│  ┌────────────┐ ┌──────────────┐ ┌──────────────────────────┐    │
│  │   libzim   │ │  filesystem  │ │   Config & validation    │    │
│  │ (Archive,  │ │  (allowed    │ │   (config.py + pydantic) │    │
│  │  Searcher) │ │   dirs only) │ │                          │    │
│  └────────────┘ └──────────────┘ └──────────────────────────┘    │
└──────────────────────────────────────────────────────────────────┘

The 8-tool consolidated surface#

v2.0.0 collapsed the 22-tool v1 advanced surface into 8 tools. Three of them take a mode or view parameter that selects the prior v1 behavior; the rest consolidated by absorbing sibling tools into one signature, or by switching on which arguments are set:

                                ┌──────────────────────────────────┐
                                │  zim_query (Simple-mode default) │
                                │  • NL routing → all 7 below      │
                                └──────────────────────────────────┘

  ┌──────────────────────────────┐  ┌──────────────────────────────┐
  │  zim_search                  │  │  zim_get                     │
  │  • mode="fulltext"  (default)│  │  • entry_path=...            │
  │  • mode="title"              │  │  • entry_paths=[...]  batch  │
  │  • mode="suggest"            │  │  • binary=True   binary blob │
  │  • cross_file=True  fan-out  │  │  • main_page=True            │
  │  • namespace= / content_type=│  │  • view="summary"            │
  │                              │  │  • view="toc"                │
  │                              │  │  • view="structure"          │
  └──────────────────────────────┘  └──────────────────────────────┘

  ┌──────────────────────────────┐  ┌──────────────────────────────┐
  │  zim_get_section             │  │  zim_browse                  │
  │  • section_id=...            │  │  • mode="page"    (default)  │
  │  • from a TOC node           │  │  • mode="walk"    deep tree  │
  └──────────────────────────────┘  └──────────────────────────────┘

  ┌──────────────────────────────┐  ┌──────────────────────────────┐
  │  zim_links                   │  │  zim_metadata                │
  │  • direction="outbound"      │  │  • metadata + namespace      │
  │  • "inbound" / "related"     │  │    counts in one payload     │
  └──────────────────────────────┘  └──────────────────────────────┘

  ┌──────────────────────────────┐
  │  zim_health                  │
  │  • no arg → health + config  │
  │    + archives (one payload)  │
  │  • path → archive validation │
  └──────────────────────────────┘

Each call-out under a consolidated tool corresponds to a v1 operation that was folded in. The detailed mapping lives in the migration table on the API reference page.

Module structure#

Abridged — retrieval-pipeline helpers (bundle.py, compact_format.py, synthesize.py, rerank.py, …) and the cli/, linkgraph/, ml/, and data/ subpackages are omitted:

openzim_mcp/
├── __init__.py
├── __main__.py              # `python -m openzim_mcp` — delegates to main.py
├── main.py                  # CLI entry — reads --mode/--transport/--host/--port
├── server.py                # OpenZimMcpServer + DI wiring + tool registration
├── http_app.py              # Streamable-HTTP / SSE Starlette app, auth, CORS, /healthz, /readyz
├── subscriptions.py         # MtimeWatcher + publish_change onto the SDK's SubscriptionBus
├── simple_tools.py          # zim_query NL handler (Simple mode)
├── intent_parser.py         # NL intent → underlying-tool routing
├── config.py                # OpenZimMcpConfig (pydantic-settings)
├── defaults.py              # Frozen-dataclass defaults referenced by config.py
├── cache.py                 # LRU+TTL cache (with optional disk persistence)
├── rate_limiter.py          # Token-bucket limiter, per-client + per-operation
├── content_processor.py     # HTML→text, summary extraction, link filtering
├── security.py              # PathValidator, sanitize_input, redact_paths_in_message
├── async_operations.py      # asyncio.to_thread wrappers around blocking ZIM I/O
├── timeout_utils.py         # bounded-time helpers
├── error_messages.py        # markdown error templates
├── exceptions.py            # OpenZimMcp* exception hierarchy
├── constants.py             # cross-module constants (input limits, thresholds)
├── tool_schemas.py          # per-tool response TypedDicts (tools annotate -> Any, no outputSchema)
├── responses.py             # tool_error() + ToolErrorPayload: {error, operation, message, context?} + extras
├── mcp_envelope.py          # EnvelopeAwareMCPServer — returned error envelope -> isError=True
├── zim_operations.py        # back-compat shim — re-exports from zim/
├── zim/                     # ZIM operations package
│   ├── __init__.py
│   ├── _ops_base.py         # Shared mixin helpers (path validation, JSON serializer)
│   ├── archive.py           # ZimOperations(_ArchiveAccessMixin, _SearchMixin, _ContentMixin, _StructureMixin, _NamespaceMixin)
│   ├── search.py            # _SearchMixin
│   ├── content.py           # _ContentMixin
│   ├── structure.py         # _StructureMixin
│   ├── namespace.py         # _NamespaceMixin
│   └── redirects.py         # Strict redirect-chain resolver (raises on cycles)
└── tools/                   # MCP tool registration — one module per v2 tool
    ├── __init__.py          # register_phase_f_tools() — tool_mode gate; simple stops after zim_query
    ├── _common.py           # shared wrapper helpers: description loader, rate limit, cursors, errors
    ├── zim_query.py         # zim_query — NL entry point via SimpleToolsHandler (both tool modes)
    ├── zim_search.py        # zim_search — mode="fulltext"|"title"|"suggest", plus cross_file fan-out
    ├── zim_get.py           # zim_get — single/batch/binary/main_page fetch; view=full|summary|toc|structure
    ├── zim_get_section.py   # zim_get_section — one named section (section_id) of an article
    ├── zim_browse.py        # zim_browse — namespace enumeration, mode="page"|"walk"
    ├── zim_metadata.py      # zim_metadata — M-namespace fields + namespace inventory in one payload
    ├── zim_links.py         # zim_links — direction="outbound"|"inbound"|"related", kind filter
    ├── zim_health.py        # zim_health — no arg → health+config+archives; path → archive validation
    ├── resource_tools.py    # zim:// resources + MIME-aware per-entry template (advanced only)
    └── prompts.py           # /research, /summarize, /explore prompts (advanced only)

Each zim_*.py module exports a single register(server) function that declares one v2 tool via @server.mcp.tool(description=...), loads its LLM-facing description from the sibling _description.md file at import time, and forwards to AsyncZimOperations so blocking ZIM I/O runs under asyncio.to_thread — the exception is zim_query, which dispatches SimpleToolsHandler.handle_zim_query on a worker thread instead. register_phase_f_tools(server) registers zim_query unconditionally, then returns early when config.tool_mode == "simple"; the other seven tools, the zim:// resources, and the prompts are advanced-mode only.

Core components#

Server core (server.py)#

OpenZimMcpServer wires the dependency graph:

config            → OpenZimMcpConfig
path_validator    → PathValidator(allowed_directories)
cache             → OpenZimMcpCache(config.cache)
content_processor → ContentProcessor(config.content)
zim_operations    → ZimOperations(config, path_validator, cache, content_processor)
async_zim_ops     → AsyncZimOperations(zim_operations)
rate_limiter      → RateLimiter(config.rate_limit)
subscription_bus  → InMemorySubscriptionBus()  (only when subscriptions_enabled and transport='http')
simple_tools_h    → SimpleToolsHandler(zim_operations)

Tool registration is dispatched from tools/__init__.py:register_phase_f_tools(server). In Simple mode only zim_query is registered, and the other seven tool modules are never even imported — zim_query reaches the business logic through SimpleToolsHandler and ZimOperations directly, not through them. The advanced tools are surfaced only when tool_mode='advanced'.

HTTP / SSE transport (http_app.py)#

Owns streamable-HTTP and legacy-SSE concerns so server.py stays focused on MCP-protocol logic:

• Starlette app with /healthz (liveness — always 200) and /readyz (returns 503 if no allowed directory is readable).
• BearerTokenAuthMiddleware — timing-safe hmac.compare_digest; never logs the attempted token; OPTIONS is not exempt; /healthz//readyz are.
• CORS — explicit allow-list via apply_cors_middleware; wildcard * rejected at config-load time. The header allow-list is the union of both protocol eras the endpoint serves: Mcp-Session-Id and Last-Event-ID are kept for browser-based handshake-era clients (2026-07-28 removed sessions and stream resumption), while Mcp-Method and Mcp-Name are what a 2026-07-28 client must send on every POST.
• check_safe_startup — refuses to bind a non-loopback host without an auth token (HTTP) or any non-loopback host (SSE has no auth middleware).
• Lifespan wrapping — the subscription watcher is started/stopped via a wrapper around the SDK’s own lifespan context (streamable_http_app() supplies its own custom lifespan, which means add_event_handler('startup', …) is silently a no-op).

See HTTP and Docker deployment for operator-level guidance.

Subscriptions (subscriptions.py)#

Polling-based mtime watcher, plus the mapping from its change kinds onto MCP notifications. Delivery belongs to the SDK: clients opt in with subscriptions/listen, and the SDK’s ListenHandler owns the listener registry, per-stream filtering, backpressure, and teardown.

• MtimeWatcher — periodically scans every allowed directory (recursively); emits change events on file add/remove and on .zim mtime or size change (catches same-size replacement via mtime, and same-mtime rewrites via size).
• publish_change — maps the two change kinds onto different notifications: an add/remove changes the membership of zim://files, so it publishes ResourcesListChanged (notifications/resources/list_changed); an in-place replacement invalidates one resource, so it publishes ResourceUpdated for zim://{name} (notifications/resources/updated).
• Tunable via OPENZIM_MCP_WATCH_INTERVAL_SECONDS (default 5, range 1-60); disable with OPENZIM_MCP_SUBSCRIPTIONS_ENABLED=false.
• Publishing is HTTP-only: server.py builds an InMemorySubscriptionBus only for transport='http', and the watcher runs under the HTTP lifespan. The SDK registers subscriptions/listen unconditionally (falling back to a bus of its own), so server.py deregisters it whenever the bus is withheld — on stdio and SSE the capability is not advertised and a listen request fails with method-not-found instead of acking a stream that would never receive anything.
• Admission control is split. The SDK’s ListenHandler caps concurrent listen streams and per-stream buffered events, ending a stream that outruns its consumer. It leaves the requested URI set unbounded, so BoundedListenHandler in subscriptions.py subclasses it and rejects an oversized list with INVALID_PARAMS before a subscription slot is taken.

Security layer (security.py)#

• PathValidator.validate_path — regex-based traversal-pattern check + Path.is_relative_to(allowed_dir) containment + canonical resolution.
• PathValidator.validate_zim_file — re-resolves the file and re-checks containment immediately before the archive is opened, closing the TOCTOU window between path validation and the eventual libzim open (defends against symlink swaps), and returns the re-resolved path so the caller opens the inode that was just checked.
• sanitize_input — strips control characters, caps length per input class.
• redact_paths_in_message / sanitize_path_for_error — regex _ABS_PATH_RE redacts absolute paths (cross-platform / and \, wrapped/quoted forms like (/opt/foo) / "/opt/bar" / file=/opt/foo, and URL-decoded forms like %2Fopt%2Fzims). Used in every error message and in diagnostics tools.

Cache (cache.py)#

Single LRU+TTL cache shared across all tools and the smart-retrieval path-mapping store. Backed by an in-memory dict plus a heap for O(log n) LRU eviction (a monotonic access counter, not a timestamp, so coarse platform clock resolution cannot break the ordering); optional disk persistence (off by default) at ~/.cache/openzim-mcp-<config-fingerprint>.json, or an explicit OPENZIM_MCP_CACHE__PERSISTENCE_PATH.

Cache stats: 13 fields — enabled, size, max_size, ancillary_entries, total_entries, size_bytes, max_bytes, ttl_seconds, hits, misses, hit_rate, background_cleanup, persistence_enabled — plus persistence_path and persistence_file_exists when persistence is on. Surfaced inside zim_health under .health.cache_performance; there are no separate management tools. Restarting flushes the cache only when CACHE__PERSISTENCE_ENABLED is off; with persistence on it is saved at exit and reloaded at start.

Rate limiter (rate_limiter.py)#

• Token bucket per global RateLimitConfig, with optional per-operation overrides (per_operation_limits dict).
• Per-client bucket map with LRU eviction (10k cap) so client identity scopes the limit.
• Global + per-operation acquire is atomic — single pass over both buckets, no transient over-consumption.
• zim_get(entry_paths=[...]) charges per-entry to prevent batch bypass.

ZIM operations (zim/ package)#

zim_operations.py is a back-compat shim. Real code lives in openzim_mcp/zim/, split by concern:

class ZimOperations(
    _ArchiveAccessMixin, _SearchMixin, _ContentMixin, _StructureMixin, _NamespaceMixin
):
    """Composed via mixins so each domain (search, content, structure,
    namespace) lives in its own file. The class proper holds the
    constructor and the handful of cross-cutting helpers (file listing,
    metadata, main-page lookup, shared entry-resolution fallback)."""

Each mixin uses the same shared services injected into the parent: config, path_validator, cache, content_processor. The shim exists because external callers (and tests) were importing from openzim_mcp.zim_operations for years; the package layout is treated as an implementation detail.

Tools package (tools/)#

One module per tool. Every zim_*.py exports a register(server) function called from tools/__init__.py:register_phase_f_tools. Tools are decorated with @server.mcp.tool() (MCPServer). Each of the seven data tools builds its own AsyncZimOperations(server.zim_operations) inside register() and forwards blocking ZIM I/O through it via asyncio.to_thread, while zim_query dispatches SimpleToolsHandler on a worker thread; server.zim_operations is called directly only for cheap synchronous helpers.

Smart retrieval fallback#

Entry lookup in _ContentMixin runs a fallback sequence when the literal path misses:

1. M/<key> routing — on a new-scheme archive, a metadata path goes to the metadata API instead of the entry API
2. Path-mapping cache check (previously resolved guesses)
3. Direct access, following redirect chains (bounded at depth 10)
4. Alternate-spelling exact probes (percent-decoded, un-rooted) — still get_entry_by_path, not search
5. Search-derived candidate terms (namespace-stripped, underscore ↔ space, URL-decoded variants) queried against libzim’s Searcher via _SearchMixin

Steps 3, 4 and 5 cache the resolved path on success; step 2 only reads that cache, dropping a mapping that has gone stale. The order is the contract: raw spelling before decoded, exact probes before search.

Successful resolutions are memoized in the shared cache so repeat lookups bypass the sequence. See Smart retrieval for the full algorithm and the diagnostics that surface which step matched.

Configuration#

class OpenZimMcpConfig(BaseSettings):
    # Top-level (abridged)
    allowed_directories: List[str]
    server_name: str = "openzim-mcp"
    tool_mode: Literal["advanced", "simple"] = "simple"
    transport: Literal["stdio", "http", "sse"] = "stdio"
    host: str = "127.0.0.1"
    port: int = 8000
    auth_token: Optional[SecretStr] = None
    insecure_disable_auth: bool = False
    cors_origins: List[str] = []
    allowed_hosts: List[str] = []
    watch_interval_seconds: int = 5
    subscriptions_enabled: bool = True
    presets_override_path: Optional[Path] = None

    # Component sub-configs
    cache: CacheConfig
    content: ContentConfig
    logging: LoggingConfig
    rate_limit: RateLimitConfig
    search: SearchConfig
    query_rewrite: QueryRewriteConfig
    synthesize: SynthesizeConfig
    meta: MetaConfig
    ml: MLConfig

There is no SecurityConfig and no InstanceConfig — security policy is hard-coded in security.py and instance tracking was removed in v1.0. See Configuration for every supported field.

Data flow — typical tool call (stdio)#

MCP client         MCPServer              Server core         async_zim_operations         ZimOperations         libzim
    │                  │                        │                          │                       │                  │
    │── tools/call ───▶│                        │                          │                       │                  │
    │                  │── @mcp.tool wrapper ──▶│                          │                       │                  │
    │                  │                        │── rate_limiter.check ───▶│                       │                  │
    │                  │                        │── sanitize_input        ─│                       │                  │
    │                  │                        │── async_op.fn ──────────▶│── to_thread(fn) ─────▶│── Archive.open ─▶│
    │                  │                        │                          │                       │── … ZIM I/O ────▶│
    │                  │                        │◀── ToolResponse / err ───│                       │                  │
    │                  │◀── tool result ────────│                          │                       │                  │
    │◀── result ───────│                        │                          │                       │                  │

Errors anywhere in the chain are returned rather than raised, as a structured ToolErrorPayload — {error: true, operation, message}, plus context when the call site supplies one, plus any self-correction extras merged in (e.g. zim_get_section attaches available_section_ids and closest_match on operation="section_not_found"). There is no status key and no hint key. Both failure paths go through responses.tool_error: the tool wrapper’s catch-all (tools/_common.py) wraps unexpected exceptions, and validation failures build the same envelope in-band at the call site. Because the envelope is returned, EnvelopeAwareMCPServer (mcp_envelope.py) recognises it at call_tool and sets isError on the CallToolResult, with the serialized body byte-identical to the plain dict return. No tool advertises an outputSchema — all eight are annotated -> Any — so nothing arrives in structuredContent; clients parse the JSON text block. Absolute paths inside error messages are redacted to ...filename.zim form before the payload is returned.

Data flow — HTTP transport#

serve_streamable_http(server) mounts MCPServer on a Starlette app, then attaches middleware in LIFO order so CORS is the outer layer and bearer-auth is inner:

client → CORS middleware → BearerTokenAuthMiddleware → MCPServer MCP routes
                              │
                              └── /healthz, /readyz exempt (auth bypass)

Why CORS-outer-than-auth: a 401 from auth must still carry Access-Control-Allow-Origin headers, otherwise browsers see an opaque CORS error instead of “401 unauthorized”.

The subscription watcher is started on app startup via a lifespan_context wrapper (the SDK’s streamable_http_app() supplies its own lifespan, so add_event_handler('startup', …) is silently a no-op).

What was removed before v2.0.0#

• instance_tracker.py (removed in v1.0) — multi-instance conflict tracking, along with its tools diagnose_server_state and resolve_server_conflicts. HTTP server instances now coexist freely.
• Cache management tools (removed in v1.0) — warm_cache, cache_stats, cache_clear. The cache itself remains; a restart flushes it only when CACHE__PERSISTENCE_ENABLED is off (see the cache notes above).
• get_random_entry (removed in v1.0) — exploratory helper that didn’t pull its weight.

v2.0.0 then collapsed the remaining 22 advanced tools into 8 consolidated tools without removing functionality — every v1 call site maps to a v2 call with a mode / view parameter.

Horizontal scaling#

The server is per-process; multiple HTTP instances coexist freely. Standard pattern:

• Run N instances behind a TLS-terminating reverse proxy (Caddy, nginx, traefik).
• Each instance has its own cache (no cross-process coordination).
• Persistent cache on shared storage is not recommended unless you handle concurrent-writer issues at the storage layer — the on-disk cache uses simple file rewrites.
• /healthz and /readyz are reverse-proxy-friendly.

For deployment specifics see HTTP and Docker deployment.

Tuning? Performance optimization. Security model? Security best practices.

The v1.x maintenance window closed when v2.5.0 shipped (2026-06-18); only the current major line is supported — see SECURITY.md for the policy. The CHANGELOG carries the v1 → v2 migration table and the v3.0.0 breaking-changes entry.