WorksComputer Science
Isabelle Pool Server: a pooled verification service for prover training and multi-agent proving
Abstract
Isabelle Pool Server (IPS), a verification service with a pool of warm headless Isabelle sessions behind an HTTP API, with session leasing and reclamation, memory-aware admission control, and a time budget on every call. Its verification interface spans single-step execution, block-level proof checking and whole-theory builds, with checkpoints for cheap branching and backtracking, and a build from scratch as the final arbiter. The service runs as a single container with monitoring and is accessible via a Python client, two MCP servers and a raw HTTP API. In our measurements IPS is level with PIDE MCP, a PIDE frontend for agents, at one worker and ahead of it at eight, at its commit of October 2026 (the tool is now PIDE AI). Once its sessions are up it scales to 32 concurrent workers on one host at 24 times the single-worker rate, and under memory pressure it refuses new sessions rather than losing existing ones. IPS is free software and publicly available at https://github.com/Deep-LCF/Isabelle-Pool-Server
Introduction
Proof assistants are now driven by machines as well as by people, in two ways. In the training scenario, a loop that builds or evaluates an LLM prover verifies thousands of candidate proof steps per hour. Isabelle provers since LISA have done this through PISA-style read-eval-print loops (REPLs), one Isabelle process per worker [7, 8, 9, 12], and two of them name the cost of attempts as the limiting factor [9, 12]. In the multi-agent serving scenario, many coding agents reach the prover over the Model Context Protocol (MCP) [2] and share one host, on which memory decides how many Isabelle sessions can run. Lean has infrastructure for both: Kimina Lean Server pools REPL workers for reinforcement learning [5, 22], and AXLE hosts isolated, strictly checked verification for agents and training alike [23]. Isabelle’s own interfaces assume other clients. Its editors each serve one person, and isabelle server checks whole theories for other tools; both are built on PIDE, the document model of the system [17]. isabelle build checks whole sessions in batch. The interfaces that training loops and agents use come from outside the distribution. The loops use the REPLs above. Agents use one of several MCP servers, which differ in how they reach Isabelle: AutoCorrode’s Isabelle/Q [3] runs inside a person’s Isabelle/jEdit session; PIDE MCP [10], since renamed PIDE AI, drives sessions of its own without an editor; and Isabelle-MCP [24] goes through the language server. Each meets some of the needs below. None, as far as we know, manages the resources of the host that its clients share.
The two scenarios need the same things of a prover service. A call must return within a known time: a training loop bounds every step so that no attempt stalls a worker, and an agent’s harness abandons a tool call that outlasts its timeout. A failed attempt must cost nothing to undo: a loop that branches at a proof state discards most of what it tries, and an agent whose step fails has to get back to where it was. And the answer to whether a proof is complete must be reliable, because a verifier that accepts an unfinished proof corrupts a reward [14, 23] and tells an agent that it has finished when it has not. Where many clients share one host, two more things are needed: each client’s work kept apart from the others’, and a component that decides who gets a session, when it is taken back, and what happens when memory runs out.
IPS is a service of this kind. Like PIDE MCP, it is a direct PIDE frontend: it drives headless sessions through Isabelle/Scala, with no editor between. What it adds is a resource-management layer, which makes those decisions. It keeps a pool of warm sessions, leases them to clients over HTTP, reclaims idle and abandoned leases, and gates admission on memory, with a wall-clock budget on every wait. Because a PIDE session is a document and not a command stream [16], IPS’s tools can roll back a failed attempt, restore a checkpoint exactly and check a chunk’s proofs in parallel; a batch build gives the final verdict. Training harnesses use a Python client; agents use one of two bundled MCP servers, one that verifies submitted proof chunks and one that follows a theory file the agent edits on disk.
IPS is built for these two scenarios. We state what each needs from the service as metrics before reporting on them (§ 3). Our contributions are:
A resource-management layer on a direct PIDE frontend. A pool of leased sessions with memory-gated admission, and on it the tools of a verification service: budgets, rollback of a failed chunk as a unit, checkpoints, a batch build as arbiter, a Python client and two MCP servers (§ 4).
Measurements for both scenarios. For the training scenario: throughput, rollout rate and branch cost on the HOL-Analysis library, with two checks of the implementation, the verdict flags and the bounded step. For the multi-agent serving scenario: memory per session, acquisition latency, behaviour under a memory limit and crash containment. All were taken on one 128 GB host, and PIDE MCP, at commit
dd46315, was measured under the same conditions for memory, throughput, latency and crash containment (§ 5, § 6 and § 7).
A comparison with PIDE MCP at one commit. Where the two direct PIDE frontends differed at the commit we measured, which of the differences are tooling that would transfer, and a feature inventory (§ 7; the full table is in the repository).
An open implementation and a benchmark. The service1 and the harness that drove both tools through one procedure are public, and so are the raw results, in a results repository2 (Appendix § 13).
Three things here are useful to readers who will never run IPS. The sizing numbers of § 6 apply to any shared Isabelle2026 host. The memory cost of treating a whole theory as visible to PIDE, found in PIDE MCP at the measured commit and reproduced on IPS (§ 7), is a finding for any PIDE frontend. And the harness is a benchmark a third connector can plug into; its verdict-flag result, an interactive PIDE verdict matching isabelle build on all 890 checks, is evidence for any frontend that reads its verdict from PIDE, not for IPS alone. § 2 first places IPS among Isabelle connectors; § 8 lists the limits.
Isabelle connectors and where IPS sits
A connector between a program and Isabelle answers three questions, which this section keeps apart:
How the client reaches the server: the server’s interface (REPL, LSP, command line, MCP, HTTP).
How the server reaches Isabelle: through Isabelle/ML alone, through a protocol such as that of
isabelle serveror of the language server, or through the Isabelle/Scala API of PIDE.What the server offers: its tools (state queries, edits, several sessions, reuse of sessions) and its background services (a memory guard, a file watcher).
The second answer fixes an architecture. The third does not: a tool set changes from one commit to the next, and a tool that one server lacks can often be written for it. So we name the version we read of each connector whose tools we describe. For PIDE MCP it is commit dd46315 of 19 October 2026 (§ 7.1).
A PIDE frontend is a program that manages one or several PIDE sessions through PIDE’s Isabelle/Scala API [10]: Isabelle/jEdit, isabelle server, PIDE MCP, IPS. A direct frontend uses PIDE itself, with no editor between; an indirect one goes through an editor’s session. Kappelmann’s own table calls the two “headless PIDE” and “editor-attached PIDE”. We use the other pair, because “headless” is easily read as implying one process, or one session, per agent, which it does not. The pair was suggested to us by Kevin Kappelmann (personal communication, November 2026). Table 1 collects the terms the report uses throughout and says whose definition each is: Wenzel’s, from the Isabelle manuals [18, 19, 20], Kappelmann’s, IPS’s for its API, or this report’s for its experiments. What the training and multi-agent serving scenarios need of a service is set out as metrics in Table 3.
| Term | Meaning | Source |
| heap | the persistent heap image of a logic session, which captures the state of its theories; built once by isabelle build and loaded by every session that starts on it | Wenzel [20], §2 |
| session | an Isabelle/PIDE session with its underlying Isabelle/ML process, started on a heap, as isabelle server’s session_start does | Wenzel [20], §4.4 |
| settled | every command of the node has finished or failed, the node status under which use_theories returns | Wenzel [20], §4.3 |
| batch build | isabelle build of the theory on its heap: the whole-file check, and the final verdict | Wenzel [20], §2 |
| perspective, overlay | the visible range of a node, which decides where PIDE runs its print functions; a query attached to a command without an edit | Wenzel [19], §3.1 |
| node | a unit of the PIDE document: a theory, or an auxiliary file such as an ML file; the preprint’s “document” | Wenzel [17] |
| print function | a function PIDE runs by itself on a checked command that is visible or carries an overlay; Isabelle’s automatic tools are print functions | Wenzel [17] |
| PIDE frontend | a program that manages one or several PIDE sessions through PIDE’s Isabelle/Scala API | Kappelmann [10], §2 |
| chunk | one document edit of one or more consecutive Isar commands, submitted as a unit | this work |
| checkpoint, restore | a recorded version of the node’s document, and the edit that takes its text back to that version | this work |
| lease | one client’s exclusive hold on a session; it expires, so an abandoned session is reclaimed | this work |
| census | one replay of the corpus at one worker; its times are the reference for every slowdown | this work |
| topology A, B | PIDE MCP at commit dd46315 as one process holding several sessions for one client; one such process per session | this work |
Table 1. Terms used throughout the report, and whose definition each is.
| Question | Metric | Measured as |
| Training scenario (§ 5) | ||
| How many verified chunks per minute? | throughput | verify_chunk calls per minute against worker count, with and without session starts |
| How many rollouts per hour? | rollout rate | scripted whole-proof and step-level loops, rollouts and verified steps per hour against worker count |
| Can I branch and backtrack cheaply? | branch cost | restore time against replaying to the same proof state |
| Multi-agent serving scenario (§ 6) | ||
| How many sessions fit in a host? | capacity | memory per session at steady state, by heap; sessions per 100 GB |
| How long until a client gets a session? | acquisition latency | a ready session, a session start and a heap build, separately |
| What happens when memory runs out? | pressure | admission refusals, evictions, kernel kills, recovery time |
| Are agents isolated from each other? | isolation | crash containment: do the other sessions answer? |
| Checks (§ 5.3) | ||
| Do the verdict flags catch what they target? | verdict flags | each flag against the failure it exists to catch, with the batch build as the reference, on the corpus and on broken proofs |
| Can one bad step stall a worker? | bounded step | looping steps returned at their budget; recovery; effect on other workers |
Table 3. Each scenario’s questions, the metric that answers each, and how it is measured. The last group checks two properties of the implementation rather than measuring a scenario.
The connectors in use today are of six kinds. The seventh item below is not a kind of connector but the layer that IPS adds.
REPLs on Isabelle/ML. Examples are PISA and its descendants [7] and qIsabelle [21]. On the Lean side the Lean REPL [11] is of the same kind, and Pantograph [1] is a variant built for proof search. Each runs one prover process per worker, takes commands one at a time, and holds the state in memory. They were built for the training scenario and are still its most common harness. Two newer ones serve agents: Isa-Mini’s REPL server, on which AoA [25] runs, and AutoCorrode’s Isabelle/REPL (I/R) [3], which can fork a REPL at a point in another. A REPL is sequential by construction, so parallel proof checking is lost [16].
Batch build. The agent edits files and runs
isabelle build, as in the Munkres auto-formalisation [4]. A build is not incremental: every attempt re-checks the whole development. AutoCorrode’s Isabelle/Check (I/C) is a file check that avoids this: it runs on I/R and reprocesses only what a change affects.
Isabelle’s own server.
isabelle server, in the distribution since Isabelle2018, is a resident process that manages several logic sessions over a TCP line protocol. It is a direct PIDE frontend, multi-session and bounded by a time limit and a watchdog, butuse_theoriesis its one checking command: it works on whole theory files and answers when they are done. There is no command for a single step, for the goal at a point in a proof, or for Sledgehammer.
Language server (LSP). The connector reaches Isabelle through its language server,
vscode_server, an approach Kappelmann assesses as only partly complete [10]. The stock server lacks PIDE requests that Isabelle-MCP [24] needs, among them the output at a position and the cancellation of a running check. Early versions of Isabelle-MCP patched the Isabelle distribution to add them; since version 0.3.0 it carries a fork of the server as a component of its own, tied to one Isabelle release. At version 0.7.0 it serves one agent per process. On the Lean sidelean-lsp-mcp[6] is of the same kind.
Indirect PIDE frontend. Isabelle/Q [3] is an MCP server inside Isabelle/jEdit. The agent works in the human’s own PIDE session, which requires a running editor and confines it to one session on one machine.
Direct PIDE frontends. Examples are PIDE MCP [10] and IPS (this report);
isabelle server above is a third, and AutoCorrode’s IC2, which manages headless sessions from the command line, a fourth. At the commit we measured PIDE MCP served one client; its later versions, renamed PIDE AI, let several agents share one server (§ 7.1).
A resource-management layer. As far as we know, none of the Isabelle connectors above, at the versions we read, decides who gets a session, when it is taken back, or what happens when memory runs out. In Lean, Kimina [5] and AXLE [23] do: they put a pool of prover processes behind HTTP, with warm workers reused, admission, isolation and a bounded call. A pool is not an architecture of its own; it is a set of tools and background services that a multi-session frontend may offer. IPS offers them on a direct PIDE frontend, and is the one pool in either prover whose workers are PIDE sessions and not REPL processes, so that its tools can use the document model for checkpoints, rollback and parallel checking within a chunk.
AutoCorrode’s tools. The AutoCorrode repository [3], a verification framework for imperative programs in Isabelle/HOL, carries four of the connectors above: I/Q, I/R, I/C and IC2, the last “similar to isabelle server and isabelle client”. Two further parts concern a human at the editor: Isabelle/Proxy (I/P) runs the prover on a remote machine while Isabelle/jEdit stays local, and Isabelle Assistant is a language-model assistant inside Isabelle/jEdit. We read the repository on 16 November 2026 and measured none of these tools; I/R’s forking is the closest relative of IPS’s checkpoints (§ 3.1).
Nothing in IPS bypasses PIDE: sessions start, load theories and are edited through the API isabelle server itself uses.3
Table 2 places the connectors on the first two of the three questions that open this section: how the client reaches the server, and how the server reaches Isabelle. The scenarios differ on the third, what the server offers. Interactive work wants the whole PIDE surface, a project of many theories, and calls that do not block; training wants a narrow contract, a bound on every call, and a pool behind it. Nothing in a direct PIDE frontend rules out offering both; no connector we know of does so today. The limit of a REPL, sequential checking, applies to IPS in part: its one-command mode is a REPL and inherits the sequential cost (§ 4.3); its chunk call submits a block as one PIDE edit with parallel proofs on, and its whole-file check is a batch build.
Requirements and metrics
The training scenario
In the training scenario a loop builds or evaluates an LLM prover. Whether it is reinforcement learning, expert iteration or search, the loop has one shape: sample candidate steps, submit them, learn from the verdicts, repeat. IPS is judged by what it does to that loop, which needs four things from the service.
Verdicts at the sampler’s rate. Thor issues up to 300 model queries per theorem [8]; DeepSeek-Prover-V2 samples 32 candidate proofs per problem at every reinforcement-learning step [14]. A Lean pool verifies about 20 whole proofs per second on 64 cores, 0.05 s per proof amortised [5]; that is the reference point for an Isabelle service.
| Connector (version read) | Client reaches it by | It reaches Isabelle by | What it keeps between calls |
| PISA, qIsabelle | REPL protocol | Isabelle/ML | one prover process per worker, named states |
| I/R (AutoCorrode) | command line, TCP, MCP wrapper | Isabelle/ML | a set of REPLs that can be forked |
isabelle build | command line | a batch process | nothing |
isabelle server (Isabelle2026) | TCP line protocol | PIDE, direct | several sessions |
| Isabelle-MCP 0.7.0 | MCP | the language server (a fork of vscode_server) | one session, one agent per process |
| I/Q (AutoCorrode) | MCP | PIDE, indirect, through Isabelle/jEdit | the editor’s session |
| IC2 (AutoCorrode) | command line | PIDE, direct | persistent headless sessions |
| PIDE MCP dd46315 | MCP on standard input and output | PIDE, direct | several sessions, one client |
| PIDE AI 2026 | MCP or commands, on standard input and output or TCP | PIDE, direct | several sessions, several clients |
| IPS (this report) | HTTP, MCP, Python client | PIDE, direct | a pool of leased sessions, admitted by memory |
Table 2. Isabelle connectors: how a client reaches each, how each reaches Isabelle (the first two questions of this section), and what each keeps between calls. Only PIDE MCP at dd46315 and IPS were measured.
Cheap branching at a proof state. Best-first and tree search in Thor [8], Magnushammer [12] and DeepSeek-Prover-V1.5 [22] all branch. In a REPL a branch resumes from a saved state when the loop kept one, as PISA and qIsabelle allow and as AutoCorrode’s I/R does by forking a REPL at a point in another [3], and is otherwise a replay from the start or a fresh process.
A bound on every step. Every one of these systems bounds each prover call, from 2 seconds in Magnushammer [12] to 300 in DeepSeek-Prover [22], because one looping tactic otherwise removes a worker from the pool.
A verdict that means what the reward assumes. DeepSeek-Prover-V2 traced inflated benchmark results to a Lean bug that failed to report sorry [14]; AXLE’s strict verifier exists because compilation alone accepts sorry, unsound axioms and restated theorems [23].
The multi-agent serving scenario
In the multi-agent serving scenario several agents or harnesses share one Isabelle host, and whoever runs it usually does not know how many will be active or what they will import. IPS is judged by capacity and by conduct under load, and the scenario needs predictability on four counts.
A known number of sessions and a known wait. Kimina’s motivation was that earlier tools paid an initialisation cost on every verification, and reusing warm workers that had already loaded the same imports halved its verification time [5]. For Isabelle the cost is larger, on disk as well as in time: PISA’s instructions for running many servers give each server its own copy of Isabelle and its heaps, 35 GB of disk with the heaps of the Archive of Formal Proofs, to avoid I/O errors [7]. Memory, not cores, limits an Isabelle host, and the wait for a session runs from milliseconds, when one is ready, to minutes, when a heap must first be built (§ 6.2).
No heap build while a client waits. An Isabelle project heap takes minutes to build (§ 6.2), so a request whose heap is not yet built waits minutes rather than seconds. IPS keeps each heap once it is built (§ 4.2).
Defined behaviour when memory runs out. Kimina caps the memory of each worker [5]. An Isabelle host has the same need, with larger processes.
Each client’s work kept apart from the others. AXLE sandboxes every request [23], and the Isabelle community has proposed container and VM setups for agents [15]. IPS isolates by lease rather than by process, which is cheaper and weaker; the isolation measurement of § 6.2 states what it provides.
A third scenario, not served
A third, the interactive serving scenario, is a single agent working on a project, alone or beside a human, with the whole of PIDE within reach. It is where we take PIDE MCP to be strongest, though not the only use its preprint names: that also speaks of deployment on servers and of background autoformalisation [10]. IPS does not primarily serve this scenario; § 2 and § 7 make the comparison.
Metrics
Table 3 turns the questions of each scenario into the metrics reported in § 5 and § 6, with the procedure for each in Appendix § 13. Agent success on a benchmark measures the agent and its prompt rather than the service, so it appears only as an anecdote in Appendix § 15.
Service overview
This section describes IPS as a client sees it. Figure 1 shows the parts; Figure 2 shows how a request for a session is resolved. A reader who knows PIDE will find the same service described in PIDE’s terms in Appendix § 11.

Figure 1. IPS as a system. A training harness or an agent reaches the server through the Python client, one of the two bundled MCP servers or plain HTTP. The server leases sessions, admits new ones by memory, keeps project heaps and runs batch builds; a gateway JVM drives one headless PIDE session for each session of the pool, each with an Isabelle/ML process started from a heap. Arrows show who calls whom; every call returns its result along the same path. The dashed arrow from the heaps shows where a prover is loaded from.

Figure 2. How a request for a session is resolved, after Kimina’s dispatch figure [5]. A request that finds no warm session needs a heap that covers the imports, memory for one more session once idle ones are reclaimed, and a free slot or an idle session to evict; the first that is missing is the reason it is refused.
Sessions and leases
A client works with a session: a running Isabelle, loaded with the imports the client asked for, holding one theory the client builds up by submitting text. The client never starts Isabelle and never sees a file. Three properties of a session are fixed: its imports cannot change, its document is one theory rather than a project, and it belongs to one client for the life of its lease. A training harness reaches sessions through the Python client; an agent through the stepwise MCP server, which verifies chunks the agent submits, or the file MCP server, which follows a theory file the agent edits with its own tools.
Acquiring a session
The client names the imports it needs. If an idle session with those imports exists, it is leased at once. Otherwise the pool starts one on a built heap, or, if the host is short of memory, refuses with an explanation. § 6.2 gives the latencies of the two cases, and § 5.2 the cost when 32 sessions start at once. Imports from the Isabelle distribution are always available. Imports from the client’s own project need a heap that IPS builds once on request and reuses until the project’s sources change. Leases expire and idle sessions are reclaimed, so a client that fails to release does not hold a session indefinitely.
Verification modes
Table 4 sets out the verification modes: one command at a time; a chunk (verify_chunk), a block of Isar checked as one unit, which is the call a training loop uses; a whole file, rebuilt from scratch in a fresh Isabelle; and, for an agent, a batch of independent chunks, each on a clean session. A submission that fails or times out is rolled back: the server removes its text from the theory, so the session is as it was before the call.
| Mode | Use it when | Returns | Rolled back on failure | Parallel checking |
| One command | the loop wants the proof state after each step | status and messages of that command | yes | no |
Chunk (verify_chunk) | a candidate step or proof is one block | per-command status; success, proof_open, used_sorry, timed_out | yes, the whole block | within the block |
| Whole file | a verdict must be final | the build’s result; cached by content | stateless | the build’s |
Batch (MCP verify-batch) | many independent chunks from one agent | one chunk verdict per item | a clean session per item, closed after | across the pool, client-side |
Table 4. The verification modes.
Between submissions the client can read goals, open proofs, facts in scope, or a Sledgehammer suggestion without changing the document; take a checkpoint and later restore it, so a search branches and backtracks without a new session; or roll back its last submission itself.
Verdict semantics
IPS separates “did anything error” from “is the theorem proved”. A chunk succeeds when no command in it errored. It is proved when it succeeded, left no proof open, and used no sorry, Isar’s fake proof that pretends to solve the pending claim [18]; the client reads this from three flags in the response. When a verdict matters beyond the current loop, the whole-file check is the arbiter: a fresh build depends on nothing the session holds. § 5.3 tests the flags against it.
What the verdict certifies. IPS looks for sorry and oops as parsed commands, not as words in the text, so a comment or a string can neither hide one nor fake one. The whole-file check runs isabelle build with Isabelle’s defaults, under which sorry is an error, so a proof the batch build accepts is one Isabelle itself accepts. Neither verdict says that the statement proved is the one asked for, and neither rejects a new axiom. A separate command guard covers the second: unless the operator switches it off, IPS refuses axiomatization, along with ML and file commands, in any text it is sent (Appendix § 14), so a candidate proof cannot postulate what it then uses. A client that checks whole theories of its own should keep the guard on. A strict verdict that also checks the statement and the oracle set is on the roadmap (§ 8).
Time budgets
Every call that waits on Isabelle carries a budget, chosen by the client and enforced by the server. A submission that exceeds its budget is rolled back, and the call returns a timeout, so the session stays usable and no submission can hold a session or a worker indefinitely. This is a property of IPS’s calls, not of its architecture (§ 7). Appendix § 11 says how the server keeps these guarantees in PIDE’s terms, and Appendix § 14 how it is configured.
Training workloads
This section covers the training scenario: how a loop uses IPS, then what we measured against the requirements of § 3.1.
Design
The loop acquires a session per worker with the imports of its problems and keeps it while the imports stay the same. Each candidate is a chunk under a budget, submitted after a checkpoint. A candidate that fails or times out is rolled back by the server. A candidate that checks without error stays in the document, and the loop either goes on from it to the next step or restores the checkpoint to discard it. A candidate that is rolled back or discarded leaves the state as it was, and no session is started to branch. The reward reads the three flags of § 4.4. A final verdict goes to the whole-file check, and a batch of independent candidates to the stepwise MCP’s verify_batch (Table 4). Appendix § 12 shows the loop’s calls in Python.
Results
All numbers come from one series of runs on the host of § 6.2 (32 cores, 128 GB, Isabelle2026). Memory is in binary units throughout the report: 1 MB is bytes and 1 GB is , as the kernel reports them. The workload is the HOL-Analysis library: its 93 theories apart from the collector Analysis.thy, split into 11 984 top-level blocks of 166 438 commands, each block submitted as one chunk. One replay of the library at one worker, the census, proved every block; its times are the reference for every slowdown below. Each theory runs on a session acquired with its own imports, as in a curriculum whose problems differ in imports. No model runs: the candidates are scripted, so the rates measure the server alone, with no model’s time in them. is the number of concurrent workers. A cell is one run at one setting, in one repeat. Appendix § 13 gives the method; the raw results are in the results repository.
Throughput. In the throughput runs workers, from 1 to 32, each replay eight theories block by block with verify_chunk, every theory on a session started for it, with three repeats per (Figure 3). We give the rate twice: verify only, the rate a loop sees once its sessions are up, and with session starts, where the start of each theory’s session is counted.

Figure 3. Verified chunks per minute against the number of workers. The dotted line is linear scaling from the single-worker rate. The verify-only line is the rate once sessions are up; it falls below the dotted line as calls slow, from 0.41 to 0.52 s at the median. The line with session starts also counts the start of each theory’s session; the gap between the two is the share of a worker’s time spent acquiring a session, 10–14 % up to 8 workers and 64 % at 32. Lines are the medians of three repeats; the small points are the repeats. Eight theories per worker.
Verify only. One worker verified 132 chunks per minute, which is 1 607 commands per minute, and 32 workers verified 3 181 chunks per minute, 24 times as many. The median latency of a call rose only from 0.41 to 0.52 s, and no call timed out.
With session starts. With the per-theory session starts counted, the rate at 32 workers is 899 chunks per minute, 8.0 times the single-worker 113 (8.6 times on the census-weighted measure of Table 9, an efficiency of 0.27). The gap to the verify-only rate is the starts. A cold HOL-Analysis start takes 6 s on an idle host; it took 8 s at 8 workers, 35 s at 16 and 156 s at 32 (medians), with no request refused. A start loads the HOL-Analysis heap while the other sessions check proofs. We did not investigate the cause of the slowdown further; memory is ruled out, at 28 GB at the peak. Acquisition is 10–14 % of a worker’s time up to 8 workers, 30 % at 16 and 64 % at 32. A loop whose problems share imports keeps its sessions and pays none of this; one whose problems differ in imports should expect the lower curve of Figure 3, or start its sessions before the work begins.
| W | chunks/min verify only | repeats min–max | commands per min | chunks/min with session starts | acquisition share | census work rate | slowdown vs census | p50 s | p95 s | PSS peak GB |
| 1 | 132 | 108–132 | 1 607 | 113 | 0.10 | 0.97 | 1.02 | 0.41 | 0.92 | 6.5 |
| 2 | 281 | 230–286 | 3 713 | 183 | 0.12 | 1.44 | 1.04 | 0.41 | 0.92 | 7.5 |
| 4 | 474 | 469–503 | 6 576 | 297 | 0.12 | 2.58 | 1.08 | 0.41 | 1.06 | 11.3 |
| 8 | 927 | 902–975 | 12 501 | 497 | 0.14 | 4.45 | 1.17 | 0.42 | 1.22 | 16.0 |
| 16 | 1 677 | 1 558–1 718 | 22 353 | 789 | 0.30 | 7.07 | 1.62 | 0.47 | 1.41 | 26.5 |
| 32 | 3 181 | 3 162–3 214 | 44 877 | 899 | 0.64 | 8.33 | 3.13 | 0.52 | 1.50 | 28.3 |
Table 9. T1 at every : the median of three repeats, with the repeats’ range of the verify-only rate (no call timed out, no theory skipped). Census work rate: the census time of the cell’s theories over the cell’s wall time, the number of single-worker replays the cell is worth; speedup and efficiency in the body are on it. Slowdown: the cell’s replay time over the census time of the same theories, acquisition included.
Reference points. Two reference points place these rates. Against the batch tool: isabelle build checks the whole library from sources in 360 s on this host (Table 8), about 27 700 commands per minute; the replay at 32 workers checks 44 900 commands per minute verify only and about 12 700 with session starts. The two are not the same work, since the build is bound by the critical path of the theory dependency graph and the replay checks every theory against the prebuilt heap, but chunked, stepwise checking at 32 workers runs in the batch tool’s range. Against Lean: 32 workers verify 15 chunks per second with session starts and 53 verify only, where Kimina’s pool verifies about 20 whole proofs per second on 64 cores [5]. A chunk and a Lean proof are different units, so this is an order of magnitude, not a ranking: IPS is in that class.
| HOL | HOL-Library | HOL-Analysis | |
| Warm hit (a released session acquired again), p50 | 0.002 s | 0.002 s | 0.002 s |
| Cold start on a built heap, p50 / p95 | 2.3 / 3.9 s | 3.4 / 5.1 s | 6.0 / 8.2 s |
| Heap build from sources (once) | 4 s† | 114 s | 360 s |
Table 8. Acquisition latency under IPS, by case. Warm hits: 4 sessions, 3 rounds, every acquisition reused the session its client had released. Cold starts: the session starts of the capacity runs. Heap builds: the base heaps on this host. †HOL was already built by the Isabelle installation; the figure is the check, not a build.
Rollouts per hour. Table 5 gives the same measurement for two scripted rollout loops. The whole-proof loop submits eight candidates per theorem, and each candidate is one rollout. Of the eight, the corpus proof always proves, four prove when they happen to and three cannot; a candidate that stays in the document is discarded by a restore. The step-level loop tries two alternatives at every proof command before the corpus step, each under a checkpoint (Appendix § 13 lists the candidates). About 15 % of the whole-proof rollouts end in a proof; that share is a property of the scripted candidates, not a result about the server. At one worker the whole-proof loop ran 6 436 rollouts per hour and the step-level loop 2 343 verified steps per hour; 32 workers ran 5.4 and 8.6 times as many. The time a worker spends on one rollout does not change with the number of workers: the server does not slow a worker’s loop at 32. Two things bound the end-to-end rate. One is setup, nearly all of it session starts: 62–68 % of a worker’s time at 32. The other is workers finishing early. With three theories each, one worker’s loop ran 26 s and another’s 255 s in one cell, so a fifth of the workers’ time at 32 workers, and up to two fifths at 8, was spent outside any call, mostly by workers already done. Checkpoint and restore each took 3 to 4 ms in every cell, and all 769 integrity checks of the document’s source passed.
| Whole-proof loop | Step-level loop | |||||||||
| W | rollouts/h | proved/h | s per rollout | setup | idle | paths/h | verified steps/h | s per path | setup | idle |
| 1 | 6 436 | 954 | 0.60 | 0.22 | 0.00 | 596 | 2 343 | 3.67 | 0.36 | 0.00 |
| 8 | 20 778 | 2 839 | 0.60 | 0.30 | 0.39 | 1 613 | 15 243 | 3.65 | 0.36 | 0.19 |
| 32 | 34 518 | 4 798 | 0.60 | 0.68 | 0.21 | 2 071 | 20 042 | 3.67 | 0.62 | 0.23 |
Table 5. Scripted rollout loops at workers, two repeats each. A path is one step-level rollout: a theorem’s proof walked command by command. Rates are end to end: session acquisition, entering the theory and replaying up to each problem included. “s per rollout” is a worker’s loop time per rollout on the theories every cell shares, the like-for-like comparison across . Setup is the share of a worker’s time in acquisition, entering and replaying to the problems; idle the share of the workers’ time outside any call, mostly workers already finished. No rollout ended in a memory, prover-death or session error; the candidates’ timeout share was 0.5–2 %.
Branch cost. We took 200 proof states across 68 theories, a third each early, in the middle and late in their files. At each we tried the next block and then went back to the state in three ways: by restoring a checkpoint, by rolling back the last submission, and by replaying the theory up to the state in a new theory node or a new session. The replay is the cost of a branch when no state was saved (Figure 4, Table 6). A restore call returns in 4 ms, before the document has settled, so the verification call that follows may have to wait. We call that extra time the settling delay: the time by which a verification call is slower after a restore than on a settled document. The branch cost, the restore call plus its settling delay, is 4 ms at the median and 20 ms on average (200 ms at the 95th percentile); for the late states its mean is 25 ms, with a 95 % confidence bound of 63 ms. Replaying to the same state takes 8 s at the median for the early states, 43 s in the middle and 91 s late in a file, 13.9, 49.6 and 93.2 s with a fresh session, so the branch cost is at least 350 times lower than a replay’s early in a file and 1 450 times late in it. Verified again after each restore and each rollback, the block gave the same report as the first time in all 1 000 repeats, and the source matched after all 600 restores and 400 rollbacks.
| Operation | p50 | p95 |
| Save a checkpoint | 3 ms | 4 ms |
| Restore a checkpoint (the call) | 4 ms | 5 ms |
| Branch cost: restore call plus settling delay | 4 ms | 200 ms |
| Roll back the last submission (settled) | 110 ms | 216 ms |
| Replay the prefix in a new node, same session | 30 s | 165 s |
| Replay the prefix in a new session | 35 s | 169 s |
Table 6. The cost of going back to a proof state, over 200 states. The replay rows pool the three positions; Figure 4 shows them against the prefix length.

Figure 4. Time to go back to a proof state, against the number of commands before it, for 200 states: by a restore call, by a rollback, and by replaying the prefix in a new theory node of the same session or in a new session. At this scale the branch cost, the restore call plus its settling delay, cannot be told from the restore call alone; Table 6 gives it.
Checks
Two properties of the implementation are checked here rather than measured, since they are things a service must get right rather than metrics on which to compare services: that each verdict flag catches the failure it exists to catch, and that a looping step cannot take a worker with it. The tables are in Appendix § 13.
Verdict flags. We tested each flag against the failure it exists to catch, with a batch build of the same text as the reference (Table 10). We sampled 200 theorem blocks across the library and spliced each into its theory in four forms: unchanged, the proof replaced by sorry, the last closing step removed, and the first fact name made undefined. Every flag caught every case of its failure: used_sorry the 200 placeholders, proof_open the 200 unfinished proofs, success the 197 failing steps, 13 of them by the budget, since the step ran until its bound and was rolled back rather than failing at once. The 200 unchanged blocks and the 93 theories as written were accepted by the flags and by the build alike. No case was proved in-session and rejected by the build, the disagreement that would corrupt a reward, and no mutated form was accepted by either. The 890 checks, the five rows of Table 10, rest on 690 batch builds, the 93 theories as written and the 597 mutated forms, since an unchanged block shares the build of its theory as written; the builds took 13.6 s at the median and 24 s at p95, four at a time. A loop can therefore read its reward from the flags and run the batch build where a verdict is final; § 4.4 says what no flag sees.
| Proof as submitted | flag it targets | n | in-session | batch build |
Proof replaced by sorry | used_sorry | 200 | used_sorry fired in 200 | rejected 200 |
| Last closing step removed | proof_open | 200 | proof_open fired in 200 | rejected 200 |
| A fact name undefined | success | 197 | success false in 197, 13 of them at the budget | rejected 197 |
| Block unchanged | none | 200 | no flag fired in 200 | accepted 200 |
| Theory as written | none | 93 | no flag fired in 93 | accepted 93 |
Table 10. Each verdict flag against the failure it is meant to catch, with a batch build of the same text as the reference. A flag that fires rejects the proof in-session. The mutations change the proof text only; statements and axioms are as in the library. Three blocks have no fact name to replace.
Bounded step. We placed 100 steps meant to loop before proved theorems of 9 theories, 50 of each kind: a rewriting loop and a metis call on a false goal (Appendix § 13). Each was submitted under a budget of 5 s, and some of those that looped were tried again under 30 and 120 s (Table 11). Every looping step returned at its budget with the stuck command named, the server rolled it back within 0.6 s at p95 in every row but one (4.2 s), the document’s source was unchanged after every attempt, and the next corpus block proved at the census’s speed (0.41 s median). Twelve of the rewriting steps failed at once and are counted apart. At 8 workers, with 15 looping steps of 120 s spread over the workers so that up to seven loops ran at once, the other workers’ blocks ran at the speed they had in the 8-worker throughput cell, where nothing loops (median ratio 1.00 over 1 130 matched blocks), and waited 4–5 ms for the server. A long rewriting loop does leave memory in the prover: 1.2 GB at the median after a 30 s loop and 7 GB after 120 s, which the rollback does not reclaim; the metis loops left 130 MB or less at the median (858 MB at most). This is Poly/ML’s own memory, which grows under a long rewrite and is not returned to the operating system afterwards. A cap on each prover’s memory (Appendix § 13) bounds it; a prover that reaches the cap fails with an out-of-memory error that the response names as such, and the memory gate sees the growth on the next call.
| Looping step | Budget | Attempts | Returned at budget | Rollback p50 / p95 | Memory left, p50 |
metis, up to 56 facts | 5 s | 50 | 50 | 0.07 s / 0.45 s | 4 MB |
| 30 s | 10 | 10 | 0.30 s / 0.60 s | 127 MB | |
| 120 s | 3 | 3 | 0.28 s / 0.35 s | 1 MB | |
subst rewrite loop | 5 s | 38 | 38 | 0.05 s / 0.52 s | 4 MB |
| 30 s | 8 | 8 | 0.54 s / 4.21 s | 1 181 MB | |
| 120 s | 1 | 1 | 0.35 s / 0.35 s | 7 116 MB |
Table 11. Looping steps under a budget, one worker. Rollback is the wall time past the budget until the call returned with the step rolled back. Memory left is the prover’s resident set after the attempt minus before it. Twelve of the 50 rewriting steps did not loop and are left out.
Multi-agent serving
This section covers the multi-agent serving scenario: how the pool behaves on a shared host, then the measurements against the requirements of § 3.2.
Design
§ 4.2 describes the pool as a client meets it; three things concern the host. Admission is gated on the container’s memory: above the threshold the pool evicts the least-recently-used idle session, then refuses a new session with an explanation, so the host reports “busy” instead of suffering an out-of-memory kill. Concurrent Sledgehammer runs are capped by a server-wide limit, since each fans out to several external provers. All sessions share one gateway JVM; its crash is detected and the pool rebuilt. Probes, metrics and the lack of authentication are in Appendix § 14.
Results
All numbers in this section come from one run on a 32-core, 128 GB Ubuntu host with Isabelle2026, both tools native and nothing else running; Appendix § 13 gives the method and the workloads, the results repository the per-cell results. We write for the number of concurrent sessions. We took the capacity and isolation measurements for IPS and, under identical conditions, for PIDE MCP at commit dd46315 (19 October 2026). The figure and the table of this section show PIDE MCP in two arrangements: PIDE MCP A, one PIDE MCP process holding sessions, and PIDE MCP B, processes with one session each (§ 7.3 says why both). This section reports the IPS numbers; § 7 reads the two side by side.
Memory measures. We report two measures. The proportional set size (PSS) of the tool’s processes, summed, counts a shared page once. Host used is the change in MemTotal minus MemAvailable over the cell’s baseline; it checks that no memory went uncounted. A prover shares only about 13 MB, so its PSS, resident and unique sizes agree within 2 %. Under a memory limit we also read the cgroup’s counters, which the limit is enforced on (Appendix § 13).
Capacity. Figure 5 plots the summed PSS of the tool’s processes against the number of concurrent sessions for three heaps, each session holding a checked theory, three repeats per point. The curves are straight to . A least-squares fit on the processes’ summed PSS (Table 7) separates the pool’s fixed cost, one server process and one gateway JVM, 3.7–4.2 GB, from the cost of one more session: 886 MB on HOL, 1 077 MB on HOL-Library and 1 263 MB on HOL-Analysis, which is the prover alone. The gateway JVM grows from 3.6 GB at one session to 5.1 GB at 32, so the fixed cost is not constant, but its growth is small against the sessions. After the fixed cost, a 100 GB budget (102 400 MB) holds 111 HOL, 91 HOL-Library or 77 HOL-Analysis sessions. The same fit on host used gives per-session costs within 1 % of these, so the process view accounts for everything the host loses. These are the settled costs of a session holding one checked theory, a planning estimate; a long proof search grows its prover beyond them (Table 11). When all sessions re-check their theories at once, after a line is inserted near the top (the lower row of Figure 5, Appendix § 13), the per-session cost rises by at most 1.4% under IPS, because the re-check runs inside the memory each prover already holds; under PIDE MCP A it rises by 4–28%, most on HOL-Library.

Figure 5. Memory of the tool’s processes (summed PSS) against the number of sessions , one column per heap, three repeats per point as small marks with the line through their median. Top: after every session has loaded and checked its theory. Bottom: at the concurrent re-check of all theories (the re-check peak, Appendix § 13). IPS: one pool. PIDE MCP A: one process holding sessions. PIDE MCP B: processes, one session each.
| Tool | Heap | Settled MB per session | Re-check peak MB per session | Sessions per 100 GB | Cold start p50/p95 s |
| IPS | HOL | 886 | 887 | 111 | 2.3 / 3.9 |
| IPS | HOL-Library | 1077 | 1092 | 91 | 3.4 / 5.1 |
| IPS | HOL-Analysis | 1263 | 1267 | 77 | 6.0 / 8.2 |
| PIDE MCP A | HOL | 1266 | 1320 | 78 | 2.0 / 3.2 |
| PIDE MCP A | HOL-Library | 1495 | 1915 | 66 | 3.0 / 4.3 |
| PIDE MCP A | HOL-Analysis | 1461 | 1602 | 67 | 5.8 / 7.1 |
| PIDE MCP B | HOL | 2630 | 2712 | 38 | 3.1 / 3.2 |
| PIDE MCP B | HOL-Library | 3017 | 3466 | 33 | 4.1 / 4.2 |
| PIDE MCP B | HOL-Analysis | 3604 | 3753 | 28 | 6.7 / 7.3 |
Table 7. Capacity and latency from the capacity runs (E1 in Appendix § 13), on the processes’ summed PSS. Settled MB per session: the least-squares marginal cost over (18 cells per row) after the theories are loaded; peak: the same at the concurrent re-check; sessions in a 100 GB budget (102 400 MB) after the fit’s fixed cost, rounded down; the cold-start latency. The fixed cost is the fit’s intercept: 3.7–4.2 GB for IPS (the server and the gateway JVM) and 2.4–3.2 GB for PIDE MCP A (its JVM); PIDE MCP B has no fixed process and its intercepts lie within 0.5 GB of zero.
Acquisition latency. Table 8 separates the three cases. A warm hit, a released session handed to the next client with the same imports, takes 2 ms and returned the same session in all 36 trials. A cold start on a built heap takes 2 to 6 seconds depending on the heap. A heap build takes minutes: 114 s for HOL-Library and 360 s for HOL-Analysis from the distribution sources on this host. A cold start costs over a thousand times a warm hit, and a heap build 30 to 60 times a cold start, which is why IPS keeps sessions warm and keeps a built heap until its sources change.
Pressure. We started the server in a systemd scope with MemoryMax=48G and added HOL-Analysis sessions one at a time, each loading and checking the theory, until the pool refused. IPS admitted 27 sessions and refused the 28th with HTTP 503 and a body that states the facts: 37.5 GB in use (78% of the limit), 10.5 GB available, every running session leased and so none evictable, retry in 5 s. The gate refused because the next session’s 4 GB estimate would cross the 85% threshold. No process died (oom_kill stayed 0; the scope’s peak was 37.5 GB). After the client released half of its sessions, the next session was admitted and loaded 8.4 s later. The stop is the gate’s margin, not the limit: these runs used the default 4 GB estimate per HOL-Analysis session, and the limit itself would hold about 35 sessions of the measured 1.2 GB. So the 27 is where the gate stops a client, not how many sessions the memory holds.
Isolation. With 8 HOL sessions leased, we killed the newest session’s prover process with SIGKILL and asked every session to re-check its theory within 30 s. The other 7 answered. The victim’s session returned “prover process died, the session has been discarded”, and its slot went back to the pool. The gateway and the server process were unaffected. A prover death is thus one client’s failure, reported as such. Cross-lease requests, the other half of isolation, are refused by the lease check; we did not measure them here.
Comparison with PIDE MCP
IPS and PIDE MCP [10] are both direct PIDE frontends (§ 2), and both expose their sessions to a language model over MCP. This section says which version of PIDE MCP we measured, where the two differed at that version and which of the differences are tooling, what we measured for both under the same conditions, and what each offered that the other did not. The full inventory, 24 concerns side by side, is docs/FEATURE_INVENTORY.md in the repository.
The version measured
Every statement about PIDE MCP in this report is about commit dd46315 of 19 October 2026 (server 0.1.0), five weeks after the preprint, unless it says otherwise. At that commit PIDE MCP was a headless, multi-session PIDE frontend with a replaceable tool set, serving one client over MCP on standard input and output. Three things have changed since, and each bears on a result below.
Name and transport. The project was renamed PIDE AI on 4 November 2026. Since its Isabelle2026 release a server also listens on a TCP port, so several agents can share one server and its sessions. Topology B of § 7.3, one process per agent, was how independent agents had to use the commit we measured; it is no longer needed.
Waiting. Since 31 October 2026
get_statetakes a timeout and waits until the commands have finished or failed, andedit_get_statejoins the edit to that wait. The polling loop of our harness (§ 7.3) is not needed against these versions.
Visibility. Ranges that an agent reads or edits are no longer made visible by default (commit
8fdfc12, 14 November 2026), a change made after we reported to its author the memory cost of § 7.3.
We have repeated no measurement on a later version. The preprint says that the core tools can be removed like any others and that “the concrete set of core tools is likely to change over time” [10]. What we measured is the tool set of one commit, and we say so wherever a difference could be read as one of architecture.
Differences at the measured commit
Where a node’s text comes from. Both tools edit the nodes of a PIDE document. In IPS a session has one theory node whose text arrives in requests: the server appends to it or replaces it, no file stands behind it, and the session’s imports are fixed when it starts. In PIDE MCP the nodes are those of a project, theories and auxiliary files such as ML files, each theory with its own header. PIDE loads them from disk and then manages them in memory, and PIDE MCP’s read and edit tools synchronise a node’s content with its file at every call [10]. That gives PIDE MCP project scope, dependency resolution, positioned messages, and co-presence with a human’s editor through the file system. IPS rolls back a chunk as a unit when it fails, records checkpoints and restores them; PIDE MCP had no tool that undoes an edit at that commit. Such a tool could be written: a PIDE session keeps a history of its document’s versions, and an edit tool could keep its modifications on a stack.
Waiting. The preprint says that PIDE MCP “decidedly does not block on checking and does not use timeouts for its core tools, but users can implement such means for their own tools if desired” [10]. At the commit we measured an edit returns when PIDE has accepted it, the agent reads a snapshot with get_state and, as the preprint notes, typically polls until the result is there, and a long-running tool call can be cancelled through the protocol. An IPS call waits for its submission to settle under a budget, and a submission that exceeds the budget is rolled back and reported as a timeout. The first suits an agent that can look again later; the second suits a loop with a fixed worker pool that cannot afford to lose a worker. A tool that waits under a bound can be written for PIDE MCP, and its later versions ship one (§ 7.1).
Decisions across sessions. IPS has a pool with leases, idle and lease timeouts, LRU eviction, a memory admission gate and a Sledgehammer cap. PIDE MCP, at the commit we measured, reports session statistics and leaves admission and reclamation to the deployment, a reasonable division of labour for one agent on one machine. For many agents on one host the deployment has to supply them.
Architecture or tooling. In the preprint’s sense the two tools have one architecture: each is a PIDE frontend that manages several headless sessions through the Isabelle/Scala API [10]. None of the differences above is architectural in that sense. One of them is a decision about what a session holds, and IPS could not change it without changing its sessions. A session’s context is fixed and its one theory has no file behind it, so sessions with the same imports can stand in for one another, which is what lets the pool reuse them. In exchange a session cannot hold a project of several theories (§ 8). Everything else in this section is tooling at one commit, and it transfers in both directions: budgets, rollback on failure, checkpoints and the verdict flags could be written as PIDE MCP tools, and positioned messages, cancellation and progress could be added to IPS. The lease belongs with the tooling too. No verification feature needs it; it is what tells the layer above the sessions which of them are in use, so that the layer evicts only the others.
Measurements
We ran the capacity and isolation experiments of § 6.2 for PIDE MCP at commit dd46315 on the same host, Isabelle build, prover memory cap and workloads, in two arrangements. Topology A is one PIDE MCP process holding sessions for one client, the multi-session use the tool was built for. Topology B gives every agent its own PIDE MCP process, which independent agents needed at that commit (§ 7.1). It is a cost study of that arrangement and not a defect of the tool.
Memory per session. Under topology A a session costs 0.2 to 0.4 GB more than under IPS on every heap, and 100 GB holds 66–78 sessions rather than 77–111 (Table 7). The gap is inside the prover processes, not the JVMs: over the cells of the series a PIDE MCP A prover holds 0.94–1.25 GB on HOL and 1.29–1.39 GB on HOL-Analysis (medians per cell), an IPS prover 0.79 and 1.17 GB, while the two tools’ JVMs end within 0.4 GB of each other at 32 sessions. The likely cause is the PIDE perspective. At this commit PIDE MCP’s edit marks the edited range and 50 lines around it as visible (option pide_mcp_range_context). Our workload writes the whole theory in one edit, so the whole theory is visible and PIDE runs its print functions on every command, among them Isabelle’s automatic tools, to which PIDE AI’s change log attributes the cost. IPS gives its nodes an empty visible range (Appendix § 11). We reproduced the gap on IPS by making its nodes fully visible: the same HOL theory then cost 0.98 to 1.22 GB per prover in seven of eight provers, against 0.80 GB with the empty range (Appendix § 13 gives the check, § 8 what it does not cover). If the perspective is the cause, the extra memory is the cost of those print functions, and it depends on the workload. At this commit only the range last read or edited stays visible, with its 50 lines of context. We therefore expect an agent that edits a few lines at a time to pay little of it, though we did not measure that case. The session options do not explain the gap: IPS ran parallel_proofs=2 and PIDE MCP Isabelle’s default of 1, which give the same checking in a headless session (Appendix § 11).
PIDE MCP’s fixed cost is lower (2.4–3.2 GB against 3.7–4.2 GB): its JVM starts at 1.5 GB on HOL and grows to 4.9 GB at 32 sessions, where the IPS gateway starts at 3.6 GB and ends at 5.1 GB. So PIDE MCP A is the smaller deployment for a few sessions and IPS for many: the totals cross between three and five sessions, depending on the heap. Under topology B a session costs 2.6–3.5 GB, three times IPS, because every agent pays a 1.5–2.2 GB JVM; 32 HOL-Analysis agents used 114 GB of the 128 GB host, 118 GB at the re-check peak.
Throughput. We ran the training workload of § 5.2 under both tools, with the same theories, census and heap, and each theory on a fresh session started inside the timed window. For PIDE MCP our harness used an edit-and-poll loop over its tools: one edit per block, then get_state every 0.1 s until the block has settled. At one worker the two were level: IPS verified 132 chunks per minute and the PIDE MCP loop 135, or 113 and 116 with the session starts counted. At eight workers IPS verified 927 chunks per minute against 813, and 497 against 381 with the starts. The loop and its interval of 0.1 s are ours and not PIDE MCP’s: we chose the interval because IPS re-reads the node at it. The two calls do not do the same work. verify_chunk does more than check the chunk: when the chunk fails it rolls the chunk back, and when the chunk succeeds it asks the prover whether a proof is still open, which costs about one 0.1 s poll. In both cases it returns a verdict with the status of each command. The PIDE MCP loop returns counts of commands, from which the client reads off a verdict itself. Both rates count verified chunks, so a block that fails or times out lowers them rather than inflating them. Some blocks fail under the PIDE MCP loop through choices of our driver and not of the tool: one block per cell that did not settle and ended its theory, and under 1 % of the blocks through the driver’s file name and its closing end (Appendix § 13).
Latency. Cold starts do not separate the two tools. A start takes 2 to 7 s depending on the heap, and the gap between the tools’ medians, 0.2 to 0.8 s, is smaller than the spread of IPS’s own start times, about 1.5 to 2 s from the median to the 95th percentile (Table 7). For the latency of one call we paired the chunks of the one-worker runs above: 2 966 chunks that both tools proved, each timed from the call to the moment the client knew it had settled, at the same 0.1 s resolution (Appendix § 13); Figure 6 shows the two distributions. Checked one after another, they took IPS 1 479 s and PIDE MCP 1 443 s. The difference is 2.4 %, or 12 ms per call, which is small against the 0.1 s polling interval. The median call is 0.41 s under IPS and 0.37 s under PIDE MCP, within one polling interval, as are 57 % of the pairs; a single command, the floor of a call, costs 0.34 s under IPS and 0.31 s under PIDE MCP on average, and chunks of ten or more commands cost the same under both (0.66 s). The IPS call does more for the time, as the paragraph above says, and an event-driven client would shorten both.

Figure 6. Empirical cumulative distribution of the latency of one call at one worker, over the same chunks under both tools, being the latency of chunk under the tool, the median over the repeats that ran it; each curve passes through every 0.5 % quantile (Appendix § 13). The vertical runs are the 0.1 s step of both tools, the interval at which IPS re-reads the node and at which our harness polls PIDE MCP; the dotted line is , the median, at 0.41 s under IPS and 0.37 s under PIDE MCP.
Under a memory limit. We make no comparison under a memory limit. PIDE MCP at that commit leaves admission to the deployment, as the previous subsection says, so a run under a limit tests the kernel and not the tool. The run we made under topology B is in the results repository.
Isolation. With one prover among 8 killed, the other 7 sessions answered under all three arrangements. PIDE MCP reports the killed session as no longer running, where IPS names the cause (§ 6.2). Under topology B all eight MCP processes survived their provers. On this test the tools are equal.
Capabilities
PIDE MCP at the measured commit, not IPS. A project of many theories and auxiliary files with dependency resolution; an edit tool that writes files, with messages at line and column; a visible range around what the agent reads or edits, on which PIDE runs its print functions; cancellation of a long-running tool call and progress notifications; entity search over a range; an extension mechanism for tools through Isabelle services; and three shipped agent skills. The first is a matter of what a session is; the others are tools.
IPS, not PIDE MCP at the measured commit. A pool with leases, reuse, eviction and a memory admission gate; rollback of a failed chunk as a unit, and checkpoints; a verification call with explicit semantics (success, open proof, sorry) and a batch build as arbiter; a wall-clock budget on every call; batch fan-out across the pool; an HTTP API beside MCP; input hardening (an ML and file-IO denylist, path roots); Prometheus metrics and probes; and a test suite. The pool with its leases and its gate is the resource-management layer. The other items are tools that need neither leases nor appended text, and they could be written for another PIDE frontend.
Both. Direct PIDE frontends on headless sessions with chosen logic and imports; per-command status; state queries that do not edit the document; Sledgehammer; MCP on standard input and output.
Limitations and roadmap
Limitations
The limits are of three kinds: of what a session is, of the deployment, and of the measurements. Two further sets are reported elsewhere. § 7 lists the tools PIDE MCP had and IPS lacks, cancellation, progress and an extension mechanism among them. And two weaknesses that the measurements found are reported where they were measured: session starts slow down as workers are added (§ 5.2), and a long rewriting loop leaves memory in the prover (§ 5.3); the pool does not recycle such a session, so the client must close it.
What a session is. A session’s imports are fixed when it is created, and it holds one theory with no file behind it. So changing imports means a new session, and project imports must be session-qualified theory names; there is no multi-file project scope; diagnostics are located at a command, not at a line within it; and the connector edits no file: the file MCP server observes a file the agent edits with its own tools, and the stepwise server has no file at all. This is the price of the decision that lets sessions be pooled (§ 7).
Deployment. There is no authentication. The trust boundary is the network: any client that can reach the server can acquire sessions, and the lease check keeps clients apart but does not identify them. IPS is meant to run on a trusted network or behind an authenticating proxy (Appendix § 14). One gateway JVM holds every session’s backend and grows with the pool (3.6 to 5.1 GB over 32 sessions); a crash of it loses every session at once.
The measurements. Four things bound what the numbers show.
They are tied to a build. Every figure in Sections 5 to 7 is for the Isabelle build, host and commits in Appendix § 13. They were taken on the second release candidate of Isabelle2026 and were not re-taken on the final release.
PIDE MCP was measured at one commit. No number was re-taken on PIDE AI, and three changes made since that commit bear on the results (§ 7.1).
The replay baseline is IPS itself. The branch-cost comparison replays a prefix in a new node or a new session of IPS, the cost of a branch from a state nobody saved. A REPL on the same host would make the comparison direct, and one that keeps states (§ 3.1) would put a saved-state branch beside the restore. We ran none: PISA dates from 2021, qIsabelle stopped at Isabelle2024, and I/R, which is built for agents and not for training loops, we did not try. The PIDE MCP figures beside IPS’s in § 7 are throughput at and 8, not branch cost, so the branch-cost comparison has no external baseline.
The perspective cost was reproduced on IPS, not isolated in PIDE MCP. Making a theory fully visible adds 0.2–0.4 GB to an IPS prover on HOL, which matches PIDE MCP A’s gap there, but PIDE MCP itself was not re-run with a narrower range, and HOL-Library and HOL-Analysis were not checked. Our harness writes the whole theory in one edit, so narrowing
pide_mcp_range_contextalone would not change what is visible; a harness that edits in pieces would. A re-run on PIDE AI, where visibility is the agent’s choice (§ 7.1), would test the attribution directly; we have not done it.
Roadmap
A strict verdict is the first item, and authentication the first for the multi-agent serving scenario. All but the last leave IPS’s session layer as it is.
A strict verdict. Reject any theorem whose oracle set is not empty, which Isabelle’s
thm_oraclesreports and which coverssorryand every other oracle, and check the statement against a reference, as AXLE’s strict verifier does for Lean.
Authentication with per-client quotas on sessions and budgets, so that a shared host can tell its clients apart and bound each one.
Agents over the AST. AoA [25] edits a Minilang proof tree by node rather than Isar text and runs today on Isa-Mini’s own REPL server; a Minilang document mode in IPS, with node-level edits and per-node verdicts, would give such agents the pool, the budgets and the checkpoints.
A command-line client. An agent that has a shell can call a command where it would call an MCP tool. PIDE AI ships command-line clients for its command protocol, and AutoCorrode’s IC2 drives headless sessions with
isabelle ic2[3]. The HTTP API already carries every IPS operation, so the client is a thin wrapper; what needs design is where the lease is kept between invocations.
Consolidation with PIDE AI. IPS and PIDE AI each wrote a Scala layer on Isabelle’s session class, and below the session model the two now do the same things: a node checked without a visible range, proof states read from the prover’s state messages, Sledgehammer as an overlay query. Sharing can start there, with IPS’s query operations as a component either system loads and the harness of Appendix § 13 as a common benchmark. The next step is a trial: one IPS operation, a step under a budget that is rolled back when the
budget runs out, written as a PIDE AI tool extension and measured. If the trial goes well, the pool, the leases and the memory gate could run on PIDE AI sessions in place of IPS’s own backend. The two session models differ (where a node’s text comes from, how a session gets its imports, what a timeout does), so we think a merge possible and have not costed it.
Conclusion
IPS is a direct PIDE frontend with a resource-management layer: a pool of warm sessions, each leased to one client at a time, admitted by memory, with a budget on every call. In architecture it does not differ from PIDE MCP, and what a training loop gets from it, rollback, exact restore, parallel checking within a chunk and a batch build as arbiter, are tools that could be written for another PIDE frontend. What is particular to IPS is narrower. Sessions with the same imports are interchangeable and can be pooled. In exchange a session cannot hold a project of several theories, and diagnostics are located at a command and not at a line. And the layer above the sessions is built and measured: a known cost per session, a gate that refuses with a reason, and isolation by lease. On the questions we measured, memory per session, throughput, conduct under a limit and isolation, IPS does what it was built for. PIDE MCP at commit dd46315, holding its sessions in one process, is within 0.4 GB of it per session and level with it in throughput at one worker. The three-fold gap in memory per session, which appears when each agent runs its own PIDE MCP process, is the price of a JVM per agent, and PIDE AI no longer requires one. On the questions we did not measure, those of the interactive serving scenario, PIDE MCP is the more capable tool, and nothing in our measurements says otherwise. A group running both would use PIDE AI for interactive work and IPS in the training cluster; the two share what lies below the session model, so a consolidation is possible (§ 8). The measurements are repeatable from Appendix § 13 and the results repository.
Acknowledgements
We thank Kevin Kappelmann for fruitful discussions and comments on an early draft of this report, and Lawrence Paulson for pointing us to the agentic workflow of AutoCorrode. The Isabelle Pool Server was developed as part of the Sketchpad project, supported by Renaissance Philanthropy’s AI for Math Fund.
IPS in PIDE’s terms
This appendix describes the service of § 4 for a reader who knows PIDE [17]. IPS is a direct PIDE frontend. A session is a Headless.Session, the class behind isabelle server; the backend starts it, loads its imports and edits its document through the Isabelle/Scala API. The session’s theory is one node of that document. Its text is held in memory, with no file behind it, and the server appends to it or replaces it. A chunk is one insert at the end of the node, and a rollback is the edit that removes it. The node is required with an empty visible range, its perspective, so PIDE checks every command, reports errors and warnings, and runs no print function. Proof states, facts in scope and Sledgehammer are query operations run as overlays on the node’s last command and removed afterwards; the document is never edited for them. A call returns when the submission has settled. A checkpoint records the node’s text and the status of its commands; a restore is the edit that takes the text back, so PIDE’s incremental checking keeps everything before the edit point and re-checks nothing. A restore call returns when its edit is accepted, before the document has settled; the next call waits until it has, which is why § 5.2 counts that settling delay in the branch cost. Proof checking is parallel inside a chunk: in a headless session any parallel-proofs above 0 runs at Isabelle’s interactive level, which forks the proofs of a command, so the value the experiments set (2) is the same checking as 1.
Quickstart and API summary
Running the server. IPS ships as one container image parameterised by Isabelle version, with a setup script for Docker and one for a native Linux host (optionally as a systemd unit). On a machine with Docker:
git clone https://github.com/Deep-LCF/Isabelle-Pool-Server.git
cd Isabelle-Pool-Server
./deploy/setup.sh --verify # writes .env, builds, starts, proves True
curl http://localhost:8000/readyz # 200 once the gateway JVM is up
./deploy/native_setup.sh --system --systemd --verify # the same, no Docker
The first build downloads Isabelle and compiles the backend (10–30 min); the last line is the native install. The configuration is a set of environment variables in .env (Appendix § 14).
A session from Python. The client is a separate package (pip install -e ./client, httpx only). Every call on a session carries the lease returned when the session was acquired.
from client.async_client import PoolAsyncClient
async with PoolAsyncClient("http://localhost:8000") as client:
s = await client.acquire_session(theories=["Main"], field="HOL")
sid, lease = s["session_id"], s["lease_id"]
try:
await client.enter_theory(sid, "T", imports=["Main"], lease_id=lease)
cp = await client.save_checkpoint(sid, lease_id=lease)
r = await client.verify_chunk(
sid, ’lemma l: "x + 0 = x" by simp’, timeout=10.0, lease_id=lease)
proved = r["success"] and not r["proof_open"] and not r["used_sorry"]
if not proved:
await client.restore_checkpoint(
sid, cp["checkpoint_id"], lease_id=lease)
finally:
await client.release_session(sid, lease_id=lease)
verify_chunk returns the verdict flags and the per-command report:
{"success": true, "proof_open": false, "pending_qed": false,
"used_sorry": false, "timed_out": false, "stuck_line": null,
"execution_time": 0.41,
"commands": [{"index": 0, "line": 1, "kind": "lemma", "status": "ok",
"messages": []}, ...]}
On a budget expiry timed_out is true and stuck_line names the command still running; pending_qed says the goals are discharged and only the closing qed is missing. load_document replaces the whole document, rollback removes the last submission, and verify_bigstep_text is the batch build.
The stepwise MCP server (13 tools). enter_theory, verify_chunk, proof_state, source, diagnostic, sledgehammer, checkpoint, restore, rollback, close_theory, verify_batch, list_my_sessions, memory_status. verify_chunk is the only tool that executes Isar; it returns the flags above as one text line, then one line per failed or running command (every command with detail=true), and a note when the proof is still open or only qed is missing.
The file MCP server. The agent edits a theory file with its own tools; the server re-reads it on every call and keeps the session in step. Tools cover open, close and sync; diagnostics, goal and proof state at a line; source, hover and definition; local and global facts; query and Sledgehammer; checkpoint, restore, rollback and history; multi-attempt and run-code on scratch sessions; and heap build and status. The naming follows lean-lsp-mcp [6].
The HTTP API. Served with an OpenAPI document at /openapi.json. The routes a client uses in order: acquire or create a session; submit a command, a chunk or a whole document; read state, goals or facts; checkpoint, restore, rollback; release. The whole-file check is a stateless route that takes a theory and returns a verdict; batch fan-out is the stepwise MCP’s verify_batch, not a route. /api/v1/memory reports the memory facts behind every admission decision.
Repeating the measurements. Appendix § 13 says where the scripts and the raw results are; SERVER_GUIDE.md in evaluation/training_metrics/ is the step-by-step procedure for a fresh Ubuntu host.
Experimental details
Everything here is reproducible from the IPS repository (evaluation/memory_scaling/ for the serving runs, evaluation/training_metrics/ for the training runs; each one config.yaml, numbered scripts and a deterministic analysis). The raw data is in the results repository4: every cell’s manifest (phases, per-session timings, environment, workload checksum, the tool’s own report), its per-second samples, the per-cell tables and the plots; the memory-scaling tree also as release asset memscale-2026-10-21b of the main repository. This appendix keeps the method and the two series the body’s figures and fits are drawn from. The per-cell tables (E1 medians per cell, E3 per session, the E2 and E4 records) are results/report-raw-tables.md in the results repository.
Environment. One x86-64 host with 32 logical CPUs and 128 GB of memory, Ubuntu with cgroup v2, nothing else running; both tools native, as one unprivileged user, from one Isabelle2026 installation (the second release candidate, § 8) with the same 9 GB memory cap on every prover process (ML_OPTIONS --maxheap 9216). IPS at commit 6e4dd6f for the memory-scaling runs and 1fe270f for the training runs (the census at 28600bb for 82 theories and cb03c0e for the 11 whose blocks a splitter fix re-cut, a server-side change judged not to touch a replay; the analysis at e144f97), memory management on at its defaults, pool size 40, no pre-warmed sessions, parallel_proofs=2; the training runs with a 1.5 GB estimate per HOL-Analysis session and four batch builds at a time; a fresh server in its own systemd scope for every memory-scaling cell. PIDE MCP at commit dd46315 of 19 October 2026 (server 0.1.0; the repository is now kappelmann/isabelle-pide-ai), reached over MCP on standard input and output, with its default options, among them pide_mcp_range_context = 50, and parallel-proofs=2 on its sessions in the training rows.
Memory-scaling workloads. One fixed theory per heap, loaded whole into every session: CTL.thy (HOL, 288 commands), Termination.thy (HOL-Library), Sum_Topology.thy (HOL-Analysis, 163 lines, importing HOL-Analysis.Abstract_Topology). The re-check inserts lemma memscale_p5_done: "True" by simp after the first paragraph under the same theory name, so every later command is re-checked on the incremental path.
Memory measure. We report the sum of PSS over the tool’s processes, read once a second from /proc/<pid>/smaps_rollup: PSS counts a page shared by processes as for each, so the sum is the physical memory the tool occupies whatever is shared. Sharing between provers is small, since Poly/ML reads the heap image into private memory. Host used (MemTotal − MemAvailable) is the independent check: its per-session slope agrees with the PSS slope within 2 % in every series of Table 7, and its intercept sits 0.54–0.67 GB below the PSS intercept in every series, most likely the file-backed pages the JVMs keep resident, which PSS counts and MemAvailable treats as reclaimable; the table therefore reports the PSS fit, and the host fits are in the results repository. The cgroup counters of the systemd scope the tool runs in (memory.current, memory.peak, oom_kill) are what a memory limit is enforced on and carry the E3 result.
E1, capacity. For each tool, heap and in {1, 2, 4, 8, 16, 32}, three repeats: P0 baseline (60 s; 10 s after the first cell of a series), P1 start sessions, P2 idle 60 s, P3 load the theory in each session, P4 settle 60 s, P5 re-check all concurrently, P6 stop and a 30 s tail. Steady state is the median of the last 30 s of a phase; host used is the delta over the cell’s own P0; the fits of Table 7 are least squares over the 18 cells of a series. The lower row of Figure 5 is the P5 series, the re-check peak; the per-cell values are results/tables/cells.csv in the results repository.
Perspective check. To test the attribution of § 7, IPS was rebuilt with its nodes’ visible range set to the whole text (the two places the backend sets the perspective) and run in its container on a 16 GB machine with Isabelle2026. Four HOL sessions each loaded the E1 HOL theory through load_document and settled for 60 s; then each prover’s PSS was read from /proc. The theory’s one axiomatization, a constant declaration without axioms, was written as consts, because the command guard now refuses the word. Empty range: 816–819 MB in all four provers. Full range, two repeats: 1007–1176 and 665–1251 MB. Proof-state output (editor_output_state) was off throughout, as in both tools’ defaults. The script and the three samples are results/perspective-check/ in the results repository.
E2, E3, E4. E2: 4 IPS sessions, released and re-acquired with the same imports for 3 rounds. E3: IPS sessions added one at a time, the server in a systemd scope with MemoryMax=48G, until a refusal, a failed load or an oom_kill; then half the sessions released and one more added. E4: 8 HOL sessions, the newest session’s prover killed with SIGKILL, every session asked to answer within 30 s (IPS: a re-check through load_document; PIDE MCP: get_state). § 6.2 reports the outcomes.
Training workloads. The raw results of the runs of § 5.2, every cell’s manifest, per-block records, memory and CPU samples and the server’s reports, are the training-metrics/ tree of the results repository.
Corpus and census. The 94 theories of HOL-Analysis in Isabelle2026 less the collector Analysis.thy, 93, with their imports qualified, each split into top-level blocks (11984; a block is one top-level command with its proof). The census replays every theory once at one worker, each on a session with the theory’s imports, with a 600 s budget per block; it proved all 11 984 blocks, and its per-block times and command counts key every later measurement. A block’s budget in the experiments is s for census commands.
throughput, latency. workers (1 to 32, three repeats), 8 theories each from a seeded assignment that is a prefix across , each theory on a fresh session acquired with its imports, replayed block by block with verify_chunk; the clock runs from the first theory to the last. Table 9 is the full series. T2, the latency of one call, is read from the T1 cells. The PIDE MCP runs use the same assignment at and 8 through a loop of our own: a session per theory, a file from create_scratch, one edit per block carrying the whole text, and get_state every 0.1 s until the block has settled. A block that does not settle has to be edited out by the client, which our driver did not do, so such a block ended its theory (one per cell). Isabelle requires a theory to bear the name of its file, so each theory took the scratch file’s name, and the 1% of blocks that refer to their own theory by its library name fail for that reason; the choice of file was the driver’s, since PIDE MCP does not name theories. The loop also closes the text with end after every edit, so a block that opens an instantiation is counted as failed, and so are some blocks after it in the same theory; the two cases together are under 1% of the blocks. For the latency comparison of § 7 its blocks pair with IPS’s by block hash over the one-worker cells, a block’s time the median of the repeats that ran it, and only blocks proved under both tools count: 2 966 of 2 985, one having timed out and 18 being proved under one tool only. branch cost. 200 proof states, a third each early, in the middle and late in their theories, each on a session created for it: checkpoint, verify the next block, three restores each followed by the block, two rollbacks each followed by the block, then the prefix replayed in a new theory node and in a new session. verdict flags. Part A: each theory as written, the census verdict against verify_bigstep_text (isabelle build on the heap). Part B: 200 census-proved theorem blocks, seeded, in four forms (unchanged, sorry, truncated before the closing step, first fact name undefined), checked in-session at the block’s position under its budget and rolled back, and in a batch build with the form spliced into its theory, one mutation per build. bounded step. 50 looping steps of each kind, a rewriting loop (apply (subst add.commute)+ on an additive goal) and a metis call with up to 56 facts on a false goal, placed before census-proved theorems of 9 theories, tried at 5 s; of those that looped, up to 10 of each kind tried again at 30 s and up to 3 at 120 s; a 1.5 s pause after every attempt for the memory sampler; at , two looping steps per worker, under 120 s, among the worker’s theories (16, of which one rewriting step did not loop). rollout loops. workers (1, 8, 32; two repeats), three theories each from the T1 assignment; whole-proof: the first 8 proved theorems of each theory, 8 candidates each (the corpus proof, by auto and by simp twice, sorry, the proof truncated, a method that fails at once) under 20 s, one that stays in the document discarded by a restore; step-level: the first 4 proved theorems of 2 to 40 proof commands, at each command two alternatives from {apply auto, apply simp, apply -, by auto, by simp} under 10 s before the corpus command. Every cell polls the server’s memory tier and samples memory and CPU per process once a second.
Deployment notes and security envelope
Sizing. About 30 GB of disk for Isabelle and the heaps; memory as measured in § 6.2: the pool’s fixed cost plus one per-session cost of Table 7 for each concurrent session, with the admission gate’s margin on top. The pool size, the number of sessions started at boot, the idle and lease timeouts, the memory thresholds and session estimates, the prover memory cap and the per-call budgets are environment variables documented in the repository’s .env.example.
Setup. The Dockerfile selects the native architecture and falls back across mirrors; Appendix B gives the setup scripts. Readiness and liveness are exposed as HTTP probes; metrics are exposed for Prometheus, with a Grafana dashboard in the repository. The memory gate reads the server’s own cgroup, so it sees a Docker memory limit and a systemd MemoryMax alike.
Security envelope. There is no authentication (§ 8), and the administrative page inlines its token, a known and deferred issue. Every endpoint that accepts Isar text rejects ML, setup, file-IO and axiomatization commands unless the operator switches the policy off. The diagnostic endpoint accepts only an allowlist of keywords. Heap project paths are confined to configured roots, and destructive heap operations require an administrative token. Measured against the MCP specification’s requirements for input validation, access control and rate limiting [13], IPS meets the first in part and neither of the others. PIDE MCP states that it includes no safety measures and refers to a separate set of sandboxed setups [15]. PIDE AI’s TCP port, like the IPS port, has no authentication.
Where to look. The backend (session start, document edits, overlay queries, the per-command report) is server/repl; the HTTP layer, the pool and the memory monitor server/app; the two MCP servers mcp_servers; the experiments evaluation. docs/DESIGN_CHOICES.md in the repository explains how the server keeps the guarantees of § 4: the tiers, the owned document, verification, the overlay queries, and the pool with its admission gate.
A worked example: agents in the loop (anecdote)
The Isabelle-Humanize project5 runs a builder/reviewer loop (the RLCR flow of humanize26) on competition problems. Two agent seats (opencode on DeepSeek V4 Pro in both roles) talk to IPS through the file MCP server over streamable HTTP, and the flow uses the IPS whole-file check as the arbiter: a problem counts as solved only when isabelle build passes with no sorry or oops and the target theorem present. Three problems were solved end to end: Putnam 2023 A1 (2 review rounds, 3.3 h wall), Putnam 2023 B5 (5 rounds, 14.3 h, with one regression and a replan) and IMO 2026 P1 (one round, a 1059-line theory with all five sorrys discharged, 4.2 h). A rerun of IMO 2026 P1 on the IPS build with incremental document sync and the memory work of this report halved the wall time (7 391 against 14 949 s) and the agent-active time (117 against 248 min), cut MCP timeouts from 12 to 0 and diagnostics calls from 47 to 18 at a median of 0.3 s, and the arbiter build from 44 to 11 s. The project’s own caveats apply: one run per problem, a prompt rule that changed between the two IMO runs, and a fourth “solve” (Putnam 2023 B6) excluded because the benchmark’s Isabelle statement is degenerate. This is an example, not an evaluation: a few problems with one model measure the agent and the prompt as much as the connector. It shows that a service built for the first two scenarios also serves the third, and that the service’s latency reaches the agent’s wall clock.
References
- [1]Leni Aniva, Chuyue Sun, Brando Miranda, Clark Barrett, and Sanmi Koyejo. Pantograph: A machine-to-machine interaction interface for advanced theorem proving, high level reasoning, and data extraction in Lean 4. In Arie Gurfinkel and Marijn Heule, editors, Tools and Algorithms for the Construction and Analysis of Systems (TACAS 2025), pages 104–123. Springer, 2025. doi: 10.1007/978-3-031-90643-5_6.DOI
- [2]Anthropic. Introducing the Model Context Protocol. Announcement, 2024. URL https://www.anthropic.com/news/model-context-protocol.
- [3]Hanno Becker, Nathan Chong, Robert Dockins, Jim Grundy, Jason Z. S. Hu, Ike Mulder, Dominic P. Mulligan, Paul Mure, Lawrence C. Paulson, and Konrad Slind. AutoCorrode software verification framework for Isabelle/HOL. https://github.com/awslabs/autocorrode, 2025.
- [4]Dustin Bryant, Jonathan Julián Huerta y Munive, Cezary Kaliszyk, and Josef Urban. Munkres’ general topology autoformalized in Isabelle/HOL. arXiv preprint arXiv:2604.07455, 2026. URL https://arxiv.org/abs/2604.07455.
- [5]Marco Dos Santos, Haiming Wang, Hugues de Saxcé, Ran Wang, Mantas Baksys, Mert Unsal, Junqi Liu, Zhengying Liu, and Jia Li. Kimina Lean server: Technical report. arXiv preprint arXiv:2504.21230, 2025. URL https://arxiv.org/abs/2504.21230. Presented as “Kimina Lean Server: A High-Performance Lean Server for Large-Scale Verification” at the MATH-AI Workshop, NeurIPS 2025. Software: https://github.com/project-numina/kimina-lean-server.
- [6]Oliver Dressler. Lean LSP MCP: Tools for agentic interaction with the Lean theorem prover. Git repository, 2026. URL https://github.com/o0o0o0o/lean-lsp-mcp.
- [7]Albert Qiaochu Jiang, Wenda Li, Jesse Michael Han, and Yuhuai Wu. LISA: Language models of Isabelle proofs. In Artificial Intelligence and Theorem Proving (AITP 2021), 2021. URL https://github.com/albertqjiang/Portal-to-ISAbelle. Software: PISA.
- [8]Albert Qiaochu Jiang, Wenda Li, Szymon Tworkowski, Konrad Czechoswki, Tomasz Odrzygóźdź, Piotr Miłoś, Yuhuai Wu, and Mateja Jamnik. Thor: Wielding hammers to integrate language models and automated theorem provers. In Advances in Neural Information Processing Systems 35 (NeurIPS 2022), 2022. arXiv:2205.10893.arxiv.org/abs/2205.10893
- [9]Albert Qiaochu Jiang, Sean Welleck, Jin Peng Zhou, Wenda Li, Jiacheng Liu, Mateja Jamnik, Timothée Lacroix, Yuhuai Wu, and Guillaume Lample. Draft, sketch, and prove: Guiding formal theorem provers with informal proofs. In International Conference on Learning Representations (ICLR 2023), 2023. arXiv:2210.12283.arxiv.org/abs/2210.12283
- [10]Kevin Kappelmann. PIDE MCP: Connecting AI agents to Isabelle. Preprint, HAL hal-05748384, submitted 14 Sep 2026. Accepted at the Workshop on Artificial Intelligence for Mathematics, CICM 2026. Software since renamed PIDE AI, https://github.com/kappelmann/isabelle-pide-ai; measured here at commit dd46315 (2026-10-19), 2026. URL https://hal.science/hal-05748384.DOI
- [11]Lean FRO. A read-eval-print-loop for Lean 4. Git repository, 2023. URL https://github.com/leanprover-community/repl. Accessed 2026-11-17.
- [12]Maciej Mikuła, Szymon Tworkowski, Szymon Antoniak, Bartosz Piotrowski, Albert Qiaochu Jiang, Jin Peng Zhou, Christian Szegedy, Łukasz Kuciński, Piotr Miłoś, and Yuhuai Wu. Magushammer: A transformer-based approach to premise selection. In International Conference on Learning Representations (ICLR 2024), 2024. arXiv:2303.04488.
- [13]Model Context Protocol. Specification 2025-11-25, 2025. URL https://modelcontextprotocol.io/specification/2025-11-25.
- [14]Z. Z. Ren, Zhihong Shao, Junxiao Song, Huajian Xin, Haocheng Wang, Wanjia Zhao, Liyue Zhang, Zhe Fu, Qihao Zhu, Dejian Yang, Z. F. Wu, Zhibin Gou, Shirong Ma, Hongxuan Tang, Yuxuan Liu, Wenjun Gao, Daya Guo, and Chong Ruan. DeepSeek-Prover-V2: Advancing formal mathematical reasoning via reinforcement learning for subgoal decomposition. arXiv preprint arXiv:2504.21801, 2025. URL https://arxiv.org/abs/2504.21801.
- [15]Maximilian Schäffeler, Lukas Stevens, Kevin Kappelmann, Mohammad Abdulaziz, Andrei Popescu, and Dmitry Traytel. Safe agentic workflows for Isabelle. In Isabelle Workshop 2026 (associated with ITP 2026 and IJCAR 2026, FLoC 2026), Lisbon, Portugal, July 2026. URL https://files.sketis.net/Isabelle_Workshop_2026/Isabelle_2026_paper9.pdf.DOI
- [16]Makarius Wenzel. READ-EVAL-PRINT in parallel and asynchronous proof-checking. In Cezary Kaliszyk and Christoph Lüth, editors, Proceedings of the 10th International Workshop on User Interfaces for Theorem Provers (UITP 2012), volume 118 of EPTCS, pages 57–71, 2012. doi: 10.4204/EPTCS.118.4.DOI
- [17]Makarius Wenzel. Asynchronous user interaction and tool integration in Isabelle/PIDE. In Gerwin Klein and Ruben Gamboa, editors, Interactive Theorem Proving (ITP 2014), volume 8558 of LNCS, pages 515–530. Springer, 2014. doi: 10.1007/978-3-319-08970-6_33.DOI
- [18]Makarius Wenzel. The Isabelle/Isar Reference Manual, 2026. URL https://isabelle.in.tum.de/doc/isar-ref.pdf.
- [19]Makarius Wenzel. Isabelle/jEdit, 2026. URL https://isabelle.in.tum.de/doc/jedit.pdf.
- [20]Makarius Wenzel. The Isabelle System Manual, 2026. URL https://isabelle.in.tum.de/doc/system.pdf.
- [21]Marcin Wrochna. QIsabelle: Query the Isabelle proof assistant with Python. Git repository, 2024. URL https://github.com/marcinwrochna/qisabelle. Accessed 2026-11-17.
- [22]Huajian Xin, Z. Z. Ren, Junxiao Song, Zhihong Shao, Wanjia Zhao, Haocheng Wang, Bo Liu, Liyue Zhang, Xuan Lu, Qiushi Du, Wenjun Gao, Qihao Zhu, Dejian Yang, Zhibin Gou, Z. F. Wu, Fuli Luo, and Chong Ruan. DeepSeek-Prover-V1.5: Harnessing proof assistant feedback for reinforcement learning and Monte-Carlo tree search. arXiv preprint arXiv:2408.08152, 2024. URL https://arxiv.org/abs/2408.08152.
- [23]Jimmy Xin, Alex Schneidman, Chris Cummins, Karun Ram, Srihari Ganesh, and Jannis Limpberg. AXLE: A cloud infrastructure for Lean 4 theorem proving utilities. arXiv preprint arXiv:2606.26442, 2026. URL https://arxiv.org/abs/2606.26442. Poster at the 3rd AI for Math Workshop, ICML 2026. Software: https://axle.axiomath.ai.
- [24]Qiyuan Xu. Isabelle-MCP: an MCP server bridging AI agents with the Isabelle proof assistant via its LSP/PIDE interface. Software, version 0.7.0, https://github.com/xqyww123/Isabelle-MCP, 2026. Accessed 2026-10-31.
- [25]Qiyuan Xu, Joshua Ong Jun Leang, Renxi Wang, Haonan Li, Wenda Li, Luke Ong, and Conrad Watt. AoA: Theorem proving agent over abstract syntax tree of redesigned language. arXiv:2607.16372, 2026. URL https://arxiv.org/abs/2607.16372.