WorksComputer Science
MITRA-Swarm: Injecting Explicit Linguistic Structure into Foundation Models for Classical Machine Translation
Abstract
Classical Sanskrit poses three challenges unresolved by single-call monolithic models: morphophonemic fusion Sandhi, dense nominal compounding Samāsa, and discourse-level zero-anaphora forced by poetic meter. We present MITRA-Swarm, a multi-agent architecture injecting explicit linguistic structure into an otherwise monolithic pipeline: a Sandhi-splitting agent, a parallel array of generalist gemma-4-26B-A4B-it and domain-specialized gemma-2-mitra-it translators, and a discourse-aware Ellipsis Restorer that consults corpus-derived register tags to avoid inventing active-voice subjects in impersonal legal and philosophical registers. Two separated judges evaluate quality: a routing judge (, same model family as the generators) driving an internal critique loop, and an independent reporting judge (gpt-4o-mini, no shared vendor lineage) scoring every output exactly once. On a stratified benchmark of 1,000 verses across five registers (23,990 scored verdicts, 12-condition pre-registered ablation), the full swarm reaches MQM against a monolithic baseline of (, , 95% CI ), and isolated discourse restoration alone accounts for MQM (), outperforming a call-count-matched self-refinement baseline by MQM (). Severe per-verse failures (MQM) fall from to , at roughly the per-verse inference cost of the baseline. Additionally, scores its own family's outputs - MQM higher than on every one of the 12 conditions, empirically motivating the two-judge split; and a preflight screen of a 192,151-entry retrieval index against all 12 evaluation works verifies 0 reference leaks across 269 runtime checks. These are small gains on a metric saturating near 85-86.
Introduction
Sanskrit is the primary linguistic repository for a vast corpus of classical Indian philosophy, science, and literature, yet remains severely under-resourced in modern NLP: the lack of large parallel corpora and the language’s grammatical complexity have historically limited MT quality [8, 3]. Foundation models trained overwhelmingly on high-resource, concatenative languages do not, by default, carry the inductive biases classical Sanskrit requires, i.e. no mechanism for undoing phonetic fusion at word boundaries, no structured representation of deeply nested compounds, and no discourse memory built to restore an argument elided clauses earlier for metrical reasons, which in turn produces translations marred by omission, semantic drift, and hallucination [12].
Three linguistic properties make this failure systematic rather than incidental:
Sandhi (Morphophonemic Fusion): phonetic mergers at word boundaries obscure lexical boundaries without pre-translation segmentation, causing out-of-vocabulary and dictionary-lookup failures.
Samāsa (Nominal Compounding): compounds, sometimes spanning entire clauses, leave the syntactic and semantic relations between their constituents implicit.
High Ellipsis (Zero Anaphora): metrical verse systematically omits subjects, objects, even verbs, requiring inference from the surrounding discourse.
A monolithic model attempts segmentation, reordering, lexical translation, and discourse-level argument restoration in a single generation call; without explicit modular guidance it loses local morphosemantic structure, hallucinates relations between compound members, and invents target-language arguments rather than interrogating the source. MITRA-Swarm instead injects each linguistic principle as an explicit, separately-verifiable pipeline stage:
A morphological Sandhi Parser decomposes phonetic mergers and compounds into a structured morphosemantic guide before translation.
A parallel generator array of a generalist frontier model (gemma-4-26B-A4B-it) and a domain-specialized fine-tune (gemma-2-mitra-it) compares specialization against raw scale rather than assuming it.
A discourse-level Ellipsis Restorer with register-aware voice guarding (§ 3.4) restores zero-anaphora referents in narrative registers while suppressing invented active-voice subjects in impersonal legal (Dharmasāstra) and philosophical (Sūtra) injunctions.
A two-judge quality architecture: a routing judge (shares a model family with the generators) drives an internal critique loop; an independent reporting judge (shares no vendor lineage with any generator or ) scores every finished output exactly once, off the critical path, for the numbers in this paper.
When ’s estimate falls below threshold, a Critique Agent diagnoses errors, a Tool Selector queries whatever retrieval tools are genuinely backed by data, and a Refinement Agent rewrites the translation.
Our contributions:
Architecture. Agents that each target one linguistic failure mode of monolithic models: fusion, compounding, and discourse ellipsis, rather than leaving all three to a single unguided generation call.
Empirical scale and significance. A pre-registered 12-condition ablation on verses (23,990 scored verdicts): full swarm beats the monolithic baseline by +0.63 MQM () and a call-count-matched self-refinement baseline by +0.51 MQM (), via paired approximate randomization with Holm–Bonferroni correction and a measured noise floor rather than an uncorrected bootstrap-tail probability.
An isolated discourse-syntax finding. Ablating only the Ellipsis Restorer costs MQM (), representing the single largest individually-significant component effect, and the clearest evidence that a linguistically-motivated discourse module, not scale or generic self-critique, is doing the work.
Evaluation and decontamination rigor. We directly quantify LLM-judge self-preference ( scores its own family’s outputs 2–7 MQM higher than on every condition, which is why only is reported) and screen a 192,151-entry retrieval index for reference leakage (0 leaks / 269 runtime checks).
Related Work
Sanskrit Computational Linguistics Prior work builds on foundational lexical-database and morphological-analysis resources [11], and more recently on neural sequence-to-sequence splitters [4] and transformer tokenizers such as TransLIST (Sandhan et al., 2022), integrated into tools like SanskritShala (Sandhan et al., 2023) and corpora such as the Digital Corpus of Sanskrit [9] (Hellwig 2021).
LLMs for Machine Translation General-purpose LLMs excel on high-resource languages [10] but struggle where parallel data is scarce [8]. Two-stage fine-tuning, monolingual pretraining followed by a small amount of high-quality parallel data (ALMA, Xu et al. (2024)), motivates our domain-aligned gemma-2-mitra-it backbone [1]; scaling efforts like NLLB-200 [3] expand coverage without addressing classical-language-specific needs like zero-anaphora resolution (Wang et al., 2023).
Multi-Agent Systems and Self-Refinement Multi-agent debate among independent LLM instances improves factuality and reasoning [5]; TransAgents (Wu et al., 2025) showed multi-role literary translation with specialized agents preferred by humans despite lower n-gram scores. Self-refinement [17] is directly implemented, call-count-matched, as our self_refine ablation. Tool-augmented reasoning (Yao et al., 2023; Schick et al., 2023) and retrieval augmentation [13] motivate § 3.6, where we report what happens when tool-use is restricted to tools actually backed by data.
Reference-Free Evaluation and Judge Reliability Reference-based metrics have documented validity limits (Reiter, 2018); reference-free QE (Rei et al., 2022) and fine-grained error-span detection [7] address this, as do LLM-as-judge frameworks [15 a] aided by chain-of-thought prompting (Wei et al., 2022). Such judges also exhibit documented biases like self-preference (Zheng et al., 2023): a judge sharing a model family with the system it evaluates is one instance, and § 5.2 measures it directly rather than assuming it away.
Methodology
The MITRA-Swarm pipeline decomposes translation into chunking, morphosemantic preprocessing, parallel drafting, discourse-aware argument restoration, automated quality routing, and agentic refinement, illustrated in Figure 1 and detailed with exact prompts in Appendix § 9.

Figure 1. System architecture of the MITRA-Swarm pipeline. (same model family as the generators) drives the internal critique loop; (independent vendor, gpt-4o-mini) scores the finished output exactly once, off the critical path, and is the only judge whose scores are reported as results (§ 5.2).
Document Segmentation (Chunking)
To manage context-window limits and preserve local discourse cohesion [16], the source document is segmented at classical sentence/verse terminators (, ), a sliding token window (, overlap ) for unpunctuated prose, or an LLM-driven semantic chunker for narrative boundaries.
Morphological Pre-processing (Sandhi Parser)
A Sandhi Parser Agent decomposes phonetic mergers and compounds (Samāsa; [4] (Sandhan et al. 2022)) into a structured morphosemantic guide, threaded through a typed TranslationRequest into every downstream translator call. gemma-4-26B-A4B-it accepts the guide via its chat template; gemma-2-mitra-it uses a legacy completion-style prompt for which guide injection was evaluated empirically and disabled, so the guide_only/guide_ellipsis contrasts reflect that per-model choice rather than a uniform assumption.
Parallel Translation Array
The chunk is translated concurrently by a parallel array of models,
comprising the domain-specialized gemma-2-mitra-it ([1]) and the generalist frontier gemma-4-26B-A4B-it ([6]), both served locally via vLLM. Each model generates a draft translation independently, and the routing judge selects among candidates.
Ellipsis and Zero-Anaphora Argument Restoration with Register-Aware Voice Guarding
The Ellipsis Restorer operates over a sliding discourse window of the preceding/succeeding chunks, , identifying zero-anaphora and interpolating the implied argument. Naively restoring an elided subject risks inventing a third-person actor in passages that are impersonal or injunctive by design, where restoration could convert a Dharmaśāstra injunction into an active clause with a hallucinated subject. MITRA-Swarm mitigates this with a voice-preservation guard: every source work ID is mapped, via a corpus-derived, manually verified register table (registers.json; six registers (vedic, epic, legal, sutra, kavya, shastra), covering over 100% of the corpus’s 52 work IDs, rather than inferred from a keyword heuristic), to a voice_preserving flag. Legal and sutra are flagged voice-preserving as characteristically impersonal/passive by genre; the Ellipsis Restorer receives the literal genre hint “legal” for any work so flagged, and the raw register name otherwise, which measurably changes its restoration behavior (Section A) without a separate model or prompt per register.
Two-Judge Quality Architecture
Quality estimation is split into two roles, enforced at startup by an automated model-family check. (Routing Judge): a locally-hosted gemma-4-26B-A4B-it instance, sharing a model family with the generators, scores intermediate drafts against threshold to decide whether the critique loop fires; because it shares a family with what it evaluates and is queried repeatedly during generation, its scores are an internal control signal only, never reported as a result. (Reporting Judge): an independent offline judge (openai: gpt-4o-mini), sharing no vendor lineage with either generator or , invoked exactly once per finished output on a separate machine from the generation cluster. Every number in Section 5 is a score.
Both judges use a GEMBA-style rubric returning structured error spans (category, severity, verbatim span, explanation) rather than a bare number. The headline MQM is recomputed deterministically from that list using fixed severity weights (critical , major , minor , floored at ), not the model’s own top-line arithmetic, which is retained separately for the judge-validation diagnostic (§ 5.2). A failed judge call yields a missing score, excluded from all aggregates, rather than a sentinel that would otherwise depress the mean and minimum and spuriously trigger the critique loop.
Agentic Feedback and Tool Retrieval
If , a Critique Agent diagnoses errors, a Tool Selector queries whichever tools are genuinely backed by data, and a Refinement Agent synthesizes the draft, critique, and retrieved context into a revised translation, re-scored up to iterations. The inventory is reported as offered: a tool with no backing index is absent rather than present and silently returning a canned string. On the air-gapped GH200 cluster used here, that inventory reduces to TRANSLATION_MEMORY, which performs fuzzy character-6-gram matching against a decontaminated corpus index (§ 4.2). A DICTIONARY tool (offline lexicon lookup, with entries OCR-sanitized to strip 19th-century scanning artifacts before reaching the Refinement Agent) exists but was not backed by a snapshot in this run; ALIGNMENT and live SEARCH require, respectively, an alignment index and network access unavailable offline. A Dedup Agent stitches overlapping segments pairwise, , feeding only the tail of the running translation (last characters) so per-step input stays bounded rather than growing with the number of merged segments.
Experimental Setup
Evaluation Dataset and Stratification
Experiments are conducted on a stratified sample of verses drawn, proportionally by register, from the 5,552-verse test partition (12 works) of the MitraSamgraha corpus ([2]). Table 1 reports the realised sample against the full test-partition population for each of the five registers represented in this partition; the register-to-work-ID mapping is manually verified (100% work-ID coverage, 0 unmapped) rather than inferred from a keyword heuristic.
| Register | Description | Pop. | Sample | Voice-preserv. |
| epic | Itihāsa/Purāṇa narrative | 1,720 | 310 | – |
| sutra | Philosophical sūtra, commentary | 1,488 | 268 | ✓ |
| vedic | Vedic saṃhitā/upaniṣad; archaic | 1,000 | 180 | – |
| kavya | Ornate metrical poetry | 844 | 152 | – |
| legal | Dharmaśāstra, nibandha | 500 | 90 | ✓ |
| Total | 5,552 | 1,000 |
Table 1. Register composition of the evaluation sample, proportional to its population in the 5,552-verse test partition. Voice-preserving marks registers routed to the impersonal-voice guard in the Ellipsis Restorer (§ 3.4).
This sample size balances statistical power with the cost of a 12-condition ablation (12,000 full pipeline runs). Because the corpus is drawn from 12 works with substantial internal stylistic variation, register-level results are an average over that variation, not a claim of uniformity within it.
Data Decontamination and Preflight Reference-Echo Screening
To ensure rigorous evaluation and prevent data leakage, the TRANSLATION_MEMORY index contains 192,151 segment-level entries, with all 12 evaluation work IDs excluded at build time (125,665 entries excluded; 0 evaluation work IDs reachable in the live index). A runtime reference-echo detector computes character 6-gram F-score similarity between every retrieved string and the gold reference that triggered it, flagging anything ; across a 200-item preflight check and 269 runtime retrievals, the maximum observed similarity is 0.375, with 0 echoes flagged. We separately audit translator outputs against the gold reference for suspected training-data memorization (independent of retrieval): 0.21% of gemma and 0.70% of mitra outputs exceed a 0.6 "suspect" threshold, with 0.15% of mitra outputs exceeding a stricter 0.8 threshold; inspection shows short sentences (e.g., "He makes offering(s) to four deities") with only a one-word paraphrase gap, not retrieval-time leakage.
Models and Hardware
The generator array comprises gemma-4-26B-A4B-it (26B-parameter MoE, 4B active) and gemma-2-mitra-it, both served locally via vLLM on NVIDIA GH200 nodes; runs on the same endpoint. runs offline, after generation, against the OpenAI API (gpt-4o-mini), since the generation cluster is air-gapped.
Hyperparameters and Compute Budget
Translation agents decode at temperature 0.0; critique/judge/tool-selector agents at 0.1; serving context window is capped at 4,096 tokens (with a 2,048-token output cap) to optimize batch throughput across verse segments, well within the models’ native context windows. Table 2 compares per-verse token budget and latency for the three conditions central to the main results (full per-condition table in Appendix Table 10), measured directly from run instrumentation, not estimated. Across the full run, generation consumed 17.61M tokens (10.71M prompt, 6.90M completion) across 85,509 LLM calls at a 54.9% cache hit rate; the routing judge alone accounts for 17,713 calls (12.57M tokens), since is queried once per candidate per stage rather than once per verse.
| Configuration | Prompt tok. | Compl. tok. | Total/verse | Latency (s) |
| baseline | 91 | 49 | 140 | 2.5 |
| self_refine | 164 | 89 | 253 | 4.9 |
| bo_n | 455 | 258 | 713 | 12.8 |
| full_swarm | 828 | 462 | 1290 | 7.1 |
Table 2. Per-verse compute, key conditions (measured, not estimated). Full swarm costs 9.2× baseline tokens for +0.63 MQM.
Ablation Matrix
We pre-register a 12-condition ablation matrix (full active-component flags in Appendix Table 8), isolating each architectural component against a monolithic baseline, a paired test-retest control, a placebo re-judging control, and two compute-matched controls (best-of-N and generic self-refinement): baseline (monolithic single-pass, no guide/ellipsis/loop); baseline_dup (different seed; the noise floor); placebo_rejudge (baseline re-judged; judge-side noise only); bo_n (compute-matched best-of-5, temperature 0.7); self_refine (call-count-matched generic self-critique, no role specialization); guide_only (Sandhi guide alone); guide_ellipsis (guide + Ellipsis Restorer, no loop/tools); no_guide, no_ellipsis (the isolated discourse-restoration ablation), no_critique, no_tools (full swarm minus one component each); and full_swarm, the complete architecture (Section 3.4–3.6).
Metrics and Statistical Plan
Quality is MQM, a reference-free [0, 100] score from (§ 3.5). Significance uses a paired approximate randomization test (the preferred test for MT system comparison rather than a bootstrap-tail probability; Riezler and Maxwell, 2005) at , with 95% BCa bootstrap CIs (); all 10 pre-registered contrasts are corrected jointly with Holm–Bonferroni. Because mean MQM is compressed near 85–86 for every condition, we also report a measured noise floor: the mean absolute difference between baseline and baseline_dup is 0.21 MQM, and a contrast is reportable only if it is both significant and exceeds this floor; a significant result smaller than the chance disagreement between two identical systems is a measurement of the harness, not the method.
Results and Analysis
Main Ablation Results
Table 3 reports the full 12-condition matrix; Table 4 reports the 10 pre-registered contrasts derived from it.
| Configuration | MQM | SD | P10 | P25 | Min | %<80 | |
| baseline | 1000 | 85.27 | 5.67 | 80.0 | 84.0 | 35 | 9.0 |
| baseline_dup | 1000 | 85.27 | 5.53 | 80.0 | 84.0 | 35 | 8.5 |
| placebo_rejudge | 1000 | 85.26 | 5.71 | 80.0 | 84.0 | 35 | 8.9 |
| bo_n | 1000 | 85.58 | 5.51 | 80.0 | 84.0 | 40 | 8.7 |
| self_refine | 1000 | 85.39 | 5.56 | 80.0 | 84.0 | 35 | 8.3 |
| guide_only | 1000 | 85.23 | 5.65 | 80.0 | 84.0 | 40 | 8.9 |
| guide_ellipsis | 999 | 85.91 | 5.03 | 80.0 | 84.0 | 60 | 7.4 |
| no_guide | 999 | 85.91 | 5.20 | 80.0 | 84.0 | 60 | 7.1 |
| no_ellipsis | 1000 | 85.47 | 5.22 | 80.0 | 84.0 | 55 | 8.3 |
| no_critique | 999 | 85.91 | 5.03 | 80.0 | 84.0 | 60 | 7.4 |
| no_tools | 999 | 85.89 | 5.29 | 80.0 | 84.0 | 40 | 7.6 |
| full_swarm | 999 | 85.90 | 5.29 | 80.0 | 84.0 | 40 | 7.6 |
Table 3. Ablation results on the stratified MitraSamgraha benchmark. MQM from (gpt-4o-mini); P10/P25 are the 10th/25th percentiles; %< 80 is the severe-failure rate.
| Contrast | MQM | 95% CI | W/L/T | |||
| Full swarm vs monolithic baseline | 999 | +0.63* | [+0.30, +0.97] | 0.002 | +0.12 | 397/294/308 |
| Re-judging alone (placebo) | 1000 | -0.01† | [-0.07, +0.01] | 1.000 | -0.02 | 2/2/996 |
| Full swarm vs best-of-N (compute matched) | 999 | +0.32 | [-0.05, +0.67] | 0.460 | +0.06 | 363/333/303 |
| Full swarm vs self-refine (specialisation) | 999 | +0.51* | [+0.16, +0.85] | 0.029 | +0.09 | 389/301/309 |
| Morphosemantic guide | 999 | -0.01† | [-0.27, +0.23] | 1.000 | -0.00 | 170/179/650 |
| Ellipsis restoration | 999 | +0.44* | [+0.12, +0.73] | 0.029 | +0.09 | 323/232/444 |
| Tool retrieval | 999 | +0.01† | [+0.00, +0.03] | 1.000 | +0.04 | 2/0/997 |
| Critique loop (isolated) | 999 | -0.02† | [-0.14, +0.09] | 1.000 | -0.01 | 24/27/948 |
| Baseline -> guide | 1000 | -0.04† | [-0.38, +0.27] | 1.000 | -0.01 | 252/256/492 |
| Guide -> guide + ellipsis | 999 | +0.69* | [+0.39, +1.00] | 0.001 | +0.14 | 324/221/454 |
Table 4. Pre-registered contrasts, paired approximate randomization test, , Holm–Bonferroni adjusted over the family of 10. * significant and exceeds the 0.21 MQM noise floor. † below the noise floor (not distinguishable from run-to-run variation regardless of nominal -value). * significant after Holm–Bonferroni over 10 pre-registered contrasts. † below the measured noise floor of 0.21 MQM (the difference between two identical pipelines under different decoding seeds) and therefore not distinguishable from run-to-run variation.
Four trends are reportable under the criterion in § 4.6 (significant and above the noise floor).
(1) The full swarm beats the monolithic baseline by +0.63 MQM (, CI [+0.30, +0.97], , 397/294/308 W/L/T), which is real but small in absolute terms (both means sit at 85.3–85.9 of 100), which we report plainly. (2) Removing only the Ellipsis Restorer costs −0.44 MQM relative to the full swarm (, [+0.12, +0.73], 323/232/444 W/L/T), constituting the single largest individually-significant component effect, confirming that discourse restoration operates effectively in isolation. (3) The swarm beats a call-count-matched generic self-refinement baseline by +0.51 MQM (, 389/301/309 W/L/T), attributable to role specialization and discourse/morphology structure rather than extra inference calls alone. (4) Adding the Ellipsis Restorer on top of the Sandhi guide alone yields +0.69 MQM (, [+0.39, +1.00], ), the largest effect size in the matrix, consistent with (2).
The other six contrasts are not reportable, and a null from a well-controlled comparison is itself informative: the Sandhi guide alone, against the full swarm (−0.01) or the baseline (−0.04, guide_only); the critique loop in isolation (−0.02); tool retrieval (+0.01; § 3.6 explains why, given the tool inventory actually available); the placebo re-judging control (−0.01, confirming judge stability under repeat scoring); and full swarm vs. best-of- (+0.32, , numerically positive and above the floor but not significant, though a large enough compute-matched sampling budget may close some of this gap).
These findings replicate under a fully deterministic, reference-based metric. To check they do not depend on a single judge’s preferences, we additionally compute chrF++ (Popović, 2017) via sacrebleu (word order 2) over the same selected outputs. The full swarm’s advantage over baseline holds under this judge-independent metric (29.27 → 30.33 chrF++, ), as does isolated ellipsis restoration (+0.73, ) and the gain over self-refine (+0.78, ); the same noise-floor check that gates the MQM contrasts flags marginal seed-to-seed chrF variance on its own (), smaller than the reported effects but disclosed rather than omitted.
Restricting to baseline, the isolated ellipsis ablation, and the full swarm (full 12-condition breakdown in Appendix Table 9), the full-swarm gain over baseline is largest in vedic (83.6 → 84.5, +0.9; heaviest ellipsis, most archaic morphology), moderate in kavya (+0.4), legal, and sutra (+0.3 each), and flat in epic (+0.0). This pattern is consistent with, though not definitive proof of, the discourse-restoration mechanism in § 3.4: the voice-preserving registers (legal, sutra) do not show the largest numeric gains, since the voice guard changes how arguments are restored (Appendix § 7) rather than guaranteeing the largest score delta.
Error-category counts (mean per segment by MQM category, Appendix Table 11) are dominated by mistranslation and omission in every condition; the full swarm shows a modest omission reduction relative to no_ellipsis/baseline, alongside a small rise in flagged mistranslation spans, since the critique loop rewrites text and not every rewrite is judged an improvement. The tail of the per-verse score distribution (Appendix Figure 2) confirms the system’s effect is concentrated in the low tail (P10, minimum, %< 80 in Table 3) rather than a uniform shift of the whole distribution, since the mean is compressed near 85–86 for every condition.
Judge Validation: Self-Preference and Score-Arithmetic Reliability
Table 5 summarizes the evaluation-methodology findings motivating the two-judge split (full per-condition self-preference in Appendix Table 12). Self-preference is large and universal: on every one of the 12 conditions, scores the output 1.9–6.8 MQM higher than . Reporting ’s own scores, as a single-judge design would, would inflate every headline number by an amount that itself varies by condition, biasing cross-condition comparison; to our knowledge, this is one of the more direct in-pipeline measurements of LLM-judge self-preference reported for a low-resource MT system. The judge’s own arithmetic disagrees with its stated rubric on 96.4% of scored items (mean between the self-reported score and the score recomputed from its own error spans), which is why we report the recomputed score as the metric and both scores as a diagnostic. Recomputing MQM is a deliberate departure from the published GEMBA-MQM protocol, so the two are not numerically interchangeable with prior work using the standard prompt. Test–retest reliability has a within-item SD of 1.45 MQM, smaller than the smallest reportable effect in Table 4; the reporting judge’s failure rate is 0.41%.
| Judge diagnostic | Value |
| Reporting judge | openai:gpt-4o-mini |
| Judge failure rate | 0.41% |
| Arithmetic disagrees with rubric | 96.4% (mean ) |
| Test–retest SD (repeated calls) | 1.45 |
| Self-preference range (min–max, 12 conditions) | +1.94 to +6.79 |
Table 5. Judge validation summary under the zero-shot judge prompt.
Few-shot prompting reduces, but does not eliminate, this arithmetic gap. Motivated by the concern that a zero-shot judge may simply be miscalibrated, we re-score a stratified subsample, the same 100 items per condition, paired across all twelve, with a three-shot variant of the judge prompt. This reduces the arithmetic-disagreement rate from 96.4% to 79.3% (mean : 16.37 → 8.46) but leaves most items still disagreeing, reinforcing that recomputing the score from error spans is not an optional refinement. Zero-shot and few-shot scores on the same items correlate only moderately (Pearson , Spearman ), indicating the judge’s output is itself non-trivially prompt-sensitive. At per condition the pre-registered contrasts do not reach significance here ( is needed to detect a 2-point effect), so we treat this as a calibration diagnostic, not a second significance test.
Table 5: Judge validation summary under the zero-shot judge prompt (the few-shot comparison above is a separate, -per-condition subsample). Self-preference range is over the 12 conditions.
Tool Utility
Table 6 reports usage for the TRANSLATION_MEMORY tool: 5,513 queries, 4.9% hit rate, zero transport errors. Its measured effect (+0.01 MQM, Table 4) is below the noise floor, representing a genuine negative result rather than an artifact: fuzzy translation-memory lookup rarely finds a close precedent for an unseen verse, and when it does, the 26B-parameter generalist model already appears to carry comparable lexical knowledge parametrically.
| Tool | Calls | Hit rate | Errors | Latency (ms) |
| TRANSLATION_MEMORY | 5513 | 0.05 | 0 | 513 |
Table 6. Tool utility, aggregated over all conditions in which the tool inventory was active.
Conclusion
We introduced MITRA-Swarm, a multi-agent architecture injecting explicit linguistic structure (Sandhi splitting, register-aware discourse ellipsis restoration, and a two-judge quality architecture) into an otherwise monolithic foundation-model pipeline for classical Sanskrit. On a pre-registered, stratified benchmark, the swarm produces small but statistically robust gains over a monolithic baseline and a call-count-matched self-refinement baseline, with the isolated discourse-restoration ablation as the single largest component effect. Equally central is what the benchmark says about evaluation itself: LLM-judge self-preference is large and universal (2–7 MQM, every condition), and a preflight decontamination sweep is directly falsifiable and, here, clean. A foundation-model system for a morphologically rich, low-resource classical language should be evaluated with rigorous self-scrutiny; we release the pipeline, matrix, and diagnostics so that findings can be independently reproduced. Future work: gold linguistic annotation to measure each agent against the phenomenon it targets rather than only aggregate MQM; extension to Classical Tibetan and Pāli; and a multi-judge reporting panel to bound the single-judge design’s residual uncertainty.
Limitations
Effects are small relative to the metric’s range. Every condition scores 85.2–85.9 of 100; the effect concentrates in the low tail, not a uniform mean shift, and should not be extrapolated to a metric with more headroom.
A single reporting judge is a single point of failure. Despite the validation in § 5.2, the headline metric rests on one model; a multi-judge panel (future work) would strengthen this.
The noise floor bounds generation variance, not judge variance, since baseline/baseline_dup differ only in seed and are judged at temperature 0 with caching.
Recomputing MQM from error spans departs from the published GEMBA-MQM protocol (96.4% disagreement with the model’s own arithmetic, § 5.2), so it is not numerically comparable to prior work reporting the raw score.
bo_n decodes at temperature 0.7 vs. 0.0 for every other condition: this is inherent to best-of-, but a genuine confound.
Single-prompt reasoning baselines: While self_refine and bo_n provide compute-matched controls, exploring unified single-prompt Chain-of-Thought reasoning (prompting a single frontier model to perform Sandhi analysis and discourse tracking in explicit scratchpad tokens before emitting a translation) represents a valuable direction for future baseline comparisons.
Ensemble aggregation vs. routing: The parallel drafting array selects the top candidate via ; evaluating multi-candidate synthesis/aggregation models rather than tournament routing remains an open architectural alternative.
Ethical considerations. Multi-agent inference raises compute cost per query relative to single-pass models (Table 2), acceptable for offline research but warranting caching and hardware-efficiency care before deployment; translating culturally and religiously loaded terminology (Dharma, Śūnya) risks interpretive drift, which the voice guard and independent evaluation aim to mitigate. We credit and respect the licenses of the open assets used: MitraSamgraha (CC-BY-SA 4.0; [2]) and the Gemma model family (Gemma Open License; [6]).
Qualitative Analysis: Register-Aware Voice Preservation
Table 7 shows an unedited pipeline output pair for a legal (Manusmṛti, voice-preserving) verse. Both conditions produce fluent, accurate English; the full swarm’s Ellipsis Restorer, informed that this work is voice-preserving, rewrites the target clause as an impersonal passive ("the mind should be turned away...") rather than an invented active third-person subject ("one should turn the mind away..."), matching the impersonal register rather than converting it to a personal directive. We report this as a mechanism demonstration, not a large per-item MQM swing: per-item routing-judge scores are a diagnostic signal, not a reported metric (§ 3.5); the statistically assessed effect of this mechanism is the register-stratified pattern (§ 5) and the isolated ablation (§ 5, trend 2), not any single verse.
| Source | *indriyārtheṣu sarveṣu na prasajyeta kāmataḥ / atiprasaktim cait eṣām manasā saṃnivartayet // |
| Gold Reference | “...let him carefully obviate an excessive attachment to them, by (reflecting on their worthlessness in) his heart.” |
| no_ellipsis (guard inactive) | “...furthermore, one should turn the mind away from excessive attachment to these.” |
| full_swarm (legal → passive) | “...furthermore, the mind should be turned away from excessive attachment to these.” |
Table 7. Register-aware voice preservation, an unedited output pair (Manusmṛti 2.98, item GSD36manu_test:130).
Supplementary Tables
Table 8: Ablation matrix: active components per configuration. Guide = Sandhi Parser; Ellip. = Ellipsis Restorer; Crit. = critique-refinement loop; Tools = tool-selector inventory; TM = translation-memory prefetch; QA = automated QA audit pass. Referenced from § 4.5.
| Configuration | Guide | Ellip. | Crit. | Tools | TM | QA | Notes |
| baseline | – | – | – | – | – | – | – |
| baseline_dup | – | – | – | – | – | – | seed +1 |
| placebo_rejudge | – | – | – | – | – | – | re-judge only |
| bo_n | – | – | – | – | – | – | |
| self_refine | – | – | ✓ | – | – | ✓ | unspecialised |
| guide_only | ✓ | – | – | – | – | – | – |
| guide_ellipsis | ✓ | ✓ | – | – | – | – | – |
| no_guide | – | ✓ | ✓ | ✓ | ✓ | ✓ | – |
| no_ellipsis | ✓ | – | ✓ | ✓ | ✓ | ✓ | – |
| no_critique | ✓ | ✓ | – | – | ✓ | ✓ | – |
| no_tools | ✓ | ✓ | ✓ | – | ✓ | ✓ | – |
| full_swarm | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | – |
Table 8. Ablation matrix: active components per configuration.
Table 9: Full 12-condition register breakdown, MQM by register and configuration (); the condensed discussion in § 5 is derived from this.
| Register | baseline | baseline_dup | bo_n | full_swarm | guide_ellipsis | guide_only | no_critique | no_ellipsis | no_guide | no_tools | placebo_rejudge | self_refine |
| epic | 86.0 | 85.8 | 85.8 | 86.0 | 85.8 | 85.9 | 85.8 | 86.0 | 86.4 | 86.0 | 86.0 | 85.9 |
| kavya | 85.4 | 85.5 | 85.3 | 85.8 | 85.7 | 85.4 | 85.7 | 85.4 | 85.7 | 85.8 | 85.4 | 85.4 |
| legal | 85.1 | 84.7 | 85.0 | 85.4 | 85.1 | 85.0 | 85.1 | 84.8 | 85.2 | 85.4 | 85.1 | 84.6 |
| sutra | 85.7 | 85.8 | 85.9 | 86.0 | 86.0 | 85.5 | 86.0 | 85.8 | 86.2 | 86.0 | 85.7 | 86.1 |
| vedic | 83.6 | 83.9 | 83.7 | 84.5 | 84.6 | 83.4 | 84.7 | 83.5 | 84.4 | 84.6 | 83.6 | 83.7 |
Table 9. Full 12-condition register breakdown, MQM by register and configuration ().
| Configuration | Prompt tok. | Compl. tok. | Total/verse | Latency (s) |
| baseline | 91 | 49 | 140 | 2.5 |
| baseline_dup | 91 | 48 | 140 | 2.4 |
| bo_n | 455 | 258 | 713 | 12.8 |
| full_swarm | 828 | 462 | 1290 | 7.1 |
| guide_ellipsis | 738 | 430 | 1169 | 5.1 |
| guide_only | 482 | 389 | 871 | 2.5 |
| no_critique | 738 | 430 | 1169 | 5.1 |
| no_ellipsis | 726 | 472 | 1198 | 7.3 |
| no_guide | 415 | 114 | 529 | 6.8 |
| no_tools | 797 | 458 | 1255 | 6.8 |
| placebo_rejudge | 91 | 49 | 140 | 2.5 |
| self_refine | 164 | 89 | 253 | 4.9 |
Table 10. Full per-verse token budget and latency by configuration (measured); Table 2 in the main text shows the three key conditions. Referenced from § 4.4.
| Configuration | addition | mistranslation | omission | untranslated | awkward | grammar |
| baseline | 0.07 | 1.62 | 1.35 | 0.02 | 0.02 | 0.13 |
| baseline_dup | 0.07 | 1.62 | 1.36 | 0.02 | 0.02 | 0.13 |
| bo_n | 0.07 | 1.63 | 1.37 | 0.01 | 0.02 | 0.13 |
| full_swarm | 0.06 | 1.68 | 1.31 | 0.01 | 0.02 | 0.13 |
| guide_ellipsis | 0.06 | 1.69 | 1.31 | 0.01 | 0.02 | 0.13 |
| guide_only | 0.06 | 1.66 | 1.32 | 0.02 | 0.02 | 0.15 |
| no_critique | 0.07 | 1.69 | 1.31 | 0.01 | 0.02 | 0.13 |
| no_ellipsis | 0.06 | 1.70 | 1.33 | 0.01 | 0.02 | 0.14 |
| no_guide | 0.07 | 1.66 | 1.34 | 0.01 | 0.02 | 0.11 |
| no_tools | 0.06 | 1.68 | 1.31 | 0.01 | 0.02 | 0.13 |
| placebo_rejudge | 0.07 | 1.62 | 1.35 | 0.02 | 0.02 | 0.13 |
| self_refine | 0.07 | 1.68 | 1.35 | 0.01 | 0.02 | 0.12 |
Table 11. Errors per segment by MQM category, . Referenced from § 5.
Errors per segment, by MQM category, as judged by the independent reporting judge.
Agent System Prompts and Prompt Templates
In this appendix, we provide the exact system instructions, prompt templates, and JSON schemas used by the various subagents in MITRA-Swarm to ensure full reproducibility of the agentic behaviors.
Morphological Pre-processing (Sandhi Parser Agent)
The Sandhi Parser Agent is responsible for decomposing complex phonetic mergers and nominal compounds before translation begins. Its output is threaded into every downstream translator call via a typed TranslationRequest.guide field (§ 3.2).
System Instruction:
You are an expert Sanskrit grammarian.
Analyze the input Sanskrit text. Identify
complex compound words (Samasa) and
phonetic joins (Sandhi). Write down the
compound word splits and explain the
individual components briefly to assist a
translator. Format your output as a concise
split analysis. If there are no complex
compounds, reply: 'No complex compounds.'
Prompt Template:
Analyze compounds/sandhi in this text:
{source_chunk}
| Judge diagnostic | Value |
| Reporting judge | openai:gpt-4o-mini |
| Judge failure rate | 0.41% |
| Judge arithmetic disagrees with rubric | 96.4% (mean = 16.37) |
| Test–retest SD (same input, repeated calls) | 1.45 |
| Self-preference, baseline () | +2.84 |
| Self-preference, baseline_dup () | +2.72 |
| Self-preference, bo_n () | +6.79 |
| Self-preference, full_swarm () | +5.43 |
| Self-preference, guide_ellipsis () | +4.49 |
| Self-preference, guide_only () | +1.94 |
| Self-preference, no_critique () | +4.48 |
| Self-preference, no_ellipsis () | +5.12 |
| Self-preference, no_guide () | +5.77 |
| Self-preference, no_tools () | +5.46 |
| Self-preference, placebo_rejudge () | +2.90 |
| Self-preference, self_refine () | +5.65 |
Table 12. Full per-condition judge-validation diagnostics; self-preference is . Table 5 in the main text summarizes this. Referenced from § 5.2.
Figure 2. Tail of the per-verse MQM distribution, full swarm vs. monolithic baseline. Referenced from § 5.
Discourse Argument Restoration (Ellipsis Restorer Agent)
This agent operates over a sliding window to interpolate omitted arguments (zero-anaphora) into the draft translation. In production, the genre field passed below is "legal" whenever the source work ID is flagged voice_preserving in the register table (covering the legal and sutra registers), and the literal register name otherwise (§ 3.4).
System Instruction:
You are an expert classical language editor.
Sanskrit frequently omits pronouns,
subjects, or verbs (ellipses) when they are
clear from context. Review the source
Sanskrit text, the surrounding discourse
context, and the draft English translation.
If the source genre is impersonal or
injunctive (e.g. legal/sutra), prefer
impersonal or passive restorations over
inventing an active third-person subject.
Identify if any implied arguments are
missing in the draft English translation. If
so, write down the restored/improved
translation. If the draft translation is
already complete, accurate, and fluent,
return the draft translation EXACTLY as is.
Return only the final English text without
explanations or headers.
Prompt Template:
Source Sanskrit:
{source}
Source genre: {genre}
Draft English Translation:
{draft_translation}
Surrounding Discourse Context:
- Preceding Chunk: {prev_chunk}
- Succeeding Chunk: {next_chunk}
Output the restored translation:
Quality Evaluation Judge (GEMBA-MQM)
Both and (§ 3.5) use this prompt; the reported metric is the deterministic score recomputed from errors using the fixed weights below, not the model’s own top-line number, which is retained separately as model_reported_score for the judge-validation diagnostic in § 5.2.
System Instruction:
You are an expert translation quality evaluator
trained in MQM (Multidimensional Quality
Metrics). You identify translation errors from
classical languages into English and assign
severity levels. Be rigorous but fair. Respond
strictly in the JSON schema requested.
Prompt Template:
Source language: {lang_name}
Source text:
"""
{source}
"""
Candidate translation (English):
"""
{translation}
"""
Identify every translation error. For each,
choose:
- category: one of [accuracy/mistranslation,
accuracy/omission, accuracy/addition,
accuracy/untranslated, fluency/grammar,
fluency/register, fluency/spelling,
fluency/punctuation, style/awkward,
terminology, other]
- severity: one of [minor, major, critical]
- span: the offending English substring
(verbatim)
- explanation: one short sentence
Then compute an overall MQM score. Use the standard weights:
critical = -25, major = -5, minor = -1
score = max(0, 100 + sum(weights))
Return ONLY valid JSON matching:
{
"errors": [
{"category": "...", "severity": "...", "span": "...",
"explanation": "..."}
],
"score": <number 0-100>,
"rationale": "<one sentence summary>"
}
Critique and Tool-Selector Agents
When a translation falls below the quality threshold, the Critique Agent diagnoses the errors, and the Tool Selector Agent retrieves necessary context strictly from the tools reported as available in the current environment (§ 3.6); for the benchmark in this paper, that inventory reduces to TRANSLATION_MEMORY.
Critique Agent System Instruction:
You are a senior translation editor. A Sanskrit-to-English translation
has received a low evaluation score from our automated judge. Perform a
detailed critique. Identify errors, mistranslations, omissions, or
terminology mismatches in the draft compared to the Sanskrit source.
Suggest specific corrections.
Tool Selector Agent System Instruction:
You are a research coordinator routing translation tasks to tools. Review
the Sanskrit source text and the critique. Determine which tools from
the inventory below are needed to resolve the issues. Only tools listed
here are available in this environment; do not request any other tool.
Available Tools (environment-reported; varies by run):
1. TRANSLATION_MEMORY: fuzzy-match the source against a decontaminated
parallel corpus for sentence-level precedents (query = Sanskrit text).
2. DICTIONARY: look up a headword in the offline lexicon snapshot, if
one is configured for this run (query = single word in IAST).
3. ALIGNMENT: Tibetan/Classical Chinese parallel alignment, if an
alignment index is configured for this run (query = Sanskrit keyword).
You must select between 0 and 3 tool calls. Return your selection ONLY
as a JSON list of dictionaries, e.g.:
[
{"tool": "TRANSLATION_MEMORY", "query": "citraśravaḥ tama havante"}
]
Do not output markdown code blocks or any
explanation.References
- [1]Anonymous. 2026a. MITRA: A Large-Scale Parallel Corpus and Multilingual Pretrained Language Model for Machine Translation and Semantic Retrieval for Pāli, Sanskrit, Buddhist Chinese, and Tibetan. Citation anonymized for double-blind review.
- [2]Anonymous. 2026b. Mitrasamgraha: A Comprehensive Classical Sanskrit Machine Translation Dataset. Citation anonymized for double-blind review.
- [3]Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, and 1 others. 2022. No Language Left Behind: Scaling Human-Centered Machine Translation. arXiv preprint arXiv:2207.04672.
- [4]Sushant Dave, Arun Kumar Singh, A. P. Prathosh, and Brajesh Lall. 2021. Neural Compound-Word (Sandhi) Generation and Splitting in Sanskrit Language. In 8th ACM IKDD CODS and 26th COMAD, pages 171–177. Association for Computing Machinery.arxiv.org/abs/2010.12940
- [5]Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. 2023. Improving Factuality and Reasoning in Language Models through Multiagent Debate. arXiv preprint arXiv:2305.14325.arxiv.org/abs/2305.14325
- [6]Google. 2026. Gemma 4: Open Models for Responsible AI Development. Technical report, Google DeepMind.
- [7]Nuno M. Guerreiro, Ricardo Rei, Daan van Stigt, Luísa Coheur, Pierre Colombo, and André F. T. Martins. 2024. xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detection. Transactions of the Association for Computational Linguistics, 12:979–995.arxiv.org/abs/2310.10482
- [8]Barry Haddow, Rachel Bawden, Antonio Valerio Miceli Barone, Jindřich Helcl, and Alexandra Birch. 2022. Survey of Low-Resource Machine Translation. Computational Linguistics, 48(3):673–732.
- [9]Oliver Hellwig. 2010–2021. DCS - The Digital Corpus of Sanskrit.
- [10]Amr Hendy, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Matushita, Young Jin Kim, Mohamed Afify, and Hany Hassan Awadalla. 2023. How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation. arXiv preprint arXiv:2302.09210.arxiv.org/abs/2302.09210
- [11]Gérard Huet. 2004. Design of a Lexical Database for Sanskrit. In Proceedings of the Workshop on Enhancing and Using Electronic Dictionaries, pages 8–14, Geneva, Switzerland. COLING.DOI
- [12]Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezhen Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin J. Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of Hallucination in Natural Language Generation. ACM Computing Surveys, 55(12):248.DOI
- [13]Urvashi Khandelwal, Angela Fan, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2021. Nearest Neighbor Machine Translation. In International Conference on Learning Representations (ICLR).arxiv.org/abs/2010.00710
- [15]Tom Kocmi and Christian Federmann. 2023b. Large Language Models Are State-of-the-Art Evaluators of Translation Quality. In Proceedings of the 24th Annual Conference of the European Association for Machine Translation, pages 193–203, Tampere, Finland. European Association for Machine Translation.arxiv.org/abs/2302.14520
- [16]Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics, 12:157–173.arxiv.org/abs/2307.03172
- [17]Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-Refine: Iterative Refinement with Self-Feedback. In Advances in Neural Information Processing Systems, volume 36, pages 46534–46594.arxiv.org/abs/2303.17651
