WorksComputer Science
What Transfers Across Cuneiform? Script-Level Substrate and Language-Specific Regularities via Segmentation and Representation
Abstract
Cuneiform recorded typologically unrelated languages for more than three millennia, raising a natural question: which of its statistical regularities are properties of the shared writing system, and which belong to individual languages? We answer this with two probe pairs targeting the same script-versus-language decomposition. The first compares two unsupervised, dictionary-free segmentation methods: transitional probability (TP), grounded in infant statistical-learning research, and Morfessor. To our knowledge, this is the first systematic evaluation of dictionary-free segmentation on cuneiform; the only prior published baseline relies on a hand-built dictionary (Max-Match, F1 = 0.74). Both our methods reach F1 > 0.97 within language (Morfessor reaches F1 > 0.99 on Akkadian and Sumerian), exceeding MaxMatch by more than 23 F1 points, but diverge sharply in cross-language transfer: TP holds F1 > 0.95 across all six language pairs, while Morfessor collapses to F1 = 0.27 from the 13K-token Elamite source. A learning curve and a Hittite stress test (34.7% foreign-script logograms, where TP drops 35 points and Morfessor only 8) further isolate the two underlying mechanisms. The second probe pair extends the same decomposition to representations: concatenating Unicode and Latin features improves classification in 11 of 12 morphosyntactic tasks (p < 0.01). We release CUNEI-TOOLS and a three-language benchmark.
Introduction
Cuneiform is the world’s oldest known writing system, used for over three millennia (c. 3400 BCE–75 CE) across the ancient Near East to write languages as typologically diverse as Sumerian (a language isolate), Akkadian (Semitic), and Elamite (whose genetic affiliation remains debated). With over half a million surviving inscribed objects and active digitization efforts through projects such as ORACC (Oracc, [23]) and the CDLI, cuneiform represents both significant cultural heritage and a compelling testbed for low-resource NLP.
A central obstacle in cuneiform NLP is the absence of stable units of analysis. When a new tablet is digitized, the sign sequence is often recorded as a continuous stream without word boundaries, and segmentation directly affects tokenization, which underlies all modern NLP methods. The standard scholarly representation, Latin transliteration, adds boundaries as part of the transcription process, but it does so at the cost of interpretive commitments: the scholar assigns reading values to each sign and resolves polyphony, determinative function, and logogram-versus-syllabic ambiguity in advance of any computational analysis. Because transliteration collapses or alters graphemic distinctions, segmentation methods operating over transliteration may obscure distributional regularities present in the original sign stream. Elamite illustrates this challenge concretely: its transliteration conventions vary across scholarly traditions (Hinz and Koch, [13]; de Blois, [6]; Basello, [2]), while Unicode cuneiform encodes the underlying graphemic identity unambiguously. The cuneiform world contains many under-documented languages of this kind (Hittite, Hurrian, Urartian, Eblaite, Old Persian, and others), and newly discovered tablets often appear in languages or periods with limited annotated data. A practical segmentation method must therefore operate in the low-resource regime where annotated corpora are small and transliteration conventions are unsettled.
Because cuneiform was used to write typologically unrelated languages, it offers a natural question: which of its statistical regularities are properties of the shared script substrate, and which belong to individual languages? We probe this with two pairs of contrasting tools that target the same script-versus-language decomposition.
The first probe pair consists of two unsupervised segmentation methods. Transitional probability (TP) predicts a boundary when falls below a threshold, where bigram probabilities are estimated from a segmented training corpus. TP operates on sign-level co-occurrence statistics and is grounded in the statistical learning framework of Saffran et al. [30], who showed that infants segment continuous speech using syllable co-occurrence statistics. Morfessor (Creutz and Lagus [4]) learns an MDL-optimal lexicon of subword units from word-frequency data and applies Viterbi decoding to segment new text; it is the standard reference baseline for unsupervised segmentation across many languages. To our knowledge, no dictionary-free segmentation method has previously been evaluated systematically on cuneiform. The only prior published work on cuneiform word segmentation (Homburg and Chiarcos [14]) relied on a hand-built dictionary (MaxMatch, ), while their unnormalized bigram baseline reached only , far short of dictionary-based performance. The present work characterizes the two principal families of dictionary-free segmentation (statistics-based and vocabulary-based) on cuneiform for the first time, and achieves an in-language improvement of more than 23 points over the prior dictionary-based state of the art. The two methods are well-suited as substrate probes precisely because they exploit different statistical properties of training data: TP exploits sign-level co-occurrence patterns, which can be language-invariant if the script substrate is shared, while Morfessor exploits lexical recurrence, which is inherently language-specific.
The second probe pair extends the same decomposition to representations. Two digital representations of cuneiform coexist: Latin transliteration, the scholarly standard, which encodes interpreted reading values (language-specific); and Unicode cuneiform, available since Unicode 5.0 (2006), which preserves sign-level distinctions (script-level). We compare them across three morphosyntactic task families (POS tagging, entity classification, lemmatization) and find that they capture complementary information: concatenating both significantly outperforms either alone in 11 of 12 comparisons (). The Unicode/Latin complementarity parallels the TP/Morfessor dissociation; both probe pairs reveal the same underlying script-level versus language-level structure.
Our contributions are:
A substrate decomposition framework for the cuneiform corpora we study. Across Akkadian (ORACC), Sumerian (ETCSL), and Elamite (Elamite Dictionary plus Utu-našu), we observe separable script-level statistical regularities (shared across these three languages, recoverable from little data) and language-specific regularities (requiring more data, not transferring).
Segmentation methods as substrate probes. We provide the first systematic comparison of dictionary-free statistical (TP) and lexical (Morfessor) unsupervised segmentation on cuneiform, against the prior dictionary-based baseline of Homburg and Chiarcos [14]. TP transfers across languages ( in all six pairs) and is recoverable from approximately tokens; Morfessor’s lexicon does not transfer ( at 13K Elamite). Hittite, where 34.7% of tokens are foreign-script logograms, produces asymmetric failure (TP drops 35 points, Morfessor 8), showing the two substrate properties are dissociable rather than redundant (§ 4.1–§ 4.2.2).
Representations as a second probe pair. Unicode and Latin encodings parallel the segmentation methods in probing the same script-versus-language decomposition; concatenating them improves classification in 11 of 12 tasks (§ 4.4).
Resources. We release CUNEI-TOOLS, a pip-installable toolkit for segmentation and script conversion, and a three-language evaluation benchmark.1
Background
The Cuneiform Writing System
Cuneiform is a logosyllabic script in which individual signs can represent syllables (phonographic use) or entire words (logographic use). Additional complexity arises from determinatives (semantic classifiers that are written but not pronounced), phonetic complements, and language-specific conventions. The script was adapted across cultures, with each language employing different subsets of the approximately 1,110 attested signs.
The three languages in our study represent distinct typological profiles: Akkadian is a Semitic language with rich templatic morphology; Sumerian is an agglutinative language isolate with complex verbal morphology; and Elamite is a largely agglutinative, non-Indo-European language whose genetic affiliation remains debated [39, 11, 27]. All three were written in cuneiform but employed the script differently, making them ideal for testing whether segmentation methods generalize across typological boundaries.
Representation Choices
Latin transliteration encodes interpreted readings: the scholar decides which of a sign’s possible values was intended in context, injecting expert knowledge but also interpretive commitments. Unicode cuneiform preserves graphemic structure without interpretation. Three phenomena illustrate the contrast. First, polyphony: the sign A U+12000 maps to both a ‘water’ and duru5` ‘to be wet’ in Latin transliteration, but remains a single Unicode glyph. Second, determinatives: the divine classifier in {d}en-ki is written but not pronounced, appearing explicitly in Latin but encoded as an ordinary sign in Unicode. Third, logographic variation: ‘king’ can be written as a single logogram LUGAL (U+12217) or syllabically as šar-ru (two signs), collapsing a two-sign sequence to one glyph or expanding it. These three phenomena indicate that the two representations encode systematically different information about the same underlying sign stream.
Related Work
Cuneiform NLP. Computational approaches to cuneiform have primarily used Latin transliteration for tasks including machine translation [12, 24, 25], POS tagging and lemmatization [31, 32, 33], named entity recognition [20], and sign-level analysis [7, 9, 42]. Jauhiainen et al. [17] demonstrated that Unicode cuneiform can function as a computational representation for language identification. For word segmentation specifically, Homburg and Chiarcos [14] tested multiple methods on Middle Babylonian Unicode cuneiform, achieving with the dictionary-based MaxMatch algorithm; their unnormalized bigram baseline achieved only . To our knowledge, no dictionary-free statistical or vocabulary-based segmentation method has been evaluated on cuneiform prior to the present work. Simmons et al. [35] released SumTablets, pairing Unicode glyphs with transliterations for 91K Sumerian tablets. Smidt et al. [37] compared transliteration and Unicode for Akkadian POS tagging, reporting accuracy losses with Unicode input. A more detailed comparison to prior work is in Appendix § 8.§ 1.
Unsupervised word segmentation. The two main families are statistics-based and vocabulary-based methods. Transitional probability as a word boundary cue was demonstrated by Saffran et al. [30], who showed that infants segment continuous speech using syllable co-occurrence statistics; to our knowledge, this principle has not previously been applied to logosyllabic writing systems. Creutz and Lagus [4] introduced the Morfessor family, MDL-based segmenters that learn a probabilistic lexicon of subword units. Subword tokenization methods such as BPE [34] and SentencePiece take a related vocabulary-learning approach for downstream NLP tasks.
Script and representation effects. Whether to use original script or romanization is a recurring question in NLP. For Chinese, Meng et al. [21] and Sun et al. [38] showed that glyph-based and fused glyph/character/pinyin representations outperform single-view models, closely paralleling our complementarity result. For byte-level models, Xue et al. [43] showed that ByT5 can process any script through raw bytes, though its effectiveness depends on pretraining coverage. Cuneiform languages exemplify the low-resource setting [1], where character-level models [19] have proven effective, motivating our use of character n-gram features.
Data and Preprocessing
Datasets
Table 1 summarizes our three datasets, which span three orders of magnitude in size.
Akkadian data comes from ORACC (Oracc, [23]), including Neo-Assyrian royal inscriptions (Frame et al., [8]) and state archives (Munich Open-access Cuneiform Corpus Initiative, 2007–2020). Sumerian data comes from ETCSL (Black et al., [3]), consisting of literary compositions. Elamite data derives from the Elamite Dictionary (Hinz and Koch, [13]) and Elamite primary sources in the CEPOAT corpus (Quintana Cifuentes, [26]) (15,601 entries across 9 POS categories), preprocessed through a two-pass Unicode conversion pipeline, including the Utu-našu corpus of 85 running-text documents. The dictionary and the running text play distinct roles by task: segmentation experiments train exclusively on the Utu-našu running text, while the Elamite Dictionary enters only the classification experiments; a training-source ablation confirming that the dictionary does no hidden segmentation work appears in Appendix § 8.13. Elamite transliteration conventions vary across scholarly traditions, introducing inconsistency that Unicode avoids by preserving graphemic sign structure directly ([13, 6, 2]).
Unicode Conversion
We convert Latin transliteration to Unicode cuneiform using merged sign lists from Nuolenna (Jauhiainen, n.d.) and Akkademia (Gordin et al., [10]) (18,272 mappings), supplemented with 136 manual corrections. Conversion rates are 98.9% for Akkadian, 99.9% for Sumerian, and 85.8% for Elamite; unconverted tokens are excluded from Unicode-dependent experiments. The lower Elamite conversion rate reflects the historical state of Elamite lexicography: the 14.2% unconverted tokens are predominantly damaged or partially legible signs in fragmentary tablets, unmapped hapax legomena from the small Elamite corpus, and rare orthographic variants for which canonical Unicode mappings have not been established. These exclusions remove unrecoverable noise rather than systematically simpler vocabulary, but we note them as a source of asymmetry between the Latin and Unicode Elamite experiments.
POS Harmonization
The three datasets use different POS tagsets. We define a unified mapping and evaluate in two configurations: (A) a unified tagset preserving named entity categories (PN, DN, GN, etc.) as distinct POS labels ([29]), and (B) a grammatical tagset collapsing all entity types to PROPN with separate NER tags ([15]).
Experiments
All classification experiments use 5-fold stratified cross-validation and report macro F1. The primary classifier is character n-gram (1–4) logistic regression with balanced class weights. We additionally report k-NN on fastText embeddings and character-level BiLSTM results. Statistical significance is assessed via paired bootstrap (2,000 resamples, two-sided).
Word Boundary Inference
We compare transitional probability and Morfessor on word boundary inference from unsegmented Unicode cuneiform streams. Both methods are trained on segmented documents and evaluated via 5-fold held-out cross-validation. Both are dictionary-free, in contrast to the only prior published baseline for cuneiform word segmentation (Homburg and Chiarcos, [14]), which used the dictionary-based MaxMatch algorithm at F1 = 0.74. To ensure a rigorous, apples-to-apples comparison, we replicate their pipeline by stripping all transliterated spaces to simulate the unbroken graphemic stream of a physical tablet, thereby isolating the segmentation algorithm as the sole independent variable.
Transitional Probability (TP). For each position in a continuous sign stream, we predict a boundary when , where bigram probabilities are estimated from a training corpus of segmented text. TP normalizes by unigram frequency, isolating the transition structure that signals boundaries; raw co-occurrence counts are dominated by high-frequency signs and uninformative about word structure. This normalization is what was missing from the bigram baseline of Homburg and Chiarcos [14], which achieved only F1 = 0.149 due to its reliance on unnormalized counts.
Morfessor. Morfessor (Creutz and Lagus, [4]) is a standard unsupervised segmenter that learns an MDL-optimal lexicon of subword units from word-frequency data. At test time, the trained model applies Viterbi decoding to find the best segmentation of an input stream using the learned units. Like TP, Morfessor requires segmented training data but no dictionary; unlike TP, it learns a parameterized lexicon rather than computing statistics directly. We use the default hyperparameters (corpusweight = 1.0). A hyperparameter-robustness analysis (Appendix § 8.11) shows that in-language F1 is stable for corpusweight but collapses catastrophically above it (F1 at corpusweight for all three languages), and that admissible source-side model selection prefers corpusweight = 0.5; every qualitative transfer conclusion is unchanged under the selected value, with tuned-Morfessor transfer still 19–26 F1 points below untuned TP from the smallest source.
Table 2 shows that both methods achieve high in-language F1 across all three languages, with Morfessor holding a slight in-language edge (0.997 vs. 0.971 on Akkadian, 0.993 vs. 0.972 on Sumerian, 0.985 vs. 0.989 on Elamite; Elamite is the only language where TP wins in-language). The in-language gap is statistically significant for Akkadian (F1 = , ) and Sumerian (F1 = , ; document-level paired bootstrap, 2,000 resamples), while TP’s small Elamite advantage is not (F1 = , 95% CI [, ], ). Both methods substantially outperform the prior dictionary-based MaxMatch baseline of Homburg and Chiarcos [14], with an improvement of more than 23 F1 points over MaxMatch’s 0.739. The in-language gap between TP and Morfessor is small, however, and the more revealing comparison between the two dictionary-free methods emerges in cross-language transfer.
| Method | Lang | F1 | P | R |
| TP | AKK | .971 | .946 | .997 |
| SUX | .972 | .946 | .999 | |
| ELX | .989 | .993 | .984 | |
| Morfessor | AKK | .997 | .997 | .996 |
| SUX | .993 | .997 | .990 | |
| ELX | .985 | .999 | .971 |
Table 2. Word boundary inference (5-fold held-out CV). Both dictionary-free methods exceed the prior dictionary-based MaxMatch baseline (Homburg and Chiarcos [14]) (F1 = 0.74 on Middle Babylonian) by more than 23 F1 points. Morfessor’s slight in-language edge does not generalize to cross-language transfer (Table 3).
Cross-Language Transfer
Cross-language transfer reveals a sharp dissociation between the two dictionary-free methods. Table 3 shows the full transfer matrix: training on language and testing on language , for both TP and Morfessor, with no test-language hyperparameter tuning.
| Train | TP (test F1) AKK | TP (test F1) SUX | TP (test F1) ELX | Morfessor (test F1) AKK | Morfessor (test F1) SUX | Morfessor (test F1) ELX | Train tokens |
| AKK | .971† | .967 | .992 | .997† | .981 | .984 | 1.26M |
| SUX | .960 | .972† | .993 | .859 | .995† | .953 | 146K |
| ELX | .959 | .969 | .989† | .304 | .267 | .985† | 13K |
Table 3. Cross-language word boundary transfer (F1). † = in-language (held-out CV). Off-diagonal cells = zero-shot transfer with no test-language hyperparameter tuning. Bold = >10 F1 point gap from TP. Train tokens are Table 1 word counts; in segmentation-stream units the corpora are 2.44M / 338K / 8.8K (ELX segmentation trains on the Utu-našu running text only, § 3). 95% bootstrap confidence intervals (1,000 document resamples) are within ±0.006 across all transfer cells; the cross-language dissociation between TP and Morfessor (65+ F1 points at ELX-as-source) is therefore well outside any reasonable noise band. Visualized in Figure 1.

Figure 1. Cross-language transfer F1 by source language and method. The dissociation is corpus-size mediated: TP (blue) maintains F1 > 0.95 across all source corpora, while Morfessor (orange) tracks TP at the largest source (AKK, 1.26M tokens), trails by approximately 10 points at the medium source (SUX, 146K tokens), and collapses to below 0.31 at the smallest source (ELX, 13K tokens). The pattern is exactly what would be expected if TP requires only stable bigram statistics (which converge from approximately tokens) while Morfessor requires a lexicon that covers the target corpus; coverage, not language identity per se, predicts its transfer (§ 4.1.1).
When the training corpus is large (Akkadian at 1.26M tokens), both methods transfer reasonably well: AKKSUX yields TP F1 = 0.967 and Morfessor F1 = 0.981; AKKELX yields TP F1 = 0.992 and Morfessor F1 = 0.984. The methods are roughly comparable in this high-resource regime, with Morfessor holding a small edge on AKK as the source. As the training corpus shrinks, however, the methods diverge sharply. Training on Sumerian (146K tokens), TP transfers to Akkadian at F1 = 0.960 while Morfessor drops to F1 = 0.859, a 10-point gap. Training on the small Elamite corpus (13K tokens), TP transfers to Akkadian at F1 = 0.959 while Morfessor collapses to F1 = 0.304; TP transfers to Sumerian at F1 = 0.969 while Morfessor collapses to F1 = 0.267. The gaps in the low-resource transfer setting exceed 65 F1 points.
This dissociation has a clean substrate-decomposition explanation. TP’s sufficient statistics (sign-level bigram probabilities) appear to function, across the three corpora we test, as properties of a shared script substrate: within-word sign sequences in Akkadian, Sumerian, and
Elamite obey similar local statistical regularities, consistent with their shared use of the cuneiform writing system, even when the underlying vocabulary and morphology differ completely.2 Morfessor’s learned lexicon, by contrast, is language-specific: a model trained on Elamite memorizes Elamite words, which appear essentially nowhere in Akkadian or Sumerian text, so cross-language failure reflects the lexicon’s coverage of the target corpus (to which both source size and source composition contribute; see below) rather than any failure of the algorithm itself. This interpretation is corroborated by the in-language learning curve below (§ 4.2.1): Morfessor at 10K Akkadian training tokens reaches F1 > 0.95 on a held-out Akkadian test, while Morfessor at 13K Elamite training tokens reaches only F1 = 0.30 on Akkadian test. The 65+ point gap between these two settings, both at comparable training-corpus size, isolates the vocabulary-transfer mechanism from any pure sample-efficiency confound. We release the TP segmenter as CUNEI-SEG, with pre-trained models for all three languages.
Coverage, not language identity, drives the collapse. A matched-size control proposed during review sharpens the mechanism claim. Training Morfessor on Sumerian subsampled to the Elamite training budget (5 seeds, whole-document subsampling, zero-shot evaluation) separates two contributions:
Both axes act through a single quantity: the
Table 4. Matched-size transfer control. Shrinking the Sumerian source to the Elamite budget costs 32 points (→AKK), yet at the same budget the Elamite source sits a further 24 points lower: size and composition both matter, and neither alone explains the collapse.
lexicon’s coverage of the target. The 13K-token Sumerian lexicon covers 96.9% of Elamite target tokens (hence SUX→ELX holds at .710), while the Elamite lexicon covers only 71.7% of Sumerian. Across a full grid of 14 source-size configurations (5 seeds each, 140 evaluations; Appendix § 8.12), coverage of the target predicts Morfessor’s transfer F1 at Pearson . A causal test completes the argument: training Morfessor on the full Sumerian corpus with its lexicon clamped to the Elamite sign inventory collapses transfer to →AKK = 0.283 (unclamped: 0.857), and the mirror clamp on Akkadian gives →SUX = 0.285 (unclamped: 0.981). Holding the language constant and intervening only on inventory coverage reproduces the Table 3 collapse. In-language sample efficiency (§ 4.2.1) and cross-language transfer are thus two views of the same coverage mechanism, while TP requires no lexicon at all.
What the script-level substrate consists of.
The coverage asymmetry is itself a quantitative trace of the shared scribal tradition sketched in the footnote above: the Sumerian lexicon blankets Elamite because syllabary conventions and high-frequency sign sequences propagated from Sumerian scribal curricula outward, while Elamite’s leaner inventory covers only a fraction of Sumerian. The regularities TP exploits are correspondingly graphemic rather than lexical: transitions out of determinatives reliably mark word-initial position across all three languages, and syllabic-structure constraints differ systematically at word edges as against word-internal sequences, so the sign-bigram statistics that signal a boundary in Akkadian remain informative in Sumerian and Elamite even though the words themselves never recur.
Recoverability of the Script-Level Substrate from Minimal Data
The cross-language transfer matrix shows that TP’s substrate property transfers across languages while Morfessor’s does not. A complementary question is how much training data each method requires to learn its respective substrate property in the first place. We run a controlled learning curve on a single language to isolate this question from the cross-language vocabulary-overlap confound. We subsample the Akkadian training corpus to tokens, train both TP and Morfessor at each scale, and evaluate on a fixed held-out Akkadian test set (Figure 2).

Figure 2. In-language learning curve for TP (blue) and Morfessor (orange) on Akkadian, evaluated on a fixed held-out test set. TP is essentially flat at F1 ≈ 0.96 across all training-corpus sizes from 1K to 1.26M tokens, confirming that the script-level substrate property is recoverable from minimal data. Morfessor starts at F1 = 0.67 at 1K tokens, climbs to F1 ≈ 0.92 at 5K, and reaches TP’s level by approximately 10K tokens; above the crossover, Morfessor slightly exceeds TP. The in-language curve isolates property-recoverability (the focus of this experiment) from cross-language vocabulary overlap (the focus of Table 3); together they account for the full cross-language dissociation.
TP is near asymptote at the smallest training size we test: F1 = 0.961 at 1K tokens, climbing
only marginally to F1 = 0.967 at 1.26M tokens. Morfessor starts at F1 = 0.67 at 1K, reaches F1 = 0.92 at 5K, and matches TP near 10K training tokens; from 50K onward, Morfessor slightly exceeds TP. The crossover at approximately tokens tells us two things. First, TP’s sufficient statistics converge from very little data, consistent with the script-level substrate interpretation. Second, Morfessor at 10K in-language tokens reaches F1 > 0.95, while Morfessor at 13K Elamite tokens tested on Akkadian collapses to F1 = 0.30 (Table 3); the 65-point gap at comparable corpus size isolates the coverage mechanism (§ 4.1.1) from sample efficiency. The two experiments together decompose the substrate, in the corpora we study, into a recoverability dimension (TP needs less data) and a transfer dimension (TP’s property holds across the three languages we test).
Hittite: Substrate Decomposition
The cross-language transfer and learning curve results confirm one prediction of the theory: simple statistical methods exploit substrate properties that converge from limited data. A second prediction is that when the substrate itself is violated, both methods should degrade, but through different mechanisms and to different degrees, depending on which aspect of substrate each depends on. Hittite provides a natural test. Hittite cuneiform inserts Akkadograms (Akkadian words read as Hittite) and Sumerograms (Sumerian logograms read as Hittite) inline within Hittite syllabic text, often mid-word. This is a within-word substrate violation: it injects a heterogeneous sign sequence between Hittite syllabic signs, breaking the within-word continuity that both methods, in their different ways, rely on.
Trained and evaluated on Hittite documents under the same protocol as the three other languages, TP achieves F1 = 0.619 (best at ) while Morfessor achieves F1 = 0.918, both below the F1 > 0.96 baseline established for Akkadian, Sumerian, and Elamite. The degradation is sharply asymmetric: TP loses 35 F1 points relative to its three-language average, Morfessor 8. We attribute this asymmetry to the structure of Hittite cuneiform, in which 34.7% of tokens are foreign-script logograms detected via a conservative multi-character uppercase Latin heuristic (representative examples include LUGAL ‘king’ with 420 attestations, NINDA.GUR₄.RA ‘thick bread’ with 1,238, and MUNUS.LUGAL ‘queen’ with 484). These insertions affect the two methods through different mechanisms. TP exploits within-word sign-level statistical continuity: a Hittite syllabic sign abutting a foreign-script logogram produces an unstable local transition probability that does not reliably mark word boundaries. Morfessor exploits lexical-unit recurrence: foreign-script logograms still recur as consistent units within the Hittite corpus, and Morfessor’s memorized vocabulary captures them despite the script-mixing. The asymmetric failure identifies the specific aspect of cuneiform substrate that each method depends on, and shows that script-mixing breaks within-word sign-level continuity more severely than lexical-unit recurrence.
Robustness of the logogram rate. The 34.7% rate is our quantitative anchor for the substrate-violation mechanism. Varying the detector heuristic leaves every variant in the 25–35% range (Appendix § 8.17), so the substantive claim, that more than a quarter of Hittite tokens are foreign-script logograms violating within-word continuity, survives under any reasonable detector. Hittite serves throughout as a substrate-violation stress test rather than a fourth evaluation language. An Old Babylonian case study on naturally mixed Akkadian–Sumerographic letters extends
the same analysis diachronically and shows both failure modes of the decomposition co-occurring (Appendix § 8.16).
POS Classification
Having compared segmentation methods, we next ask whether the choice of representation (Latin vs. Unicode) affects downstream classification, using the same script-versus-language probe-pair logic. We compare Latin and Unicode on token-level part-of-speech classification using character n-gram logistic regression. The unified ORACC POS tagset combines grammatical categories with named-entity tags (Robson and Tinney, [29]). Table 5 reports macro-F1. Latin outperforms Unicode for Akkadian and Sumerian, while Unicode wins for Elamite, hinting that corpus size mediates the representation advantage. Sumerian’s high absolute scores partly reflect ETCSL’s curated literary annotation together with its smaller tagset (14 classes versus 24 for Akkadian; Table 1), which raises macro-F1 chance baselines. The separated grammatical-POS-versus-entity analysis, which reveals a hidden Unicode advantage on grammatical categories that the unified tagset masks, is reported in Appendix § 8.§ 2.
| AKK | SUX | ELX | HIT | |
| Tokens | 1,255,669 | 146,143 | 13,383 | 170,496 |
| Texts | 13,720 | 394 | 85 | 7,099 |
| Vocabulary | 123,001 | 31,906 | 13,189 | — |
| Lemmas | 28,953 | 3,765 | 748 | — |
| POS classes | 24 | 14 | 7 | — |
| Entity types | 11 | 8 | 3 | — |
| Entity ratio | 12.6% | 9.2% | 50.9% | — |
| Source | ORACC | ETCSL | D.+C. | Corpus |
| Unicode rate | 98.9% | 99.9% | 85.8% | 98.6% |
Table 1. Dataset statistics. AKK = Akkadian, SUX = Sumerian, ELX = Elamite, HIT = Hittite (Yavasan and Gordin, [44]). The ELX token count comprises 13K lemma-dictionary entries plus the Utu-našu running corpus; segmentation experiments train on running text only (§ 3), and an expanded 871-document Elamite corpus is analyzed in Appendix § 8.14. Hittite serves exclusively as the substrate-violation stress test (§ 4.2.2), so annotation-dependent rows do not apply.
| Task | Repr. | AKK | SUX | ELX |
| Unified | Latin | .693 | .847 | .636 |
| Unified | Unicode | .621 | .750 | .659** |
| Unified | Concat | .712*** | .870*** | .666*** |
Table 5. POS classification (F1-macro, n-gram LR). Significance markers are paired-bootstrap -values comparing Concat against the better of Latin and Unicode within each column: ***; **. Concatenation wins across all three languages. Separated grammatical-vs-entity results in Appendix § 8.§ 2.
Representation Concatenation as a Second Substrate Probe
The segmentation results identify two separable substrate properties in the cuneiform corpora we study: a script-level property that transfers across the three languages tested and is recoverable from minimal data, and a language-level property that does not transfer. We now ask whether the same decomposition appears at the representational level. Unicode preserves graphemic distinctions and is largely language-invariant (the same Unicode glyph appears in Akkadian, Sumerian, Elamite, and Hittite text). Latin transliteration encodes interpreted reading values that are language-specific (the same sign A U+12000 transliterates as ‘a’ (water) in Sumerian but enters Akkadian text as the syllabic value /a/, with different conventional surroundings in transliteration). If Unicode and Latin behave like a second probe pair for the same script-versus-language decomposition, we should observe complementary information: Unicode for script-level patterns, Latin for language-specific interpretation, and the best system combining both.
We test this by concatenating Latin and Unicode character n-gram feature vectors and training a single logistic regression classifier on the combined representation. If the representations were redundant, this should not help. The Concat row in Table 5 shows the opposite: concatenation wins across all three languages with the unified tagset, and the same pattern holds on the separated grammatical and entity subtasks (Appendix § 8.§ 2). Across all 12 task-language comparisons, concatenation wins 12 of 12 with 11 statistically significant at (paired bootstrap); the sole exception is Akkadian entity typing (). The complementarity is robust to architecture: a 2-layer character Transformer trained from scratch shows concatenation wins 5 of 6 task-language pairs, combining to 17 of 18 wins across both architectures (Appendix § 8.§ 4). Character-level seq2seq lemmatization is the one task where Latin alone wins across all three languages, since lemma targets are themselves in Latin script (Appendix § 8.§ 3).
The complementarity reproduces the segmentation result at the representational level: Unicode supplies sign-level distinctions (paralleling TP’s substrate property), Latin supplies language-specific reading-value interpretation (paralleling Morfessor’s). The two probe pairs converge on the same finding.
The CUNEI-TOOLS Toolkit
We release CUNEI-TOOLS, a pip-installable Python package comprising: (1) CUNEI-SEG, which implements transitional-probability word boundary detection with pre-trained models for all three languages; and (2) CUNEI-CONV, which provides bidirectional Latin–Unicode conversion using 18,272 sign mappings.
Conclusion
This paper characterizes the three cuneiform corpora we study as exhibiting separable script-level and language-specific statistical regularities, exposed through two pairs of probes. Within language, transitional probability reaches and Morfessor reaches on Akkadian and Sumerian given sufficient data, surpassing the prior dictionary-based MaxMatch baseline by more than 23 F1 points and constituting, to our knowledge, the first systematic evaluation of dictionary-free segmentation methods on cuneiform. The two dictionary-free methods diverge under stress in informative ways. In cross-language transfer, TP holds across all six language pairs, consistent with sign-level sufficient statistics that function as properties of a shared script substrate across these corpora, while Morfessor’s transfer F1 falls to 0.27 from the small Elamite source, not because of sample size (the in-language learning curve shows Morfessor matches TP at approximately tokens) but because its learned lexicon fails to cover the target: across a 14-configuration size grid, lexicon coverage of the target predicts Morfessor’s transfer at , and clamping a large source to a small sign inventory suffices to reproduce the collapse (§ 4.1.1). In the Hittite substrate-violation test, the two methods degrade asymmetrically (TP drops 35 points, Morfessor 8), showing that within-word sign-level continuity is more fragile to script-mixing than lexical recurrence. The same decomposition appears at the representational level: Unicode preserves orthographic distinctions, Latin encodes language-specific interpretation, and the two are complementary across 11 of 12 morphosyntactic tasks. The magnitudes (65+ points cross-language; a 27-point Hittite asymmetry) make method choice binary rather than marginal.
In practice: use TP for cross-language segmentation or in-language tasks with fewer than 10K segmented tokens, and Morfessor above that scale; concatenate Latin and Unicode for classification, but use Latin alone for lemmatization, where target-script alignment matters. Supervision does not change this calculus: a character-BiLSTM tagger needs roughly 50K gold-segmented tokens merely to match label-free Morfessor (Appendix § 8.15), and offers no label-free path to a new language.
Limitations
Our Elamite corpus is small (13K dictionary entries, 85 running-text documents); results on this language should be interpreted cautiously. A expansion of the Elamite running corpus assembled during review (871 documents; Appendix § 8.14) leaves TP’s transfer values essentially unchanged and improves Morfessor’s, consistent with the coverage account, while reversing the in-language ordering on the noisier administrative register it introduces. Unicode conversion excluded 14.2% of Elamite entries; these exclusions may skew comparisons against Latin.
Our datasets also ingest damaged material rather than filtering it: damage markers are stripped but the flagged signs are kept, and only fully illegible or unlemmatizable tokens are excluded, so all reported results are measured on damage-inclusive text. The damage-marked share of ingested tokens is 27.4% for Akkadian (a further 22.9% of raw rows are excluded up front as unlemmatizable), 8.4% for the Utu-našu Elamite corpus, 21.9% for the expanded Susa material, and 0.6% for Sumerian, where ETCSL’s composite editions resolve damage editorially. Segmentation performance on damage-heavy text is therefore already reflected in our numbers, but genre and preservation quality co-vary with language in ways we cannot fully disentangle. ETCSL Sumerian consists of literary texts; administrative Sumerian may behave differently. Our Hittite logogram detector is a conservative heuristic (multi-character uppercase Latin); the true logogram rate may be higher than the 25–35% range we report. Our sample-efficiency learning curve is run on Akkadian only; we expect the qualitative shape (TP plateau before Morfessor) to hold for other languages and other corpus sizes, but verifying this for individual under-documented cuneiform languages requires more annotated data than is currently available. Our substrate-level claims should be read as scoped to the specific corpora and dialect/register combinations we evaluate (ORACC Neo-Assyrian and state archives, ETCSL literary Sumerian, the Elamite Dictionary and Utu-našu corpus). Diachronic variation within cuneiform languages (for example, Old Babylonian versus Neo-Assyrian Akkadian) and register variation (literary versus administrative texts) may produce different boundary statistics; whether the substrate property we identify generalizes across these dimensions is an empirical question we leave to future work.
Our evaluation focuses on morphosyntactic tasks. Representation effects on higher-level tasks such as machine translation remain unexplored. Our ByT5 evaluation reveals that pretrained byte-level models cannot effectively exploit cuneiform Unicode without script-specific pretraining. Sign lists have gaps, and some Unicode mappings are approximations. Both Unicode encoding and scholarly transliteration necessarily abstract over historically variable scribal practices.
Ethics Statement
All data comes from published scholarly editions of ancient texts. No human subjects were involved. We release all code and data to support reproducibility.
Use of AI Assistance
We used AI assistance (a large language model) for prose editing, LaTeX formatting, and code debugging during paper preparation. All research design, experimental methodology, data collection, results, and scientific claims are the authors’ own work. AI was not used for research ideation, experimental design, or interpretation of results.
Acknowledgments
We thank the anonymous ARR reviewers and area chair, whose suggested controls materially improved the paper’s analysis; the ORACC, ETCSL, CEPOAT, and CDLI projects for making their data openly available; and Timo Homburg, together with the creators of the Nuolenna (Jauhiainen, n.d.) and Akkademia (Gordin et al., [10]) sign lists, whose resources underlie our conversion pipeline.
Supplementary Results
Expanded Related Work
This appendix expands the main-body Related Work with detailed citation context. Gutherz et al. [12] developed neural machine translation for Akkadian. Sahala [31] presented a comprehensive computational Assyriology framework. Sahala et al. [32] released BabyLemmatizer, a neural POS tagger and lemmatizer for Akkadian. Sahala and Lindén [33] developed a neural pipeline for cuneiform POS tagging and lemmatization. For Sumerian, Pagé-Perron et al. [24] explored machine translation and Liu et al. [20] performed unsupervised named entity recognition. Punia et al. [25] developed the first neural MT system for Sumerian transliterations. At the sign level, Dencker et al. [7] applied deep learning for cuneiform sign detection, and Gordin et al. [9] developed OCR systems for cuneiform. Williams et al. [42] applied deep learning to Elamite cuneiform sign classification. Smidt et al. [37] reported accuracy losses of up to 24.7 points with Unicode input on Akkadian POS tagging.
For Arabic NLP, diacritization and romanization effects have been studied extensively [5]. Li et al. [18] compared character-level, word-level, and pinyin representations for Chinese segmentation. The substrate decomposition framework we propose generalizes these script-versus-language tradeoffs by isolating the underlying mechanism: which method or representation exploits writing-system-level versus language-level statistical structure.
Separated POS vs. Entity Classification
The unified ORACC POS tagset conflates two fundamentally different classification problems: grammatical category assignment and named entity recognition. We separate the evaluation into three subtasks: (1) grammatical POS with all entity tokens excluded, (2) binary entity detection, and (3) fine-grained entity typing. This separation uncovers a hidden finding. Table 6 shows that Unicode significantly outperforms Latin on Akkadian grammatical POS when entities are excluded (), a result invisible in the unified tagset. Latin’s strong advantage on entity classification (, ) was masking Unicode’s grammatical advantage. For Elamite, Unicode wins or ties on all three subtasks.
| Task | Repr. | AKK | SUX | ELX |
| Gram. | Latin | .669 | .872 | .615 |
| Unicode | .657* | .832 | .634** | |
| Concat | .682* | .888* | .642* | |
| Ent. det. | Latin | .937 | .947 | .873 |
| Unicode | .839 | .896 | .876ns | |
| Concat | .939* | .954* | .882* | |
| Ent. type | Latin | .897 | .957 | .871 |
| Unicode | .833 | .923 | .881* | |
| Concat | .910ns | .963* | .883* |
Table 6. Separated grammatical POS, entity detection, and entity typing (F1-macro). Significance markers are paired-bootstrap -values comparing Concat against the better of Latin and Unicode within each column: ***; **; *. Concatenation wins all 9 comparisons; 8 are significant.
Lemmatization
Character-level LSTM seq2seq models map surface forms to lemmas. Latin wins across all three languages (AKK: 68.8% vs. 64.4%; SUX: 81.9% vs. 77.5%; ELX: 42.5% vs. 25.8% exact match).
This is expected, since lemma targets are themselves in Latin script, and confirms that representation advantages are task-dependent.
Architecture Robustness
To test whether the complementarity effect is an artifact of linear models, we train a 2-layer character Transformer encoder (128-dim, 4-head, trained from scratch) on the same tasks. For the concatenated condition, both representations are fed as a single sequence separated by a delimiter.
The Transformer’s Unicode preference parallels Chinese NLP findings ([21, 38]), where self-attention benefits disproportionately from logographic glyph information. ByT5-small (Xue et al., [43]), a byte-level pretrained model, strongly favors Latin over Unicode (e.g., AKK unified: .571 vs. .334). This result is consistent with the substrate-decomposition framework rather than against it: ByT5’s pretraining corpus is overwhelmingly Latin-script and contains essentially no cuneiform Unicode, so the model’s representational machinery is tuned to recover language-specific lexical patterns from Latin-encoded text. Unicode cuneiform offers the same script-level substrate to ByT5 that it offers to TP, but ByT5 cannot exploit it because its prior knowledge does not include the relevant sign-level statistical regularities. The from-scratch character Transformer above, by contrast, can learn the script-level substrate during training and shows the predicted Unicode advantage. ByT5 therefore illustrates that the script-level substrate is exploitable only by models whose representational capacity is aligned with it, either by training from scratch (the character Transformer) or by carrying no prior bias (TP’s parameter-free statistics).
Feature Importance: What Does Each Representation Encode?
To understand why concatenation helps, we examine the logistic regression model trained on concatenated features. For each POS class, we compute what fraction of total absolute weight falls on Unicode-side versus Latin-side features (Figure 3).

Figure 3. Unicode feature weight share by POS category. Elamite is Unicode-dominant; Akkadian and Sumerian show balanced contributions for open-class words but Latin dominance for entities and closed-class items.
For Akkadian and Sumerian, open-class content words (NOUN, VERB) draw roughly equally from both representations (49–54% Unicode share), while closed-class categories (DET: 29%, CONJ: 34%) and named entities (DN: 38–41%, GN: 39–40%) lean heavily toward Latin. (Per-language class inventories and entity ratios for these categories are those of Table 1.) For Elamite, Unicode features dominate every category (56–69%). Nouns and verbs in cuneiform tend to be compositionally rich (multiple signs encoding stem, tense, person), where Unicode captures sign-level compositional patterns naturally. Closed-class functional words are typically short (one or two signs), where the Latin reading value provides more discriminative power than sign identity alone.
Error Analysis
For Akkadian, 18.4% of tokens are correctly classified by one representation but not the other (Table 8). The effect is mediated by sign count: for Sumerian, Latin wins on single-sign words by a 13:1 ratio, where polyphony makes a single Unicode glyph ambiguous. As sign count increases, Unicode’s compositional advantage grows, reaching parity at 4–5 signs.
| Both correct | Both wrong | Latin only | Uni. only | |
| AKK | 63.1% | 18.5% | 12.4% | 6.0% |
| SUX | 85.2% | 5.2% | 7.5% | 2.1% |
| ELX | 66.9% | 20.2% | 5.5% | 7.3% |
Table 8. Error outcome distribution. “Latin only” and “Unicode only” columns represent the complementary pool (9.6–18.4%).
The confusion patterns are linguistically revealing. Unicode’s most common Akkadian error is misclassifying nouns as divine names (DN): it recognizes the DINGIR determinative sign but cannot distinguish its semantic from its logographic use. Latin resolves this via contextual reading. The complementary pool is largest for rare tokens (Table 9): in Akkadian, 28.9% of rare tokens are correctly classified by only one representation, compared to 19.1% for frequent tokens.
| Rare (≤26) | Frequent (>26) | |
| AKK | 28.9% | 19.1% |
| SUX | 16.3% | 8.5% |
Table 9. Percentage of tokens correctly classified by exactly one representation, split by token frequency.
Scale Effects and Embedding Space
Unicode’s individual advantage decreases with corpus size: Elamite favors Unicode on all tasks while larger corpora generally favor Latin. Yet the concatenation benefit is consistent across all scales, suggesting the complementarity is fundamental rather than a small-data artifact. FastText embeddings reveal a complementary dissociation: Unicode achieves higher morpheme coherence across all three languages (AKK: .796 vs. .764; SUX: .839 vs. .791; ELX: .644 vs. .639), while Latin produces better POS-based silhouette scores and higher POS agreement in nearest-neighbor graphs (AKK: .645 vs. .512; SUX: .741 vs. .638). Unicode organizes by form, Latin by function.
Embedding Space Metrics
Table 10 provides full embedding space metrics.
| Metric | Repr. | AKK | SUX | ELX |
| Silhouette | Latin | −.100 | −.071 | −.037 |
| Unicode | −.093 | −.147 | −.063 | |
| Morph. coh. | Latin | .764 | .791 | .639 |
| Unicode | .796 | .839 | .644 | |
| Graph agr. | Latin | .645 | .741 | .460 |
| Unicode | .512 | .638 | .452 |
Table 10. Embedding metrics (cosine similarity).
ByT5 Detailed Results
Table 11 provides ByT5-small POS classification results across both task configurations. Latin strongly outperforms Unicode in every cell, consistent with ByT5’s pretraining bias toward Latin-script text. Concatenating Latin and Unicode does not improve over Latin alone in any cell, in contrast to the from-scratch character Transformer (Table 7). This confirms that pretrained byte-level models cannot exploit the cuneiform script substrate without script-specific pretraining: ByT5’s representational machinery has been tuned on Latin-script data and cannot recover sign-level statistical regularities from Unicode cuneiform, even when the relevant graphemic information is present in the input.
| Task | Lang | Best | Cat. | |
| Uni. | AKK | .373 | .412 | +.039 |
| SUX | .428 | .451 | +.023 | |
| ELX | .471 | .489 | +.018 | |
| Gram. | AKK | .516 | .513 | −.003 |
| SUX | .630 | .654 | +.024 | |
| ELX | .465 | .469 | +.004 |
Table 7. Character Transformer: best single representation vs. concatenation. Concat wins 5/6 Transformer comparisons; combined with logistic regression (Table 5): concat wins 17/18 across both architectures.
| Task | Repr. | AKK | SUX | ELX |
| Unified | Latin | .571 | .671 | .564 |
| Unicode | .334 | .422 | .543 | |
| Concat | .548 | .650 | .571 | |
| Gram. | Latin | .542 | .728 | .529 |
| Unicode | .401 | .508 | .516 | |
| Concat | .529 | .705 | .534 |
Table 11. ByT5-small POS classification (F1-macro). Latin strongly outperforms Unicode, consistent with Latin-script pretraining bias.
TP Threshold Sweep
Table 12 reports TP word boundary F1 across a sweep of decision thresholds for each language. The F1 surface is relatively flat for Akkadian and Sumerian across a wide range (within 0.01 of the maximum for on both languages), while Elamite shows a steeper dependence on threshold (F1 climbs from 0.591 at to 0.989 at ). The differing optima reflect language-specific bigram distributions: Elamite’s small corpus produces sparser bigram counts, requiring a more permissive threshold to detect word boundaries. The flatness near the optimum on Akkadian and Sumerian explains why source-tuned thresholds transfer reasonably well across these two languages despite slight per-language differences, and why the cross-language transfer matrix (Table 3) is robust to the precise threshold choice.
| AKK | SUX | ELX | |
| 0.20 | .963 | .957 | .591 |
| 0.30 | .967 | .966 | .685 |
| 0.50 | .968 | .971 | .897 |
| 0.70 | .971 | .971 | .954 |
| 0.90 | .961 | .956 | .984 |
| 0.95 | .949 | .943 | .989 |
Table 12. TP word boundary F1 at selected thresholds.
Hyperparameter Robustness: Morfessor Corpusweight
Morfessor’s single hyperparameter, corpusweight, controls the lexicon-size/likelihood trade-off. Table 13 sweeps it in-language: F1 is stable for corpusweight but collapses catastrophically above it, correcting an overly broad stability statement in the submitted version of § 4.1. Source-side selection (in-language CV on the source, the only tuning admissible for zero-shot transfer) picks corpusweight = 0.5 in all three languages, decisively so only for Elamite (+0.014 over 1.0, beyond fold noise; Akkadian +0.0001 and Sumerian +0.0014 are within it). Table 14 re-runs the full transfer matrix at the selected value: Morfessor improves everywhere, most dramatically from the Elamite source (→AKK 0.304 → 0.765), yet remains 19–26 F1 points below untuned TP from that source. Hyperparameter tuning therefore softens but does not overturn the dissociation, and the coverage mechanism of § 4.1.1 is unchanged: the clamped-inventory intervention at corpusweight 0.5 still collapses transfer (AKK 0.732 from clamped full Sumerian vs. 0.977 unclamped).
| corpusweight | ELX | SUX | AKK |
| 0.25 | .998 | .994 | .996 |
| 0.5 | .999 | .994 | .997 |
| 1.0 (paper) | .985 | .993 | .996 |
| 2.0 | .063 | .076 | .122 |
| 4.0 | .057 | .065 | .065 |
caption: tbl:13: In-language 5-fold CV F1 by corpusweight. Bold = source-side selected value.
:::
| Src | Tgt | cw=1.0 | cw=0.5 | TP |
| AKK | SUX | .981 | .982 | .967 |
| AKK | ELX | .984 | .997 | .992 |
| SUX | AKK | .857 | .977 | .960 |
| SUX | ELX | .953 | .993 | .993 |
| ELX | AKK | .304 | .765 | .959 |
| ELX | SUX | .267 | .713 | .969 |
caption: tbl:14: Zero-shot Morfessor transfer at the published default vs. the source-selected corpusweight, against untuned TP. Bold = TP’s advantage from the smallest source survives tuning.
:::
Source-Size Transfer Grid
Table 15 reports the full grid behind the coverage analysis of § 4.1.1: 14 source-size configurations 5 seeds (140 evaluations; document-level subsampling; corpusweight 1.0). Across the grid, the Morfessor lexicon’s coverage of the target corpus predicts transfer F1 at Pearson ; transfer swings by more than 60 points as size and composition vary, while TP’s published values (0.959–0.995) hold from every source.
| Source | Size | AKK | SUX | ELX |
| AKK | 5K | — | .615 | .826 |
| AKK | 13K | — | .743 | .955 |
| AKK | 25K | — | .826 | .973 |
| AKK | full | — | .981 | .984 |
| SUX | 5K | .482 | — | .554 |
| SUX | 13K | .595 | — | .710 |
| SUX | 25K | .662 | — | .757 |
| SUX | full | .858 | — | .953 |
| ELX-exp | 5K | .517 | .414 | — |
| ELX-exp | 13K | .587 | .478 | — |
| ELX-exp | 25K | .651 | .527 | — |
| ELX-exp | full | .783 | .712 | — |
| ELX-orig | 5K | .273 | .247 | — |
| ELX-orig | full | .304 | .267 | — |
caption: Table 15: Morfessor zero-shot transfer across the source-size grid (mean over 5 seeds; seed std .091, typically .03). ELX-exp = the 871-document expanded corpus (Appendix § 8.14).
:::
Elamite Training-Source Ablation
The published Elamite segmentation results train on the Utu-našu running text alone; the Elamite Dictionary enters only classification (§ 3). Table 16 verifies with identical held-out running-text folds that the dictionary does no hidden segmentation work: TP barely moves across training sources, and for Morfessor, isolated citation forms provide a worse lexicon than one-tenth the volume of running text, the same coverage effect as § 4.1.1.
| Training source | TP | Morf. | Tokens |
| Dictionary only | .994 | .727 | 68,251 |
| Utu-našu only (paper) | .989 | .985 | 6,759 |
| Combined | .995 | .991 | 75,010 |
caption: tbl:16: Elamite segmentation by training source, evaluated on identical held-out running-text folds.
:::
Expanded Elamite Corpus
During review we expanded the Elamite running corpus from 84 to 871 documents (8.8K to 111,733 segmentation-stream tokens, 12.7) via 787 collated Susa documents from the CEPOAT corpus, using the identical conversion pipeline (90.2% token conversion). Re-running segmentation: in-language TP falls to 0.923 .007 (the original-corpus value re-runs unchanged at 0.989) while Morfessor rises to 0.993 .000, reversing the in-language ordering (F1 = −.069, p < .001). The reversal is a corpus-composition effect: the Susa material introduces numerals, residual unconverted logograms (\approx$10% of tokens), and an administrative register that disrupt TP’s local transition statistics, while Morfessor’s lexicon absorbs them. This sharpens the practical recommendation of the Conclusion: TP for low-resource settings with cleaner data, Morfessor once corpus size and register heterogeneity grow. Transfer from the expanded source improves for Morfessor (AKK .783, SUX .712; Table 15) yet remains 18–25 points below tuned TP, and TP’s own transfer is essentially unchanged (.963/.968 target-tuned; .924/.918 zero-shot at ).
Supervised Neural Baseline
A character-BiLSTM boundary tagger (the same character-BiLSTM family as Appendix § 8.§ 4) trained on gold-segmented Akkadian tokens and evaluated on a fixed 500-document held-out set, against TP and Morfessor trained on the identical subsamples:
| Tokens | BiLSTM (sup.) | TP | Morf. |
| 1,000 | .957 | .961 | .637 |
| 5,000 | .957 | .964 | .908 |
| 13,000 | .962 | .968 | .963 |
| 50,000 | .994 | .968 | .991 |
Table 17. Supervised vs. unsupervised boundary detection on identical training subsamples (seed std .063; .01 except Morfessor at 1K). For calibration, the all-boundaries trivial baseline reaches F1 = 0.961 on this boundary-dense test set; the substantive comparisons are therefore precision-driven and lie above that floor.
Below roughly 50K labeled tokens the supervised tagger does not outperform the unsupervised methods; at 50K it essentially matches what Morfessor learns from the same text without labels (.994 vs. .991), while presupposing gold segmentation, precisely the resource unavailable for under-documented cuneiform languages.
Old Babylonian Case Study
We test the substrate decomposition diachronically on an Old Babylonian corpus of 132 CDLI documents (13,614 words; letters and administrative texts, including the Iltani archive) whose transliteration mixes syllabic Akkadian with Sumerographic writing word by word. Writing mode is labeled per token from the ATF edition markup itself: tokens inside underscore spans are Sumerographic, giving ground-truth labels for 50.3% of tokens. (An orthographic detector of the kind our Hittite analysis uses reaches only P = 0.74 / R = 0.65 against this ATF truth, underscoring the value of native markup over heuristics.) Illegible tokens are excluded, and Unicode conversion (91.4% of tokens; unconverted words split the stream into 1,246 analysis runs totaling 11,589 tokens) uses the paper’s sign mappings. Both methods are trained on the full Neo-Assyrian Akkadian corpus and applied zero-shot, with TP’s threshold target-tuned as in the tuned cells of Table 3.
The register is an extreme version of the Hittite condition: 51% of word boundaries are writing-mode switches, and words average under three signs. Both methods collapse toward the all-boundaries trivial baseline (F1 = 0.540 on this boundary-dense stream): TP reaches F1 = 0.567 at its best threshold, and Morfessor over-segments nearly every position (F1 = 0.551; within-word false-boundary rate 0.92–0.96). Diagnostics attribute the two collapses to the two different mechanisms the paper identifies. Morfessor’s failure is coverage: only 35.4% of OB tokens appear in the Neo-Assyrian lexicon, exactly the regime of § 4.1.1. TP’s failure is not sign novelty (100% of OB signs and 98.6% of sign bigrams are attested in training) but transition instability under dense mode-switching, and its residual behavior remains ordered as the substrate account predicts: boundary recall is highest at switch points (0.865) and lowest inside syllabic text (0.709), while false boundaries are markedly rarer inside Sumerographic spans (0.510 vs. 0.659 per within-word position), whose logographic sequences recur as lexicalized units with stable internal transitions. The case study thus shows the framework’s two failure modes, lexicon under-coverage and within-word continuity violation, co-occurring in one naturally occurring register, and cautions that segmentation statistics trained on one dialect and register of Akkadian do not transfer to another unchanged (cf. Limitations).
Hittite Experiment Details
Hittite corpus ([44]): 170,496 tokens, 7,099 texts. Unicode conversion rate: 98.6%. Documents used for segmentation: 5,896 (filtered to those with 2 clean Unicode words).
In-language 5-fold CV (held-out, matching the same fold splits across both methods):
TP: best F1 = 0.619 at = 0.06.
Morfessor: F1 = 0.918 (0.004), precision = 0.866, recall = 0.977.
Logogram detection: 34.7% of tokens flagged by the default heuristic (multi-character uppercase Latin after stripping determinative wrappers). Robustness variants: 31.2% (require 3 uppercase letters), 28.4% (uppercase-only, no mixed case), 26.1% (exclude period-containing forms). Top detected logograms: NINDA.GUR.RA (1,238), 1-ŠU (1,225), LUGAL-uš (1,122), GAL (1,101), LUGAL-i (857), KI.MIN (592), ŠIR-RU (572), MUNUS.LUGAL (484), GEŠTIN (397), and others.
References
- [1]Rachit Bansal, Himanshu Choudhary, Ravneet Punia, Niko Schenk, Émilie Pagé-Perron, and Jacob Dahl. 2021. How low is too low? A computational perspective on extremely low-resource languages. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: Student Research Workshop, pages 44–59, Online. Association for Computational Linguistics.
- [2]Gian Pietro Basello. 2004. Elam between Assyriology and Iranian studies. In Antonio Panaino and Andrea Piras, editors, Schools of Oriental Studies and the Development of Modern Historiography. Proceedings of the Fourth Annual Symposium of the Assyrian and Babylonian Intellectual Heritage Project, Held in Ravenna, Italy, October 13–17, 2001, volume IV of Melammu Symposia, pages 1–40. Università di Bologna & IsIAO, Milan.
- [3]Jeremy A. Black, Graham Cunningham, Jarle Ebeling, Esther Flückiger-Hawker, Eleanor Robson, Jon Taylor, and Gábor Zólyomi. 2006. The electronic text corpus of Sumerian literature (ETCSL). https://etcsl.orinst.ox.ac.uk/. University of Oxford.
- [4]Mathias Creutz and Krista Lagus. 2007. Unsupervised models for morpheme segmentation and morphology learning. ACM Transactions on Speech and Language Processing, 4(1):1–34.
- [5]Kareem Darwish. 2014. Arabizi detection and conversion to Arabic. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), Doha, Qatar.arxiv.org/abs/1306.6755
- [6]François de Blois. 1992. Review of W. Hinz & H. Koch, Elamisches Wörterbuch. Bibliotheca Orientalis, 49:195–197.
- [7]Tobias Dencker, Pablo Klinkisch, Stefan M. Maul, and Björn Ommer. 2020. Deep learning of cuneiform sign detection with weak supervision using transliteration alignment. PLoS ONE, 15(12):e0243039.DOI
- [8]Grant Frame, Jamie Novotny, and Joshua Jeffers. 2023. Royal inscriptions of the Neo-Assyrian period (RINAP) online. https://oracc.museum.upenn.edu/rinap/. Oracc project. Accessed August 2026.
- [9]Shai Gordin, Morris Alper, Avital Romach, Luis Saenz Santos, Naama Yochai, and Roey Lalazar. 2024. CuReD: Deep learning optical character recognition for cuneiform text editions and legacy materials. In Proceedings of the 1st Workshop on Machine Learning for Ancient Languages (MLAAL 2024), pages 130–140, Hybrid in Bangkok, Thailand and online. Association for Computational Linguistics.
- [10]Shai Gordin, Gai Gutherz, Ariel Elazary, Avital Romach, Enrique Jiménez, Jonathan Berant, and Omer Cohen. 2020. Reading Akkadian cuneiform using natural language processing. PLoS ONE, 15(10):e0240511.DOI
- [11]Françoise Grillot-Susini. 2008. L’Élamite: Éléments de grammaire. Librairie Orientaliste Paul Geuthner, Paris.
- [12]Gai Gutherz, Shai Gordin, Luis Sáenz, Omer Levy, and Jonathan Berant. 2023. Translating Akkadian to English with neural machine translation. PNAS Nexus, 2(5):pgad096.DOI
- [13]Walther Hinz and Heidemarie Koch. 1987. Elamisches Wörterbuch. Archäologische Mitteilungen aus Iran, Ergänzungsband 17. Dietrich Reimer Verlag, Berlin.
- [14]Timo Homburg and Christian Chiarcos. 2016. Word segmentation for Akkadian cuneiform. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 4067–4074, Portorož, Slovenia. European Language Resources Association (ELRA).DOI
- [15]Matthew Honnibal and Ines Montani. 2017. spaCy 2: Natural language understanding with bloom embeddings, convolutional neural networks and incremental parsing. https://spacy.io.
- [17]Tommi Jauhiainen, Heidi Jauhiainen, Tero Alstola, and Krister Lindén. 2019. Language and dialect identification of cuneiform texts. In Proceedings of the Sixth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2019), pages 89–98. Association for Computational Linguistics.arxiv.org/abs/1903.01891
- [18]Xiaoya Li, Yuxian Meng, Xiaofei Sun, Qinghong Han, Arianna Yuan, and Jiwei Li. 2019. Is word segmentation necessary for deep learning of Chinese representations? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3242–3252, Florence, Italy. Association for Computational Linguistics.arxiv.org/abs/1905.05526
- [19]Wang Ling, Chris Dyer, Alan W. Black, Isabel Trancoso, Ramón Fernandez, Silvio Amir, Luís Marujo, and Tiago Luís. 2015. Finding function in form: Compositional character models for open vocabulary word representation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1520–1530, Lisbon, Portugal. Association for Computational Linguistics.arxiv.org/abs/1508.02096
- [20]Yudong Liu, Clinton Burkhart, James Hearne, and Liang Luo. 2015. Enhancing Sumerian lemmatization by unsupervised named-entity recognition. In Proceedings of NAACL-HLT.DOI
- [21]Yuxian Meng, Wei Wu, Fei Wang, Xiaoya Li, Ping Nie, Fan Yin, Muyi Li, Qinghong Han, Xiaofei Sun, and Jiwei Li. 2019. Glyce: Glyph-vectors for Chinese character representations. In Proceedings of NeurIPS.
- [23]Oracc. 2014. Oracc: The open richly annotated cuneiform corpus. https://oracc.museum.upenn.edu/. Steered by Jamie Novotny, Eleanor Robson, Steve Tinney, and Niek Veldhuis. Federated corpus infrastructure for linguistically annotated cuneiform texts. Accessed August 2026.
- [24]Émilie Pagé-Perron, Maria Sukhareva, Ilya Khait, and Christian Chiarcos. 2017. Machine translation and automated analysis of the Sumerian language. In Proceedings of the Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature, pages 10–16, Vancouver, Canada. Association for Computational Linguistics.
- [25]Ravneet Punia, Niko Schenk, Christian Chiarcos, and Émilie Pagé-Perron. 2020. Towards the first machine translation system for Sumerian transliterations. In Proceedings of COLING.
- [26]Enrique Quintana Cifuentes. 2014. Elamita: Web de divulgación de la lengua Elamita. https://www.um.es/cepoat/elamita/. University of Murcia. Accessed August 2026.
- [27]Erica Reiner. 1969. The Elamite Language. In Altkleinasiatische Sprachen, Handbuch der Orientalistik, 1. Abteilung, 2. Band: Keilschriftforschung und Alte Geschichte Vorderasiens, 1–2. Abschnitt, Lieferung 2, pages 54–118. E. J. Brill, Leiden.DOI
- [28]Eleanor Robson. 2001. The tablet house: A scribal school in Old Babylonian Nippur. Revue d’Assyriologie et d’archéologie orientale, 95:39–66.DOI
- [29]Eleanor Robson and Steve Tinney. 2019. QPN: Oracc linguistic annotation for proper nouns. http://oracc.museum.upenn.edu/doc/help/languages/propernouns/. Oracc: The Open Richly Annotated Cuneiform Corpus.
- [30]Jenny R. Saffran, Richard N. Aslin, and Elissa L. Newport. 1996. Statistical learning by 8-month-old infants. Science, 274:1926–1928.pubmed.ncbi.nlm.nih.gov/8943209
- [31]Aleksi Sahala. 2021. Contributions to Computational Assyriology. Ph.D. thesis, University of Helsinki.DOI
- [32]Aleksi Sahala, Tero Alstola, Jonathan Valk, and Krister Lindén. 2022. BabyLemmatizer: A lemmatizer and POS-tagger for Akkadian. In Proceedings of the CLARIN Annual Conference 2022, pages 14–18, Prague, Czech Republic. CLARIN ERIC.
- [33]Aleksi Sahala and Krister Lindén. 2023. A neural pipeline for POS-tagging and lemmatizing cuneiform languages. In Proceedings of the Ancient Language Processing Workshop, pages 203–212, Varna, Bulgaria. INCOMA Ltd., Shoumen, Bulgaria.
- [34]Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of ACL.arxiv.org/abs/1508.07909
- [35]Cole Simmons, Richard Diehl Martinez, and Dan Jurafsky. 2024. SumTablets: A transliteration dataset of Sumerian tablets. In Proceedings of the 1st Workshop on Machine Learning for Ancient Languages (ML4AL 2024), pages 192–202, Bangkok, Thailand. Association for Computational Linguistics.
- [36]Åke W. Sjöberg. 1976. The Old Babylonian Eduba. In Stephen J. Lieberman, editor, Sumerological Studies in Honor of Thorkild Jacobsen on His Seventieth Birthday, June 7, 1974, volume 20 of Assyriological Studies, pages 179–179. University of Chicago Press, Chicago.
- [37]Gustav Ryberg Smidt, Els Lefever, and Katrien de Graef. 2024. At the crossroad of cuneiform and NLP: Challenges for fine-grained part-of-speech tagging. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 1745–1755, Torino, Italia. ELRA and ICCL.
- [38]Zijun Sun, Xiaoya Li, Xiaofei Sun, Yuxian Meng, Xiang Ao, Qing He, Fei Wu, and Jiwei Li. 2021. ChineseBERT: Chinese pretraining enhanced by glyph and pinyin information. In Proceedings of ACL.
- [39]Jan Tavernier. 2020. Elamite. In Rebecca Hasselbach-Andee, editor, A Companion to Ancient Near Eastern Languages, pages 163–184. John Wiley & Sons, Hoboken, NJ.DOI
- [40]Steve Tinney. 1999. On the curricular setting of Sumerian literature. Iraq, 61:159–172.DOI
- [41]Niek Veldhuis. 2004. Religion, Literature, and Scholarship: The Sumerian Composition Našše and the Birds, with a Catalogue of Sumerian Bird Names, volume 22 of Cuneiform Monographs. Brill, Leiden.
- [42]Edward C. Williams, Grace Su, Sandra R. Schloen, Miller C. Prosser, Susanne Paulus, and Sanjay Krishnan. 2025. DeepScribe: Localization and classification of Elamite cuneiform signs via deep learning. ACM Journal on Computing and Cultural Heritage, 18(2):31:1–31:32.arxiv.org/abs/2306.01268
- [43]Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, and Colin Raffel. 2022. ByT5: Towards a token-free future with pre-trained byte-to-byte models. Transactions of the Association for Computational Linguistics, 10:291–306.
- [44]Emma Yavasan and Shai Gordin. 2024. Glossed Hittite texts with German translation for machine learning. Zenodo.DOI