betaFeedback

WorksComputer Science

Mandarin Wh-Indefinites Follow Nonveridicality in Multilingual LLMs, Unlike English any

Abstract

Multilingual LLMs learn mostly from English. Do they carry English polarity licensing over to other languages? English any is licensed by downward-entailing (DE) contexts, Mandarin wh-indefinites such as shenme 'what/something' by nonveridical ones, and the two classes come apart: perhaps licenses shenme but not any, while only licenses any but entails existence. We test this dissociation in a pre-registered study of twelve open LLMs (0.5B-9B, four families), using 48 matched bilingual frames in 21 contexts and a log-probability difference-in-differences normalised by negation. All 11 models that pass the measurement gate license Mandarin wh-indefinites under possibility, necessity and evidential adverbs, at 0.42-2.33 times the strength of negation, while English any gets nothing in the same contexts. An exploratory extension to Qwen2.5-32B shows the same. The English side is weaker: any under only, first and regret is licensed but at a fraction of negation, so only 3 models show the full double dissociation in sentence probabilities. When prompted to choose between sentences, most gated models show it. In exploratory directional ablations, a direction that carries English licensing, held-out classes included, does not reach Mandarin licensing in most models. The models do not simply apply the English rule to Mandarin.

Introduction

Multilingual LLMs learn most of what they know from English, and their internal representations are partly English-anchored ([20]). For a phenomenon that exists in many languages but works differently in each, this raises a concrete question. Does the model learn each language’s own rule, or does it learn the English rule and apply it everywhere?

Polarity-sensitive indefinites are a sharp test case. English any is licensed by downward-entailing

(DE) operators: Tom didn’t buy anything is fine, Tom bought anything is not ([10]). Mandarin 什么 ‘what’ can also be read as ‘something’, but only in licensing contexts. These contexts are not the DE ones. Mandarin wh-indefinites are licensed by nonveridicality, that is, by any context that does not entail the event ([12, 21, 11]). The two classes overlap under negation, questions and conditionals, and they come apart in both directions:

  • 张伟昨天可能买了什么东西 ‘Zhang Wei perhaps bought something’ is natural, but Perhaps Tom bought anything is not: epistemic adverbs are nonveridical but not DE.

  • Only Tom bought anything is natural, but 只有张伟买了什么东西 is predicted to be degraded: only is Strawson-DE but entails that Zhang Wei bought something.

A model with one shared, English-shaped licensing detector would get the second contrast and miss the first.

We test this in a pre-registered study with twelve open LLMs from four families (0.5B–9B), and extend it in an exploratory run to Qwen2.5-32B-Instruct. The items are 48 frames in 21 contexts, with matched Mandarin and English versions. The measure is a log-probability difference-in-differences that cancels frame plausibility and context length, normalised by negation. All 11 gated models show the Mandarin side of the dissociation. Three LLM raters reading the same items show the same contrast. A second, exploratory protocol asks how the two systems relate inside the model. Ablating a direction that carries English licensing removes English licensing in 11 of 11 models, but Mandarin licensing beyond random in only 4. Asked directly to choose between sentences, most gated models also show the English side of the dissociation.

Contributions.

  • A controlled bilingual test of DE versus nonveridical licensing, released with code, items and all model outputs. It covers contexts that existing Chinese benchmarks do not: existential 什么/谁 under possibility, necessity and evidential adverbs.

  • A pre-registered finding that holds in all 11 gated models and both base models. Mandarin licensing under nonveridical, non-DE adverbs is as strong as half to twice that of negation (LzhL_{\mathrm{zh}} 0.42–2.33), while English any in the same contexts gets none (LenL_{\mathrm{en}} −0.26–0.15). The effect survives two controls, two object families, an alternative denominator and two adverb baselines (165/165 cells).

  • An account of the weaker English side. Any under only/first/regret is licensed but under-learned, and the Mandarin prediction in those contexts is itself context-dependent.

  • Two exploratory follow-ups, locked before their outputs. Directional ablation shows that the English licensing direction stays in English in most models, while a Mandarin direction works in some families and not in others. A prompted forced choice shows the full double dissociation in most gated models.

Design

The contrast

English any is licensed by downward-entailing (DE) operators, including Strawson-DE ones such as only, superlatives, and emotive factives [10, 17]. Mandarin existential 什么/谁 (‘something’/‘someone’ on their indefinite reading) is licensed more broadly. Any nonveridical context licenses them, that is, any context that does not entail that the event happened [12, 21, 11]. Nonveridicality includes DE but goes beyond it. Two predictions follow, one on each side:

  • zh-advantage (ZA). Epistemic possibility (可能 ‘perhaps’), necessity (肯定 ‘surely’) and evidentials (好像 ‘apparently’) are nonveridical but not DE. They should license 什么 but not any: 张伟昨天可能买了什么东西 ‘Zhang Wei perhaps bought something’ versus *Perhaps Tom bought anything.

  • en-advantage (EA). Only, the first to and regret are Strawson-DE, but they entail that the event happened. They should license any but not 什么, since 只有张伟买了 ‘only Zhang Wei bought’ entails that he bought (the non-entailment-of-existence condition, Lin [12]).

A single English-shaped NPI detector predicts the English pattern in both languages. Two language-specific licensing systems predict the double dissociation.

Items

Each item is a frame placed in one of 21 contexts (Table 2 in the appendix). There are 48 frames: 24 with a thing object (买了什么东西 / bought anything) and 24 with a person object (见了谁 / met anyone). Each frame has three objects:

Contextzh/enMandarin (T)English (T)
BASE−/−张伟昨天买了什么东西…Tom bought anything yesterday…
VER1−/−张伟昨天确实买了什么东西…Tom indeed bought anything yesterday…
VER2−/−张伟昨天果然买了什么东西…Tom predictably bought anything yesterday…
VER3−/−张伟昨天终于买了什么东西…Tom finally bought anything yesterday…
VER4−/−张伟昨天已经买了什么东西…Tom already bought anything yesterday…
MAN1−/−张伟昨天偷偷买了什么东西…Tom secretly bought anything yesterday…
MAN2−/−张伟昨天匆匆买了什么东西…Tom hastily bought anything yesterday…
NEG+/+张伟昨天没买什么东西…Tom didn’t buy anything yesterday…
QYN+/+张伟昨天买了什么东西吗?Did Tom buy anything yesterday?
COND+/+如果张伟昨天买了什么东西,请马上告诉我。If Tom bought anything yesterday, please tell me right away.
POSS1+/−张伟昨天可能买了什么东西…Perhaps Tom bought anything yesterday…
POSS2+/−张伟昨天也许买了什么东西…Tom might have bought anything yesterday…
POSS3+/−张伟昨天大概买了什么东西…Tom probably bought anything yesterday…
POSS4+/−张伟昨天或许买了什么东西…Maybe Tom bought anything yesterday…
NEC1+/−张伟昨天肯定买了什么东西…Tom surely bought anything yesterday…
NEC2+/−张伟昨天一定买了什么东西…Tom must have bought anything yesterday…
EVID1+/−张伟昨天好像买了什么东西…Tom apparently bought anything yesterday…
EVID2+/−张伟昨天似乎买了什么东西…Tom seemingly bought anything yesterday…
ONLY−/+昨天只有张伟买了什么东西…Only Tom bought anything yesterday…
FIRST−/+张伟是昨天第一个买什么东西的人…Tom was the first to buy anything yesterday…
REGRET−/+张伟后悔昨天买了什么东西…Tom regretted having bought anything yesterday…

Table 2. The 21 contexts, shown for thing frame 0 with the target object; the second column gives the predicted licensing. … stands for the shared continuation ,然后就回家了。 /, and then went home. The controls replace the object with 那些东西/those things (C1) or 东西/something (C2). Person frames use 谁, 那些人, 某个人 and anyone, those people, someone.

  • the target T (什么东西, 谁; anything, anyone);

  • a definite control C1 (那些东西, 那些人; those things, those people);

  • an indefinite control C2 (东西, 某个人; something, someone).

This gives 48×21×3×2=6,04848 \times21 \times3 \times2 = 6{,}048 sentences per model.

The contexts form seven classes:

  • an episodic baseline (BASE);

  • four veridical adverbs (VERC: indeed, predictably, finally, already);

  • two manner adverbs (MAN: secretly, hastily);

  • the three shared licensers, negation (NEG), polar questions (QYN) and conditionals (COND);

  • the ZA classes POSS (4 adverbs), NEC (2) and EVID (2);

  • the three EA contexts.

The two adverb classes test whether a ZA effect merely reflects an extra preverbal adverb.

All contexts except QYN and COND, which have their own endings, close with the same continuation (,然后就回家了 / and then went home). The continuation forces the declarative parse, because without it a bare 张伟昨天买了什么东西 is a perfectly good question. Pilot items without it allowed an embedded-question parse, so it was fixed before the main study.

Measure

For frame ff and context cc, let LP be the summed log-probability of the whole sentence. We score with the model’s BOS token, in bf16, and without a chat template. The licensing effect is a difference-in-differences:

DiD⁡(f,c)=[LP⁡(T)−LP⁡(C1)]f,c−[LP⁡(T)−LP⁡(C1)]f,BASE.\operatorname{DiD}(f,c) = [\operatorname{LP}(\mathrm{T})-\operatorname{LP}(\mathrm{C1})]_{f,c} -[\operatorname{LP}(\mathrm{T})-\operatorname{LP}(\mathrm{C1})]_{f,\mathrm{BASE}}.

The control C1 removes everything that the target and the control share, such as frame plausibility and context length. The BASE term removes the target’s baseline cost.

We call this relative effect licensing throughout; it is graded and does not by itself separate grammatical from ungrammatical sentences. We normalise by negation, L(c)=DiD⁡‾(c)/DiD⁡‾(NEG)L(c)=\overline{\operatorname{DiD}}(c)/\overline{\operatorname{DiD}}(\mathrm{NEG}), so L=1L=1 means “as licensed as under negation”. The cross-language contrast is Δ(c)=Lzh(c)−Len(c)\Delta(c)=L_{\mathrm{zh}}(c)-L_{\mathrm{en}}(c).

Contexts within a class are averaged frame by frame. Uncertainty comes from a frame bootstrap stratified by object family (10,000 resamples, seed 0). Each resample recomputes the denominator.

Pre-registered criteria

The protocol, criteria and model list were committed before any main-study output existed. Only report-only amendments were added later, and each one is dated in the released log.

Measurement gate (M). NEG and QYN must have DiD⁡>0\operatorname{DiD}>0 in both languages, with a CI that excludes 0 and ≥70%\ge70\% of frames positive. Models that fail M are reported but not counted.

ZA(k)ZA(k), for k∈{POSS,NEC,EVID}k \in\{\mathrm{POSS},\mathrm{NEC},\mathrm{EVID}\}, requires all of the following:

  • Δ(k)>0\Delta(k)>0, with a CI that excludes 0 and ≥60%\ge60\% of frames positive;

  • Lzh(k)>0L_{\mathrm{zh}}(k)>0, with a CI that excludes 0;

  • Lzh(k)L_{\mathrm{zh}}(k) above both Lzh(VERc)L_{\mathrm{zh}}(\mathrm{VERc}) and Lzh(MAN)L_{\mathrm{zh}}(\mathrm{MAN}), each with a CI that excludes 0.

EA(k)EA(k), for k∈{ONLY,FIRST,REGRET}k \in\{\mathrm{ONLY},\mathrm{FIRST},\mathrm{REGRET}\}, is the mirror image: Δ(k)<0\Delta(k)<0 and Len(k)>0L_{\mathrm{en}}(k)>0.

A model’s zh side holds if ZA passes in ≥2\ge2 of the 3 classes; its en side holds if EA passes in ≥2\ge2 of 3.

H1 requires ≥75%\ge75\% of the gated models to show the zh side, including at least one model that is neither Qwen nor Llama. H2 requires ≥50%\ge50\% to show the en side.

Models

We test twelve open models from four families. All are scored with Transformers 5 on single GPUs:

  • Qwen2.5: 0.5B, 1.5B, 3B and 7B Instruct, plus 7B base;

  • Llama 3.x: 3.2-1B and 3.2-3B Instruct, 3.1-8B Instruct, plus 3.1-8B base;

  • Qwen3-8B;

  • two Chinese-centric models from other developers, GLM-4-9B-Chat and Yi-1.5-9B-Chat.

Behavioural results

Gate. 11 of the 12 models pass the measurement gate. Qwen3-8B fails: its Mandarin negation DiD is only 0.47 (95% CI [−0.14-0.14, 1.07]), with 60% of frames positive, while its QYN and COND cells behave normally. Because LL divides by the negation cell, its LzhL_{\mathrm{zh}} values are not interpretable. Following the protocol, we report it (Table 1) but do not count it.

Model\multicolumn{6}{c}{Nonveridical, not DE (prediction: zh +, en −)}zh\multicolumn{6}{c}{DE, existence-entailing (prediction: zh −, en +)}en
\multicolumn{3}{c}{LzhL_{\mathrm{zh}}}\multicolumn{3}{c}{LenL_{\mathrm{en}}}\multicolumn{3}{c}{LzhL_{\mathrm{zh}}}\multicolumn{3}{c}{LenL_{\mathrm{en}}}
POSSNECEVIDPOSSNECEVIDONLYFIRSTREGRONLYFIRSTREGR
Qwen2.5-0.5B-Inst0.920.731.630.130.070.15✓0.570.490.470.240.510.28
Qwen2.5-1.5B-Inst0.680.611.760.060.030.06✓0.03−0.280.860.500.500.34
Qwen2.5-3B-Inst0.560.420.95−0.09−0.08−0.00✓0.44−0.330.460.100.590.38
Qwen2.5-7B-Inst0.760.481.510.08−0.010.10✓0.620.36−0.430.190.450.21
Qwen2.5-7B0.700.451.260.01−0.05−0.04✓0.670.69−0.210.380.500.17
Llama-3.2-1B-Inst0.640.520.900.060.010.09✓−0.330.820.410.390.560.23
Llama-3.2-3B-Inst0.520.671.29−0.06−0.000.00✓0.080.670.440.470.560.14
Llama-3.1-8B-Inst0.821.081.480.010.03−0.01✓0.100.320.340.420.530.29✓
Llama-3.1-8B0.950.891.60−0.03−0.020.01✓−0.280.330.030.500.520.24✓
GLM-4-9B-Chat1.190.882.33−0.03−0.020.02✓0.731.08−0.560.430.460.27
Yi-1.5-9B-Chat0.850.592.16−0.16−0.26−0.02✓0.62−0.28−0.350.250.720.22✓
Qwen3-8B†0.28−3.341.930.110.120.13−6.94−3.43−3.510.100.580.15

Table 1. Licensing strength LL (DiD relative to negation; L=1L = 1 means as licensed as under negation, L=0L = 0 no licensing beyond the episodic baseline). zh/en: the model passes the pre-registered zh-side (ZA in ≥2 of 3 classes) or en-side (EA in ≥2 of 3) criterion. †Fails the measurement gate (its Mandarin negation cell collapses), so it is reported but not counted.

The Mandarin side holds everywhere (H1). All 11 gated models show the zh side, and every one passes all three ZA classes (Figure 1, blue). This includes GLM-4 and Yi-1.5, the two models outside the Qwen and Llama families. The effect is large:

Figure 1. Licensing strength in Mandarin (LzhL_{\mathrm{zh}}) against English (LenL_{\mathrm{en}}), one point per gated model and context class. Negation is 1 on both axes by definition; the dashed line is Lzh=LenL_{\mathrm{zh}}=L_{\mathrm{en}}. Nonveridical, non-DE adverbs (ZA, blue) license Mandarin wh-indefinites but not English any. DE, existence-entailing contexts (EA, orange) license any weakly, and Mandarin 什么 in some models. Black outlines mark the exploratory Qwen2.5-32B-Instruct, which falls in the same regions.

  • LzhL_{\mathrm{zh}} ranges from 0.42 to 2.33, so possibility, necessity and evidential adverbs license 什么/谁 at roughly half to twice the strength of negation;

  • LenL_{\mathrm{en}} in the same contexts ranges from −0.26 to 0.15, so perhaps, surely and apparently do nothing for any.

The effect is not an adverb artefact: all 66/66 comparisons against the veridical and manner adverb classes are positive with CIs excluding 0. The sign of Δ is also stable across all 165/165 model × class × variant cells. The variants are the C1 and C2 controls, a three-licenser denominator, and the thing and person families on their own.

The English side is weaker and model-dependent (H2 not met). Only 3 of the 11 models show the en side, against the pre-registered 50%. The failure is not that any goes unlicensed. LenL_{\mathrm{en}} is reliably above 0 under only (10/11 models), the first to (11/11) and regret (11/11). Its size, however, is only 0.10–0.72 of negation. Two things make Δ fail:

  • English licensing is weak in these contexts (gated-model means: ONLY 0.35, FIRST 0.54, REGRET 0.25);

  • many models also give Mandarin 什么 some credit there (ONLY 0.30, FIRST 0.35, REGRET 0.13).

Δ < 0 holds in only 5, 7 and 5 models respectively. Following the protocol, the claim narrows to its pre-registered fallback. Mandarin nonveridical licensing is robust across families and scales; the English DE side varies by model.

Scale and post-training. The zh-side result has no scale threshold: it holds from 0.5B to 9B. It also holds for both base models (Qwen2.5-7B, Llama-3.1-8B), so it is not a product of instruction tuning. An exploratory extension outside the pre-registered list runs Qwen2.5-32B-Instruct through the unchanged pipeline, with its reading fixed before any output. The model passes the gate, and the Mandarin side extends to it: all 3 ZA classes are licensed, at 0.85–1.92 times negation, while English any gets at most 0.05 there. The English side does not improve with scale. Only 1 of the three EA classes holds, and mean English licensing under EA (0.31) lies within the range of the 0.5B–7B Qwen2.5 models (0.28–0.45).

Do the items mean what the theory says?

The predictions come from the theoretical literature. This section asks how the target sentences actually read, using two report-only checks that were added to the protocol before all models were in. Neither check enters the criteria.

Native-speaker screening. A native Mandarin speaker (one of the authors) screened all 48 frames and rated 168 target sentences on a 1–5 scale. No frame was flagged as unnatural, so the planned sensitivity analysis that drops flagged frames is identical to the main analysis.

The screener rated almost every sentence as acceptable, including the EA contexts. We then showed the screener the 11 sentences that all three LLM raters scored ≤2\le2. They judged 8 of them acceptable once a discourse context was supplied, for example 昨天只有张伟见了谁 and 高翔是昨天第一个推谁的人. In each such context, 谁 picks out a particular person, such as the person one had been hoping to see. The other 3 sentences stayed degraded: 已经见了谁 and two 第一个见/遇到谁的人 sentences. We return to this context sensitivity in § 7.

LLM ratings. Three LLM raters from different developers rated all 2,016 target sentences and all 2,016 C1 controls (temperature 0, shuffled batches of 12). We treat these as LLM ratings, not human judgements. Inter-rater Spearman correlations per sentence are 0.44–0.53 for Mandarin and 0.72–0.85 for English.

Averaged over raters, the ratings reproduce the theoretical contrast:

  • Mandarin targets under POSS/NEC/EVID score close to the shared licensers, while English any there scores near the floor (1.4 on 1–5 at BASE);

  • English any under ONLY/FIRST/REGRET scores 4.1/4.0/3.9, while the Mandarin targets score 2.5/1.9/2.8.

Across the 20 non-baseline contexts, the mean rating tracks the gated models’ mean LL, with Spearman ρ=0.80\rho= 0.80 for Mandarin and 0.72 for English. A rating DiD built like the model DiD gives ρ=0.80\rho= 0.80 and 0.67.

The ratings also locate the English shortfall. The raters treat any under only, the first to and regret as nearly as good as under negation. The models license it at only a fraction of negation. The English side therefore fails because the models under-license Strawson-DE any, not because the items are bad.

Is the licensing signal shared across languages?

The behavioural dissociation leaves open how the two systems relate inside the model. An English-shaped detector could still exist and be overridden for Mandarin, or the two languages could rely on separate signals. We test this with directional interventions and a prompted forced choice. This second protocol was locked after the behavioural verdict and before any of its outputs.

Directions. At every layer ℓ\ell we take the residual stream at the last token before the object, hℓh_\ell. For language gg, the licensing direction is

dg(ℓ)=hˉℓ[LIC]−hˉℓ[UNL],d_g(\ell) = \bar{h}_\ell[\mathrm{LIC}] - \bar{h}_\ell[\mathrm{UNL}],

where LIC is NEG, QYN and COND and UNL is BASE, VERc and MAN. This is the difference of means between the shared licensers and the unlicensed contexts, paired over the same 48 frames. The held-out classes, POSS/NEC/EVID and ONLY/FIRST/REGRET, never enter a direction. Three kinds of control direction are also built:

  • a lexical check dgNEGd_g^{\mathrm{NEG}}, built from QYN and COND only and tested on the negation cell it never saw;

  • an adverb control dgVER=VERc−BASEd_g^{\mathrm{VER}} = \mathrm{VERc} - \mathrm{BASE};

  • 20 random Gaussian directions per layer.

Interventions. Ablation projects each layer’s own unit direction ug(ℓ)u_g(\ell) out of that layer’s block output, h←h−(h⋅u)uh \leftarrow h-(h\cdot u)u, at every position and every layer. A sensitivity variant ablates only the first half of the layers. The effect on class kk is the removal share R(k)=1−DiDabl(k)/DiD(k)R(k) = 1-\mathrm{DiD}_{\mathrm{abl}}(k)/\mathrm{DiD}(k). Addition adds αug\alpha u_g at the middle layer to BASE sentences, with α\alpha set to the norm of the target language’s dgd_g at that layer.

The pre-registered test ablates uenu_{\mathrm{en}} and asks what happens to Mandarin. A shared English-shaped signal predicts that uenu_{\mathrm{en}} removes Mandarin negation licensing and, to a similar degree, Mandarin ZA licensing. The categories are:

  • shared: RNEGR_{\mathrm{NEG}} and RZAR_{\mathrm{ZA}} are both above the random 95th percentile, and RZAR_{\mathrm{ZA}} is not more than 0.25 below RNEGR_{\mathrm{NEG}} (bootstrap CI);

  • separate: RNEGR_{\mathrm{NEG}} is above the random 95th percentile, but RZAR_{\mathrm{ZA}} is not, or RZAR_{\mathrm{ZA}} is more than 0.25 below RNEGR_{\mathrm{NEG}} (CI);

  • neither: RNEGR_{\mathrm{NEG}} is not above the random 95th percentile;

  • partial: anything else.

A category describes the model set if it holds in a majority of the assessable models.

Measure, and an amendment. The protocol scored only the object word given the prefix. The first model’s measurement check showed that this misses most of the Mandarin effect: across models it tracks the behavioural DiD only at ρ=0.57\rho= 0.57–0.890.89 for Mandarin. The reason is linguistic. 张伟昨天买了什么东西 is a fine question, so 什么 is not dispreferred at the object; the cost arrives with the declarative continuation. (For QYN the Mandarin prefix equals BASE, so the object-only DiD is zero by construction.) After inspecting that first model we amended the protocol to also score object plus continuation, and reran all models. This measure scores the same tokens as the behavioural DiD, so its agreement with it (ρ≥0.90\rho\ge0.90) confirms the alignment rather than replicating the result. Directions, layers, random baselines and categories are unchanged. We report both measures below; the continuation measure is the post-hoc one.

Results: the English direction stays in English. Figure 2 (left) shows the test. Ablating uenu_{\mathrm{en}} removes English negation licensing beyond random in 11 of 11 models (R=0.21R = 0.21–0.760.76), and it also removes the held-out EA licensing in 8. The direction therefore captures English licensing, including classes it was not built from. The same direction removes Mandarin negation licensing beyond random in only 4 models, and Mandarin ZA licensing in 4. Under the pre-registered categories the model set reads neither: 7 of 11 models, against 1 shared, 2 separate and 1 partial. Ablating only the first half of the layers gives the same reading (7 neither, 3 shared). On the pre-specified object-only measure the Mandarin NEG or ZA effect is not positive in 4 models, which therefore cannot be classified. No category reaches a majority of the other 7 when all layers are ablated; with the first half ablated, 6 of them read neither (Table 4).

Figure 2. Directional ablation per model, on the continuation measure. Each point is a removal share RR; the grey dash next to it is the 95th percentile of the same share over 20 random directions. Filled markers are the negation cell, which the direction was built with; hollow markers are held-out classes (ZA: POSS/NEC/EVID; EA: ONLY/FIRST/REGRET). The word under each model in the left panel is its pre-registered category. Values are in Table 3; the object-only measure is in Table 4.

ModelMandarin licensing removed: by νen\nu_{\mathrm{en}} NEGMandarin licensing removed: by νen\nu_{\mathrm{en}} ZAMandarin licensing removed: random r95 NEGMandarin licensing removed: random r95 ZAMandarin licensing removed: by νzh\nu_{\mathrm{zh}} NEGMandarin licensing removed: by νzh\nu_{\mathrm{zh}} ZAEnglish licensing removed: by νen\nu_{\mathrm{en}} NEGEnglish licensing removed: by νen\nu_{\mathrm{en}} EAEnglish licensing removed: random r95 NEGEnglish licensing removed: random r95 EAEnglish licensing removed: by νzh\nu_{\mathrm{zh}} EAcategory
Qwen2.5-0.5B-Inst2.821.791.480.494.931.821.312.950.161.261.04partial
Qwen2.5-1.5B-Inst−0.42−0.431.600.753.881.241.011.710.120.310.54neither
Qwen2.5-3B-Inst−0.37−0.240.220.56−0.360.150.832.060.100.320.30neither
Qwen2.5-7B-Inst0.75−0.390.670.181.390.090.711.540.070.51−0.27separate
Qwen2.5-7B1.310.030.280.140.550.350.761.200.050.22−0.22separate
Llama-3.2-1B-Inst3.781.300.610.323.181.220.292.250.100.400.11partial
Llama-3.2-3B-Inst–1.13–0.32–1.380.721.870.090.280.70n/a
Llama-3.1-8B-Inst–0.20–0.21–1.460.831.420.070.20−0.05n/a
Llama-3.1-8B0.450.170.230.101.860.860.881.910.050.180.83partial
GLM-4-9B-Chat–0.90–0.30–2.461.052.140.040.131.72n/a
Yi-1.5-9B-Chat––––––0.541.960.120.220.25n/a

Table 4. The same ablation on the pre-specified object-only measure. It misses most of the Mandarin effect (§ 5); – marks classes whose unablated DiD is ≤0\le0 (not assessable).

The Mandarin direction is model-dependent. The null for uenu_{\mathrm{en}} means something only where a Mandarin direction works at all (Figure 2, right). Ablating uzhu_{\mathrm{zh}} removes Mandarin negation and held-out ZA licensing together in 7 models: all Llama, GLM and Yi models and Qwen2.5-0.5B. In Qwen2.5-1.5B to 7B it removes neither, so in these models our difference-of-means direction has no detectable effect on Mandarin licensing, and their neither says nothing about sharing. Among the 7 models where uzhu_{\mathrm{zh}} works, only Llama-3.1-8B-Inst is shared. In the others the English direction misses Mandarin negation (three models), removes it without the ZA classes (two), or removes both by clearly different amounts (Yi-1.5). Our conclusion is therefore limited: the English licensing direction reliably affects English licensing, but it does not reliably reach Mandarin licensing, and the Mandarin direction’s effect depends on the model. This does not exclude a shared representation that is non-linear or sits elsewhere.

Addition. Adding a direction to BASE sentences at the middle layer raises English any above random for uenu_{\mathrm{en}} in 10 models, and also for uzhu_{\mathrm{zh}} in 9. Mandarin 什么 rarely rises above random (uzhu_{\mathrm{zh}}: 1 model; uenu_{\mathrm{en}}: 3). The addition test is thus uninformative for Mandarin, and the English result suggests that both directions share a generic licenser component, such as negation, that English any responds to.

Prompted forced choice. The second protocol also asks each model which of two sentences is more natural, with both orders averaged. 8 models pass the negation gate in both languages. In these models the preferences show the full double dissociation. The Mandarin advantage under the ZA classes has a CI above zero in 7, 7 and 7 of the 8 models (POSS, NEC, EVID). The English advantage under only, first and regret has a CI below zero in 8, 7 and 6. The prompted LL values track the log-probability LL values (ρ=0.58\rho= 0.58–0.920.92). These are prompted preferences of the same models, not human judgements, but they show that the English side, weak in sentence probabilities, is present when the models are asked directly. The exploratory 32B model shows the full double dissociation in forced choice as well.

Related work

Polarity items in language models. Most of this work is about English.

  • NPI licensing was among the first targeted syntactic evaluations ([14, 8, 18]).

  • BLiMP includes NPI paradigms ([19]).

  • Jumelet et al. [7] show that LMs use monotonicity information to judge NPI licensing.

  • Bylinina and Tikhonov [2] probe polarity sensitivity in neural LMs.

  • Kumon and Yanaka [9] use activation patching on English constructions. Filler-gap dependencies share a localised mechanism across constructions, but NPI licensing does not.

Chinese benchmarks cover 任何 ‘any’ under negation, conditionals, questions and non-factive verbs [13], as well as 什么 under negation [16]. The multilingual BLiMPPI covers polarity items in 33 languages but excludes conditionals [5]. None of these tests existential 什么/谁 in nonveridical contexts that are not DE. None contrasts DE with nonveridical licensing within one design, or sets Mandarin against English.

Theory. Ladusaw [10] tied NPIs to downward entailment, and von Fintel [17] extended this to Strawson-DE operators such as only. Giannakidou [4] argued that many polarity items depend on nonveridicality instead. For Mandarin wh-indefinites, Lin [12] proposed a non-entailment-of-existence condition, and Xie [21] and Lin and Giannakidou [11] recast it as nonveridical licensing. Chen [3] shows that 什么 can also appear in some veridical episodic sentences, where it carries a speaker-ignorance reading. This matters for our en-advantage cells (§ 7).

Shared multilingual representations. Latent representations in multilingual LMs are partly English-anchored [20]. For negation specifically, Nguyen et al. [15] find a shared cross-lingual representation: a direction extracted in one language adds or removes negation in others. Our intervention asks the next question. Is the licensing that negation, questions and conditionals provide also carried by a shared direction, and does such a direction cover the Mandarin-specific licensors? We use difference-of-means directions with directional ablation and addition, following Arditi et al. [1]. Our forced-choice probe follows the caution of Hu and Levy [6] that prompted judgements and probabilities can come apart.

Discussion

What the models learned. Every gated model, from 0.5B to 9B in four families and the exploratory 32B model, licenses Mandarin 什么/谁 under possibility, necessity and evidential adverbs. None licenses English any there. In these contexts both languages offer the same surface cue, a preverbal adverb, yet only Mandarin licenses the target. A single English-calibrated NPI detector cannot produce this. In Mandarin, the models’ scores follow the property that the theory says matters there, nonveridicality, rather than the DE property that governs any. The ablations point the same way. A direction that carries English licensing, held-out classes included, does not reach Mandarin licensing in most models (§ 5).

Why the English side is weaker. Two separate things happen in the en-advantage cells.

  • Any under only, the first to and regret is licensed, but at a fraction of negation. LLM raters, by contrast, accept these sentences almost as readily as negated ones. Strawson-DE licensing is the part of the English system that these models under-learn, at least at the scales we test: in Qwen2.5, the 32B model is no better at it than the 0.5B–7B models. This fits its rarity and the presupposition it requires [17].

  • Several models give Mandarin 什么 partial credit under 只有 and 第一个. That is not obviously a model error. Our native screener accepted such sentences once a context made the wh-phrase refer to a particular person or thing. Chen [3] documents a speaker-ignorance use of 什么 in veridical sentences. The non-entailment-of-existence prediction for these cells is therefore weaker than the nonveridical prediction for the ZA cells, and the asymmetry in our results mirrors that.

Limitations

  • Relative, not categorical, licensing. LL measures how much a context raises the target over a definite control, relative to negation. It does not separate grammatical from ungrammatical. The native-speaker follow-up shows that several “unlicensed” Mandarin sentences improve in a supporting context, so the EA-side predictions are graded.

  • Human judgements. The only human check is a screening by one native speaker, who is an author. The three LLM raters reproduce the theoretical contrasts, but they are LLM ratings, not human data. A rating study with naive native speakers of both languages remains to be done.

  • Items. The design uses 48 templated frames with one continuation, so the effects are measured in a controlled but narrow construction. Naturalistic sentences, other wh-words and embedded contexts (for example, non-factive verbs) are untested.

  • Models. The pre-registered models go up to 9B. One exploratory 32B model, from the Qwen2.5 family, extends this; other families at that scale and larger models are untested. One model, Qwen3-8B, fails the measurement gate because its Mandarin negation cell collapses. We report it rather than adjusting the gate.

  • Mechanism. The interventions use linear difference-of-means directions read at one position. A null effect for a direction does not show that no shared representation exists, and in Qwen2.5-1.5B to 7B even the Mandarin direction has no effect on Mandarin. The continuation measure was added after the first model’s measurement check; on the pre-specified object-only measure the cross-model reading is inconclusive.

Items

Ablation values

Table 3 gives the values plotted in Figure 2. Table 4 gives the same ablation on the pre-specified object-only measure. Rerunning that measure with the amended code reproduces the first run (Pearson r≥0.979r \ge0.979 on the three models run twice).

ModelMandarin licensing removed by uenu_{\mathrm{en}} NEGMandarin licensing removed by uenu_{\mathrm{en}} ZAMandarin licensing removed random r95 NEGMandarin licensing removed random r95 ZAMandarin licensing removed by uzhu_{\mathrm{zh}} NEGMandarin licensing removed by uzhu_{\mathrm{zh}} ZAEnglish licensing removed by uenu_{\mathrm{en}} NEGEnglish licensing removed by uenu_{\mathrm{en}} EAEnglish licensing removed random r95 NEGEnglish licensing removed random r95 EAEnglish licensing removed by uzhu_{\mathrm{zh}} EAcategory
Qwen2.5-0.5B-Inst−0.150.130.290.171.220.710.580.360.170.440.05neither
Qwen2.5-1.5B-Inst0.10−0.000.320.39−0.430.030.650.480.060.160.27neither
Qwen2.5-3B-Inst0.100.190.250.19−0.31−0.390.490.770.100.350.09neither
Qwen2.5-7B-Inst−0.00−0.300.160.12−0.49−0.680.270.390.050.13−0.00neither
Qwen2.5-7B−0.09−0.560.090.08−0.09−0.180.23−0.070.030.07−0.03neither
Llama-3.2-1B-Inst0.840.500.130.130.940.760.210.300.050.12−0.08separate
Llama-3.2-3B-Inst0.180.090.190.140.450.530.370.410.090.110.10neither
Llama-3.1-8B-Inst0.420.280.120.090.420.230.310.100.050.11−0.10shared
Llama-3.1-8B0.150.070.130.090.580.300.350.300.040.100.08separate
GLM-4-9B-Chat−0.210.280.100.101.500.930.761.110.020.040.61neither
Yi-1.5-9B-Chat2.411.730.380.300.691.590.721.150.060.080.05partial

Table 3. Directional ablation (every layer projects out its own direction). Entries are removal shares R=1−DiDabl/DiDR = 1-\mathrm{DiD}_{\mathrm{abl}}/\mathrm{DiD} on the sentence-continuation measure; r95 is the 95th percentile over 20 random directions. ZA averages POSS/NEC/EVID; EA averages ONLY/FIRST/REGRET. The directions are built only from NEG/QYN/COND versus unlicensed contexts, so ZA and EA are held out.

References

  1. [1]Andy Arditi, Oscar Obeso, Aaquisb Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. 2024. Refusal in language models is mediated by a single direction. In Advances in Neural Information Processing Systems.arxiv.org/abs/2406.11717
  2. [2]Lisa Bylinina and Alexey Tikhonov. 2022. Transformers in the loop: Polarity in neural models of language. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6601–6610, Dublin, Ireland. Association for Computational Linguistics.
  3. [3]Zhuo Chen. 2017. The epistemic indefinite shenme in Mandarin. In Proceedings of the 34th West Coast Conference on Formal Linguistics, pages 115–121. Cascadilla Proceedings Project.
  4. [4]Anastasia Giannakidou. 1998. Polarity Sensitivity as (Non)Veridical Dependency. John Benjamins, Amsterdam.DOI
  5. [5]Arzu Burcu Güven, Anna Rogers, Leonie Weissweiler, Joakim Nivre, and Rob van der Goot. 2026. BLiMPPI: Benchmark of linguistic minimal pairs for polarity items. arXiv preprint arXiv:2610.13607.arxiv.org/abs/2610.13607
  6. [6]Jennifer Hu and Roger Levy. 2023. Prompting is not a substitute for probability measurements in large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5040–5060, Singapore. Association for Computational Linguistics.arxiv.org/abs/2305.13264
  7. [7]Jaap Jumelet, Milica Denic, Jakub Szymanik, Dieuwke Hupkes, and Shane Steinert-Threlkeld. 2021. Language models use monotonicity to assess NPI licensing. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4958–4969. Online. Association for Computational Linguistics.
  8. [8]Jaap Jumelet and Dieuwke Hupkes. 2018. Do language models understand anything? on the ability of LSTMs to understand negative polarity items. In Proceedings of the 2018 EMNLP Workshop Black-boxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 222–231, Brussels, Belgium. Association for Computational Linguistics.
  9. [9]Ryoma Kumon and Hitomi Yanaka. 2026. Fine-grained analysis of shared syntactic mechanisms in language models. arXiv preprint arXiv:2604.22166.arxiv.org/abs/2604.22166
  10. [10]William A. Ladusaw. 1979. Polarity Sensitivity as Inherent Scope Relations. Ph.D. thesis, University of Texas at Austin.
  11. [11]Jing Lin and Anastasia Giannakidou. 2015. No exhaustivity for the Mandarin NPI shenme. Manuscript, University of Amsterdam and University of Chicago.
  12. [12]Jo-Wang Lin. 1998. On existential polarity wh-phrases in Chinese. Journal of East Asian Linguistics, 7(3):219–255.
  13. [13]Yikang Liu, Yeting Shen, Hongao Zhu, Lilong Xu, Zhiheng Qian, Siyuan Song, Kejia Zhang, Jialong Tang, Pei Zhang, Baosong Yang, Rui Wang, and Hai Hu. 2026. A systematic assessment of language models with linguistic minimal pairs in Chinese. Transactions of the Association for Computational Linguistics, 14:755–771.arxiv.org/abs/2411.06096
  14. [14]Rebecca Marvin and Tal Linzen. 2018. Targeted syntactic evaluation of language models. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1192–1202, Brussels, Belgium. Association for Computational Linguistics.
  15. [15]Giang Son Nguyen, Wray Buntine, Dung D. Le, and Wenya Wang. 2026. “not” is not lost in translation: A causal analysis of negation representations in multilingual LLMs. arXiv preprint arXiv:2610.32281.arxiv.org/abs/2610.32281
  16. [16]Yixiao Song, Kalpesh Krishna, Rajesh Bhatt, and Mohit Iyer. 2022. SLING: Sino linguistic evaluation of large language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4606–4634, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.arxiv.org/abs/2210.11689
  17. [17]Kai von Fintel. 1999. NPI licensing, Strawson entailment, and context dependency. Journal of Semantics, 16(2):97–148.DOI
  18. [18]Alex Warstadt, Yu Cao, Ioana Grosu, Wei Peng, Hagen Blix, Yining Nie, Anna Alsop, Shikha Bordia, Haokun Liu, Alicia Parrish, Sheng-Fu Wang, Jason Phang, Anhad Mohanane, Phu Mon Htut, Paloma Jeretic, and Samuel R. Bowman. 2019. Investigating BERT’s knowledge of language: Five analysis methods with NPIs. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2877–2887, Hong Kong, China. Association for Computational Linguistics.arxiv.org/abs/1909.02597
  19. [19]Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohanane, Wei Peng, Sheng-Fu Wang, and Samuel R. Bowman. 2020. BLiMP: The benchmark of linguistic minimal pairs for English. Transactions of the Association for Computational Linguistics, 8:377–392.arxiv.org/abs/1912.00582
  20. [20]Chris Wendler, Veniamin Veselovsky, Giovanni Monea, and Robert West. 2024. Do llamas work in English? on the latent language of multilingual transformers. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15366–15394, Bangkok, Thailand. Association for Computational Linguistics.arxiv.org/abs/2402.10588
  21. [21]Zhiguo Xie. 2007. Nonveridicality and existential polarity wh-phrases in Mandarin. In Proceedings from the 43rd Annual Meeting of the Chicago Linguistic Society, volume 43, pages 121–135.

Paper details

vlnc:W_mvt6zpn8
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents