Assertion and Belief: Moore's Paradox in Language Models

Introduction

I may say that $it$ $is$ $raining$ and that $you$ $do$ $not$ $believe$ $it$. I may say that $it$ $rained$ $yesterday$ and that $I$ $did$ $not$ $then$ $believe$ $it$. I may even suppose, for the sake of argument, that $it$ $is$ $raining$ and that $I$ $do$ $not$ $believe$ $it$. But I cannot say, in my own voice and of the present moment, “It is raining, but I don’t believe that it is.” Both conjuncts may well be true together, for there is plenty of rain I don’t know about; and yet to say them together is absurd [22]. This is Moore’s paradox, and sentences that produce it, such as $p \land\neg B(p)$, are called Moorean [8].

Classic accounts of meaning, such as truth-conditional semantics, fail to predict this absurdity as the sentence is satisfiable, and a satisfiable

Act	State	Felicitous
Assertion	It is raining; Robin is ignorant of the rain. She shares her view over the weather.	✘ No
Reading	Robin is asked to read a line on a worksheet “It is raining”; she does not believe it.	✔ Yes

sentence is, for them, a sound one. What these accounts overlook, Moore’s paradox reveals, is that in saying something a speaker also performs an act with meaning of its own. To assert $p$ is to represent oneself as standing in an epistemic relation to $p$, as having grounds for it or believing it [38, 24]. In a Moorean sentence, the speaker’s own self-ascription then represents her otherwise, and the two clash [22, 37]. Reading a sentence aloud, by contrast, requires only that the speaker say what is printed, not that she believe it, so the same words cause no clash (Table 1). The paradox thus brings out what a competent language user must grasp beyond literal meaning, i.e., what a speech act requires of the speaker (e.g., for assertion, the epistemic state she should be in).

Large language models (LLMs) have shown competence with linguistic form [21], semantics, such as modality and aspect [11, 20], and pragmatics, such as conversational maxims and implicature [12, 26]. A separate line of work probes their epistemic understanding such as whether they distinguish between belief, knowledge, and fact, and whether they track the beliefs of themselves and others [27, 29, 31] (Zhu et al. 2024). Yet how epistemic understanding enters into meaning has been little studied. Moore’s paradox offers this bridge, since the paradox arises at the level of the speech act, which links the meaning of language to the speaker’s epistemic state.

We present the first test of Moore’s paradox in LLMs, with a carefully curated set of Moorean sentences, each placed in contexts. First, across 12 open-weight and proprietary models, we treat each model as a listener and test how the model understands the Moorean sentence in three ways: (1) whether the model recognizes the infelicity out of context (§ 3), (2) whether the judgement relies on memorized form, by testing Moorean sentences of different forms with the same meaning (§ 4.1), and (3) whether the judgement responds to the speech act, by placing the speaker in a context where she reads the sentence rather than asserts it (§ 4.2, Table 1).

Second, we test whether these judgements follow from reasoning about the speaker’s epistemic state. We adopt the Rational Speech Act framework (RSA; Goodman and Frank, [7]), which allows us to model a speaker who chooses what to say given her epistemic state and a listener who infers that state from what she says. We construct states and utterances that are relevant to the paradox (§ 5-§ 6). We first elicit the model’s speaker by asking which utterances it could say in each state ($S_1(u \mid s)$). To find what norms of assertion this speaker follows, we compare the speaker with alternative ones built from different norms, e.g., asserting only what one believes (BNA) or only what one’s evidence supports (JNA) (Pagin and Marsili, [24]). We then elicit the model’s listener separately by asking which states a speaker could be in when she says each utterance ($L_1(s \mid u)$). If the model reasons about the speaker, its listener should consider only the states where its own speaker would say the utterance, and we measure how well the two match. We additionally examine recurring patterns in the models’ reasoning traces (Appendix § 25).

Our results show that models do not judge Moorean sentences by reasoning about its own speaker, even though they largely recognize the pragmatic profile of them. (1) Models recognize the pragmatic profile of Moorean sentences. They rank the Moorean sentence between the coherent sentence and the contradiction. The judgement is insensitive to the forms of Moorean sentences while being sensitive to the speech act change in the context (§ 3 and § 4). (2) Their speakers follow a belief norm of assertion. They assert what they believe even against their evidence. When the speaker’s evidence shows $p$ but she does not believe $p$, the models’ speakers would only assert $\neg p$ (§ 6.1). (3) Yet their listeners do not follow their own speakers. They take the Moorean sentence to also come from such a speaker, and reasoning makes such mismatches more frequent (Section 6.2). (4) Instead, the models rely on a recalled rule. Their reasoning traces judge the sentence by applying or reinterpreting a known rule of the paradox (Appendix § 25).

Background

Moore [22] noticed that a speaker cannot sensibly say the following:

(1) # I went to the pictures last Tuesday, but I don’t believe that I did.

Hearers of such a sentence “hear contradiction, yet are puzzled to find no contradiction in what has been asserted” (Williams, [37]). We study this omissive form, $p \land\neg\mathrm{B}(p)$, and set aside the commissive form $p \land\mathrm{B}(\neg p)$ and other forms.

Since Moore, the absurdity has been located in the act of assertion rather than in the content asserted. The explanation rests on a pragmatic principle: “when a man asserts anything that might be true or false, he implies that he himself, at the time of speaking, believes or knows the thing in question” (Moore [22]; see also Black [1]; Wittgenstein [39]; Davidson [3]; Pagin and Marsili [24]). Hence the first conjunct of a Moorean sentence, by the pragmatic principle, represents the speaker as believing $p$ at the time of speaking, while the second has the same speaker deny that very belief, which leads to the contradiction (Moore, [23]). It follows that the absurdity disappears when the utterance is not an assertion, as when it is recited or read aloud (Williams, [37]): the words then represent nothing about the speaker’s epistemic state.

Beneath the surface: Epistemic structure of the Moorean state

So what may a speaker assert? This concerns two questions: What epistemic standing does assertion require of the speaker? When can the position she expresses be rationally held?

Norms of assertion Epistemic norms of assertion govern when a speaker can appropriately assert a proposition (Williamson, [38]; Pagin and Marsili, [24]). On the account employed in Lackey [17]’s treatment of Moorean assertions, a speaker should assert $p$ only if it is reasonable for her to believe $p$ and her assertion is based at least partly on that reasonableness. Hence, a Moorean assertion requires the speaker to have grounds that support $p$ and $\neg B(p)$, as in (2).

(2) The skywalk is safe, but I do not believe it is.

For a speaker to appropriately assert (2), she might have inspection results and repeated safe crossings that provide the grounds for asserting safety ($p$). In the meantime, the speaker notices that she steps back and still expects the glass to give way. Recognising these persistent doubts can support her self-ascription of non-belief ($\neg B(p)$).

Epistemic rationality Suppose both assertions have adequate grounds: the speaker has evidence supporting safety and accurately recognises her non-belief. A further question remains: given the grounds available to her, is it reasonable for her to remain unconvinced? If she has considered the safety evidence but persistently withholds belief without countervailing grounds, her position is irrational [4, 6, 5]. Both claims may be assertable but the assertion is not a rational one. Fileva and Brakel [5] argue that some Moorean positions can be rationally maintained. In the skywalk case, engineering evidence supports the assertion that the bridge is safe, yet the bridge looks dangerous to her. Such an appearance is a first-person seeming, the way things strike the speaker herself through automatic processes such as perception and learned association. The seeming sustains her non-belief. Recognising the doubt supports her self-ascription without necessarily undermining the grounds for asserting safety.

Related Work

The paradox concerns three issues: meaning beyond truth-conditions (e.g., pragmatics), epistemic understanding, and speech acts. LLMs have been tested on many pragmatic inferences, from implicature and presupposition cast as natural language inference [14] to broader benchmarks and comparisons with human judgements [12, 26, 28, 19]. A separate line of work evaluates whether models track others’ false beliefs [33, 16, 27, 31], represent the beliefs of themselves and others (Zhu et al., 2024), and distinguish belief from knowledge and fact [29]. Yet thus far, pragmatic inference tests rarely probe the speaker’s epistemic state, at least not directly. The domain of speech acts is where the two lines of work can intersect. Work on LLMs and speech acts has remained philosophical, debating whether a model can assert at all [10, 36] rather than whether a model takes the speech act into account when computing meaning. Moorean sentences have been tested only with human participants [34, 35].

The Rational Speech Act framework (RSA) models communication as a speaker and a listener reasoning about each other [7]. Prior work has used RSA to explain LLM pragmatics or to improve model performance [13, 2, 25]. For Moore’s paradox, RSA lets us derive the pragmatic inference from the interaction between a speaker and a listener, and we can design this interaction around the factors the paradox concerns, such as the speaker’s epistemic state.

Do models recognize the infelicity of Moorean assertions?

We first test whether models, as listeners, recognize the absurdity of classic Moorean assertions made in an underspecified context.

Materials. We construct stimuli from 200 propositions, which we select manually from LLM-generated candidates to cover diverse predicates and domains (details in Appendix § 10). For each proposition $p$, we instantiate three templates: a Moorean sentence (“$p$, but I don’t believe that”), a coherent control (“$p$, but that’s okay”), and a logical contradiction (“$p$, but it isn’t”).

Method. We adapt two judgement tasks from human studies: binary sense judgements [35] and seven-point acceptability ratings [34]. We introduce each sentence with an underspecified context: “Imagine that Robin sincerely made the following statement.” Models then have to answer, “Does what Robin says make sense?” (Yes/No) or rate the statement’s acceptability from 1 (Unacceptable) to 7 (Acceptable), where acceptable means that a native speaker could use the sentence. To reduce dependence on a particular wording, we use five versions of each question in separate prompts (Appendix § 27.§ 2) and average judgements for each proposition. We report averaged Yes rates and acceptability ratings across 200 propositions, with 95% confidence intervals from 1,000 bootstrap.

Models. We evaluate eleven open-weight models from three families (Qwen3.5, K2-Horizon and Gemma 4) at different sizes, and Claude Haiku 5.5 (Appendix § 11, Table 3). We run each model with reasoning off and on.

Results. We report sense judgements (Figure 1) and as acceptability ratings show similar results (Appendix § 13, Figure 6), we report sense judgements only hereafter. As the bar charts at the top of Figure 1 show, models overall rank the sentences coherent > Moorean > contradiction, as people do in the same sense-judgement task (Willemsen and Baumgartner, [35]). Reasoning mode lowers acceptance of the Moorean sentence in every family, but by very different amounts; see Appendix § 25, Appendix § 13, and Figure 5 for further discussions.

Underlying the Judgements

All models but Haiku with reasoning rank the sentences as people do (§ 3). We now test how they reach their judgements. A model may have memorized that sentences of the shape “p, but I don’t believe that” are bad as a string. We test this in two ways: changing the form but not the situation, and the situation but not the form.

Changing the form does not alter judgements

We keep the base prompt of § 3 and change only the wording of the Moorean sentence while preserving its meaning. For each Moorean sentence, we write three paraphrases that reword the second half (e.g., “p, but I don’t think it is”) and two insertions that place coherent material between the two halves (e.g., “The side gate is closed, the back gate is closed, but I don’t believe that the side gate is closed”; full set in Appendix § 27.§ 3). If the judgement rested on string memorization, the judgement should change once the string changes. Figure 2 (left) shows that the judgement survives the changes of form. For every family, the changed forms are accepted no more often than the original (per model in Appendix § 14, Figure 7).

Accommodating the infelicity

We then keep the sentence fixed and change the context to one where Robin does not make an assertion but reads it aloud, a context argued to remove the oddness (adapted from Williams [37]); prompts in Appendix § 27.§ 4):

(3) A teacher asks each student to read aloud the sentence on their worksheet and add a comment of their own. Robin’s worksheet says “The side gate is closed.” It is Robin’s turn. Robin says, “The side gate is closed, but I don’t believe that.”

We compare how often models accept the Moorean sentence when Robin asserts it and when she reads it aloud. Two controls separate the speech act from the reading frame. The coherent sentence tests whether the frame raises acceptance of any sentence. The unread Moorean sentence “It’s my turn, but I don’t believe that” tests whether the frame rescues any Moorean sentence or only the one Robin reads (i.e., speech act sensitivity). In Figure 2 (right), a point on the diagonal means the frame makes no difference, and the further a point lies above the diagonal, the more reading aloud raises acceptance (i.e., the more the model accommodates the sentence). Reading aloud raises acceptance of the Moorean sentence by 53 points on average with reasoning and 40 without, against 6 and 15 for the coherent sentence and −4 and 10 for the unread Moorean sentence. For most models, accommodation thus responds to whether Robin reads or asserts, not to the frame (Appendix § 15, Figure 8).

These results show that models do not judge by form alone. They accommodate the Moorean sentence when the context allows it. However, this could come from knowledge of the paradox (Appendix § 16 shows that e.g., Qwen family and Haiku name “Moore” in more than 89% of their reasoning traces) rather than reasoning about Robin’s state. In the next section, we test how far the judgements follow from reasoning about the speaker based on rational speech act theory.

Rational Speech Act Theory

The Rational Speech Act framework (RSA; Goodman and Frank [7]) models how a listener interprets an utterance by reasoning about the speaker with three components. A literal listener ($L_0$) interprets an utterance $u$ by its literal meaning alone, checking whether $u$ is true in each state $s$:

where $\llbracket u \rrbracket(s)$ is 1 if $u$ is true in $s$ and 0 otherwise. A pragmatic speaker ($S_1$) in state $s$ chooses among alternative utterances, preferring those that lead the literal listener to her actual state,

A pragmatic listener ($L_1$) interprets $u$ by asking which states lead such a speaker to say it:

Interpreting Moorean Assertions via RSA We first show how RSA accounts for the pragmatics of Moorean sentences, using theory-driven states.

States. We use four states of the speaker, Robin, $s_1$-$s_4$ (Table 2). In $s_1$, her evidence, belief and the world agree. $s_2$ is Moore’s classic case: Robin has no evidence and lacks belief in $p$. In $s_3$, her evidence shows $p$, but she maintains her belief $\neg p$ due to irrationality or a first-person seeming (Fileva and Brakel, [5]). In $s_4$, she reads the line “$p$” aloud and so does not commit herself to $p$.

Utterances. We consider four relevant utterances about $p$: the base form ($p$), the negation ($\neg p$), a statement of ignorance ($DK$, “I don’t know whether $p$”) and the Moorean form ($M$). The first three assert Robin’s view of $p$; $M$ asserts $p$ and denies believing it.

As Table 2 shows, on hearing $M$, an ideal listener should consider $s_3$ and $s_4$ and rule out $s_1$ and $s_2$. We follow the computation in Figure 3b and derive the target profile ($L_1$ Figure 3b).

Literal listener. For each state and utterance, we check against Table 2 whether the utterance is true, which gives $\llbracket u\rrbracket(s)$ (Truth in Figure 3a). $M$ is true in $s_2$ and $s_3$, where $p$ holds and Robin does not believe $p$, and false in $s_4$, where $p$ is false. The literal listener normalizes each column of Truth over states (Eq. (1); $L_0$ in Figure 3b), so hearing $M$, $L_0$ splits its probability between $s_2$ and $s_3$. Literal meaning thus misses the target in $s_2$ and $s_4$.

Speaker. A vanilla speaker may say only what is true in her state, and prefers the utterances that lead $L_0$ to it (Eq. (2); Appendix § 18). What she may say is thus the Truth table (Figure 3a). Hence, she says $M$ in $s_2$, where $M$ is true, and never in $s_4$, where $M$ is false, i.e., she cannot correct what literal meaning misses (red outlines). To reach the target, the speaker must go beyond truth: (1) the speaker may say something that is false, e.g., in $s_4$, she says $p$ although she knows it is not true (i.e., reading does not require truth); (2) the speaker may not say something even if it is true, e.g., $M$ in $s_2$, Moore’s classic case.

We therefore add to the canonical $S_1$ the norms of speech acts. The speaker of Eq. (2) maximizes

(Eq. (1) with a uniform prior; derivation in Appendix § 17). The first term is whether $u$ is true in her state, and the second how informative $u$ is to $L_0$. As RSA models of non-literal language revise the speaker’s utility (Kao et al., [15]), we revise the first term from $\log\llbracket u \rrbracket(s)$ to $\log A(s,u)$, so that what she may say goes beyond literal meaning and follows the norms of speech acts (see Appendix § 19),

We define $A(s,u)$ as 1 where the norm of her act lets her say $u$ in $s$ and 0 otherwise. The intuition is that the speaker maximizes informativity only among the utterances her norm permits. Where no norm applies, $A(s,u)$ is the literal $\llbracket u \rrbracket(s)$, so Eq. (4) reduces to Eq. (2). $A$ thus depends on which act and which norm one considers, and each norm overrides some cells of the Truth table (bold in Figure 3a). In the case of Moore’s paradox, we consider two instantiations of $A$, one for non-assertive acts and one for assertion, each with its own norm.

acts and one for assertion, each with its own norm. For non-assertive acts, we instantiate the norm with reading, which requires saying what is printed and places no requirement on the speaker’s epistemic state (discussion in Appendix § 20). The norm of assertion instead sets what her epistemic state must be. It takes different forms (Williamson, [38]; Lackey, [17]; Pagin and Marsili, [24]), and we consider two of them, echoing the two submaxims of Grice’s Quality (Grice, [9]). The justification norm of assertion (JNA) says “Assert only that for which you have proper justification” (see also Lackey, [17]), and the belief norm of assertion (BNA) “Assert only what you believe”. Neither instantiation of $A$ changes the literal meaning of an utterance, so $L_0$ stays the same.

With the norm of reading (NA in Figure 3a), the speaker in $s_4$ may say $p$, as the norm of reading requires, so $A_{\mathrm{NA}}(s_4,M)=1$ and $S_1(M \mid s_4)$ is possible now. The reading norm does not affect her choice in $s_2$, where Robin asserts. Her choice there still rests on truth alone, so she says $M$ like the vanilla speaker, although $M$ should be ruled out (red outline). The norm of assertion corrects this. Both JNA and BNA set $A_{\mathrm{JNA}}(s_2,M)=A_{\mathrm{BNA}}(s_2,M)=0$, JNA because Robin lacks evidence for $p$ and BNA because she lacks belief in $p$. The two norms part ways in $s_3$, where $M$ should remain possible. There her evidence shows $p$, but she believes $\neg p$. JNA lets her assert $p$, since her evidence justifies it, so $A_{\mathrm{JNA}}(s_3,M)=1$ (NA + JNA). BNA does not, since she does not believe $p$, so $A_{\mathrm{BNA}}(s_3,M)=0$, and $M$ is ruled out where it should be possible (NA + BNA). Figure 3b computes the speaker with NA and JNA. By Eq. (4), she weighs each utterance she may say by its informativity $1/\sum_{s'}\llbracket u\rrbracket(s')$, which is $L_0(s \mid u)$ in any state where $u$ is true (.33 for $p$, 1 for $\neg p$, .50 for DK and $M$), and normalizes over utterances. In $s_4$, for instance, she may say $p$ or $M$, and so says $p$ (.40) or $M$ (.60).

Pragmatic listener. A pragmatic listener therefore can interpret the Moorean sentence correctly by reasoning about a speaker who follows the norm of reading and JNA. Such a speaker says $M$ only in $s_3$ and $s_4$ (Eq. (4)), so, hearing $M$, the listener considers only these two states, as the target requires ($L_1$ in Figure 3b).

Deriving the Moorean Judgement in Models

Section 5 derived the pragmatic profile of Moorean sentences with RSA. We now prompt each of the 12 LLMs separately to obtain its $\hat{L}_0$, $\hat{S}_1$ and $\hat{L}_1$, and test whether they relate as in RSA. We ask which norms of speech acts the model’s $\hat{S}_1$ follows (§ 6.1) and whether its $\hat{L}_1$ follows from its $\hat{S}_1$ (Section 6.2).

Materials. Utterances. We keep the four utterances of § 5, which cover how Robin’s view can relate to $p$: $p$, $\neg p$, ignorance, and the Moorean sentence. Each is instantiated with a set $Q$ of 50 of the 200 propositions of § 3.

States. Rather than fixing the states in advance, we sample them from the models. For each of the 4 utterances, we asked each of the 12 test models why Robin might have said it (prompt in Appendix § 27.§ 6), repeating this with 3 different propositions, and collected about 10.5k proposals per utterance. We apply LLM judge for clustering each utterance’s proposals, and combine with manual inspection, we arrive at 12 states in Table 4. Details can be found in Appendix § 27.§ 5.

Eliciting $\hat{L}_0$, $\hat{S}_1$ and $\hat{L}_1$. For each of the test models, we obtain their $\hat{L}_0$, $\hat{S}_1$ and $\hat{L}_1$ by prompting them as below (prompts in Appendix § 27.§ 7):

$\hat{L}_0(s|u)$: for each state and utterance, we ask the model whether $u$ is true in $s$ ($[\![u]\!](s)$).

$\hat{S}_1(u|s)$: for each state, the model rates how likely Robin is to say each of the four alternative utterances ($u'$) on the scale of 0-10. Writing $r_q(u,s)$ for the model’s rating of utterance $u$ in state $s$ for proposition $q$, we normalize over utterances and average over $Q$,

$\hat{L}_1(s|u)$: for each utterance, the model rates how likely Robin was in each state when she said it on the scale of 0-10. We normalize these ratings $r_q(s,u)$ in the same way, over the 12 states,

Appendix § 23 reports the complete state-utterance tables for each model’s elicited $\hat{L}_0$, $\hat{S}_1$, and $\hat{L}_1$.

What kind of speaker do the models adopt?

We compare $\hat{S}_1$ with four candidate speakers. They differ only in which utterances Robin may say in each state ($S_1^A$ is Eq. (4) with $A \in\{A_{\mathrm{Van}}, A_{\mathrm{NA}}, A_{\mathrm{JNA}}, A_{\mathrm{BNA}}\}$). (1) Vanilla ($S_1^{\mathrm{Van}}$): Robin may say anything true. (2) Non-assertive ($S_1^{\mathrm{NA}}$): where Robin does not assert, as when she reads aloud, teases, or misspeaks, the norm of her act rather than truth decides what she may say, so $M$ becomes possible there. (3) JNA ($S_1^{\mathrm{JNA}}$): as (2), and where Robin asserts, she may assert only what her evidence justifies. (4) BNA ($S_1^{\mathrm{BNA}}$): as (2), and where Robin asserts, she may assert only what she believes.

Method. For each candidate speaker, we manually check, for each of the 12 states ($S$) and the four utterances ($U$), whether Robin may say the utterance under the candidate’s norms, which gives its $A$. We then obtain $S_1^A$ by applying Eq. (4) with $A$ to the model’s own $\hat{L}_0$ (Appendix § 23). Like the elicited $\hat{S}_1$, each candidate $S_1^A$ then decides which utterances Robin may say in each state. We compare $\hat{S}_1$, after calibration (Appendix § 22), with $S_1^A$ in each state $s$ on whether they consider $u$ feasible, writing $S_1(u \mid s)=1$ if feasible and 0 if not. We report the agreement as

Results. The models’ speakers go beyond literal truth. No elicited speaker is closest to the vanilla speaker, except K2 7B without reasoning. For example, in $s_2$, Moore’s classic case, Robin has no evidence and lacks belief in $p$. $p$ and $M$ are literally true, so the vanilla speaker would say them, yet no speaker, with or without reasoning, treats either as feasible.

Models likely adopt a BNA speaker. Qwen, Gemma and Haiku agree most with BNA, with or without reasoning, and reasoning widens BNA’s lead over JNA (Figure 4, left). Take $s_3$, a case like the skywalk in (2). Robin has the engineering evidence that the skywalk is safe, but she does not believe it. Only BNA lets her assert that the skywalk is not safe, on belief alone and against her evidence. The models’ speakers do the same. With or without reasoning, all speakers treat this assertion as feasible, except K2 3.7B without reasoning (undetermined). K2 agrees with JNA as much as with BNA with reasoning. Three of its four speakers then treat $M$ as feasible in $s_3$, which only JNA allows, while every Qwen, Gemma and Haiku speaker but Qwen 4B rules it out.

Does the listener follow its own speaker?

In RSA, the listener interprets $M$ through the speaker, $L_1(s\mid M)\propto P(s)S_1(M\mid s)$, so a listener that follows its own speaker considers exactly the states where its speaker would say $M$. We compare $\hat{S}_1(M\mid s)$ with $\hat{L}_1(s\mid M)$ in each of the 12 states. The listener can depart from its speaker in two ways: (1) It can accept too much, considering a state where its speaker would never say $M$ (a false positive); (2) It can also reject too much, ruling out $M$ in a state where its speaker would say $M$ (a false negative; details in Appendix § 24). Over the 12 states, the two rates are

with $\hat{L}_1(s\mid M)=1$ if $s$ is feasible given $M$ and 0 if not, from the same calibration. We report both rates in Figure 4 (right), with each state’s $\hat{S}_1$ and $\hat{L}_1$ averaged over $Q$ (per-model results in Appendix § 23, Figure 26 and Figure 27).

Reasoning off
Model	$p$	$\neg p$	DK	M
Qwen 4B	.00	.00	.00	.00
Qwen 9B	.00	.00	.00	.25
Qwen 27B	.00	.25	.00	.25
Qwen 35B-A3B	.00	.00	.00	.33
K2 3.7B	.00	.00	.00	.00
K2 7B	.00	.00	.00	.00
K2 32B	.00	.00	.00	.09
K2 36B-A4B	.00	.00	.00	.00
Gemma 12B	.00	.50	.33	.00
Gemma 26B-A4B	.00	.00	.00	.29
Gemma 31B	.00	.25	.33	.73
Haiku 5.5	.00	.00	.00	.20
Reasoning on
Model	$p$	$\neg p$	DK	M
Qwen 4B	.00	.20	.25	.67
Qwen 9B	.00	.20	.00	.44
Qwen 27B	.00	.20	.00	.67
Qwen 35B-A3B	.00	.20	.17	.55
K2 3.7B	.00	.17	.00	.50
K2 7B	.00	.20	.00	.33
K2 32B	.00	.20	.00	.25
K2 36B-A4B	.00	.20	.25	.00
Gemma 12B	.14	.25	.25	.60
Gemma 26B-A4B	.00	.20	.00	.38
Gemma 31B	.00	.25	.50	.64
Haiku 5.5	.00	.17	.00	.44

Reasoning off
Model	$p$	$\neg p$	DK	M
Qwen 4B	.00	.40	.00	.00
Qwen 9B	.00	.40	.00	.00
Qwen 27B	.20	.50	.12	1.00
Qwen 35B-A3B	.00	.33	.00	.00
K2 3.7B	.67	.67	.00	1.00
K2 7B	.00	.00	.00	.00
K2 32B	.00	.40	.00	.00
K2 36B-A4B	.00	.20	.00	.00
Gemma 12B	.00	.14	.00	.17
Gemma 26B-A4B	.00	.17	.00	.00
Gemma 31B	.00	.29	.00	.00
Haiku 5.5	.25	.60	.17	.00
Reasoning on
Model	$p$	$\neg p$	DK	M
Qwen 4B	.25	.60	.00	1.00
Qwen 9B	.00	.50	.14	.00
Qwen 27B	.00	.33	.25	.00
Qwen 35B-A3B	.00	.40	.00	.00
K2 3.7B	.33	.60	.40	.25
K2 7B	.00	.20	.00	.00
K2 32B	.00	.33	.00	.33
K2 36B-A4B	.00	.33	.00	.00
Gemma 12B	.00	.14	.00	.00
Gemma 26B-A4B	.00	.17	.00	.50
Gemma 31B	.00	.17	.14	.00
Haiku 5.5	.00	.40	.00	.00

Results. The models’ listeners do not follow their own speakers, as the non-zero false positives and false negatives in Figure 4 (right) show. Only Qwen 4B, K2 7B and K2 36B-A4B without reasoning, and K2 36B-A4B with reasoning, follow their speakers in every state. The failures are mostly false positives, and reasoning boosts them. Without reasoning, the two rates are comparable (false positives .02–.34, false negatives .00–.25 per family). With reasoning, false positives rise in every family (by .20–.37) to .27–.58, while false negatives change by at most .11.

False positives fall mostly in $s_3$, $s_7$ and $s_{10}$, where Robin believes $\neg p$ although $p$ is true. In $s_3$ and $s_7$, her evidence shows $p$. The Qwen 9B–35B, Gemma and Haiku speakers rule out $M$ there and say $\neg p$ instead, as a BNA speaker would, yet their listeners take $M$ to come from $s_3$ and $s_7$, as if the speaker followed JNA. The few false negatives come from contexts such as a misspeaking Robin ($s_{12}$). Exhausted, she meant to say another sentence, and the K2 32B and Gemma 26B-A4B speakers would say $M$ there as a slip, but the models reject $M$. These results reveal a mismatch between the models’ speakers and listeners.

A complementary analysis of reasoning traces suggests that models often invoke rules about assertion or reinterpret the utterance to reach their judgements (see Appendix § 25).

Conclusions

Using Moore’s paradox as a test of whether language models take the speech act into account when computing meaning, we find that models largely recognize the infelicity of Moorean sentences, but their judgements do not arise from a listener reasoning about the speaker. Their speakers follow a norm of assertion, yet their listeners do not use it, and their traces point to a recalled rule instead. This reveals a gap between judging an utterance correctly and deriving the judgement from what the act requires of the speaker. The gap matters as LLMs increasingly interpret what people say, where meaning depends on who says it and in what act. We hope that deriving a model’s judgements from its own speaker and listener provides a testbed for future work, including other speech acts such as promises and questions that depend on the speaker’s state, and training or prompting that leads models to reason from the speaker.

Limitations

Several choices define the scope of our findings. We use templated sentences of the omissive form in English, which gives us full control over the stimuli and allows matched comparisons across the Moorean, coherent and contradictory sentences; commissive and knowledge forms, naturally occurring Moorean sentences and other languages remain for future work. We elicit each model’s speaker and listener through ratings, normalized and calibrated per model. Token probabilities could complement the ratings for models without reasoning, but with reasoning the answer follows a sampled trace, so its probability reflects that one trace rather than the model’s judgement, and such probabilities are not available for every model. We test 12 models across four families and sizes, with and without reasoning. Larger proprietary models may differ, although the gap we find holds across families and sizes. Finally, reasoning traces need not be faithful to the computation behind a judgement ([32]), so we treat them as behavior that complements the RSA analysis rather than as evidence of mechanism on their own.

Acknowledgments

We would like to thank Kanishka Misra for the insightful discussion. This work was partially supported by NSF grants 2107524 and 2145479, as well as Good Systems,¹ a UT Austin Grand Challenge to develop responsible AI technologies.

Proposition Sampling and Selection

We sample candidate scenarios using Claude Fable 5.1 (prompt in Appendix § 27.§ 1). Each scenario has Robin check three objects of one kind for one property, such as three gates of an office complex and whether they are closed. Each object gives a proposition, e.g., “The side gate is closed”. We manually select 100 scenarios to cover diverse predicates and domains, with 98 domains and 61 distinct predicates in the final set. All propositions are stative and in the simple present, describing a state of an object at the time of speaking, so that Robin can have evidence and a belief about that state. Each scenario is instantiated with two propositions, leading to a set of 200 propositions. The other two objects of the scenario supply the coherent material for the insertions of the form test (§ 4).

Models

Table 3 lists the twelve test models with their sources and reasoning settings.

Model	Parameters	Source	Reasoning on
Qwen3.5-4B	4B	Qwen/Qwen3.5-4B	native thinking
Qwen3.5-9B	9B	Qwen/Qwen3.5-9B	native thinking
Qwen3.5-27B	27B	Qwen/Qwen3.5-27B	native thinking
Qwen3.5-35B-A3B	35B (3B active)	Qwen/Qwen3.5-35B-A3B	native thinking
K2-Horizon-3.7B	3.7B	IFM/K2-Horizon-3.7B	effort high
K2-Horizon-7B	7B	IFM/K2-Horizon-7B	effort high
K2-Horizon-32B	32B	IFM/K2-Horizon-32B	effort high
K2-Horizon-MoVA-36B-A4B	36B (4B active)	IFM/K2-Horizon-MoVA-36B-A4B	effort high
Gemma 4 12B[30]	12B	google/gemma-4-12B-it	native thinking
Gemma 4 26B-A4B	26B (4B active)	google/gemma-4-26B-A4B-it	native thinking
Gemma 4 31B	31B	google/gemma-4-31B-it	native thinking
Claude Haiku 5.5	undisclosed	Anthropic API, claude-haiku-5-5	adaptive, effort high, summarized

Compute

All the experiments are done with 8 NVIDIA H200s. The experiments take around 2-3 GPU hours on average.

Baseline Judgements by Model

Figure 5 gives each model’s sense judgements of § 3, and Figure 6 each model’s acceptability ratings.

Form Test by Model

Figure 7 gives each model’s results on the form test of § 4.1. Per model, the main exception is “p, but I don’t think it is”, which Gemma 31B and Haiku without reasoning accept more often than the original, possibly because the sentence reads as “I think it isn’t”. K2 32B with reasoning accepts every paraphrase more often than the original.

Context Test by Model

Figure 8 gives each model’s results on the context test of § 4.2.

Moore Mentions

Figure 9 gives the share of reasoning traces that name Moore, in the form test (§ 4.1) and the context test (§ 4.2).

The Prior and the Reduction of $S_1$

$L_0$ with a prior. (1) written out, with its normalizer over states, is

Step 1. $P(s)$ cancels in $S_1$ under any prior. Taking the log of $L_0$ gives

$S_1$ normalizes $\exp(\log L_0(s \mid u)) = L_0(s \mid u)$ over utterances in a fixed state $s$,

$P(s)$ is the same for every $u'$, so it comes out of the sum over $u'$ and cancels. The denominator is then the same for every $u$, and Eq. (2) becomes

Step 2. The uniform prior turns the second term into a count. With $P(s)=1/|S|$ for every state in the set $S$ of states, $\sum_{s'} P(s')[[u]](s') = \frac{1}{|S|}\sum_{s'}[[u]](s')$. Substituting into Eq. (6) and normalizing over utterances,

$|S|$ is the same for every $u'$ and cancels in the same way, which gives the form of Section 5,

$\sum_{s'} [\![u]\!](s')$ is the number of states where $u$ is true, so $1/\sum_{s'} [\![u]\!](s')$ is the informativity of $u$.

(4) before and after the reduction. The norms replace only the first term, $\log[\![u]\!](s)$, with $\log A(s,u)$. The informativity term still comes from $L_0$, so the prior enters as in Eq. (6). Under any prior,

and under the uniform prior,

Where $A(s,u)=0$, $\log A(s,u)=-\infty$, so $S_1(u\mid s)=0$ and the norm rules $u$ out. Where $A(s,u)=[\![u]\!](s)$, Eq. (4) is Eq. (2) under the same prior.

Example. In $s_4$ with NA and JNA, $A(s_4,u)$ is 1 for $p$ and $M$ and 0 otherwise. $p$ is true in 3 states and $M$ in 2, so $S_1(p\mid s_4)=\frac{1/3}{1/3+1/2}=.40$ and $S_1(M\mid s_4)=\frac{1/2}{1/3+1/2}=.60$, as in Figure 3b.

RSA with a Vanilla Speaker

Figure 10 gives the derivation of Section 5 with a vanilla speaker. A vanilla speaker $S_1$ considers, in each state (row), the utterances that are literally true there, and normalizes over them (Eq. (2)). In $s_2$, $p$, $DK$ and $M$ are true, so she may say any of them. The pragmatic listener $L_1$ considers, for each utterance (column), the states where $S_1$ would say it (Eq. (3)). This speaker misses the target of Table 2 in two ways (Figure 10). First, $M$ is literally true in $s_2$, where $p$ is true and Robin has no view, so she says $M$ there, although $M$ is infelicitous. Second, $M$ is literally false in $s_4$, where $p$ is false, so the reader never says $M$, although $M$ is felicitous there. The pragmatic listener inherits both errors. Hearing $M$, $L_1$ considers $s_2$, where $M$ should be ruled out, and assigns nothing to $s_4$, where $M$ should be possible.

World	Evidence	Belief	Act	$M$
$s_1$	$p$	$p$	$p$	asserts
$s_2$	$p$	none	none	asserts
$s_3$	$p$	$p$	$\neg p$	asserts ✓
$s_4$	$\neg p$	$\neg p$	$\neg p$	reads “$p$” ✓

The Speaker Table $A(s,u)$

A norm of a speech act overrides cells of $A$ in two ways. It can forbid a true utterance, as JNA and BNA forbid $M$ in $s_2$, and it can permit a false one, as reading permits $p$ and $M$ in $s_4$ and BNA permits $\neg p$ in $s_3$. Since the speaker revises only the first

term of her utility, a permitted false utterance keeps its informativity to the literal listener. $\hat{S}_1(u \mid s) > 0$ exactly when $A(s,u) = 1$, which is the only property the tests of § 6 use.

For the 12 states of § 6, $A_{\mathrm{VAN}}(s,u) = \llbracket u \rrbracket(s)$. $A_{\mathrm{NA}}$ equals $A_{\mathrm{VAN}}$ except in the non-assertive states $s_4$, $s_{11}$ and $s_{12}$, where the norm of the act, not truth, decides what Robin may say. $A_{\mathrm{JNA}}$ and $A_{\mathrm{BNA}}$ equal $A_{\mathrm{NA}}$ except in the states where Robin asserts, where she may assert $u$ only if her evidence justifies it (JNA) or only if she believes it (BNA).

Non-Assertive Acts

Non-assertive acts do not present the content as the speaker’s own claim, and they differ in their norms. Quoting, acting a part or reciting resemble reading, while acts such as guessing or promising commit the speaker in other ways, and each act would bring its own norm and so its own instantiation of $A$. In § 5, we instantiate the norm of non-assertive acts with reading, which requires saying what is printed and places no requirement on the speaker’s epistemic state. The states of § 6 also include teasing ($s_{11}$) and misspeaking ($s_{12}$), which we treat in the same way, as acts without the epistemic requirement of assertion (the Non-assertive candidate). We define this group loosely for controllability; a finer-grained account would give each act its own norm.

States

Table 4 describes the twelve states of § 6 by the world, Robin’s evidence, belief and act.

Calibrating the Elicited Speaker/Listener

We elicit $\hat{S}_1$ by asking the model to rate each utterance from 0 to 10 and normalizing the ratings over utterances. A model may rate an utterance Robin will never say 1 rather than 0, which leaves a small probability on that utterance. To determine which utterances $\hat{S}_1$ considers truly feasible, we calibrate $\hat{S}_1$ on two states where the ratings should be clear. In $s_1$, Robin’s evidence shows $p$ and she believes $p$, so among the four alternatives she should say only $p$. In $s_5$, her evidence shows $\neg p$ and she believes $\neg p$, so among the alternatives she should say only $\neg p$. Each model’s calibration baseline is the largest probability it gives to the wrong alternatives in $s_1$ or $s_5$ (the upper end of its 95% bootstrap interval over propositions; .04–.15). We take $\hat{S}_1$ to consider an utterance feasible in a state if the lower end of the rating’s interval exceeds the baseline, not feasible if the upper end is below it, and undetermined otherwise.

Model Grids

Each model’s $\hat{L}_0$, $\hat{S}_1$ and $\hat{L}_1$ on the 12 elicited states (reasoning on), with each $\hat{S}_1$ state-utterance pair classified against the model’s hedging baseline (§ 6.1).

Testing the Listener against Its Speaker

The listener weighs each state by the prior and by its speaker, $L_1(s \mid M) \propto P(s)S_1(M \mid s)$. A false positive therefore cannot come from the prior. Where the speaker rules out $M$, $S_1(M \mid s) = 0$, so $L_1(s \mid M) = 0$ under any prior, and a listener that still considers $s$ departs from its speaker. A false negative is different. Out of context, a low prior $P(s)$ alone can make the model reject $M$ even where its speaker would say $M$, so a rejection out of context does not show a departure. We therefore test false negatives in context. For each state where the model’s speaker would say $M$, we give the model the state without its belief sentence and ask whether what Robin says makes sense (prompt in Appendix § 27.§ 7). For false negatives, we read $\hat{L}_1(s \mid M)$ from this accommodation in context, with $\hat{L}_1(s \mid M) = 0$ if the model, given $s$, accepts $M$ less than half the time. Undetermined states are left out of both rates. Figure 27 gives the results for every utterance and context.

State	The world is $p$. Robin checked, and it looked $p$ (or a sign she trusts says $p$); she believes $p$.	World	Evidence	Belief	Act	$M$	Share (%)
$s_1$	The world is $p$. Robin checked, and it looked $p$ (or a sign she trusts says $p$); she believes $p$.	$p$	$p$	$p$	asserts		20.0
$s_2$	The world is $p$. Robin has no evidence about $p$ and no view.	$p$	none	none	asserts		16.0
$s_3$	The world is $p$. Robin checked, and it looked $p$, but a feeling or emotion makes her believe $\neg p$.	$p$	$p$	$\neg p$	asserts	✓	2.0
$s_4$	The world is $\neg p$. Robin’s notes say $p$, and she is asked to read them aloud; she looked, and it did not look $p$; she believes $\neg p$.	$\neg p$	$\neg p$	$\neg p$	reads “$p$”	✓	4.7
$s_5$	The world is $\neg p$. Robin checked, and it did not look $p$; she believes $\neg p$.	$\neg p$	$\neg p$	$\neg p$	asserts		21.4
$s_6$	The world is $p$. From one side it looked $p$, from the other it did not; she has no view.	$p$	both	none	asserts		7.1
$s_7$	The world is $p$. Robin checked, and it looked $p$; she nonetheless believes $\neg p$.	$p$	$p$	$\neg p$	asserts	✓	0.5
$s_8$	The world is $p$. A sign says $p$, but Robin does not trust it; she has not checked; she has no view.	$p$	$p$ (sign)	none	asserts		3.7
$s_9$	The world is $p$. A sign says $p$; her glance says $\neg p$; she finds $p$ more likely, yet believes $\neg p$.	$p$	both	$\neg p$	asserts	✓	6.6
$s_{10}$	As $s_9$, but she finds $\neg p$ more likely.	$p$	both	$\neg p$	asserts
$s_{11}$	As $s_1$, and Robin teases a worried friend by pretending to doubt $p$.	$p$	$p$	$p$	teases	✓	6.5
$s_{12}$	As $s_1$, but Robin, exhausted, meant to say another sentence.	$p$	$p$	$p$	misfires	✓	0.4
	Others (e.g., ambiguity in literal meaning, purpose only)						11.1

Into the Reasoning Traces

§ 6 shows that the models’ judgements do not follow from reasoning over the speaker’s state of the kind RSA describes. To further explore what the models rely on, we turn to their reasoning traces. The traces may also explain why the reasoning mode lowers the acceptance of Moorean sentences far more for Qwen and Haiku than for K2 and Gemma (§ 3). The traces give access to the models’ explicit judgements ((Zaman and Srivastava, 2026)), but prior work shows that they are not always faithful to model computation ([32, 18]). We therefore treat the traces as further behavior to analyze, rather than as a probe of the underlying mechanism ((Zheng et al., 2026)).

Materials. We collect 1741 reasoning traces for the Moorean sentence from three settings, each over the same 50 propositions: out of context (from § 3), the skywalk case ($s_3$), and the reader ($s_4$; both from Section 6.2).

Finding the reasoning chains. We use an LLM annotator (DeepSeek V4.1 Flash; prompts in Appendix § 27.§ 8) in three steps. First, it summarizes each trace as an ordered list of two to six reasoning steps. Second, it groups these steps into types, which yields an inventory of 13 moves (Appendix § 26, Table 5). The inventory thus comes from the traces rather than from a taxonomy fixed in advance. Third, it labels each trace with the inventory to obtain the reasoning chains.

Exact chains rarely repeat, since they differ in

length and in incidental moves. Several moves, however, play the same role in judging the sentence. Testing whether the sentence is a formal contradiction and separating the truth of $p$ from Robin’s belief in it, for instance, both check whether the two halves of the sentence can hold together. We therefore group the moves manually by the role they play, yielding three major moves rule (naming Moore’s paradox or stating that sincere assertion implies belief), reinterpretation (another reading of the sentence, such as an inner conflict or reading aloud) and logic checks (testing whether the two halves can hold together). The chains then fall into six shapes (Figure 11; grouping and examples in Appendix § 26).

Results. Figure 11 shows how the reasoning traces reach their judgements across the three settings. In the infelicitous setting (no context), the models reject the Moorean sentence mainly by invoking the rule that marks it as odd (the dark blue rule → No bars, 70% of chains). In the felicitous settings (the skywalk case and the reader), they accept it mainly by reinterpreting it, with or without invoking the rule first (the orange and brown bars), e.g., taking the first half as “a report [of the notes], not an assertion of fact” for the reader. The models thus rely on two things, a known rule about assertion or the paradox that marks the sentence as odd, and a reinterpretation that makes it acceptable. No move in any chain goes from Robin’s evidence and belief to what she may assert, which is consistent with Section 6.2.

Some models hold to the rule more stubbornly than others. In the skywalk case, Claude Haiku invokes the rule in every chain and lets it decide in about half (.52), rejecting the sentence although the context makes it acceptable. Without context, Qwen and Haiku likewise let the rule decide almost every chain (dark blue, .91 and 1.00), whereas K2 and Gemma often set it aside and reinterpret the sentence or check its logic (orange and beige). This is why, with reasoning, Qwen and Haiku reject the Moorean sentence while K2 and Gemma still accept it about half the time (§ 3).

Reasoning Traces

Figure 28 gives each model’s reasoning traces by the shape of their chain, Table 5 the 13 moves, and Table 6 the share of each chain shape in each setting.

Chain	No context	$s_3$	$s_4$
rule → No	.70	.11	.06
rule → reinterpretation → Yes	.11	.32	.31
reinterpretation → Yes	.06	.41	.52
rule → logic checks → Yes	.12	.10	.05
logic checks → Yes	.01	.06	.04

Move	Traces
States the verdict	.92
Parses the sentence into a claim and a denial of belief	.73
Restates the scenario (Robin’s evidence, notes, feelings)	.59
Checks for a logical contradiction	.57
States that sincere assertion implies belief	.48
Names Moore’s paradox	.47
Reads the sentence as expressing an inner conflict	.34
Reads the first half as reading or quoting	.25
Considers other readings (irony, doubt, reference of that)	.22
Separates truth or evidence from belief	.21
Separates asserting from believing	.20
Asks what makes sense means	.16
Distinguishes belief from alief or gut feeling	.03

Example chains. One trace per chain shape, with the annotator’s steps.

rule → No (Haiku 5.5, no context, No):

1. Identifies the statement as an instance of Moore’s paradox.

2. Notes that such assertions are absurd despite not being logically contradictory.

3. Concludes that sincere utterance of the statement does not make sense.

rule → reinterpretation → Yes (K2 36B-A4B, no context, Yes):

1. Parsed the statement into a factual assertion and a belief report, noting the ’but’ suggests contrast.

2. Recognized that asserting a proposition typically implies belief, creating a tension with the second clause.

3. Considered that sincere utterances can honestly report or describe a fact while expressing personal disbelief, like ’It’s raining, but I don’t believe it.’

4. Concluded that the statement is coherent because it can express skepticism or surprise without logical contradiction.

reinterpretation → Yes (Gemma 12B, $s_{4}$, Yes):

1. Identify the conflicting information: the notes say the service entrance is unlocked, but the actual state and Robin’s observation show it is locked.

2. Interpret Robin’s statement as having two parts: a report of what the notes say (’The service entrance is unlocked’) and an expression of her own belief (’but I don’t believe that’).

3. Recognize that the context of reading notes aloud justifies the first part as a report, not an assertion of fact.

4. Conclude that the sentence coherently conveys both the source information and Robin’s personal disbelief, making it sensible.

rule → logic checks → Yes (K2 3.7B, no context, Yes):

1. The model identifies that the statement has two parts: a factual claim about the crate and a belief claim about that fact.

2. It questions whether there is a contradiction between the fact and the disbelief, noting that belief can be independent of objective truth.

3. It considers the role of sincerity and whether Robin knowing the fact would create inconsistency, but decides the context is unspecified.

4. It concludes that the statement is logically coherent because one can state a fact while not believing it.

logic checks → Yes (K2 32B, $s_{3}$, Yes):

1. Interpreted Robin’s statement as asserting the barrier is raised while denying belief in that assertion.

2. Considered whether asserting a proposition without believing it is coherent, noting that assertion does not always imply belief.

3. Examined the context of conflicting evidence: recent observation suggests raised, but past pattern suggests not raised.

4. Concluded that the statement is not self-contradictory and can make sense as an expression of uncertainty.

Prompts

Placeholders are in braces: {P} is the proposition as a sentence (e.g., “The side gate is closed”), {p} the same in lower case, and {A}, {pred} another object of the item and the predicate. The {instruction} is “Answer with exactly one word: Yes or No.” without reasoning and “Think it through, then end your reply with "Answer: " followed by Yes or No.” with reasoning; in the baseline, the one-word instruction is used in both modes. With reasoning, the rating prompts end with “Think it through, then end your reply with a JSON object, . . . ”. The order of utterances and of states is shuffled.

Propositions

Listing 1: Proposition scenarios

Generate scenarios for a study of what speakers can sincerely assert.

In each scenario, Robin and Casey are checking three objects of the same kind (e.g., three gates of an office complex) for one property that can be checked by looking (e.g., closed). Casey asks Robin which objects have the property, and Robin answers sincerely.

Each scenario must satisfy:

1. The property is a state of the object at the time of speaking, not an event, so that "The <object> is <property>." is a stative, present-tense sentence (e.g., "The side gate is closed.").

2. The three objects are distinct members of the same kind, each named by a short definite noun phrase (e.g., "the front gate", "the side gate", "the rear gate").

3. Whether an object has the property can be settled by a quick check, but could also be believed or doubted without checking.

4. Across scenarios, vary the domain (e.g., buildings, kitchens, vehicles, labs, offices) and the property; do not repeat a domain, and use each property at most a few times.

Return one scenario per line as CSV with the columns: domain,context,q_subject,p_subject,r_subject,predicate

Baseline

Listing 2: Baseline prompt

Imagine that Robin made the following statement.

Robin: "{sentence}"

{question}

{instruction}

Listing 3: Baseline sentences and questions

Sentences

    Moorean: {P}, but I don't believe that.

    Coherent: {P}, but that's okay.

    Contradiction: {P}, but it isn't.

Sense questions (instruction: Answer with exactly one word: Yes or No.)

1. Does what Robin says make sense?

2. Does Robin's statement make sense?

3. Is what Robin says a statement that makes sense?

4. Does the statement Robin made make sense?

5. Would you say that what Robin says makes sense?

Rating questions (instruction: Answer with a single number from 1 to 7.)

1. On a scale from 1 (Unacceptable) to 7 (Acceptable), how acceptable is what Robin says? "Acceptable" does not mean that what Robin says is true; it means that a native English speaker could use that sentence.

2. How acceptable is Robin's statement, on a scale from 1 (Unacceptable) to 7 (Acceptable)? Acceptable here does not mean true; it means that a native English speaker could use the sentence.

3. Rate what Robin says from 1 (Unacceptable) to 7 (Acceptable). Acceptable does not mean that the statement is true, only that a native English speaker could use the sentence.

4. Using a scale from 1 (Unacceptable) to 7 (Acceptable), how acceptable is the statement Robin made? "Acceptable" means that a native English speaker could use the sentence, not that it is true.

5. On a 1-to-7 scale, where 1 is Unacceptable and 7 is Acceptable, how acceptable is what Robin says? Acceptable does not mean true; it means that a native English speaker could use that sentence.

Form test

The baseline prompt with the sense question “Does what Robin says make sense?”, with one of the following sentences. {a}, {b}: two other objects of the item.

Listing 4: Form-test sentences

Original

    {P}, but I don't believe that.

Paraphrased

    {P}, but I don't think it is.

    {P}, but I don't believe that {p}.

    {P}, but I don't believe that is the case.

Inserted

    {P}, {A} is {pred}, but I don't believe that {p}.

    {P}, and we checked {a} and {b} an hour ago, but I don't believe that {p}.

Context test

Listing 5: Read aloud

Imagine you are Casey, a teacher. You ask each student to read aloud the sentence printed on their worksheet and then add a comment of their own. Robin's worksheet says: “{P}.” It is Robin's turn. Robin says: "{sentence}"

Does what Robin says make sense?

{instruction}

Listing 6: Assertion

Imagine you are Casey, a teacher. You ask each student to make a comment of their own. It is Robin's turn. Robin says: "{sentence}"

Does what Robin says make sense?

{instruction}

Proposal Clustering

We then use Claude Opus 5.5, which is not among the test models, to cluster each utterance’s proposals. Then we manually map each group to a state with respect to the world, Robin’s evidence, belief and act (Appendix § 21, Table 4). We set aside proposals that focus on lexical meaning that is less relevant to either speech act or speaker’s epistemic state (e.g., “that” is anaphoric to something other than the proposition $q$, or the verb is ambiguous between two meanings, with a different meaning in each conjunct).

State proposals

Listing 7: State proposals

Imagine that Robin made the following statement. Robin: "{utterance}"

Why has Robin said this? List the situations Robin could be in that would lead her to say it.

Write the situations as a numbered list.

Model grids and accommodation

Each {state} is a scenario of Table 4.

Listing 8: Example state

s3 (seeming), p = "the side gate is closed":

The side gate is in fact closed. Robin checked the side gate a minute ago, and it looked closed, so she has good reason to think it is closed. But the side gate has always been otherwise at this time of day, and she cannot shake the feeling that it is not closed; she keeps wanting to go back and check. She nonetheless believes it is not closed.

Listing 9: Literal meaning ($\hat{L}_0$)

{state} Robin says: "{utterance}"

Is what Robin says true?

{instruction}

Listing 10: Speaker ($\hat{S}_1$)

{state} Robin is about to say something about whether {p}. How likely is she to say each of the following?

A. "{utterance 1}" B. "{utterance 2}" C. "{utterance 3}" D. "{utterance 4}"

Rate each from 0 (she would not say it) to 10 (she is very likely to say it). Answer with a JSON object, for example {"A": 3, "B": 0, "C": 7, "D": 1}.

Listing 11: Listener ($\hat{L}_1$)

Imagine that Robin made the following statement. Robin: "{utterance}"

Below are situations Robin could have been in. For each one, rate how likely it is that this was her situation when she made the statement, from 0 (not at all likely) to 10 (very likely).

A. {state 1} B. {state 2} ... R. None of these situations would lead Robin to say this.

Answer with a JSON object that gives a rating for every letter, for example {"A": 3, "B": 0, ...}.

Listing 12: Accommodation

{state without its belief sentence} Imagine that Robin made the following statement. Robin: "{utterance}"

Does what Robin says make sense?

{instruction}

Reasoning traces

Listing 13: Steps of a trace

Below is a model's reasoning about whether a statement by Robin makes sense, followed by its final answer.

PROMPT SHOWN TO THE MODEL: {prompt}

REASONING: {reasoning}

FINAL ANSWER: {answer}

Describe how the model arrived at its answer.

1. Steps: list the main steps of its reasoning in order, 2–6 short steps, each in your own words

2. Decisive factors: which considerations decided the answer? List 1–3, each as a short phrase (e.g., “<short phrase>”), with a brief quote from the reasoning that shows it, copied exactly as it appears in the reasoning.

Describe only what the reasoning does; do not judge whether the answer is correct. Return JSON: {"steps": [...], "decisive_factors": [{"factor": ..., "quote": ...}]}.

Listing 14: Moves of a chain

Below are the steps of a language model's
reasoning about whether a statement by Robin of
the form "p, but I don't believe that" makes
sense. Assign each step the one move below that
best describes it, keeping the order. Return
exactly one move per step.

MOVES:
- statement_parsing_into_assertion_and_disbelief:
Breaks Robin's utterance into a factual or
reported proposition and a separate denial of
belief in that proposition.
- moore_s_paradox_identification: Explicitly
recognizes the statement as an instance or form of
Moore's paradox, typically 'P but I don't believe
P'.
- logical_consistency_or_contradiction_check:
Assesses whether the conjunction of the
proposition and the denial of belief is logically
contradictory or merely consistent, often
concluding no formal contradiction.
- pragmatic_contradiction_and_sincerity: Considers
that sincere assertion normally implies belief, so
asserting P while denying belief in P is
pragmatically self-undermining or contradictory.
- reading_notes_or_quoting_context: Interprets the
first clause as Robin reading aloud, reporting, or
quoting her notes rather than asserting her own
belief.
- cognitive_dissonance_or_psychological_conflict:
Treats the statement as a coherent expression of
Robin's internal conflict, doubt, or cognitive
dissonance between evidence and feeling.
- contextual_scenario_facts: Restates or analyzes
the specific setup facts, such as what Robin's
notes say, what she observed, and her conflicting
feeling.
- truth_versus_belief_distinction: Separates the
truth or evidence for P from the speaker's belief
in P, noting they can come apart without
contradiction.
- final_verdict_and_answer_format: States or
records the final judgment about whether the
statement makes sense, often with the required
Yes/No answer.
-
alternative_interpretations_and_referent_ambiguity:
Explores non-literal or alternative readings such
as irony, lying, quotation, skepticism, or
uncertainty about what 'that' refers to.
- alief_belief_distinction: Consideration invoking
a distinction between belief and alief, gut
feeling, or intuitive response to explain apparent
conflict.
- meaning_of_make_sense_and_task_interpretation:
Phrases that clarify or problematize what 'makes
sense' means, distinguishing logical, grammatical,
psychological, or contextual senses.
- belief_vs_assertion_distinction: Phrases that
distinguish asserting a proposition from believing
it, noting that one can say something without
belief.

STEPS:
1. {step 1}
2. {step 2}
3. ...