Why Do LLMs Accept Generics? Explaining Acceptability using Psycholinguistic and Model-Derived Factors

Introduction

People readily accept “Lions have manes” but not “Lions are male,” even though the latter is true of more lions than the former. This contrast suggests that judgments about generic statements, which concern types rather than individuals, cannot be explained simply by the frequency of a property or the number of individuals for which a statement is true ([22, 31]). Instead, these judgments reflect how people represent categories. Generics have been argued to express the mind’s default mode of generalization ([22]). People accept them on little evidence, particularly when the property is dangerous ([5])

treat properties that are principled consequences of being a kind differently from merely common ones ([31]), and even extend characteristic generics to universal claims such as “All ducks lay eggs” ([23]). Ryu et al. [37] showed that the acceptability of generics depends on two interacting factors: prevalence, or how common a property is within a kind, and cue validity, or how strongly that property distinguishes the kind from relevant alternatives. Other accounts of how people judge generics (e.g. [15, 13]) treat people as intuitive statisticians who weigh how common a property is in the kind against how common it is in alternative kinds: a highly diagnostic property can support a generic statement even when it is relatively uncommon within the kind. Having a mane distinguishes lions from other animals, whereas being male does not.

Large language models (LLMs) are increasingly used as proxies for human judgments and as models of human language processing [2, 7]. Their agreement with human judgments, however, does not establish that models arrive at those judgments in the same way as people. Generics have long posed a challenge for theories of language acquisition: although their truth conditions are difficult to specify, children understand them at an early age and with little explicit evidence [22]. LLMs offer a useful comparison because they learn from text, in which generics are common. Previous work on generics in LLMs has examined whether models overgeneralize or accept the same statements as humans [35], as well as how generics relate to quantified statements [4]. It remains unclear, however, whether models rely on the same factors that shape human judgments. We thus ask the following questions:

• To what extent can LLM acceptability be explained by factors used in human psycholinguistic studies;

• What additional factors the models themselves point to when explaining their judgments, and how they relate to the psychological ones;

• Whether shifting the models’ internal states along each factor’s direction changes their judgments as the factor predicts.

Related work

What makes generic statements acceptable has long been a central question for linguists. Leslie [22] argued that people’s acceptance of generics depends on more than prevalence, or how common a property is within a kind. Ryu et al. [37] further showed that acceptance also depends on cue validity, or how strongly a property predicts kind membership, while Hoorens et al. [15] and Hermans et al. [13] highlighted the role of relative prevalence, showing that acceptance depends on how common a property is in one kind compared with others. However, people’s judgments reflect more than statistical patterns. Characteristic properties are judged differently from merely common ones [31], sometimes supporting universal claims such as All ducks lay eggs [23]. Similarly, Leslie [22] argued that striking properties require little evidence to support a generic, and Cimpian et al. [5] showed that dangerous properties are accepted at lower prevalence levels, although this effect seems to be limited to intermediate prevalence [13].

Research on language models has also engaged with generics. Ralethe and Buys [35] and Allaway et al. [1] found that models overgeneralize from generics to universal claims and struggle with exceptions. Although models show human-like nonmonotonic reasoning when given exceptions, their judgments often shift when unrelated or even supporting examples are added [21]. In how they judge generics, Peters et al. [29] found that LLMs rate scientific generics as more generalizable and credible than people and Hu et al. [17] showed that they largely fail to distinguish principled from statistical properties once prevalence is controlled. These studies leave open the issue of which factors shape models’ judgments and whether they resemble those underlying human judgments.

A second line of work infers model judgments from probabilities rather than written answers. Direct probability measures can reveal knowledge that metalinguistic prompts miss [16, 19] and have long been used to model acceptability [20]. For generics, Cilleruelo et al. [4] introduced ConGen, using surprisal to estimate their implicit quantifiers; Calderón et al. [3] later argued that no single quantifier preserves a generic’s meaning. However, neither study examines the factors shaping human acceptance. We use surprisal to test whether models reflect these factors and compare their written answers with their expected ratings.

Even when models agree with humans, how they reach their judgments remains unclear. Interpretability research shows that numeric attributes are encoded along linear directions that can shift model outputs when manipulated [12], and that similar interventions can flip truth judgments across datasets [24]. Building on El-Shangiti et al. [8], we test whether intervening along directions associated with individual factors shifts model ratings.

Statistical accounts of acceptability

Experimental design

We first investigate whether LLMs rely on the same information as people when judging generics, and whether they combine it in the same way. Building on statistical accounts of generic acceptability ([37, 15]), we base our experimental design on Ryu et al. [37], in which participants rated each generic on two factors. Prevalence is how common people expect a property to be within a kind. For “Ducks lay eggs,” participants were asked “Suppose you encounter a duck X. How likely is it that X lays eggs?”. Ratings range from 0 to 100; where they label the ends 0 (definitely not) and 100 (definitely yes). Cue validity is how likely something with the property is to belong to the kind, among members of a broader comparison (superordinate) category. Using birds as the comparison category, participants were asked, “Suppose a bird X lays eggs. How likely is it that X is a duck?” Ratings range from $-3$ (very unlikely) to $+3$ (very likely). A separate group of participants rated acceptability, from $-3$ (definitely reject) to $+3$ (definitely accept).

When analyzing their results on human participants, Ryu et al. [37] distinguish two types of generics. Some state that one category belongs to another, e.g. “Dogs are mammals” (IS_A generics); others attribute a property to a category, e.g. “Ducks lay eggs” (feature-describing generics). For the first type, they find that acceptability depended mainly on prevalence. For the second, both factors mattered and interacted, as in Equation (1):

where $A$ is acceptability, $P$ prevalence and $C$ cue validity. Properties with high cue validity were accepted even when they were rare (“Mosquitoes carry malaria”), whereas for properties with low cue validity, acceptability depended mainly on prevalence. Refitting this regression on their published item ratings reproduced the coefficients, so we use it as the reference for human judgments.

For our experiment, we prompted each model to rate every item for acceptability, prevalence and cue validity with the same question template and scales they used (full prompts in Appendix § 7). We then applied the same analyses to each model’s ratings: the correlation between prevalence and cue validity; for IS_A generics, the correlations of acceptability with each factor; for feature-describing generics, the regression of acceptability on prevalence, cue validity and their interaction; and the same correlations within the high- and low-cue-validity groups.

Data

Ryu et al.’s items. Our first dataset contains the 84 generics used by Ryu et al. [37], drawn from Prasada et al. [31]: 20 IS_A and 64 feature-describing generics. Of the latter, 30 have high cue validity (above 1.3) and 34 have lower cue validity. The published mean ratings of acceptability, prevalence and cue validity serve as our human reference, which makes this the dataset on which model and human judgments can be compared directly. They obtained more general comparison categories for cue validity from human participants but did not publish a complete list, so we supplied one for each item using the procedure described for ConGen in Appendix § 8. Ours match the examples in their paper, such as “a bird” for “Ducks lay eggs” and “a toy” for “Dolls wear dresses.”

ConGen. To test our findings on naturally occurring generics, we constructed a second dataset from the 970 bare generics in ConGen (Cilleruelo et al., [4]). We automatically extracted category and property phrases and assigned comparison categories for the cue-validity questions. An LLM judge checked the conversions, with flagged items reviewed manually. Two annotators reviewed the comparison categories, and a third resolved disagreements (see Appendix § 8 for details). The final dataset contains 732 items: 582 feature-describing and 150 IS_A generics. Since human acceptability ratings are unavailable for ConGen, we test whether each model’s ratings follow the pattern predicted by Ryu et al. [37].

Models

Our main analyses use six open-weight instruction-tuned language models with 7–9B parameters: Qwen2.5-7B-Instruct (Qwen et al., [33]), Llama-3.1-8B-Instruct (Grattafiori et al., [11]), Mistral-7B-Instruct-v0.3 (Jiang et al., [18]), OLMo-3-7B-Instruct (Team Olmo et al., [39]), Gemma-2-9B-it (Gemma Team et al., [9]) and Qwen3-8B (Qwen Team, [34]), the latter with disabled reasoning mode, to keep the task closer to the original one for humans. To test whether the results generalize, we study two further groups of models. The first contains four larger models that, like the open models above, answer directly: Llama-3.1-70B-Instruct (Grattafiori et al., [11]) and Qwen2.5-72B-Instruct (Qwen et al., [33]), larger versions of two of those models, so that size is the main difference; and two proprietary models of undisclosed size, GPT-4.1 ([28]) and Gemini-2.5-Flash ([6]), the latter with reasoning switched off. The second contains two reasoning models, Gemini-3.8-Flash ([10]) and Qwen3.8-Flash ([32]), a mixture-of-experts model with 125B parameters (6B active per token).

Do models follow the human rule?

We ran each analysis with two sets of predictors. First, we used each model’s own prevalence and cue-validity ratings to test whether its acceptability judgments follow the human pattern given its own estimates of these factors. Each model, however, provides a single integer rating per item, whereas human ratings are averaged across participants. Noise in individual ratings may therefore obscure the interaction even when a model combines the factors as people do. To reduce this source of noise, we repeated the analyses using human ratings for Ryu et al.’s items and, for ConGen, ratings averaged within each model’s group (7–9B, larger direct-answer or reasoning models). On Ryu et al.’s items, these group averages correlate closely with human ratings ($r = .79$–.87 for prevalence and $r = .73$–.75 for cue validity; cr. Table 3 in the Appendix for details). We assess whether model judgments follow the human pattern using either the model’s own estimates or estimates averaged across participants or models.

Group	Model	A	P	C
Open, 7–9B	Qwen2.5-7B	.74	.75	.66
Open, 7–9B	Mistral-7B	.62	.69	.73
Open, 7–9B	OLMo-3-7B	.47	.38	.41
Open, 7–9B	Gemma-2-9B	.83	.79	.69
Open, 7–9B	Qwen3-8B	.60	.61	.54
Open, 7–9B	Llama-3.1-8B	.72	.65	.34
Open, 7–9B	Group average	.77	.79	.73
Large, direct answer	Llama-3.1-70B	.77	.74	.69
Large, direct answer	Qwen2.5-72B	.82	.78	.77
Large, direct answer	GPT-4.1	.79	.84	.70
Large, direct answer	Gemini-2.5-Flash	.66	.81	.48
Large, direct answer	Group average	.84	.87	.75
Reasoning	Gemini-3.8-Flash	.68	.84	.68
Reasoning	Qwen3.8-Flash	.58	.85	.71
Reasoning	Group average	.71	.87	.73

IS_A generics. For IS_A generics, Ryu et al. [37] found that acceptability increases with prevalence and decreases with cue validity, while the two factors are negatively correlated. All twelve models reproduce the positive relationship between acceptability and their own prevalence ratings on Ryu et al.’s items, while eleven show a negative correlation between prevalence and cue validity. On ConGen, however, prevalence and cue validity are positively correlated for all but the reasoning models. The relationship between prevalence and acceptability is also weaker for all twelve models. We analyze the reason for the positive correlation between the two factors for ConGen and find that it is mainly a reflection of the difference in dataset construction. Ryu et al.’s items pair a kind with a broader category (Dogs are mammals: high prevalence, low cue validity) or reverse that relation (Mammals are dogs: low prevalence, high cue validity), placing the factors in opposition by design. ConGen contains few reversed statements and many definitional ones, whose properties are both common within the kind and diagnostic of it.

The weaker relationship between prevalence and acceptability appears to reflect uncertainty about prevalence. Models agree closely on prevalence for Ryu et al.’s items (mean inter-model $r = .92$; correlation between their average and human ratings: $r = .98$), but less so for ConGen ($r = .57$). ConGen includes negations (Raspberries are not bushes) and technical descriptions (Rabbits are hind gut fermenters) that may make prevalence harder to assess. We take half of ConGen’s items with greater agreement among models and find that the correlations between prevalence and acceptability exceed those on the full set for all twelve models, hence suggesting that models rely on prevalence more strongly whenever the prevalence of a property can be clearly assessed. Figure 2 shows the relationship between prevalence and acceptability for one Qwen model from each group, while Appendix C.2 reports both correlations for all models. We return to this pattern in § 4, where we examine whether prevalence predicts acceptability because it tracks factuality.

Feature-describing generics. For feature-describing generics, Ryu et al. [37] found that people weigh prevalence and cue validity against each other. Both make a generic more acceptable, but the higher the cue validity, the less the prevalence matters. Models are sensitive to both factors and follow the human pattern in both datasets. On ConGen, prevalence and cue validity both predict acceptability in nearly all models. On Ryu et al.’s items, both factors again predict acceptability, more strongly with the human ratings of the two factors than with the models’ own, but the interaction is weak: it usually points in the human direction but is rarely significant for individual models. Averaged ratings, whether from people or models show the human pattern clearly.

Figure 3 illustrates this for Qwen3-8B, and the same pattern recurs across models (Appendix § 9.§ 3). As for humans, cue validity matters most when prevalence is low, and its effect weakens as prevalence increases. The interaction is clearer in the larger models, and averaging ratings across models brings it out in the smaller ones as well; with group-averaged ratings on ConGen, most models combine the two factors significantly in the same way as humans. Overall, the models apply the human rule in its direction, and the larger ones come close to its strength. The interaction is about as strong on Ryu et al.’s items as on ConGen, but with nine times as many items, it reaches significance for many more models on ConGen, suggesting that its weakness on Ryu et al.’s items is due to the small sample size.

Beyond prevalence and cue validity

Prevalence and cue validity explain only part of the variance in acceptability ratings (55% of the variance in people’s ratings and 3–42% in each model’s own ratings on Ryu et al.’s feature-describing items). We therefore examined four further factors that might account for how models judge generics.

In the psychology of generics, two further factors explain judgments that prevalence cannot. Some generics are rejected although almost every member has the property, as in Humans are over three years old. Prasada et al. [31] account for this with characteristicness: a generic is accepted when the property is characteristic of the kind itself, not merely common among its members. Other generics are accepted although only a few members have the property, as in Mosquitoes carry malaria. Leslie [22] and Cimpian et al. [5] attribute this to strikingness: a dangerous or otherwise remarkable property licenses a generic on little evidence. Thus, both are candidates to be additional factors shaping the ratings of the models.

The second source is the models themselves. To see whether they attend to considerations that the accounts of human judgment leave out, we asked each 7–9B model to explain its reasoning before rating Ryu et al.’s items, and compared the words in explanations that ended in rejection with those that ended in acceptance (Appendix § 10.§ 1). Two kinds of consideration stood out. Explanations ending in acceptance appealed to whether the statement is factually correct, and those ending in rejection objected that it is inaccurate or overgeneralizes, applying to more members than it should. We therefore added two new factors: factuality, the model’s judgment of whether the statement is accurate; and overreach, its judgment of whether the statement claims more than is warranted. Overreach connects to two earlier findings: people tend to accept universal claims such as All ducks lay eggs when the corresponding generic is true (Leslie et al. [23]), and language models overgeneralize from generics in a similar way (Allaway et al. [1]).

Each model rated every ConGen and Ryu et al.’s item on the four factors, with questions built like those for prevalence and cue validity (Appendix § 7). We test the factors separately for IS_A and feature-describing generics, since Ryu et al. [37] found different rules for the two on humans. To test what the new factors contribute, we needed a baseline built from the two established ones. We use $A \sim P+C$ as the baseline for both types, since adding their interaction rarely improves held-out predictions (Appendix § 10.§ 2).

Factors from the psychology of generics

We first test whether either factor restates prevalence or cue validity. Across the twelve models and both datasets, no correlation of characteristicness or strikingness with a model’s prevalence or cue validity exceeds .63 in absolute value. Both relate to prevalence in the direction the psychological accounts predict. Characteristicness rises with prevalence in every model and dataset and strikingness falls with it in all but one case (Appendix § 10.§ 3; Figure 6). The characteristicness ratings also capture what this factor describes. Ryu et al.’s items come from Prasada et al. [31], who sorted them into seven types, separating properties that are part of what the kind is (Ducks lay eggs) from ones that are merely common (Cars have radios) or striking (Mosquitoes carry malaria). Every model rates the first group as more characteristic, and the models’ ranking of the seven types follows people’s judgments in Prasada et al.’s study (Appendix § 10.§ 4).

Within each generic type, we add both factors to the baseline and standardise all ratings, so that each factor is credited only with what it explains beyond the others, then compare how much of the variance the model explains with and without them.

Characteristicness. Characteristicness raises acceptability in almost every model, for both types of generic (Figure 4). Its effect is largest where the baseline does worst. On Ryu et al.’s feature-describing items, prevalence and cue validity explain little of the ratings of most larger and reasoning models (3–24% of the variance, against 55% of people’s), but with characteristicness and strikingness added they explain 33–50%. On ConGen the effect is consistent but small. Characteristicness also accounts for items the statistical factors get wrong: Canadians are right-handed has high prevalence, yet most larger models reject it, as people do, and nearly all models rate the property as not characteristic of Canadians. More generally, the baseline’s errors track characteristicness in most models for both types. The exception is Ryu et al.’s 20 IS_A items, where prevalence and cue validity already explain most of the variance and the factors are too closely related to separate (Appendix § 10.5).

Strikingness. Strikingness, in contrast, adds little for either type of generic. Its weights are close to zero in most models, and where they are not, they mostly point the wrong way. This is not a failure of measurement, since every model recognises the striking generics as the most striking (Appendix § 10.§ 4). The models know which properties are striking but do not treat strikingness as a reason to accept a generic, nor does it matter more for rare properties, as Leslie [22] predicts (Appendix § 10.§ 6). People may not either: Hermans et al. [13] found that dangerousness affected how justifiable people found generics only at intermediate prevalence. On ConGen, most models find generics about striking properties less acceptable, but only because such properties are rare. They treat them like any other generic about a rare property.

Taken together, the two factors behave very differently. The models attend to characteristicness beyond prevalence and cue validity, most clearly in the larger and reasoning models, but they do not use strikingness. Yet even with both factors, much of the variance in the ratings remains unexplained, especially on ConGen’s natural generics. Next, we analyze the factors coming from the self-explanations of the models.

Factors from the models’ justification

Adding factuality to the four psychological factors has the largest weight in almost every model, for both types of generic and on both datasets, and together with overreach it raises the variance explained significantly in every model (Figure 4; all models in Appendix § 10.5). The gain is largest where the psychological factors explain least: on ConGen, factuality often doubles the variance explained. Factuality is not the same as acceptability, though. A statement can be true of most members of a kind and still make a poor generic, as with Humans are over three years old (Leslie et al. [23]), and the models make this distinction too: they reject up to a quarter of the statements they judge accurate, and on ConGen factuality alone often accounts for less than half of their acceptability ratings. Overreach plays a smaller role. Where it matters, it lowers acceptability, mainly on ConGen and in the larger models, and in several models it adds nothing once factuality is known.

However, factuality and overreach are not independent of the psychological factors. Prevalence, cue validity, characteristicness and strikingness explain a substantial part of the models’ factuality ratings: statements about properties that most members have, or that are characteristic of the kind, are judged more accurate and less overreaching in almost every model. As a result, almost everything the psychological factors explain about acceptability is also explained by factuality and overreach (Appendix § 10.7). The human factors thus still matter for the models, but they reach acceptability largely through judgments of whether a statement is true and whether it claims too much. On top of this shared part, factuality and overreach explain about as much again of their own. Appendix § 12 discusses individual generics on which people and models depart from the statistical rule, and what the models’ factor ratings suggest about why.

Signal from probabilistic measures

Everything so far rests on what the models answer when asked. Previous NLP research showed that responses to prompts can understate what a model knows, and measures read directly from its probabilities are often more reliable (Hu and Levy, [16]; Kauf et al., [19]). We therefore also measured acceptability without asking a question: we gave each 7–9B model only the bare generic and read off how probable it finds the property given the kind, as the average log-probability of carry malaria after Mosquitoes (P-Acceptability; Cilleruelo et al., [4]). For generics, this signal agrees worse with the ratings of people than the models’ answers do, probably because the generics differ in every word, so their probabilities also reflect word frequency, word length, and how often such statements are written (Kauf et al., [19]). Still, with people’s prevalence and cue validity as predictors, it shows the same pattern as in § 3: IS_A generics become more probable with prevalence, feature-describing ones with cue validity, and the interaction points in the human direction in every model, though too weakly to be significant (Appendix § 10.8). Following Cilleruelo et al. [4], we also placed All, Most or Some before each generic and recorded which version makes the property most probable. This best-fitting quantifier tracks prevalence, with All for the most prevalent properties and Some for the least. Whether we ask the models or read their probabilities, both show which factors go together with acceptability, not which ones the model uses to reach it. We therefore look inside the models, testing whether moving their hidden states along each factor’s direction changes their judgment.

Do factor directions change the models’ judgments?

We examined the internal states of the six 7–9B models, adapting the method of El-Shangiti et al. [8]. For each factor, we find a direction in the model’s hidden states and ask whether moving the hidden state along that direction changes the model’s acceptability judgment.

Method

For each factor, we recorded the model’s hidden states while it read the question about that factor, for the feature-describing items in ConGen. At each layer, we identified the factor’s direction by regressing the hidden states on the model’s ratings of all six factors, so that each direction captures one factor with the others held constant. As ratings, we used the model’s expected rating, the average of the possible answers weighted by their probabilities, rather than the number it wrote, because expected ratings are continuous and read from the same input as the hidden states. Fitting to the written ratings gives essentially the same result (Appendix § 11.§ 1), and the two kinds of rating agree closely, though not perfectly (Appendix § 11.§ 2).

While the model reads the acceptability prompt, we added a factor’s direction to its hidden state at one layer, in doses of ±0.5 and ±1 times the typical distance of the states from their mean, and measured the change in its expected acceptability rating. A random direction pushed by the same amount served as a control. Interventions were run on 200 items per model, at layers between 40% and 85% of the model’s depth. We read and changed the hidden state at the last token of the question, before the instruction that specifies the answer format. Five of the factor questions share the acceptability scale, so where the model is about to answer, a factor’s direction partly encodes the number it will produce, and pushing along any such direction raises the rating regardless of what the factor means. Because models process text from left to right, the hidden state at the end of the question cannot contain information about the answer format.

Steering moves every item by the same, fixed amount. To test whether the differences that actually exist between items matter, we also swapped positions along each direction. Items in the bottom quarter of a factor’s ratings were given the position along its direction of a randomly chosen item from the top quarter, leaving the rest of the hidden state unchanged, and vice versa. We report the change in expected acceptability for the items moved up minus that for the items moved down, at the same layer and token as the steering.

Results

Pushing along each factor’s direction changes the acceptability rating, in the direction the factor’s meaning predicts, and more than pushing along a random direction of the same size, in almost all cases by far (Table 1). Factuality has the largest positive effect in every model, tied with cue validity in Gemma. Prevalence, cue validity and characteristicness follow, in varying order, and strikingness, which plays almost no role in behavior, has a smaller but clear effect. Notably, cue validity, and in some models characteristicness, both of which carry little unique weight once factuality is in the behavioral model, change the judgment about as much as prevalence, so their small behavioral weights may reflect information shared with factuality rather than a lack of use. Overreach shows that the directions carry what a factor means rather than a tendency to give a higher number. Its question uses the same scale as acceptability, but a high value means that a statement claims too much, so a direction encoding only a higher answer would raise acceptability, whereas one encoding overreach should lower it. Pushing along it lowers acceptability in every model (Table 1), and giving items the position of overreaching ones lowers their rating in five of the six models (Appendix § 11.§ 3).

Model	Layer	F	P	C	Ch	O	S	Random
Qwen2.5-7B	15 / 28	.72∗	.51∗	.30∗	.18∗	−.11∗	.22∗	−.06∗
Llama-3.1-8B	13 / 32	1.88∗	1.23∗	1.65∗	1.52∗	−.16∗	.48∗	−.09∗
Mistral-7B	13 / 32	2.07∗	1.82∗	1.81∗	1.45∗	−.88∗	.51∗	.05∗
OLMo-3-7B	13 / 32	2.50∗	1.42∗	1.44∗	.89∗	−.57∗	.39∗	.11∗
Gemma-2-9B	17 / 42	.59∗	.47∗	.59∗	.35∗	−.64∗	.36∗	.01
Qwen3-8B	20 / 36	1.22∗	1.10∗	1.00∗	1.11∗	−.57∗	.43∗	.07∗

Swapping the positions of items along a factor’s direction shows which of these differences matter at the size they actually take (Appendix § 11.§ 3). Giving items the position of items rated more factual raises their rating by up to 1.8 points in five of the six models, and the corresponding swaps raise it for prevalence and cue validity in every model and lower it for overreach in five of the six. Swaps along strikingness shift it only slightly, by at most .07 points, and swaps along the random direction leave it unchanged. At realistic sizes, factuality thus matters most and strikingness barely at all, in line with their roles in behavior, while strikingness has a clear effect only when pushed far beyond the differences between items.

Conclusion and future work

Psycholinguistic factors explain part of how LLMs judge generics. The models combine prevalence and cue validity as the human rule predicts, though less clearly, and favor characteristic properties but, not striking ones. Above all, they judge whether a generic is true, and the human factors shape their judgments largely through that. Inside the models, each factor has a direction that moves the judgment as its meaning predicts. Although the factors explain much of how LLMs judge generics, a large share of the variance still remains unexplained.

Future work should identify the factors behind the unexplained variance, especially for naturally occurring generics, and collect human acceptability ratings for them so that models and people can be compared on more items. It should also trace how models combine factors beyond single directions and test generics in context, in other languages, and about social groups, where overgeneralizing may cause the most harm ([30]).

Limitations

Human ratings exist only for Ryu et al. [37]’s 84 items, so the comparison with people on ConGen is indirect as we test whether models follow Ryu et al. [37]’s human pattern, not whether they agree with people on natural generics, and the small item set limits power for individual models.

All factors are elicited from the models with one prompt each and, in the behavioural analyses, a single integer rating; apart from prevalence and cue validity, the questions are our own operationalizations, and, to the best of our knowledge, no human ratings with them exist, only Prasada et al. [31]’s related ratings of principled connections.

The internal analysis covers only the six 7–9B open-weight models, assumes a single linear direction per factor, and intervenes at one layer and token position on feature-describing ConGen items.

All items are in English, presented out of context, so the results may not carry over to other languages, to generics in discourse, or to generics about social groups.

Ethics and broader impact

We collected no new human-subject data. We use only published resources: the ratings of Ryu et al. [37] and the ConGen dataset of Cilleruelo et al. [4], under their respective licences, together with openly released and commercially available language models used within their terms of use. The comparison categories were reviewed by the authors, so no external annotators were involved.

Generics can convey and reinforce stereotypes, and naturally occurring generics may include statements about social groups; we analyze such statements only as linguistic data and do not endorse their content. Our findings caution against treating LLMs’ agreement with people as evidence of human-like reasoning, and against using LLMs as substitutes for human participants without checking their judgments. The experiments were run on NVIDIA A100 GPUs (80 GB) for the open models and through paid APIs for the proprietary ones. AI assistants helped with coding and editing; the authors designed the study and checked all code, results and wording.

Prompts

Each question was sent as a single user message in the model’s chat format, with no system message. For Qwen3, /no_think was added on a new line after the question and thinking was switched off in the chat template. Table 2 gives all seven prompts: Ryu et al.’s three questions and the four additional factors. The placeholders in angle brackets were filled from each item’s materials: the generic sentence, the category (e.g. a lion), the feature (has a mane) and the comparison category (an animal).

Factor	Prompt
Acceptability	Please rate how acceptable the following sentence is: "<generic>" Answer with a single number from -3 to +3, where -3 = "Definitely reject", 0 = "Not sure", and +3 = "Definitely accept." Respond with only the number, nothing else.
Prevalence	Suppose you encounter <category> X. How likely is it that X <feature>? Answer with a single number from 0 to 100, where 0 = "definitely not" and 100 = "definitely yes." Respond with only the number, nothing else.
Cue validity	Suppose <comparison category> X <feature>. How likely is it that X is <category>? Answer with a single number from -3 to +3, where -3 = "Very unlikely", 0 = "Not sure", and +3 = "Very likely." Respond with only the number, nothing else.
Factuality	How factually accurate is the following statement? "<generic>" Answer with a single number from -3 to +3, where -3 = "factually wrong", 0 = "Not sure", and +3 = "factually correct." Respond with only the number, nothing else.
Overreach	Sentences that generalise about a kind vary in how strong a claim they make. Some say no more than the facts support; others assert more about the kind than is warranted. Does the following sentence claim more than is warranted? "<generic>" Answer with a single number from -3 to +3, where -3 = "claims no more than is warranted", 0 = "Not sure", and +3 = "claims far more than is warranted." Respond with only the number, nothing else.
Characteristicness	Some properties are characteristic of a kind: the kind is disposed to have them, whether or not every member does. Others are merely common among members without being characteristic of the kind itself. Is "<feature>" a characteristic property of <category>? Answer with a single number from -3 to +3, where -3 = "not characteristic at all, just something many happen to have", 0 = "Not sure", and +3 = "strongly characteristic of the kind." Respond with only the number, nothing else.
Strikingness	Suppose you encounter <category> X, and you learn that X <feature>. How alarming, dangerous or noteworthy would that be? Answer with a single number from -3 to +3, where -3 = "completely unremarkable", 0 = "Not sure", and +3 = "highly alarming or noteworthy." Respond with only the number, nothing else.

Constructing the ConGen items

Parsing. We parsed each sentence with spaCy (Honnibal et al., [14]) and extracted the subtree rooted at the main verb’s nominal subject as the category phrase. The head noun was singularised and given an indefinite article (“Millipedes” → “a millipede”), except for nouns identified as uncountable, such as “furniture.” A noun was treated as uncountable if its singular and plural forms were identical or if its plural was much less frequent than its singular in a word-frequency list (Speer, [38]). The remaining clause formed the property phrase, with the verb inflected for a third-person singular subject (“eat” → “eats”; “do not grow” → “does not grow”). Following Ryu et al. [37], we labelled items as IS_A when the root was a copula with a noun phrase complement (“Joshua trees are desert plants”), and as feature-describing otherwise.

Rejections and repairs. A sentence was rejected only if no item could be built from it: its subject was a singular proper noun, which refers to an individual rather than a kind; it had no identifiable subject; its property phrase became empty after repair; its subject was a countable singular noun, which does not express a generic; or its subject was not a noun. Otherwise, sentences were repaired where needed. This produced 868 items, 120 of which had been repaired.

Deriving comparison categories. We used WordNet (Miller, [25]) to identify a comparison category for each item. To select the sense of the category noun, we embedded the generic and its phrases, along with each candidate sense’s gloss, using all-MiniLM-L6-v2, a Sentence-Transformers model (Reimers and Gurevych, [36]) based on MiniLM (Wang et al., [40]). We selected the sense with the highest cosine similarity only if it exceeded the similarity of the most frequent sense by more than 0.20; otherwise, we retained the most frequent sense. We then searched all hypernym paths from the selected sense for the nearest ancestor meeting two criteria. First, it needed a SemCor (Miller et al., [26]) sense frequency of at least 5, to favour familiar categories over technical terms such as “anseriform bird.” Second, it needed at least 100 descendant synsets, to exclude categories that were too narrow, such as “educator” for “teacher.” Candidates were ranked by distance and then by SemCor frequency. For IS_A generics, the comparison category also had to be strictly broader than the category named in the property phrase.

Verifying the conversion. An LLM judge (Gemini 2.5 Pro (Comanici et al., [6])) assessed whether each generated prevalence question preserved the meaning of the original sentence. It received the original sentence and the generated question, and returned a verdict and an error category, without being told whether the item had been repaired. The authors reviewed the 253 flagged items, correcting category or property phrases, shortening sentences or removing items. This left 766 items.

Reviewing comparison categories. The same judge assessed whether each comparison category captured the alternatives people would consider for the target kind, following Ryu et al. [37], for example, “a toy” for “Dolls wear dresses.” It approved 209 categories. Two annotators independently reviewed the remaining 557, either retaining the proposed category, suggesting an alternative or requesting that the item be removed. The annotators agreed on 394 items. A third annotator resolved disagreements by choosing between options presented anonymously and in random order, without being told that they arose from a disagreement. The final dataset contained 732 items.

Detailed results: statistical measures

Agreement of model and human ratings

Table 3 shows how closely each model’s ratings, and its group’s averaged ratings, track the human means on Ryu et al.’s feature-describing generics. Individual models vary widely: their prevalence ratings correlate with people’s at $r = .38\text{–}.85$ and their cue-validity ratings at $r = .34\text{–}.77$, with

OLMo-3-7B lowest on prevalence and Llama-3.1-8B on cue validity. Averaging within a group reduces this variation. The group averages correlate with human prevalence at $r = .79$–$.87$ and with human cue validity at $r = .73$–$.75$, matching or exceeding the best single model in each group in all but one case. This is why, on ConGen, where human ratings are unavailable, we also test the interaction with group-averaged prevalence and cue validity (§ 3.4).

IS_A generics: all models

Table 4 gives, for every model, the correlations summarised in § 3.4: prevalence with cue validity, and prevalence with acceptability, on Ryu et al.’s items and on ConGen, with ConGen split by how much the models agree on prevalence.

The prevalence × cue validity interaction

Table 5 reports the interaction for every model under each analysis described in § 3.4. Taken together, the estimates show a consistent direction. Of the 48 estimates obtained with linear models (twelve models under four analyses), 43 are negative, in the direction of the human interaction, and 14 are significantly so; none is significantly positive. The models therefore do not combine prevalence and cue validity in the opposite way to humans; they differ in how clearly they show the human pattern.

The same consistency appears in the shape of the relationship (Figure 5). For each model and analysis, we fitted acceptability against prevalence separately for properties with high and low cue validity. With the human ratings of prevalence and cue validity, and with the ratings averaged over each group of models, the advantage of a high cue validity is largest at low prevalence and smallest at high prevalence (negative interaction) for all twelve models, though not always significant. Against the human ratings, the interaction is negative for every model, as for humans, although it is weaker than in humans for most models, and significant only for the two models where it is stronger (Llama-3.1-70B and Gemini-3.8-Flash). With the models’ own ratings the same shape appears in a flatter form; it is nearly flat for Mistral-7B on ConGen and Gemini-3.8-Flash on Ryu et al.’s items, and reversed only for Llama-3.1-8B on Ryu et al.’s items.

Whether an individual model shows a significant interaction depends on the data and on the noise in its ratings. On Ryu et al.’s 64 feature-describing items, few models reach significance, and which ones do is unrelated to which do so on ConGen: the models’ estimates on the two datasets are uncorrelated. The small item set offers too little power to rank models, which is why ConGen provides the more informative test. Noise in the models’ own ratings matters mainly for the smaller models. Averaging prevalence and cue validity within each group strengthens the interaction for five of the six 7–9B models, and makes it significant for three, but leaves the larger models largely unchanged. The judgments of the smaller models may thus already reflect the interaction, but their own answers about the two factors are too noisy to reveal it, whereas the larger models’ answers are consistent enough with their judgments for the interaction to appear without averaging.

Additional factors

What the models’ explanations distinguish

Each 7–9B model rated Ryu et al.’s 84 items after first explaining its reasoning under three chain-of-thought prompts, which gave 1,510 explanations with a valid rating. To see what the models attend to without deciding in advance what to look for, we split the explanations by the rating they ended in, rejection (rating ≤ 0) or acceptance (≥ 2), and asked which words are much more common in one group. We counted every word and two-word phrase, after removing the final answer line, the words of the

Group	Rater	$r_{P,C}$ Ryu	$r_{P,C}$ ConGen	$r_{A,P}$ Ryu	$r_{A,P}$ ConGen, prevalence agreed	$r_{A,P}$ ConGen, prevalence disputed
Humans	Ryu et al. [37]	$-.64^{*}$	–	$.87^{*}$	–	–
Open, 7–9B	Qwen3-8B	$-.68^{*}$	$.24^{*}$	$.84^{*}$	$.51^{*}$	$.64^{*}$
Open, 7–9B	Gemma-2-9B	$-.72^{*}$	$.29^{*}$	$.84^{*}$	$.59^{*}$	$.75^{*}$
Open, 7–9B	Mistral-7B	$-.63^{*}$	$.35^{*}$	$.82^{*}$	$.37^{*}$	$.62^{*}$
Open, 7–9B	Llama-3.1-8B	$.10$	$.35^{*}$	$.81^{*}$	$.44^{*}$	$.47^{*}$
Open, 7–9B	Qwen2.5-7B	$-.68^{*}$	$.32^{*}$	$.62^{*}$	$.50^{*}$	$.64^{*}$
Open, 7–9B	OLMo-3-7B	$-.47^{*}$	$.31^{*}$	$.61^{*}$	$.37^{*}$	$.66^{*}$
Large, direct answer	Llama-3.1-70B	$-.70^{*}$	$.35^{*}$	$.80^{*}$	$.49^{*}$	$.72^{*}$
Large, direct answer	Qwen2.5-72B	$-.59^{*}$	$.39^{*}$	$.77^{*}$	$.60^{*}$	$.84^{*}$
Large, direct answer	Gemini-2.5-Flash	$-.70^{*}$	$.27^{*}$	$.74^{*}$	$.59^{*}$	$.72^{*}$
Large, direct answer	GPT-4.1	$-.73^{*}$	$.25^{*}$	$.74^{*}$	$.56^{*}$	$.72^{*}$
Reasoning	Gemini-3.8-Flash	$-.71^{*}$	$-.17^{*}$	$.78^{*}$	$.38^{*}$	$.49^{*}$
Reasoning	Qwen3.8-Flash	$-.72^{*}$	$.01$	$.71^{*}$	$.54^{*}$	$.66^{*}$

Group	Rater	Ryu et al.’s items: own	Ryu et al.’s items: human P, C	ConGen: own	ConGen: group avg. P, C	ConGen: own, ordinal
Humans	Ryu et al. [37]	$-.166^{*}$	–	–	–	–
Open, 7–9B	Qwen3-8B	$-.242^{*}$	$-.161$	$-.022$	$-.104^{*}$	$.02$
Open, 7–9B	Gemma-2-9B	$-.112$	$-.124$	$-.047$	$-.110^{*}$	$.00$
Open, 7–9B	Mistral-7B	$-.117$	$-.034$	$.001$	$-.038$	$.04$
Open, 7–9B	Llama-3.1-8B	$.205$	$-.129$	$-.024$	$-.066$	$.05$
Open, 7–9B	Qwen2.5-7B	$.000$	$-.149$	$-.020$	$-.103^{*}$	$.02$
Open, 7–9B	OLMo-3-7B	$-.099$	$-.027$	$-.037$	$-.028$	$.06$
Large, direct answer	Llama-3.1-70B	$-.118$	$-.229^{*}$	$-.135^{*}$	$-.182^{*}$	$-.14$
Large, direct answer	Qwen2.5-72B	$.037$	$-.120$	$-.139^{*}$	$-.090^{*}$	$-.10$
Large, direct answer	Gemini-2.5-Flash	$.014$	$-.018$	$-.021$	$-.018$	$-.03$
Large, direct answer	GPT-4.1	$-.109$	$-.077$	$-.104^{*}$	$-.093^{*}$	$-.12$
Reasoning	Gemini-3.8-Flash	$-.093$	$-.185^{*}$	$-.022$	$-.020$	$.02$
Reasoning	Qwen3.8-Flash	$-.047$	$-.021$	$-.106^{*}$	$-.103^{*}$	$-.16^{*}$

Rejecting		Accepting
Word	$z$	Word	$z$
not	20.1	clear	10.7
all	14.8	correct	9.6
incorrect	11.5	known	8.3
specific	9.7	accurately	8.3
implies	9.2	verb	8.0
generalization	8.8	widely	8.0
not accurate	8.6	subject	7.7
misleading	8.1	grammatically correct	7.5
broad	8.0	fact	7.5
all not	7.9	simple	7.3
colors	7.5	accepted	7.1
inaccurate	7.5	grammatically	7.0
weighing	7.3	generally	6.7
diversity	7.0	common	6.6
general	7.0	ambiguity	6.5
not entirely	6.7	characteristic	6.2
hair	6.6	biological	6.0
specific type	6.5	no	6.0
implies all	6.5	conveys	5.9
entirely	6.4	accurately describes	5.9

generic itself, common function words, and words used in fewer than 15 explanations. We scored each by its weighted log-odds ratio with an informative prior (Monroe et al., [27]), which measures how reliably it distinguishes the two groups.

Table 6 lists the most distinctive words. Rejections are marked by two kinds of objection: that the statement overgeneralises (not, all, implies all, generalization, broad, misleading) and that it is factually wrong (incorrect, inaccurate, not accurate). Acceptances are marked by the statement being factually correct (correct, accurately, fact, known, widely accepted) and clearly worded (grammatically correct, clear), and, less strongly, by the property being common or characteristic.

Choice of baseline

Table 7 compares three baselines within each type of generic by how well they predict held-out items (10-fold cross-validated $R^2$, mean of five splits): prevalence alone, prevalence and cue validity, and Ryu et al.’s rule with their interaction.

Group	Rater	IS_A Ryu et al. P	IS_A Ryu et al. P+C	IS_A ConGen P	IS_A ConGen P+C	Feature-describing Ryu et al. P	Feature-describing Ryu et al. P+C	Feature-describing Ryu et al. P×C	Feature-describing ConGen P	Feature-describing ConGen P+C	Feature-describing ConGen P×C
Humans	Ryu et al. [37]	.71	.82	–	–	.34	.50	.53	–	–	–
Open, 7–9B	Qwen2.5-7B	.26	.23	.22	.23	.16	.23	.14	.23	.26	.25
Open, 7–9B	Mistral-7B	.54	.48	.11	.11	.12	.12	.07	.24	.26	.26
Open, 7–9B	OLMo-3-7B	.27	−.84	.12	.13	−.01	−.04	−.06	.22	.23	.23
Open, 7–9B	Gemma-2-9B	.66	.79	.33	.36	.04	.31	.29	.28	.32	.32
Open, 7–9B	Qwen3-8B	.66	.81	.24	.25	.07	.14	.16	.14	.17	.17
Open, 7–9B	Llama-3.1-8B	.60	.60	.16	.17	.12	.05	−.08	.10	.10	.10
Large, direct answer	Llama-3.1-70B	.59	.66	.22	.20	.15	.34	.31	.12	.16	.17
Large, direct answer	Qwen2.5-72B	.48	.53	.34	.35	.07	.16	.13	.32	.38	.40
Large, direct answer	GPT-4.1	.47	.90	.30	.30	−.04	.07	.04	.13	.15	.16
Large, direct answer	Gemini-2.5-Flash	.48	.97	.34	.33	−.04	−.07	−.11	.16	.23	.22
Reasoning	Gemini-3.8-Flash	.55	.83	.10	.09	−.06	.03	.00	.03	.03	.03
Reasoning	Qwen3.8-Flash	.42	.67	.27	.27	−.05	.02	−.01	.08	.08	.08

For IS_A generics, prevalence and cue validity are the best or tied-best baseline in 9 of 12 models on both datasets. On Ryu et al.’s items cue validity takes a negative weight in every model, as it does for people, whose ratings it also predicts better (.71 vs. .82): inverses such as Mammals are dogs are highly diagnostic but rarely true. Ryu et al. describe IS_A acceptability as following prevalence alone, on the basis of each factor’s correlation with acceptability, but did not fit the two together.

For feature-describing generics, prevalence and cue validity are the best baseline, or within .01 of it, in 8 of the 12 models on Ryu et al.’s items and in at least 10 on ConGen. Adding their interaction improves the predictions for only one model on Ryu et al.’s items (Qwen3-8B) and three on ConGen, by at most .02.

Characteristicness and strikingness against prevalence

Figure 6 shows how each model’s characteristicness and strikingness ratings relate to its prevalence ratings. Characteristicness rises with prevalence in every model and dataset ($r = .24$–$.63$). This is expected as a principled connection between a kind and a property leads one to expect most members to have it, though a common property need not be a principled one (Prasada et al., [31]). Strikingness falls with prevalence in all but one case ($r$ down to $-.54$; the exception is Llama-3.1-8B on ConGen), since striking properties are typically had by few members of a kind (Leslie, [22]). The figure also shows a difference between the groups as the 7–9B models rate most generics as mildly striking whatever their prevalence, whereas the larger and reasoning models rate common properties as clearly unremarkable.

Validation of characteristicness and strikingness

Ryu et al.’s 84 items are the materials of Prasada et al. [31], who sorted them into seven types of twelve items each. Their account predicts which types involve a principled connection between kind and property, and so should be rated characteristic: quasi-definitions (Ants are insects) and majority and minority characteristic generics (Tigers are striped, Ducks lay eggs). The other four types should not be: majority statistical generics (Cars have radios), striking generics (Mosquitoes carry malaria), and majority and minority false generalisations (Canadians are right-handed, Rooms are round). Table 8 tests each model’s ratings against

Group	Model	min. char. $-$ maj. stat. raw	min. char. $-$ maj. stat. given P	principled $-$ other	$\rho$ with humans	striking $-$ other
Open, 7–9B	Qwen2.5-7B	.83*	.81*	2.44*	.68	2.03*
	Mistral-7B	2.00*	1.97*	2.41*	.68	3.67*
	OLMo-3-7B	$-$ .17	$-$ .23	1.02*	.68	2.28*
	Gemma-2-9B	.25	.24	2.45*	.93*	4.18*
	Qwen3-8B	2.50*	2.16*	1.94*	.85*	2.36*
	Llama-3.1-8B	.50	$-$ .17	1.12*	.85*	2.39*
Large, direct answer	Llama-3.1-70B	$-$1.17	$-$1.12	2.03*	.82*	5.19*
	Qwen2.5-72B	1.50*	1.41*	2.94*	.82*	4.25*
	GPT-4.1	1.45*	1.49*	3.76*	.93*	3.94*
	Gemini-2.5-Flash	.75	1.24	4.00*	.95*	5.33*
Reasoning	Gemini-3.8-Flash	3.42*	3.41*	4.97*	.95*	5.32*
	Qwen3.8-Flash	2.83*	3.10*	4.20*	.88*	4.17*

these predictions. All twelve models rate the three principled types as more characteristic than the other four, and for nine of them the ranking of the seven types follows people’s ratings of principled connections in Prasada et al.’s study.

The hardest case contrasts minority characteristic generics such as Ducks lay eggs with majority statistical ones such as Cars have radios. Elsewhere, characteristic properties are also the more common ones, so a model could pass by tracking prevalence; here prevalence points the other way, as people judge the former less common, so a model relying on prevalence would rate them lower. We therefore also compare the two types while controlling for each model’s own prevalence ratings. Seven of the twelve models rate minority characteristic generics as more characteristic even then. In the other five the difference is not significant: it is near zero or reverses for OLMo-3-7B, Gemma-2-9B and the two Llama models, and is positive but not significant for Gemini-2.5-Flash ($p = .055$). For Llama-3.1-8B, the raw difference disappears once prevalence is controlled, so its characteristicness ratings partly re-express its prevalence estimates. Several of these models also rate striking properties as characteristic, which Prasada et al. [31] do not count as principled. With twelve items per type, these tests have limited power, but they suggest that the characteristicness weights in

Section 4 are most trustworthy for the seven models that pass.

Full results for the additional factors

Table 9 adds characteristicness and strikingness to the baseline, and Table 10 adds factuality and overreach to all four psychological factors, for feature-describing generics on both datasets and IS_A generics on ConGen. Together they give, for every model, the variance explained at each step summarised in Figure 4. Characteristicness raises acceptability in most models (eleven of twelve on Ryu et al.’s feature-describing items, nine on ConGen’s, eleven on ConGen’s IS_A items), while strikingness is mostly near zero and, where significant, more often negative. Factuality has a significant positive weight in every model and dataset; overreach, where significant, mostly lowers acceptability. Most of what the four psychological factors explain in acceptability is also explained by factuality and overreach.

Ryu et al.’s 20 IS_A items are left out of both tables. Across so few items, prevalence, cue validity and characteristicness are so closely related that each can largely be predicted from the others (variance inflation factors up to 60, against at most 2.2 elsewhere), and the four psychological factors explain 75–99% of the variance in factuality, so the weights are not interpretable. Strikingness also cannot be estimated for four models, which rate every IS_A item as equally unremarkable.

Interactions of the psychological factors

The psychological accounts predict two interactions with prevalence. Leslie [22] proposes that a striking property licenses a generic even when it is rare, so strikingness should matter more at low prevalence (a negative prevalence $\times$ strikingness weight). A characteristic property may likewise compensate for low prevalence, as in Ducks lay eggs (a negative prevalence $\times$ characteristicness weight). To test them, we add each interaction, one at a time, to the four-factor regression of Table 9 and report its weight in Table 11, together with the change in held-out $R^2$ when all pairwise interactions are added at once.

Group

Model

Feature, Ryu et al. $R^{2}_{P,C}$

Feature, Ryu et al. Ch

Feature, Ryu et al. S

Feature, Ryu et al. $R^2$

Feature, ConGen $R^{2}_{P,C}$

Feature, ConGen Ch

Feature, ConGen S

Feature, ConGen $R^2$

IS_A, ConGen $R^{2}_{P,C}$

IS_A, ConGen Ch

IS_A, ConGen S

IS_A, ConGen $R^2$

Open, 7–9B

Qwen2.5-7B

.36

.35*

.26*

.47*

.27

.09*

−.04

.28*

.27

.28*

−.01

.32*

Open, 7–9B

Mistral-7B

.23

.32*

.17

.33*

.27

.22*

.02

.31*

.14

.27*

−.08

.22*

Open, 7–9B

OLMo-3-7B

.05

.50*

.02

.26*

.24

.06

−.06

.25

.16

.22*

.06

.20*

Open, 7–9B

Gemma-2-9B

.37

.39*

.09

.51*

.33

.17*

−.01

.35*

.38

.33*

.01

.45*

Open, 7–9B

Qwen3-8B

.21

.18

−.10

.24

.18

.13*

.05

.20*

.28

.12

−.09

.30

Open, 7–9B

Llama-3.1-8B

.22

.47*

−.12

.37*

.12

.09*

.08*

.13*

.21

.25*

−.01

.26*

Large, direct answer

Llama-3.1-70B

.42

.26*

.14

.49*

.17

.06

.01

.17

.25

.31*

−.05

.33*

Large, direct answer

Qwen2.5-72B

.24

.59*

.09

.47*

.39

.22*

.02

.42*

.38

.19*

−.09

.41*

Large, direct answer

GPT-4.1

.18

.61*

−.03

.44*

.16

.13*

−.18*

.20*

.33

.22*

−.10

.38*

Large, direct answer

Gemini-2.5-Flash

.03

.76*

−.35*

.50*

.23

.18*

−.07

.26*

.35

.32*

−.21*

.44*

Reasoning

Gemini-3.8-Flash

.10

.59*

.03

.36*

.05

.10*

−.14*

.07*

.14

.24*

−.17*

.20*

Reasoning

Qwen3.8-Flash

.12

.50*

.05

.33*

.09

.00

−.05

.09

.31

.24*

−.02

.34*

Group	Model	Feature, Ryu et al. P×Ch	Feature, Ryu et al. P×S	Feature, Ryu et al. $\Delta R^{2}_{\mathrm{cv}}$	Feature, ConGen P×Ch	Feature, ConGen P×S	Feature, ConGen $\Delta R^{2}_{\mathrm{cv}}$	IS_A, ConGen P×Ch	IS_A, ConGen P×S	IS_A, ConGen $\Delta R^{2}_{\mathrm{cv}}$
Open, 7–9B	Qwen2.5-7B	−.09	−.02	−.074	.00	−.06	−.011	−.07	.00	−.073
Open, 7–9B	Mistral-7B	.01	−.23	−.154	−.04	.00	−.019	−.01	.02	−.115
Open, 7–9B	OLMo-3-7B	−.12	.25*	−.154	−.01	.09*	−.007	−.17	.01	−.051
Open, 7–9B	Gemma-2-9B	.04	−.16	−.428	−.01	−.02	−.018	.00	−.11	−.046
Open, 7–9B	Qwen3-8B	−.09	.13	−.170	−.12*	−.02	−.016	−.04	−.10	−.064
Open, 7–9B	Llama-3.1-8B	.16	.12	−.622	−.07	.00	−.020	−.08	.05	−.079
Large, direct answer	Llama-3.1-70B	.13	.03	.014	−.10*	.07*	−.001	−.01	.00	−.193
Large, direct answer	Qwen2.5-72B	.17	−.11	−.238	−.06*	−.04	.000	−.07	.11	−.028
Large, direct answer	GPT-4.1	.19	.16	−.024	−.04	.06	−.021	.10	−.08	−.098
Large, direct answer	Gemini-2.5-Flash	.22*	−.10	−1.154	.01	.02	−.008	.11	−.02	−.098
Reasoning	Gemini-3.8-Flash	.18	.08	−.049	−.07	.09*	−.024	.17	.00	−.147
Reasoning	Qwen3.8-Flash	.25*	−.01	−.082	−.01	−.01	−.013	.13	−.01	−.116

Neither prediction holds reliably. No model shows the first: where the prevalence $\times$ strikingness weight is significant, it is positive. The second appears in three models for feature-describing generics on ConGen, with small weights, and in the opposite direction in two on Ryu et al.’s items. Adding all pairwise interactions among the factors at once makes the predictions for held-out items worse in 11 of 12 models for feature-describing generics on both datasets and in all 12 for IS_A generics on ConGen. We therefore report the additive models.

Shared variance of the two sets of factors

Figure 7 splits what the six factors explain in each model’s acceptability into three parts (a commonality analysis): the part explained only by the psychological factors, the part explained by either set, and the part explained only by factuality and overreach. The shared part is $R^2_{\mathrm{psych}} + R^2_{\mathrm{F,O}} - R^2_{\mathrm{all}}$, where each $R^2$ comes from a model with that set of factors.

The part explained only by the psychological factors is small in almost every model, especially on ConGen: almost everything they explain about acceptability is also explained by factuality and overreach. This is because the models’ factuality and overreach ratings themselves depend on the psychological factors. On ConGen, prevalence and characteristicness predict higher factuality in 11–12 of the 12 models and lower overreach in 9–10, cue validity predicts higher factuality for feature-describing generics in 11, and strikingness predicts neither. On top of this shared part, factuality and overreach explain about as much again of their own.

overreach explain about as much again of their own.

Surprisal-based acceptability

For the six 7–9B models, whose token probabilities are available, we computed two measures of acceptability that involve no question. P-Acceptability is the negative cross-entropy of the property tokens given the subject, so that a property the model finds predictable for the kind scores high (Cilleruelo et al., [4]). SLOR is the log-probability of the whole sentence, normalised for the frequency of its words and its length (Lau et al., [20]). On Ryu et al.’s items, P-Acceptability correlates with people’s acceptability at $r = .39$–.46 and SLOR at $.29$–.43$, both below the models’ answers ($r = .51$–.84; Table 12).

Model	$r$ with people’s acceptability P-Acc.	$r$ with people’s acceptability SLOR	$r$ with people’s acceptability answers	P-Acceptability from people’s P and C: IS_A P	P-Acceptability from people’s P and C: IS_A C	P-Acceptability from people’s P and C: Feature-describing P	P-Acceptability from people’s P and C: Feature-describing C	P-Acceptability from people’s P and C: Feature-describing P×C
Qwen2.5-7B	.44*	.43*	.72*	.46	−.16	.20	.32*	−.07
Mistral-7B	.42*	.37*	.67*	.61*	−.08	.12	.45*	−.03
OLMo-3-7B	.44*	.38*	.51*	.41	−.42	.23	.30*	.00
Gemma-2-9B	.46*	.29*	.84*	.77*	.04	.42*	.25*	−.09
Qwen3-8B	.39*	.30*	.69*	.21	−.49	.18	.23	−.11
Llama-3.1-8B	.44*	.38*	.77*	.51	−.10	.21	.31*	−.07

Ryu et al.’s analysis. Table 12 also repeats Ryu et al.’s analysis with P-Acceptability as the measure of acceptability and people’s prevalence and cue validity as predictors. For IS_A generics, prevalence raises P-Acceptability in all six models (significantly in two, with 20 items) and cue validity lowers it in five, as for people. For feature-describing generics, cue validity raises it in all six (significantly in five), and the interaction of prevalence and cue validity is negative, the human direction, in all six, but significant in none. The human pattern thus appears not only in what the models answer but in how probable they find the generic itself: its main effects are clearly present, while the interaction, as with the written ratings, points the right way but is too weak to establish for individual models.

Quantifiers. Following Cilleruelo et al. [4], we compared each generic with its quantified versions and recorded the version (All, Most, Some, or the bare plural) whose property the model finds most predictable. All is the most frequent best fit for every model (37–71% of Ryu et al.’s items, 30–42% of ConGen’s), Some accounts for 19–27% of ConGen’s, close to the fifth that Cilleruelo et al. [4] report, and the bare plural fits best for only 2–24%. We also find that the best-fitting quantifier tracks prevalence (Figure 8): in every model and dataset except OLMo on Ryu et al.’s items, the generics best fitted by All have the highest prevalence and those best fitted by Some the lowest (people’s prevalence on Ryu et al.’s items, the model’s own on ConGen).

Group

Model

Feature, Ryu et al. F

Feature, Ryu et al. O

Feature, Ryu et al. $R^2_6$

Feature, Ryu et al. shared

Feature, ConGen F

Feature, ConGen O

Feature, ConGen $R^2_6$

Feature, ConGen shared

IS_A, ConGen F

IS_A, ConGen O

IS_A, ConGen $R^2_6$

IS_A, ConGen shared

Open, 7–9B

Qwen2.5-7B

.58*

.19*

.68*

90%

.72*

.01

.64*

96%

.72*

.06

.66*

96%

Open, 7–9B

Mistral-7B

.59*

−.28*

.64*

98%

.58*

−.12*

.58*

91%

.65*

−.11

.56*

97%

Open, 7–9B

OLMo-3-7B

.35*

−.16

.42*

43%

.57*

.03

.49*

85%

.77*

−.13*

.71*

96%

Open, 7–9B

Gemma-2-9B

.42*

−.22

.67*

89%

.56*

−.20*

.63*

97%

.59*

−.30*

.80*

99%

Open, 7–9B

Qwen3-8B

.59*

−.25*

.57*

86%

.56*

−.17*

.51*

94%

.70*

−.06

.70*

89%

Open, 7–9B

Llama-3.1-8B

.63*

−.05

.64*

80%

.42*

.08*

.29*

62%

.28*

.09

.32*

53%

Large, direct answer

Llama-3.1-70B

.55*

.00

.63*

89%

.56*

−.05

.44*

91%

.53*

−.07

.54*

88%

Large, direct answer

Qwen2.5-72B

.83*

−.03

.85*

95%

.65*

−.12*

.68*

96%

.70*

−.10

.71*

98%

Large, direct answer

GPT-4.1

.71*

−.06

.71*

81%

.62*

−.10*

.49*

97%

.60*

−.21*

.64*

98%

Large, direct answer

Gemini-2.5-Flash

.34*

−.09

.57*

54%

.48*

−.12*

.44*

92%

.53*

−.17*

.64*

96%

Reasoning

Gemini-3.8-Flash

.49*

−.20

.54*

73%

.21*

−.01

.10*

74%

.27*

−.10

.26*

86%

Reasoning

Qwen3.8-Flash

.64*

−.10

.49*

73%

.35*

−.13*

.20*

88%

.42*

−.23*

.49*

94%

Mechanistic analysis

Results with the written ratings

In the main analysis, each factor’s direction is fitted to the model’s expected rating of the factor, the average of the possible answers weighted by their probabilities. Fitting the directions to the ratings the models actually wrote instead gives the same results. Across all layers and directions, the steering effects of the two versions correlate at $r = .993$–$.998$ for every model, and the layer at which factuality’s effect peaks is the same in all six (Table 13).

Model	Layer	F	P	C	Ch	O	S	Random
Qwen2.5-7B	15 / 28	.64$^{*}$	.50$^{*}$	.31$^{*}$	.18$^{*}$	−.09$^{*}$	.21$^{*}$	−.07$^{*}$
Llama-3.1-8B	13 / 32	1.68$^{*}$	1.27$^{*}$	1.57$^{*}$	1.44$^{*}$	−.07$^{*}$	.43$^{*}$	−.08$^{*}$
Mistral-7B	13 / 32	1.99$^{*}$	1.74$^{*}$	1.40$^{*}$	1.09$^{*}$	−.87$^{*}$	.50$^{*}$	.05$^{*}$
OLMo-3-7B	13 / 32	2.44$^{*}$	1.58$^{*}$	1.35$^{*}$	.93$^{*}$	−.47$^{*}$	.38$^{*}$	.11$^{*}$
Gemma-2-9B	17 / 42	.67$^{*}$	.51$^{*}$	.57$^{*}$	.40$^{*}$	−.59$^{*}$	.36$^{*}$	.01
Qwen3-8B	20 / 36	1.22$^{*}$	1.11$^{*}$	.99$^{*}$	1.10$^{*}$	−.60$^{*}$	.43$^{*}$	−.10$^{*}$

What the models write and what their probabilities say

A model’s written rating need not be the answer its output probabilities favor, so we compared the two for every question (Table 14). For each of ConGen’s feature-describing generics, the expected rating is the average of the possible answers weighted by their probabilities when the model is about to answer, and the most probable answer is the single answer with the highest probability; the written rating comes from greedy generation at temperature 0.

Model	A $r$	A =mode	P $r$	P =mode	C $r$	C =mode	F $r$	F =mode	Ch $r$	Ch =mode	O $r$	O =mode	S $r$	S =mode	mass
Qwen2.5-7B	.98	90%	.97	80%	.96	86%	.97	86%	.96	73%	.92	79%	.94	88%	1.00
Mistral-7B	.92	87%	.87	71%	.89	83%	.92	86%	.91	77%	.87	74%	.90	87%	1.00
OLMo-3-7B	.87	82%	.87	58%	.81	79%	.86	73%	.89	91%	.77	97%	.91	95%	1.00
Gemma-2-9B	.97	86%	.95	82%	.97	90%	.97	92%	.96	87%	.94	87%	.95	86%	1.00
Qwen3-8B	.99	97%	.98	95%	.99	97%	.99	99%	.99	96%	.97	98%	.98	98%	1.00
Llama-3.1-8B	.94	90%	.95	96%	.94	91%	.89	95%	.92	93%	.84	71%	.87	88%	1.00

The two agree closely but not perfectly. The written rating correlates with the expected rating at $r = .77$–$.99$, depending on model and question, and nearly all probability falls on valid answers for every model and question. Agreement differs more between models than between questions: Qwen3-8B writes its most probable answer for 95–99% of items ($r = .97$–$.99$), whereas OLMo does so for as few as 58% ($r = .77$–$.91$). Among the questions, overreach shows the least agreement ($r = .77$–$.97$).

Even with greedy decoding, a written rating can differ from the most probable answer for technical reasons: a number is written token by token, so the first token chosen need not begin the most probable number (as for prevalence, from 0 to 100), and the prompts of the written runs and of the scoring differ slightly in tokenization for some models. The disagreement is therefore an upper bound on how far what a model writes departs from what it encodes about the answer. In line with Hu and Levy [16], it suggests that written ratings are a close but imperfect window on a model’s own judgment, which is why the main analysis uses the expected ratings; as Appendix § 11.§ 1 shows, the conclusions do not depend on the choice.

Swaps

Table 15 gives the result of swapping positions along each factor’s direction for every factor and model, at the layer of Table 1. Factuality gives the largest shift in five of the six models, up to 1.79 points; in Gemma it is not distinguishable from zero. Swaps along prevalence and cue validity raise the rating in every model, swaps along characteristicness in five, and swaps along overreach lower it in five. Swaps along strikingness shift the rating only slightly, by at most .07 points, although consistently in the same direction, and swaps along the random direction leave it unchanged.

Model	Layer	F	P	C	Ch	O	S	Random
Qwen2.5-7B	15 / 28	.41{sup}*`	.11{sup}*`	.04{sup}*`	.00	.00	.04{sup}*`	.00
Llama-3.1-8B	13 / 32	1.28{sup}*`	.72{sup}*`	.43{sup}*`	.38{sup}*`	−.10{sup}*`	.04{sup}*`	.00
Mistral-7B	13 / 32	1.79{sup}*`	.68{sup}*`	.58{sup}*`	.26{sup}*`	−.58{sup}*`	.04{sup}*`	.00
OLMo-3-7B	13 / 32	1.37{sup}*`	.32{sup}*`	.18{sup}*`	.10{sup}*`	−.13{sup}*`	.02{sup}*`	.01
Gemma-2-9B	17 / 42	.05	.11{sup}*`	.06{sup}*`	.04{sup}*`	−.19{sup}*`	.03{sup}*`	.01
Qwen3-8B	20 / 36	.51{sup}*`	.46{sup}*`	.22{sup}*`	.40{sup}*`	−.30{sup}*`	.07{sup}*`	.02

Where people and models depart from the rule

For each of the 64 feature-describing items, we compared people’s and each model’s acceptability rating with what Ryu et al.’s rule predicts from their own prevalence and cue-validity ratings. The items the rule gets wrong are mostly the same for people and models, and the models’ ratings of the other factors suggest why. Both reject human-related generalizations, such as Engineers are male and Teachers are female, more than the rule predicts. Ryu et al. [37] suggested that people may know more counterexamples about human beings and avoid generalizations that create stereotypes, and the models show this tendency in a sharper form: they agree with people that most engineers are male, yet judge the statement inaccurate and overreaching, treating a common stereotype as a false claim about the group; on average, all twelve reject Ryu et al.’s generalizations about human beings more than the rule predicts. In the other direction, people and models both accept generics such as Car crashes kill people and Ducks lay eggs more than the rule predicts, and all twelve models rate these as fully accurate, consistent with their judgments following truth rather than frequency.

The models are stricter than people with generics whose counterinstances have an alternative property, which Leslie [22] argues are judged false because they take on an almost universal reading: people are nearly neutral about Ducks are female and Lions are male, while almost every model rejects both and rates them as inaccurate, as Leslie’s account predicts. The smaller models diverge further on generics about harm: they reject Plastic bags suffocate small children and Rip tides drown swimmers, which people and most of the larger models accept, and five of the six reject Knives cut people although all six rate it as accurate. Factuality cannot explain this last rejection; it may reflect safety tuning that makes smaller models reluctant to endorse unqualified statements about harm to people, although they accept others, such as Car crashes kill people, so the boundary is not clear-cut. Finally, almost every model accepts Tables are 10 feet long, although half of them rate it as inaccurate; here the models may judge whether the sentence is well formed rather than whether it is true, a reading their explanations sometimes adopt.