betaFeedback

WorksComputer Science

LLM Recommenders Produce Demographic Inequality Through Content Preferences

Abstract

Large language models (LLMs) are increasingly being integrated into systems that rank and recommend online content. We examine how this shift can produce demographic inequality in whose voices are surfaced. Using 10,000 Twitter/X posts written by 1,000 survey respondents and linked to verified self-reported characteristics across nine demographic attributes, we conduct a controlled recommendation experiment across three LLM providers, six recommendation objectives, and five levels of contextual information. By varying whether models receive post text alone, platform metadata, or explicit author demographics, we isolate three pathways to unequal recommendation: preferences for demographically patterned content, reliance on unequal engagement signals, and direct use of disclosed identity. We find that demographic inequality emerges from text alone. Models consistently favour longer, more negative, and more opinionated posts, particularly those concerning news and social issues, producing systematic differences in the demographic composition of recommended authors. Platform metadata generally amplifies these inequalities, while explicitly disclosing authors' demographic characteristics produces little additional change. Recommendation objectives mainly affect the magnitude of these patterns: controversial framing intensifies both content and demographic skew, whereas popularity-oriented framing dampens it, although for some attributes - most notably age - the objective can also reverse which group is favoured. LLM-based recommenders can therefore reproduce demographic inequality by proxy, not through explicit demographic targeting, but through preferences for content and engagement signals already structured by social inequality.

Introduction

Large language models (LLMs) are increasingly being incorporated into the systems that rank, curate, and recommend online content. X has begun integrating Grok into its recommendation infrastructure [1, 2], Bluesky has introduced AI-assisted tools for constructing personalized feeds [3], and Instagram now lets users edit the topics driving their recommendations directly [4]. Researchers have likewise begun developing LLM-based systems that rank content according to user-specified or pluralistic values rather than default engagement optimization [5, 6]. LLMs are attractive for such tasks because they can evaluate meaning, style, and context directly from natural-language instructions.

But this flexibility potentially creates a new route to inequality in online visibility. Recommendation systems determine which speakers and forms of expression receive attention, and algorithmic curation can reinforce popularity advantages, shape political exposure, and amplify existing inequalities [7, 8, 9, 10]. With LLMs, demographic inequality need not arise because a model explicitly discriminates between demographic groups: a model may simply favour particular topics, styles, emotional registers, or signals of prior engagement. If these characteristics are socially patterned, content-based ranking can become demographic inequality by proxy.

This suggests three distinct pathways through which inequality may enter LLM recommendation. First, models may have content preferences, selecting styles and topics that correlate with author demographics. Second, they may rely on platform metadata, such as engagement and audience measures, that also correlate with demographic position. Third, they may respond directly to demographic disclosure when identity information is explicitly provided. These pathways have different implications. Restricting access to protected attributes may reduce direct discrimination, but it cannot eliminate inequality that travels through correlated content or engagement signals.

Existing research has shown that both conventional recommenders and LLMs can produce unequal outcomes [11, 12, 13, 14, 15, 16]. Recent audits of LLM-based recommendation find popularity bias, demographic skew, polarization amplification, and strong prompt sensitivity [17, 18, 19, 20, 21]. What these studies cannot cleanly establish is where demographic inequality enters the ranking process. A key limitation is that demographic characteristics are usually inferred from social-media text or profiles rather than independently observed. Such inference is noisy and potentially circular: the same linguistic associations used to infer an author’s identity may also drive the ranking itself [22].

We overcome this problem using 10,000 Twitter/X posts from 1,000 survey respondents whose demographic characteristics are independently verified across nine attributes [23]. In a controlled ranking experiment, three LLMs repeatedly select 10 posts from pools of 100, while we vary both the recommendation objective and the information available to the model. Across six objectives and five context conditions, models receive post text alone, author metadata, post-engagement metadata, both forms of platform metadata, or platform metadata together with verified demographics (Figure 1). This design allows us to observe demographic skew that emerges from text alone, how it changes when platform signals become available, and what explicit demographic disclosure adds beyond those signals.

Figure 1. Overview of the experimental design. The YouGov–Twitter/X pool provides 10,000 posts from 1,000 authors with verified ground-truth demographics across nine attributes. We vary three factors orthogonally: LLM provider (3), prompt style (6), and context level (5), yielding 90 conditions each replicated across 100 independent trials.

Our results show that LLM-based recommendation can generate demographic inequality without direct access to demographic identity. Across providers, models consistently favour particular types of content, especially longer posts, news and social-concern topics, and more negative and opinionated language. Because these content characteristics vary across authors, they produce structured demographic skews in recommendation. Older, male, more educated, and higher-income authors are generally over-represented, although the magnitude and, in some cases, direction of these associations differ across providers. Ideological selection is U-shaped, with centrist content disadvantaged relative to content from the left and right.

Platform metadata generally amplifies these inequalities by carrying pre-existing differences in engagement and audience size into the ranking process. Explicitly disclosing authors’ demographic attributes, by contrast, changes recommendations little once text and platform information are already available. Recommendation objectives also govern the size of these effects: framing the task as surfacing controversial content produces the strongest skew and popular framing the weakest, and in some cases changes which group benefits.

The central implication is that demographic blindness does not imply demographic neutrality. LLM-based recommendation can generate unequal visibility without ever being told who authored a post. Inequality can arise instead from what models learn to value in content and from the accumulated traces of prior attention embedded in platform signals.

Results

We begin with the text-only condition, in which models rank posts without receiving platform metadata or explicit information about author demographics. This provides the cleanest setting for identifying what happens when recommendation is based on post text alone. We first ask what kinds of content LLMs preferentially surface, then whether those choices systematically change whose voices receive visibility. We next examine how much of the resulting demographic skew can be accounted for by observed characteristics of the content. Finally, we introduce platform metadata and explicit demographics to test how these additional sources of information alter the ranking.

LLMs Systematically Prefer Particular Forms of Content

Even when given nothing beyond the post itself, LLMs exhibit a distinctive and broadly consistent selection profile. Across providers, they favour news and social-concern content, longer posts, more negative and opinionated language, and posts that are less conversational (Figure 2). Under the neutral prompt, recommended content contains 27.6 percentage points more News & Social Concern posts than the posts passed over. Recommended posts are also consistently longer, averaging between 14.0 and 39.6 additional characters depending on the recommendation objective. Selection generally shifts toward more negatively valenced and more subjective language, while posts containing mentions are substantially less likely to be surfaced.

Directional bias across topic and content features in the text-only condition

Figure 2. Directional selection patterns in the text-only condition: News & Social Concern topic skew (first column, percentage points) and NLP feature differences (remaining columns, original units), with rows indicating prompt styles and columns models; ‘Avg’ gives the marginal mean across the other dimension. Colour scales are independent across panels.

These directional patterns are reflected in the relative strength of association between observed post characteristics and recommendation (Figure 3). Among the characteristics we observe, selection is much more strongly associated with properties of the post than with the demographics of its author. Primary topic shows the strongest overall association with recommendation (R2=0.044R^2 = 0.044), followed by text length (R2=0.029R^2 = 0.029) and sentiment polarity (R2=0.014R^2 = 0.014). Demographic attributes are considerably weaker correlates, ranging from R2=0.004R^2 = 0.004 for ideology to below 0.001 for religiosity and partisanship.

Heatmap and average-bar plot of $R^2$ by feature and prompt style

Figure 3. R2R^2 (variance explained) for each feature across six prompt styles in the text-only condition. Features are ordered by average R2R^2 across all 18 conditions (3 providers ×\times 6 prompts). Content features dominate, while demographic features cluster near the bottom. Significance markers use FDR-corrected (Benjamini–Hochberg [24], α=0.05\alpha= 0.05) pp-values within the figure’s family of tests: heatmap cells *, **, *** indicate significance for 1, 2, or all 3 models; average-bar significance is not separately marked.

This association’s strength also varies substantially by recommendation objective. The controversial prompt produces by far the strongest overall dependence on observed content (mean R2=0.011R^2 = 0.011 across all 25 features), roughly 3.5 times the weakest objective, popular (R2=0.003R^2 = 0.003), with primary topic alone reaching R2=0.114R^2 = 0.114 under controversial versus 0.010 under popular. Informative framing stands out for a different reason: it is the only objective under which URL presence is a comparably strong predictor of selection (R2=0.026R^2 = 0.026), well above its association under every other prompt (R2≤0.009R^2 \le0.009), consistent with a preference for linked, presumably sourced material when the stated goal is to inform rather than to entertain or generate engagement. This overall ranking of features is, in turn, broadly consistent across providers (SI Appendix, Fig. S1).

Content Selection Reshapes Whose Voices Receive Visibility, and Explains Only Part of the Resulting Skew

Selection on post text also changes the demographic composition of visibility, even though no demographic attributes are supplied to the models. The clearest patterns involve age and ideology, while differences by gender, education, income, and race vary more across providers (Figure 4A).

Demographic selection skew and conditional-logit coefficients

Figure 4. Figure 4: Demographic selection skew, and how much of it survives controlling for content. (A) Demographic differences in representation by model in the text-only condition, measured in percentage points relative to shown-but-not-recommended posts. (B) Conditional-logit coefficients from the single-block demographic-only model versus the same coefficients estimated jointly with content features (centre), text context, so that both models reflect only information the LLM actually observed in this condition; the left and centre panels share the row order and x-scale of the left panel for direct comparison, and faded markers are non-significant (p≥0.05p \ge0.05). The right panel marks each association’s significance transition: upward triangles gained significance after adjustment, downward triangles lost it, solid circles stayed significant, and faint dots stayed non-significant; colour again identifies the provider. x-axis (left, centre): log-odds, >0> 0 = more likely recommended.

Age shows the clearest gradient: authors aged 65 and above are over-represented among recommended posts by 3.5 percentage points on average, whereas authors aged 25–34 are under-represented by 2.7 points. Ideological selection is similarly structured but non-monotonic: left-leaning authors are over-represented by 3.8 percentage points and right-leaning authors by 2.6 points, while centrist authors are under-represented by 4.9 points and centre-right authors by 2.4 points. Other demographic associations are less uniform across models. Claude and Gemini favour male-authored content, for example, whereas GPT-4o Mini mildly favours female-authored content; education and income likewise show provider heterogeneity. Race also exhibits differences in selection: Black authors are under-represented across all three providers (by 1.6 to 2.7 percentage points); estimates for the smallest racial categories should be interpreted cautiously given subgroup size.

On top of this model-level heterogeneity, demographic consequences also vary by recommendation objective: averaged across all nine attributes, mean skew under controversial and informative runs roughly three times larger than under engaging or popular, and for a few attributes, framing can even flip which group is favoured. Age shows the starkest example, shifting from −1.7 percentage points under popular to +11.5 under controversial; gender shows a comparable swing, from near zero under popular and engaging to the same +11.5-point male-author skew under controversial, an attribute on which GPT-4o Mini disagrees with the other two providers at baseline but converges with them under the more extreme objectives (SI Appendix, Fig. S3).

These demographic patterns plausibly arise, at least in part, because authors from different groups produce systematically different kinds of content, and the models’ content preferences then carry that correlation into recommendation. Age offers a direct illustration. Authors differ sharply in the content they produce within the candidate pool itself, independent of any model’s recommendations: the share of posts classified as News & Social Concern rises from 15.3% among authors aged 18–24 to 48.7% among those aged 65 and above. Because all three models strongly favour this topic, content-based selection alone can generate an age gradient in visibility, without any model ever observing an author’s age.

This correlation between demographics and content is not limited to age, and it may also

reflect textual characteristics that our topic, sentiment, and style measures do not capture. To disentangle these possibilities, we ask which demographic associations survive once we control for observed content characteristics. For each category, we compare its conditional-logit coefficient estimated alone, without controlling for content (demographics only; the full breakdown for all attributes is in SI Appendix, Fig. S6), against the same coefficient estimated jointly with content features (Figure 4B).

Controlling for content substantially reduces the demographic associations, but does not eliminate most of them: among the 78 provider–demographic-category associations significant before adjustment, the median coefficient declines by 32.8%, 25 (32%) fall below the significance threshold, and three reverse sign.

Which associations survive differs sharply by category. Gender differences persist for all three providers, essentially unchanged in direction though attenuated in size (e.g., Claude’s preference for male-authored content falls from b=0.10b = 0.10 to 0.04). The ideological U-shape is most robust at its outer edges: left-leaning and center-right authors remain significantly over- and under-represented, respectively, for all three providers, and centrist authors for two of three, but the right-leaning category, previously significant for Claude and Gemini, is not significant for any provider once content is controlled, so the models’ aversion to right-leaning authors, unlike their preference for left-leaning ones, is substantially a content effect; predictive margins from these same models confirm the U-shape holds consistently where it survives (SI Appendix, Fig. S2). Race shows a comparable asymmetry: Asian over-representation remains significant for all three providers and barely attenuates, whereas Black under-representation, significant for all three beforehand, survives content controls only for Anthropic (Figure 4B).

Among the other attributes, the age extremes are the most robust: both the under-representation of 25–34-year-olds and the over-representation of authors 65 and older remain significant for all three providers, while the weaker middle categories, 35–44 and 55–64, mostly lose significance (and reverse sign at 55–64 for Claude and Gemini). Education and income associations are more fragile and provider-specific: the largest before-adjustment differences, the over-representation of authors with less than a high-school education (Claude, Gemini) and of the highest income bracket (Gemini), are among those that disappear, while narrower patterns such as high-school under-representation (Claude, Gemini) persist.

Observed topic, length, sentiment, and related features therefore account statistically for a meaningful share of the demographic differences generated under text-only ranking, but not for all of them. LLMs distinguish among forms of expression along dimensions that are socially patterned, while our measured features capture only part of that distinction. The remaining demographic associations may reflect other properties of language, combinations of signals not represented by our feature set, or information correlated with author characteristics that the models encode without explicitly reporting it.

Platform Signals Amplify Existing Skews, While Demographic Disclosure Adds Little

We next introduce information that is absent from the text-only condition, in two steps (Table S1; § 5). The platform condition adds author-level metadata (followers, following, tweet count) and post-level engagement (likes, retweets) on top of the post text; the demographics condition then adds each author’s verified survey attributes (gender, age, race, ideology, partisanship, education, income, marital status, religiosity) on top of platform. Models make substantial use of the newly visible engagement and account signals, particularly post-level measures, once these are supplied (Figure 5A): the average R2R^2 for post likes rises from 0.003 under text-only ranking to 0.023 when post metadata is visible, and retweet count rises from 0.001 to 0.008. Author metadata alone has a much smaller effect (follower-count R2R^2 rising only from 0.004 to 0.006), yet follower count becomes a considerably stronger predictor once post, not author, metadata is supplied (R2=0.012R^2 = 0.012): because post engagement is itself correlated with account size, models that favour high-engagement posts end up selecting disproportionately from high-follower authors even when follower counts are never shown to them. Providers also differ in how they combine these signals: Anthropic’s and Gemini’s use of engagement features changes little once author metadata is added on top of post metadata, but OpenAI’s drops markedly across nearly every engagement feature, for instance from R2=0.020R^2 = 0.020 to 0.009 for post likes and from 0.012 to 0.005 for follower count, suggesting GPT-4o Mini partly substitutes explicit author information for the engagement proxies it otherwise relies on.

Effects of platform context on feature predictability and recommended values

Figure 5. Effects of platform context on feature predictability and recommended values. (A) R2R^2 for 11 content and platform features across four context levels, by model and averaged across models. Text: no metadata; author: author metadata; post: post-engagement metadata; platform: author and post metadata combined. (B) Mean recommended values for three engagement-related features by context level. Dashed lines indicate means among shown-but-not-recommended posts.

LevelAuthor metadataPost metadataSurvey demographics
text
author✓
post✓
platform✓✓
demographics✓✓✓

Table S1. Context levels. Each column indicates which information block is included alongside the post text. Author and post are independent additions; platform combines both; demographics adds verified survey attributes on top of platform. Author metadata: followers, following, tweet count, likes given. Post metadata: like count, retweet count, retweet flag. Survey demographics: gender, age, race, ideology, partisanship, education, income, marital status, religiosity.

This growing reliance on engagement signals has a side effect on content itself. Overall content association with recommendation remains strong at every context level (Figure 5A), but its components shift: primary topic stays the strongest single predictor throughout (R2=0.039R^2 = 0.039–0.0450.045), and text length actually grows in importance (from R2=0.029R^2 = 0.029 under text-only ranking to 0.037 once platform metadata is added), while some finer-grained content signals shrink as engagement variables come to dominate: mentions and URL presence roughly halve in explanatory power (R2=0.010→0.006R^2 = 0.010 \to0.006 and 0.009 \to 0.005), and toxicity and sentiment polarity decline more modestly.

The composition of selected posts changes accordingly (Figure 5B). Even under text-only ranking, the posts selected by the models come disproportionately from authors with large audiences: recommended authors have a mean of roughly 57,000 followers, compared with 6,300 among shown-but-not-recommended posts. This association arises without follower counts being displayed to the model, reflecting correlations between platform standing and the text being selected. Once post engagement is made explicit, the gap widens further: the mean follower count of recommended authors rises to approximately 116,000, while mean post likes increase from 2.8 under text-only ranking to 7.8. These distributions are highly right-skewed (SI Appendix, Table S4), but the shift shows that engagement information gives models an additional basis for favouring content that has already accumulated attention.

The same shift toward engagement-driven ranking also reshapes whose content is recommended. Figure 6 plots each demographic category’s average skew under one context level against its skew under the next, one point per provider-averaged category, with points falling farther from the diagonal (in the direction a category already leaned) marking amplification. Moving from text to platform context generally pushes existing demographic skews farther in the same direction (Figure 6, left; per-category detail across all nine attributes is in SI Appendix, Fig. S4): groups already over-represented under text-only ranking tend to gain further visibility, while groups already under-represented tend to lose more. This pattern is not universal. Older authors, for example, lose part of their text-only advantage once platform metadata is supplied, showing that engagement signals can also counteract particular content-based skews.

Per-category demographic skew across successive context levels

Figure 6. Per-category demographic skew across successive context levels, one point per provider-averaged demographic category; shaded wedges mark amplification, where a category’s skew grows farther from zero in the same direction it already leaned. Left: text-only skew versus skew once platform metadata is added. Right: platform skew versus skew once demographics are also disclosed. Points on the diagonal indicate no change between the two context levels being compared.

Explicit demographic disclosure produces a markedly different result from platform metadata (Figure 6, right). Adding those nine survey attributes on top of the same platform metadata changes recommendations comparatively little relative to the platform condition alone: most demographic categories lie close to the y=xy = x line, so once text and platform signals are already available, explicitly naming the author’s demographic characteristics contributes comparatively little additional change to who is surfaced. But where that skew comes from in the first place depends sharply on the recommendation objective (SI Appendix, Fig. S5). Under controversial, skew is overwhelmingly a content effect: 72% of the total comes from text alone, and platform metadata adds comparatively little on top of it (15%). Engaging is close to the opposite case: content accounts for only 29%, and platform metadata does most of the work (58%), so an engagement-oriented recommender’s demographic skew mainly reflects amplification of already-visible engagement signals rather than the text it selects. Demographic disclosure itself is the smallest of the three sources for five of six objectives, the exception being informative, where it edges out platform metadata.

The context experiment therefore distinguishes two very different consequences of providing additional information. Platform metadata supplies signals that models readily use and that generally amplify or reshape inequalities already present under text-only ranking. Explicit demographic information adds little on top of those same platform signals. Demographic skew thus emerges well before protected attributes are disclosed: first when models rank socially patterned forms of expression, and then, in many cases, when engagement signals carry forward earlier inequalities in attention.

Discussion

The central finding is that LLMs can produce demographic inequality through the kinds of content they prefer. Even when given nothing but post text, the models do not rank different forms of expression neutrally: they favour particular topics, styles, and emotional registers, while passing over others. Those preferences have consequences for the visibility of different demographic groups. In our data, the content LLMs favour is disproportionately produced by some groups and less often by others, generating systematic differences in whose voices are surfaced before the model is ever told anything about an author’s identity. The precise beneficiaries vary across models and attributes, but the mechanism is consistent: LLM preferences over content become demographic differences in visibility. Platform metadata adds a second layer. Likes, reposts, and audience size give models access to inequalities already accumulated in platform attention and generally strengthen or restructure the skews visible under text-only ranking. Explicit demographic disclosure, by contrast, adds comparatively little once those signals are present. Demographic inequality in LLM-based curation can therefore begin with what the model considers worth seeing, rather than with what it knows about who produced it.

This changes how we should think about demographic fairness in LLM-based recommendation. A model need not “dislike” a demographic group in any direct sense to systematically disadvantage its content. It may instead favour forms of speech more commonly produced by other groups, or penalise forms of expression in which a group is disproportionately represented. In our experiment, for example, the strong LLM preference for news and social-concern content contributes to an age gradient in visibility; other content preferences coincide with differences by gender, ideology, race, and education, although their direction and robustness vary across providers. Withholding race, gender, age, or political identity therefore removes only one possible route to unequal treatment. The harder problem is that the model itself brings a hierarchy of content into the ranking task: some topics, styles, and registers are treated as more worthy of attention than others, and those judgments are not demographically neutral in their consequences. For LLM-based recommendation, fairness is consequently a question not only of what the model knows about an author, but of whose forms of expression it has learned to value.

Our results also resist an overly simple proxy explanation. Controlling for the content features we measure shrinks significant demographic coefficients by a median of 32.8%, and almost a third lose statistical significance, yet most remain. The contribution is therefore that demographic differences in visibility emerge from content-based ranking even when demographics are absent, and that measured content characteristics account for a substantial but incomplete part of this process. This parallels recent work using path-specific approaches to distinguish direct and indirect sources of disparity in LLM-based decision systems [25].

The addition of platform metadata shows why this distinction matters. Engagement signals carry the history of previous selections, audiences, and inequalities in visibility. Once made available, post-level engagement becomes a much stronger determinant of ranking, and existing demographic differences generally widen, although not universally. In this sense, LLM-based ranking can connect two familiar mechanisms of inequality: semantic judgments about what content appears valuable and popularity dynamics that reward what has already received attention. This is consistent with broader evidence that recommender systems can reinforce popularity advantages and feedback loops [9, 10, 13]. It also means that the removal of protected attributes and the balancing of downstream exposure address different parts of the problem. Exposure-aware re-ranking can correct disparities after they arise [26, 27]; restricting demographic information can constrain direct discrimination. Neither, by itself, reveals or alters the criteria through which particular forms of expression come to be valued in the first place.

The ranking objective itself provides another source of variation. Asking models to surface controversial material greatly strengthens both content selection and many of its demographic consequences, whereas popularity-oriented ranking generally dampens them. Some demographic associations, especially those involving age, even reverse direction between objectives. Providers likewise differ in the magnitude and occasionally the direction of their preferences. These differences show that the inequalities documented here depend on specific LLM design decisions. At the same time, the results offer little reason to view prompt engineering as a simple fairness intervention: no objective consistently removes demographic skew, and reducing one disparity may leave another unchanged or shift who benefits. What counts as “informative,” “popular,” “engaging,” or “controversial” is therefore more than a product-design choice. Once encoded in a ranking instruction and applied at scale, it becomes a criterion for distributing attention.

This is where LLM-based recommendation differs in an important way from many conventional ranking systems. Traditional recommenders already convert predicted relevance, popularity, and engagement into unequal exposure. LLMs add the capacity to judge meaning, tone, argument, style, and apparent social value directly [28, 29]. That flexibility is precisely what makes them attractive as curators: a platform can ask for content that is constructive rather than inflammatory, informative rather than entertaining, or thought-provoking rather than merely popular. But each of these objectives also defines a particular form of expression as more deserving of attention. When such judgments become part of the infrastructure of large-scale recommendation, qualitative preferences over speech acquire distributional consequences. The question of LLM fairness therefore cannot be separated from the question of what kinds of expression these systems are built to reward.

Limitations. Our design isolates the ranking component of recommendation rather than reproducing a deployed feed. Models choose 10 posts from fixed pools of 100 without personalization, session history, repeated exposure, or behavioural feedback. Real systems would combine ranking with retrieval, user modelling, repeated interaction, and feedback loops, any of which could reinforce or counteract the patterns documented here. The advantage of the simplified design is precisely that it identifies demographic skew before those additional dynamics enter the system.

The linked YouGov data provide verified demographic information that is rarely available for social media authors, but the sample is not representative of the broader Twitter/X population. It is older and more politically centrist than the platform population [30], and subgroup sizes are uneven (Table S2). Estimates for small groups should therefore be treated cautiously, particularly the Asian subgroup (N=17N = 17). The persistence of some associations across providers is evidence that they are not specific to a single model, but it does not make them precise population estimates of which demographic groups would gain or lose under LLM-based curation.

Category%Category%
Race / EthnicityGender
White78.7Female54.1
Black8.6Male45.9
Hispanic5.2Education
Mixed3.2<High school1.2
Asian1.7High school18.2
Other2.6Some college36.6
Age groupCollege26.9
18–241.6Postgraduate17.1
25–3415.7Income
35–4419.1<$30k | 20.4
45–5419.3$30–60k | 30.3
55–6426.3$60–100k | 23.8
65+18.0$100k+ | 20.6
PartisanshipPrefer not to say4.9
Democrat43.8Marital status
Independent28.7Married59.8
Republican22.4Single16.4
Other5.1Divorced14.0
IdeologyOther9.8
Left20.4Religiosity
Center-left20.7Not religious60.2
Center31.3Somewhat religious20.4
Center-right17.2Religious12.4
Right10.4Very religious7.0

Table S2. Author survey demographics (N=1,000N = 1{,}000, YouGov). Complete coverage across all nine attributes. Race/Ethnicity ‘Other’ includes Native American (1.1%) and Middle Eastern (0.1%).

Our content-adjustment analysis is likewise descriptive rather than a causal mediation design. Attenuation after controlling for measured features shows that those characteristics account statistically for part of the demographic association; it does not establish that they constitute the complete causal pathway. Unmeasured linguistic characteristics may remain. Nor can we rule out that information correlated with demographic identity is represented implicitly by the models. A supplementary text-only inference exercise (SI Appendix, Fig. S8) provides little evidence that providers can reliably report fine-grained author demographics from individual posts: Claude’s apparent accuracy is largely explained by majority-category prediction, while GPT-4o Mini and Gemini refuse to make most inferences. But explicit reporting is not equivalent to what information a model may encode or use during ranking. A stronger test would need to connect recoverable demographic information directly to selection behaviour.

Finally, the demographics condition represents explicit survey attributes inserted into the prompt, a cleaner experimental manipulation than most real-world uses of demographic information but also an artificial one. The small incremental effect we observe should therefore be interpreted as evidence about explicit disclosure in this setting, not as proof that demographic information could never affect ranking under other implementations. The study also covers three providers at one point in time. Model updates, different training procedures, alternative candidate pools, and future LLM architectures may change the specific content and demographic patterns we observe.

Conclusion

As LLMs move into the machinery that decides what people see, the question of demographic inequality shifts upstream. It is no longer enough to ask whether a system was given protected attributes or whether its outputs contain demographic disparities. We also need to ask what forms of expression the system consistently rewards, and what histories of attention are carried into the signals on which it ranks. Our experiment shows that demographic inequality can emerge at this earlier stage: from content alone, before protected attributes are supplied, and then be strengthened or reshaped when engagement metadata becomes visible. Explicit demographic disclosure, once those signals are present, adds little in comparison.

The practical consequence is that demographic blindness is a poor synonym for neutrality. Withholding protected attributes remains important for preventing direct discrimination, but it cannot guarantee equal visibility when language, content, and prior attention are already socially patterned. As LLMs increasingly become gatekeepers of attention, inequality need not arrive labelled with an identity. It can travel through what is said, how it is said, and the accumulated traces of who has already been heard.

Materials and Methods

Dataset. The YouGov-Twitter/X dataset links Twitter/X posts to verified survey demographics from a YouGov panel study [23]. Each user provided self-reported demographics via questionnaire; their Twitter/X accounts were subsequently matched, and a sample of their posts was collected, at regular intervals, via the Twitter API. After merging and anonymizing identifiers, the pool contains 10,000 posts from 1,000 unique authors. Each author has complete, verified ground-truth values for nine attributes (Table S2); full platform and NLP feature statistics are in SI Appendix, Table S3–Table S4.

FeatureValue
NLP features
Toxicity (0–1)0.063±0.1920.063 \pm0.192
Sentiment polarity (−1 to +1)0.18±0.440.18 \pm0.44
Sentiment subjectivity (0–1)0.21±0.200.21 \pm0.20
Text length (chars)92.1±51.692.1 \pm51.6
Avg word length (chars/word)5.07±1.505.07 \pm1.50
Has emoji4.2%
Has hashtag26.4%
Has mention54.4%
Has URL55.6%
Primary topic [36] (top 8)
News & social concern34.7%
Diaries & daily life15.8%
Sports8.2%
Business & entrepreneurs8.2%
Film, TV & video5.8%
Food & dining5.7%
Celebrity & pop culture4.5%
Music3.3%

Table S3. Post NLP feature statistics (10,000 posts). Numerical: mean ± std; binary: % of posts; topics: pool frequency (top 8 of 18 classes).

FeatureValue
Author metadata
Followers14,385 ± 319,147 (med. 133)
Following1,276 ± 4,237 (med. 370)
Tweet count14,128 ± 66,330 (med. 2,241)
Likes given13,269 ± 39,321 (med. 1,022)
Post engagement
Likes1.18 ± 15.43 (med. 0)
Retweets0.33 ± 8.70 (med. 0)
Is reply32.9%
Is quote tweet6.7%

Table S4. Platform feature statistics (10,000 post-level rows). Numerical: mean ± std (median); author metadata distributions are highly right-skewed. Post engagement is sparse: median likes and retweets are both 0.

Experimental design. We cross three factors orthogonally to obtain 3×6×5=903 \times6 \times5 = 90 conditions: (i) LLM provider: OpenAI GPT-4o Mini, Anthropic Claude Sonnet 4.5, Google Gemini 2.5 Flash; (ii) prompt style: six styles from [20], each varying only the recommendation-objective header while keeping the post list and task instructions identical: neutral (“rank these posts”), general (broad audience appeal), popular (predicted virality), engaging (interaction metrics), informative (educational value), controversial (debate generation); and (iii) context level: five levels of contextual information provided alongside the post text (ranging from text only to author metadata, post metadata, their combination, and verified survey demographics; see Table S1; the full prompt template is in SI Appendix). For each of the 90 conditions, we conduct 100 recommendation trials. To ensure temporal coherence, the 10,000-post pool is first sorted by posting date and partitioned into 100 equal-count temporal buckets of 100 posts each; each trial’s pool is the corresponding bucket (randomly reordered), so posts presented together span a similar time window. The same 100 trial pools are used identically across all 90 conditions, every provider, prompt style, and context level sees the same post set for a given trial ID, so the design is fully paired across conditions. The LLM returns the top 10, yielding 1,000 recommended posts per condition. Temperature is fixed at 0.3 following [20].

Skew measurement. For each condition we compare recommended posts against the shown-but-not-recommended posts from the same trial pools (disjoint groups). We use Cramér’s VV [31] for categorical features and Cohen’s dd [32] for numerical and binary features, both converted to R2R^2 for comparability (R2=V2R^2 = V^2; for Cohen’s dd, R2=r2R^2 = r^2 with r=d/d2+1/(p(1−p))r = d/\sqrt{d^2 + 1/(p(1-p))}, the unequal-nn point-biserial conversion, where pp is each condition’s recommended-vs-shown proportion). Directional skew is the signed difference between a group’s share among recommended posts and its share among the not-recommended posts: Δpi=prec,i−pnot−rec,i\Delta p_i = p_{\mathrm{rec},i} - p_{\mathrm{not-rec},i} (categorical) and Δμ=xˉrec−xˉnot−rec\Delta\mu= \bar{x}_{\mathrm{rec}} - \bar{x}_{\mathrm{not-rec}} (numerical). Significance is assessed via chi-square and Welch’s tt-test (α=0.05\alpha= 0.05).

Because each trial presents the model with a fixed choice set (10 selections out of the same 100 candidate posts), recommendation is not an independent binary outcome across posts: treating it as one, with only a marginal logistic regression and trial-clustered standard errors, would mismodel the choice structure and bias coefficients toward whatever happens to distinguish one trial’s 100-post pool from another’s, independent of the features being estimated. We instead fit a conditional (fixed-effects) logistic regression (exact conditional MLE, the standard estimator for a fixed number of successes per matched stratum, mathematically the same model used for matched case-control [33] and discrete-choice [34] designs), stratified by the exact trial pool shown to the model (provider ×\times prompt-style ×\times context-level ×\times trial ID). Conditioning on this stratum absorbs provider, prompt-style, and pool-composition differences directly into the likelihood, rather than requiring them to be modelled as separate fixed effects or corrected for after the fact via clustered standard errors. Formally, for stratum ss with candidate pool Ns\mathcal{N}_s (∣Ns∣=100)(|\mathcal{N}_s| = 100) and the k=10k = 10 posts actually selected, Ks⊂Ns\mathcal{K}_s \subset\mathcal{N}_s, the conditional likelihood is

Ls(β)=exp⁡(∑i∈KsxiTβ)∑S∈(Nsk)exp⁡(∑i∈SxiTβ)(1)L_s(\beta) = \frac{\exp\left(\sum_{i\in\mathcal{K}_s} x_i^{\mathsf{T}}\beta\right)}{\sum_{S\in\binom{\mathcal{N}_s}{k}}\exp\left(\sum_{i\in S} x_i^{\mathsf{T}}\beta\right)} \tag*{(1)}

where the denominator sums over all (10010)\binom{100}{10} possible 10-post subsets of the pool, and β\beta is estimated by maximising ∏sLs(β)\prod_s L_s(\beta) across all 9,000 strata (computed via a recursive algorithm, not literal enumeration, using statsmodels’ ConditionalLogit). We fit this specification separately for content, platform, and demographic feature groups, and, to test whether demographic associations are mediated by content, jointly with content on the same rows and strata; the exact specifications and the resulting attenuation test are presented alongside their results in § 2.2 and, in full detail, SI Appendix.

Data, Materials, and Software Availability

All analysis code is available at https://github.com/paganick/llm-recsys-twitter-yougov-PNAS and is permanently archived on Zenodo [35]. The underlying YouGov-Twitter/X pool linking posts to respondent identities cannot be shared publicly, as approved under New York University IRB protocol IRB-FY2026-11028; the repository includes a synthetic example dataset that reproduces the required input format.

Author Contributions

N.P., P.T., C.A.B., and C.B. designed research; N.P. and C.B. performed research and analyzed data; N.P. wrote the paper with input from all authors.

Competing Interests

The authors declare no competing interests.

Acknowledgments

We thank J. Nagler (New York University) for providing the YouGov survey data used in this study.

SI Appendix

Prompt Template

Each prompt begins with a style-specific instruction (one of six variants; see below), followed by the numbered post list and a fixed closing instruction. Metadata blocks are injected inline per post: the author block appears on the line immediately before the post text; the post-engagement and demographics blocks appear on indented lines immediately after.

Opening instruction (one of six prompt styles):

general: “Recommend posts that would be most interesting to a general audience.”

popular: “Recommend posts that would be most popular/viral with a general audience.”

engaging: “Recommend posts that would generate the most engagement (likes, shares, comments).”

informative: “Recommend posts that are most informative and educational for a general audience.”

controversial: “Recommend posts that are thought-provoking or would generate debate and discussion.”

neutral: “Rank these posts.”

Per-post metadata blocks (present only in the indicated context levels):

author, platform, demographics: [Author -- Followers: NN | Following: NN | Tweets: NN | Likes given: NN] (prepended on the line before the post text)

post, platform, demographics: [Posted: YYYY-MM-DD | Likes: NN | Retweets: NN | Retweeted: yes/no] (appended on the line after the post text)

demographics only: [Demographics -- Gender: XX | Age: XX | Race: XX | Ideology: XX | Partisanship: XX | Education: XX | Income: XX | Marital status: XX | Religiosity: XX] (appended after the post-engagement block)

Closing instruction (identical across all conditions):

Task: Rank these posts from most to least relevant. Return ONLY the top kk post numbers as a comma-separated list. Example format: 5,12,3,8,1,...

Pool size N=100N = 100; recommendations requested k=10k = 10; temperature = 0.3.

Context Levels

Author Demographics

The panel is predominantly White (78.7%), with Black (8.6%) and Hispanic (5.2%) the next largest groups and all other racial categories below 5%. Gender is roughly balanced with a slight female majority (54.1%). On politics, the panel’s two self-reported political measures paint the same picture (SI Appendix, Figure S7 shows they are closely correlated): Democrats (43.8%) outnumber Republicans (22.4%), and ideology likewise leans center-left (Left + Center-left = 41.1%; Center = 31.3%). This is a statement about internal coherence between partisanship and ideology within this panel, not a claim about how representative either measure is of the broader Twitter/X population (Limitations). The panel is predominantly older, with 44% of authors aged 55 or above (55–64 is the largest single age group at 26.3%); it is also well-educated, with over 80% having attended at least some college, and income is spread across brackets, with $30–60k the most common range (30.3%). Most authors are married (59.8%) and not religious (60.2%).

Demographic Category Correlations (Phi Coefficient)

Figure S7. Phi-coefficient (Pearson rr on binary indicators) between all 44 demographic category dummies (N=1,000N = 1{,}000 YouGov panel authors). Blue = positive co-occurrence; red = negative/mutual exclusion. The strongest off-diagonal clusters are the partisanship–ideology alignment (Democrat with Left/Center-left; Republican with Right/Center-right), older age co-varying with married and religious, and education co-varying with higher income.

Platform features. Both author metadata and post engagement records are drawn from the Twitter API, snapshotted at the time each post was collected. Author metadata profiles each user’s platform standing: follower count (audience reach), following count (reciprocal connectivity), cumulative tweet volume (activity level), and lifetime likes given (engagement propensity). Author-level distributions are highly right-skewed: despite a mean of 14,385 followers, the median is only 133, confirming a panel of mostly ordinary users with a small number of high-visibility outliers. Median tweet volume (2,241) and lifetime likes given (1,022) indicate that these are nonetheless active users of the platform. Post engagement captures how each post was received: like count, retweet count, and binary flags for reply and quote-tweet status. Engagement at the post level is extremely sparse: median likes and retweets are both 0, with means inflated by a few high-engagement posts.

Post NLP features. NLP features describe the semantic and stylistic properties of each post. Topic is classified into 18 categories using Cardiff NLP RoBERTa [36]; the pool is dominated by News & Social Concern (34.7%) and Diaries & Daily Life (15.8%), with Sports and Business & Entrepreneurs each at 8.2%. Toxicity (Detoxify [37]) and sentiment polarity and subjectivity (VADER [38]; subjectivity is a proxy, 1 − neutral score, since VADER does not natively output

one) characterise the affective and safety-relevant content dimensions; the pool is generally benign and mildly positive (μtox=0.063\mu_{\mathrm{tox}} = 0.063, μpol=+0.18\mu_{\mathrm{pol}} = +0.18, μsubj=0.21\mu_{\mathrm{subj}} = 0.21). Surface style indicators capture structural and formatting properties: text length, average word length (a proxy for vocabulary complexity and writing register), and binary flags for URL, mention, hashtag, and emoji presence.

Skew Computation Formulas

Recommended vs. not-recommended posts are disjoint groups drawn from the same trial pools. Cramér’s VV: V=χ2/(n⋅min⁡(r−1,c−1))V = \sqrt{\chi^2/(n \cdot\min(r - 1, c - 1))}. Cohen’s dd: d=(xˉrec−xˉnot−rec)/σpooledd = (\bar{x}_{\mathrm{rec}} - \bar{x}_{\mathrm{not-rec}})/\sigma_{\mathrm{pooled}}.

Conversion to R2R^2: R2=V2R^2 = V^2; for Cohen’s dd, R2=r2R^2 = r^2 with r=d/d2+1/(p(1−p))r = d/\sqrt{d^2 + 1/(p(1-p))} (unequal-nn point-biserial conversion; pp = recommended-vs-shown proportion in the condition). Directional skew: Δpi=prec,i−pnot−rec,i\Delta p_i = p_{\mathrm{rec},i} - p_{\mathrm{not-rec},i} (categorical); Δμ=xˉrec−xˉnot−rec\Delta\mu= \bar{x}_{\mathrm{rec}} - \bar{x}_{\mathrm{not-rec}} (numerical).

R2R^2 by Model

R2 by feature and prompt style — disaggregated by LLM (text context)

Figure S1. R2R^2 for each feature by LLM provider (averaged across prompt styles), text context. Complements Figure 3, and uses the same FDR-corrected significance family.

Figure S1 disaggregates the R2R^2 landscape (Results A) by provider. The ranking is broadly stable across models; the addition here is that demographic feature R2R^2 values stay similarly low across all three providers (≤0.007\le0.007), and Claude’s next-highest content feature after text length is sentiment polarity (0.0180.018).

Ideology Predictive Margins

Demographic Skew by Prompt Style

Figure S3 shows that the amplification pattern reported in the main text is general rather than confined to age and gender. Averaged across all nine attributes, mean ∣skew∣|\mathrm{skew}| is 1.0–1.4 percentage points under engaging, popular, and general, rising to 2.6 under informative and 3.1 under controversial, roughly a three-fold difference that holds attribute by attribute rather than being driven by one or two outliers. Ideology follows the same pattern already visible for age in the main text: centrist under-representation deepens from −2.3 and −2.7 points under popular and engaging to −7.8 and −7.9 under informative and controversial, while the right-leaning category flips from roughly neutral (−0.6 to +0.3 points) to clearly over-represented (+5.3 to +6.9 points) under those same two objectives, the same two objectives that make primary topic the strongest single predictor of selection. Gender shows the same pattern in even starker form. Averaged across providers, the male-author skew is close to zero under popular and engaging (−0.8 and +0.8 points) but reaches +11.5 points under controversial, the single largest attribute-level skew in the panel; this shift is partly convergence rather than uniform amplification, since GPT-4o Mini favours female-authored content under popular and engaging (−6.6 and −7.0 points) but switches to favouring male authors under informative and controversial (+1.7 and +7.3 points), converging in direction with Claude and Gemini, who favour male-authored content under every objective (+2.9 to +14.2 points). The objectives that lean hardest on post content are also the ones under which political moderates lose the most visibility.

Demographic directional bias by prompt style across nine survey-verified attributes

Figure S3. Directional demographic skew across nine survey-verified attributes by prompt style (averaged across providers), text context, all values in percentage points. Complements Figure 4A.

Predicted selection probability by author ideology

Figure S2. Predicted selection probability by author ideology (predictive margins from the single-block demographic-only conditional-logit model, text context). The dashed line is the no-effect baseline: the rate at which a post would be picked if ideology made no difference to selection at all. Points above are picked more often than that; points below, less. Shaded bands are 95% CIs.

Demographic Skew by Context Level

Figure S4 shows the text-to-platform-to-demographics pattern disaggregated by category. Ideology exhibits the strongest and most consistent amplification: left-leaning over-representation grows from +4.0 percentage points under text to +8.2 under demographics, and centrist under-representation deepens from −5.0 to −9.1 points, while the right-leaning category is a partial exception, declining from +2.1 to +1.0 points as context is added. Partisanship shows a similar but starker pattern: Republican-authored content moves from over-representation (+1.3 points under text) to under-representation (−2.5 under demographics), while Democrat over-representation grows from near zero to +5.3 points. Education and income move in step with each other: high-school under-representation deepens from −3.2 to −5.7 points and the highest income bracket’s over-representation grows from +3.7 to +6.8. Race, by contrast, is comparatively stable across all three context levels (e.g., Black under-representation moves only from −2.1 to −2.4 points), consistent with the near-diagonal clustering of racial categories in Figure 6.

Median demographic skew across demographic categories under text, platform, and demographics conditions

Figure S4. Median demographic skew, with IQR across models and prompt styles, under text, platform, and demographics conditions, disaggregated by category. Complements the aggregate text-vs-platform-vs-demographics comparison in Figure 6.

Where Demographic Skew Enters the Pipeline, by Prompt Style

Figure S5 decomposes the text-to-platform-to-demographics pattern of Figure 6 by prompt style, and shows that its “demographic disclosure adds little” finding is an average over two different regimes. Under controversial and informative, content selection alone already accounts for 72% and 60% of each style’s total stage-wise skew (panel C), leaving comparatively little for platform metadata and demographic disclosure to add: adding platform metadata displaces categories by only 0.7–0.8 points on average under these two objectives, against a text-only base of 2.7–3.4 points. Under engaging, popular, and neutral, by contrast, content alone accounts for only 29–34% of total skew, and platform metadata does most of the remaining work (45–58%), displacing categories by 1.6–2.3 points on top of a much smaller (1.1–1.7-point) text-only base. Across the six styles, this text-only base and the displacement from adding platform metadata are strongly negatively correlated (r=−0.81r = -0.81): objectives that already skew heavily from content alone leave less room for platform signals to add further skew, and vice versa.

Directional demographic skew decomposed into three additive pipeline stages, by prompt style

Figure S5. Directional demographic skew decomposed into three additive pipeline stages, by prompt style (Average pools all six styles). For each category in Figure 6, skew under demographics is decomposed as skew under text plus the increment from adding platform metadata plus the increment from adding demographics on top of platform; the three terms sum exactly to the final skew. (A) Distribution of ∣stage-wise skew delta∣|\text{stage-wise skew delta}| (percentage points) across categories. (B) Mean ∣skew∣|\text{skew}| accumulated so far at each stage (the level, not the delta), i.e., how much bias is present in total by that point in the pipeline. (C) Each stage’s share of a style’s total stage-wise skew mass.

Despite these different starting points, panel BB shows that total skew converges by the final stage. Informative and controversial, the two most content-skewed objectives at text-only, remain the two most skewed overall once demographics are disclosed (3.7 and 3.6 points), but general, engaging, neutral, and popular close most of the gap over the two subsequent stages, ending between 2.4 and 3.1 points. Informative in fact overtakes controversial by the final stage, despite trailing it through platform, because it continues to accumulate skew at each successive stage rather than plateauing after text.

Single-Block Conditional-Logit Results and Full-Joint Robustness Check

Conditional-logit models that account for the fixed 100-post choice sets (§ 5) show that the content preferences documented in the main text persist after the measured content characteristics are considered jointly. News & Social Concern remains the strongest positive topic predictor across all three providers (b=0.88b = 0.88–1.031.03), while text length is strongly positive (b=0.40b = 0.40–0.600.60).

Sentiment polarity is negative (b=−0.28b = -0.28 to −0.10-0.10), indicating a preference for more negatively valenced posts, and subjectivity is positive (b=0.12b = 0.12–0.280.28). Posts containing mentions are consistently penalised. The selection profile is therefore not reducible to a single feature or to simple correlations among topic, length, and affect: the models systematically distinguish among several dimensions of expression even when ranking the same candidate pools.

§ 2.2, in the main text, reports headline attenuation statistics from the demographic-plus-content specification (demographic and content features fitted jointly, both fit on the text-only condition so that neither model uses information the LLM did not observe; § 5; main-text Figure 4B). This section summarizes those statistics, then isolates platform’s own incremental contribution as a separate robustness check, using the full-joint specification (demographic, content, and platform features fitted jointly) even though platform metadata is never shown to the model in the text-only condition. Among the 78 provider-category combinations significant in the single-block model, the median attenuation once content alone is added is 32.8%, with 3 sign flips and 25 (32%) losing significance. Adding platform characteristics on top of an already content-adjusted model shrinks the remaining associations by only a further 1.1% at the median (among the 53 still significant after content controls alone; 0.8% across all 78), with zero further sign flips and zero further losses of significance: platform metadata adds essentially no further confounding beyond content, consistent with the content-based mechanism (topic, length, sentiment, and style) documented throughout this section. The race asymmetry noted in the main text is already present in the single-block model and is simply unchanged once content, and then platform, are added. Full per-category coefficients are in the supplementary demo_attenuation_after_content_controls.csv (single-block vs. demographic-plus-content) and demo_attenuation_platform_increment.csv (demographicplus-content vs. full-joint) files.

Demographic Correlations

Figure S7 shows the phi-coefficient (Pearson rr on binary indicators) between all 44 demographic category dummies (N=1,000N = 1{,}000 authors). The strongest off-diagonal positive correlations are between partisanship and ideology: Democrat co-occurs strongly with Left and Center-left, and Republican with Right and Center-right, confirming that the two attributes capture largely the same underlying political dimension in this panel. Older age groups (55–64, 65+) correlate positively with being married and with higher religiosity. Education and income are positively associated: College and Postgraduate categories co-occur with the $60–100k and $100k+ income brackets. Negative (red) off-diagonal entries largely reflect within-attribute mutual exclusion (e.g., Democrat vs. Republican, or different age groups).

Demographic Inference Accuracy

Referenced from Limitations: a supplementary test asked each provider to infer an author’s demographics from post text alone, independent of and run separately from the recommendation task (Figure S8). The reference column against which accuracy is judged is a harder, more meaningful bar than guessing uniformly at random among categories of very unequal size: the accuracy achievable by ignoring the post entirely and always naming an attribute’s single most common category (e.g., 79% White for race).

Demographic Inference from Post Text Alone

Figure S8. Demographic inference from post text alone. (A) Share of posts each provider gave a specific (non-refusal) guess for, by attribute. (B) Accuracy of that guess among answered posts, against a reference column showing the share of authors already in that attribute’s single most common category (the accuracy achievable by always naming that one category, ignoring the post). Cells built from fewer than 20% of posts are marked insufficient data rather than plotted. (C) For race, and Claude only (the only provider with usable data), recall broken down by each true category, showing that the aggregate 79% accuracy in Panel B reflects near-perfect recall on White authors (96%) and near-chance recall on every other group.

Panel AA shows why most of Panel BB has to be read with caution: Claude answers the large majority of posts for every attribute (75–99%), but GPT-4o Mini and Gemini refuse for 76–99% of posts depending on attribute. Cells in Panel BB built from fewer than 20% of posts are marked insufficient data rather than plotted, since an accuracy computed on that small, self-selected remainder (the posts a provider happened to feel confident enough to answer) is not a representative estimate.

Even where Panel BB is estimated on enough data, the accuracy gains over that reference column are modest to nonexistent. Claude’s race accuracy (79%) is nearly identical to the 79% already achievable by always naming the panel’s most common category, White. Panel CC confirms this directly by breaking that 79% down into recall for each true race category: Claude correctly identifies White authors 96% of the time, but Black authors only 17%, Hispanic 22%, and Mixed, Asian, and Other authors 0–3%. The aggregate accuracy is high only because the panel is dominated by White authors; the model shows essentially no ability to distinguish the other five categories from one another. Gender (69% vs. 54%) and ideology (35% vs. 31%) show a modest genuine edge over the reference column; accuracy is at or below it for the remaining six attributes. Overall, this test finds little evidence that any provider performs genuine fine-grained demographic inference from post text beyond exploiting category base rates.

Conditional-logit coefficients for NLP/content, platform records, and demographic features

Figure S6. Conditional-logit coefficients (bb) in the text-only condition, estimated by exact conditional MLE stratified by the trial pool shown to the model (§ 5). Left panels show NLP/content features (top) and platform records (bottom); the right panel shows demographic features. Platform and demographic variables in this figure are observed characteristics of the posts and authors but were not supplied to the LLM in the text-only condition. Features are zz-scored; bars show 95% CIs. Filled diamonds indicate p<0.05p < 0.05.

References

  1. [1]Andrew Hutchinson. X’s algorithm is shifting to a Grok-powered AI model, October 2025. URL https://www.socialmediatoday.com/news/x-formerly-twitter-switching-to-fully-ai-powered-grok-algorithm/803174/. Accessed January 6, 2026.
  2. [2]Andrew Hutchinson. X now algorithmically ranks posts in following feed, November 2025. URL https://www.socialmediatoday.com/news/ x-formerly-twitter-sorts-following-feed-algorithm-ai-grok/806617/. Accessed January 6, 2026.
  3. [3]Terrence O’Brien. Bluesky’s new app is an AI for customizing your feed. The Verge, 2026. URL https://www.theverge.com/ai-artificial-intelligence/903190/bluesky-attie-ai-custom-feeds.
  4. [4]Aisha Malik. Instagram’s new “your algorithm” tool gives you more control over the reels you see, December 2025. URL https://techcrunch.com/2025/12/10/instagrams-new-your-algorithm-tool-gives-you-more-control-over-the-reels-you-see/. Accessed January 30, 2026.
  5. [5]Akaash Kolluri, Renn Su, Farnaz Jahanbakhsh, Dora Zhao, Tiziano Piccardi, and Michael S. Bernstein. Alexandria: A Library of Pluralistic Values for Realtime Re-Ranking of Social Media Feeds, 2025. URL https://arxiv.org/abs/2505.10839.
  6. [6]Omar El Malki, Marianne Aubin Le Quéré, Andrés Monroy-Hernández, and Manoel Horta Ribeiro. BONSAI: Intentional and personalized social media feeds. arXiv preprint arXiv:2509.10776, 2025.
  7. [7]Germain Gauthier, Roland Hodler, Philine Widmer, and Ekaterina Zhuravskaya. The political effects of X’s feed algorithm. Nature, 2026. doi: 10.1038/s41586-026-10098-2. URL https://www.nature.com/articles/s41586-026-10098-2.DOI
  8. [8]Tiziano Piccardi, Martin Saveski, Chenyan Jia, Jeffrey T. Hancock, James S. Tsai, and Michael S. Bernstein. Reranking partisan animosity in algorithmic social media feeds alters affective polarization. Science, 390(6776), 2025.DOI
  9. [9]Himan Abdollahpour, Robin Burke, and Bamshad Mobasher. Managing popularity bias in recommender systems with personalized re-ranking. In Proceedings of the 32nd International FLAIRS Conference, pages 413–418, 2019.arxiv.org/abs/1901.07555
  10. [10]Allison JB Chaney, Brandon M Stewart, and Barbara E Engelhardt. How algorithmic confounding in recommendation systems increases homogeneity and decreases utility. In Proceedings of the 12th ACM conference on recommender systems, pages 224–232, 2018.DOI
  11. [11]Michael D. Ekstrand, Anubrata Das, Robin Burke, and Fernando Diaz. Fairness in information access systems. Found. Trends Inf. Retr., 16(1–2):1–177, July 2022. ISSN 1554-0669. doi: 10.1561/1500000079. URL https://doi.org/10.1561/1500000079.
  12. [12]Michael D Ekstrand, Mucun Tian, Mohammed R Imran Kazi, Hoda Mehrpouyan, and Daniel Kluver. Exploring author gender in book rating and recommendation. In Proceedings of the 12th ACM conference on recommender systems, pages 242–250, 2018.
  13. [13]Masoud Mansoury, Himan Abdollahpour, Mykola Pechenizkiy, Bamshad Mobasher, and Robin Burke. Feedback loop and bias amplification in recommender systems. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management (CIKM), 2020.arxiv.org/abs/2007.13019
  14. [14]Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T. Kalai. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems, 29, 2016.arxiv.org/abs/1607.06520
  15. [15]Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186, 2017.DOI
  16. [16]Isabel O Gallegos, Ryan A Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K Ahmed. Bias and fairness in large language models: A survey. Computational linguistics, 50(3):1097–1179, 2024.arxiv.org/abs/2309.00770
  17. [17]Yupeng Hou, Junjie Zhang, Zihan Lin, Hongyu Lu, Ruobing Xie, Julian McAuley, and Wayne Xin Zhao. Large language models are zero-shot rankers for recommender systems. In European Conference on Information Retrieval, pages 364–381. Springer, 2024.arxiv.org/abs/2305.08845
  18. [18]Jizhi Zhang, Keqin Bao, Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He. Is Chat-GPT fair for recommendation? evaluating fairness in large language model recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems, pages 993–999, 2023.
  19. [19]Yashar Deldjoo. Understanding biases in ChatGPT-based recommender systems: Provider fairness, temporal stability, and recency. ACM Transactions on Recommender Systems, 4 (2):1–35, 2025.arxiv.org/abs/2401.10545
  20. [20]Nicolò Pagan, Christopher Barrie, Chris Andrew Bail, and Petter Törnberg. LLMs as Content Curators: A Large-Scale Audit of Recommendation Bias Across Providers and Platforms. In Conference on Language Modeling (COLM), 2026. URL https://arxiv.org/abs/2604.15937. Preprint (under the title “Polarization by Default: Auditing Recommendation Bias in LLM-Based Content Curation”) available as arXiv:2604.15937.
  21. [21]Amir Ghasemian, Homa Hosseinmardi, Upasana Dutta, and Duncan J. Watts. Llm-assisted reranking to operationalize nuanced objectives in recommender systems, 2026. URL https://arxiv.org/abs/2606.02883.
  22. [22]Byunghwee Lee, Sangyeon Kim, Filippo Menczer, Yong-Yeol Ahn, Haewoon Kwak, and Jisun An. LLMs can infer political alignment from online conversations. arXiv preprint arXiv:2603.11253, 2026.arxiv.org/abs/2603.11253
  23. [23]Gregory Eady, Jonathan Nagler, Andy Guess, Jan Zilinsky, and Joshua A. Tucker. How many people live in political bubbles on social media? evidence from linked survey and twitter data. SAGE Open, 9(1), 2019. doi: 10.1177/2158244019832705. URL https://doi.org/10.1177/2158244019832705.
  24. [24]Yoav Benjamini and Yosef Hochberg. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57(1):289–300, 1995.DOI
  25. [25]Xiao Yu, Joon Sung Park, and Suhong Moon. PopResume: Causal fairness evaluation of LLM/VLM resume screeners with population-representative dataset. arXiv preprint arXiv:2603.22714, 2026.arxiv.org/abs/2603.22714
  26. [26]Gourab K Patro, Arpita Biswas, Niloy Ganguly, Krishna P Gummadi, and Abhijnan Chakraborty. Fairrec: Two-sided fairness for personalized recommendations in two-sided platforms. In Proceedings of the web conference 2020, pages 1194–1204, 2020.DOI
  27. [27]Mohammadmehdi Naghiaei, Hossein A Rahmani, and Yashar Deldjoo. Cpfair: Personalized consumer and producer fairness re-ranking for recommender systems. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 770–779, 2022.arxiv.org/abs/2204.08085
  28. [28]Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He. Bias and debias in recommender system: A survey and future directions. ACM Transactions on Information Systems, 41(3):1–39, 2023.arxiv.org/abs/2010.03240
  29. [29]Yifan Wang, Weizhi Ma, Min Zhang, Yiqun Liu, and Shaoping Ma. A survey on the fairness of recommender systems. ACM Transactions on Information Systems, 41(3):1–43, 2023.DOI
  30. [30]Stefan Wojcik and Adam Hughes. Sizing up twitter users. Technical report, Pew Research Center, April 2019. URL https://www.pewresearch.org/internet/2019/04/24/sizing-up-twitter-users/.
  31. [31]Harald Cramér. Mathematical methods of statistics, volume 9. Princeton university press, 1999.
  32. [32]Jacob Cohen. Statistical power analysis for the behavioral sciences. routledge, 2013.
  33. [33]Norman E. Breslow and Nicholas E. Day. Statistical Methods in Cancer Research, Volume I: The Analysis of Case-Control Studies. IARC Scientific Publications No. 32. International Agency for Research on Cancer, Lyon, France, 1980.pubmed.ncbi.nlm.nih.gov/7216345
  34. [34]Daniel McFadden. Conditional logit analysis of qualitative choice behavior. In Paul Zarembka, editor, Frontiers in Econometrics, pages 105–142. Academic Press, New York, 1974.
  35. [35]Nicolò Pagan. paganick/llm-recsys-twitter-yougov-PNAS: Release for PNAS submission, September 2026. URL https://doi.org/10.5281/zenodo.22898507. https://doi.org/10.5281/zenodo.22898507.
  36. [36]Dimosthenis Antypas, Asahi Ushio, Jose Camacho-Collados, Vitor Silva, Leonardo Neves, and Francesco Barbieri. Twitter topic classification. In Proceedings of the 29th International Conference on Computational Linguistics, pages 3386–3400, 2022.
  37. [37]Laura Hanu and Unitary team. Detoxify. https://github.com/unitaryai/detoxify, 2020.DOI
  38. [38]Clayton Hutto and Eric Gilbert. Vader: A parsimonious rule-based model for sentiment analysis of social media text. In Proceedings of the international AAAI conference on web and social media, volume 8, pages 216–225, 2014.

Paper details

vlnc:W_x84w9stv
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 8

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents