betaFeedback

WorksComputer Science

Speak vs. Know: Balancing Behavior and Knowledge Distributions of Large Language Models

Abstract

Recent work reports that the distribution verbalized directly in the output text aligns far better with human opinion distributions than the next-token log-probability distribution of LLMs. However, the verbalized distribution reflects what LLMs know about a group's opinions (knowledge distribution), while what they actually output at deployment is governed by their log-probability distribution (behavior distribution). We find that when prompting LLMs to predict the opinion of a specific group, the most likely answer under the verbalized distribution differs from that under the log-probability distribution for 37% of World Values Survey questions, averaged over three LLMs. Thus, LLMs' knowledge does not match their behavior. This matters for evaluation: a verbalized distribution can score well against a group while its most likely answer is one the model never outputs, so the score no longer describes the model. The alignment scores for these two distributions reflect only the model's behavior or its knowledge, separately. We therefore propose a learnable combination of the two distributions that follows the most likely answer of the behavior distribution while preserving the shape of the knowledge distribution, with a single hyperparameter trading off behavioral fidelity against distribution quality. Our experiments on three opinion survey datasets and six models show that the balanced distribution improves behavioral consistency over the verbalized distribution from 71.6% to 95.6% on average, while the total variation distance to the human opinion distributions rises only slightly from 28.8% to 33.8%, and it remains effective under temperature scaling and cross-dataset settings.

Introduction

Large language models (LLMs) are increasingly serving as an alternative to human opinion survey respondents: to conduct a pre-test of a survey before launching a large-scale human survey (Argyle et al., [2]), to simulate human behavior in agent-based social simulations (Park et al., 2023), and to evaluate how well models represent the opinions of demographic and national groups (Santurkar et al., 2023; Liu et al., 2026). In human opinion survey questions, no single answer is “correct”; what matters is whether a model’s output distribution matches a target group’s opinion distribution, a property called distributional alignment (Meister et al., 2025).

A model’s opinion distribution can be expressed in two ways: behavior distribution and knowledge distribution. The behavior distribution (e.g., log-probability distribution; (Santurkar et al., 2023) refers to the next-token log-probabilities over option letters, which determine a model’s output answer during inference due to greedy decoding. The knowledge distribution (e.g., verbalized distribution; (Meister et al., 2025)) refers to the distribution that the model provides directly as a percentage list in textual format. This percentage list reflects the model’s level of understanding of a group’s opinions. Regarding distributional alignment, the knowledge distribution is reported to be the clear winner (Meister et al., 2025): when simulating human opinion distribution, the shape of a model’s verbalized distribution consistently aligns more closely with the shape of human opinion distribution than the log-probability distribution, which tends to be overly concentrated on one or two options (Meister et al., 2025; Tian et al., 2023).

The verbalized distribution, however, also leads to some anomalous results. As shown in Figure 1, we rank the 66 countries and regions in the World Values Survey [7] by how closely OLMo-3.1-32B-Instruct’s verbalized distribution aligns with their opinion distributions. Mongolia is a low-resource language country, linguistically and culturally distant from the predominantly English, Western-centric data on which the model was trained. Nevertheless, it ranks second, achieving alignment comparable to that of the United States and other Western countries. In contrast, other linguistically distant countries, such as Myanmar and Egypt, occupy the lower end of the ranking. Because OLMo-3.1-32B-Instruct has seen virtually no Mongolian text (Soldaini et al., 2024), its alignment score with Mongolia is difficult to interpret. We do not rule out the possibility that the model’s opinions may actually align more closely

Figure 1. The alignment score depends on the shape of the target. Mongolia ranks second among the 66 countries and regions of the WVS in alignment with OLMo-3.1-32B-Instruct’s verbalized distribution, under no steering, because its opinion distribution is nearly uniform; ○ marks the alignment of each shown country and region’s opinion distribution with the uniform distribution.

with those of Mongolia, even though the model has barely seen any Mongolian corpora.

This uncertainty leads us to examine how the knowledge and behavior distributions differ. A high alignment score only indicates that the shapes of the verbalized and human opinion distributions are similar. It does not imply that the verbalized distribution reflects the most likely answer of the model. Because the verbalized distribution is generated as text, there is no factor linking it to the probability of the next token that actually determines the model’s answer. In contrast, the log-probability distribution is constructed in a way that inherently predicts that answer. Under greedy decoding, the model outputs the most likely answer from its log-probability distribution. Therefore, two distributions may disagree on the most likely answer. When they do disagree, a metric based purely on distribution distance will favor a distribution that would never be observed when sampling from the model. Thus, neither expression method alone can serve as a faithful simulator. In this study, we aim to construct a distribution that preserves the shape of the knowledge distribution while following the most likely answer of the behavior distribution.

Our study makes three major contributions. 1) We show that the two distributions capture different things: the verbalized distribution matches the shape of human opinion, while the log-probability distribution reflects what the model would answer (§2). 2) We introduce a learnable, behaviorally constrained mixing method to combine the knowledge and behavior distributions to obtain a balanced distribution (§3). 3) Finally, our experimental results show that the net utility obtained from the balanced distribution is higher than that from the knowledge distribution and the behavior distribution, and it outperforms temperature scaling settings § 5.

Motivation

We use the alignment score (Santurkar et al., 2023; Liu et al., 2026) to investigate what the alignment score measures on the WVS dataset. The alignment score is as follows:

A(DM,Dc;Q)=1∣Q∣∑q∈Q(1−WD⁡(DM(q),Dc(q))∣N∣−1)(1)\mathcal{A}(D_{\mathcal{M}}, D_c; Q) = \frac{1}{|Q|}\sum_{q\in Q}\left(1-\frac{\operatorname{WD}(D_{\mathcal{M}}(q),D_c(q))}{|\mathcal{N}|-1}\right) \tag*{(1)}

where QQ is the question set, and DM(q)D_{\mathcal{M}}(q), Dc(q)D_c(q) denote the opinion distributions of model M\mathcal{M} and country cc over the ∣N∣|\mathcal{N}| ordered Likert-scale options of qq. The factor ∣N∣−1|\mathcal{N}|-1 normalizes the Wasserstein distance (WD)2, so that the alignment score lies in [0,1][0,1], with 1 denoting a perfect match. For each country, we compute the alignment score between a model’s and the country’s opinion distribution; the score ranges from 0 to 1, with higher values indicating better alignment. Appendix § 9.§ 2 provides the prompts, and Appendix § 10 provides all numerical results of this section.

As shown in Table 1, the behavior distribution performs worse than the knowledge distribution under the persona steering setting. In particular, the average alignment scores for the knowledge distribution range from 0.843 to 0.875, whereas the alignment scores for the behavior distribution range only from 0.765 to 0.804. However, when the most likely answer of each distribution is compared with that of the human opinion distribution, the two are comparable in soft accuracy (AlKhamissi et al., [1]).3

ModelAlignment scoreSoft accuracy
Behav.Knowl.Behav.Knowl.
Qwen2.5-72B0.7650.8750.7990.809
Llama-3.1-70B0.8040.8650.8130.791
OLMo-3.1-32B0.7790.8430.7480.765

Table 1. The alignment score of the behavior (Behav.) and knowledge (Knowl.) distributions with the 66 WVS countries and regions under persona steering, and the soft accuracy of their most likely answers.

Instead, the advantage lies in the shape: the dispersion of the knowledge distribution is far closer to that of the human opinion distribution, whereas the behavior distribution is over-concentrated. As shown in Appendix § 10 (Table 9), the human opinion distribution on the WVS dataset is relatively flat (average entropy of 0.766), the knowledge distribution is even flatter (0.83 to 0.87), while the behavior distribution is much steeper (0.17 to 0.51).

ModelBehaviorKnowledge
Qwen2.5-72B-Instruct0.168 [0.161, 0.174]0.854 [0.848, 0.861]
Llama-3.1-70B-Instruct0.514 [0.503, 0.525]0.827 [0.822, 0.830]
OLMo-3.1-32B-Instruct0.415 [0.397, 0.424]0.871 [0.866, 0.875]
Human opinion0.766 [0.663, 0.885]

Table 9. The average normalized entropy of the behavior and knowledge distributions on the WVS dataset under no steering. The numbers in brackets are the minimum and maximum entropy over the 66 countries and regions.

The behavior distribution, however, determines the model’s actual output: under greedy decoding, the model outputs the most likely answer from its log-probability distribution. An ideal distribution should therefore reflect both the model’s output behavior and its knowledge.

The knowledge distribution does not meet the first requirement, that of behavioral fidelity: its most likely answer often differs from that of the behavior distribution. These two distributions share the same most likely answer for only 56.2%–68.1% of the questions (see Table 2).

ModelNo steeringPersona steering
Qwen2.5-72B68.1%64.9%
Llama-3.1-70B61.1%62.5%
OLMo-3.1-32B56.2%60.5%

Table 2. The behavioral consistency on the WVS: the percentage of questions on which the most likely answer of the knowledge distribution matches that of the behavior distribution.

Therefore, when an LLM is steered to simulate a target group’s opinions, as shown in Figure 2, our goal is to create a single distribution that follows the most likely answer of the behavior distribution while preserving the shape of the knowledge distribution.

Figure 2. For each survey question, the LLM exposes a behavior distribution (log-probability) whose most likely answer reflects what the model actually outputs, and a knowledge distribution (verbalized) whose shape is closer to the human opinion distribution. We learn a combination of the two that keeps the most likely answer of the behavior distribution while preserving the shape of the knowledge distribution, yielding the balanced distribution.

Method

Problem Setup

For a group gg and question qq with nn answer choices, let DgD_g be the group’s ground-truth answer distribution over the nn options. We consider the following two distribution expression methods as behavior and knowledge distributions:

  • Log-probability distribution DlpD_{lp}: the model’s normalized next-token probabilities over the option letters (e.g., “A”, “B”, …);

  • Verbalized distribution DvbD_{vb}: the model is

asked to verbalize the whole distribution in a JSON format of per-option percentages (e.g., {A:40%, B:35%, C:10%, D:15%});

As § 2 shows, DlpD_{lp} is behaviorally faithful but over-concentrated (low entropy), whereas DvbD_{vb} captures the shape of a group’s opinion but need not match the model’s behavior. Our goal is a single distribution D\mathcal{D} that balances the two types of distributions.

Training Objective

We construct the balanced distribution D\mathcal{D} as a learnable combination of the behavior distribution and the knowledge distribution. By adjusting its weights, we trade off between the shape of the distribution and the behavior fidelity. Then, we define the balanced distribution D\mathcal{D} and its two constraint terms: the knowledge alignment term Lknowledge\mathcal{L}_{\mathrm{knowledge}} and the behavior constraint term Lbehavior\mathcal{L}_{\mathrm{behavior}}. Finally, we integrate them into the training objective.

Balanced Distribution. By default, D\mathcal{D} uses a single weight to combine the log-probability and verbalized distributions:

D=wDlp+(1−w)Dvb,w∈[0,1],(2)\mathcal{D}=wD_{lp}+(1-w)D_{vb},\qquad w\in[0,1], \tag*{(2)}

where w=1w=1 recovers DlpD_{lp} and w=0w=0 recovers DvbD_{vb}, and the intermediate ww retains the shape of the verbalized distribution while pulling mass toward the model’s actual output. As both distributions are normalized and the weights are convex, D\mathcal{D} is automatically a valid distribution.

Knowledge Alignment Term. We quantify how much the balanced distribution differs in shape from the verbalized distribution DvbD_{vb}. Specifically, we adopt the Total Variation (TV) distance as our distributional distance measure, which compares the two distributions option by option:

Lknowledge(D,Dvb)=12∑k=1n∣D(k)−Dvb(k)∣.(3)\mathcal{L}_{\mathrm{knowledge}}(\mathcal{D},D_{vb})=\frac{1}{2}\sum_{k=1}^{n}\left|D^{(k)}-D_{vb}^{(k)}\right|. \tag*{(3)}

The values for Lknowledge(D,Dvb)\mathcal{L}_{\mathrm{knowledge}}(\mathcal{D},D_{vb}) range from 0 to 1 and are lower when the balanced distribution D\mathcal{D} is closer to the verbalized distribution DvbD_{vb}.

Behavior Constraint Term. We aim to make the most likely answer of the balanced distribution as close as possible to the most likely answer under the behavior distribution. Let t=arg⁡max⁡kDlp(k)t=\mathop{\arg\max}_{k}D_{lp}^{(k)} be the answer produced by the model at inference.

We require arg⁡max⁡kD(k)=t\mathop{\arg\max}_{k}D^{(k)}=t so that balancing never changes the model’s most likely answer, and we enforce it with a soft hinge penalty:

Lbehavior(D)=max⁡(0,max⁡k≠tD(k)−D(t)+m),(4)\mathcal{L}_{\mathrm{behavior}}(\mathcal{D})=\max\left(0,\max_{k\ne t}D^{(k)}-D^{(t)}+m\right), \tag*{(4)}

where m≥0m\geq0 is a margin. When tt leads all other options by at least mm, the penalty is zero; otherwise, the penalty increases linearly. Therefore, the larger mm is, the greater the lead required for the most likely answer. We set m=0m=0 throughout and vary only λ\lambda, so that the penalty is active exactly when tt is not the most likely answer of D\mathcal{D}. The effect of mm is analyzed in Appendix C.

In addition, we quantify the consistency between a distribution and the behavior distribution. To this end, we introduce Behavioral Consistency (BC), an evaluation metric that measures how often the mode of the knowledge distribution (or the balanced distribution) agrees with the mode of the behavior distribution.

Objective Function. We fit the weights of D\mathcal{D} by minimizing a single loss that combines the two terms:

L=Lknowledge(D,Dvb)⏟knowledge alignment+λLbehavior(D)⏟behavior constraint,(5)\mathcal{L}=\underbrace{\mathcal{L}_{\mathrm{knowledge}}(\mathcal{D},D_{vb})}_{\text{knowledge alignment}}+\lambda\underbrace{\mathcal{L}_{\mathrm{behavior}}(\mathcal{D})}_{\text{behavior constraint}}, \tag*{(5)}

where λ\lambda is a constraint strength used to control the relative weights of Lbehavior(D)\mathcal{L}_{\mathrm{behavior}}(\mathcal{D}). Eq. (5) is the Lagrangian relaxation (Geoffrion, [4]) of minimizing Lknowledge\mathcal{L}_{\mathrm{knowledge}} subject to Lbehavior≤ϵ\mathcal{L}_{\mathrm{behavior}}\leq\epsilon. Therefore, λ\lambda is the Lagrange multiplier, the shadow price of BC; adjusting λ\lambda controls the trade-off between the model’s knowledge and behavior.

Metrics

In economics, a rational agent facing a decision weighs the benefit of an option against its cost and chooses the option that maximizes the net utility, namely the benefit minus the cost (Mas-Colell et al., 1995). In this paper, we use net utility to describe the trade-off between knowledge and behavior:

NU=BC−TV.(6)\mathrm{NU}=\mathrm{BC}-\mathrm{TV}. \tag*{(6)}

Here, TV is measured against the human opinion distribution, unlike Lknowledge\mathcal{L}_{\mathrm{knowledge}} in Eq. (3), which is measured against DvbD_{vb}. We report BC and TV as percentages and NU as a decimal. For a balanced distribution, a higher BC indicates better alignment with the model’s behavior, while a lower TV indicates better alignment with the human opinion distribution. In this study, we consider BC and TV to be equally important, so their coefficients are both set to 1. Then, the net utility NU ranges from -1 to 1; a higher NU indicates that the distribution better satisfies the requirements of both the model’s behavior and the model’s knowledge.

Experimental Setup

Datasets. We evaluate the three opinion survey datasets used in Meister et al. (2025)’s work, which span demographic and national groups. OpinionQA (Santurkar et al., 2023) consists of 100 contentious PEW4 American Trends questions, spanning topics from science to politics and personal relationships, on which demographic subgroups frequently disagree; its ground truth is the answer distribution of six United States groups by party (Democrat and Republican), sex (Male and Female), and race (Black and White). NYT Book Opinions (Meister et al., 2025) captures non-political subjective preferences: 346 annotators give a 4-point Likert response to “how likely are you to read it?” for 235 books from the past two decades judged by The New York Times, and the annotators are grouped by party (Democrat and Republican) and sex (Male and Female). GlobalOpinionQA (Durmus et al., [3]) is a global opinion dataset, taking its 100 questions of highest cross-country disagreement from the WVS and the PEW Global Attitudes Survey, with each of the 26 countries forming a group. For every group and question pair, the ground truth is that group’s normalized answer distribution from the original survey.

Models. We obtain the original verbalized and log-probability distributions using three popular open-source LLMs: OLMo-3.1-32B-Instruct5 (Olmo et al., 2025), Qwen2.5-72B-Instruct6 (Qwen et al., 2025), and Llama-3.1-70B-Instruct7 (Grattafiori et al., [5]). These three LLMs come from three different developers. In addition to open-source LLMs, we also validate our method on three production-grade APIs that support outputting the next-token log-probability: GPT-4o (gpt-4o-2024-08-06; OpenAI et al., 2024), GPT-4.1 (gpt-4.1-2025-04-14; OpenAI, 2025), and DeepSeek-V4-Flash (deepseek-v4-flash; Xu et al., 2026).

Steering Methods. We fit the balanced distribution only under steering, where the model is actually asked to simulate a target group. Without steering, the model returns the same distribution for every group, so there is no group-specific knowledge or behavior to balance.

We focus on two types of steering conditions that genuinely require LLMs to simulate target groups: 1) persona steering, which instructs the model in the prompt to respond as a typical member of that group, and 2) few-shot steering, which includes five additional context questions from the same group in the prompt, each with the group’s actual opinion distribution to steer the model toward that group. We create the few-shot demonstrations following Meister et al. (2025). For the OpinionQA and NYT Book Opinions datasets, we keep the top five most similar questions based on the question similarity rankings in the benchmark. Each question is accompanied by the corresponding ground truth distribution from that group. For the GlobalOpinionQA dataset, as each country responds to only a small set of questions, we use five other questions answered by the same country as few-shot demonstrations. For the few countries that answered fewer than five questions, we use all of the other answered questions.

Training Settings. We learn the weights for the balanced distribution by minimizing the objective function of Eq. (5) using Adam (Kingma and Ba, [11]) in full-batch training for 1,000 training steps, with a learning rate of 5×10−25 \times10^{-2}. In Appendix § 12, we provide loss and weight change curves to clarify that 1,000 steps are sufficient. We assess generalization with grouped 5-fold cross-validation over questions: for each dataset, we split it along the question dimension so that questions do not overlap between the training and validation sets. This prevents data leakage. For each fold, we fit ww based on the 4 training folds and report the average results of NU, TV, and BC across the 5 folds.

Experimental Results

Behavioral Fidelity at Near-Zero Cost

Fitting this balanced distribution requires balancing the knowledge alignment term Lknowledge\mathcal{L}_{\mathrm{knowledge}} and the behavior constraint term Lbehavior\mathcal{L}_{\text{behavior}} in Eq. (5). As shown in Figure 3, sweeping λ\lambda traces a curve in the TV and BC space. At λ=0\lambda= 0 the behavior constraint is inactive and the balanced distribution recovers the verbalized distribution, so the left endpoint of the curve is the verbalized distribution itself, where BC is 71.6%.8 As λ\lambda increases, the balanced distribution incorporates the log-probability distribution, raising BC while TV increases only slightly.

Figure 3. Trade-offs between TV and BC under different λ\lambda. Average results across six models and three datasets, under persona steering.

We select a single λ∗\lambda^* for each model, shared across its three datasets: the λ\lambda that maximizes the model’s average net utility across the datasets. Averaged over the six models and three datasets, BC reaches 95.6% at this λ∗\lambda^*, while TV rises only slightly from 28.8% at λ=0\lambda= 0 to 33.8%. All results in this paper use this model-specific λ∗\lambda^*. The weights ww are trained under λ∗\lambda^* for each dataset. Moreover, all λ∗\lambda^* and weights ww are provided in Appendix H (Table 14 and Table 15).

Modelλ∗\lambda^*OpinionQAOpinionQANYT BooksNYT BooksGlobalOpinionQAGlobalOpinionQA
wlpw_{lp}wvbw_{vb}wlpw_{lp}wvbw_{vb}wlpw_{lp}wvbw_{vb}
Llama-3.1-70B180.700.300.430.570.580.42
Qwen2.5-72B370.410.590.350.650.360.64
OLMo-3.1-32B160.610.390.450.550.580.42
GPT-4o380.400.600.440.560.380.62
GPT-4.1160.240.760.200.800.230.77
DeepSeek-V4340.890.110.370.630.390.61

Table 14. The per-model λ∗\lambda^* (a single λ\lambda per model, shared across its three datasets, that maximizes the model's average net utility) and the corresponding mixing weights wlp,wvbw_{lp}, w_{vb} of Dlp+DvbD_{lp} + D_{vb} under persona steering.

Modelλ∗\lambda^*OpinionQA wlpw_{lp}OpinionQA wvbw_{vb}NYT Books wlpw_{lp}NYT Books wvbw_{vb}GlobalOpinionQA wlpw_{lp}GlobalOpinionQA wvbw_{vb}
Llama-3.1-70B200.540.460.310.690.840.16
Qwen2.5-72B300.310.690.220.780.360.64
OLMo-3.1-32B12.10.590.410.330.670.650.35
GPT-4o250.290.710.370.630.320.68
GPT-4.1200.180.820.210.790.300.70
DeepSeek-V4200.530.470.440.560.700.30

Table 15. The per-model λ∗\lambda^* (a single λ\lambda per model, shared across its three datasets, that maximizes the model's average net utility) and the corresponding mixing weights wlp,wvbw_{lp}, w_{vb} of Dlp+DvbD_{lp} + D_{vb} under few-shot steering.

Table 3 compares the net utility NU of the balanced distribution against each single distribution expression method under persona steering for Qwen2.5-72B-Instruct and GPT-4.1. The balanced distribution achieves the highest NU on all three datasets for both models, exceeding the strongest single method by 0.14 to 0.25. On other models, the balanced distribution also outperforms all baseline distribution expression methods when averaged over the three datasets.

MethodOpinionQANYT BooksGOQA
Qwen2.5-72B-Instruct (open-source)
LogProb0.47910.34890.5002
Verbalized0.43300.47940.4479
Ours0.63750.61960.6432
GPT-4.1 (closed-source)
LogProb0.44750.30080.5240
Verbalized0.54600.34440.5297
Ours0.69460.59580.6926

Table 3. The comparison of net utility NU of each distribution when prompting the models using persona steering across three datasets.

In addition, we experimented by replacing the distribution distance metric TV with the Wasserstein distance (Villani et al., 2009) and substituting persona steering with few-shot steering. The conclusions are consistent with those based on TV and persona steering. The full results are provided in Appendices E and F.

Robustness to Decoding Temperature

Meister et al. (2025) demonstrated that direct temperature scaling of the log-probability distribution (TS-LogProb; Guo et al., [6]), i.e., pi(T)∝pi1/Tp_i(T) \propto p_i^{1/T}, can change the sharpness of the model’s log-probability distribution. In particular, increasing the temperature can flatten the distribution. Because temperature scaling is strictly monotonic, it does not change the model’s most likely answer. We select the deployable temperature parameters of LLMs T∈{0,0.5,1,1.5,2}T \in\{0, 0.5, 1, 1.5, 2\}. 9 We replace the log-probability distribution with its temperature scaled version. Here, T=0T=0 corresponds to the one-hot encoding distribution, while T=1T=1 represents the identity mapping of the log-probability distribution. We then retrain all weights from scratch under temperature scaling settings.

Figure 4 plots the net utility NU of Qwen2.5-72B-Instruct and GPT-4.1 against the temperature TT on the OpinionQA dataset. Two findings stand out in particular. First, the net utility of the learned balanced distribution remains nearly flat as temperature varies, suggesting that the balanced distribution requires behavioral signals from the log-probability distribution rather than its shape. Second, the pure TS-LogProb is extremely sensitive to temperature: its net utility rises, averaged over the two panels, from 0.39 at T=0.0T=0.0 to 0.53 at T=2.0T=2.0, yet it never surpasses the balanced distribution at any temperature. As shown above, our method is insensitive to temperature scaling and outperforms temperature scaling.

Figure 4. The comparison of net utility between the balanced distribution and the temperature-scaled log-probability distribution at different temperatures. The shadows indicate the standard deviation of the 5-fold cross-validation.

The results for the other two datasets and the remaining models, which show the same pattern, are provided in Appendix § 15.

Distribution Shape: Between Too-Flat and Too-Sharp

To characterize the shape of the balanced distribution, we compute the normalized entropy of each distribution, which ranges from 0 (a sharp, one-hot shape) to 1 (a uniform shape). Table 4 reports the absolute difference from the human entropy averaged over the three datasets under persona steering. The log-probability distribution is severely over-concentrated (average entropy 0.32), whereas the verbalized distribution is too flat (0.85), even flatter than the human opinion distribution (0.75). The entropy of the balanced distribution almost perfectly matches that of the human opinion distribution (0.73 versus 0.75), although the objective never optimizes entropy. For every one of the six models, the balanced distribution is the expression method closest to human entropy; it retains the shape information of the verbalized distribution while avoiding the log-probability distribution’s tendency to concentrate on one or two options. Thus, the balanced distribution balances model behavior and knowledge: its shape approximates the human opinion distribution while reflecting the model’s actual behavioral choices.

ModelLogProbVerb.Ours
Llama-3.1-70B0.2860.0800.045
Qwen2.5-72B0.5540.1160.006
OLMo-3.1-32B0.3610.0810.062
GPT-4o0.4580.1130.022
GPT-4.10.6650.0760.006
DeepSeek-V4-F.0.2530.1030.008
Average0.4290.0950.025

Table 4. Absolute difference between each distribution’s normalized entropy and the human entropy (0.751), under persona steering, averaged over the three datasets (smaller means closer to the human opinion distribution). Ours is the closest to the human opinion distribution for every model. All open-source models are instruct models. F. indicates Flash.

Cross-Dataset Validation

To investigate whether the mixing weights ww in Eq. (2), which were trained on dataset AA, also generalize to another dataset BB, we conduct tests on the balanced distribution Dlp+DvbD_{lp}+D_{vb}. Because a single λ∗\lambda^{*} is shared across a model’s datasets, only the weights ww are dataset specific. For each source

dataset AA, we take the weights wAw_A fitted in-domain on AA and apply them to each target dataset BB; the resulting net utility is denoted as transferred net utility.

Figure 5 shows the results for Qwen2.5-72B-Instruct and GPT-4.1. For each target dataset, the transferred net utility converges to the in-domain net utility and exceeds that of the log-probability and verbalized distributions. Among the results for all models in Appendix § 16, these transferred weights ww are effective. The only exception is DeepSeek-V4-Flash on the OpinionQA dataset, where the original log-probability distribution is already close to optimal (NU = 0.66), leaving little space for improvement. Therefore, the effectiveness of the transferred weights ww enables generalization across different datasets.

Figure 5. The cross-dataset validation of the fitted weights ww at λ∗\lambda^{*} for Qwen2.5-72B-Instruct and GPT-4.1. The weights fitted on one dataset still outperform all baseline distributions on another dataset. Blue bars use the λ∗\lambda^{*} of each model; green bars use the average λ∗\lambda^{*} of all six models.

Moreover, we replace each model’s λ∗\lambda^{*} with the average λ∗\lambda^{*} of all six models. As shown by the green bars in Figure 5, the net utility remains essentially unchanged, indicating that a λ∗\lambda^{*} itself need not be tuned for each model.

Revisiting the Country Ranking

In this section, we investigate two questions: how the balanced distribution changes the country ranking on the WVS, and whether the alignment scores are supported by an external measure of the model’s cultural knowledge. The weights for the balanced distribution are learned under persona steering and averaged over the three datasets. These weights are then applied under no steering on the WVS. The full results are provided in Appendix § 17.

Country Ranking Changes. Table 5 lists the countries with the largest changes in ranking when

Largest FallsLargest Rises
CountryRankΔ\DeltaCountryRankΔ\Delta
Mongolia14.0↓\downarrow11.3Netherlands36.0↑\uparrow15.2
Mexico27.3↓\downarrow9.0Germany28.2↑\uparrow12.2
Thailand18.8↓\downarrow7.3New Zealand17.5↑\uparrow9.7
Serbia35.2↓\downarrow6.7Philippines45.0↑\uparrow8.0
Slovakia8.5↓\downarrow6.3Malaysia37.7↑\uparrow6.5

Table 5. The change in the country’s ranking among the 66 countries when the balanced distribution is substituted for the verbalized distribution; this change is calculated as the average of the six models. Rank is the country’s average ranking under the verbalized distribution, and Δ\Delta is the number of places the country falls (↓\downarrow) or rises (↑\uparrow).

the balanced distribution is substituted for the verbalized distribution. Among the 66 countries, Mongolia experiences the largest fall in ranking, dropping 11.3 places; under OLMo-3.1-32B-Instruct it falls from second place to tenth, dissolving the anomaly of § 2. As shown in Appendix § 17 (Table 16), this adjustment is selective rather than uniform. The alignment scores for English-speaking countries (e.g., the United States, Canada, and New Zealand) show minimal change (at most 0.012). The largest declines occur for Mongolia and Slovakia, whose scores drop by 0.034 and 0.026 (up to 0.041 for Qwen2.5-72B-Instruct).

CountryVerbalizedBalanced
United States0.9100.898 ↓\downarrow.012
Canada0.8940.890 ↓\downarrow.004
New Zealand0.8790.879 ↓\downarrow.000
Hong Kong0.8830.867 ↓\downarrow.016
Mongolia0.8820.848 ↓\downarrow.034
Slovakia0.8880.862 ↓\downarrow.026
Egypt0.7930.776 ↓\downarrow.017

Table 16. The alignment scores of the verbalized and balanced distributions for seven representative countries under no steering, averaged over the six models.

External Validation. If a balanced distribution can correct a certain artifact, its alignment score should reflect the model’s actual understanding of each country. We evaluate this on BLEnD (Myung et al., 2024), a benchmark dataset for general cultural knowledge that overlaps with 11 of the 66 WVS countries and regions. For each model, we

calculate the correlation between its BLEnD accuracy across countries on two tasks (multiple-choice questions (MCQ) and short-answer Q&A) and the country’s alignment score under each distribution expression method. See Appendix § 18 for the complete experimental setup. As shown in Table 6, in 7 out of 12 cases, the balanced distribution outperforms the log-probability and verbalized distributions (10 of 12 against the verbalized distribution alone). Therefore, a country’s alignment scores under the balanced distribution better reflect the model’s understanding of that country, which supports the ranking results of Table 5. It is worth noting that the log-probability distribution exhibits the highest Pearson’s rr in 4 cases, which supports our claim that the log-probability distribution reflects the behavior of the model itself.

ModelMCQShort-Answer
LogProbVerb.Bal.LogProbVerb.Bal.
Llama-3.1-70B.622.621.634.753.666.759
Qwen2.5-72B.648.596.667.718.731.791
OLMo-3.1-32B.781.738.787.863.796.864
GPT-4o.525.508.494.657.597.615
GPT-4.1.500.552.546.589.620.621
DeepSeek-V4-F..550.507.516.663.504.594

Table 6. The Pearson’s rr between a country’s BLEnD accuracy and its alignment score across the 11 overlapping countries, under the log-probability (LogProb), verbalized (Verb.), and balanced (Bal.) distributions. Bold indicates the highest Pearson’s rr of each model and task.

Related Work

Simulating human opinions with LLMs. A growing number of studies are using LLMs to replace human survey respondents, both for piloting research and for populating agent-based simulations. Argyle et al. [2] introduced “silicon samples” as a way to use LLMs as cost-effective proxies for diverse demographic groups. Santurkar et al. (2023) curated OpinionQA from the Pew American Trends Panel and showed that LLM responses systematically over-represent some demographics and under-represent others. Durmus et al. [3] extended this to non-Western opinions with GlobalOpinionQA, exposing similar misrepresentations. Later studies report that LLM responses are unstable and diverge from human response biases (Liu, 2024b,a), and that opinion probes are highly sensitive to how questions are asked ([8] (Liu and Chu 2025; Liu and Chu 2026)). Steering LLMs toward a target group with personas or in-context examples can help narrow these gaps, but cannot eliminate them entirely [9] (Oba et al. 2024; Liu et al. 2026).

Eliciting probabilistic outputs from LLMs.

There are two main methods for extracting distributions from LLMs: reading the next-token log-probabilities over the option tokens (Santurkar et al., 2023) and verbalizing probabilities directly in the LLM’s output (Tian et al., 2023; Meister et al., 2025). Tian et al. (2023) showed that the verbalized distribution is better calibrated than log-probability distributions on correctness tasks, and Kadavath et al. [10] found that models can assess the validity of their own answers. The closest phenomenon to ours is the generator-validator gap, where an LLM’s generated answer contradicts its own judgment of that answer (Li et al., 2024). We study this tension in a new setting, subjective opinion distributions, and quantify the most likely answer disagreement between the verbalized and log-probability distributions elicited for the same group and question.

Conclusion

In this paper, we distinguished what an LLM knows about a group’s opinions, captured by its verbalized distribution, from how it would actually behave at deployment, captured by its log-probability distribution. Because the two disagreed on the most likely answer for a large fraction of questions, a high verbalized alignment score did not describe what a practitioner observed once the model was sampled, and a high alignment score did not, by itself, indicate that the distribution reflected the model. Motivated by this, we proposed a learnable, behavior-constrained balanced distribution that preserves the shape of the knowledge distribution while anchoring its mode to the model’s behavior, controlled by a single hyperparameter. Across six models and three datasets, the balanced distribution consistently achieved higher net utility on average than the baseline distributions. We recommend that future work on LLM opinion simulation report behavioral consistency alongside distributional alignment.

Acknowledgments

This work was supported by JST BOOST, Grant Number JPMJBS2407.

More Settings

Alignment Score

Following Santurkar et al. (2023) and Liu et al. (2026), the alignment score between a model distribution D\mathcal{D} and a ground-truth distribution Dgt\mathcal{D}_{gt} over nn ordered options is as follows:

A(D,Dgt)=1−1n−1∑k=1n∣∑j≤kD(j)−∑j≤kDgt(j)∣(7)\mathcal{A}(\mathcal{D},\mathcal{D}_{gt})=1-\frac{1}{n-1}\sum_{k=1}^{n}\left|\sum_{j\leq k}\mathcal{D}^{(j)}-\sum_{j\leq k}\mathcal{D}_{gt}^{(j)}\right| \tag*{(7)}

which ranges from 0 to 1, with higher values indicating better alignment.

Prompts

All prompts follow the structure of Meister et al. (2025): a task instruction, five formatting examples, and the question block (the question text followed by lettered options). Under persona steering, the instruction asks the model to answer as the target group; the placeholder <group> below is instantiated as, e.g., “a group of ‘Democrat’” for demographic groups or “a typical respondent from Mongolia” for countries. Under no steering, the group phrase is simply omitted. The instructions of the two distribution expression methods are shown below.

Log-probability instruction, persona steering

I will provide a multiple choice response to
a series of questions from a random group to
help you understand the formatting of this task.
After the 5 examples, please simulate an answer
from <group> for the question asked. Please
only respond with a single multiple choice
answer, no extra spaces, characters, quotes or
text. Please only produce 1 character. Answers
with more than one characters will not be
accepted. First I will provide a few examples
of this task just to give you a sense of the
answer formatting, then you will perform the
task yourself.

Verbalized instruction, persona steering

I will provide a distribution over answer
choices on a series of questions from a random
group to help you understand the formatting
of this task. After the 5 examples, please
express the distribution of answers from
<group> for the question asked. Do not follow
the distribution expressed in the examples
as they are only to give you an example of
formatting, express the distribution based on
your understanding of how humans may respond.
Please only respond in the exact format of
a dictionary mapping answer choice letter
to probability, no extra spaces, characters,
quotes or text. Please only produce 1 sentence
in this format. Answers outside of this format
will not be accepted.

Under few-shot steering, the five formatting examples are replaced by up to five questions answered by the same group, each preceded by a context line that states the group’s true response distribution. The prompt ends with a reminder before the target question. The verbalized instruction is shown below; the log-probability variant differs only in the answer-format sentences, as above.

Verbalized instruction, few-shot steering

In this task you will receive information
on the distribution of responses from <group
members> to related questions. Given this
data, your task is to simulate an answer
to a new question from <group members>.
First, I will provide the distribution of
responses from <group members> to a series
of questions. Afterwards, I will provide
example responses to the question to help you
understand the formatting of this task. After
the examples, please express the distribution
of answers from <group> for the question
asked. Please only respond in the exact format
of a dictionary mapping answer choice letter
to probability, no extra spaces, characters,
quotes or text. Please only produce 1 sentence
in this format. Answers outside of this format
will not be accepted. For the new question,
there will be no distribution provided, this
is for you to estimate!

Context: Let the probability that <group>
responds to the following question with
<option 1> be <percent>%, <option 2>
be <percent>%, ... (one clause per answer
option, covering the group’s full response
distribution).

Your turn! Please answer this question
for <group members>. As a reminder, this group
is the exact same group as the group in the
previous examples. The previous examples are
used to provide an example of formatting and
to give you insight into how this group would
respond to such questions.

Each question block lists the question text and its lettered options, and the model completes the final line after “Answer:.” The expected answer formats of the two methods are illustrated below with a WVS question.

ModelBehaviorKnowledge
Qwen2.5-72B-Instruct0.7650.875
Llama-3.1-70B-Instruct0.8040.866
OLMo-3.1-32B-Instruct0.7790.843

Table 7. Alignment score under persona steering, averaged over the 66 WVS countries. The knowledge distribution is the strongest single expression method, consistent with Meister et al. (2025).

Question block and answer formats (WVS example)

Figure question. Question block and answer formats (WVS example)

A.3

Decoding Settings

The log-probability distribution is read from the first generated token: we request the top 20 token log-probabilities and limit the completion to one token. The verbalized distribution is generated greedily (T=0T=0); on the rare prompts where the output cannot be parsed as a distribution, it is re-sampled at increasing temperatures until a valid one is returned. For the API models, the log-probability distribution is queried at T=0T=0, except for DeepSeek-V4-Flash, which collapses to a one-hot distribution at T=0T=0 and is therefore queried at T=1T=1 to recover the raw next-token distribution.

Details of the Motivation Analysis

Tables 7 and 8 report the full per-model numbers behind the motivation analysis in § 2: the average alignment scores of the two distribution expression methods under persona steering and the alignment scores with Mongolia, Myanmar, and the United States under no steering (including a uniform-distribution baseline).

Section 2 also compares the shapes of the two distributions.

ModelMongoliaMyanmarUnited States
Qwen2.5-72B-Instruct0.8790.7830.905
Llama-3.1-70B-Instruct0.8750.7720.906
OLMo-3.1-32B-Instruct0.8880.7670.892
Uniform0.8630.7500.800

Table 8. Alignment score with Mongolia, Myanmar, and the United States under no steering, plus a uniform-distribution baseline.

Table 9 reports the normalized entropy behind that comparison: on the WVS, the human opinion distribution is relatively flat, the knowledge distribution is even flatter, and the behavior distribution is far more concentrated. The normalized entropy ranges from 0 for a one-hot distribution to 1 for a uniform one.

Effect of the Margin

The behavior constraint term of § 3.2 includes a margin mm, which sets how far the model’s most likely answer tt must lead the remaining options before the penalty vanishes. The main text fixes m=0m=0 and varies only λ\lambda. Here we justify that choice: mm and λ\lambda move the balanced distribution along the same trade-off.

We scan m∈{0,0.05,0.1,0.15,0.2}m \in\{0, 0.05, 0.1, 0.15, 0.2\} under two settings:

  • λ\lambda is held at the λ∗\lambda^* fitted at m=0m=0.

  • λ\lambda is re-optimized for every value of mm.

All numbers are averaged over the six models and the three datasets under persona steering.

Figure 6 shows the result. With λ\lambda held fixed, raising mm from 0 to 0.2 sharpens the balanced distribution (its normalized entropy falls from 0.699 to 0.501) and raises behavioral consistency from 95.6% to 99.2%. A larger margin, therefore, gains behavioral fidelity at the cost of distribution shape, which is exactly the trade-off that λ\lambda controls. Once λ\lambda is re-optimized at each mm, both curves return to their m=0m=0 values (at m=0.2m=0.2, entropy 0.698 and behavioral consistency 95.8%), and the fitted $\lambda^* absorbs the change instead, falling to 19% of its value at m=0m=0.

Figure 6. Effect of the margin mm on the normalized entropy (top) and the behavioral consistency (bottom) of the balanced distribution, averaged over the six models and three datasets under persona steering. Red: λ\lambda fixed at the value fitted at m=0m=0; blue: λ\lambda re-optimized at each mm. The shaded areas are the standard error across the six models.

The margin thus adds no expressive power that λ\lambda does not already provide: any shape reachable by increasing mm is also reachable by decreasing λ\lambda. We keep m=0m=0 so that a single hyperparameter governs the trade-off between distribution shape and behavioral consistency.

Convergence of the Weight Fitting

The weights of the balanced distribution are fitted with a fixed budget of 1,000 Adam steps and no early stopping, so the fit must converge within that budget for every λ\lambda of the sweep. Figure 7 instruments exactly that training loop and records the training loss of the objective function and the verbalized weight wvbw_{\mathrm{vb}} at every step for all 321 values of λ\lambda in the training grid and for three (model, dataset) pairs that cover the three datasets.

Figure 7. Training loss (left) and the fitted verbalized weight wvbw_{vb} (right) at every one of the 1,000 Adam steps, for all values of λ\lambda in the training grid (color: λ\lambda on a log scale) and three model and dataset pairs under persona steering. Both the loss and the weight are flat long before the step budget ends.

Both quantities flatten within the first few hundred steps. Across the 321 values of λ\lambda, the loss enters and stays within 10−410^{-4} of its final value at a median step of 111 (Qwen2.5-72B / OpinionQA), 76 (GPT-4.1 / NYT Books), and 681 (DeepSeek-V4-Flash / GlobalOpinionQA). The fitted weight is likewise frozen well before the budget ends: the change of wvbw_{\mathrm{vb}} over the entire second half of training, ∣wvb(500)−wvb(1000)∣\lvert w_{\mathrm{vb}}^{(500)}-w_{\mathrm{vb}}^{(1000)}\rvert, has a median of 8×10−48\times10^{-4} and a maximum of 2.2×10−22.2\times10^{-2} over all λ\lambda and all three pairs, and the corresponding loss change is at most 1.9×10−31.9\times10^{-3}. The only trajectories that keep moving after step 500 are those at λ≈0\lambda\approx0 (the darkest curves), where the behavior constraint is inactive and the loss decays asymptotically toward the unconstrained optimum; these are also the trajectories whose weight has already saturated. Training for 1,000 steps is therefore sufficient, and the reported results are insensitive to the exact step budget.

Net Utility for All Models

Table 10 and Table 11 report the net utility of every distribution expression method for all six models under persona and few-shot steering, complementing the two models shown in Table 3.

ModelDatasetLog-probVerbalizedOurs
Llama-3.1-70BOpinionQA0.63230.44200.6495
Llama-3.1-70BNYT Books0.46750.32540.5649
Llama-3.1-70BGOQA0.59800.46040.6222
Qwen2.5-72BOpinionQA0.47910.43300.6375
Qwen2.5-72BNYT Books0.34890.47940.6196
Qwen2.5-72BGOQA0.50020.44790.6432
OLMo-3.1-32BOpinionQA0.50330.32360.5387
OLMo-3.1-32BNYT Books0.40780.38090.5356
OLMo-3.1-32BGOQA0.46650.31710.5017
GPT-4oOpinionQA0.52620.45780.6668
GPT-4oNYT Books0.41920.43920.5882
GPT-4oGOQA0.54700.50170.6652
GPT-4.1OpinionQA0.44750.54600.6946
GPT-4.1NYT Books0.30080.34440.5958
GPT-4.1GOQA0.52400.52970.6926
DeepSeek-V4-FlashOpinionQA0.66490.36500.6425
DeepSeek-V4-FlashNYT Books0.42320.42410.5826
DeepSeek-V4-FlashGOQA0.62190.49440.6837

Table 10. Net utility NU=BC−TVNU = BC - TV of each distribution expression method under persona steering, for all six models.

ModelDatasetLog-probVerbalizedOurs
Llama-3.1-70BOpinionQA0.56310.39250.6304
NYT Books0.48550.34580.6870
GOQA0.61160.41050.6193
Qwen2.5-72BOpinionQA0.44920.43260.6401
NYT Books0.38050.32010.7096
GOQA0.53160.44040.6349
OLMo-3.1-32BOpinionQA0.49630.34490.5384
NYT Books0.42350.20330.6346
GOQA0.49730.26440.5357
GPT-4oOpinionQA0.50730.48690.6757
NYT Books0.49920.32210.6608
GOQA0.57990.53830.6953
GPT-4.1OpinionQA0.43050.56810.7072
NYT Books0.35170.38670.7077
GOQA0.54640.59930.7171
DeepSeek-V4-FlashOpinionQA0.57110.40330.6432
NYT Books0.49800.32700.6357
GOQA0.61170.46000.6500

Table 11. Net utility NU=BC−TVNU = BC - TV of each distribution expression method under few-shot steering, for all six models.

Net Utility under the Wasserstein Distance

The main text measures the distance between the model distribution and the ground-truth distribution with the total variation. Here we replace it with the normalized Wasserstein distance NWD=1−A\mathrm{NWD}=1-\mathcal{A}, where A\mathcal{A} is the alignment score of §A, i.e., the earth-mover distance over the ordinal answer options divided by n−1n-1. Unlike the total variation, this distance is ordinal: moving probability mass to a neighboring option is penalized less than moving it to a distant one. The net utility becomes NU=BC−NWD\mathrm{NU}=\mathrm{BC}-\mathrm{NWD}, the balanced distribution is refitted from scratch with the Wasserstein loss, and λ∗\lambda^* is re-selected per model under the same rule as in § 5.1. Because NWD≤TV\mathrm{NWD}\leq\mathrm{TV} for ordinal options, all values are larger than in Table 10 and Table 11; only the comparison between the three columns is meaningful.

Table 12 and Table 13 report the results. The conclusion of § 5.1 is unchanged. The balanced distribution obtains the highest net utility on 13 of the 18 (model, dataset) combinations under persona steering and on 16 of 18 under few-shot steering; in every remaining case, the winner is the log-probability distribution and the gap is at most 0.007, which is within the fold-to-fold spread of the cross-validation. Averaged over the three datasets, the balanced distribution is the best expression method for all six models under both steering methods, by 0.006 to 0.080 (persona) and 0.019 to 0.097 (few-shot). Compared with the total-variation results, the Wasserstein distance narrows the gap to the log-probability distribution, which is expected: the log-probability distribution concentrates its mass on a single option that is usually adjacent to the human mode, so an ordinal distance forgives its over-concentration far more than the total variation does.

ModelDatasetλ∗\lambda^*Log-probVerbalizedOurs
Llama-3.1-70BOpinionQA240.81580.52470.8207
Llama-3.1-70BNYT Books240.75420.43700.8181
Llama-3.1-70BGlobalOpinionQA240.83850.57190.8323
Qwen2.5-72BOpinionQA470.78000.57610.8321
Qwen2.5-72BNYT Books470.71740.40910.8280
Qwen2.5-72BGlobalOpinionQA470.81040.60620.8365
OLMo-3.1-32BOpinionQA420.80100.50930.7989
OLMo-3.1-32BNYT Books420.70910.30310.7824
OLMo-3.1-32BGlobalOpinionQA420.78840.46800.7954
GPT-4oOpinionQA280.77870.61940.8366
GPT-4oNYT Books280.74870.41340.7968
GPT-4oGlobalOpinionQA280.81690.67500.8506
GPT-4.1OpinionQA440.75710.69280.8499
GPT-4.1NYT Books440.69450.47780.8355
GPT-4.1GlobalOpinionQA440.81500.72040.8711
DeepSeek-V4-FlashOpinionQA310.81250.54020.8168
DeepSeek-V4-FlashNYT Books310.73670.43170.7865
DeepSeek-V4-FlashGlobalOpinionQA310.82750.59900.8318

Table 12. Net utility NU=BC−NWDNU = BC - NWD of each distribution expression method under persona steering, for all six models, when the total variation is replaced by the normalized Wasserstein distance NWD=1−ANWD = 1 - A. The balanced distribution is re-fitted with the Wasserstein loss and λ∗\lambda^* is re-selected per model under the same rule.

Modelλ∗\lambda^*OpinionQAOpinionQANYT BooksNYT BooksGlobalOpinionQAGlobalOpinionQA
wlpw_{lp}wvbw_{vb}wlpw_{lp}wvbw_{vb}wlpw_{lp}wvbw_{vb}
Llama-3.1-70B180.700.300.430.570.580.42
Qwen2.5-72B370.410.590.350.650.360.64
OLMo-3.1-32B160.610.390.450.550.580.42
GPT-4o380.400.600.440.560.380.62
GPT-4.1160.240.760.200.800.230.77
DeepSeek-V4340.890.110.370.630.390.61

Table 13. Net utility NU=BC−NWDNU = BC - NWD of each distribution expression method under few-shot steering, for all six models, when the total variation is replaced by the normalized Wasserstein distance NWD=1−ANWD = 1 - A. The balanced distribution is re-fitted with the Wasserstein loss and λ∗\lambda^* is re-selected per model under the same rule.

Temperature Scaling for All Models and Datasets

Figure 8 extends the temperature analysis of § 5.2 to all six models and all three datasets. The overall pattern holds: the balanced distribution is nearly flat in temperature, whereas the temperature-scaled log-probability distribution is highly sensitive. On a few open-weight pairs (e.g., Llama-3.1-70B-Instruct and DeepSeek-V4-Flash on OpinionQA), the temperature-scaled distribution catches up with the balanced distribution at the highest temperatures, but it does not surpass it by a meaningful margin, and on NYT Books, it remains clearly below throughout.

Figure 8. The comparison of net utility between the balanced distribution and the temperature-scaled log-probability distribution at different temperatures. The shadows indicate the standard deviation of the 5-fold cross-validation.

Cross-Dataset Transfer for All Models

Table 14 and Table 15 report the learned λ∗\lambda^{*} and weights. Figure 9 and Figure 10 extend the cross-dataset validation of § 5.4 to all six models under persona and few-shot steering. The transferred net utility converges to the in-domain net utility and surpasses all baseline distributions on nearly every target; the only exception is DeepSeek-V4-Flash on OpinionQA, where the log-probability distribution is already close to optimal. Moreover, replacing the per-model λ∗\lambda^{*} with the average λ∗\lambda^{*} of all six models (green bars) leaves the net utility essentially unchanged, indicating that λ∗\lambda^{*} itself need not be tuned for each model.

Figure 9. Cross-dataset transfer of the fitted weights ww (at the per-model λ∗\lambda^{*}) for all six models under persona steering. Blue bars use the λ∗\lambda^{*} of each model; green bars use the average λ∗\lambda^{*} of all six models.

Figure 10. Cross-dataset transfer of the fitted weights ww (at the per-model λ∗\lambda^{*}) for all six models under few-shot steering. Blue bars use the λ∗\lambda^{*} of each model; green bars use the average λ∗\lambda^{*} of all six models.

Country Ranking for All Models

§ 6 re-scores the 66 WVS countries and regions with the balanced distribution and reports the largest ranking changes. Here we report the complete picture: for every one of the six models, all 66 WVS countries and regions are re-scored, and we compare how they are ranked by the verbalized distribution and by the balanced distribution. The setting is that of § 6: the Dlp+Dvb\mathcal{D}_{lp}+\mathcal{D}_{vb} mixing weights are learned under persona steering at each model’s λ∗\lambda^{*} (Table 14), averaged over the three datasets, and then applied unchanged to the WVS no-steering distributions. Table 17 lists every country’s alignment score and its rank under both distributions.

CountryLlama VerbLlama OursQwen VerbQwen OursOLMo VerbOLMo OursGPT-4o VerbGPT-4o OursGPT-4.1 VerbGPT-4.1 OursDeepSeek VerbDeepSeek Ours
United States0.90610.906_{1}0.88930.889_{3}0.90510.905_{1}0.90210.902_{1}0.89210.892_{1}0.87530.875_{3}0.91710.917_{1}0.90310.903_{1}0.92710.927_{1}0.91810.918_{1}0.91110.911_{1}0.90310.903_{1}
Northern Ireland0.90220.902_{2}0.89910.899_{1}0.90220.902_{2}0.89620.896_{2}0.87680.876_{8}0.86350.863_{5}0.90820.908_{2}0.89320.893_{2}0.92120.921_{2}0.91220.912_{2}0.89950.899_{5}0.88920.889_{2}
Chile0.89040.890_{4}0.856150.856_{15}0.89940.899_{4}0.870110.870_{11}0.88730.887_{3}0.86170.861_{7}0.90830.908_{3}0.88170.881_{7}0.89670.896_{7}0.878110.878_{11}0.90820.908_{2}0.88640.886_{4}
Canada0.88470.884_{7}0.89020.890_{2}0.89930.899_{3}0.89230.892_{3}0.88450.884_{5}0.88010.880_{1}0.90240.902_{4}0.89230.892_{3}0.90630.906_{3}0.89740.897_{4}0.89290.892_{9}0.88830.888_{3}
Slovakia0.878110.878_{11}0.851190.851_{19}0.884120.884_{12}0.853200.853_{20}0.88170.881_{7}0.852150.852_{15}0.89650.896_{5}0.87590.875_{9}0.887120.887_{12}0.862200.862_{20}0.90040.900_{4}0.88060.880_{6}
Argentina0.88380.883_{8}0.857140.857_{14}0.877190.877_{19}0.867140.867_{14}0.875110.875_{11}0.855130.855_{13}0.89260.892_{6}0.866160.866_{16}0.90250.902_{5}0.88780.887_{8}0.89670.896_{7}0.874120.874_{12}
Cyprus0.89330.893_{3}0.861120.861_{12}0.885110.885_{11}0.864160.864_{16}0.865200.865_{20}0.827300.827_{30}0.89280.892_{8}0.873120.873_{12}0.890110.890_{11}0.876120.876_{12}0.89960.899_{6}0.87980.879_{8}
Australia0.88550.885_{5}0.88840.888_{4}0.88970.889_{7}0.89240.892_{4}0.869170.869_{17}0.859110.859_{11}0.89190.891_{9}0.88640.886_{4}0.90440.904_{4}0.89930.899_{3}0.882210.882_{21}0.87970.879_{7}
Singapore0.88560.885_{6}0.86590.865_{9}0.89350.893_{5}0.88180.881_{8}0.869150.869_{15}0.858120.858_{12}0.886120.886_{12}0.87680.876_{8}0.881180.881_{18}0.868190.868_{19}0.889100.889_{10}0.87790.877_{9}
Hong Kong0.878100.878_{10}0.851180.851_{18}0.89060.890_{6}0.868130.868_{13}0.88540.885_{4}0.87620.876_{2}0.883160.883_{16}0.872130.872_{13}0.878200.878_{20}0.860230.860_{23}0.885120.885_{12}0.872130.872_{13}
Great Britain0.88390.883_{9}0.88750.887_{5}0.88880.888_{8}0.88660.886_{6}0.866180.866_{18}0.86180.861_{8}0.891100.891_{10}0.88260.882_{6}0.90160.901_{6}0.89650.896_{5}0.878250.878_{25}0.875100.875_{10}
Ukraine0.871190.871_{19}0.842230.842_{23}0.873240.873_{24}0.843270.843_{27}0.876100.876_{10}0.844190.844_{19}0.888110.888_{11}0.860210.860_{21}0.881150.881_{15}0.861220.861_{22}0.90130.901_{3}0.88250.882_{5}
Uruguay0.868200.868_{20}0.864100.864_{10}0.884130.884_{13}0.869120.869_{12}0.872130.872_{13}0.841210.841_{21}0.89270.892_{7}0.873110.873_{11}0.89580.895_{8}0.878100.878_{10}0.881220.881_{22}0.870170.870_{17}
Mongolia0.875150.875_{15}0.821390.821_{39}0.879180.879_{18}0.838300.838_{30}0.88820.888_{2}0.860100.860_{10}0.884150.884_{15}0.853270.853_{27}0.871260.871_{26}0.845320.845_{32}0.89380.893_{8}0.872140.872_{14}
Czech Republic0.867220.867_{22}0.844210.844_{21}0.881160.881_{16}0.849230.849_{23}0.88360.883_{6}0.86260.862_{6}0.885130.885_{13}0.870140.870_{14}0.874230.874_{23}0.849290.849_{29}0.883150.883_{15}0.868200.868_{20}
Taiwan0.872160.872_{16}0.864110.864_{11}0.884140.884_{14}0.872100.872_{10}0.860240.860_{24}0.848160.848_{16}0.885140.885_{14}0.88250.882_{5}0.880190.880_{19}0.868180.868_{18}0.883140.883_{14}0.875110.875_{11}
Andorra0.875130.875_{13}0.87870.878_{7}0.88790.887_{9}0.88470.884_{7}0.858250.858_{25}0.846170.846_{17}0.882170.882_{17}0.868150.868_{15}0.89490.894_{9}0.88870.888_{7}0.875300.875_{30}0.868210.868_{21}
New Zealand0.877120.877_{12}0.88460.884_{6}0.883150.883_{15}0.88950.889_{5}0.865190.865_{19}0.86340.863_{4}0.881180.881_{18}0.874100.874_{10}0.894100.894_{10}0.89160.891_{6}0.871310.871_{31}0.871160.871_{16}
Russia0.866230.866_{23}0.834270.834_{27}0.867260.867_{26}0.835310.835_{31}0.87690.876_{9}0.841200.841_{20}0.880190.880_{19}0.861190.861_{19}0.873240.873_{24}0.850280.850_{28}0.888110.888_{11}0.868220.868_{22}
Thailand0.868210.868_{21}0.838250.838_{25}0.885100.885_{10}0.846240.846_{24}0.870140.870_{14}0.830280.830_{28}0.878210.878_{21}0.854250.854_{25}0.861310.861_{31}0.838370.838_{37}0.883160.883_{16}0.869180.869_{18}
Brazil0.872180.872_{18}0.842220.842_{22}0.874230.874_{23}0.856190.856_{19}0.863210.863_{21}0.831250.831_{25}0.879200.879_{20}0.858230.858_{23}0.885130.885_{13}0.869160.869_{16}0.882200.882_{20}0.859310.859_{31}
Greece0.875140.875_{14}0.853170.853_{17}0.863280.863_{28}0.852210.852_{21}0.854280.854_{28}0.831260.831_{26}0.877220.877_{22}0.865180.865_{18}0.881170.881_{17}0.869150.869_{15}0.879240.879_{24}0.869190.869_{19}
Venezuela0.865250.865_{25}0.824350.824_{35}0.857300.857_{30}0.840280.840_{28}0.862230.862_{23}0.828290.828_{29}0.875240.875_{24}0.849320.849_{32}0.874220.874_{22}0.861210.861_{21}0.884130.884_{13}0.861280.861_{28}
Kazakhstan0.859300.859_{30}0.836260.836_{26}0.875200.875_{20}0.843260.843_{26}0.869160.869_{16}0.836240.836_{24}0.873260.873_{26}0.860200.860_{20}0.857350.857_{35}0.836390.836_{39}0.882170.882_{17}0.871150.871_{15}
Puerto Rico0.866240.866_{24}0.853160.853_{16}0.866270.866_{27}0.860180.860_{18}0.844410.844_{41}0.806420.806_{42}0.872230.872_{23}0.854260.854_{26}0.885140.885_{14}0.873130.873_{13}0.875280.875_{28}0.858320.858_{32}
Romania0.872170.872_{17}0.841240.841_{24}0.853340.853_{34}0.844450.844_{45}0.845380.845_{38}0.808380.808_{38}0.870280.870_{28}0.853280.853_{28}0.874210.874_{21}0.868170.868_{17}0.882190.882_{19}0.862270.862_{27}
Mexico0.862270.862_{27}0.815480.815_{48}0.852350.852_{35}0.829350.829_{35}0.855270.855_{27}0.817360.817_{36}0.874250.874_{25}0.841390.841_{39}0.869270.869_{27}0.853250.853_{25}0.880230.880_{23}0.853350.853_{35}
Guatemala0.861280.861_{28}0.816470.816_{47}0.849380.849_{38}0.830340.830_{34}0.857260.857_{26}0.820330.820_{33}0.870300.870_{30}0.844360.844_{36}0.868280.868_{28}0.854240.854_{24}0.882180.882_{18}0.862260.862_{26}
Germany0.863260.863_{26}0.87180.871_{8}0.874210.874_{21}0.87790.877_{9}0.849340.849_{34}0.840230.840_{23}0.869310.869_{31}0.866170.866_{17}0.881160.881_{16}0.87890.878_{9}0.860410.860_{41}0.860300.860_{30}
Macao SAR0.853350.853_{35}0.831290.831_{29}0.880170.880_{17}0.851220.851_{22}0.873120.873_{12}0.86190.861_{9}0.863340.863_{34}0.844340.844_{34}0.852380.852_{38}0.832400.832_{40}0.869340.869_{34}0.866320.866_{32}
Ecuador0.857310.857_{31}0.824390.824_{39}0.855520.855_{52}0.834320.834_{32}0.852290.852_{29}0.823320.823_{32}0.827270.827_{27}0.852290.852_{29}0.860330.860_{33}0.846300.846_{30}0.876270.876_{27}0.858340.858_{34}
Japan0.860290.860_{29}0.850200.850_{20}0.874220.874_{22}0.863170.863_{17}0.852300.852_{30}0.840220.840_{22}0.863350.863_{35}0.852300.852_{30}0.864300.864_{30}0.852260.852_{26}0.859420.859_{42}0.852360.852_{36}
Morocco0.856320.856_{32}0.828380.828_{38}0.850370.850_{37}0.826400.826_{40}0.849350.849_{35}0.806400.806_{40}0.865320.865_{32}0.843380.843_{38}0.859340.859_{34}0.844330.844_{33}0.877260.877_{26}0.860290.860_{29}
South Korea0.852370.852_{37}0.823370.823_{37}0.868250.868_{25}0.839290.839_{29}0.862220.862_{22}0.854140.854_{14}0.861370.861_{37}0.844370.844_{37}0.848410.848_{41}0.830410.830_{41}0.868350.868_{35}0.852370.852_{37}
Kenya0.850390.850_{39}0.827310.827_{31}0.857310.857_{31}0.830330.830_{33}0.850320.850_{32}0.819340.819_{34}0.879290.879_{29}0.852310.852_{31}0.852390.852_{39}0.837380.837_{38}0.875290.875_{29}0.863250.863_{25}
Serbia0.850400.850_{40}0.812520.812_{52}0.844410.844_{41}0.819480.819_{48}0.849330.849_{33}0.818350.818_{35}0.864330.864_{33}0.839420.839_{42}0.861320.861_{32}0.843340.843_{34}0.870320.870_{32}0.850400.850_{40}
Colombia0.853360.853_{36}0.815490.815_{49}0.845390.845_{39}0.827380.827_{38}0.846370.846_{37}0.807390.807_{39}0.862360.862_{36}0.841400.841_{40}0.865290.865_{29}0.850270.850_{27}0.868370.868_{37}0.843460.843_{46}
Netherlands0.855330.855_{33}0.858130.858_{13}0.860290.860_{29}0.866150.866_{15}0.844400.844_{40}0.845180.845_{18}0.855410.855_{41}0.856220.856_{22}0.871250.871_{25}0.869140.869_{14}0.848480.848_{48}0.848410.848_{41}
Malaysia0.846430.846_{43}0.827320.827_{32}0.852360.852_{36}0.827370.827_{37}0.850310.850_{31}0.831270.831_{27}0.860380.860_{38}0.858220.858_{22}0.87450.874_{5}0.822450.822_{45}0.870330.870_{33}0.864420.864_{42}
Bolivia0.854340.854_{34}0.818440.818_{44}0.841440.841_{44}0.828360.828_{36}0.843420.843_{42}0.812370.812_{37}0.859390.859_{39}0.844450.844_{45}0.856360.856_{36}0.845310.845_{31}0.868360.868_{36}0.848420.848_{42}
Nicaragua0.845440.845_{44}0.808550.808_{55}0.844400.844_{40}0.823420.823_{42}0.848360.848_{36}0.806410.806_{41}0.857400.857_{40}0.830480.830_{48}0.856370.856_{37}0.840360.840_{36}0.864380.864_{38}0.847440.847_{44}
Turkey0.851380.851_{38}0.823380.823_{38}0.855330.855_{33}0.827390.827_{39}0.840430.840_{43}0.797480.797_{48}0.850440.850_{44}0.840410.840_{41}0.835460.835_{46}0.821460.821_{46}0.859430.859_{43}0.850390.850_{39}
Lebanon0.850410.850_{41}0.807560.807_{56}0.838460.838_{46}0.819460.819_{46}0.834440.834_{44}0.799460.799_{46}0.847450.847_{45}0.831470.831_{47}0.840420.840_{42}0.829430.829_{43}0.860400.860_{40}0.847450.847_{45}
Peru0.848420.848_{42}0.811530.811_{53}0.828510.828_{51}0.819450.819_{45}0.826480.826_{48}0.800450.800_{45}0.847460.847_{46}0.838430.838_{43}0.849400.849_{40}0.842350.842_{35}0.862390.862_{39}0.840480.840_{48}
Philippines0.827520.827_{52}0.826340.826_{34}0.843430.843_{43}0.825410.825_{41}0.845390.845_{39}0.825310.825_{31}0.852420.852_{42}0.848330.848_{33}0.825490.825_{49}0.812500.812_{50}0.854450.854_{45}0.858330.858_{33}
India0.830510.830_{51}0.818420.818_{42}0.843420.843_{42}0.809500.809_{50}0.832460.832_{46}0.786510.786_{51}0.851430.851_{43}0.835450.835_{45}0.827480.827_{48}0.808540.808_{54}0.855440.855_{44}0.850380.850_{38}
Zimbabwe0.838460.838_{46}0.827330.827_{33}0.837470.837_{47}0.819470.819_{47}0.820500.820_{50}0.787500.787_{50}0.846470.846_{47}0.833460.833_{46}0.838430.838_{43}0.830420.830_{42}0.849470.849_{47}0.841470.841_{47}
Iran0.840450.840_{45}0.832280.832_{28}0.836480.836_{48}0.817490.817_{49}0.830470.830_{47}0.791490.791_{49}0.845480.845_{48}0.836440.836_{44}0.831470.831_{47}0.819470.819_{47}0.852460.852_{46}0.848430.848_{43}
Nigeria0.832490.832_{49}0.817460.817_{46}0.833500.833_{50}0.808510.808_{51}0.820510.820_{51}0.780540.780_{54}0.840490.840_{49}0.823500.823_{50}0.825500.825_{50}0.813490.813_{49}0.846490.846_{49}0.838490.838_{49}
Armenia0.837470.837_{47}0.809540.809_{54}0.826520.826_{52}0.805530.805_{53}0.816540.816_{54}0.773560.773_{56}0.833520.833_{52}0.815570.815_{57}0.837440.837_{44}0.827440.827_{44}0.842500.842_{50}0.825580.825_{58}
Uzbekistan0.819550.819_{55}0.818410.818_{41}0.840450.840_{45}0.821430.821_{43}0.832450.832_{45}0.803440.803_{44}0.836500.836_{50}0.825490.825_{49}0.820540.820_{54}0.811520.811_{52}0.832570.832_{57}0.833530.833_{53}
China0.822540.822_{54}0.819400.819_{40}0.836490.836_{49}0.820440.820_{44}0.822490.822_{49}0.798470.798_{47}0.826570.826_{57}0.811580.811_{58}0.820530.820_{53}0.807550.807_{55}0.831580.831_{58}0.829550.829_{55}
Maldives0.830500.830_{50}0.817450.817_{45}0.819570.819_{57}0.801550.801_{55}0.805610.805_{61}0.767590.767_{59}0.833510.833_{51}0.823510.823_{51}0.820520.820_{52}0.811530.811_{53}0.840510.840_{51}0.834510.834_{51}
Tunisia0.835480.835_{48}0.795590.795_{59}0.814590.814_{59}0.798580.798_{58}0.808580.808_{58}0.767580.767_{58}0.825580.825_{58}0.805590.805_{59}0.823510.823_{51}0.813480.813_{48}0.835530.835_{53}0.819610.819_{61}
Indonesia0.819560.819_{56}0.818430.818_{43}0.823550.823_{55}0.807520.807_{52}0.815550.815_{55}0.783530.783_{53}0.830530.830_{53}0.815560.815_{56}0.813570.813_{57}0.806570.806_{57}0.834540.834_{54}0.829540.829_{54}
Tajikistan0.819580.819_{58}0.814510.814_{51}0.825530.825_{53}0.816560.816_{56}0.813560.813_{56}0.795520.795_{52}0.828550.828_{55}0.819520.819_{52}0.807590.807_{59}0.796590.796_{59}0.837520.837_{52}0.836500.836_{50}
Ethiopia0.825530.825_{53}0.815500.815_{50}0.824540.824_{54}0.802540.802_{54}0.809590.809_{59}0.763600.763_{60}0.826560.826_{56}0.818530.818_{53}0.817550.817_{55}0.808530.808_{53}0.834550.834_{55}0.825570.825_{57}
Kyrgyzstan0.819570.819_{57}0.805570.805_{57}0.819560.819_{56}0.799570.799_{57}0.809570.809_{57}0.769570.769_{57}0.829540.829_{54}0.816550.816_{55}0.816560.816_{56}0.806560.806_{56}0.834560.834_{56}0.828560.828_{56}
Iraq0.813610.813_{61}0.777650.777_{65}0.805610.805_{61}0.796610.796_{61}0.811600.811_{60}0.774550.774_{55}0.825590.825_{59}0.795620.795_{62}0.811580.811_{58}0.792600.792_{60}0.829590.829_{59}0.819600.819_{60}
Viet Nam0.791640.791_{64}0.798580.798_{58}0.816580.816_{58}0.796590.796_{59}0.819520.819_{52}0.806430.806_{43}0.821600.821_{60}0.817540.817_{54}0.796630.796_{63}0.782640.782_{64}0.822610.822_{61}0.833520.833_{52}
Pakistan0.814590.814_{59}0.793600.793_{60}0.813600.813_{60}0.781600.781_{60}0.806620.806_{62}0.757610.757_{61}0.821610.821_{61}0.800600.800_{60}0.803610.803_{61}0.787630.787_{63}0.826600.826_{60}0.820590.820_{59}
Jordan0.814600.814_{60}0.787610.787_{61}0.792620.792_{62}0.776620.776_{62}0.786620.786_{62}0.746630.746_{63}0.806620.806_{62}0.795610.795_{61}0.799620.799_{62}0.787620.787_{62}0.813620.813_{62}0.807630.807_{63}
Libya0.808630.808_{63}0.784630.784_{63}0.788650.788_{65}0.776630.776_{63}0.787630.787_{63}0.746640.746_{64}0.803630.803_{63}0.790640.790_{64}0.804600.804_{60}0.798580.798_{58}0.810630.810_{63}0.799640.799_{64}
Egypt0.812620.812_{62}0.784640.784_{64}0.789640.789_{64}0.774640.774_{64}0.772650.772_{65}0.736650.736_{65}0.796640.796_{64}0.785650.785_{65}0.794640.794_{64}0.788610.788_{61}0.798650.798_{65}0.787660.787_{66}
Bangladesh0.788650.788_{65}0.766620.766_{62}0.791630.791_{63}0.770650.770_{65}0.779640.779_{64}0.748620.748_{62}0.792650.792_{65}0.791630.791_{63}0.776650.776_{65}0.768650.768_{65}0.801640.801_{64}0.807620.807_{62}
Myanmar0.772660.772_{66}0.770660.770_{66}0.783660.783_{66}0.760660.760_{66}0.767660.767_{66}0.729660.729_{66}0.784660.784_{66}0.778660.778_{66}0.766660.766_{66}0.753660.753_{66}0.792660.792_{66}0.790650.790_{65}

Table 17. Alignment score and country ranking of the verbalized distribution (Verb) and the balanced distribution (Ours) on the 66 WVS countries and regions, under no steering. The subscript is the rank of the country among the 66 under that distribution (1 = highest alignment score). The balanced distribution uses the persona-steering weights of Table 14, averaged over the three datasets, applied unchanged to the WVS distributions. Countries are sorted by their average verbalized rank over the six models.

Table 16 summarizes the alignment score changes mentioned in § 6 for seven representative countries averaged over the six models. The adjustment is selective rather than uniform. English-speaking countries barely move (at most 0.012, and New Zealand is unchanged), and Hong Kong, as a partially English-speaking region, sees a slight decrease (0.016). The largest declines occur for Mongolia and Slovakia (0.034 and 0.026; up to 0.041 for Qwen2.5-72B-Instruct), while Egypt, whose verbalized score is already the lowest, drops by only about half as much as Mongolia. The size of the decrease therefore measures how much of a country’s alignment score rests on a shape that the model’s own behavior does not support.

The two rankings agree on the broad ordering (Spearman ρ\rho between 0.88 and 0.99), so anchoring the distribution to the model’s behavior does not overturn the country ranking; it corrects it locally. The corrections are systematic. Averaged over the six models, the country that falls the most is Mongolia (+11.3 ranks), followed by Mexico (+9.0), Thailand (+7.3), Serbia (+6.7), and Slovakia (+6.3); the largest single fall is Mongolia under Llama-3.1-70B-Instruct, from rank 15 to rank 39. The countries that rise the most are the Netherlands (−15.2), Germany (−12.2), New Zealand (−9.7), the Philippines (−8.0), and Malaysia (−6.5). Mongolia falls under all six models, from ranks 15, 18, 2, 15, 26, and 8 to ranks 39, 30, 10, 27, 32, and 14 for Llama-3.1-70B-Instruct, Qwen2.5-72B-Instruct, OLMo-3.1-32B-Instruct, GPT-4o, GPT-4.1, and DeepSeek-V4-Flash, respectively.

models, from ranks 15, 18, 2, 15, 26, and 8 to ranks 39, 30, 10, 27, 32, and 14 for Llama-3.1-70B-Instruct, Qwen2.5-72B-Instruct, OLMo-3.1-32B-Instruct, GPT-4o, GPT-4.1, and DeepSeek-V4-Flash, respectively. Under OLMo-3.1-32B-Instruct, in particular, the verbalized distribution places Mongolia second among all 66 countries, ahead of every English-speaking country except the United States, a position it loses once behavior is taken into account.

The countries at the two extremes of the ranking are essentially unaffected. The United States and Northern Ireland remain in the top two under every model, and Myanmar, Egypt, Libya, and Iraq stay at the bottom: where the verbalized alignment score is genuinely high or genuinely low, there is no flatness artifact to remove. The movement is concentrated in the middle of the ranking, exactly where a nearly uniform verbalized distribution can be rewarded for matching a flat human opinion distribution without predicting what the model would actually answer.

External Validation with a Cultural Knowledge Benchmark

§ 6 argues that anchoring the distribution to the model’s behavior corrects the country ranking rather than merely changing it. This section describes the external test of that claim and reports the full results.

Benchmark. BLEnD (Myung et al., 2024) measures everyday cultural knowledge (food, celebrations, education, work, and daily life) with questions written and answered by native annotators. Eleven of its countries and regions map onto the 66 WVS groups: the United States, Great Britain, China, Greece, Mexico, Iran, South Korea, Ethiopia, Indonesia (West Java), India (Assam), and Nigeria (Northern Nigeria).

Tasks and Scoring. We evaluate the six models of § 4 on the two English tasks. For multiple-choice QA, we sample 2,300 questions per country with a fixed seed, matching the per country scale of the original benchmark, and score the emitted option letter against the gold answer. For short-answer QA, we use all 500 questions per country and follow BLEnD’s answer-containment rule: a response counts as correct if it contains any annotator answer. The local-language short-answer split is excluded because BLEnD provides no local-language multiple-choice task, the United States and Great Britain have no local-language split, and only nine scorable countries would remain. Open-weight models are run with greedy decoding; GPT-4o and GPT-4.1 are queried through the Batch API.

Correlation. For every model and task, we compute the Pearson correlation between the 11 countries’ BLEnD accuracy and their WVS alignment scores under the log-probability, verbalized, and balanced distributions. The setting is that of §I: no steering, with the balanced distribution using the persona-steering weights of Table 14 averaged over the three datasets. Table 6 reports all 12 model and task pairs. The balanced distribution correlates better than the verbalized one in 10 of the 12 pairs (two-sided sign test, p=0.039p = 0.039). The conclusion does not depend on the choice of statistic: Kendall’s τ\tau also prefers the balanced distribution in 10 of 12 pairs (p=0.039p = 0.039), and Spearman’s ρ\rho in 9 of 12. The two exceptions are GPT-4o and GPT-4.1 on multiple-choice QA, the two models with the smallest transferred log-probability weight wlpw_{\mathrm{lp}} (0.41 and 0.23): their balanced distribution stays closest to their verbalized one and has the least room to move. The log-probability distribution attains the highest correlation in four of the pairs. As it is the model’s behavior itself, this corroborates that the gain of the balanced distribution comes from anchoring to the behavioral signal. Unlike the raw log-probability distribution, the balanced distribution preserves the distribution quality reported in the main results.

ModelDatasetλ∗\lambda^*Log-probVerbalizedOurs
Llama-3.1-70BOpinionQA590.83270.57340.8300
Llama-3.1-70BNYT Books590.66900.41810.6941
Llama-3.1-70BGlobalOpinionQA590.82690.61560.8241
Qwen2.5-72BOpinionQA180.78590.57720.8230
Qwen2.5-72BNYT Books180.66500.56880.7641
Qwen2.5-72BGlobalOpinionQA180.79910.61790.8354
OLMo-3.1-32BOpinionQA440.81320.50100.8103
OLMo-3.1-32BNYT Books440.66130.49120.6997
OLMo-3.1-32BGlobalOpinionQA440.77740.54040.7764
GPT-4oOpinionQA190.77690.58570.8189
GPT-4oNYT Books190.68330.53080.7389
GPT-4oGlobalOpinionQA190.79690.64460.8268
GPT-4.1OpinionQA340.76030.66740.8487
GPT-4.1NYT Books340.63250.44670.7375
GPT-4.1GlobalOpinionQA340.80900.66470.8544
DeepSeek-V4-FlashOpinionQA650.83310.50200.8262
DeepSeek-V4-FlashNYT Books650.65240.51720.7082
DeepSeek-V4-FlashGlobalOpinionQA650.82100.63390.8278

Table 18.

References

  1. [1]Badr AlKhamissi, Muhammad ElNokrashy, Mai Alkhamissi, and Mona Diab. 2024. Investigating cultural alignment of large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12404–12422.arxiv.org/abs/2402.13231
  2. [2]Lisa P Argyle, Ethan C Busby, Nancy Fulda, Joshua R Gubler, Christopher Rytting, and David Wingate. 2023. Out of one, many: Using language models to simulate human samples. Political Analysis, 31(3):337–351.DOI
  3. [3]Esin Durmus, Karina Nguyen, Thomas I Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, et al. 2023. Towards measuring the representation of subjective global opinions in language models. arXiv preprint arXiv:2306.16388.arxiv.org/abs/2306.16388
  4. [4]Arthur M Geoffrion. 2009. Lagrangean relaxation for integer programming. In Approaches to integer programming, pages 82–114. Springer.
  5. [5]Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.arxiv.org/abs/2407.21783
  6. [6]Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. 2017. On calibration of modern neural networks. In International conference on machine learning, pages 1321–1330. PMLR.arxiv.org/abs/1706.04599
  7. [7]Christian Haerpfer, Ronald Inglehart, Alejandro Moreno, Christian Welzel, Kseniya Kizilova, Jaime Diez-Medrano, Marta Lagos, Pippa Norris, Eduard Ponarin, and Bi Puranen. 2022. World values survey: Round seven – country-pooled datafile version 5.0. Madrid, Spain & Vienna, Austria: JD Systems Institute & WVSA Secretariat. Doi:10.14281/18241.24.
  8. [8]Rem Hida, Masahiro Kaneko, and Naoaki Okazaki. 2025. Social bias evaluation for large language models requires prompt variations. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 14507–14530.arxiv.org/abs/2407.03129
  9. [9]EunJeong Hwang, Bodhisattwa Majumder, and Niket Tandon. 2023. Aligning language models to user opinions. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5906–5919.arxiv.org/abs/2305.14929
  10. [10]Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, et al. 2022. Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221.arxiv.org/abs/2207.05221
  11. [11]Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.arxiv.org/abs/1412.6980

Paper details

vlnc:W_3347c5ht
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 8

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents