WorksComputer Science
A Unified Choice Model of News Selection: Experimental Evidence on Stance, Rank, and Engagement Cues
Abstract
Social media platforms often display visual engagement cues alongside ranked news articles, yet their influence on clicks remains poorly understood when considered jointly with ranking position and users' political preferences. We propose a unified discrete choice framework using a conditional multinomial logit model to jointly estimate and compare the influence of these factors on news selection. We evaluate the model using 1,480 choices from 370 human participants across four news topics in a controlled study with and without visual engagement cues. The framework allows us to assess the incremental contribution of each factor through nested-model likelihood-ratio tests. We find that including visual engagement cues significantly improves model fit across all topics after accounting for rank and stance. Engagement cues alone also predict clicks better than either of these features, although the full model achieves the highest held-out log-likelihood. Finally, we quantify via counterfactual analysis the impact of ranking position and displayed engagement counts on click probabilities. These findings can inform platform owners, researchers, and policymakers evaluating how platform design affects the selection of political news.
CCS Concepts
• Human-centered computing → Empirical studies in collaborative and social computing; • Information systems → Content ranking.
Keywords
news selection, click behavior, engagement cues, position bias, conditional multinomial logit, randomized experiment
ACM Reference Format:
Jacopo D’Ignazi, Andreas Kaltenbrunner, Michele Starnini, and Vicenç Gómez. 2027. A Unified Choice Model of News Selection: Experimental Evidence on Stance, Rank, and Engagement Cues. In Proceedings of The ACM Web Conference 2027 (TheWebConf ’27). ACM, New York, NY, USA, 5 pages.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).
TheWebConf ’27, Dublin, Ireland
© 2027 Copyright held by the owner/author(s).
Introduction
Social media platforms play a central role in political discourse and news consumption. Even small changes in displayed information can alter user behavior [15], while ranking algorithms and interface design shape news consumption [2, 10] and have been linked to the amplification of extreme political content, polarization, and misinformation [1, 3, 7, 12].
Experiments have shown how ranking position [8, 18], political preferences, and social cues [4, 5, 6] influence user choices. In particular, displaying likes and shares can outweigh partisan preferences, reducing selective exposure to chance levels [14]. However, the joint influence of these factors on political news selection remains poorly understood. For example, Messing and Westwood Messing and Westwood [14] compare endorsements and partisan sources without varying item position, while Epstein et al. Epstein et al. [6] examine reactions to individual items rather than choices among ranked alternatives. Closer to our approach, Hogg and Lerman Hogg and Lerman [11] model position, content, and social signals together, and Krummen et al. Krummen et al. [13] study position and popularity, but both focus on non-political content (science stories and music).
We propose a conditional multinomial logit model that jointly represents political stance, ranking position, and engagement cues as components of utility in a discrete choice setting. Such models have been applied to brand selection [9], product rankings [8], network formation [16], and cultural markets [13], with extensions for interpretable feature effects [17].
We illustrate the benefits of the proposed framework through an interventional study examining how visual engagement cues influence news selection. We use likelihood-ratio tests to assess the contribution of each feature to model fit and cross-validate to compare their predictive power. Our work builds on the experimental setup of D’Ignazi et al. [3]. We reformulate their clicking model as a special case of our conditional multinomial logit model and analyze data collected with and without visible engagement cues.
Our main contributions are (C1) a unified discrete choice model incorporating political stance, ranking position, and visual engagement cues; (C2) causal evidence that ranking position and the magnitude of displayed engagement counts affect article selection; and (C3) an evaluation of predictive performance and effect sizes, showing that engagement cues alone predict clicks better than either stance or ranking position. Code and data to reproduce the experiment and all results are available in an anonymized repository.1
A Discrete Choice Model for News Selection
The conditional multinomial logit (CML) model describes the selection of one alternative from a finite choice set through a utility function of observed features. We use it to model which news article a user clicks in a ranked list. For a user choosing from a set of news articles , the CML model specifies
where is the systematic utility of news article for user , parameterized by .
We specify utility using three groups of features: user–article ideological stances, ranking position, and visual engagement cues. Stances of a user and an article are denoted by , ranging from extreme left to extreme right. Let denote the ranking position of article , with corresponding to the highest position. Finally, visual engagement cues are captured by means of the engagement counts associated with article , which can be, for example, likes or shares.
The utility is a linear combination of features and their corresponding parameters:
Here, contains indicators for user–article stance combinations, the rank feature , and the visibility-dependent engagement feature . The parameter vector collects their corresponding free coefficients. The indicator is 1 for the experiment with visible engagement counts and 0 otherwise.
The stance parameters form a matrix , where captures the preference of users with stance for articles with stance . Since adding a constant to an entire row leaves choice probabilities unchanged, we set for every , leaving 20 free parameters.
The rank feature has coefficient . A positive favors articles higher in the ranking. For otherwise identical articles, an article one position higher is times as likely to be clicked.
For engagement, we use the feature , with coefficient . The logarithm reduces the scale of the typically heavy-tailed engagement counts. Positive values of indicate a preference for articles with higher displayed counts, while removes this effect when counts are hidden. Note that setting yields a reduced model with only stance and position effects, nested within the full model and including the clicking model of D’Ignazi et al. [3] as a special case.
To estimate the model parameters from a dataset of observations , we maximize the log-likelihood
Because utility is linear in the parameters, the log-likelihood is concave. Therefore, any finite local maximum is global. We fit a common parameter vector across all topics and separate parameter vectors for each topic.
Experimental setup
Our experiment investigates whether displaying visual engagement cues in ranked news lists influences the selection of news articles. We conducted two experiments, one without visible engagement cues (E1, March 2026 with 166 participants) and one displaying likes and shares alongside news articles (E2, September 2026 with 204 participants). Each participant completed four tasks, one on each of the following topics: climate change, immigration, gender policies, and vaccination. The experimental platform, task design, and news selection build on D’Ignazi et al. [3], whose implementation and data are publicly available.2
User Recruitment. We recruited participants from the US through Prolific, requiring an approval rate above 90%. We used quotas to balance self-reported political affiliation (Republican, Democratic, or Independent) and gender (male or female). No quotas were imposed on other demographic characteristics. The age distribution of participants was centered around 40–50 years.
To mitigate the participation of bots, we included an attention check which asked the users to retrieve a number from an external web page.3 Participants were then asked to write that number on a second page, and those who failed to write one in the expected range were immediately excluded from the experiment. Approximately 80% of recruited participants passed this check. Participants who passed and accepted the study conditions completed the four tasks within 40 minutes and received approximately £1.5 (equivalent to £9 per hour based on the median completion time).
News Selection. For each of the four topics, we selected 10 articles through Ground News4, two from each of the five ideological categories defined in § 2. To reduce the variation associated with differences in the events covered, we selected articles from different news outlets reporting on the same event.
Platform Design. We implemented the experiment using oTree, with an interface resembling a social media news feed. In each task, participants first read a brief description of the topic and reported their stance on a five-point scale, from strongly left to strongly right. They then selected one article from a scrollable ranked list of 10 articles. The order of the four topics varied across participants.
Article headlines and coarse stance labels (left, center, or right) were always visible, while the news source and the distinction between moderate and extreme stances were hidden. In E2, each article also displayed its number of likes and shares (Figure 1). These counts were hidden in E1.

Figure 1. Example of a randomized ranked news list users see during one task of our experiment (E2). Engagement metrics are shown at the bottom right of each box. Users can scroll to see all 10 news items before deciding which one to click.
For each participant and topic, article positions were randomized. In E2, pairs of like and share counts were also randomly assigned to articles, independently of their ranking positions. These values were derived from the engagement recorded in [3] and rescaled exponentially to make differences between low and high counts more apparent to participants. The smallest values were on the order of some units and the largest of .
Results
We first assess the stability of parameter estimates, followed by likelihood-ratio tests to evaluate the contribution of each model component. We then compare predictive performance through cross-validation and, finally, quantify the effects of ranking position and displayed engagement counts on click probabilities.
Stability Test. To assess the stability of parameter estimates as the dataset size increases, we first fit the model to all E2 data to obtain reference parameters . We consider dataset sizes in the range from 10 to 816 clicks. For each size, we generate 1,000 synthetic datasets and re-estimate the model parameters. We then compute the relative absolute differences of the estimates from the reference values, and report the mean and standard deviation of such differences. For stance, we convert the coefficients into normalized preference probabilities by applying a softmax over article stances for each user stance. This accounts for the fact that stance utilities are defined up to an additive constant. For rank and engagement, we compute relative absolute differences directly from and .
These estimation errors decrease rapidly as dataset size increases. For datasets of 200 clicks, the relative absolute differences are approximately 20–40% for stance parameters and 10–20% for rank and engagement parameters. With the largest dataset, these values decrease to 10–20% and 5–10%, respectively. Across all dataset sizes, shows slightly larger deviations than . The estimates show little further improvement beyond approximately 600 clicks.
Ablation Analysis. We assess the contribution of each model component using a balanced pooled dataset combining E1 and E2 (selecting the first 166 users of E2 in chronological order), for a total of 1,328 clicks with the same news articles in both conditions. For the overall dataset and each topic, we compare the full model with three nested models, each omitting one component: stance, rank, or engagement. We use likelihood-ratio tests, with 20 degrees of freedom for stance and one for rank and engagement.
| Omitted Component | Overall | Gender | Vaccines | Immigration | Climate |
| Stance | 0.041 | 0.127 | |||
| Rank | 0.127 | ||||
| Engagement (Likes) |
Table 1. Likelihood-ratio tests comparing the full model with the model that omits one component (-values Holm-adjusted across all 15 tests). Omitting any of the components is significant in the pooled data, and engagement is significant on every topic. Results on stance and rank are mixed.
Table 1 reports the resulting -values, adjusted for multiple comparisons using Holm’s method.
All three components significantly improve model fit across the full dataset. Engagement is significant for every topic, with in all cases, while stance and rank show more variation across topics. In particular, stance is not significant for immigration (), nor is rank for climate change ().
The engagement component differs from the others because it enters the utility only when counts are visible (). Thus, E1 contributes to estimating the common stance and rank parameters, while is identified through the displayed counts in E2.
Since ranking positions and engagement counts were randomized independently, both effects admit a causal interpretation. Articles appearing higher in the ranking are more likely to be selected, as are articles displaying higher engagement counts in E2.
Predictivity analysis. The previous likelihood-ratio tests assess whether each feature significantly improves model fit.
We now examine how well different combinations of features predict unseen clicks using cross-validation. We evaluate each model specification on E2 data using 10 repetitions of 10-fold cross-validation. For each repetition, we sum the out-of-sample log-likelihood across the 10 folds. Figure 2 reports the mean and standard deviation across repetitions.
Figure 2. Mean and standard deviation of out-of-sample log-likelihood over 10 repetitions of 10-fold cross-validation on E2. Colors distinguish the engagement features included in each model. The dashed line indicates the uniform random baseline.
Within each repetition, all four choices from the same participant were assigned to the same fold, ensuring that participants in the test set were not included in the training set.
As a baseline, we consider uniform random selection, a model that assigns probability to each article. For the 816 clicks in E2, this corresponds to a log-likelihood of , shown as a dashed line in Figure 2.
For the engagement component, we consider three specifications: Likes, using the number of likes; Shares, using the number of shares; and Likes + Shares, using the sum of both counts. Each specification uses a single engagement feature and coefficient.
Engagement cues alone achieve higher predictive performance than either stance or rank alone. The differences between engagement measures are small, with likes yielding higher log-likelihoods than shares. Using their sum offers no clear improvement over likes alone. The highest out-of-sample log-likelihood is obtained when stance, rank, and engagement are included together.
Counterfactual Click Probabilities. To quantify the effects of ranking position and displayed engagement on click probabilities, we use the model fitted on E2 to compute counterfactual predictions for the 816 rankings presented to participants. For each ranking, we first swap the positions of the top- and bottom-ranked articles. We then consider a separate counterfactual in which the articles with the lowest and highest engagement exchange their like and share counts. In each case, we compare the predicted click probability of the article initially at the bottom or with the lowest engagement, before and after the swap, keeping all other features unchanged. We report the interquartile ranges (25th–75th percentiles) of these probabilities and their relative changes across rankings.
Moving an article from the bottom to the top increases its predicted click probability from 3.5–7.7% to 9.2–18.9%, corresponding to relative increases of 141–162%. Similarly, exchanging the lowest and highest engagement counts increases the predicted probability from 3.9–8.4% to 12.5–24.4%, with relative increases of 158–268%.
The observed clicks show a similar pattern. Bottom-ranked articles received 7.8% of clicks, compared with 17.8% for top-ranked articles. Articles displaying the lowest like counts received 7.2% of clicks, compared with 21% for those displaying the highest counts.
Discussion
We have presented a conditional multinomial logit model that jointly accounts for political stance, ranking position, and visual engagement cues in news selection. The model includes the clicking model of D’Ignazi et al. [3] as a special case and incorporates engagement cues into the same likelihood-based framework.
Cross-validation shows that engagement cues alone predict clicks better than stance or rank, while combining all components yields the highest out-of-sample log-likelihood.
The fitted model estimates how click probabilities change when an article’s ranking position or displayed engagement counts are modified. Since both features were randomized in our experiment, these counterfactual estimates can be interpreted causally. These predictions show that moving an article from the bottom to the top of a ranking more than doubles its click probability, while assigning it the highest instead of the lowest engagement counts roughly triples it.
Overall, our findings highlight the importance of visual engagement cues in shaping political news selection, beyond ranking position and ideological preferences. Our results suggest that the way platforms display popularity information can affect which political news articles attract attention. Whether these effects persist in more realistic browsing environments or have implications for news diversity and political polarization remains an open question.
Limitations. The two experimental conditions were conducted six months apart (March and September 2026), without random assignment of participants between conditions. Both conditions used the same articles and recruitment criteria, but differences between the two samples may remain. Our causal evidence for engagement comes from the randomized counts within E2 and concerns their displayed magnitude, not the effect of making engagement cues visible.
Ethical Use of Data and Informed Consent. The study followed ACM’s Publications Policy on Research Involving Human Participants and Subjects (2021) and applicable institutional and disciplinary ethical standards. All procedures received ethics board approval, and participants provided informed consent. The consent form stated that participation “does not entail risks greater than those ordinarily encountered in daily life”. Political affiliations previously disclosed to Prolific were used for quota sampling.
References
- [1]Mihai Avram, Nicholas Micallef, Sameer Patil, and Filippo Menczer. 2020. Exposure to Social Engagement Metrics Increases Vulnerability to Misinformation. Harvard Kennedy School Misinformation Review 1, 5 (2020). doi:10.37016/mr-2020-033DOI
- [2]Eytan Bakshy, Solomon Messing, and Lada A. Adamic. 2015. Exposure to Ideologically Diverse News and Opinion on Facebook. Science 348, 6239 (2015). doi:10.1126/science.aaa1160DOI
- [3]Jacopo D’Ignazi, Emma Fraxanet, Andreas Kaltenbrunner, Gaël Le Mens, Fabrizio Germano, and Vicenç Gómez. 2026. Rewarding Engagement and Personalization in Popularity-Based Rankings Amplifies Extremism and Polarization. In Proceedings of the 32nd ACM SIGKDD Conference. 875–886. doi:10.1145/3770855.3818037DOI
- [4]Shira Dvir-Gvirsman. 2019. I Like What I See: Studying the Influence of Popularity Cues on Attention Allocation and News Selection. Information, Communication & Society 22, 2 (2019), 286–305. doi:10.1080/1369118X.2017.1379550
- [5]Ines Engelmann and Manuel Wendelin. 2017. Comment Counts or News Factors or Both? Influences on News Websites’ News Selection. International Journal of Communication 11 (2017), 2501–2519.
- [6]Ziv Epstein, Hause Lin, Gordon Pennycook, and David Rand. 2022. How Many Others Have Shared This? Experimentally Investigating the Effects of Social Cues on Engagement, Misinformation, and Unpredictability on Social Media. arXiv:2207.07562 [cs.SI] doi:10.48550/arXiv.2207.07562
- [7]Fabrizio Germano, Vicenç Gómez, and Francesco Sobbrio. 2026. Ranking for engagement: How social media algorithms fuel misinformation and polarization. Journal of Public Economics 255 (2026), 105589.
- [8]Anindya Ghose, Panagiotis G. Ipeirotis, and Beibei Li. 2014. Examining the Impact of Ranking on Consumer Behavior and Search Engine Revenue. Management Science 60, 7 (2014), 1632–1654. doi:10.1287/mnsc.2013.1828DOI
- [9]Peter M. Guadagni and John D. C. Little. 1983. A Logit Model of Brand Choice Calibrated on Scanner Data. Marketing Science 2, 3 (1983), 203–238. doi:10.1287/mksc.2.3.203
- [10]Andrew M. Guess, Neil Malhotra, Jennifer Pan, Pablo Barberá, Hunt Allcott, et al. 2023. How Do Social Media Feed Algorithms Affect Attitudes and Behavior in an Election Campaign? Science 381, 6656 (2023), 398–404. doi:10.1126/science.abp9364DOI
- [11]Tad Hogg and Kristina Lerman. 2015. Disentangling the Effects of Social Signals. Human Computation 2, 2 (2015). doi:10.15346/hc.v2i2.4DOI
- [12]Ferenc Huszár, Sofia Ira Ktena, Conor O’Brien, Luca Bellini, Andrew Schlaikjer, and Moritz Hardt. 2022. Algorithmic Amplification of Politics on Twitter. Proceedings of the National Academy of Sciences 119, 1 (2022), e2025334119. doi:10.1073/pnas.2025334119
- [13]Coco Krumme, Manuel Cebrian, Galen Pickard, and Sandy Pentland. 2012. Quantifying Social Influence in an Online Cultural Market. PLoS ONE 7, 5 (2012), e33785. doi:10.1371/journal.pone.0033785DOI
- [14]Solomon Messing and Sean J. Westwood. 2014. Selective Exposure in the Age of Social Media: Endorsements Trump Partisan Source Affiliation When Selecting News Online. Communication Research 41, 8 (2014), 1042–1063. doi:10.1177/0093650212466406
- [15]Lew Muchnik, Sinan Aral, and Sean J. Taylor. 2013. Social Influence Bias: A Randomized Experiment. Science 341, 6146 (2013), 647–651. doi:10.1126/science.1240466
- [16]Jan Overgoor, Austin R. Benson, and Johan Ugander. 2019. Choosing to Grow a Graph: Modeling Network Formation as Discrete Choice. In Proceedings of the World Wide Web Conference (WWW). 1409–1420. doi:10.1145/3308558.3313662DOI
- [17]Kiran Tomlinson and Austin R. Benson. 2021. Learning Interpretable Feature Context Effects in Discrete Choice. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (KDD). 1582–1592. doi:10.1145/3447548.3467250DOI
- [18]Raluca M. Ursu. 2018. The Power of Rankings: Quantifying the Effect of Rankings on Online Consumer Search and Purchase Decisions. Marketing Science 37, 4 (2018), 530–552. doi:10.1287/mksc.2017.1072
