betaFeedback

WorksComputer Science

A Model of Diverse Sampling from Language Models

Abstract

Language models often produce high-quality individual samples but poor sets, with repeated samples clustering around the same semantic modes. We formalize diverse generation as sampling a size-KK subset of complete strings from a determinantal point process (DPP), where the likelihood encodes item quality, and the embedding geometry encodes repulsion between similar outputs. Exact inference over all strings is intractable, so we propose PoolDPP: draw a finite candidate pool from a tractable proposal, importance-weight the candidates toward the target quality, and run a KK-DPP sampler on the induced pool kernel; a greedy MAP variant provides deterministic selection. We prove a finite-sample bound with an explicit constant: the KL divergence between the sampler's output and the target KK-DPP decays as O(1/N) in the pool size. Empirically, PoolDPP extends the quality–diversity frontier on open-ended generation and the precision–recall frontier on ambiguous text-to-SQL (Ambrosia), and on mathematical reasoning (GSM8K), it raises diversity at about the same pass@KK.

https://github.com/rycolab/pool-dpp

Introduction

Search engines gave us pages of results; we browsed and chose. In contrast, language models often give us just one authoritative-sounding answer. However, for important questions—where multiple perspectives exist, or the right answer depends on context—we might want that plurality back. For example, when answering ambiguous questions, a model should surface multiple valid interpretations rather than commit to a single one [33]. Pushing the language model’s regenerate button 10 times yields 10 distinct outputs, but they often cluster around the same idea, phrased slightly differently.1 This makes sense because the model generates each string independently; it has no notion of what it means for a set of 10 strings to be diverse.

Why do samples cluster? Because nothing in the sampling process discourages similarity. Temperature sampling broadens the distribution over next tokens, but it does not model relationships between outputs—two samples may independently land in the same high-probability region.2 Diverse decoding methods based on beam search [36, 30] inject heuristics during decoding, but they lack a globally coherent notion of what it means for two strings to be similar or for a set to be diverse. These approaches are attractive because they are fast, but they are documented to produce artifacts such as incoherence or sudden shifts in topic [17, 35]. Other work has explored adopting sampling-without-replacement schemes in LMs [23, 29], but the notion of similarity in these models is exact equality. Thus, these methods tend to cluster like independent samples. To get diverse outputs, we need to define similarity and then explicitly discourage it.

We formalize diverse sampling as a probabilistic model over subsets of strings using determinantal point processes (DPPs; [25]).3 The key ingredient is a similarity function: we represent each string by its embedding (e.g., from a pretrained sentence embedder) scaled by its quality, so that the direction encodes meaning and the magnitude encodes quality. Two strings are similar if their embeddings point in roughly the same direction. A DPP assigns probability to subsets proportional to the squared volume spanned by their quality-scaled embeddings—large when items are high-quality and point in different directions, near-zero when they are redundant. This gives us a well-defined distribution over sets: one that assigns high probability to sets of KK strings that are individually good and collectively spread out, with a parameter that controls the balance between quality and diversity. Figure 1 illustrates this in the context of generating meaningfully different sorting algorithm implementations.

Figure 1. Diverse implementations of sorting. Each point is a sorting-algorithm implementation, embedded by a code language model and projected to 3D with PCA; colors denote algorithm family. The 3-DPP assigns subset probability proportional to the squared volume of the parallelepiped spanned by the selected quality-scaled embeddings—large when the selected items are individually high-quality (large norms) and mutually dissimilar. The highlighted triple has high probability because it combines implementations from distinct algorithmic strategies, yielding a large spanned volume; by contrast, choosing three variants from the same cluster (e.g., three quicksorts) produces nearly parallel embeddings and thus near-zero volume despite high individual quality.

Since exact inference in this DPP over all strings is intractable, we propose PoolDPP, a finite-pool approximation. PoolDPP draws NN candidate strings from a tractable proposal, importance-weights them toward a globally tempered quality distribution, and applies a KK-DPP on the resulting pool. The same pool kernel supports two inference modes: stochastic KK-DPP sampling (PoolDPP-Sampling) and greedy MAP selection (PoolDPP-MAP). For the sampling mode, we prove a non-asymptotic approximation guarantee: the KL divergence between the marginal output law of PoolDPP-Sampling (its output distribution, averaged over random pools) and the target KK-DPP decreases as O(1/N)\mathcal{O}(1/N) with the pool size. The bound’s constant is explicit, and we evaluate it on each task, on instances built from its pools rather than on the full proposal; the MAP mode has no distributional guarantee. Empirically, PoolDPP extends the quality–diversity frontier on open-ended generation and the precision–recall frontier on ambiguous text-to-SQL. On mathematical reasoning, it raises diversity at about the same pass@KK (within 0.01; § 6.3), and we measure how weakly the embedders separate correct from incorrect reasoning chains.

Language Modeling

A language model pp is a probability distribution over the set Σ∗\Sigma^{*} of all strings over an alphabet Σ\Sigma. Every language model pp admits the factorization p(y)=p~(EOS∣y)∏t=1∣y∣p~(yt∣y<t)p(\boldsymbol{y}) = \tilde{p}(\mathrm{EOS} \mid\boldsymbol{y}) \prod_{t=1}^{\lvert\boldsymbol{y}\rvert} \tilde{p}(y_t \mid y_{<t}), where y<t=y1,…,yt−1\boldsymbol{y}_{<t} = y_1, \ldots, y_{t-1}, ∣y∣\lvert\boldsymbol{y}\rvert denotes the string’s length, and EOS∉Σ\mathrm{EOS} \notin\Sigma is a distinguished end-of-string marker. Here, p~(y)=∑y′∈Σ∗p(yy′)\tilde{p}(\boldsymbol{y}) = \sum_{\boldsymbol{y}' \in\Sigma^{*}} p(\boldsymbol{y}\boldsymbol{y}') is the prefix probability, the probability that a string drawn from pp starts with y\boldsymbol{y}. Its conditionals are p~(y∣y)=p~(yy)/p~(y)\tilde{p}(y \mid\boldsymbol{y}) = \tilde{p}(\boldsymbol{y}y)/\tilde{p}(\boldsymbol{y}) for y∈Σy \in\Sigma and p~(EOS∣y)=p(y)/p~(y)\tilde{p}(\mathrm{EOS} \mid\boldsymbol{y}) = p(\boldsymbol{y})/\tilde{p}(\boldsymbol{y}) (if p~(y)=0\tilde{p}(\boldsymbol{y}) = 0, we set p~(y∣y)=1{y=EOS}\tilde{p}(y \mid\boldsymbol{y}) = \mathbb{1}\{y = \mathrm{EOS}\} for all y∈Σ∪{EOS}y \in\Sigma\cup\{\mathrm{EOS}\}). In our experiments, the language model is conditioned on a prompt x∈Σ∗x \in\Sigma^{*}: p(y∣x)=defp(xy)/p⃗(x)p(y \mid x) \mathrel{\stackrel{\mathrm{def}}{=}} p(xy)/\vec{p}(x), the probability that a string starting with xx continues with yy and then ends. We write p(y)p(y) when the prompt is fixed. This factorization gives a generation procedure, known as ancestral sampling: starting from the empty prefix, we sample tokens left-to-right as yt∼p⃗(⋅∣y<t)y_t \sim\vec{p}(\cdot\mid y_{<t}) until EOS is drawn.

We define local annealing by temperature T′T' as the following change to the conditional distribution, and the locally annealed language model as the product of the annealed conditionals:

p⃗T′(yt∣y<t)=defp⃗(yt∣y<t)1/T′∑y∈Σ∪{EOS}p⃗(y∣y<t)1/T′(1)\vec{p}_{T'}(y_t \mid y_{<t}) \mathrel{\stackrel{\mathrm{def}}{=}} \frac{\vec{p}(y_t \mid y_{<t})^{1/T'}}{\sum_{y \in\Sigma\cup\{\mathrm{EOS}\}} \vec{p}(y \mid y_{<t})^{1/T'}} \tag*{(1)}
p⃗T′(y)=defp⃗T′(EOS∣y)∏t=1∣y∣p⃗T′(yt∣y<t)(2)\vec{p}_{T'}(y) \mathrel{\stackrel{\mathrm{def}}{=}} \vec{p}_{T'}(\mathrm{EOS} \mid y)\prod_{t=1}^{|y|}\vec{p}_{T'}(y_t \mid y_{<t}) \tag*{(2)}

for all yt∈Σ∪{EOS}y_t \in\Sigma\cup\{\mathrm{EOS}\} and y∈Σ∗y \in\Sigma^{*}. Ancestral sampling with the annealed conditionals draws from p⃗T′\vec{p}_{T'}. As T′→0T' \to0, each conditional concentrates on its mode, yielding greedy decoding; in contrast, as T′→∞T' \to\infty, each conditional approaches the uniform distribution over Σ∪{EOS}\Sigma\cup\{\mathrm{EOS}\}. High local temperatures increase diversity at the cost of coherence; low temperatures increase coherence at the cost of diversity [11, 15]. Local annealing is efficient, as it only requires O(∣Σ∣)O(|\Sigma|) additional work per time step to adjust its conditional distribution.

In contrast, global annealing by temperature TT modifies the probability of entire sequences:

pT(y)=defp(y)1/T∑y′∈Σ∗p(y′)1/T(3)p_T(y) \mathrel{\stackrel{\mathrm{def}}{=}} \frac{p(y)^{1/T}}{\sum_{y' \in\Sigma^{*}}p(y')^{1/T}} \tag*{(3)}

when the sum is finite, which holds for every T≤1T \le1, since p(y)1/T≤p(y)p(y)^{1/T} \le p(y). Unlike local annealing, which modifies each conditional independently, global annealing linearly scales log-probability differences between sequences: high TT compresses these differences, while low TT amplifies them. Global annealing tends to preserve the coherence and fluency of the underlying language model. Kempton and Burrell [20] show that local annealing does not generally yield a good approximation to the globally annealed distribution pTp_T. Unfortunately, exact sampling from the global distribution is intractable.4

Determinantal Point Processes

Determinantal point processes (DPPs) are distributions over subsets that naturally encode diversity: items that are similar are unlikely to be selected together. We use DPPs to define a distribution over sets of strings where high-probability sets contain strings that are both high-quality and well-dispersed.

The many faces of diversity. There are many ways to operationalize the notion of a diverse set of strings. In natural language, we can distinguish semantic diversity (variation in meaning), syntactic diversity (variation in surface form), and stylistic diversity (variation in tone and register). In specialized domains such as code generation, one can also consider algorithmic diversity, i.e., variation in the underlying algorithms implemented by the generated code. Which notion matters depends on the application. In this work, we focus on notions of diversity that can be expressed through embeddings. This lets us quantify similarity geometrically and swap in a different embedder depending on the domain.

Embeddings. To measure similarity between strings, we represent each string yy by an embedding ϕ(y)∈RD\phi(y) \in\mathbb{R}^{D} from a pretrained embedder. The choice of embedder determines what notion of diversity the DPP captures—semantic, syntactic, or domain-specific. We try three options across our experiments: a small off-the-shelf semantic embedder (MiniLM-L6), a larger one (Qwen3-Embedding-0.6B), and the generating LM’s own mean-pooled last-layer hidden states. On open-ended tasks like FineWeb (§ 6.1), the two off-the-shelf embedders give very similar quality–diversity Pareto frontiers across language models (Figure 8), so we use the smaller MiniLM for these tasks; on more constrained tasks like math reasoning on GSM8K (§ 6.3), the larger Qwen3-Embedding-0.6B picks up structural distinctions between reasoning chains that MiniLM blurs, and we adopt it as the default. The self-contained LM-hidden-state variant performs on par with the small semantic embedder in several settings but does not consistently improve over it; layer and pooling choices that we did not exhaustively optimize, and a semantic-oriented fine-tune, are a plausible route to closing that gap.

Figure 8. LM/embedding robustness (FineWeb, K=10K = 10, N=400N = 400). Pareto frontiers across two LMs and two embedders. The qualitative ordering is stable across all four configurations.

Building a DPP. Given a base set Y\mathcal{Y}, a quality measure κ:Y→R≥0\kappa:\mathcal{Y}\to\mathbb{R}_{\ge0}, and a (unit-norm) embedding ϕ:Y→RD\phi:\mathcal{Y}\to\mathbb{R}^{D} (i.e., ∥ϕ(y)∥2=1\lVert\phi(y)\rVert_{2}=1 for all y∈Yy\in\mathcal{Y}),5 we define the (quality–diversity) kernel matrix [25 §3.1]:

L(x,y):def=κ(x)ϕ(x)Tϕ(y)κ(y)(4)L(x,y) \mathrel{\mathop:}^{\mathrm{def}}=\sqrt{\kappa(x)}\phi(x)^{\mathsf{T}}\phi(y)\sqrt{\kappa(y)} \tag*{(4)}

Intuitively, in the quality-scaled embedding κ(y)ϕ(y)\sqrt{\kappa(y)}\phi(y), the direction encodes meaning (for diversity) and the magnitude encodes quality. Let S⊆YS\subseteq\mathcal{Y} be a finite set, and LS:def=[L(x,y)]x,y∈S∈RS×SL_{S}\mathrel{\mathop:}^{\mathrm{def}}=[L(x,y)]_{x,y\in S}\in\mathbb{R}^{S\times S} be the kernel matrix indexed by SS. A determinantal point process (DPP) is a probability distribution over subsets S⊆YS\subseteq\mathcal{Y} where the probability of the set SS is proportional to det⁡(LS)\det(L_{S}). This determinant has a geometric interpretation: it equals the squared volume of the parallelepiped spanned by the quality-scaled embeddings {κ(y)ϕ(y)}y∈S\{\sqrt{\kappa(y)}\phi(y)\}_{y\in S} [25 §3]. This volume is large when strings are high-quality and point in different directions, and collapses to zero when two strings are nearly identical. More precisely, under Eq. (4), the determinant decomposes as

det⁡(LS)=(∏y∈Sκ(y))det⁡(ΦSTΦS),\det(L_{S})=\left(\prod_{y\in S}\kappa(y)\right)\det(\Phi_{S}^{\mathsf{T}}\Phi_{S}),

where ΦS:def=[ϕ(y)]y∈S∈RD×S\Phi_{S}\mathrel{\mathop:}^{\mathrm{def}}=[\phi(y)]_{y\in S}\in\mathbb{R}^{D\times S}. Here, the quality factor ∏y∈Sκ(y)\prod_{y\in S}\kappa(y) rewards individual quality, while the diversity factor det⁡(ΦSTΦS)\det(\Phi_{S}^{\mathsf{T}}\Phi_{S}) penalizes collective similarity (the squared volume spanned by the embeddings).

K-DPPs. Our work uses the KK-DPP variant, which introduces a set-size parameter KK. This parameter constrains the underlying DPP to generate sets of exactly KK items; thus, it can be used to encourage diversity by controlling the number of elements. The KK-DPP’s probability is defined below:6

P(S):def=det⁡(LS)1{∣S∣=K}ZP(S)\mathrel{\mathop:}^{\mathrm{def}}=\frac{\det(L_{S})\mathbf{1}\{|S|=K\}}{Z}

where

Z:def=∑S∈(YK)det⁡(LS)(5)Z\mathrel{\mathop:}^{\mathrm{def}}=\sum_{S\in\binom{\mathcal{Y}}{K}}\det(L_{S}) \tag*{(5)}

and (YK)\binom{\mathcal{Y}}{K} is the set of size-KK subsets from Y\mathcal{Y}. P(S)P(S) is only well-defined when 0<Z<∞0<Z<\infty.7

Trading off quality and diversity. We also make use of a parameter T>0T>0, the global temperature, to trade off quality and diversity. Specifically, we use

κ(y)=p(y)1/T(6)\kappa(y)=p(y)^{1/T} \tag*{(6)}

References

  1. [1]Nima Anari, Shayan Oveis Gharan, and Alireza Rezaei. Monte Carlo Markov chain algorithms for sampling strongly Rayleigh distributions and determinantal point processes. In Proceedings of the Annual Conference on Learning Theory, 2016. URL https://proceedings.mlr.press/v49/anari16.html.arxiv.org/abs/1602.05242
  2. [2]David Arthur and Sergei Vassilvitskii. k-means++: The advantages of careful seeding. In Proceedings of the Annual ACM-SIAM Symposium on Discrete Algorithms, 2007. URL https://dl.acm.org/doi/10.5555/1283383.1283494.DOI
  3. [3]Peter J. Bickel and Kjell A. Doksum. Mathematical Statistics: Basic Ideas and Selected Topics, volume I. Chapman and Hall/CRC, 2nd edition, 2015. URL https://www.routledge.com/Mathematical-Statistics-Basic-Ideas-and-Selected-Topics-Volume-I-Second/Bickel-Doksum/p/book/9781498723800.
  4. [4]Julius Borcea, Petter Brändén, and Thomas M. Liggett. Negative dependence and the geometry of polynomials. Journal of the American Mathematical Society, 22(2), 2009. URL https://doi.org/10.1090/S0894-0347-08-00618-8.
  5. [5]Jaime G. Carbonell and Jade Goldstein. The use of MMR, diversity-based reranking for reordering documents and producing summaries. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 1998. URL https://doi.org/10.1145/290941.291025.
  6. [6]Laming Chen, Guoxin Zhang, and Hanning Zhou. Fast greedy MAP inference for determinantal point process to improve recommendation diversity. In Advances in Neural Information Processing Systems, 2018. URL https://proceedings.neurips.cc/paper_files/paper/2018/hash/dbf603ff0e99629dda57b5b6f75f96e-Abstract.html.arxiv.org/abs/1709.05135
  7. [7]Yilei Chen, Souradip Chakraborty, Lorenz Wolf, Ioannis Paschalidis, and Aldo Pacchiano. Post-training large language models for diverse high-quality responses. In International Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=BV3bLKaJ3Y.arxiv.org/abs/2509.04784
  8. [8]Ali Çivril and Malik Magdon-Ismail. On selecting a maximum volume sub-matrix of a matrix and related problems. Theoretical Computer Science, 410(47–49), 2009. URL https://doi.org/10.1016/j.tcs.2009.06.018.
  9. [9]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems, 2021. URL https://arxiv.org/abs/2110.14168.
  10. [10]Ryan Cotterell. The Probabilistic Typology of Vowel Systems. PhD thesis, Johns Hopkins University, 2021. URL https://jscholarship.library.jhu.edu/server/api/core/bitstreams/a6243442-b807-405d-a4a4-ac408baa8752/content.
  11. [11]DeepSeek-AI. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning, 2025. URL https://arxiv.org/abs/2501.12948.
  12. [12]Boqing Gong, Wei-Lun Chao, Kristen Grauman, and Fei Sha. Diverse sequential subset selection for supervised video summarization. In Advances in Neural Information Processing Systems, 2014. URL https://proceedings.neurips.cc/paper_files/paper/2014/file/5d3b9e06117de70a7e5076cc3ed89e18-Paper.pdf.
  13. [13]Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, 2021. URL https://datasets-benchmarks-proceedings.neurips.cc/paper_files/paper/2021/hash/be83ab3ecd0db773eb2dc1b0a17836a1-Abstract-round2.html.
  14. [14]Wassily Hoeffding. A class of statistics with asymptotically normal distribution. The Annals of Mathematical Statistics, 19(3), 1948. URL https://doi.org/10.1214/aoms/1177730196.
  15. [15]Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=rygGQyrFvH.
  16. [16]Jonathan H. Huggins. An information-theoretic analysis of resampling in sequential Monte Carlo. Master’s thesis, Massachusetts Institute of Technology, 2014. URL http://hdl.handle.net/1721.1/91033.
  17. [17]Daphne Ippolito, Reno Kriz, João Sedoc, Maria Kustikova, and Chris Callison-Burch. Comparison of diverse decoding methods from conditional language models. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, 2019. URL https://aclanthology.org/P19-1365/.arxiv.org/abs/1906.06362
  18. [18]Aayush Karan and Yilun Du. Reasoning with sampling: Your base model is smarter than you think. In International Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=Vsgq2ldr4K.arxiv.org/abs/2510.14901
  19. [19]Tahira Kazimi, Connor Dunlop, and Pinar Yanardag. Diverse video generation with determinantal point process-guided policy optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2026. URL https://openaccess.thecvf.com/content/CVPR2026/html/Kazimi_Diverse_Video_Generation_with_Determinantal_Point_Process-Guided_Policy_Optimization_CVPR_2026_paper.html.arxiv.org/abs/2511.20647
  20. [20]Tom Kempton and Stuart Burrell. Local normalization distortion and the thermodynamic formalism of decoding strategies for large language models. In Findings of the Association for Computational Linguistics: EMNLP, 2025. URL https://aclanthology.org/2025.findings-emnlp.1210/.arxiv.org/abs/2503.21929
  21. [21]Kevin Knight. Decoding complexity in word-replacement translation models. Computational Linguistics, 25(4), 1999. URL https://aclanthology.org/J99-4005/.DOI
  22. [22]Chun-Wa Ko, Jon Lee, and Maurice Queyranne. An exact algorithm for maximum entropy sampling. Operations Research, 43(4), 1995. URL https://doi.org/10.1287/opre.43.4.684.
  23. [23]Wouter Kool, Herke van Hoof, and Max Welling. Stochastic beams and where to find them: The Gumbel-top-k trick for sampling sequences without replacement. In Proceedings of the International Conference on Machine Learning, 2019. URL https://proceedings.mlr.press/v97/kool19a.html.arxiv.org/abs/1903.06059
  24. [24]Alex Kulesza and Ben Taskar. Structured determinantal point processes. In Advances in Neural Information Processing Systems, 2010. URL https://proceedings.neurips.cc/paper/2010/file/1f50893f80d6830d62765ffa7d721742-Paper.pdf.
  25. [25]Alex Kulesza and Ben Taskar. Determinantal point processes for machine learning. Foundations and Trends in Machine Learning, 5(2–3), 2012. URL https://doi.org/10.1561/2200000044.
  26. [26]Shankar Kumar and William Byrne. Minimum Bayes-risk decoding for statistical machine translation. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL, 2004. URL https://aclanthology.org/N04-1022/.
  27. [27]Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=v8L0pN6E0i.arxiv.org/abs/2305.20050
  28. [28]Odile Macchi. The coincidence approach to stochastic point processes. Advances in Applied Probability, 7(1), 1975. URL https://doi.org/10.2307/1425855.
  29. [29]Clara Meister, Afra Amini, Tim Vieira, and Ryan Cotterell. Conditional Poisson stochastic beams. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2021a. URL https://aclanthology.org/2021.emnlp-main.52.
  30. [30]Clara Meister, Martina Forster, and Ryan Cotterell. Determinantal beam search. In Proceedings of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing, 2021b. URL https://aclanthology.org/2021.acl-long.512.
  31. [31]Clara Meister, Tiago Pimentel, Luca Malagutti, Ethan Wilcox, and Ryan Cotterell. On the efficacy of sampling adapters. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, 2023. URL https://aclanthology.org/2023.acl-long.80/.
  32. [32]Guilherme Penedo, Hynek Kydlíček, Loubna Ben Allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, and Thomas Wolf. The FineWeb datasets: Decanting the web for the finest text data at scale. In Advances in Neural Information Processing Systems, 2024. URL https://proceedings.neurips.cc/paper_files/paper/2024/hash/370df50ccfdf8bde18f8f9c2d9151bda-Abstract-Datasets_and_Benchmarks_Track.html.arxiv.org/abs/2406.17557
  33. [33]Irina Saparina and Mirella Lapata. AMBROSIA: A benchmark for parsing ambiguous questions into database queries. In Advances in Neural Information Processing Systems, 2024. URL https://proceedings.neurips.cc/paper_files/paper/2024/hash/a4c942a8405cc910f0a833d28d2573cc-Abstract-Datasets_and_Benchmarks_Track.html.arxiv.org/abs/2406.19073
  34. [34]Solomon E. Shimony. Finding MAPs for belief networks is NP-hard. Artificial Intelligence, 68(2), 1994. URL https://doi.org/10.1016/0004-3702(94)90072-8.
  35. [35]Guy Tevet and Jonathan Berant. Evaluating the evaluation of diversity in natural language generation. In Proceedings of the Conference of the European Chapter of the Association for Computational Linguistics, 2021. URL https://aclanthology.org/2021.eacl-main.25/.arxiv.org/abs/2004.02990
  36. [36]Ashwin K Vijayakumar, Michael Cogswell, Ramprasaath R. Selvaraju, Qing Sun, Stefan Lee, David Crandall, and Dhruv Batra. Diverse beam search: Decoding diverse solutions from neural sequence models, 2018. URL https://arxiv.org/abs/1610.02424.
  37. [37]Mark Wilhelm, Ajith Ramanathan, Alexander Bonomo, Sagar Jain, Ed H. Chi, and Jennifer Gillenwater. Practical diversified recommendations on YouTube with determinantal point processes. In Proceedings of the ACM International Conference on Information and Knowledge Management, 2018. URL https://doi.org/10.1145/3269206.3272018.
  38. [38]David P. Woodruff. Sketching as a tool for numerical linear algebra. Foundations and Trends in Theoretical Computer Science, 10(1–2), 2014. URL https://doi.org/10.1561/0400000060.
  39. [39]Hugh Zhang, Daniel Duckworth, Daphne Ippolito, and Arvind Neelakantan. Trading off diversity and quality in natural language generation. In Proceedings of the Workshop on Human Evaluation of NLP Systems, 2021. URL https://aclanthology.org/2021.humeval-1.3/.arxiv.org/abs/2004.10450

Paper details

vlnc:W_7epq7k49
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 5

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

agentic coding utilities

View statement
Looking for related papers…
Contents