WorksComputer Science
Using AI Predictions to Augment Rather Than Replace Surveys
Abstract
LLMs can predict human survey responses with nontrivial accuracy. What this means for the future of social surveys remains hotly contested. Optimists anticipate a coming era of synthetic respondents; skeptics see a fragile and overhyped substitute. This article advocates a position: AI predictions cannot replace surveys, but they can augment them. Using prediction-powered inference (PPI), a statistical framework rooted in survey methodology and known in sociology as "mixed subjects design", the study tests whether AI predictions can improve the statistical efficiency of social surveys when combined with census-scale auxiliary information. Applying the framework to 83 social outcomes in the General Social Survey (GSS) and National Health Interview Survey (NHIS), with American Community Survey (ACS) being the auxiliary file, we find that supervised predictors yield effective-sample-size gains of about 7-12 percent when estimating the mean, while LLM gains are smaller but positive.
Transparency and Reproducibility Statement. The replication repository for this study is hosted on the Open Science Framework (OSF).
Acknowledgements. This research was supported by the Research Grants Council of the Hong Kong Special Administrative Region, China, through its Early Career Scheme (Project No. 27623826, “Powering Survey Research with Census Data and Predictions from Machine Learning and LLM”).
Introduction
AI is no longer a distant future; it is already part of the infrastructure of social science. Across fields, predictive systems now support tasks ranging from classification to simulation. Sociology has entered this moment as well, as generative models increasingly shape debates about measurement and routine research practice [3, 9].
A separate trend sociologists are facing is the increasing difficulty of fielding social surveys. Probability surveys remain an indispensable source of information for understanding society, but the classical operating model of large samples, high response rates, and stable fieldwork costs has become harder to sustain [27, 32, 33, 50].
When these two trends meet, we see an emerging interest in the use of AI for social surveys. Large language models can be prompted with demographic profiles and then requested to answer common social survey questions, and they often produce responses that resemble human answers in nontrivial ways [6, 11, 38, 54, 59, 68]. Supervised learning trained on earlier survey waves can also predict many survey outcomes from demographic and socioeconomic covariates [52, 57, 76]. These results make it tempting to imagine a future in which surveys become smaller, with predictive models filling in what humans would have said.
That temptation requires caution. AI-generated survey responses can contain arbitrary and hard-to-audit bias. They can be too centered or under-dispersed, muting the heterogeneity that surveys are designed to measure. They can also be too discriminative, reproducing stereotypes or group-specific distortions in the name of prediction [11, 40, 59, 65, 68].
Treating AI predictions as replacements for survey responses gives models more authority than their bias can support. Discarding those predictions altogether leaves useful information unused. We advocate a middle path, using AI predictions to augment actual surveys. A prediction can be wrong in level or distribution and still be useful if it is correlated with the true outcome. That correlation lets the model summarize the part of the outcome that is predictable from observed covariates in a large auxiliary file. The probability survey then measures prediction error by comparing predicted and observed responses among sampled humans. The resulting estimator uses AI for the predictable component and uses the survey to correct what AI misses, preserving design-based validity while reducing sampling variability.
Prediction-powered inference (PPI) formalizes this augmentation strategy [4, 5, 78]. Its central operation is outcome prediction followed by correction. Predictions on a large unlabeled file are combined with a correction estimated from observed outcomes in a smaller labeled sample. In our application, we also incorporate survey weights to account for the sampling designs, whereas prediction-powered augmentation uses additional information to improve precision. For mean estimation, this correction follows the logic of the classical difference estimator, linking PPI to the model-assisted survey tradition [13, 17, 23, 48, 60].
PPI is especially useful when collecting additional observed outcomes is expensive but a much larger unlabeled file from the same population distribution is available. The objective is to improve precision for the same inferential target, not to establish representativeness or justify extrapolation to a different population. Gains depend on whether predictions sufficiently reduce the variation that must be estimated from the labeled sample; a large unlabeled file alone does not guarantee improvement. Social surveys offer an important application because additional interviews are costly, while files such as the American Community Survey (ACS), harmonized through IPUMS, provide compatible covariates for many more individuals without measuring the focal survey outcomes.
We test how much PPI can help with routine quantitative tasks in sociology, including estimating population means and estimating regression coefficients. We apply the framework to a range of opinion outcomes in the General Social Survey (GSS) and health outcomes in the National Health Interview Survey (NHIS), with ACS as the auxiliary file. We compare historical supervised learning, cross-fitted supervised learning, and LLM respondent simulation, and we evaluate both predictive performance and downstream inferential gain. For mean estimation, supervised predictors yield effective-sample-size gains of about 7 to 12 percent, while LLM gains are smaller. In this benchmark, gains are generally larger for health outcomes than opinion outcomes and for means than regression coefficients. These differences matter for how researchers use PPI: a predictor that improves estimation of an outcome mean may provide little benefit for an adjusted demographic association. Predictors should therefore be evaluated for the intended estimand, rather than by outcome-level predictive accuracy alone.
In summary, this article illustrates a design principle for AI-era survey research, which uses cheap AI predictions to augment expensive human responses. The analyses show how predictions can be incorporated into probability survey inference without treating model outputs as respondents. The empirical results establish a conservative baseline using public surveys, ACS covariates, and contemporary prediction tools. Better auxiliary data and stronger prediction systems may increase those gains, while weakly predicted outcomes and groups identify where human measurement remains indispensable. We invite survey methodologists, sociologists, and machine-learning researchers to test this design in new surveys and build richer auxiliary frames. We also invite researchers to release PPI-ready prediction artifacts and evaluate predictors by the precision they add after survey data correct prediction error. These exercises will help improve the statistical efficiency of social surveys without conducting additional interviews, thus reducing costs for survey organizers and improving power for data users.
Related Literature
Three literatures define the problem this article addresses. Survey methodology shows how auxiliary information can improve precision when probability sampling remains the source of validity. The predictive turn in social science shows that many social outcomes contain measurable but uneven signal. Research on LLM-generated respondents shows that AI predictions are cheap and flexible, but also that uncorrected synthetic responses cannot support population inference.
Model-assisted and prediction-powered inference
Survey methodology has long allowed models to improve precision without giving them final authority over the estimand. Poststratification and ratio adjustment align survey estimates with known population totals. Other model-assisted estimators use working models, adjusted weights, or modeled cells to exploit auxiliary covariates [13, 17, 23, 30, 31, 35, 43, 45, 48, 51, 60] (Wu and Sitter 2001). Their implementations differ, but the inferential discipline is the same. Model structure can lower variance only because the probability sample still defines the population target and supplies a correction for model error.
Prediction-powered inference gives this correction logic a modern form. In the canonical unweighted PPI setting, a small labeled sample contains gold-standard outcomes and a larger unlabeled file contains covariates, with both samples drawn from a common distribution. An external predictor assigns provisional outcomes to both files [4]. The estimator does not analyze predicted outcomes as if they were observations. It uses predictions on the large file to estimate the predictable component, then uses the labeled sample to estimate and correct prediction error. This construction does not require reweighting the labeled sample; survey weights enter our application separately to account for its sampling designs.
A related design has entered sociology under the term “mixed subjects design”, which applies PPI to combine human responses with LLM predictions [14]. That work establishes a way to use synthetic responses without treating them as a substitute for human data. What remains to be demonstrated is whether the residual-correction logic can improve probability-sample inference for social surveys. This article helps establish the applicability of mixed subjects design in large and public social survey settings, in combination with population censuses.
PPI is one approach within the broader literature on inference with predicted data. Salerno et al. [56] review how these methods address bias and uncertainty and connect them to survey sampling, missing-data analysis, measurement error, and semi-supervised inference. Semi-supervised inference studies how unlabeled covariates can improve estimation of means and regression coefficients [7, 18, 73]. Post-prediction and surrogate-label methods show how downstream inference can be biased when predicted outcomes are used without correction [28, 64]. Orthogonalized estimators similarly separate flexible nuisance prediction from valid inference for target parameters [19]. Recent PPI extensions improve rectification and allocation while extending the framework to harder outcomes, missing covariates, nonuniform sampling, and survey-like LLM allocation problems [5, 22, 29, 39, 77 b] (Li and Ignatiadis 2025; Ye and Yoganarasimhan 2026). We use PPI’s residual-correction formulation to evaluate augmentation of probability surveys with a large auxiliary file.
Predictability as a survey-design quantity
Auxiliary predictions improve precision when they reduce the variation the labeled survey must estimate. A recent literature in sociology asks how much social outcomes can be predicted at all. The predictive turn in social science argues that out-of-sample prediction is a useful complement to explanation because it exposes whether models generalize beyond the data on which they were fit [34, 52, 71] (Verhagen 2022). The Fragile Families Challenge is the landmark benchmark in this literature. In a common-task competition using a rich longitudinal cohort, 160 teams predicted six age-15 outcomes from thousands of earlier-life covariates [57, 58]. Even the best models achieved only modest holdout performance. The out-of-sample R-square is roughly .23 for material hardship and .19 for GPA, with lower values for the remaining outcomes. Subsequent papers showed that intensive feature engineering and flexible algorithms helped, but often only modestly relative to simpler baselines [16, 47, 53, 62].
Those results have usually been interpreted as evidence about social complexity rather than as inputs to survey design. Qualitative follow-up to the Fragile Families Challenge emphasizes contingency, timing, and local context beyond structured covariates [46, 61]. Yan and Rahal [69] distinguish irreducible uncertainty from prediction errors that could shrink with better measurement, while studies across several domains show that predictability varies with the outcome and research setting [8, 24, 42, 76]. These studies take predictability as a quantity that contains theoretical meaning by itself and link it to concepts like social rigidity or contingency.
This article uses predictability as a survey-design quantity. Earlier scholars reported how predictable a range of life outcomes are, and what predictability tells us about society. We ask how these predictions can improve the efficiency of our usual research tasks. A modest may be unpersuasive for individual forecasting but still valuable for survey estimation when predictions can be generated at auxiliary scale.
LLMs as external predictors for survey augmentation
Survey augmentation requires predictions that are external to the outcomes being rectified. Supervised machine learning can supply such predictions when repeated survey modules exist, but many modules are new or redesigned across time. LLMs relax this constraint because a respondent profile can be paired with the question and response scale to create a prediction task without first collecting a large outcome-specific training set. This makes LLMs attractive for augmented surveys, especially when researchers need provisional predictions before enough human labels exist for conventional supervised learning.
A growing literature asks whether LLMs can simulate survey respondents or experiment participants [2, 6, 36, 54]. This literature sheds initial light on the prospect of using LLMs for survey augmentation. LLMs can recover nontrivial structure in human responses across many tasks [15, 38, 70, 75]. These findings imply that LLM outputs may contain enough signal to be useful after correction.
The same literature explains why correction is mandatory. LLM-generated responses can be under-dispersed and demographically misaligned. They can also shift over time and across prompts [11, 12, 25, 26, 40, 59]. A recent high-profile study shows that LLM-generated social science data can appear realistic while failing to reproduce the distributional, associational, and sequential structure needed for social inference [68]. Related work warns that AI-generated content can homogenize social data and threaten survey integrity [65, 67, 74]. Methodological critiques therefore separate exploratory uses such as piloting and design checks from confirmatory population inference, which requires validated and transparent calibration [1, 37, 41, 55]. This failure pattern makes augmentation preferable to replacement. A probability sample can estimate the gap between predicted and observed responses, but only if human responses remain the source of validity.
Method
[Figure 1 about here]
Figure 1 gives the basic workflow of prediction-powered survey augmentation. The survey has both covariates and outcomes of interest. Apart from the survey, we also have an auxiliary file that is much larger than the survey, referred to as the census. The census need not literally include everyone in the population. The census has only covariates but not outcomes. We also have access to an external predictor that takes covariates as input and predicts outcomes. We predict outcomes for both survey respondents and census respondents, for whom outcomes are uncollected. The final estimator uses both pieces: a prediction mean over the large auxiliary file and a residual correction from the labeled survey. We first present this idea in the simplest unweighted mean case, because it makes the correction and efficiency mechanism transparent, and then extend the same logic to weighted surveys and regression coefficients.
Figure 1. Basic workflow of prediction-powered survey augmentation. The survey observes both covariates and outcomes for a relatively small labeled sample. The census-scale auxiliary file is much larger but contains covariates only. A predictor supplies predicted outcomes for both sources. PPI combines the census prediction mean with the survey-estimated residual correction, producing an unbiased estimate that can be more precise when predictions contain useful signal.
Prediction-powered estimation
Suppose the target is the population mean . A labeled survey sample of size contains both covariates and outcomes, . A much larger auxiliary file of size contains only covariates, . For this unweighted example, assume the survey and auxiliary samples are independent random samples from the same population distribution. Let be an external predictor of the outcome from covariates. The ordinary survey estimator is the simple mean
A prediction-only estimator, , is not trustworthy by itself because the predictor can be biased. Prediction-powered inference keeps the prediction mean but adds a human-survey correction for prediction error:
The first term estimates the predictable component of the outcome at auxiliary-file scale. The second term estimates the average residual that the predictor misses in the labeled survey. If the predictor is systematically too high, the residual correction is negative; if it is too low, the correction is positive. Taking expectations gives , so the estimator preserves the same target as the survey mean even when . PPI can use biased predictions because the human survey estimates and corrects such bias.
The correction can often improve precision. Ignoring finite-population corrections and treating the survey and auxiliary samples as independent,
When the auxiliary file is much larger than the survey, as ACS is here, the first term is negligible. PPI is then more efficient than the survey mean whenever the residual varies less than the original outcome . Put plainly, the survey no longer has to estimate the full outcome variation. It only has to estimate the part of the outcome that cannot be predicted.
The gain in precision can be captured intuitively by effective sample size. In the simple mean case, predictive accuracy links directly to that gain. If is linearly calibrated so that the residual variance is approximately , then
A predictor with out-of-sample therefore gives the precision of a survey about times as large, or an 11 percent effective-sample-size gain. An of 0.20 gives a 25 percent gain, and an of 0.40 gives a 67 percent gain. These numbers explain why modest individual-level predictive accuracy can still be useful for survey inference when predictions are cheap, the auxiliary file is large, and collection of an actual data point is expensive.
The same unweighted construction extends from means to regression coefficients and other moment-defined estimands. For a target parameter satisfying
define . The unweighted PPI estimating equation is
This is the score-level analogue of the mean estimator above: prediction-implied scores are averaged over the large auxiliary file, and the labeled survey estimates the residual score correction. When , the equation reduces to the unweighted PPI mean. For linear regression, with an intercept included in . For logistic or other generalized linear models, is the corresponding score.
Social surveys are usually not simple random samples. We therefore use a survey-weighted formulation of PPI. Weighted residual correction has been developed within inverse-probability-weighted PPI, and related prediction-assisted methods accommodate weighted, stratified, and clustered samples [20, 39]. Let denote the labeled probability survey sample, with outcomes , covariates , and design weights . Let denote the auxiliary probability sample, with covariates and weights but without the focal outcome. Define normalized weights and . The weighted mean estimator is
The justification requires both files to represent the same target population, with compatible covariate measurements and weights appropriate to their respective sampling designs. Suppose that the weighted averages consistently estimate the relevant population averages. For a fixed external predictor, the auxiliary term then converges to , while the survey residual term converges to . Their sum converges to , regardless of whether the predictor is unbiased. Because the normalized weights produce ratio estimators, the claim is design consistency rather than exact finite-sample unbiasedness. The predictions can improve precision, but their bias correction remains grounded in observed survey outcomes.
The same correction applies to parameters defined by population moment conditions. The survey-weighted PPI estimator solves
Under the same population-alignment and weighting conditions, both weighted averages of converge to the same population moment, so their difference converges to zero. Consequently, converges to . With uniform convergence of the weighted moments and an identified population solution, the corresponding estimator is consistent for . This argument treats the predictor as fixed and external to the outcomes used for correction.
For coefficients and other score-based estimands, the relevant predictive signal is not individual-level alone. What matters is how much the predictor reduces the variation in the estimating score that the survey must estimate after rectification.
Uncertainty follows a first-order linearization of the estimating equation [10]. Let and let denote its covariance under the sampling designs, holding and the external predictor fixed. The large-sample sandwich variance estimator is
Here is a design-based covariance estimate constructed from score contributions evaluated at the fitted parameter. It is not the literal variance of the solved equation . The online supplement (Section A.1) states the regularity conditions and provides the detailed derivation of Equation (9).
To estimate , the survey contributes residual scores , and the auxiliary sample contributes prediction-implied scores . Taylor linearization or replicate weights must respect each sampling design, including weight normalization, strata, and clusters where applicable. The two covariance contributions add when the sampling designs are independent; otherwise, their cross-covariance must also be included. Treating auxiliary uncertainty as negligible is an approximation justified by its variance contribution relative to that of the survey residual correction, not by file size alone.
The main efficiency quantities are confidence-interval width reduction and effective-sample-size gain. Let be the standard error of the ordinary weighted survey estimator and the standard error of the augmented estimator. The effective-sample multiplier is
and the ESS gain is this multiplier minus one. A 10 percent ESS gain means that the augmented estimator has the precision of a survey-only estimator with roughly 10 percent more observations, holding the design otherwise fixed. We also report confidence-interval width reduction as , where is the 95 percent interval width.
Prediction engines
The benchmark uses three prediction engines, each designed to produce predictions external to the outcome values being rectified. The first is historical supervised learning. For each target outcome, a Super Learner ensemble is trained on historical survey waves and then applied to the target-year survey and auxiliary file [63]. The learner library contains generalized linear models, elastic-net regularization, and random forests. Historical training is strongest when survey instruments are stable across time and when prior waves contain many labeled examples. It is fully out-of-time relative to the target wave, so the predictions used for rectification are not fitted on the target outcomes.
The second regime is within-wave cross-fitting. The target-year survey is partitioned into folds; models are trained on all but one fold and then used to predict the held-out fold. Each labeled unit’s prediction is therefore generated by a model that did not use that unit’s outcome. This reduces direct in-sample overfitting and follows the logic of cross-prediction-powered inference [78]. It is useful when historical training data are unavailable or when temporal drift makes historical prediction suspect. After out-of-fold predictions are obtained for the labeled survey, a final model is trained on the full target-year survey to score the auxiliary ACS file. Extending the fixed-predictor variance argument to this regime requires additional stability conditions and agreement between the fold-trained and full-wave predictors at the first-order scale; held-out predictions alone do not establish these conditions (see the online supplement, Section A.1).
The third regime is LLM respondent simulation. Each respondent’s covariates are rendered as a structured demographic profile, paired with the verbatim survey question and admissible response options, and passed to a language model with a constrained output format. Prompted models include GPT-5-mini, Gemini-2.5-flash, and Qwen3, and one Qwen3 variant is fine-tuned on historical respondent–outcome pairs. Prompting and fine-tuning address the same prediction task, which is a survey response conditional on a respondent’s demographic profile and the question being asked. In-context learning supplies observed examples within the prompt, while supervised fine-tuning learns from observed respondent–outcome pairs. In both approaches, the final predicted response provides auxiliary predictive information for the inference analysis. More implementation details can be found in the online supplement (Section B).
Data
The auxiliary frame is the American Community Survey as harmonized by IPUMS USA. IPUMS USA provides annual ACS public-use microdata for the period since 2000 and harmonizes demographic, household, and economic variables across years. We use ACS 2017 as the auxiliary frame for the NHIS benchmark and ACS 2022 for the GSS benchmark. In the processed files used for the analyses, ACS 2017 contributes roughly 3.19 million person records and ACS 2022 contributes roughly 3.37 million person records. Although we describe ACS as census-scale, the analysis uses ACS public-use microdata rather than a complete census. We treat ACS as a large weighted auxiliary file and align it to the target survey populations as closely as possible using age, household, and person-level restrictions. For the GSS benchmark, the target is U.S. adults in the 2022 GSS universe; for the NHIS benchmark, the target is the civilian noninstitutionalized population represented by NHIS 2017. ACS person weights are used for auxiliary prediction means, while GSS and NHIS design weights define the labeled-survey residual correction.
The opinion outcomes come from the 2022 General Social Survey, a national survey of U.S. adults conducted by NORC. The public-use GSS file contains 3,544 respondents in the 2022 target wave, and historical waves from 1988 through 2021 provide 46,971 observations for training and prompt exemplars [21]. We selected 37 opinion-related outcomes from GSS, with details in the online supplement (Section C.2). These outcomes roughly fall under three categories: political attitudes, social attitudes, and well-being and quality of life.
The health outcomes come from the 2017 National Health Interview Survey, a major U.S. health survey conducted by the National Center for Health Statistics. NHIS is designed to monitor the health of the civilian noninstitutionalized population and to relate health characteristics to demographic and socioeconomic variables [49]. The target wave contains 78,132 observations in the processed benchmark, and pre-2017 waves provide a large historical training corpus of more than six million observations. We selected 46 health-related outcomes, with details in the online supplement (Section C.3). These outcomes roughly fall under three categories: health behavior, health status, and mental health. We use NHIS 2017 because more recent waves contain more missingness in some harmonized demographic covariates needed for ACS alignment.
For both GSS and NHIS, binary outcomes are coded 0/1, ordered categorical outcomes retain their substantive ordering, and count or continuous outcomes are analyzed on their released numeric scale after invalid and not-in-universe codes are removed. The benchmark excludes unordered categorical outcomes with more than two categories. This restriction applies only to the outcomes being predicted and analyzed, not to the covariates; nominal categorical covariates, such as race, are included among the predictors. LLM predictions are parsed as valid response labels and then mapped to the same numeric codes used for survey responses.
[Table 1 about here]
The predictor space is deliberately limited to variables that can be represented in both the target surveys and ACS. The harmonized covariates include sex, age, marital status, race, nativity, citizenship, educational attainment, employment status, occupation, industry, personal income, and poverty status. Occupation and industry are recoded to common census-based categories; income and education are harmonized to comparable ordered categories; and special missing, refused, not-in-universe, and invalid codes are excluded before modeling or estimation. Survey design variables, including person or household weights, strata, and primary sampling units where available, are retained for design-consistent estimation.
The outcome set is intentionally heterogeneous. This heterogeneity is useful because prediction-powered gains should not be expected to be uniform. Health status, activity limitation, and home ownership are plausibly more tied to demographic and socioeconomic covariates than many political opinions or moral attitudes.
Results
The main results proceed in three steps. We first pair GSS and NHIS outcomes with ACS auxiliary covariates to compare expected gains for means and regression coefficients. We then examine when LLM predictors may be useful under label scarcity and how PPI changes hypothesis-testing behavior in small samples. The online supplement (Section E) provides supplementary analyses of repeated-sampling performance and the differences between prediction-only, PPI, and survey-only point estimates.
Predictability and expected efficiency gain using GSS and NHIS data
The first analysis pairs NHIS and GSS outcomes with ACS auxiliary covariates to evaluate expected PPI efficiency gains for two types of inferential targets: survey-weighted outcome means and regression coefficients. For each outcome, we specify a survey-weighted linear regression with an intercept and sex, age, education, and race entered jointly. We examine four coefficients: female relative to male, age in years, the harmonized education scale, and White relative to the non-White reference group. These coefficients describe demographic associations adjusted for the other covariates and allow us to assess whether PPI improves precision for associations as well as means. The prediction engines described in § 3.2 supply the outcome predictions used to augment estimation of these targets.
Table 2 averages the same outcome-estimand panels used in Figure 2 and Figure 3. Each cell gives confidence-interval width reduction followed by effective-sample-size gain. For NHIS means, the historical learner reduces interval width by 5.4 percent and increases effective sample size by 12.2 percent, while the cross learner yields 4.3 percent and 9.8 percent. For GSS means, the corresponding gains are 3.4 percent and 7.5 percent for the historical learner and 3.0 percent and 6.5 percent for the cross learner. The best LLM rows are positive but smaller, with mean-level gains of 1.2 percent and 2.6 percent in NHIS and 1.4 percent and 2.8 percent in GSS. Coefficient targets are less uniform and usually smaller, especially in the GSS supervised rows, because prediction must reduce the variation in the estimating contributions relevant to each coefficient. These gains are modest for any single estimate but meaningful at survey scale, where a widely used public survey effectively becomes larger for many estimands without additional interviews.

Figure 2. Predictability and implied efficiency gains for NHIS 2017 health outcomes. The first column reports gains for survey-weighted means. The four coefficient columns report gains for terms from survey-weighted linear regressions of each outcome on sex, age, education, and race. These terms are female relative to male, age in years, the harmonized education scale, and White relative to the non-White reference category. Health status, functional limitations, hearing, and activity outcomes tend to yield larger gains than weakly predictable behavioral or mental-health intensity measures.

Figure 3. Predictability and implied efficiency gains for GSS 2022 opinion and social-indicator outcomes. The first column reports gains for survey-weighted means. The four coefficient columns report gains for terms from survey-weighted linear regressions of each outcome on sex, age, education, and race. These terms are female relative to male, age in years, the harmonized education scale, and White relative to the non-White reference category. Qwen CoT coefficient results for natenergy are omitted because four complete cases cannot identify five regression parameters. Gains are generally smaller than in NHIS and differ across outcomes and estimands, reflecting the weaker demographic predictability of many attitudes.
[Table 2 about here]
Table 2 summarizes average gains across outcomes. Figure 2 disaggregates these gains by outcome and estimand for NHIS, and Figure 3 presents the corresponding results for GSS.
In NHIS, the strongest mean-level gains occur for health status, functional limitations, and physical-activity measures. The largest mean effective-sample-size gains are roughly 34 to 36 percent for self-rated health, limitation measures, and minutes of vigorous physical activity. Smoking frequency and some mental-health intensity measures are less predictable from the harmonized covariates and yield smaller gains. In GSS, the largest mean gains are about 21 to 23 percent for computer use, marijuana legalization, home ownership, and social class. Home ownership, political ideology, gun ownership, and financial satisfaction carry useful predictive signal, while many social and political attitudes remain weakly predicted by demographic and socioeconomic covariates alone. Mean estimation benefits when predictions reduce the outcome variation left in the survey residual correction. For a regression coefficient, gains depend on reductions in the variation of the estimating contributions relevant to that coefficient. In this benchmark, coefficient-level gains are usually smaller and more heterogeneous than mean-level gains. This distinction matters when the scientific target is an adjusted group difference or association rather than a marginal mean.
[Figure 2 about here]
[Figure 3 about here]
Figure 4 extends this heterogeneity over time. Here we report results based on historical supervised learners. Across repeated survey years, NHIS health-status outcomes have median effective-sample-size gains ranging from about 4 to 25 percent, health behaviors range from about 6 to 14 percent, and mental-health outcomes range from about 4 to 6 percent. GSS political attitudes, social attitudes, and well-being measures mostly range from the low single digits to about 8 percent. This pattern does not imply that attitudes are intrinsically unpredictable. NHIS has a larger training corpus than GSS, which could explain part of the difference in predictability. Richer auxiliary data could also increase predictability.

Figure 4. Median effective-sample-size gains over time by outcome family. Health-related outcomes are generally more predictable from harmonized census-style covariates than political and social attitudes, though the magnitude varies across years and survey domains.
[Figure 4 about here]
Table 2 and Figure 2–Figure 4 summarize expected confidence-interval width reductions and effective-sample-size gains for the available ACS covariates. These comparisons show which outcomes and estimands are most likely to benefit from augmentation in this benchmark. Where expected gains are small, additional observed outcomes or richer auxiliary covariates may be more useful.
| Dataset | Predictor | Mean | Female | Age | Education | White |
| NHIS health | Historical learner | 5.4 / 12.2 | 2.9 / 6.4 | 2.6 / 5.6 | 3.8 / 8.3 | 3.2 / 7.0 |
| NHIS health | Cross learner | 4.3 / 9.8 | 2.1 / 4.6 | 1.8 / 3.8 | 2.6 / 5.8 | 2.3 / 5.0 |
| NHIS health | Best LLM (GPT-5-mini) | 1.2 / 2.6 | 0.1 / 0.3 | 0.2 / 0.4 | 0.3 / 0.6 | 0.3 / 0.7 |
| GSS opinions | Historical learner | 3.4 / 7.5 | 0.6 / 1.3 | 1.0 / 2.1 | 1.3 / 2.7 | 1.2 / 2.4 |
| GSS opinions | Cross learner | 3.0 / 6.5 | 0.5 / 1.0 | 0.5 / 1.0 | 1.0 / 2.1 | 0.6 / 1.2 |
| GSS opinions | Best LLM (GPT-5 CoT) | 1.4 / 2.8 | 0.6 / 1.3 | 0.8 / 1.7 | 0.6 / 1.3 | 1.4 / 3.3 |
Table 2. Average efficiency gains from design-consistent PPI in the outcome-level panels Notes: Each cell reports two percentages, confidence-interval width reduction followed by effective-sample-size gain. Female, age, education, and white refer to coefficients from regressions of each outcome on sex, age, education, and race. The summaries use the same outcome-estimand panels as Figure 2 and Figure 3; the NHIS panel omits the three weekly physical-activity variables excluded from Figure 2. The best LLM row selects the LLM predictor with the largest mean-level effective-sample-size gain within each dataset.
When LLM predictions may still help
The second analysis asks whether LLMs still have a role despite the stronger performance of supervised learners. The limitation of supervised learning is that it needs labeled outcome data, while LLM prediction does not. Historical and cross learners require labeled outcomes from prior waves or within-wave training folds, while an LLM can generate external predictions before a new survey item has accumulated training data.
Figure 5 illustrates this role under limited training data for self-rated health. Cross-fitted supervised learning reaches a prediction-response correlation of about 0.33 with 100 training observations, about 0.46 with 1,000 observations, and about 0.49 with 5,000 observations. GPT-5-mini has a correlation of about 0.443, roughly comparable to the supervised learner near 1,000 training observations. Gemini 2.5 Flash is lower at about 0.422. As the supervised training sample grows, the data-based model continues to improve and eventually surpasses the LLM benchmarks. However, LLMs can be useful in the early stage of a measurement task, before outcome-specific training data are abundant.
[Figure 5 about here]
Small-sample implications for hypothesis testing
Small samples often arise within large surveys when researchers compare groups defined by intersecting social positions or local contexts. Imprecise group-specific estimates can obscure substantively important differences or encourage pooling categories that matter to the research question. Prediction-powered augmentation may improve precision when compatible auxiliary data and observed survey responses are available for the relevant groups. The final analysis illustrates the underlying mechanism with a mean tested against a fixed reference, holding the effect size fixed while sample size varies. Figure 6 is an analytic design illustration, with the setup described in the online supplement (Section D).
Figure 6 describes the non-linearity in the gain for statistical tests. The upper panel translates predictive strength and auxiliary-file size into effective sample size. At a survey sample size of 1,000, survey-only analysis has an effective sample size of 1,000. PPI raises this to about 1,370 under a scenario with predictive and an auxiliary file with 10,000 observations. With predictive , effective sample size rises to about 2,174 when the auxiliary file has 10,000 observations and about 2,463 when it has 100,000 observations. At a survey sample size of 5,000, the same scenarios yield effective sample sizes of about 5,882, 7,143, and 11,628. Given a predictive model and auxiliary data, effective sample size increases roughly linearly with actual sample size.

(a) (a) Effective sample size

(b) (b) Two-sided p-value
Figure 6. Small-sample implications of prediction-powered inference. In this analytic design illustration, PPI has greater consequences for the test conclusion when the survey-only calculation is near the significance threshold, the auxiliary file is large, and predictions substantially reduce residual outcome variation.
On the other hand, it is well known that the p-value does not change linearly with sample size. The lower panel gives the corresponding two-sided test behavior. At a survey sample size of 1,000, survey-only analysis gives . PPI with and lowers the p-value to about 0.064. PPI with lowers it to about 0.020 when and about 0.013 when . Conventional significance is reached at about 1,550 survey cases under survey-only analysis, about 1,150 cases with , about 700 cases with and , and about 650 cases with and . The largest changes in the illustrated test conclusion occur where the survey-only calculation lies near the significance threshold.
[Figure 6 about here]
For subgroup comparisons, predictive performance and the residual correction must be assessed within the relevant groups. Overall accuracy can conceal weak prediction for a small group. The variance of a group difference must incorporate uncertainty in both corrected group estimates and any covariance between them. Where group-specific predictions contain useful signal, augmentation may improve precision without pooling substantively distinct groups.
Discussion
The central claim of this article is deliberately conservative: AI predictions cannot replace social surveys, but they can augment them. AI predictions are too biased to be trusted, but are too informative to ignore. The empirical evidence supports both halves of the claim. Replacement fails because synthetic responses can be biased, and more synthetic observations can make that bias look deceptively precise. Augmentation works because predictions can contain useful information about outcomes, and design-consistent rectification can use that information without trusting it unconditionally. The results identify where survey augmentation is most promising in this benchmark. Gains are generally larger for outcome means than regression coefficients and for health measures than social and political attitudes. Mean estimation benefits when predictions reduce outcome variation left for the survey to estimate; coefficient estimation requires predictive information relevant to the particular association being estimated. Researchers studying adjusted associations should therefore assess coefficient-specific gains rather than infer them from performance in predicting the outcome. Weakly predictable outcomes may offer limited returns from augmentation with the available covariates. Supervised learning provides the strongest gains when labeled training data are available, while LLMs may be useful for new survey items when such data are scarce. The practical question is how much precision the available predictions add for the researcher’s intended estimand.
Our findings also encourage survey agencies and data producers to be prediction-aware. Prediction-aware design means that harmonized links to population-scale covariates and predictive algorithms can be released together with public-use files. Analysts could report both survey-only and augmented estimates, with precision gains evaluated separately for each estimand.
Prediction-aware design can also change fieldwork decisions. Historical predictability profiles can identify which outcomes are likely to benefit from auxiliary prediction and which require direct measurement. During data collection, early interviews can be used to assess whether a focal estimand has reached a target precision; if so, fieldwork can be shifted toward outcomes, modules, or groups where predictive signal is weak. In questionnaire design, agencies might keep a stable core of outcomes measured on all respondents while rotating or splitting additional modules, using PPI to recover precision for outcomes that are easier to predict and reserving scarce interview time for outcomes that are not.
LLMs occupy a specific place in this future. They are useful as flexible external predictors, especially for new modules, rare outcomes, or early fieldwork when historical training data are unavailable. But their role should remain auxiliary. LLM outputs should be audited and compared against simple and supervised baselines. Most importantly, LLM predictions should be tied to a probability sample through rectification before they enter population inference.
PPI depends on the labeled survey and auxiliary file representing the same target population after weighting and population restrictions. Differences in coverage or covariate measurement can prevent the prediction terms from cancelling, and PPI does not itself correct unaccounted-for nonresponse or coverage bias in the survey. Precision gains are also conditional in the sense that weak or poorly scaled predictions can leave residual variance unchanged or increase it, and auxiliary-file uncertainty can offset part of the gain. Inaccurate historical predictions can still be corrected under the population-alignment conditions, but may contribute little precision. Very small labeled samples, subgroup cells, or numbers of independent sampling units can make residual corrections unstable and large-sample confidence intervals unreliable.
The benchmark also has limitations. It uses harmonized demographic and socioeconomic covariates available in ACS. Richer auxiliary data, such as geography, administrative records, neighborhood context, or prior panel responses, could increase gains, especially for opinions and behaviors weakly predicted by demographics alone. But richer data also create privacy, governance, and fairness concerns. The analysis focuses on the United States, where ACS provides unusually large public-use microdata; other countries will differ in the availability of census, registry, or administrative frames. The benchmark reports average gains across selected GSS and NHIS outcomes; other outcomes, surveys, and estimands may differ.
In our opinion, the best path forward for surveys is neither pretending that the data environment has not changed nor abandoning human measurement for synthetic substitutes. The path is integrating human responses, auxiliary frames, and predictions in ways that preserve the inferential virtues of probability sampling. AI can help make surveys cheaper, more powerful, and more adaptive. It should do so by increasing the return on each human interview, not by erasing the need for interviews altogether.
| Data source | Role | Years | Content | Sample size |
| ACS/IPUMS | Auxiliary frame | 2017 | Covariates | 3,190,040 |
| ACS/IPUMS | Auxiliary frame | 2022 | Covariates | 3,373,378 |
| GSS | Target survey | 2022 | 37 outcomes + Covariates | 3,544 |
| GSS | Training corpus | 1988–2021 | Outcomes + Covariates | 46,971 |
| NHIS | Target survey | 2017 | 46 outcomes + Covariates | 78,132 |
| NHIS | Training corpus | 1963–2016 | Outcomes + Covariates | 6,141,631 |
Table 1. Data sources and roles in the benchmark. Notes: Sample sizes refer to processed benchmark files before outcome-specific missingness restrictions. Outcome-specific sample sizes vary because not every question is asked of every respondent and because invalid or not-in-universe codes are treated as missing.

Figure 5. Predictive accuracy for self-rated health. Cross-fitted supervised learning is compared with LLM respondent simulation. LLM predictions contain useful signal, especially relative to very small training samples, but supervised learning improves as labeled survey data accumulate.
Online Supplement
Statistical justification
Derivation of the sandwich variance estimator
This subsection derives Equation (9) as a first-order asymptotic covariance estimator, rather than an exact finite-sample variance identity. The argument uses the estimating-equation linearization underlying complex-survey variance estimation [10]. Its scope is a fixed external predictor and sampling designs whose weighted moments and covariance estimates satisfy the conditions below. It does not require the prediction model to be correctly specified.
Target, notation, and conditions. For design-based inference, let be a sequence of finite target populations and define
The finite-population target is the counterpart of the expectation-defined target in § 3.1 of the main article. Here indexes the sequence of populations and sampling designs, not an additional data source. Write for Equation (8) along this sequence, , and . The subscript means that probability and covariance are taken over the joint sampling designs, conditional on the finite population and external predictor. The parameter dimension is fixed. The proof requires the following conditions:
Both samples refer to the same target population and use compatible covariates. Their positive design weights yield uniformly consistent weighted moment estimates on a neighborhood of the target. Score contributions have finite second moments, and the predictor is held fixed independently of the samples used for correction.
An identified interior root of is selected with . The sample equation is continuously differentiable locally. Its negative derivative converges uniformly to the population negative derivative, which is uniformly continuous locally along the sequence and converges at to a nonsingular matrix .
(iii) For a deterministic scale , a centered design central limit theorem holds:
where is a finite covariance matrix. Any ratio-estimator bias is negligible at this scale.
The scale reflects the sampling designs and relative sample sizes; it need not be the square root of the raw survey sample size.
(iv) A design-based score covariance estimator , evaluated with the fitted parameter held fixed, satisfies . This condition requires a variance procedure valid for the actual sampling designs and normalized weights; positive weights alone do not establish it.
§ 3.1 of the main article explains why the weighted moments have the correct population limit; condition (ii) makes consistency of the selected root explicit. Conditions (ii)–(iv) are the additional assumptions needed for the variance result. The derivation proceeds by linearizing the root, applying the design central limit theorem, and replacing the derivative and score covariance by consistent estimates.
Linearization of the root. Define the integrated negative derivative along the segment joining the target and fitted parameter,
By consistency, that segment lies in the differentiable neighborhood with design probability tending to one. The fundamental theorem of calculus then gives the exact local identity
Uniform derivative convergence, local uniform continuity, and consistency imply .
Because is nonsingular, is invertible with probability tending to one. Hence
The remainder follows because and the central limit theorem implies . Slutsky’s theorem therefore yields
This identifies the first-order asymptotic covariance. With , the same derivative conditions give . Condition (iv) and continuity of matrix inversion imply
Removing the sequence subscripts gives (9). This proves consistency for the first-order covariance under the stated conditions, not exact finite-sample unbiasedness of either the estimator or its estimated variance.
Design covariance of the score contributions. At a fixed parameter, decompose the sample equation as , where
Their design covariance satisfies
Independent sampling designs make the two cross-covariance terms zero. Dependent designs, including designs in which one sample is nested within the other, require their joint covariance to be accounted for. A genuinely complete auxiliary census has no auxiliary sampling. variance conditional on the population; ACS public-use microdata, by contrast, are a sample. Neglecting their contribution is justified only when it is small relative to the residual-score contribution.
Both and use normalized weights and are ratio estimators. Taylor linearization must therefore account for the estimated weight totals in their denominators, as well as strata and clusters where applicable. Alternatively, a design-valid replicate-weight procedure can recompute the two normalized weighted averages in each replicate, holding and the external predictor fixed. Their replicate sums estimate the covariance of the score contributions at that fixed parameter. Computing the variance of after resolving the root within every replicate would instead return zero and is not the construction of .
As a check, for mean estimation and , so is the weighted mean of and . Equation (9) then reduces to the design variance of the prediction mean plus the residual correction, including their covariance when necessary. With independent simple random samples, negligible sampling fractions, and equal weights, this reproduces the unweighted variance expression in Section 3.1 of the main article.
Scope for learned predictors. The argument above conditions on an external predictor. If predictions are learned from the labeled survey, an additional first-order equivalence argument is required. A sufficient condition is
for a suitable fixed-predictor reference equation, together with the corresponding derivative and covariance consistency conditions. Cross-prediction uses fold-trained predictors on both the held-out labeled fold and the auxiliary sample and imposes learner-stability conditions [78]. When a full-wave model instead scores the auxiliary file, equivalence between that model and the fold-trained predictions must also be justified at the first-order scale. Merely excluding a respondent’s own outcome from its prediction does not by itself prove Equation (9) for this learned-predictor regime.
LLM implementation details
This section describes the LLM survey-response prediction procedures, including in-context learning and supervised fine-tuning. Each prediction case combines a respondent’s harmonized demographic profile, a survey question and its admissible responses. The final predicted response is mapped to the survey’s numeric coding for the statistical analysis. Supervised fine-tuning uses GSS and NHIS respondent–outcome examples under this common prediction task.
Experimental settings
In-context examples pair respondent demographic profiles with observed survey responses. The GSS few-shot specification uses 20 historical examples per outcome from 2000–2021. Examples are sampled without replacement from respondents with an observed answer to the focal question, and the same outcome-specific set is used across target respondents. The NHIS few-shot specification uses 15 examples. Prompted chain-of-thought configurations allow intermediate reasoning, while the fine-tuning targets contain only the observed survey response.
Temperature controls variation in generated responses. We use temperature 0.6 for both in-context learning and inference from the fine-tuned model. This setting permits response variation, including intermediate reasoning in the chain-of-thought configurations. GSS negative special codes (e.g., -100 to -97) and NHIS NIU/high missing codes (e.g., 0 or 996--999) are treated as missing in unified files. Survey design weights are normalized in the estimating equation.
Prompt templates and consistency across settings
Shared prediction task. Across in-context learning and supervised fine-tuning, the input combines a standardized demographic profile with a survey question and its valid response options. Each respondent–outcome pair contributes one final predicted response to the analysis. The response is mapped to the numeric coding used for the observed survey outcome. Chain-of-thought prompting allows intermediate reasoning, whereas the supervised fine-tuning targets contain response labels without reasoning annotations. Only the final predicted response enters the statistical analysis.
ICL (few-shot with reasoning): System template
You are simulating a survey respondent. Answer exactly as instructed.
1. Analyze the demographic profile and the question
to determine the most likely response.
2. Then output the final answer on the last line.
### Output format (strict)
PREDICTION: <verbatim answer>
(Use exactly one valid option label/number.
No extra text after the PREDICTION line.)
As you answer, consider how similar respondents answered this question:
[FEW-SHOT EXEMPLARS: K examples for this outcome]
# Assistant target (SFT label)
PREDICTION: <verbatim answer>
SFT training recipe (Qwen3-14B, LoRA)
Model and objective. We fine-tune Qwen3-14B to map (profile, question) to a single-line prediction PREDICTION: .... Training minimizes next-token cross-entropy on the assistant target under a chat-style messages format with System/User/Assistant roles.
Training data construction. Each training instance is one respondent–outcome pair. The System input provides the survey-aware persona framing, enforces a strict no-reasoning instruction, and specifies the output-format constraint. The User input contains a standardized demographic profile and exactly one outcome definition with enumerated valid options. The Assistant target is exactly one line of the form PREDICTION: <verbatim answer> with no additional tokens. To keep the distribution balanced without collapsing outcome coverage, we downsample respondents rather than outcomes and mix GSS and NHIS examples under the unified predictor schema.
Train/validation split. We use year-disjoint splits to mitigate leakage and to make validation reflect a genuine temporal holdout rather than random re-partitioning within the same wave. Concretely, we build the SFT dataset from postprocessed, harmonized survey files and define non-overlapping year windows separately for GSS and NHIS: for GSS, we use train years 2000–2021; for NHIS, we use train years 2000–2016. The benchmark-year inference tasks reported in the main paper are evaluated on held-out survey waves (e.g., the 2022 benchmark when used) and are not included in the SFT training window.
Sample selection. We select respondents before expanding their observed answers into respondent–outcome training examples. This controls the representation of each survey while retaining the questions answered by the selected respondents. Sample selection determines the composition of the training material; the training budget determines how much of that material is processed during fine-tuning.
Mixing GSS and NHIS. Both surveys use a harmonized demographic representation and a common question–response prediction task. Pooling was intended to draw on the generalization capacity of LLMs and allow useful response patterns to transfer across survey domains. We control source representation at the respondent level so that the size of the NHIS respondent pool does not dominate the training mixture. We did not compare pooled and survey-specific models, so their relative predictive performance in this application remains unknown.
LoRA configuration. We use parameter-efficient fine-tuning with LoRA, with a typical starting point of rank 32 and learning rate , training for 1 epoch (formatting-focused tasks can overfit quickly). We set max sequence length to 2048 (increasing it if prompts truncate) and use AdamW as the optimizer.
Response format and inference. The supervised fine-tuning targets consist of observed survey response labels without reasoning annotations. Answer-only instructions request responses in this format. Prompted chain-of-thought configurations allow intermediate reasoning. For both approaches, only the final predicted response is used in the statistical analysis. Fine-tuned-model inference uses temperature 0.6, as described in Section B.1.
Variable lists
Harmonized predictors
| Variable | Category | Original question |
| survey_year | Survey metadata | Survey year |
| sex | Demographics | Respondent sex |
| age | Demographics | Age in years |
| marital_status | Household / family | Marital status |
| race | Race / ethnicity | Race |
| born_in_us | Nativity / citizenship | Born in United States |
| citizenship | Nativity / citizenship | U.S. citizenship status |
| education | Education | Educational attainment |
| employment_status | Labor market | Employment status |
| occupation | Occupation / industry | Occupation code |
| sei | Occupational status | |
| prestige | Occupational status | |
| industry | Occupation / industry | Industry code |
| personal_income | Income / poverty | Personal income |
| poverty_status | Income / poverty | Poverty status |
| pid | Identifiers | |
| year_pid | Identifiers | |
| person_number | Identifiers | Person number within household |
| respondent_id | Identifiers | Respondent identifier |
| survey_weight | Survey design (weights) | |
| Variable | Category | Original question |
perweight | Survey design (weights) | |
sampweight | Survey design (weights) | |
fweight | Survey design (weights) | |
hhweight | Survey design (weights) | |
longweight | Survey design (weights) | |
partweight | Survey design (weights) | |
strata | Survey design (strata/PSU) | Sampling strata |
psu | Survey design (strata/PSU) | Primary sampling unit |
Table 3.
GSS outcomes
| Variable | Category | Original question |
polviews | Political Attitudes | We hear a lot of talk these days about liberals and conservatives. I’m going to show you a seven-point scale on which the political views that people might hold are arranged from extremely liberal—point 1—to extremely conservative—point 7. Where would you place yourself on this scale? |
Table 4.
| Variable | Category | Original question |
| natspac | Federal Government Spending | We are faced with many problems in this country, none of which can be solved easily or inexpensively. I’m going to name some of these problems, and for each one I’d like you to tell me whether you think we’re spending too much money on it, too little money, or about the right amount. Space exploration program. |
| natenvir | Federal Government Spending | We are faced with many problems in this country, none of which can be solved easily or inexpensively. I’m going to name some of these problems, and for each one I’d like you to tell me whether you think we’re spending too much money on it, too little money, or about the right amount. Improving and protecting the environment. |
| natheal | Federal Government Spending | We are faced with many problems in this country, none of which can be solved easily or inexpensively. I’m going to name some of these problems, and for each one I’d like you to tell me whether you think we’re spending too much money on it, too little money, or about the right amount. Improving and protecting the nation’s health. |
| natcity | Federal Government Spending | We are faced with many problems in this country, none of which can be solved easily or inexpensively. I’m going to name some of these problems, and for each one I’d like you to tell me whether you think we’re spending too much money on it, too little money, or about the right amount. Solving the problems of the big cities. |
Table 5.
| Variable | Category | Original question |
| natcrime | Federal Government Spending | We are faced with many problems in this country, none of which can be solved easily or inexpensively. I’m going to name some of these problems, and for each one I’d like you to tell me whether you think we’re spending too much money on it, too little money, or about the right amount. Halting the rising crime rate. |
| natdrug | Federal Government Spending | We are faced with many problems in this country, none of which can be solved easily or inexpensively. I’m going to name some of these problems, and for each one I’d like you to tell me whether you think we’re spending too much money on it, too little money, or about the right amount. Dealing with drug addiction. |
| nateduc | Federal Government Spending | We are faced with many problems in this country, none of which can be solved easily or inexpensively. I’m going to name some of these problems, and for each one I’d like you to tell me whether you think we’re spending too much money on it, too little money, or about the right amount. Improving the nation’s education system. |
| natrace | Federal Government Spending | We are faced with many problems in this country, none of which can be solved easily or inexpensively. I’m going to name some of these problems, and for each one I’d like you to tell me whether you think we’re spending too much money on it, too little money, or about the right amount. Improving the conditions of Black people. |
Table 6.
| Variable | Category | Original question |
| natarms | Federal Government Spending | We are faced with many problems in this country, none of which can be solved easily or inexpensively. I’m going to name some of these problems, and for each one I’d like you to tell me whether you think we’re spending too much money on it, too little money, or about the right amount. The military, armaments and defense. |
| natsci | Federal Government Spending | We are faced with many problems in this country, none of which can be solved easily or inexpensively. I’m going to name some of these problems, and for each one I’d like you to tell me whether you think we’re spending too much money on it, too little money, or about the right amount. Supporting scientific research. |
| natenrgy | Federal Government Spending |
Table 1 (PDF p. 54).
| Variable | Category | Original question |
| eqwlth | Inequality and Income | Some people think that the government in Washington ought to reduce the income differences between the rich and the poor, perhaps by raising the taxes of wealthy families or by giving income assistance to the poor. Others think that the government should not concern itself with reducing this income difference between the rich and the poor. Here is a card with a scale from 1 to 7. Think of a score of 1 as meaning that the government ought to reduce the income differences between rich and poor, and a score of 7 meaning that the government should not concern itself with reducing income differences. What score between 1 and 7 comes closest to the way you feel? |
| cappun | Capital Punishment | Do you favor or oppose the death penalty for persons convicted of murder? |
| gunlaw | Guns | Would you favor or oppose a law which would require a person to obtain a police permit before he or she could buy a gun? |
| grass | Marijuana | Do you think the use of marijuana should be made legal or not? |
| reliten | Religion | Would you call yourself a strong (PREFERENCE NAMED IN RELIG) or a not very strong (PREFERENCE NAMED IN RELIG)? |
Table 1 (PDF p. 55).
| Variable | Category | Original question |
| affrmact | Affirmative Action | Some people say that because of past discrimination, Black people should be given preference in hiring and promotion. Others say that such preference in hiring and promotion of Black people is wrong because it discriminates against whites. What about your opinion – are you for or against preferential hiring and promotion of Black people? |
| happy | Happiness | Taken all together, how would you say things are these days–would you say that you are very happy, pretty happy, or not too happy? |
| hapmar | Happiness / Marital Status | Taking things all together, how would you describe your marriage? Would you say that your marriage is very happy, pretty happy, or not too happy? |
| health | Health | Would you say your own health, in general, is excellent, good, fair, or poor? |
| life | Life Outlook | In general, do you find life exciting, pretty routine, or dull? |
| trust | Social Trust | Generally speaking, would you say that most people can be trusted or that you can’t be too careful in dealing with people? |
| satjob | Job Satisfaction | On the whole, how satisfied are you with the work you do–would you say you are very satisfied, moderately satisfied, a little dissatisfied, or very dissatisfied? |
| satfin | Income / Satisfaction | We are interested in how people are getting along financially these days. So far as you and your family are concerned, would you say that you are pretty well satisfied with your present financial situation, more or less satisfied, or not satisfied at all? |
| class | Social Class | |
| Variable | Category | Original question |
| parsol | Parents / Social Mobility | Compared to your parents when they were the age you are now, do you think your own standard of living now is much better, somewhat better, about the same, somewhat worse, or much worse than theirs was? |
| abany | Abortion | Please tell me whether or not you think it should be possible for a pregnant woman to obtain a legal abortion if the woman wants it for any reason. |
| sexeduc | Education / Sex | Would you be for or against sex education in the public schools? |
| premarsx | Sex | If a man and woman have sex relations before marriage, do you think it is always wrong, almost always wrong, wrong only sometimes, or not wrong at all? |
| fear | Crime / Residence | Is there any area right around here–that is, within a mile–where you would be afraid to walk alone at night? |
| owngun | Guns | Do you happen to have in your home (IF HOUSE: or garage) any guns or revolvers? |
| god | Religion | Which statement comes closest to expressing what you believe about God? |
| compuse | Computers and Internet / Information Society | Do you personally ever use a computer at home, at work, or at some other location? |
| goodlife | Social Mobility / ISSP Social Inequality | The way things are in America, people like me and my family have a good chance of improving our standard of living – do you agree or disagree? |
| sexornt | Sexual Orientation / High Risk Behaviors | Which of the following best describes you? |
Table 7 (PDF p. 56).
| Variable | Category | Original question |
| dwelown | Residence / Dwelling | Do you/Does your family own your (home/apartment), pay rent, or what? |
Table 8.
NHIS outcomes
| Variable | Category | Original question |
| health | Health Status | Would you say [person’s] health in general is excellent, very good, good, fair, or poor? |
| bmicalc | Physical Health | Body Mass Index calculated from publicly released height and weight variables |
| wldayr | Health Impact | During the past 12 months, about how many days did illness or injury keep [person] in bed more than half of the day? |
| latotal | Functional Status | Is [person] limited in any way in any activities because of an impairment or health problem? |
| lany | Functional Status | Is [person] limited in any way in any activities because of physical, mental, or emotional problems? |
| flany | Functional Status | Does [person] have any functional limitations? |
| fanylct | Family Health | Number of family members with any limitation |
| alc1yr | Substance Use | In [person’s] entire life, has [he/she] had at least 12 drinks of any type of alcoholic beverage in any one year? |
| alclife | Substance Use | In [person’s] entire life, has [he/she] had at least 12 drinks of any type of alcoholic beverage? |
| alc5upyr | Substance Use | In the past year, how many days did [person] have 5 or more drinks of any alcoholic beverage? |
| alcamt | Substance Use | In the past year, on the days when [person] drank, on the average, how many drinks did [he/she] have? |
| Variable | Category | Original question |
| smokev | Substance Use | Has [person] smoked at least 100 cigarettes in [his/her] entire life? |
| smokagereg | Substance Use | How old was [person] when [he/she] first started smoking cigarettes fairly regularly? |
| cigdaymo | Substance Use | During the past 30 days, on how many days did [person] smoke cigarettes? |
| cigsday | Substance Use | On the average, how many cigarettes does [person] smoke per day? |
| cigsday1 | Substance Use | On the average, how many cigarettes does [person] smoke per day? (Daily smokers) |
| cigsday2 | Substance Use | On the average, how many cigarettes does [person] smoke per day? (Some day smokers) |
| smokestatus2 | Substance Use | Cigarette smoking recode combining current detailed smoking status, former smoking status, and never smoking status |
| smokfreqnow | Substance Use | Does [person] now smoke cigarettes every day, some days, or not at all? |
| smklsev | Substance Use | Has [person] ever used smokeless tobacco? |
| smklsfreqnow | Substance Use | How often does [person] now use smokeless tobacco? |
| ecigev | Substance Use | Has [person] ever used an e-cigarette, even once or twice? |
| eciged | Substance Use | Does [person] now use e-cigarettes every day, some days, or not at all? |
| mod10dmin | Physical Activity | About how long does [person] do these moderate activities each time? |
| mod10fwk | Physical Activity | In a typical week, on how many days does [person] do moderate activities for at least 10 minutes? |
| vig10dmin | Physical Activity | About how long does [person] do these vigorous activities each time? |
| Variable | Category | Original question |
vig10fwk | Physical Activity | In a typical week, on how many days does [person] do vigorous activities for at least 10 minutes? |
strongfwk | Physical Activity | In a typical week, on how many days does [person] do physical activities specifically designed to strengthen muscles? |
hearing | Sensory Function | Which statement best describes [person’s] hearing (without a hearing aid): excellent, good, a little trouble, moderate trouble, a lot of trouble, or deaf? |
vdifamount | Sensory Function | How much difficulty does [person] have with vision? |
hrsleep | Health Behaviors | On average, how many hours of sleep does [person] get in a 24-hour period? |
tirefreq3mo | Health Status | During the past 3 months, how often did [person] feel very tired or exhausted? |
aeffort | Mental Health | In the past 30 days, how often did [person] feel that everything was an effort? |
afeelint1mo | Mental Health | During the past 30 days, how much did these feelings interfere with [person’s] life? |
ahopeless | Mental Health | In the past 30 days, how often did [person] feel hopeless? |
anervous | Mental Health | In the past 30 days, how often did [person] feel nervous? |
arestless | Mental Health | In the past 30 days, how often did [person] feel restless or fidgety? |
asad | Mental Health | In the past 30 days, how often did [person] feel sad? |
aworthless | Mental Health | In the past 30 days, how often did [person] feel worthless? |
worfreq | Mental Health | How often does [person] feel worried, nervous, or anxious? |
worrx | Mental Health | Does [person] take medication for worried, nervous, or anxious feelings? |
| Variable | Category | Original question |
| worfeelelvl | Mental Health | The last time [person] felt worried, nervous, or anxious, how would [he/she] describe the level of these feelings? |
| depfreq | Mental Health | How often does [person] feel depressed? |
| deprx | Mental Health | Does [person] take medication for depression? |
| depfeelelvl | Mental Health | The last time [person] felt depressed, how would [he/she] describe the level of depression? |
| sawment | Healthcare Utilization | During the past 12 months, did [person] see or talk to a mental health professional such as a psychiatrist, psychologist, psychiatric nurse, or clinical social worker about [his/her] health? |
Table 9 (PDF p. 58).
Small-sample hypothesis-testing illustration
Figure 6 is an analytic design illustration rather than a Monte Carlo simulation over generated outcomes. The calculation considers a standardized outcome with variance equal to one and a true mean difference of standard deviations. Survey sample size ranges from 50 to 5,000, and the two-sided testing level is . The survey-only baseline has no auxiliary prediction signal, represented as . The three PPI scenarios use with , with , and with .
The calculation uses the large-sample variance approximation for a PPI mean,
The first term is the contribution of the auxiliary prediction mean, and the second term is the survey-estimated residual component. For the survey-only baseline, and , so the variance reduces to . Effective sample size is defined as
because the standardized survey-only mean has variance . The test statistic under the stylized alternative is , and the reported two-sided p-value is
This setup is intentionally simple. It translates a fixed amount of predictive signal and auxiliary-file size into effective sample size and hypothesis-test behavior, holding the target effect size fixed so that Figure 6 isolates the nonlinear consequences of precision gain across survey sample sizes.
Supplementary results
Pseudo-census simulation
This supplementary analysis directly evaluates bias and repeated-sampling variability against a known finite-population target, complementing the point-estimate comparisons and expected efficiency gains in the survey applications. Figure E1 uses the NHIS 2017 validation wave as a pseudo-census and compares four estimators across repeated 10 percent subsamples. Here, predictions come from historical supervised learning, a learner trained on all data in NHIS prior to 2017. Panel A reports standardized bias. Survey-only and design-consistent PPI estimates are close to zero. Across outcomes, the largest absolute standardized bias is about 0.009 for survey-only estimation and 0.006 for design-consistent PPI, compared with about 0.049 for silicon-only estimation and 0.104 for conventional unweighted PPI. The results suggest that more predicted observations do not remove model bias, and the residual correction must respect the survey design.

Figure E1. Pseudo-census Monte Carlo using NHIS 2017. The full wave is treated as a finite population; repeated 10 percent subsamples mimic smaller surveys. In panel A, circles denote design-consistent PPI, triangles survey-only estimation, squares unweighted PPI, and diamonds prediction-only estimation. The dashed line indicates zero bias. Panel B reports percentage reductions in Monte Carlo standard deviation relative to survey-only estimation.
Panel B adds the precision result that compares the Monte Carlo standard deviation of PPI estimates and survey-only estimates. Design-consistent PPI reduces Monte Carlo sampling variability by about 32 percent for days of muscle-strengthening activity, 23 percent for days of vigorous activity, and 20 percent for days of moderate activity. The gain is much smaller for weakly predicted outcomes. The simulation demonstrates the benefit of applying PPI. Compared with survey-only estimates, design-consistent PPI estimates are similarly centered but more precise. Prediction-only estimates can be biased and therefore should not be used to infer the population directly.
Prediction-only and corrected point estimates
Figure E2 shows how predictions change estimated outcome levels in the GSS and NHIS applications. Figure E1 evaluates bias and sampling variability against a known population truth, while Table 2 and Figure 2 and Figure 3 report precision gains. Comparing prediction-only and corrected PPI estimates with observed survey means illustrates the difference between replacing survey responses and augmenting them with predictions. Each point reports the difference from the survey mean in outcome standard deviations. Positive values indicate higher estimates; negative values indicate lower estimates. Zero marks agreement with the observed survey mean, rather than a known population truth.

Figure E2. Prediction-only and residual-corrected outcome means relative to observed survey means. The horizontal axis reports the estimate minus the survey mean, divided by the survey standard deviation of the outcome. All means in this descriptive comparison are unweighted and use the corresponding valid cases. Historical and cross-fitted supervised prediction means are computed over ACS; their corrected PPI estimates appear as green circles. The LLM series reports prediction-only means for survey respondents with valid LLM predictions. The dashed line marks agreement with the observed survey mean.
The corrected PPI estimates differ relatively little from the survey means across outcomes, with a median absolute standardized difference of about 0.04 and a 90th percentile of about 0.24. Prediction-only estimates sometimes differ substantially more: the largest absolute differences for the LLM series are about 1.37 outcome standard deviations in NHIS and 0.98 in GSS. PPI does not require the corrected estimate to equal the survey mean. Its remaining difference reflects the difference between the auxiliary-file and survey prediction means after the survey’s average prediction error has been removed.
References
- [1]Abdurahman, Suhaib, Alireza Salkhordeh Ziabarı, Alexander K. Moore, Daniel M. Bartels, and Morteza Dehghani. 2025. “A Primer for Evaluating Large Language Models in Social-Science Research.” Advances in Methods and Practices in Psychological Science 8(2). doi:10.1177/25152459251325174.DOI
- [2]Aher, Gati V., Rosa I. Arriaga, and Adam Tauman Kalai. 2023. “Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies.” In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 337–371. PMLR.arxiv.org/abs/2208.10264
- [3]Alvero, A. J., Dustin Stoltz, Oscar Stuhler, and Marshall Taylor. 2026. “Generative AI in Sociological Research: State of the Discipline.” Sociological Science 13:45–62. doi:10.15195/v13.a3.DOI
- [4]Angelopoulos, Anastasios N., Stephen Bates, Clara Fannjiang, Michael I. Jordan, and Tijana Zrnic. 2023. “Prediction-Powered Inference.” Science 382(6671):669–674. doi:10.1126/science.adi6000.DOI
- [5]Angelopoulos, Anastasios N., John C. Duchi, and Tijana Zrnic. 2024. “PPI++: Efficient Prediction-Powered Inference.” doi:10.48550/arXiv.2311.01453. ArXiv:2311.01453.arxiv.org/abs/2311.01453
- [6]Argyle, Lisa P., Ethan C. Busby, Nancy Fulda, Joshua Gubler, Christopher Rytting, and David Wingate. 2023. “Out of One, Many: Using Language Models to Simulate Human Samples.” Political Analysis 31(3):337–351. doi:10.1017/pan.2023.2.DOI
- [7]Azriel, David, Lawrence D. Brown, Michael Sklar, Richard Berk, Andreas Buja, and Linda Zhao. 2022. “Semi-Supervised Linear Regression.” Journal of the American Statistical Association 117(540):2238–2251. doi:10.1080/01621459.2021.1910230.
- [8]Badolato, Luca, Ari Gabriel Decter-Frain, Nicolas Irons, Maria Laura Miranda, Erin Walk, Elnura Zhalieva, Monica Alexander, Ugofilippo Basellini, and Emilio Zagheni. 2023. “The Limits of Predicting Individual-Level Longevity.” Working Paper WP-2023-008, Max Planck Institute for Demographic Research. doi:10.4054/MPIDR-WP-2023-008. Revised January 2024.
- [9]Bail, Christopher A. 2024. “Can Generative AI Improve Social Science?” Proceedings of the National Academy of Sciences 121(21):e2314021121. doi:10.1073/pnas.2314021121.DOI
- [10]Binder, David A. 1983. “On the Variances of Asymptotically Normal Estimators from Complex Surveys.” International Statistical Review 51(3):279–292. doi:10.2307/1402588.DOI
- [11]Bisbee, James, Joshua D. Clinton, Cassy Dorff, Brenton Kenkel, and Jennifer M. Larson. 2024. “Synthetic Replacements for Human Survey Data? The Perils of Large Language Models.” Political Analysis 32(4):401–416. doi:10.1017/pan.2024.5.DOI
- [12]Boelaert, Julien, Samuel Coavoux, Étienne Ollion, Ivaylo Petev, and Patrick Präg. 2025. “Machine Bias. How Do Generative Language Models Answer Opinion Polls?” Sociological Methods & Research 54(3):1156–1196. doi:10.1177/00491241251330582.DOI
- [13]Breidt, F. Jay and Jean D. Opsomer. 2017. “Model-Assisted Survey Estimation with Modern Prediction Techniques.” Statistical Science 32(2):190–205. doi:10.1214/16-STS589.DOI
- [14]Broska, David, Michael Howes, and Austin van Loon. 2025. “The Mixed Subjects Design: Treating Large Language Models as Potentially Informative Observations.” Sociological Methods & Research 54(3):1074–1109. doi:10.1177/00491241251326865.DOI
- [15]Cao, Yong, Haijiang Liu, Arnav Arora, Isabelle Augenstein, Paul Röttger, and Daniel Hershcovich. 2025. “Specializing Large Language Models to Simulate Survey Response Distributions for Global Populations.” In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 3141–3154. Association for Computational Linguistics. doi:10.18653/v1/2025.naacl-long.162.DOI
- [16]Carnegie, Nicole Bohme and James Wu. 2019. “Variable Selection and Parameter Tuning for BART Modeling in the Fragile Families Challenge.” Socius 5:2378023119825886. doi:10.1177/2378023119825886.DOI
- [17]Cassel, Carl M., Carl-Erik Särndal, and Jan H. Wretman. 1976. “Some Results on Generalized Difference Estimation and Generalized Regression Estimation for Finite Populations.” Biometrika 63(3):615–620. doi:10.1093/biomet/63.3.615.DOI
- [18]Chakrabortty, Abhishek and T. Tony Cai. 2018. “Efficient and Adaptive Linear Regression in Semi-Supervised Settings.” The Annals of Statistics 46(4):1541–1572. doi:10.1214/17-AOS1594.DOI
- [19]Chernozhukov, Victor, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. 2018. “Double/Debiased Machine Learning for Treatment and Structural Parameters.” The Econometrics Journal 21(1):C1–C68. doi:10.1111/ectj.12097.DOI
- [20]Datta, Jyotishka and Nicholas G. Polson. 2025. “Prediction-Powered Inference with Inverse Probability Weighting.” doi:10.48550/arXiv.2508.10149. ArXiv:2508.10149, version 2 (March 2026).DOI
- [22]Demirel, Ilker, Ahmed Alaa, Anthony Philippakis, and David Sontag. 2024. “Prediction-Powered Generalization of Causal Inferences.” In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 10385–10408. PMLR.arxiv.org/abs/2406.02873
- [23]Deville, Jean-Claude and Carl-Erik Särndal. 1992. “Calibration Estimators in Survey Sampling.” Journal of the American Statistical Association 87(418):376–382. doi:10.1080/01621459.1992.10475217.DOI
- [24]Domingue, Benjamin W., Charles Rahal, Jessica Faul, Jeremy Freese, Klint Kanopka, Alexandros Rigos, Ben Stenhaug, and Ajay Shanker Tripathi. 2025. “The InterModel Vigorish (IMV) as a Flexible and Portable Approach for Quantifying Predictive Accuracy with Binary Outcomes.” PLOS ONE 20(3):e0316491. doi:10.1371/journal.pone.0316491.DOI
- [25]Dominguez-Olmedo, Ricardo, Moritz Hardt, and Celestine Mendler-Dünner. 2024. “Questioning the Survey Responses of Large Language Models.” arXiv doi:10.48550/arXiv.2306.07951.
- [26]Durmus, Esin, Karina Nguyen, Thomas I. Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, Liane Lovitt, Sam McCandlish, Orowa Sikder, Alex Tamkin, Janel Thamkul, Jared Kaplan, Jack Clark, and Deep Ganguli. 2023. “Towards Measuring the Representation of Subjective Global Opinions in Language Models.” doi:10.48550/arXiv.2306.16388. ArXiv:2306.16388.DOI
- [27]Dutwin, David and Trent D. Buskirk. 2021. “Telephone Sample Surveys: Dearly Beloved or Nearly Departed? Trends in Survey Errors in the Era of Declining Response Rates.” Journal of Survey Statistics and Methodology 9(3):353–380. doi:10.1093/jssam/smz044.
- [28]Egami, Naoki, Michael Hinck, Brandon M. Stewart, and Hanying Wei. 2023. “Using Imperfect Surrogates for Downstream Inference: Design-Based Supervised Learning for Social Science Applications of Large Language Models.” arXiv doi:10.48550/arXiv.2306.04746.
- [29]Fan, Shuxian, Adam Visokay, Kentaro Hoffman, Stephen Salerno, Li Liu, Jeffrey T. Leek, and Tyler H. McCormick. 2024. “From Narratives to Numbers: Valid Inference Using Language Model Predictions from Verbal Autopsy Narratives.” doi:10.48550/arXiv.2404.02438. ArXiv:2404.02438.DOI
- [30]Gelman, Andrew and Thomas C. Little. 1997. “Poststratification into Many Categories Using Hierarchical Logistic Regression.” Survey Methodology 23(2):127–135.
- [31]Ghitza, Yair and Andrew Gelman. 2013. “Deep Interactions with MRP: Election Turnout and Voting Patterns Among Small Electoral Subgroups.” American Journal of Political Science 57(3):762–776. doi:10.1111/ajps.12004.DOI
- [32]Groves, Robert M. 2006. “Nonresponse Rates and Nonresponse Bias in Household Surveys.” Public Opinion Quarterly 70(5):646–675. doi:10.1093/poq/nfl033.DOI
- [33]Groves, Robert M. and Emilia Peytcheva. 2008. “The Impact of Nonresponse Rates on Nonresponse Bias: A Meta-Analysis.” Public Opinion Quarterly 72(2):167–189. doi:10.1093/poq/nfn011.DOI
- [34]Hofman, Jake M., Duncan J. Watts, Susan Athey, Filiz Garip, Thomas L. Griffiths, Jon Kleinberg, Helen Margetts, Sendhil Mullainathan, Matthew J. Salganik, and Simine Vazire. 2021. “Integrating Explanation and Prediction in Computational Social Science.” Nature 595:181–188. doi:10.1038/s41586-021-03659-0.DOI
- [35]Holt, D. and T. M. F. Smith. 1979. “Post Stratification.” Journal of the Royal Statistical Society. Series A (General) 142(1):33–46. doi:10.2307/2344652.DOI
- [36]Horton, John J., Apostolos Filippas, and Benjamin S. Manning. 2023. “Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus?” Working Paper 31122, National Bureau of Economic Research. doi:10.3386/w31122. Revised February 2026.DOI
- [37]Hullman, Jessica, David Broska, Huaman Sun, and Aaron Shaw. 2026. “This Human Study Did Not Involve Human Subjects: Validating LLM Simulations as Behavioral Evidence.” doi:10.48550/arXiv.2602.15785. ArXiv:2602.15785.DOI
- [38]Kim, Junsol and Byungkyu Lee. 2023. “AI-Augmented Surveys: Leveraging Large Language Models and Surveys for Opinion Prediction.” arXiv doi:10.48550/arXiv.2305.09620.
- [39]Kluger, Dan M., Kerri Lu, Tijana Zrnic, Sherrie Wang, and Stephen Bates. 2025. “Prediction-Powered Inference with Imputed Covariates and Nonuniform Sampling.” doi:10.48550/arXiv.2501.18577. ArXiv:2501.18577.DOI
- [40]Kozlowski, Austin C. and James A. Evans. 2025. “Simulating Subjects: The Promise and Peril of Artificial Intelligence Stand-Ins for Social Agents and Interactions.” Sociological Methods & Research 54(3):1017–1073. doi:10.1177/00491241251337316.DOI
- [41]Krsteski, Stefan, Giuseppe Russo, Serina Chang, Robert West, and Kristina Gligorić. 2025. “Valid Survey Simulations with Limited Human Data: The Roles of Prompting, Fine-Tuning, and Rectification.” doi:10.48550/arXiv.2510.11408. ArXiv:2510.11408.arxiv.org/abs/2510.11408
- [42]Kuikka, Sanni. 2024. “The (Un)Predictability of Early (Un)Employment: A Machine Learning Approach.” Socius 10:23780231241286655. doi:10.1177/23780231241286655.DOI
- [43]Lax, Jeffrey R. and Justin H. Phillips. 2009. “How Should We Estimate Public Opinion in the States?” American Journal of Political Science 53(1):107–121. doi:10.1111/j.1540-5907.2008.00360.x.DOI
- [45]Little, Roderick J. A. 1993. “Post-Stratification: A Modeler’s Perspective.” Journal of the American Statistical Association 88(423):1001–1012. doi:10.1080/01621459.1993.10476368.DOI
- [46]Lundberg, Ian, Rachel Brown-Weinstock, Susan Clampet-Lundquist, Sarah Pachman, Timothy J. Nelson, Vicki Yang, Kathryn Edin, and Matthew J. Salganik. 2024. “The Origins of Unpredictability in Life Outcome Prediction Tasks.” Proceedings of the National Academy of Sciences 121(24):e2322973121. doi:10.1073/pnas.2322973121.DOI
- [47]McKay, Stephen. 2019. “When 4 ≈ 10,000: The Power of Social Science Knowledge in Predictive Performance.” Socius 5:2378023118811774. doi:10.1177/2378023118811774.DOI
- [48]Mozer, Reagan. 2026. “PPI Is the Difference Estimator: Recognizing the Survey Sampling Roots of Prediction-Powered Inference.” doi:10.48550/arXiv.2603.19160. ArXiv:2603.19160.DOI
- [49]National Center for Health Statistics. 2017. “National Health Interview Survey Description [2017].” Technical report, National Center for Health Statistics, Hyattsville, MD. National Health Interview Survey Public Use Data Release.
- [51]Park, David K., Andrew Gelman, and Joseph Bafumi. 2004. “Bayesian Multilevel Estimation with Poststratification: State-Level Estimates from National Polls.” Political Analysis 12(4):375–385. doi:10.1093/pan/mph024.DOI
- [52]Rahal, Charles, Mark Verhagen, and David Kirk. 2024. “The Rise of Machine Learning in the Academic Social Sciences.” AI & Society 39:799–801. doi:10.1007/s00146-022-01540-w.
- [53]Rigobon, Daniel E., Eaman Jahani, Yoshihiko Suhara, Khaled AlGhোনeim, Abdulaziz Alghunaim, Alex Pentland, and Abdullah Almaatouq. 2019. “Winning Models for Grade Point Average, Grit, and Layoff in the Fragile Families Challenge.” Socius 5:2378023118820418. doi:10.1177/2378023118820418.DOI
- [54]Rosenbusch, Hannes, Claire E. Stevenson, and Han L. J. van der Maas. 2023. “How Accurate Are GPT-3’s Hypotheses About Social Science Phenomena?” Digital Society 2(2):26. doi:10.1007/s44206-023-00054-2.DOI
- [55]Rossi, Luca, Katherine Harrison, and Irina Shklovski. 2024. “The Problems of LLM-Generated Data in Social Science Research.” Sociologica 18(2):145–168. doi:10.6092/issn.1971-8853/19576.DOI
- [56]Salerno, Stephen, Kentaro Hoffman, Awan Afiaz, Anna Neufeld, Tyler H. McCormick, and Jeffrey T. Leek. 2025. “Do We Really Even Need Data? A Modern Look at Drawing Inference with Predicted Data.” doi:10.48550/arXiv.2512.05456. ArXiv:2512.05456.DOI
- [57]Salganik, Matthew J., Ian Lundberg, Alexander T. Kindel, Caitlin E. Ahearn, Khaled Al-Ghoneim, Abdullah Almaatouq, Drew M. Altschul, Jennie E. Brand, Nicole Bohme Carnegie, Ryan J. Compton, Debanjan Datta, Thomas Davidson, Anna Filippova, Connor Gilroy, Brian J. Goode, Eaman Jahani, Ridhi Kashyap, Antje Kirchner, Stephen McKay, Allison C. Morgan, Alex Pentland, Kivan Polimis, Louis Raes, Daniel E. Rigobon, Claudia V. Roberts, Diana M. Stanescu, Yoshihiko Suhara, Adaner Usmani, Erik H. Wang, Muna Adem, Ahmed Alhajri, Bedoor AlShebli, Rediet Amin, Ryan B. Amos, Lisa P. Argyle, Lily Baer-Bositis, Arnaud Béjar, Judith Borja, Rutuja Choksi, Damien Chung, William Cummings, Christine Forrester, Rayid Ghani, Thomas Griffiths, Farhan Hussain, Priyanka Jain, Dharam Jaithlia, Mohammad Usman Khan, Paritosh Khandelwal, Monica Leventhal, Stephen J. Mooney, Richard Niu, Laura Ong, Sachet Parikh, Niranjan Pisharody, Taha Razavi, Nikki Rebel, Alessandra Reid, Kimberly Rink, Naman Sambhukumar, Michael Samuel, Samuel Scheer, Avantika Sharma, Emaan Sheikh, Anand Siththaranjan, Dmitry Sorokin, Eric Walsh, James Wang, Ankith Wijenayake, Peter Yankov, Taha Yasseri, Aaron Ying, and Talal Zuberi. 2020. “Measuring the Predictability of Life Outcomes with a Scientific Mass Collaboration.” Proceedings of the National Academy of Sciences 117(15):8398–8403. doi:10.1073/pnas.1915006117.
- [58]Salganik, Matthew J., Ian Lundberg, Alexander T. Kindel, and Sara S. McLanahan. 2019. “Introduction to the Special Collection on the Fragile Families Challenge.” Socius 5:2378023119871580. doi:10.1177/2378023119871580.DOI
- [59]Santurkar, Shibani, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. “Whose Opinions Do Language Models Reflect?” In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 29971–30004. PMLR.arxiv.org/abs/2303.17548
- [60]Särndal, Carl-Erik, Bengt Swensson, and Jan Wretman. 1992. Model Assisted Survey Sampling. New York: Springer.
- [61]Small, Mario L. 2024. “The Value of Qualitative Data in Understanding Failure in Prediction.” Proceedings of the National Academy of Sciences 121(28):e2409327121. doi:10.1073/pnas.2409327121.DOI
- [62]Stanescu, Diana, Erik Wang, and Soichiro Yamauchi. 2019. “Using LASSO to Assist Imputation and Predict Child Well-Being.” Socius 5:2378023118814623. doi:10.1177/2378023118814623.DOI
- [63]van der Laan, Mark J., Eric C. Polley, and Alan E. Hubbard. 2007. “Super Learner.” Statistical Applications in Genetics and Molecular Biology 6(1). doi:10.2202/1544-6115.1309.DOI
- [64]Wang, Shuo, Tyler H. McCormick, and Jeffrey T. Leek. 2020. “Methods for Correcting Inference Based on Outcomes Predicted by Machine Learning.” Proceedings of the National Academy of Sciences 117(48):30266–30275. doi:10.1073/pnas.2001238117.DOI
- [65]Westwood, Sean J. 2025. “The Potential Existential Threat of Large Language Models to Online Survey Research.” Proceedings of the National Academy of Sciences 122(47):e2518075122. doi:10.1073/pnas.2518075122.DOI
- [67]Xie, Yu and Yueqi Xie. 2026. “When Artificial Intelligence Makes Everything Similar: The Risks of Content Homogenization.” Chinese Journal of Sociology 12(2):157–172. doi:10.1177/2057150X261419573.DOI
- [68]Xie, Yueqi, Lemeng Liang, Shuzhen Li, Yifu Lu, Zhiwen Xiao, Mengdi Shi, Junming Huang, Mengdi Wang, and Yu Xie. 2026. “Evaluating the Statistical Realism of LLM-Generated Social Science Data.” Proceedings of the National Academy of Sciences 123(19):e2538145123. doi:10.1073/pnas.2538145123.DOI
- [69]Yan, Jiani and Charles Rahal. 2025. “On the Unknowable Limits to Prediction.” Nature Computational Science 5(3). doi:10.1038/s43588-025-00776-y.DOI
- [70]Yang, Kaiqi, Hang Li, Hongzhi Wen, Tai-Quan Peng, Jiliang Tang, and Hui Liu. 2024. “Are Large Language Models (LLMs) Good Social Predictors?” In Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 2718–2730. Association for Computational Linguistics. doi:10.18653/v1/2024.findings-emnlp.153.DOI
- [71]Yarkoni, Tal and Jacob Westfall. 2017. “Choosing Prediction Over Explanation in Psychology: Lessons from Machine Learning.” Perspectives on Psychological Science 12(6):1100–1122. doi:10.1177/1745691617693393.DOI
- [73]Zhang, Anru, Lawrence D. Brown, and T. Tony Cai. 2019. “Semi-Supervised Inference: General Theory and Estimation of Means.” The Annals of Statistics 47(5):2538–2566. doi:10.1214/18-AOS1756.
- [74]Zhang, Simone, Janet Xu, and A. J. Alvero. 2025. “Generative AI Meets Open-Ended Survey Responses: Research Participant Use of AI and Homogenization.” Sociological Methods & Research 54(3):1197–1242. doi:10.1177/00491241251327130.DOI
- [75]Zhao, Jianpeng, Chenyu Yuan, Weiming Luo, Haoling Xie, Guangwei Zhang, Steven Jige Quan, Zixuan Yuan, Pengyang Wang, and Dehui Zhang. 2025. “Large Language Models as Virtual Survey Respondents: Evaluating Sociodemographic Response Generation.” doi:10.48550/arXiv.2509.06337. ArXiv:2509.06337.
- [77]Zrnic, Tijana and Emmanuel J. Candès. 2024a. “Active Statistical Inference.” In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research. PMLR.arxiv.org/abs/2403.03208
- [78]Zrnic, Tijana and Emmanuel J. Candès. 2024b. “Cross-Prediction-Powered Inference.” Proceedings of the National Academy of Sciences 121(15):e2322083121. doi:10.1073/pnas.2322083121.DOI
