WorksMathematics
Reorganizing Graduate Statistical Inference for Modern Biostatistical Practice
Abstract
First-year graduate courses in statistical inference within PhD programs in biostatistics often follow a classical mathematical statistics progression through probability models, data reduction, point estimation, testing and interval estimation, and asymptotic evaluation. This organization has clear pedagogical strengths, but may provide less guidance when the probability model is not the natural starting point. We propose a complementary target-first organization for a graduate statistical inference course in biostatistics. The course begins with the scientific question and target parameter, then introduces statistical model classes, estimation, first-order sampling behavior, inference, and efficiency within a common framework. Classical methods such as maximum likelihood, method of moments, estimating equations, confidence intervals, hypothesis tests, bootstrap and jackknife methods, and the Cramér–Rao bound are organized within a common inferential workflow. A central advantage of this organization is that the same workflow provides a natural bridge from classical inference to modern statistical problems, including flexible estimation using machine learning and artificial intelligence, policy learning, anytime-valid inference, and modern Bayesian methods.
Introduction
Many first-year graduate courses in statistical inference in biostatistics programs build on probability and follow a classical mathematical statistics progression: probability and models, data reduction, point estimation, testing and interval estimation, followed by asymptotic evaluation. This sequence has clear pedagogical advantages. Parametric families provide concrete probability models with tractable calculations, sufficiency and completeness formalize data reduction, likelihood and related methods provide systematic approaches to point estimation, and tests and confidence intervals introduce formal uncertainty quantification. This progression naturally encourages a model-first view of inference: begin by specifying a parametric probability model for the observed data, then develop estimation and inference within that model. Texts such as Casella and Berger [6] follow this pedagogy closely. This sequence is intended to represent a common organizing pattern rather than to characterize every graduate inference course.
The model-first perspective is natural when a parametric model provides a useful starting point. Modern biostatistical problems, however, increasingly arise in settings where no such model is obvious. Electronic health records, genomics, imaging, and other complex data structures may require flexible, data-adaptive methods, including machine learning and artificial intelligence tools. The challenge is therefore not that classical inferential principles cease to apply, but that a model-first pedagogy provides less guidance about how to organize inference when the model is no longer the natural starting point. This may make it more difficult for students to connect classical theory with contemporary biostatistical practice.
A complementary way to organize statistical inference is to begin with the scientific question and define the statistical target, or quantity, that answers it. We refer to this as a target-first view of inference. Once the target has been specified, students can consider what statistical model class or structure to impose on the data-generating distribution, how to construct an estimator of the target, and how to study its sampling behavior, uncertainty, and efficiency. This places classical methods within a common inferential workflow that can remain useful as estimation becomes more flexible and statistical problems evolve.
A target-first organization can also help students connect frequentist and Bayesian approaches. Both can begin from the same scientific question and statistical target, while differing in how inference about that target is constructed and uncertainty is quantified. Framing the two approaches around a common target can therefore make their similarities and differences easier to understand.
To better prepare graduate students for contemporary biostatistical practice, we propose reorganizing the first graduate course in statistical inference around a target-first workflow. The contribution is pedagogical rather than methodological: classical inferential ideas remain central, but are reorganized around a common sequence linking scientific questions, statistical targets, models, estimation, sampling behavior, and inference. A central advantage of this organization is that the same sequence provides a natural bridge from classical inference to modern problems involving flexible machine-learning and artificial-intelligence methods, policy and reinforcement learning, sequential inference, and modern Bayesian methods.
A Target-First Framework for Statistical Inference
At a high level, the course is organized around a sequence that begins with the scientific question and target parameter, then proceeds through model class choice, estimation, sampling behavior, inference, and efficiency.
Within this organization, classical topics such as maximum likelihood estimation, estimating equations, hypothesis testing, confidence intervals, and the Cramér–Rao bound are introduced as components of a broader inferential workflow rather than as the organizing framework itself. The same structure also provides natural places for bootstrap and jackknife methods and can subsequently be reused to introduce multiple testing, regression and working models, and basic Bayesian inference.
The proposed organization is summarized in Table 1, while Figure 1 contrasts a classical model-first sequence with the target-first workflow. More advanced semiparametric machinery is deferred to a subsequent course, where the same framework can be extended to flexible machine-learning estimators and other modern inferential problems.

Figure 1. Two ways of organizing a first graduate course in statistical inference. A classical model-first sequence begins with probability models and develops estimation, testing, and large-sample theory within that structure. The target-first sequence begins with the scientific question and target parameter, then proceeds through model choice, observed data, estimation, sampling behavior, inference, and efficiency.
| Topic | Pedagogical goal |
| I. Scientific questions and target parameters | Foreground the target of inference and distinguish the unknown distribution , the target , and an estimator of that target |
| II. Statistical model classes | Present the statistical model as a choice about how much structure to impose on after the target has been defined |
| III. Estimation | Present plug-in, and M-/Z-estimation as alternative routes from a specified target and model to an estimator within a common inferential framework |
| IV. Population-level sensitivity | Introduce the score as a local direction of perturbation of and the pathwise derivative as the corresponding first-order change in the target |
| V. First-order sampling behavior | Organize consistency, rates of convergence, and asymptotic distributions around the structure of the estimation error |
| VI. Regularity and influence functions | Connect local perturbations of , first-order changes in the target, and the first-order sampling behavior of regular estimators |
| VII. Statistical inference | Show how standard errors, confidence intervals, and hypothesis tests follow from the estimator’s first-order sampling behavior |
| VIII. Bootstrap and jackknife | Present resampling as a computational approach to studying sampling behavior, while using the jackknife to connect finite-sample perturbations with influence functions |
| IX. Efficiency | Frame efficiency as a comparison of regular estimators through first-order variance, with Fisher information and the Cramér–Rao bound as the parametric realization of these ideas |
| X. Multiple targets and multiple testing | Extend the target-first framework from a single target to a collection of targets and motivate multiple-testing procedures through the need to control inferential errors across that collection |
| XI. Regression and working models | Apply the target-first framework to regression, distinguishing regression coefficients as population-level summaries from regression models used to estimate separately defined targets, without requiring correct specification of the working model |
| XII. A Bayesian route to inference | Use the common target-first structure to introduce Bayesian inference as an alternative route to uncertainty quantification, while making explicit the role of the prior and the interpretation of uncertainty |
| XIII. Bridges to modern statistical inference | Show how the same organizing ideas provide a foundation for flexible machine-learning estimators, policy learning, anytime-valid inference, and modern Bayesian methods |
Table 1. A target-first organization for a first graduate course in statistical inference.
To make this progression concrete, Section A follows a simple example to show how familiar classical results emerge naturally from the target-first organization.
From Scientific Question to Target Parameter
The first objective is to translate a scientific question into a precise target of inference. When inference is organized around a model-first progression, the quantity being learned can remain implicit. In a target-first course, students first ask what quantity would answer the scientific question if the relevant population distribution were known.
Formally, let denote the observed data for one unit, let denote a broad collection of distributions for , and let denote the true data-generating distribution. A target parameter is a functional of this distribution:
The true target value is
This notation separates three objects that students often conflate: the unknown distribution , the target value , and the estimator . If were known, there would be no statistical problem; could be computed directly. The need for inference arises because is unknown.
Defining the target parameter requires close collaboration with subject-matter experts and making explicit the outcome of interest, its scale and time point, the population of interest, and, when relevant, the treatment, exposure, or policy comparison, as well as whether the target is causal or associational.
For example, suppose a clinician is interested in how three-year survival varies with a baseline biomarker among patients with newly diagnosed cancer. The population of interest is patients with newly diagnosed cancer, the outcome is survival, the time point is three years, and the scale is the probability of survival. Let denote the indicator of survival beyond three years and let denote the unknown joint distribution of in this population. A natural associational target is
Here, is the target feature of that answers the scientific question.
Different scientific questions lead to different targets. For example, interest in overall three-year survival would lead to
while changing the outcome, time point, population, scale, or scientific comparison would define a different target. If the scientific question were instead causal, for example the effect of a new treatment on three-year survival, a corresponding causal estimand might be the risk difference
where denotes the counterfactual three-year survival outcome under treatment level . Unlike the associational targets above, this counterfactual quantity is not generally identified from the observed-data distribution without additional causal assumptions. Such assumptions would be needed to express the causal estimand as a functional of , but are not developed here (see [23]). At this stage, the target has only been defined. The next step is to consider what structure, if any, to impose on the underlying distribution for studying our target.
Pedagogical goal: The pedagogical goal is to understand and identify the target of inference, while making explicit the distinction between the unknown data-generating distribution , the target , and an estimator of that target.
Statistical Model Classes
Once students have learned to define the target parameter, they can next consider what structure, if any, to impose on the unknown data-generating distribution . Recall that the target is the feature of of interest. A statistical model can then be introduced as the collection of distributions considered possible for the observed data, with .
For example, a parametric model restricts to a finite-dimensional family,
so functionals of interest can be expressed as functions of a finite-dimensional parameter . Structure may also be introduced through moment restrictions. For a vector-valued function , one may consider
Unlike a parametric model, moment restrictions constrain selected features of the distribution rather than the full distribution. This approach provides the basis for method-of-moments and generalized method-of-moments estimation ([14, 22]).
For the marginal survival target introduced above, specific examples are as follows. One may specify , and impose a parametric model
The same target can also be characterized using the moment-restriction formulation by taking and . Then
The same perspective extends to semiparametric and nonparametric models, where less structure is imposed on . These model classes are not developed here, but they illustrate why distinguishing the target from the model is useful: the same target can be studied under increasingly flexible assumptions.
Pedagogical goal: The pedagogical goal is to present the statistical model as a choice about how much structure to impose on after the target has been defined. Parametric models and moment restrictions provide two different ways of introducing such structure.
Estimation of the Target
Students can next learn how a population-level target is connected to observed data through estimation. The target parameter and statistical model remain population-level objects: is unknown, specifies the feature of we wish to learn, and represents the modeling assumptions imposed on . Estimation begins with an observed sample,
and constructs an estimator to approximate . Several broad estimation strategies fit naturally within this framework. A natural approach is the plug-in strategy, which , which simply estimates relevant parts of the distribution , then plugs in those estimates to a formula for the target parameter as if we had used the truth. Mathematically, it is described as . In the example of Eq. (1), this means estimating directly using proportions.
A Z-estimation approach can also be used. Define the empirical measure by . Then, a Z-estimator solves where is a moment or estimating function. An example of this is given by using logistic regression to model Eq. (1) in which the estimating function is a minimizer/maximizer of the gradient function. Method of moments and estimating equations fit naturally within Z-estimation, with the related M-estimation defining estimators through optimization of an empirical criterion ([27, 22, 12]), also falling under the class of Z-estimation.
Pedagogical goal: The pedagogical goal is to present plug-in and Z-estimation as different routes for estimation, yet working toward the same goal of estimating a target.
Sampling Behavior, Statistical Inference, and Efficiency
Once an estimator has been constructed, the next objective is to teach students how the target and estimator behave under small changes in the data-generating distribution, how these behaviors are linked, and how this connection leads to uncertainty quantification and efficiency. We begin with the population-level sensitivity of the target, study the first-order sampling behavior of the estimator, and finally connect the two through regularity and influence functions.
Population-Level Sensitivity: Local Perturbations and Scores
Students can first learn how the target changes under small, smooth perturbations of the data-generating distribution. To describe such perturbations, consider a smooth one-dimensional path given by
subject to . This represents a smooth local variation of the distribution, indexed by the scalar , with . Along this path, the target becomes a function of , namely , whose derivative at zero describes the sensitivity to this perturbation.
To rigorously quantify how distributions change along this submodel, the score is introduced by
An example of such a submodel is given by
where for parametric models, the score is the usual derivative of the log-likelihood.
Finally, to describe how this translates the target parameter, the pathwise derivative of at is introduced along the submodels , [27, 3, 25]
Pedagogical goal: The pedagogical goal is to introduce the score as describing a local direction of perturbation of , and the pathwise derivative as describing the corresponding first-order change in the target, rather than presenting the score as an object tied specifically to maximum likelihood.
First-Order Representation of the Estimation Error
Students can next learn how to study the sampling behavior of an estimator through a common first-order representation of its estimation error. Letting be the empirical measure, consider the following expansion
where is mean 0, , and collects the non-first order error. The function plays a crucial role in determining the estimator’s first-order sampling behavior, referred to as the influence function of the estimator [13, 27, 3].
This representation provides a common framework for familiar large-sample properties of a general target parameter. If the leading term and remainder vanish in probability, the estimator is consistent. If , the estimator is asymptotically linear [27, 3], and
which determines the first-order rate and limiting behavior by the central limit theorem.
Pedagogical goal: The pedagogical goal is to provide a common first-order representation of the estimation error, from which students can study various statistical properties.
Linking Population-Level Sensitivity and Estimator-Level Behavior: Regularity and Influence Functions
Students can next learn how the first-order sensitivity of the target and the first-order sampling behavior of an estimator are connected. The former is described through the pathwise derivative, while the latter is described through the estimator’s asymptotic linear expansion. Specifically, under regularity, these two first-order behaviors are linked by
where is the influence function in the expansion of a regular asymptotically linear estimator. Regularity requires this identity to hold for every score of a regular submodel through [3, 27, 25].
Loosely speaking, regularity means that the estimator’s limiting distribution, centered at the perturbed target, remains stable under local -scale perturbations of . This ensures that the first-order sampling behavior is compatible with the corresponding first-order change in the target.
This relationship gives students a common interpretation of the central objects:
: local direction of perturbation of
: pathwise derivative, or first-order change in the target
: influence function describing the estimator’s first-order sampling behavior
Pedagogical goal: The pedagogical goal is to introduce the influence function as the object that links a local direction of perturbation of , represented by the score , to first-order change in the target and the first-order sampling behavior of a regular estimator via regularity.
From Influence Functions to Statistical Inference
Students can next learn how the estimator’s first-order representation leads directly to statistical inference. Under the asymptotic linearity conditions above, the central limit theorem [27] gives
so first order, . Thus, the influence function directly determines the asymptotic variance of the estimator. Writing
a large-sample confidence interval is then given by a Wald-type construction [20]. Let the Wald statistic be given by
Then, students can form an asymptotically valid confidence interval by
which also connects to a hypothesis test and p-value for
Pedagogical goal: The pedagogical goal is to show how standard errors, confidence intervals, and hypothesis tests follow from the estimator’s first-order sampling behavior within a common inferential framework.
Resampling and the Jackknife
Students can next learn how resampling provides a computational way to study the sampling behavior of an estimator. Once a target and estimator have been specified, the bootstrap approximates the estimator’s sampling distribution by repeatedly applying the same estimation procedure to resampled data (Efron, [9]; Efron and Tibshirani, [10]). Typically, bootstrap is seen as a computational route to standard errors, confidence intervals, and other features of sampling behavior when analytic approximations are difficult, but the connection to estimation theory can be made explicit and clear using this approach.
The jackknife offers a related connection to the influence-function perspective (Boos and Stefanski, [4]). For sufficiently smooth estimators, the change induced by deleting one observation provides a finite-sample approximation to that observation’s first-order influence on the estimator. Jackknife pseudovalues are viewed as empirical analogues of influence-function contributions, linking classical resampling ideas to the first-order representation developed above.
Pedagogical goal: The pedagogical goal is to present the bootstrap and jackknife from the basis of the target first approach, providing an alternative to studying the sampling behavior of a specified estimator. In particular, the jackknife can be used to reinforce the connection between finite-sample perturbations and influence functions.
Efficiency
Once students understand that the influence function determines first-order variance, , efficiency can be introduced as the question of how small this variance can be among regular estimators of the same target (Lehmann and Casella, [19]; van der Vaart, [27]; Bickel et al., [3]; Tsiatis, [25]).
Parametric models provide a concrete setting in which to develop this idea. For a model , let the score vector be given by
and the Fisher information by
The one-dimensional score corresponding to a local direction through the parametric model is obtained from this score vector. For a smooth scalar target , the efficient influence function in the parametric model is
with variance
recovering the Cramér–Rao lower bound ([19, 6]). One can leverage much of the classical pedagogy to present efficiency in this view, and proceed to discuss equivalent ideas when moving past parametric models.
Pedagogical goal: The pedagogical goal is to frame efficiency as a comparison of regular estimators through their first-order variance, and to present the likelihood score, Fisher information, and Cramér–Rao lower bound as the parametric realization of these more general ideas.
Extensions of the Target-First Framework
We have presented an outline for a target first framework, going from the scientific question to target, the model class assumptions, the estimator choice, and an overview of understanding the sampling behavior, inference, and efficiency. Moving forward, we discuss two useful and natural extensions after developing this foundation: multiple testing, where the inferential target becomes multivariate, and regression, where the distinction between the target and the model becomes especially important.
Multiple Targets and Multiple Testing
Once students have learned to formulate inference around a single target parameter, the same framework can be extended to a collection of targets,
The inferential problem then includes not only estimating each target and quantifying its uncertainty, but also controlling errors across the collection. Multiple-testing procedures can therefore be introduced as an extension of the same target-first sequence, with family-wise error rate and false discovery rate providing different ways to formalize error control across several inferential targets ([16, 2]).
Pedagogical goal: The pedagogical goal is to extend the target-first framework from a single inferential target to a collection of targets and to motivate multiple-testing procedures through the need to control uncertainty across that collection.
Regression and Working Models
Once students have learned to distinguish the target parameter from the statistical model and estimator, the same framework can be used to revisit regression from a target-first perspective. Regression is often introduced by selecting a model based on the outcome type, such as linear regression for continuous outcomes or logistic regression for binary outcomes, and then interpreting the resulting model coefficients. However, these coefficients may not correspond to the original scientific question, and their usual interpretations may depend on correct model specification.
A target-first approach instead begins by asking which population quantity is of scientific interest and then considers how regression can be used to summarize or estimate that quantity. This provides two complementary ways of introducing regression within the same inferential framework.
First, regression can be used to define a lower-dimensional summary of a potentially complex population relationship. For example, consider the population conditional mean
without assuming a particular functional form. Rather than postulating that this relationship is linear, we may define the coefficients of its best linear approximation:
The corresponding working model is
Here, is a well-defined population quantity, provided the projection is unique, even when the true conditional mean is nonlinear. The coefficients describe a particular summary of the population relationship rather than parameters of an assumed true linear model [28, 5]. Their scientific relevance must still be justified by the research question.
Second, regression can be used as an estimation tool without its coefficients being the target of inference. For example, suppose the scientific question concerns the marginal mean
A regression model for the conditional mean can be used to help estimate , but the target remains the marginal mean rather than any individual regression coefficient.
These examples illustrate how familiar regression methods can be incorporated into the target-first framework without requiring the regression model to determine the scientific target. Regression can provide a useful population summary or serve as a tool for estimating a separately defined target, depending on the scientific question.
Pedagogical goal: The pedagogical goal is to extend the target-first framework to regression by teaching students to distinguish the scientific target from the role of the regression model. Regression coefficients may be defined as population-level summaries without assuming correct model specification, while regression models may also be used to estimate targets that are not themselves regression coefficients.
A Bayesian Route to Inference
Once students have learned to separate the scientific question, target parameter, statistical model, estimator, and uncertainty quantification, the same framework provides a natural way to introduce basic Bayesian inference. The scientific question, target parameter, statistical model, and observed data remain the same; Bayesian inference introduces a prior distribution on the unknown parameters of the model and updates this distribution using the observed data.
For example, under a parametric model , a prior and likelihood lead to the posterior [11]
If the target is , the posterior for induces a posterior distribution for the target, which can be summarized using posterior means, credible intervals, or posterior probabilities.
This presentation emphasizes the common structure of the inferential problem rather than framing Bayesian and frequentist methods as competing approaches. Both begin with the same scientific question and target, but they differ in how uncertainty about that target is constructed and interpreted.
Pedagogical goal: The pedagogical goal is to use the common target-first structure to introduce Bayesian inference as an alternative route to uncertainty quantification.
A Target-First Course Design
The framework above describes organizing principles rather than a syllabus. In this section, we illustrate how it could be implemented in a 14-week first graduate course in statistical inference. Table 1 summarizes the main components of the proposed organization and their pedagogical roles, while Table 2 gives one possible week-by-week implementation. The course develops the target-first sequence through estimation, first-order sampling behavior, inference, resampling, and efficiency, and then uses the same framework to revisit multiple testing, regression and working models, and basic Bayesian inference. The final component provides connections to a subsequent course in modern statistical inference. Concepts such as pathwise differentiability, regularity, and influence functions can be introduced at a conceptual and computational level, while more technical semiparametric theory and efficiency calculations are deferred to subsequent coursework.
| Week | Topic | Representative material |
| 1 | Scientific questions, targets, and models | Observed data, populations, target parameters, parametric models, moment restrictions |
| 2 | General approaches to estimation | Plug-in estimation, Z-estimation, M-estimation |
| 3 | Classical estimators in the common framework | Maximum likelihood, method of moments, generalized method of moments, estimating equations |
| 4 | First-order estimation error | Error decompositions, LLN, CLT, delta method, consistency, convergence rates |
| 5 | Local perturbations and target sensitivity | Parametric submodels, scores, pathwise derivatives |
| 6 | Regularity and influence functions | Asymptotic linearity, remainder terms, regular estimators, influence functions |
| 7 | Statistical inference | First-order variance, standard errors, confidence intervals, Wald tests |
| 8 | Bootstrap | Bootstrap distributions, standard errors, confidence intervals, relation to sampling distributions |
| 9 | Jackknife | Leave-one-out perturbations, pseudovalues, variance estimation, connection to influence functions |
| 10 | Efficiency | Efficient influence functions, likelihood scores, Fisher information, Cramér–Rao bounds |
| 11 | Multiple targets and multiple testing | Collections of target parameters and hypotheses, family-wise error rate, false discovery rate |
| 12 | Regression and working models | Conditional-mean targets, parametric regression models, regression coefficients, working models, low-dimensional summaries |
| 13 | A Bayesian route to inference | Priors, likelihoods, posterior distributions, posterior summaries, credible intervals |
| 14 | Bridges to modern statistical inference | Flexible ML- and AI-based estimation, policy learning, anytime-valid inference, modern Bayesian methods |
Table 2. One possible 14-week implementation of the target-first framework.
Bridging the Target-First Framework to a Second Course in Inference
The target-first framework provides a natural foundation for a subsequent course in modern statistical inference, where the same ideas of target specification, model choice, first-order approximation, and uncertainty quantification can be extended to more complex settings. We discuss the presentation and possible topics.
Flexible estimation with machine learning and AI. Flexible estimation provides a natural setting in which to revisit valid inference and introduce influence-function-based corrections, orthogonality, and cross-fitting [15, 18, 29, 7].
Policy learning and decision making. Replacing a fixed target with the value of a treatment rule or policy creates a bridge to individualized treatment rules, dynamic treatment regimes, contextual bandits, reinforcement learning, and regret-based decision making [8, 17, 1].
Anytime-valid inference. Sequential monitoring and adaptive stopping provide a natural extension of fixed-sample uncertainty quantification to confidence sequences and anytimevalid inference [24].
Modern Bayesian inference. The same target-first perspective can also connect to modern Bayesian methods, including amortized approaches such as Prior-Data Fitted Networks [21].
Higher-Order Influence Functions For nonlinear functionals, higher-order influence functions and classical U- and V-statistics provide natural extensions of the first-order framework. This can be casted in the framework of above, providing exposure to non-standard functional targets.
Discussion
The goal of the proposed framework is not to replace classical statistical inference, but to reorganize its central ideas around a common inferential workflow. Beginning with the scientific question and target parameter makes the distinction among the target, statistical model, estimator, and uncertainty explicit. Estimation, first-order sampling behavior, influence functions, resampling, and efficiency can then be introduced as successive components of the same inferential problem.
An important advantage of this organization is that the same framework provides a natural bridge to modern statistical inference. The questions developed here, including what the target is, what structure is imposed, how it is estimated, and how uncertainty is characterized, continue to apply in subsequent courses when estimation uses machine learning or artificial intelligence, when the target becomes a treatment policy or decision rule, when inference is sequential, or when uncertainty is represented through a Bayesian posterior.
This target-first perspective is closely related in spirit to the targeted-learning literature, which organizes estimation and inference around a prespecified target parameter and the statistical model in which it is defined ([26]). Our emphasis here is pedagogical: we use this organizing principle to connect classical and modern inferential methods rather than to advocate for a particular estimation procedure.
This reorganization necessarily changes the relative emphasis placed on some topics traditionally covered in mathematical statistics courses. Classical results on data reduction and finite-sample optimality, including sufficiency and minimal sufficiency, completeness, Rao–Blackwell and Lehmann–Scheffé results, and Neyman–Pearson or uniformly most powerful testing theory, may receive less attention. These remain valuable ideas, and programs wishing to retain exposure to them could reserve a lecture for selected classical finite-sample results while preserving the target-first organization of the course.
Finally, this is a curricular proposal rather than an empirical evaluation of learning outcomes, and the effects of this organization on student learning remain to be studied.
Acknowledgements
We acknowledge Ivan Diaz for the helpful discussions.
A Worked Example of the Target-First Framework
This appendix illustrates the proposed organization using a deliberately simple example. Suppose the scientific question concerns three-year survival among patients with newly diagnosed cancer. Let
indicate survival beyond three years. To keep the calculations transparent, we initially take ; a baseline biomarker is reintroduced in the extensions below. The simplicity of the example allows the same problem to be followed from target specification through estimation, first-order behavior, inference, resampling, and efficiency.
Scientific Question and Target
The scientific question is: what proportion of patients in the population survive beyond three years? Following § 2.1, the corresponding target is
where denotes the unknown population distribution of .
At this stage, the target has been defined without specifying either a statistical model or an estimator.
Pedagogical goal illustrated (§ 2.1): separate the unknown distribution , the target , and the estimator that will later be used to approximate it.
Statistical Model
Because is binary, its marginal distribution can be represented as
so that
In this example the Bernoulli family exhausts the possible distributions on , so the parametric representation imposes no additional restriction beyond the binary support.
Pedagogical goal illustrated (§ 2.2): the target is specified first, while the statistical model describes the structure imposed on the unknown distribution.
Estimation
Suppose
The natural estimator is the sample proportion
This single estimator illustrates the three estimation strategies introduced in § 2.3.
First, it is a plug-in estimator:
Second, it is a Z-estimator because it solves
Third, under the Bernoulli model it is the maximum likelihood estimator,
and hence an M-estimator with negative log-likelihood as its criterion.
Thus, plug-in estimation, Z-estimation, and M-estimation provide different representations of the same familiar estimator.
Pedagogical goal illustrated (§ 2.3): place familiar estimation methods within a common inferential workflow and show how plug-in, Z-estimation, and M-estimation can provide different representations of the same estimator.
Population-Level Sensitivity
Following § 2.4, consider a local path through the true Bernoulli distribution,
for sufficiently small . The corresponding score is
Because the target is ,
The score therefore describes the local direction in which is perturbed, while the pathwise derivative describes the corresponding first-order change in the survival probability.
Pedagogical goal illustrated (§ 2.4): connect a local perturbation of the data-generating distribution to the resulting first-order change in the target.
First-Order Sampling Behavior
For the sample proportion,
Thus the general first-order representation becomes
with
In this simple example, the first-order representation is exact, and is the function governing the estimator’s first-order sampling behavior. The next step is to verify that this same function is compatible with the local behavior of the target. Consistency follows from the law of large numbers, while the central limit theorem determines the root- limiting behavior.
Pedagogical goal illustrated (§ 2.4): organize consistency, rates, and limiting distributions around the structure of the estimation error.
Regularity and the Influence Function
The function identified from the first-order representation can now be connected to the population-level perturbation. For the path indexed by ,
Hence
Thus the same that describes the estimator’s first-order sampling behavior also represents the pathwise derivative through the influence function relationship. The example therefore makes explicit the connection among the score, the first-order change in the target, and the first-order sampling behavior of the estimator described in § 2.4.
Pedagogical goal illustrated (§ 2.4): show concretely how the influence function links population-level sensitivity to the first-order sampling behavior of a regular estimator.
Statistical Inference
Since
the central limit theorem gives
The corresponding estimated standard error is
A large-sample confidence interval is therefore
and for , the corresponding Wald statistic is
Thus standard errors, confidence intervals, and hypothesis tests arise directly from the first-order representation.
Pedagogical goal illustrated (§ 2.4): derive familiar inferential procedures from the estimator’s first-order sampling behavior.
Bootstrap and Jackknife
The same estimator also illustrates the role of resampling. A nonparametric bootstrap sample
is drawn from the empirical distribution , and the estimator is recomputed as
The conditional distribution of provides a computational approximation to the sampling distribution of .
For the jackknife, deleting observation gives
The corresponding jackknife pseudovalue is
so that
The centered pseudovalues therefore coincide with the empirical version of the influence-function contribution.
Pedagogical goal illustrated (§ 2.4): present bootstrap and jackknife methods as tools for studying the sampling behavior of the same estimator, while making the connection between jackknife perturbations and influence functions explicit.
Efficiency
For the Bernoulli model, the likelihood score is
and the Fisher information is
Because ,
Thus
and the corresponding variance bound for an estimator based on observations is
The influence function of the sample proportion is therefore the efficient influence function, and the estimator attains the Cramér–Rao bound in this model.
Pedagogical goal illustrated (§ 2.4): connect first-order variance, likelihood scores, Fisher information, and the Cramér–Rao bound as different parts of the same efficiency calculation.
References
- [1]Athey, S. and Wager, S. (2021). Policy learning with observational data. Econometrica, 89(1):133–161.arxiv.org/abs/1702.02896
- [2]Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B, 57(1):289–300.DOI
- [3]Bickel, P. J., Klaassen, C. A. J., Ritov, Y., and Wellner, J. A. (1993). Efficient and Adaptive Estimation for Semiparametric Models. Johns Hopkins University Press.
- [4]Boos, D. D. and Stefanski, L. A. (2013). Essential statistical inference: theory and methods, volume 591. Springer.DOI
- [5]Buja, A., Berk, R., Brown, L., George, E., Pitkin, E., Traskin, M., Zhang, K., and Zhao, L. (2019). Models as approximations i: Consequences illustrated with linear regression. Statistical Science, 34(4):523–544.arxiv.org/abs/1404.1578
- [6]Casella, G. and Berger, R. L. (2002). Statistical Inference. Duxbury, 2 edition.
- [7]Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., and Robins, J. (2018). Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21(1):C1–C68.
- [8]Dudík, M., Langford, J., and Li, L. (2011). Doubly robust policy evaluation and learning. In Proceedings of the 28th International Conference on Machine Learning, pages 1097–1104.arxiv.org/abs/1103.4601
- [9]Efron, B. (1982). The Jackknife, the Bootstrap and Other Resampling Plans. Society for Industrial and Applied Mathematics.
- [10]Efron, B. and Tibshirani, R. J. (1993). An Introduction to the Bootstrap. Chapman & Hall.
- [11]Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., and Rubin, D. B. (2013). Bayesian Data Analysis. Chapman & Hall/CRC, 3 edition.
- [12]Godambe, V. P., editor (1991). Estimating Functions. Clarendon Press, Oxford.
- [13]Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J., and Stahel, W. A. (1986). Robust Statistics: The Approach Based on Influence Functions. John Wiley & Sons.
- [14]Hansen, L. P. (1982). Large sample properties of generalized method of moments estimators. Econometrica, 50(4):1029–1054.DOI
- [15]Hines, O., Dukes, O., Diaz-Ordaz, K., and Vansteelandt, S. (2022). Demystifying statistical learning based on efficient influence functions. The American Statistician, 76(3):292–304.DOI
- [16]Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2):65–70.DOI
- [17]Kallus, N. (2018). Balanced policy evaluation and learning. In Advances in Neural Information Processing Systems, volume 31.arxiv.org/abs/1705.07384
- [18]Kennedy, E. H. (2024). Semiparametric doubly robust targeted double machine learning: a review. Handbook of statistical methods for precision medicine, pages 207–236.arxiv.org/abs/2203.06469
- [19]Lehmann, E. L. and Casella, G. (1998). Theory of Point Estimation. Springer, 2 edition.
- [20]Lehmann, E. L. and Romano, J. P. (2005). Testing Statistical Hypotheses. Springer, 3 edition.
- [21]Müller, S., Hollmann, N., Pineda Arango, S., Grabocka, J., and Hutter, F. (2022). Transformers can do bayesian inference. In International Conference on Learning Representations.arxiv.org/abs/2112.10510
- [22]Newey, W. K. and McFadden, D. (1994). Large sample estimation and hypothesis testing. In Engle, R. F. and McFadden, D. L., editors, Handbook of Econometrics, volume 4, pages 2111–2245. Elsevier.
- [23]Pearl, J. (2010). An introduction to causal inference. The international journal of biostatistics, 6(2):7.
- [24]Ramdas, A., Grünwald, P., Vovk, V., and Shafer, G. (2023). Game-theoretic statistics and safe anytime-valid inference. Statistical Science, 38(4):576–601.arxiv.org/abs/2210.01948
- [25]Tsiatis, A. A. (2006). Semiparametric Theory and Missing Data. Springer.DOI
- [26]van der Laan, M. J. and Rose, S. (2011). Targeted Learning: Causal Inference for Observational and Experimental Data. Springer.DOI
- [27]van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge University Press.DOI
- [28]White, H. (1982). Maximum likelihood estimation of misspecified models. Econometrica, 50(1):1–25.
- [29]Williams, N. T., Hines, O. J., and Rudolph, K. E. (2026). Riesz represents for the rest of us. Epidemiology, pages 10–1097.DOI