Introduction
Biodiversity studies increasingly seek to characterize how species compositions vary across ecological communities and geographic regions. Quantifying this between-community heterogeneity, commonly referred to as β \beta β -diversity, is fundamental for understanding ecological processes, monitoring environmental change, and prioritizing conservation efforts [40 ] . A central scientific objective is to infer the extent of species sharing across locations and to predict how many previously unobserved species would be discovered through additional sampling. Such questions arise naturally in large-scale biodiversity monitoring projects, where species occurrence data are collected across multiple habitats, ecosystems, or geographic regions.
Statistical inference on β \beta β -diversity is challenging for several reasons. Species occurrence data are typically high dimensional and sparse, with many species remaining unobserved even after extensive sampling. Ecological communities often exhibit substantial heterogeneity across locations due to differences in environmental conditions and habitat characteristics, making exchangeability assumptions unrealistic. In addition, biodiversity surveys are affected by imperfect detection: species that are present may fail to be observed because of limitations in sampling technologies and measurement protocols. Ignoring detection error can substantially bias estimates of species richness, species sharing, and between-community heterogeneity. These challenges are particularly pronounced in modern DNA meta-barcoding studies. We are motivated by the Global Spore Sampling Project (GSSP) [27 , 1 ] , which collected airborne fungal spore samples at locations around the world using cyclone samplers. Ecologists typically analyze these data in terms of operational taxonomic units (OTUs), obtained by assigning taxonomic labels to DNA sequences [35 ] . Because the proportion of sequences mapped to each OTU is not a reliable indicator of abundance, meta-barcoding studies commonly reduce the observations to binary occurrence data indicating whether a species is detected in each sample. These data are characterized by large numbers of rare species, substantial heterogeneity across locations, and non-negligible false negative rates arising from low DNA concentrations, PCR biases, and sequencing errors [11 , 33 ] .
Despite the importance of inference on β \beta β -diversity, existing approaches remain largely based on empirical dissimilarity indices with limited probabilistic interpretation and inadequate uncertainty quantification [31 ] . Occupancy models explicitly account for imperfect detection by separating ecological and observational processes [22 , 32 , 41 , 39 ] , and have become standard tools in ecology. However, these models are not designed to characterize latent species sharing structure across heterogeneous communities or to accommodate the continual discovery of previously unobserved species in highly diverse ecosystems.
Bayesian nonparametric feature allocation models provide a natural probabilistic framework for species occurrence data. Indian buffet processes [38 , 16 , 5 ] and related models [20 , 4 , 23 , 6 , 13 ] characterize binary latent feature matrices in which rows correspond to samples and columns to a potentially unbounded collection of features. In biodiversity applications the features are species, so that new species emerge as sampling proceeds and the latent feature matrix encodes precisely the pattern of species sharing that β \beta β -diversity seeks to quantify. Existing models, however, are ill-suited to this task on two counts. First, they assume exchangeability, that is, invariance of the occurrence matrix under permutations of the samples, which precludes the systematic variation in species composition across communities that β \beta β -diversity is about. Second, they assume perfect detection, and thus attribute every absence to genuine absence rather than to a failure to observe. Partially exchangeable constructions have recently received growing attention for species sampling models (e.g., [12 ] ; [9 ] ), but substantially fewer results are available for feature allocations: [34 ] proposed a bivariate beta process for genetic variants across two heterogeneous populations, though the approach is restricted to two groups and does not admit a tractable analytical characterization, while [14 ] introduced a broad class of priors for partially exchangeable trait allocations, without addressing imperfect detection or developing inferential results for biodiversity functionals such as species sharing and β \beta β -diversity.
Motivated by these challenges, we propose the class of Multi-site Occupancy-aware Species Allocation with Imperfect deteCtion (MOSAIC) models, which addresses both issues at once. Partial exchangeability allows species compositions to vary systematically across sites, while an occupancy-based detection layer separates the probability that a species occupies a site from the probability that it is recorded there, so that an absence in the data need not be an absence in the field. The number of species is finite but random, a choice that preserves substantial modeling flexibility while enabling analytical tractability and scalable posterior computation; by contrast, natural extensions of infinite-activity beta process constructions to heterogeneous populations may violate fundamental finiteness conditions or require computationally prohibitive inference procedures ([34 ] ).
The resulting framework supports a coherent probabilistic treatment of diversity at every level. While α \alpha α -diversity and γ \gamma γ -diversity, measuring within-community and global diversity, respectively, have received extensive attention in the Bayesian nonparametric literature ([21 ] ; [10 ] ; [13 ] ), this is not the case for multi-site incidence data, for which even the distribution of the number of distinct species at each site has not been characterized; and substantially less work has addressed principled inference on β \beta β -diversity, despite its central ecological importance ([31 ] ). We therefore develop the α \alpha α - and γ \gamma γ -diversity theory first, obtaining prior and posterior distributions for the site-specific and pooled species counts, along with predictive distributions for the species that would be discovered under further sampling. Beyond their own interest, these results underpin our treatment of β \beta β -diversity, since the same machinery delivers the distribution of the number of species shared between any pair of sites, the main ingredient of our proposed index. The predictive quantities are moreover directly relevant for biodiversity monitoring and adaptive sampling design, where decisions must be made regarding whether and where further sampling effort should be allocated. The resulting summaries of compositional heterogeneity are interpretable and account simultaneously for latent species richness, partial exchangeability, and imperfect detection. We further specify a hierarchical prior in which site-specific covariates drive the occurrence probabilities, inducing borrowing of information across sites.
The remainder of the paper is organized as follows. Section 2 introduces the proposed class of partially exchangeable feature allocation models with imperfect detection, while Section 3 develops their theoretical properties and studies inference on biodiversity functionals, including β \beta β -diversity. Section 4 presents simulation studies evaluating finite-sample performance, while Section 5 analyzes global fungal biodiversity data from the GSSP study.
The MOSAIC model
Notation and model
In this section, we define the notation and introduce our proposed class of feature allocation models for partially exchangeable data with imperfect detection. The data are sampled from Q Q Q distinct sites, indexed by q ∈ { 1 , … , Q } q \in\{1,\ldots,Q\} q ∈ { 1 , … , Q } ; we denote by n q n_q n q the number of samples from site q q q and by n = ( n 1 , … , n Q ) \mathbf{n}=(n_1,\ldots,n_Q) n = ( n 1 , … , n Q ) the corresponding vector of sample sizes. We assume a common species reference set across sites, and we denote by ( X ~ j ) j ≥ 1 (\widetilde{X}_j)_{j\geq1} ( X j ) j ≥ 1 the sequence of all possible species labels, which are unknown a priori. Let X q \mathcal{X}_q X q be the set of distinct species actually observed at site q q q and let K q , n q = ∣ X q ∣ = k q K_{q,n_q}=|\mathcal{X}_q|=k_q K q , n q = ∣ X q ∣ = k q be its cardinality. The total number of distinct species observed across all sites is K n = ∣ ⋃ q = 1 Q X q ∣ = k K_n=\left|\bigcup_{q=1}^{Q}\mathcal{X}_q\right|=k K n = ⋃ q = 1 Q X q = k , and we write X 1 , … , X k X_1,\ldots,X_k X 1 , … , X k for the labels of the species actually observed, which form a subset of the complete list ( X ~ j ) j ≥ 1 (\widetilde{X}_j)_{j\geq1} ( X j ) j ≥ 1 . This notation explicitly accounts for the fact that some species may remain unseen. Finally, let C n q , n r = ∣ X q ∩ X r ∣ = c q , r C_{n_q,n_r}=|\mathcal{X}_q\cap\mathcal{X}_r|=c_{q,r} C n q , n r = ∣ X q ∩ X r ∣ = c q , r denote the number of species shared between the samples from sites q q q and r r r with q ≠ r q\ne r q = r . Because sites may share species, in general K n ≤ ∑ q = 1 Q K q , n q K_n\leq\sum_{q=1}^{Q}K_{q,n_q} K n ≤ ∑ q = 1 Q K q , n q ; for instance, with Q = 2 Q=2 Q = 2 we have K n = K 1 , n 1 + K 2 , n 2 − C n 1 , n 2 K_n=K_{1,n_1}+K_{2,n_2}-C_{n_1,n_2} K n = K 1 , n 1 + K 2 , n 2 − C n 1 , n 2 . Throughout, we further assume that the species pool is finite, that is, the complete list reduces to X ~ 1 , … , X ~ N \widetilde{X}_1,\ldots,\widetilde{X}_N X 1 , … , X N , so that the number of distinct species observed in the full sample satisfies K n ≤ N K_n\leq N K n ≤ N . The total number of species N N N is itself unknown and is inferred from the data. Accordingly, we adopt the beta–Bernoulli (BB) class of feature allocation models ([16 ] , [13 ] ), under which the distribution of K n K_n K n has bounded support given N N N . This choice is motivated by analytical tractability: known infinite-species models lead to considerably less tractable expressions ([34 ] ).
The data can be encoded as binary occurrence indicators, one for each combination of sample, species, and site, and we distinguish between a version involving the full species list and its observed counterpart. Let Y ~ = ( Y ~ i j q ) \widetilde{\mathbf{Y}}=(\widetilde{Y}_{ijq}) Y = ( Y ij q ) with i = 1 , … , n q i=1,\ldots,n_q i = 1 , … , n q ; j = 1 , … , N j=1,\ldots,N j = 1 , … , N ; q = 1 , … , Q q=1,\ldots,Q q = 1 , … , Q , where Y ~ i j q = 1 \widetilde{Y}_{ijq}=1 Y ij q = 1 if species X ~ j \widetilde{X}_j X j is recorded in sample i i i of site q q q , and Y ~ i j q = 0 \widetilde{Y}_{ijq}=0 Y ij q = 0 otherwise. A zero entry is ambiguous: species X ~ j \widetilde{X}_j X j may be absent from site q q q at the time of sampling, or present but not detected. We collect the observed occurrences in Y = ( Y i j q ) \mathbf{Y} = (Y_{ijq}) Y = ( Y ij q ) with i = 1 , … , n q i = 1,\ldots,n_q i = 1 , … , n q ; j = 1 , … , k j = 1,\ldots,k j = 1 , … , k ; q = 1 , … , Q q = 1,\ldots,Q q = 1 , … , Q , obtained from Y ~ \widetilde{\mathbf{Y}} Y by discarding the species that are never recorded, namely those j j j for which Y ~ i j q = 0 \widetilde{Y}_{ijq} = 0 Y ij q = 0 for every i i i and q q q ; equivalently, each species retained in Y \mathbf{Y} Y has at least one non-zero entry. The distinction between Y ~ \widetilde{\mathbf{Y}} Y and Y \mathbf{Y} Y would be inconsequential if N N N were known, since the discarded species could then be reinstated, or if all species were discovered, in which case N = k N = k N = k and the two arrays coincide. These are not realistic assumptions in ecological applications, where neither the labels nor the number of undiscovered species is known a priori , and hence Y ~ \widetilde{\mathbf{Y}} Y carries strictly more information than Y \mathbf{Y} Y . In what follows we specify a model for Y ~ \widetilde{\mathbf{Y}} Y , which induces a model for the observed indicators Y \mathbf{Y} Y .
Let η q = ( η 1 q , … , η N q ) \boldsymbol{\eta}_q = (\eta_{1q},\ldots,\eta_{Nq}) η q = ( η 1 q , … , η N q ) and γ q = ( γ 1 q , … , γ N q ) \boldsymbol{\gamma}_q = (\gamma_{1q},\ldots,\gamma_{Nq}) γ q = ( γ 1 q , … , γ N q ) , for q = 1 , … , Q q = 1,\ldots,Q q = 1 , … , Q , collect independent random probabilities representing the species- and site-specific occupancy and detectability components, respectively. In our MOSAIC model, given these parameters, the entries of Y ~ \widetilde{\mathbf{Y}} Y are conditionally independent Bernoulli random variables,
Y ~ i j q ∣ η j q , γ j q ∼ i n d Bernoulli ( η j q γ j q ) , (1) \widetilde{Y}_{ijq} \mid\eta_{jq},\gamma_{jq} \mathrel{\overset{\mathrm{ind}}{\sim}} \operatorname{Bernoulli}(\eta_{jq}\gamma_{jq}), \tag*{(1)} Y ij q ∣ η j q , γ j q ∼ ind Bernoulli ( η j q γ j q ) , (1)
for i = 1 , … , n q i = 1,\ldots,n_q i = 1 , … , n q , j = 1 , … , N j = 1,\ldots,N j = 1 , … , N and q = 1 , … , Q q = 1,\ldots,Q q = 1 , … , Q . The factorization in (1) separates the occupancy and detection components of the model: η j q \eta_{jq} η j q is the probability that species X ~ j \widetilde{X}_j X j occupies site q q q , and γ j q \gamma_{jq} γ j q is the probability that it is detected, given presence. The two differ because a species present at a site may go unrecorded, so that the observed occurrences are a thinned version of the underlying presence/absence pattern. This is made precise by a data augmentation, in which each Y ~ i j q \widetilde{Y}_{ijq} Y ij q is written as the product of two independent Bernoulli random variables representing the occupancy and detection events separately. Let W ~ = ( W ~ i j q ) \widetilde{\mathbf{W}} = (\widetilde{W}_{ijq}) W = ( W ij q ) and T ~ = ( T ~ i j q ) \widetilde{\mathbf{T}} = (\widetilde{T}_{ijq}) T = ( T ij q ) , with i = 1 , … , n q i = 1,\ldots,n_q i = 1 , … , n q , j = 1 , … , N j = 1,\ldots,N j = 1 , … , N and q = 1 , … , Q q = 1,\ldots,Q q = 1 , … , Q , be augmented binary arrays encoding respectively the true presence of each species and its potential detection, so that W ~ i j q ∣ η j q ∼ i n d Bernoulli ( η j q ) \widetilde{W}_{ijq} \mid\eta_{jq} \mathrel{\overset{\mathrm{ind}}{\sim}} \operatorname{Bernoulli}(\eta_{jq}) W ij q ∣ η j q ∼ ind Bernoulli ( η j q ) and T ~ i j q ∣ γ j q ∼ i n d Bernoulli ( γ j q ) \widetilde{T}_{ijq} \mid\gamma_{jq} \mathrel{\overset{\mathrm{ind}}{\sim}} \operatorname{Bernoulli}(\gamma_{jq}) T ij q ∣ γ j q ∼ ind Bernoulli ( γ j q ) . Then Y ~ i j q = W ~ i j q T ~ i j q \widetilde{Y}_{ijq} = \widetilde{W}_{ijq}\widetilde{T}_{ijq} Y ij q = W ij q T ij q , so that a species is recorded if and only if it is both present and detected. Analogously, let W q \mathcal{W}_q W q be the set of distinct species present in at least one of the n q n_q n q samples from site q q q according to W ~ \widetilde{\mathbf{W}} W , and let L q , n q = ∣ W q ∣ = ℓ q L_{q,n_q} = |\mathcal{W}_q| = \ell_q L q , n q = ∣ W q ∣ = ℓ q be its cardinality. We write L n = ∣ ⋃ q = 1 Q W q ∣ = ℓ L_n = \left|\bigcup_{q=1}^{Q}\mathcal{W}_q\right| = \ell L n = ⋃ q = 1 Q W q = ℓ for the number of species present in at least one sample across sites, and U n q , n r = ∣ W q ∩ W r ∣ = u q , r U_{n_q,n_r} = |\mathcal{W}_q \cap\mathcal{W}_r| = u_{q,r} U n q , n r = ∣ W q ∩ W r ∣ = u q , r , with q ≠ r q \ne r q = r , for the number of species present at both sites q q q and r r r . We then let W = ( W i j q ) \mathbf{W} = (W_{ijq}) W = ( W ij q ) , with i = 1 , … , n q i = 1,\ldots,n_q i = 1 , … , n q , j = 1 , … , ℓ j = 1,\ldots,\ell j = 1 , … , ℓ and q = 1 , … , Q q = 1,\ldots,Q q = 1 , … , Q , be the array obtained from W ~ \widetilde{\mathbf{W}} W by discarding the species that are never present, so that each retained species has at least one non-zero entry. Unlike their observed counterparts, none of these quantities is available in practice, and they satisfy K q , n q ≤ L q , n q K_{q,n_q} \le L_{q,n_q} K q , n q ≤ L q , n q , K n ≤ L n ≤ N K_n \le L_n \le N K n ≤ L n ≤ N and C n q , n r ≤ U n q , n r C_{n_q,n_r} \le U_{n_q,n_r} C n q , n r ≤ U n q , n r .
We now specify priors for our MOSAIC model, with two inferential goals in mind: (i) estimating the site and species specific parameters η j q \eta_{jq} η j q and γ j q \gamma_{jq} γ j q ; (ii) estimating the number of unseen species N − k N - k N − k . To achieve the latter goal, we need to estimate N N N , the total number of species. To this end, we treat N N N as random and assume N ∼ Poisson ( λ ) N \sim\operatorname{Poisson}(\lambda) N ∼ Poisson ( λ ) , with λ > 0 \lambda> 0 λ > 0 . Moreover, we assume that the parameters γ j q \gamma_{jq} γ j q and η j q \eta_{jq} η j q are conditionally independent for j = 1 , … , N j=1,\ldots,N j = 1 , … , N and q = 1 , … , Q q=1,\ldots,Q q = 1 , … , Q given a set of parameters ( α q , θ q , σ q ) (\alpha_q,\theta_q,\sigma_q) ( α q , θ q , σ q ) and follow a beta distribution. We will further denote the corresponding vectors of parameters as α = ( α 1 , … , α Q ) \boldsymbol{\alpha}=(\alpha_1,\ldots,\alpha_Q) α = ( α 1 , … , α Q ) , θ = ( θ 1 , … , θ Q ) \boldsymbol{\theta}=(\theta_1,\ldots,\theta_Q) θ = ( θ 1 , … , θ Q ) and σ = ( σ 1 , … , σ Q ) \boldsymbol{\sigma}=(\sigma_1,\ldots,\sigma_Q) σ = ( σ 1 , … , σ Q ) . This implies the following specification for the MOSAIC model with parameters ( λ , α , θ , σ ) (\lambda,\boldsymbol{\alpha},\boldsymbol{\theta},\boldsymbol{\sigma}) ( λ , α , θ , σ )
N ∼ Poisson ( λ ) , N \sim\operatorname{Poisson}(\lambda), N ∼ Poisson ( λ ) ,
η j q ∼ i n d Beta { − α q , σ q ( α q + θ q ) } , ( j = 1 , … , N , q = 1 , … , Q ) , (2) \eta_{jq} \mathrel{\overset{\mathrm{ind}}{\sim}} \operatorname{Beta}\left\{-\alpha_q,\sigma_q(\alpha_q+\theta_q)\right\}, \qquad(j=1,\ldots,N,q=1,\ldots,Q), \tag*{(2)} η j q ∼ ind Beta { − α q , σ q ( α q + θ q ) } , ( j = 1 , … , N , q = 1 , … , Q ) , (2)
γ j q ∼ i n d Beta { − α q + σ q ( α q + θ q ) , ( 1 − σ q ) ( α q + θ q ) } , ( j = 1 , … , N , q = 1 , … , Q ) , \gamma_{jq} \mathrel{\overset{\mathrm{ind}}{\sim}} \operatorname{Beta}\left\{-\alpha_q+\sigma_q(\alpha_q+\theta_q),(1-\sigma_q)(\alpha_q+\theta_q)\right\}, \qquad(j=1,\ldots,N,q=1,\ldots,Q), γ j q ∼ ind Beta { − α q + σ q ( α q + θ q ) , ( 1 − σ q ) ( α q + θ q ) } , ( j = 1 , … , N , q = 1 , … , Q ) ,
where σ q ∈ ( 0 , 1 ) \sigma_q\in(0,1) σ q ∈ ( 0 , 1 ) , α q < 0 \alpha_q<0 α q < 0 , θ q > − α q \theta_q>-\alpha_q θ q > − α q for q = 1 , … , Q q=1,\ldots,Q q = 1 , … , Q . By a result of [19 ] on products of independent beta random variables,
η j q γ j q ∼ i n d Beta ( − α q , α q + θ q ) , ( j = 1 , … , N , q = 1 , … , Q ) , (3) \eta_{jq}\gamma_{jq} \mathrel{\overset{\mathrm{ind}}{\sim}} \operatorname{Beta}(-\alpha_q,\alpha_q+\theta_q), \qquad(j=1,\ldots,N,q=1,\ldots,Q), \tag*{(3)} η j q γ j q ∼ ind Beta ( − α q , α q + θ q ) , ( j = 1 , … , N , q = 1 , … , Q ) , (3)
and in particular E ( η j q γ j q ) = − α q / θ q E(\eta_{jq}\gamma_{jq})=-\alpha_q/\theta_q E ( η j q γ j q ) = − α q / θ q . Crucially, (3) holds for every value of σ q ∈ ( 0 , 1 ) \sigma_q\in(0,1) σ q ∈ ( 0 , 1 ) : the parameter σ q \sigma_q σ q redistributes mass between occupancy and detectability. Its role is made explicit by the expectations
E ( η j q ) = − α q σ q ( α q + θ q ) − α q , E ( γ j q ) = − α q + σ q ( α q + θ q ) θ q , E(\eta_{jq})=\frac{-\alpha_q}{\sigma_q(\alpha_q+\theta_q)-\alpha_q}, \qquad E(\gamma_{jq})=\frac{-\alpha_q+\sigma_q(\alpha_q+\theta_q)}{\theta_q}, E ( η j q ) = σ q ( α q + θ q ) − α q − α q , E ( γ j q ) = θ q − α q + σ q ( α q + θ q ) ,
which are respectively decreasing and increasing in σ q \sigma_q σ q . Values of σ q \sigma_q σ q close to 1 thus correspond to near-perfect detectability, whereas values close to 0 describe the least favorable regime, in which every species occupies every site and the observed sparsity is ascribed entirely to detection failure. In the limit σ q → 1 \sigma_q\to1 σ q → 1 the law of γ j q \gamma_{jq} γ j q degenerates at one, giving E ( η j q ) = − α q / θ q E(\eta_{jq})=-\alpha_q/\theta_q E ( η j q ) = − α q / θ q , which recovers the expected occurrence probability of a partially exchangeable BB model with perfect detection.
This framework borrows information across sites in two ways. First, all sites share a common total number of species N N N . Second, it introduces dependence among species occurrence probabilities through site-specific covariates via a hierarchical formulation, detailed in Section 3.3.
Remark 1. The occupancy indicators W ~ i j q \widetilde{W}_{ijq} W ij q vary across samples i i i within a site, so that occupancy is understood as presence at the time of collection of sample i i i rather than as a fixed site attribute. This departs from standard occupancy formulations (e.g., [22 ] ; [41 ] ), in which a single indicator is shared by all samples from a site, but it yields substantially simpler expressions for several quantities of interest, as discussed in Section 2.2 and Section 3.1. The price to pay is that the observed occurrences depend on η j q \eta_{jq} η j q and γ j q \gamma_{jq} γ j q only through their product, so that occupancy and detectability cannot be separated on the basis of Y ~ \widetilde{Y} Y alone. The parameter σ q \sigma_q σ q trades one against the other while leaving the law of the observed occurrences unchanged; the split between the two components is therefore driven entirely by the choice of σ q \sigma_q σ q , which cannot be learned from the data. Accordingly, we do not estimate σ q \sigma_q σ q and treat it instead as a sensitivity parameter, exploring different detectability scenarios ranging from σ q → 1 \sigma_q\to1 σ q → 1 (perfect detection) to values implying substantial under-detection.
Relationship with the beta–Bernoulli model and distribution theory
In this section, we relate MOSAIC to the partially exchangeable BB model of [14 ] (2025), which did not consider undetected species or develop inferences for beta diversity. We show that the two induce the same law for the observed occurrences and therefore differ only by data augmentation. Our detectability layer provides an interpretation of the BB parameters that has not been exploited previously. Let π j q \pi_{jq} π j q , for j = 1 , … , N j=1,\ldots,N j = 1 , … , N and q = 1 , … , Q q=1,\ldots,Q q = 1 , … , Q , be independent random probabilities, which we collect into the vectors π q = ( π 1 q , … , π N q ) \boldsymbol{\pi}_q=(\pi_{1q},\ldots,\pi_{Nq}) π q = ( π 1 q , … , π N q ) for each site q q q . The partially exchangeable BB model is
Y ~ i j q ∣ π j q ∼ i n d Bernoulli ( π j q ) , ( i = 1 , … , n q , j = 1 , … , N , q = 1 , … , Q ) , \widetilde{Y}_{ijq}\mid\pi_{jq}\mathrel{\overset{\mathrm{ind}}{\sim}}\operatorname{Bernoulli}(\pi_{jq}),\qquad(i=1,\ldots,n_q,j=1,\ldots,N,q=1,\ldots,Q), Y ij q ∣ π j q ∼ ind Bernoulli ( π j q ) , ( i = 1 , … , n q , j = 1 , … , N , q = 1 , … , Q ) ,
N ∼ Poisson ( λ ) , (4) N\sim\operatorname{Poisson}(\lambda), \tag*{(4)} N ∼ Poisson ( λ ) , (4)
π j q ∼ i n d Beta ( − α q , α q + θ q ) , ( j = 1 , … , N , q = 1 , … , Q ) , \pi_{jq}\mathrel{\overset{\mathrm{ind}}{\sim}}\operatorname{Beta}(-\alpha_q,\alpha_q+\theta_q),\qquad(j=1,\ldots,N,q=1,\ldots,Q), π j q ∼ ind Beta ( − α q , α q + θ q ) , ( j = 1 , … , N , q = 1 , … , Q ) ,
with α q < 0 \alpha_q<0 α q < 0 and θ q > − α q \theta_q>-\alpha_q θ q > − α q . As can be recognized by combining (2) with (3) , our MOSAIC model is equivalent to the partially exchangeable BB model of [14 ] (2025), having set π j q = η j q γ j q \pi_{jq}=\eta_{jq}\gamma_{jq} π j q = η j q γ j q , in the sense that it leads to the same marginal distribution for Y \mathbf{Y} Y , for any fixed value of the parameters ( λ , α , θ ) (\lambda,\boldsymbol{\alpha},\boldsymbol{\theta}) ( λ , α , θ ) . Due to the crucial relevance of this result, we state it in the following theorem, although it is essentially a consequence of [19 ] (1972).
Theorem 1. Let Y \mathbf{Y} Y follow the MOSAIC model with parameters ( λ , α , θ , σ ) (\lambda,\boldsymbol{\alpha},\boldsymbol{\theta},\boldsymbol{\sigma}) ( λ , α , θ , σ ) and set π j q = η j q γ j q \pi_{jq}=\eta_{jq}\gamma_{jq} π j q = η j q γ j q . Then, for every σ = ( σ 1 , … , σ Q ) \boldsymbol{\sigma}=(\sigma_1,\ldots,\sigma_Q) σ = ( σ 1 , … , σ Q ) , the marginal distribution of ( Y , K n ) (\mathbf{Y},K_n) ( Y , K n ) coincides with that of the partially exchangeable BB model (4) with parameters ( λ , α , θ ) (\lambda,\boldsymbol{\alpha},\boldsymbol{\theta}) ( λ , α , θ ) .
Theorem 1 has practical, and not merely conceptual, consequences. Under the model of [14 ] (2025) and MOSAIC, the marginal distribution of Y \mathbf{Y} Y , that is, the likelihood function for ( λ , α , θ ) (\lambda,\boldsymbol{\alpha},\boldsymbol{\theta}) ( λ , α , θ ) , and the posterior distributions of π j q \pi_{jq} π j q and N N N are all available in closed form. The marginal law of Y \mathbf{Y} Y under MOSAIC, or more precisely the joint law of ( Y = y , K n = k ) (\mathbf{Y}=\mathbf{y},K_n=k) ( Y = y , K n = k ) , depends on y \mathbf{y} y only through the sufficient statistics given by the marginal occurrence frequencies of each species. Let M j q = ∑ i = 1 n q Y i j q M_{jq}=\sum_{i=1}^{n_q}Y_{ijq} M j q = ∑ i = 1 n q Y ij q denote the frequency of species X j X_j X j in site q q q , let M = ( M j q ) \mathbf{M}=(M_{jq}) M = ( M j q ) be the corresponding matrix, and let m = ( m j q ) \mathbf{m}=(m_{jq}) m = ( m j q ) be its observed counterpart, for j = 1 , … , k j=1,\ldots,k j = 1 , … , k and q = 1 , … , Q q=1,\ldots,Q q = 1 , … , Q . Then
Pr ( Y = y , K n = k ∣ λ , α , θ ) = λ k k ! exp ( − λ { 1 − ∏ q = 1 Q ( α q + θ q ) n q ( θ q ) n q } ) \Pr(\mathbf{Y}=\mathbf{y},K_n=k\mid\lambda,\boldsymbol{\alpha},\boldsymbol{\theta})=\frac{\lambda^k}{k!}\exp\left(-\lambda\left\{1-\prod_{q=1}^{Q}\frac{(\alpha_q+\theta_q)_{n_q}}{(\theta_q)_{n_q}}\right\}\right) Pr ( Y = y , K n = k ∣ λ , α , θ ) = k ! λ k exp ( − λ { 1 − q = 1 ∏ Q ( θ q ) n q ( α q + θ q ) n q } )
× ∏ q = 1 Q [ − α q ( θ q ) n q ] k ∏ j = 1 k ( 1 − α q ) m j q − 1 ( α q + θ q ) n q − m j q , (5) {}\times\prod_{q=1}^{Q}\left[\frac{-\alpha_q}{(\theta_q)_{n_q}}\right]^k\prod_{j=1}^{k}(1-\alpha_q)_{m_{jq}-1}(\alpha_q+\theta_q)_{n_q-m_{jq}}, \tag*{(5)} × q = 1 ∏ Q [ ( θ q ) n q − α q ] k j = 1 ∏ k ( 1 − α q ) m j q − 1 ( α q + θ q ) n q − m j q , (5)
where ( x ) m = Γ ( x + m ) / Γ ( x ) (x)_m=\Gamma(x+m)/\Gamma(x) ( x ) m = Γ ( x + m ) /Γ ( x ) is the Pochhammer symbol and Γ ( x ) \Gamma(x) Γ ( x ) is the gamma function. The above result is a special case of Theorem 1 in [14 ] (2025); for completeness, the derivation is reported in the Supplementary Material. We refer to (5) as a partially exchangeable feature probability function (pEFPF), since it specializes their partially exchangeable trait probability function to the case of binary traits. It is also related to the partially exchangeable partition probability function of [12 ] .
We now characterize the posterior distributions of the occurrence probabilities π j q = η j q γ j q \pi_{jq} = \eta_{jq}\gamma_{jq} π j q = η j q γ j q and of the total number of species N N N , both of which admit simple closed-form expressions. Let N ′ = N − k N' = N-k N ′ = N − k denote the number of unseen species given Y \mathbf{Y} Y , and split the occurrence probabilities into two groups: those associated with the observed species X 1 , … , X k X_1,\ldots,X_k X 1 , … , X k , denoted by π j q ∗ \pi_{jq}^{*} π j q ∗ for j = 1 , … , k j=1,\ldots,k j = 1 , … , k , and those associated with the unobserved ones, denoted by π j q ′ \pi_{jq}' π j q ′ for j = 1 , … , N ′ j=1,\ldots,N' j = 1 , … , N ′ , in both cases with q = 1 , … , Q q=1,\ldots,Q q = 1 , … , Q .
Theorem 2 ([14 ] ). Let Y \mathbf{Y} Y follow the MOSAIC model with parameters ( λ , α , θ , σ ) (\lambda,\boldsymbol{\alpha},\boldsymbol{\theta},\boldsymbol{\sigma}) ( λ , α , θ , σ ) and set π j q = η j q γ j q \pi_{jq} = \eta_{jq}\gamma_{jq} π j q = η j q γ j q . Then the posterior distribution of the number of unseen species is
N ′ ∣ Y ∼ Poisson { λ p 0 ( α , θ , n ) } , p 0 ( α , θ , n ) = ∏ q = 1 Q p 0 , q ( α q , θ q , n q ) . N' \mid\mathbf{Y} \sim\operatorname{Poisson}\{\lambda p_0(\boldsymbol{\alpha},\boldsymbol{\theta},\mathbf{n})\}, \qquad p_0(\boldsymbol{\alpha},\boldsymbol{\theta},\mathbf{n}) = \prod_{q=1}^{Q} p_{0,q}(\alpha_q,\theta_q,n_q). N ′ ∣ Y ∼ Poisson { λ p 0 ( α , θ , n )} , p 0 ( α , θ , n ) = q = 1 ∏ Q p 0 , q ( α q , θ q , n q ) .
with p 0 , q ( α q , θ q , n q ) = ( α q + θ q ) n q / ( θ q ) n q p_{0,q}(\alpha_q,\theta_q,n_q) = (\alpha_q+\theta_q)_{n_q}/(\theta_q)_{n_q} p 0 , q ( α q , θ q , n q ) = ( α q + θ q ) n q / ( θ q ) n q . Moreover, the occurrence probabilities are independent across species and sites given Y \mathbf{Y} Y , with
π j q ∗ ∣ Y ∼ i n d Beta ( m j q − α q , α q + θ q + n q − m j q ) , ( j = 1 , … , k ) , \pi_{jq}^{*} \mid\mathbf{Y} \mathrel{\overset{\mathrm{ind}}{\sim}} \operatorname{Beta}(m_{jq}-\alpha_q,\alpha_q+\theta_q+n_q-m_{jq}), \qquad(j=1,\ldots,k), π j q ∗ ∣ Y ∼ ind Beta ( m j q − α q , α q + θ q + n q − m j q ) , ( j = 1 , … , k ) ,
π j q ′ ∣ Y ∼ i n d Beta ( − α q , α q + θ q + n q ) , ( j = 1 , … , N ′ ) , \pi_{jq}' \mid\mathbf{Y} \mathrel{\overset{\mathrm{ind}}{\sim}} \operatorname{Beta}(-\alpha_q,\alpha_q+\theta_q+n_q), \qquad(j=1,\ldots,N'), π j q ′ ∣ Y ∼ ind Beta ( − α q , α q + θ q + n q ) , ( j = 1 , … , N ′ ) ,
for q = 1 , … , Q q=1,\ldots,Q q = 1 , … , Q . In particular, the posterior does not depend on σ \boldsymbol{\sigma} σ .
We note that p 0 , q ( α q , θ q , n q ) = ( α q + θ q ) n q / ( θ q ) n q = E { ( 1 − π j q ) n q } p_{0,q}(\alpha_q,\theta_q,n_q) = (\alpha_q+\theta_q)_{n_q}/(\theta_q)_{n_q} = E\{(1-\pi_{jq})^{n_q}\} p 0 , q ( α q , θ q , n q ) = ( α q + θ q ) n q / ( θ q ) n q = E {( 1 − π j q ) n q } is the probability that a species goes unrecorded in the n q n_q n q samples from site q q q , and that p 0 ( α , θ , n ) = ∏ q = 1 Q p 0 , q ( α q , θ q , n q ) p_0(\boldsymbol{\alpha},\boldsymbol{\theta},\mathbf{n}) = \prod_{q=1}^{Q}p_{0,q}(\alpha_q,\theta_q,n_q) p 0 ( α , θ , n ) = ∏ q = 1 Q p 0 , q ( α q , θ q , n q ) is the probability that it goes unrecorded at every site. We will show in Section 3.1 that E ( K n ) = λ { 1 − p 0 ( α , θ , n ) } E(K_n) = \lambda\{1-p_0(\boldsymbol{\alpha},\boldsymbol{\theta},\mathbf{n})\} E ( K n ) = λ { 1 − p 0 ( α , θ , n )} , so that 1 − p 0 ( α , θ , n ) 1-p_0(\boldsymbol{\alpha},\boldsymbol{\theta},\mathbf{n}) 1 − p 0 ( α , θ , n ) and p 0 ( α , θ , n ) p_0(\boldsymbol{\alpha},\boldsymbol{\theta},\mathbf{n}) p 0 ( α , θ , n ) admit a direct interpretation as the expected fractions of the species pool that are respectively discovered and left undiscovered. Moreover, the posterior expectations of the occurrence probabilities are E ( π j q ∗ ∣ Y ) = ( m j q − α q ) / ( θ q + n q ) E(\pi_{jq}^{*}\mid\mathbf{Y}) = (m_{jq}-\alpha_q)/(\theta_q+n_q) E ( π j q ∗ ∣ Y ) = ( m j q − α q ) / ( θ q + n q ) and E ( π j q ′ ∣ Y ) = − α q / ( θ q + n q ) E(\pi_{jq}'\mid\mathbf{Y}) = -\alpha_q/(\theta_q+n_q) E ( π j q ′ ∣ Y ) = − α q / ( θ q + n q ) , for j = 1 , … , k j=1,\ldots,k j = 1 , … , k and j = 1 , … , N ′ j=1,\ldots,N' j = 1 , … , N ′ respectively, and q = 1 , … , Q q=1,\ldots,Q q = 1 , … , Q . The first is a shrunken version of the empirical occurrence frequency m j q / n q m_{jq}/n_q m j q / n q , with − α q -\alpha_q − α q acting as a pseudo-count and θ q \theta_q θ q as a pseudo-sample size. Finally, the parameters above are not known in practice and must themselves be inferred from the data, possibly borrowing information across the Q Q Q sites. This is achieved in Section 3.3 through a hierarchical specification inducing priors on α q \alpha_q α q and θ q \theta_q θ q and a Gamma prior for λ \lambda λ . The expressions of Theorem 2 nonetheless remain valuable, since they are transparent to interpret and can be embedded within an MCMC scheme, where the availability of conditional distributions in closed form yields substantial computational savings. These expressions can moreover be extended beyond the Poisson case: placing a Gamma prior on λ \lambda λ induces a negative binomial distribution for N N N , for which analogous closed forms are available, as detailed in the Supplementary Material.
Biodiversity quantification and inference
Species richness and shared species
In this section, we investigate the theoretical properties of MOSAIC for key quantities of interest in biodiversity studies. Among the many diversity measures, species richness, the total number of species in a community, is arguably the simplest and most widely used. In the multi-site setting a distinction is needed: the richness of a single site, or α \alpha α -diversity, refers to one site at a time, whereas the richness across sites, or γ \gamma γ -diversity, refers to the pooled species list. The results below concern both, together with the number of shared species, a priori and a posteriori. The study of species richness is well established in the Bayesian nonparametric literature for exchangeable data: see [21 ] ; [10 ] ; [43 ] ; [31 ] for abundance data and [13 ] for incidence data. In the multi-site case, that is, under partial exchangeability, closed-form expressions are instead scarce, with the notable exception of [9 ] for abundance data. The findings below are new and were not investigated in [14 ] ; involving essentially only Poisson distributions, they are among the first available in the literature.
We provide results based both on the observed data Y \mathbf{Y} Y and on the latent occupancies W \mathbf{W} W . Recall that K q , n q K_{q,n_q} K q , n q and K n K_n K n denote the number of distinct species observed at site q q q and across all sites, respectively, and let L q , n q L_{q,n_q} L q , n q and L n L_n L n be their latent counterparts based on W \mathbf{W} W , so that K n ≤ L n ≤ N K_n \le L_n \le N K n ≤ L n ≤ N . In the terminology above, K q , n q K_{q,n_q} K q , n q and L q , n q L_{q,n_q} L q , n q are the observed and latent α \alpha α -diversities of site q q q , whereas K n K_n K n and L n L_n L n are their γ \gamma γ -diversity analogues; all four depend on the sampling effort and are bounded by the size N N N of the species pool. In ecological applications, the expectations E ( K 1 ) , … , E ( K n ) E(K_1),\ldots,E(K_n) E ( K 1 ) , … , E ( K n ) provide model-based estimates of the global rarefaction curve ([15 ] , [43 ] ) in terms of the global sampling effort n \mathbf{n} n . Analogously, the sequence E ( K q , 1 ) , … , E ( K q , n q ) E(K_{q,1}),\ldots,E(K_{q,n_q}) E ( K q , 1 ) , … , E ( K q , n q ) defines a model-based estimate of the rarefaction curve at site q q q as a function of the sampling effort n q n_q n q . Recall also that p 0 , q ( α q , θ q , n q ) = ( α q + θ q ) n q ( θ q ) n q p_{0,q}(\alpha_q,\theta_q,n_q) = \frac{(\alpha_q+\theta_q)_{n_q}}{(\theta_q)_{n_q}} p 0 , q ( α q , θ q , n q ) = ( θ q ) n q ( α q + θ q ) n q is the probability that a species goes unrecorded in the n q n_q n q samples from site q q q , and that p 0 ( α , θ , n ) = ∏ q = 1 Q p 0 , q ( α q , θ q , n q ) p_0(\boldsymbol{\alpha},\boldsymbol{\theta},\mathbf{n}) = \prod_{q=1}^{Q}p_{0,q}(\alpha_q,\theta_q,n_q) p 0 ( α , θ , n ) = ∏ q = 1 Q p 0 , q ( α q , θ q , n q ) is the probability that it goes unrecorded at every site. Conversely, we let p 1 , q ( α q , θ q , n q ) : = 1 − p 0 , q ( α q , θ q , n q ) p_{1,q}(\alpha_q,\theta_q,n_q) := 1-p_{0,q}(\alpha_q,\theta_q,n_q) p 1 , q ( α q , θ q , n q ) := 1 − p 0 , q ( α q , θ q , n q ) be the probability that a species is recorded at least once at site q q q , and p 1 ( α , θ , n ) : = 1 − p 0 ( α , θ , n ) p_1(\boldsymbol{\alpha},\boldsymbol{\theta},\mathbf{n}) := 1-p_0(\boldsymbol{\alpha},\boldsymbol{\theta},\mathbf{n}) p 1 ( α , θ , n ) := 1 − p 0 ( α , θ , n ) be the probability that it is recorded at least once overall. Finally, we write θ q ( σ q ) : = σ q ( α q + θ q ) − α q \theta_q(\sigma_q) := \sigma_q(\alpha_q+\theta_q)-\alpha_q θ q ( σ q ) := σ q ( α q + θ q ) − α q and θ ( σ ) : = { θ 1 ( σ 1 ) , … , θ Q ( σ Q ) } \boldsymbol{\theta}(\boldsymbol{\sigma}) := \{\theta_1(\sigma_1),\ldots,\theta_Q(\sigma_Q)\} θ ( σ ) := { θ 1 ( σ 1 ) , … , θ Q ( σ Q )} , noting that θ q ( 1 ) = θ q \theta_q(1)=\theta_q θ q ( 1 ) = θ q . The following proposition provides the marginal distributions of these quantities.
Proposition 1 (A priori distinct species). Under the MOSAIC model with parameters ( λ , α , θ , σ ) (\lambda,\boldsymbol{\alpha},\boldsymbol{\theta},\boldsymbol{\sigma}) ( λ , α , θ , σ ) , the number of distinct species observed at site q q q and across sites is distributed as
K q , n q ∼ Poisson { λ p 1 , q ( α q , θ q , n q ) } , K n ∼ Poisson { λ p 1 ( α , θ , n ) } , ( q = 1 , … , Q ) , K_{q,n_q} \sim\operatorname{Poisson}\{\lambda p_{1,q}(\alpha_q,\theta_q,n_q)\},\qquad K_n \sim\operatorname{Poisson}\{\lambda p_1(\boldsymbol{\alpha},\boldsymbol{\theta},\mathbf{n})\},\qquad(q=1,\ldots,Q), K q , n q ∼ Poisson { λ p 1 , q ( α q , θ q , n q )} , K n ∼ Poisson { λ p 1 ( α , θ , n )} , ( q = 1 , … , Q ) ,
whereas the number of distinct species present at site q q q and across sites is distributed as
L q , n q ∼ Poisson { λ p 1 , q ( α q , θ q ( σ q ) , n q ) } , L n ∼ Poisson { λ p 1 ( α , θ ( σ ) , n ) } , ( q = 1 , … , Q ) . L_{q,n_q} \sim\operatorname{Poisson}\{\lambda p_{1,q}(\alpha_q,\theta_q(\sigma_q),n_q)\},\quad L_n \sim\operatorname{Poisson}\{\lambda p_1(\alpha,\theta(\sigma),n)\},\quad(q=1,\ldots,Q). L q , n q ∼ Poisson { λ p 1 , q ( α q , θ q ( σ q ) , n q )} , L n ∼ Poisson { λ p 1 ( α , θ ( σ ) , n )} , ( q = 1 , … , Q ) .
Proposition 1 shows that the number of distinct species under the MOSAIC model admits a simple, analytically tractable and highly interpretable form. In the expected value E ( K n ) = λ p 1 ( α , θ , n ) E(K_n)=\lambda p_1(\alpha,\theta,n) E ( K n ) = λ p 1 ( α , θ , n ) , the parameter λ \lambda λ is the expected total number of species across sites and p 1 ( α , θ , n ) p_1(\alpha,\theta,n) p 1 ( α , θ , n ) is the fraction of these that is observed under sampling effort n \mathbf{n} n . The site-specific quantities admit the same reading, since E ( K q , n q ) = λ p 1 , q ( α q , θ q , n q ) E(K_{q,n_q})=\lambda p_{1,q}(\alpha_q,\theta_q,n_q) E ( K q , n q ) = λ p 1 , q ( α q , θ q , n q ) . The laws of K q , n q K_{q,n_q} K q , n q and K n K_n K n are recovered from those of L q , n q L_{q,n_q} L q , n q and L n L_n L n in the limit σ q → 1 \sigma_q \to1 σ q → 1 , that is, under perfect detection.
Of key ecological interest in the study of community similarity is the number of species shared between two sites q q q and r r r , denoted by C n q , n r C_{n_q,n_r} C n q , n r , together with its latent counterpart U n q , n r U_{n_q,n_r} U n q , n r based on the occupancies W \mathbf{W} W , so that C n q , n r ≤ U n q , n r C_{n_q,n_r} \le U_{n_q,n_r} C n q , n r ≤ U n q , n r . When Q = 2 Q=2 Q = 2 , the number of shared species is linked to the site-specific and total counts through C n 1 , n 2 = K 1 , n 1 + K 2 , n 2 − K n C_{n_1,n_2}=K_{1,n_1}+K_{2,n_2}-K_n C n 1 , n 2 = K 1 , n 1 + K 2 , n 2 − K n . The following proposition provides the distribution of the number of species shared between any pair of sites.
Proposition 2 (A priori shared species). Under the MOSAIC model with parameters ( λ , α , θ , σ ) (\lambda,\alpha,\theta,\boldsymbol{\sigma}) ( λ , α , θ , σ ) , the number of species shared between sites q q q and r r r , with q ≠ r q \ne r q = r , is distributed as
C n q , n r ∼ Poisson { λ p 1 , q ( α q , θ q , n q ) p 1 , r ( α r , θ r , n r ) } . C_{n_q,n_r} \sim\operatorname{Poisson}\{\lambda p_{1,q}(\alpha_q,\theta_q,n_q)p_{1,r}(\alpha_r,\theta_r,n_r)\}. C n q , n r ∼ Poisson { λ p 1 , q ( α q , θ q , n q ) p 1 , r ( α r , θ r , n r )} .
whereas its latent counterpart is distributed as
U n q , n r ∼ Poisson { λ p 1 , q ( α q , θ q ( σ q ) , n q ) p 1 , r ( α r , θ r ( σ r ) , n r ) } . U_{n_q,n_r} \sim\operatorname{Poisson}\{\lambda p_{1,q}(\alpha_q,\theta_q(\sigma_q),n_q)p_{1,r}(\alpha_r,\theta_r(\sigma_r),n_r)\}. U n q , n r ∼ Poisson { λ p 1 , q ( α q , θ q ( σ q ) , n q ) p 1 , r ( α r , θ r ( σ r ) , n r )} .
Proposition 2 is the first result characterizing the distribution of the number of shared species for partially exchangeable feature allocation models with finitely many features. The mean of C n q , n r C_{n_q,n_r} C n q , n r is the expected species richness λ \lambda λ multiplied by the probability that a species is recorded at both sites, which factorizes because the occurrence probabilities are independent across sites. The same reading applies to U n q , n r U_{n_q,n_r} U n q , n r , which counts the species genuinely present at both sites, and the two coincide in the limit σ q , σ r → 1 \sigma_q,\sigma_r \to1 σ q , σ r → 1 .
The above result allows us to define model-based estimates of rarefaction for shared species between pairs of sites, providing a measure of overlap in species composition. Although the definition of a single-site rarefaction curve is straightforward, its extension to shared species is less direct. The shared rarefaction surface is the matrix of dimension n q × n r n_q \times n_r n q × n r with entries E ( C i q , i r ) E(C_{i_q,i_r}) E ( C i q , i r ) , for i q = 1 , … , n q i_q=1,\ldots,n_q i q = 1 , … , n q and i r = 1 , … , n r i_r=1,\ldots,n_r i r = 1 , … , n r , representing the expected number of shared species as sampling effort increases at both sites. This two-dimensional surface can be summarized by the one-dimensional curve E ( C 1 , 1 ) , E ( C 2 , 2 ) , … , E ( C n ˉ , n ˉ ) E(C_{1,1}),E(C_{2,2}),\ldots,E(C_{\bar{n},\bar{n}}) E ( C 1 , 1 ) , E ( C 2 , 2 ) , … , E ( C n ˉ , n ˉ ) , with n ˉ = min ( n q , n r ) \bar{n}=\min(n_q,n_r) n ˉ = min ( n q , n r ) , obtained by increasing sampling effort jointly and equally at the two sites. The resulting curve is a symmetric and interpretable summary of how overlap accumulates under balanced sampling, and it is most informative when the two sites have similar sample sizes, since the diagonal then spans the entire range of sampling effort. Otherwise the curve stops at n ˉ \bar{n} n ˉ and the full surface should be inspected instead.
Propositions 1-2 concern a priori properties of K q , n q K_{q,n_q} K q , n q , K n K_n K n , C n q , n r C_{n_q,n_r} C n q , n r and of their latent counterparts.
We now consider the important problem of predicting the number of new species that would be discovered if s = ( s 1 , … , s Q ) \mathbf{s} = (s_1,\ldots,s_Q) s = ( s 1 , … , s Q ) additional samples were collected after an initial n \mathbf{n} n , resulting in an enlarged sample of size n + s \mathbf{n}+\mathbf{s} n + s . The key ingredient is the posterior distribution of the number of unseen species N ′ N' N ′ , reported in Theorem 2 together with the posterior of the occurrence probabilities. We denote by K q , s q ( n q ) K_{q,s_q}^{(n_q)} K q , s q ( n q ) and K s ( n ) K_s^{(n)} K s ( n ) the number of new species that would be discovered at site q q q and across all sites, respectively, and by L q , s q ( n q ) L_{q,s_q}^{(n_q)} L q , s q ( n q ) and L s ( n ) L_s^{(n)} L s ( n ) their latent counterparts based on W \mathbf{W} W . These quantities extrapolate the accumulation curves beyond the observed effort, since E ( K n + s ∣ Y ) = k + E ( K s ( n ) ∣ Y ) E(K_{\mathbf{n}+\mathbf{s}} \mid\mathbf{Y}) = k + E(K_s^{(n)} \mid\mathbf{Y}) E ( K n + s ∣ Y ) = k + E ( K s ( n ) ∣ Y ) .
Two groups of species contribute to the predictive of K q , s q ( n q ) K_{q,s_q}^{(n_q)} K q , s q ( n q ) : those not yet recorded anywhere, whose number is random; and the k − k q k-k_q k − k q species recorded at some other site but not at q q q , whose number is known given Y \mathbf{Y} Y . The same reading applies to L q , s q ( n q ) L_{q,s_q}^{(n_q)} L q , s q ( n q ) , with ℓ \ell ℓ and ℓ q \ell_q ℓ q in place of k k k and k q k_q k q .
Theorem 3 (A posteriori distinct species). Under the MOSAIC model with parameters ( λ , α , θ , σ ) (\lambda,\boldsymbol{\alpha},\boldsymbol{\theta},\boldsymbol{\sigma}) ( λ , α , θ , σ ) , the number of new species discovered at site q q q and across sites satisfies
K q , s q ( n q ) ∣ Y = d Poisson { λ p 0 ( α , θ , n ) p 1 , q ( α q , θ q + n q , s q ) } + Binomial { k − k q , p 1 , q ( α q , θ q + n q , s q ) } , ( q = 1 , … , Q ) , K_{q,s_q}^{(n_q)} \mid\mathbf{Y} \mathrel{\overset{d}{=}} \operatorname{Poisson}\left\{\lambda p_0(\boldsymbol{\alpha},\boldsymbol{\theta},\mathbf{n})p_{1,q}(\alpha_q,\theta_q+n_q,s_q)\right\}
+ \operatorname{Binomial}\left\{k-k_q,p_{1,q}(\alpha_q,\theta_q+n_q,s_q)\right\}, \qquad(q=1,\ldots,Q), K q , s q ( n q ) ∣ Y = d Poisson { λ p 0 ( α , θ , n ) p 1 , q ( α q , θ q + n q , s q ) } + Binomial { k − k q , p 1 , q ( α q , θ q + n q , s q ) } , ( q = 1 , … , Q ) ,
K s ( n ) ∣ Y ∼ Poisson { λ p 0 ( α , θ , n ) p 1 ( α , θ + n , s ) } , K_s^{(n)} \mid\mathbf{Y} \sim\operatorname{Poisson}\left\{\lambda p_0(\boldsymbol{\alpha},\boldsymbol{\theta},\mathbf{n})p_1(\boldsymbol{\alpha},\boldsymbol{\theta}+\mathbf{n},\mathbf{s})\right\}, K s ( n ) ∣ Y ∼ Poisson { λ p 0 ( α , θ , n ) p 1 ( α , θ + n , s ) } ,
whereas their latent counterparts satisfy
L q , s q ( n q ) ∣ W = d Poisson { λ p 0 ( α , θ ( σ ) , n ) p 1 , q ( α q , θ q ( σ q ) + n q , s q ) } + Binomial { ℓ − ℓ q , p 1 , q ( α q , θ q ( σ q ) + n q , s q ) } , ( q = 1 , … , Q ) , L_{q,s_q}^{(n_q)} \mid\mathbf{W} \mathrel{\overset{d}{=}} \operatorname{Poisson}\left\{\lambda p_0(\boldsymbol{\alpha},\boldsymbol{\theta}(\boldsymbol{\sigma}),\mathbf{n})p_{1,q}(\alpha_q,\theta_q(\sigma_q)+n_q,s_q)\right\}
+ \operatorname{Binomial}\left\{\ell-\ell_q,p_{1,q}(\alpha_q,\theta_q(\sigma_q)+n_q,s_q)\right\}, \qquad(q=1,\ldots,Q), L q , s q ( n q ) ∣ W = d Poisson { λ p 0 ( α , θ ( σ ) , n ) p 1 , q ( α q , θ q ( σ q ) + n q , s q ) } + Binomial { ℓ − ℓ q , p 1 , q ( α q , θ q ( σ q ) + n q , s q ) } , ( q = 1 , … , Q ) ,
L s ( n ) ∣ W ∼ Poisson { λ p 0 ( α , θ ( σ ) , n ) p 1 ( α , θ ( σ ) + n , s ) } . L_s^{(n)} \mid\mathbf{W} \sim\operatorname{Poisson}\left\{\lambda p_0(\boldsymbol{\alpha},\boldsymbol{\theta}(\boldsymbol{\sigma}),\mathbf{n})p_1(\boldsymbol{\alpha},\boldsymbol{\theta}(\boldsymbol{\sigma})+\mathbf{n},\mathbf{s})\right\}. L s ( n ) ∣ W ∼ Poisson { λ p 0 ( α , θ ( σ ) , n ) p 1 ( α , θ ( σ ) + n , s ) } .
Here and in the following, the sum of two named distributions denotes the distribution of the sum of two independent random variables with those laws. The latent quantities are conditional on W \mathbf{W} W , which is itself unobserved, and are therefore available only within the sampler of Section S.2.1 in the Supplementary Material, where W ~ \widetilde{\mathbf{W}} W is imputed at each iteration. Theorem 3 admits the same interpretation as its a priori counterpart, with the roles of the two factors made explicit: λ p 0 ( α , θ , n ) \lambda p_0(\boldsymbol{\alpha},\boldsymbol{\theta},\mathbf{n}) λ p 0 ( α , θ , n ) is the expected number of species left undiscovered after the initial n \mathbf{n} n samples, and p 1 ( α , θ + n , s ) p_1(\boldsymbol{\alpha},\boldsymbol{\theta}+\mathbf{n},\mathbf{s}) p 1 ( α , θ + n , s ) is the probability that one such species is recorded at least once in the s \mathbf{s} s additional samples. The Bayesian estimator of the number of new species is therefore E ( K s ( n ) ∣ Y ) = λ p 0 ( α , θ , n ) p 1 ( α , θ + n , s ) E(K_s^{(n)} \mid\mathbf{Y}) = \lambda p_0(\boldsymbol{\alpha},\boldsymbol{\theta},\mathbf{n})p_1(\boldsymbol{\alpha},\boldsymbol{\theta}+\mathbf{n},\mathbf{s}) E ( K s ( n ) ∣ Y ) = λ p 0 ( α , θ , n ) p 1 ( α , θ + n , s ) , and the site-specific expressions read analogously, with λ p 0 , q ( α q , θ q , n q ) \lambda p_{0,q}(\alpha_q,\theta_q,n_q) λ p 0 , q ( α q , θ q , n q ) the expected number of species not yet recorded at site q q q . The latter includes species already recorded elsewhere, so that K q , s q ( n q ) K_{q,s_q}^{(n_q)} K q , s q ( n q ) counts species new to site q q q rather than new to the study. These results generalize those of [13 ] in the exchangeable case.
Finally, we characterize the predictive distribution of the number of new species shared between two sites q q q and r r r , denoted by C s q , s r ( n q , n r ) C_{s_q,s_r}^{(n_q,n_r)} C s q , s r ( n q , n r ) , and of its latent counterpart U s q , s r ( n q , n r ) U_{s_q,s_r}^{(n_q,n_r)} U s q , s r ( n q , n r ) based on W \mathbf{W} W . Let k q , r = k q + k r − c q , r k_{q,r}=k_q+k_r-c_{q,r} k q , r = k q + k r − c q , r be the number of distinct species recorded at site q q q or at site r r r , and let ℓ q , r = ℓ q + ℓ r − u q , r \ell_{q,r}=\ell_q+\ell_r-u_{q,r} ℓ q , r = ℓ q + ℓ r − u q , r be its latent counterpart. Three groups of species contribute to the predictive: those not yet recorded at either site, which must appear at both; those recorded only at site q q q , which must appear at site r r r ; and those recorded only at site r r r , which must appear at site q q q .
Theorem 4 (A posteriori shared species). Under the MOSAIC model with parameters ( λ , α , σ ) (\lambda,\alpha,\boldsymbol{\sigma}) ( λ , α , σ ) , the number of species shared between sites q q q and r r r , with q ≠ r q \ne r q = r , that would be discovered in ( s q , s r ) (s_q,s_r) ( s q , s r ) additional samples satisfies
C s q , s r ( n q , n r ) ∣ Y = d Poisson { λ p 0 ( α , θ , n ) p 1 , q ( α q , θ q + n q , s q ) p 1 , r ( α r , θ r + n r , s r ) } + Binomial { k − k q , r , p 1 , q ( α q , θ q + n q , s q ) p 1 , r ( α r , θ r + n r , s r ) } + Binomial { k q − c q , r , p 1 , r ( α r , θ r + n r , s r ) } + Binomial { k r − c q , r , p 1 , q ( α q , θ q + n q , s q ) } , C_{s_q,s_r}^{(n_q,n_r)} \mid\mathbf{Y} \mathrel{\stackrel{d}{=}} \operatorname{Poisson}\left\{\lambda p_0(\alpha,\boldsymbol{\theta},n)p_{1,q}(\alpha_q,\theta_q+n_q,s_q)p_{1,r}(\alpha_r,\theta_r+n_r,s_r)\right\}
\\
+ \operatorname{Binomial}\left\{k-k_{q,r},p_{1,q}(\alpha_q,\theta_q+n_q,s_q)p_{1,r}(\alpha_r,\theta_r+n_r,s_r)\right\}
\\
+ \operatorname{Binomial}\left\{k_q-c_{q,r},p_{1,r}(\alpha_r,\theta_r+n_r,s_r)\right\}
\\
+ \operatorname{Binomial}\left\{k_r-c_{q,r},p_{1,q}(\alpha_q,\theta_q+n_q,s_q)\right\}, C s q , s r ( n q , n r ) ∣ Y = d Poisson { λ p 0 ( α , θ , n ) p 1 , q ( α q , θ q + n q , s q ) p 1 , r ( α r , θ r + n r , s r ) } + Binomial { k − k q , r , p 1 , q ( α q , θ q + n q , s q ) p 1 , r ( α r , θ r + n r , s r ) } + Binomial { k q − c q , r , p 1 , r ( α r , θ r + n r , s r ) } + Binomial { k r − c q , r , p 1 , q ( α q , θ q + n q , s q ) } ,
whereas its latent counterpart satisfies
U s q , s r ( n q , n r ) ∣ W = d Poisson { λ p 0 ( α , θ ( σ ) , n ) p 1 , q ( α q , θ q ( σ q ) + n q , s q ) p 1 , r ( α r , θ r ( σ r ) + n r , s r ) } + Binomial { ℓ − ℓ q , r , p 1 , q ( α q , θ q ( σ q ) + n q , s q ) p 1 , r ( α r , θ r ( σ r ) + n r , s r ) } + Binomial { ℓ q − u q , r , p 1 , r ( α r , θ r ( σ r ) + n r , s r ) } + Binomial { ℓ r − u q , r , p 1 , q ( α q , θ q ( σ q ) + n q , s q ) } , U_{s_q,s_r}^{(n_q,n_r)} \mid\mathbf{W} \mathrel{\stackrel{d}{=}} \operatorname{Poisson}\left\{\lambda p_0(\alpha,\boldsymbol{\theta}(\boldsymbol{\sigma}),n)p_{1,q}(\alpha_q,\theta_q(\sigma_q)+n_q,s_q)p_{1,r}(\alpha_r,\theta_r(\sigma_r)+n_r,s_r)\right\}
\\
+ \operatorname{Binomial}\left\{\ell-\ell_{q,r},p_{1,q}(\alpha_q,\theta_q(\sigma_q)+n_q,s_q)p_{1,r}(\alpha_r,\theta_r(\sigma_r)+n_r,s_r)\right\}
\\
+ \operatorname{Binomial}\left\{\ell_q-u_{q,r},p_{1,r}(\alpha_r,\theta_r(\sigma_r)+n_r,s_r)\right\}
\\
+ \operatorname{Binomial}\left\{\ell_r-u_{q,r},p_{1,q}(\alpha_q,\theta_q(\sigma_q)+n_q,s_q)\right\}, U s q , s r ( n q , n r ) ∣ W = d Poisson { λ p 0 ( α , θ ( σ ) , n ) p 1 , q ( α q , θ q ( σ q ) + n q , s q ) p 1 , r ( α r , θ r ( σ r ) + n r , s r ) } + Binomial { ℓ − ℓ q , r , p 1 , q ( α q , θ q ( σ q ) + n q , s q ) p 1 , r ( α r , θ r ( σ r ) + n r , s r ) } + Binomial { ℓ q − u q , r , p 1 , r ( α r , θ r ( σ r ) + n r , s r ) } + Binomial { ℓ r − u q , r , p 1 , q ( α q , θ q ( σ q ) + n q , s q ) } ,
where in both displays the four terms are independent.
Theorem 4 provides a simple closed-form expression for extrapolating the shared rarefaction surface between a pair of sites, and is the first result analytically characterizing the posterior distribution of the number of shared species for partially exchangeable feature allocation models. Each term corresponds to a distinct group of species. The Poisson and the first binomial term account for species not yet recorded at either site, which must appear at both: the former counts those never recorded anywhere, whose number is itself random and Poisson distributed, whereas the latter counts the k − k q , r k-k_{q,r} k − k q , r species already recorded at some other site, whose number is known given Y \mathbf{Y} Y . The last two binomial terms count species recorded at only one of the two sites, which need appear at the other site alone.
All the above results extend to the case in which the total number of species in (2) follows a negative binomial distribution, which accommodates overdispersion while retaining the same analytical tractability and interpretability. The Supplementary Material reports the details, together with a closed-form expression for the marginal distribution analogous to (5) .
A model-based index of β \beta β -diversity
Compared to species richness, comparatively little work in the Bayesian nonparametric literature has addressed the so-called β \beta β -diversity, namely the heterogeneity of species composition across different sampling regions, despite its central role in the quantification of biodiversity. While the pairwise rarefaction and extrapolation curves of Section 3.1 based on shared species are valuable, it is often convenient to summarize heterogeneity across regions in a single number. Many measures of β \beta β -diversity have been introduced in ecology, but there is no overall consensus on which one is the most appropriate ([2 ] ). The oldest is the decomposition of [42 ] , in which the γ \gamma γ -diversity of a set of sites is expressed as the product of the average α \alpha α -diversity of the individual sites and a multiplicative factor, the β \beta β -diversity, measuring how much the pooled species list exceeds what a typical site contains. A second family of measures is instead based on dissimilarities; for incidence data the most popular are the Jaccard and Sørensen indices ([40 ] ). Jaccard similarity is the number of shared species divided by the total number of species, namely ∣ X q ∩ X r ∣ / ∣ X q ∪ X r ∣ |\mathcal{X}_q \cap\mathcal{X}_r|/|\mathcal{X}_q \cup\mathcal{X}_r| ∣ X q ∩ X r ∣/∣ X q ∪ X r ∣ in our notation, for q ≠ r q \ne r q = r ; Sørensen is similar but places more weight on the overlap, so that for the same two sets it never returns a lower value than Jaccard. [3 ] point out that many commonly used β \beta β -diversity indices are highly correlated and redundant, yet do not properly account for undetected shared and unshared species. Our proposed MOSAIC model offers a principled way to solve this problem, based on a ratio of shared species between and within sites.
Definition 1. The β \beta β -diversity between sites q q q and r r r , with q ≠ r q \ne r q = r , is
β q r = 1 − ∑ j = 1 N η j q η j r ( ∑ j = 1 N η j q 2 ∑ j = 1 N η j r 2 ) 1 / 2 \beta_{qr}=1-\frac{\sum_{j=1}^{N}\eta_{jq}\eta_{jr}}{\left(\sum_{j=1}^{N}\eta_{jq}^{2}\sum_{j=1}^{N}\eta_{jr}^{2}\right)^{1/2}} β q r = 1 − ( ∑ j = 1 N η j q 2 ∑ j = 1 N η j r 2 ) 1/2 ∑ j = 1 N η j q η j r
The proposed index is the cosine dissimilarity between the occupancy profiles of the two sites; it is therefore symmetric, β q r = β r q \beta_{qr}=\beta_{rq} β q r = β r q , and, since the occupancy probabilities are nonnegative, conveniently bounded between 0 and 1. It equals 0 when the two profiles are proportional, and 1 when no species has positive occupancy probability at both sites; in general, increasing values of β q r \beta_{qr} β q r indicate greater heterogeneity between sites. The numerator of 1 − β q r 1-\beta_{qr} 1 − β q r is the expected number of species shared in two samples drawn from different sites, say the i i i th sample from site q q q and the i ′ i' i ′ th from site r r r ,
∑ j = 1 N Pr ( W ~ i j q = W ~ i ′ j r = 1 ∣ η q , η r ) = ∑ j = 1 N η j q η j r , \sum_{j=1}^{N}\Pr(\widetilde{W}_{ijq}=\widetilde{W}_{i'jr}=1\mid\boldsymbol{\eta}_q,\boldsymbol{\eta}_r)=\sum_{j=1}^{N}\eta_{jq}\eta_{jr}, j = 1 ∑ N Pr ( W ij q = W i ′ j r = 1 ∣ η q , η r ) = j = 1 ∑ N η j q η j r ,
which equals E ( U 1 , 1 ∣ η q , η r ) E(U_{1,1}\mid\boldsymbol{\eta}_q,\boldsymbol{\eta}_r) E ( U 1 , 1 ∣ η q , η r ) in the notation of Section 3.1, whereas each term in the denominator is the expected number of species shared in two samples i ≠ i ′ i\ne i' i = i ′ drawn from the same site,
∑ j = 1 N Pr ( W ~ i j q = W ~ i ′ j q = 1 ∣ η q ) = ∑ j = 1 N η j q 2 . \sum_{j=1}^{N}\Pr(\widetilde{W}_{ijq}=\widetilde{W}_{i'jq}=1\mid\boldsymbol{\eta}_q)=\sum_{j=1}^{N}\eta_{jq}^{2}. j = 1 ∑ N Pr ( W ij q = W i ′ j q = 1 ∣ η q ) = j = 1 ∑ N η j q 2 .
The denominator thus rescales the between-site co-occurrence by the geometric mean of the two within-site co-occurrences, so that the index measures similarity of occupancy patterns rather than their overall magnitude: two sites whose profiles differ by a common multiplicative factor are regarded as maximally similar.
The index β q r \beta_{qr} β q r is monotonically equivalent to the chord distance of [26 ] , although the latter is commonly applied to empirical relative abundances rather than to model-based probabilities. Replacing each η j q \eta_{jq} η j q with the empirical indicator 1 ( X ~ j ∈ X q ) 1(\widetilde{X}_j \in\mathcal{X}_q) 1 ( X j ∈ X q ) yields instead one minus the Ochiai similarity coefficient [25 ] , 1 − ∣ X q ∩ X r ∣ ∣ X q ∣ ∣ X r ∣ 1 - \frac{|\mathcal{X}_q \cap\mathcal{X}_r|}{\sqrt{|\mathcal{X}_q||\mathcal{X}_r|}} 1 − ∣ X q ∣∣ X r ∣ ∣ X q ∩ X r ∣ , since the sums appearing in β q r \beta_{qr} β q r then count the species shared by the two sites and those observed at each of them; the Jaccard index differs in that it normalizes the shared species by ∣ X q ∪ X r ∣ |\mathcal{X}_q \cup\mathcal{X}_r| ∣ X q ∪ X r ∣ . A further connection is with the correlation between random probability measures in Proposition 4 of [12 ] , which equals the probability of a tie across two groups, normalized by the geometric mean of the two within-group tie probabilities: our index has the same structure, with co-occurrences of binary features replacing ties among species labels, and thus provides its feature allocation counterpart. We note in passing that their quantity would in turn provide a natural definition of β \beta β -diversity for multivariate species sampling models, based on relative abundances rather than on presence–absence patterns.
Our definition is based on the occupancy probabilities η q \eta_q η q rather than on the occurrence probabilities π q \pi_q π q , since under-detection thins the recorded occurrences and would otherwise bias the resulting measure of biodiversity; it is moreover defined over the complete list of N N N species, and hence accounts for the species left unobserved. Since β q r \beta_{qr} β q r is a function of occupancy probabilities whose posterior is available, it admits full uncertainty quantification in addition to point estimation. However, as noted in Remark 1, the occupancy probabilities η q \eta_q η q cannot be learned from the data, and their posterior law is entirely driven by the choice of σ q \sigma_q σ q . The induced β \beta β -diversity β q r \beta_{qr} β q r is nonetheless considerably more stable, because varying σ q \sigma_q σ q shrinks all the occupancy probabilities at site q q q by a common factor and the cosine similarity is invariant to such a global rescaling. Finally, the same construction applies more broadly to any probabilistic framework that yields species-occurrence probabilities, such as joint species distribution models [28 ] , on replacing η q \eta_q η q by π q \pi_q π q and N N N by k k k ; the resulting index no longer corrects for under-detection or for unseen species, but remains well defined.
Hierarchical prior specification with site-specific covariates
In this section, we discuss the prior specification for the MOSAIC model. In order to induce borrowing of information across sites, we define hierarchical priors for the parameters α \alpha α and θ \theta θ . Given the formal equivalence with the BB model shown in Theorem 1, we focus on this construction for easier computation and interpretation. Hence, we consider π j q ∣ α q , θ q ∼ Beta ( − α q , α q + θ q ) \pi_{jq} \mid\alpha_q,\theta_q \sim\operatorname{Beta}(-\alpha_q,\alpha_q+\theta_q) π j q ∣ α q , θ q ∼ Beta ( − α q , α q + θ q ) with α q < 0 \alpha_q < 0 α q < 0 and θ q > − α q \theta_q > -\alpha_q θ q > − α q in (4) , and we need to define a hyperprior for α q \alpha_q α q and θ q \theta_q θ q , for q = 1 , … , Q q = 1,\ldots,Q q = 1 , … , Q . For convenience, we reparametrize the beta distribution in terms of mean and precision, considering
μ q = − α q θ q , ϕ q = θ q , with μ q ∈ ( 0 , 1 ) , ϕ q > 0 , (6) \mu_q = \frac{-\alpha_q}{\theta_q}, \quad\phi_q = \theta_q, \quad\text{with } \mu_q \in(0,1), \ \phi_q > 0, \tag*{(6)} μ q = θ q − α q , ϕ q = θ q , with μ q ∈ ( 0 , 1 ) , ϕ q > 0 , (6)
for q = 1 , … , Q q = 1,\ldots,Q q = 1 , … , Q . Therefore, π j q ∣ μ q , ϕ q ∼ Beta { μ q ϕ q , ( 1 − μ q ) ϕ q } \pi_{jq} \mid\mu_q,\phi_q \sim\operatorname{Beta}\{\mu_q\phi_q,(1-\mu_q)\phi_q\} π j q ∣ μ q , ϕ q ∼ Beta { μ q ϕ q , ( 1 − μ q ) ϕ q } , with
E ( π j q ∣ μ q , ϕ q ) = μ q , var ( π j q ∣ μ q , ϕ q ) = μ q ( 1 − μ q ) ϕ q + 1 . E(\pi_{jq} \mid\mu_q,\phi_q) = \mu_q, \quad\operatorname{var}(\pi_{jq} \mid\mu_q,\phi_q) = \frac{\mu_q(1-\mu_q)}{\phi_q+1}. E ( π j q ∣ μ q , ϕ q ) = μ q , var ( π j q ∣ μ q , ϕ q ) = ϕ q + 1 μ q ( 1 − μ q ) .
In biodiversity studies, it is common to have site-specific covariates, such as mean wind speed, mean temperature, and mean precipitation. We incorporate this information into a regression model for μ q \mu_q μ q . Letting ζ = ( ζ 1 , … , ζ H ) T \boldsymbol{\zeta} = (\zeta_1,\ldots,\zeta_H)^{\mathsf{T}} ζ = ( ζ 1 , … , ζ H ) T denote regression coefficients and z q = ( z q 1 , … , z q H ) T \mathbf{z}_q = (z_{q1},\ldots,z_{qH})^{\mathsf{T}} z q = ( z q 1 , … , z q H ) T site-specific covariates, we assume
log ( μ q 1 − μ q ) = z q T ζ , ζ ∼ N ( b 0 , B 0 ) , (7) \log\left(\frac{\mu_q}{1-\mu_q}\right) = \mathbf{z}_q^{\mathsf{T}}\boldsymbol{\zeta}, \quad\boldsymbol{\zeta} \sim\mathcal{N}(\mathbf{b}_0,\mathbf{B}_0), \tag*{(7)} log ( 1 − μ q μ q ) = z q T ζ , ζ ∼ N ( b 0 , B 0 ) , (7)
for q = 1 , … , Q q = 1,\ldots,Q q = 1 , … , Q , and we let Z Z Z denote the Q × H Q \times H Q × H covariate matrix. This specification admits a direct interpretation of the regression coefficients in terms of odds ratios, analogous to standard logistic regression. For the dispersion parameters ϕ = ( ϕ 1 , … , ϕ Q ) T \boldsymbol{\phi} = (\phi_1,\ldots,\phi_Q)^{\mathsf{T}} ϕ = ( ϕ 1 , … , ϕ Q ) T , we assume a common distribution across sites, ϕ q ∼ Gamma ( a ϕ , b ϕ ) \phi_q \sim\operatorname{Gamma}(a_\phi,b_\phi) ϕ q ∼ Gamma ( a ϕ , b ϕ ) for q = 1 , … , Q q = 1,\ldots,Q q = 1 , … , Q . Since the detectability parameter σ q \sigma_q σ q is not identifiable, we take it to be common across sites and conduct a sensitivity analysis over a set of plausible values, for instance { 0.4 , 0.6 , 0.8 } \{0.4, 0.6, 0.8\} { 0.4 , 0.6 , 0.8 } ; ideally, prior information on the detectability of species should guide this choice. For the parameter of the Poisson distribution for N N N , we assume λ ∼ Gamma ( ρ 0 , ρ 0 / ν 0 ) \lambda\sim\operatorname{Gamma}(\rho_0,\rho_0/\nu_0) λ ∼ Gamma ( ρ 0 , ρ 0 / ν 0 ) , which implies N ∼ NegBinomial ( r 0 , ν 0 ) N \sim\operatorname{NegBinomial}(r_0,\nu_0) N ∼ NegBinomial ( r 0 , ν 0 ) . We recall that all the theoretical results in Section 3.1 extend to the negative binomial case, as detailed in the Supplementary Material.
Simulation studies
In this section, we evaluate the performance of the proposed approach with synthetic data. Specifically, we assess estimation accuracy for the regression coefficients ζ \boldsymbol{\zeta} ζ , which capture covariate effects, and the dispersion parameters ϕ \boldsymbol{\phi} ϕ , as well as predictive performance for the number of shared species between site pairs, C s q , s r ( n q , n r ) C_{s_q,s_r}^{(n_q,n_r)} C s q , s r ( n q , n r ) , and the number of site-specific distinct species, K q , s q ( n q ) K_{q,s_q}^{(n_q)} K q , s q ( n q ) . To the best of our knowledge, there are no existing methods specifically designed for predicting the number of shared species across sites, as considered here. However, for the prediction of K q , s q ( n q ) K_{q,s_q}^{(n_q)} K q , s q ( n q ) , different feature allocation models are available. Accordingly, we adopt as a natural benchmark the exchangeable negative binomial mixture of beta–Bernoulli models proposed by [13 ] .
We begin by evaluating the performance of the proposed approach in estimating the number of shared species in a simple setting with Q = 3 Q = 3 Q = 3 sites. We generate occurrence data under the hierarchical formulation of the MOSAIC model with N = 3,000 N = 3{,}000 N = 3 , 000 , regression coefficients ζ = ( − 3 , 0.5 , − 0.3 ) T \boldsymbol{\zeta} = (-3,0.5,-0.3)^{\mathsf{T}} ζ = ( − 3 , 0.5 , − 0.3 ) T , where the negative intercept reflects the presence of many rare species, and draw ϕ \boldsymbol{\phi} ϕ from a Gamma distribution
######## Additional results for simulation study with shape 4 and rate 2. The elements of the design matrix Z Z Z are simulated from standard normal variables. We consider sample sizes n = ( 100 , 80 , 80 ) \boldsymbol{n} = (100, 80, 80) n = ( 100 , 80 , 80 ) and split the data into training and test sets, using 60% of the observations in each site for training. The MOSAIC model is fitted to the training data using the following hyperparameter specification: b 0 = ( − 2 , 0 , 0 ) T \boldsymbol{b}_0 = (-2, 0, 0)^{\mathsf{T}} b 0 = ( − 2 , 0 , 0 ) T , B 0 = I B_0 = I B 0 = I , ρ 0 = 10 \rho_0 = 10 ρ 0 = 10 , ν 0 = 1.5 k \nu_0 = 1.5k ν 0 = 1.5 k , a ϕ = 2 a_{\phi} = 2 a ϕ = 2 and b ϕ = 2 b_{\phi} = 2 b ϕ = 2 . The MCMC algorithm is run for 8,000 iterations, of which the first half is discarded as burn-in. We predict the number of shared species C s q , s r ( n q , n r ) C_{s_q,s_r}^{(n_q,n_r)} C s q , s r ( n q , n r ) for each pair of sites in the test set, with s = ( 40 , 32 , 32 ) \boldsymbol{s} = (40, 32, 32) s = ( 40 , 32 , 32 ) .
Figure 1 displays the rarefaction surfaces for shared species on the test set, together with the corresponding model-based predictions. The posterior mean closely follows the empirical surfaces for all pairs, indicating a strong agreement between observed and predicted values. For the pair with the same number of samples, (2, 3), we additionally report the full one-dimensional rarefaction curve, corresponding to the diagonal of the surface, together with its model-based prediction on the test set. This representation further highlights the strong agreement between empirical and predicted values and provides a clearer view of how the number of shared species increases under balanced sampling effort. Figure S2 in the Supplementary Material shows the 95% posterior credible intervals for the predicted rarefaction surfaces for all site pairs.
Figure 1. Top panel: empirical rarefaction surfaces for shared species in the test set, together with the corresponding posterior mean estimates of C s q , s r ( n q , n r ) C_{s_q,s_r}^{(n_q,n_r)} C s q , s r ( n q , n r ) for all pairs of sites. Bottom panel: simplified one-dimensional rarefaction curve for shared species (black dots), alongside the posterior mean of C s 2 , s 3 ( n 2 , n 3 ) C_{s_2,s_3}^{(n_2,n_3)} C s 2 , s 3 ( n 2 , n 3 ) (solid line) and its 95% credible intervals for site pair 2-3. Gray vertical line indicates the training set.
Figure S2. Posterior summaries of the rarefaction surfaces for shared species across site pairs, showing the upper bounds of the 95% credible intervals (bottom panels), posterior means (middle panels), and lower bounds (top panels).
We next consider a more complex setting with Q = 15 Q = 15 Q = 15 sites, where the sample sizes n q n_q n q are drawn uniformly between 15 and 40 and N N N follows a negative binomial distribution with mean 5,000 and size 1,000. The parameters ζ \zeta ζ and ϕ \phi ϕ are set as in the previous experiment. This configuration reflects typical biodiversity data. Data are generated under two scenarios: (A) the correctly specified MOSAIC model and (B) a misspecified version in which a random intercept is added to the linear predictor in (7) , log { μ q / ( 1 − μ q ) } = z q T ζ + u q \log\{\mu_q/(1-\mu_q)\} = \boldsymbol{z}_q^{\mathsf{T}}\zeta+ u_q log { μ q / ( 1 − μ q )} = z q T ζ + u q , with u q ∼ N ( 0 , τ 2 ) u_q \sim\mathcal{N}(0,\tau^2) u q ∼ N ( 0 , τ 2 ) and τ = 0.1 \tau= 0.1 τ = 0.1 . For each scenario, we replicate the experiment 25 times. Each dataset is split into training and test sets, with 10 observations in the test set for all sites. The MOSAIC model is fitted to the training data using the same hyperparameter configuration as in the previous experiment. Based on the fitted model, we predict the number of new species at each site, K q , s q ( n q ) K_{q,s_q}^{(n_q)} K q , s q ( n q ) , with s q = 10 s_q = 10 s q = 10 for q = 1 , … , Q q = 1,\ldots,Q q = 1 , … , Q . Predictive performance is evaluated using the average mean squared error across sites, scaled by the true values, for each s q = 1 , … , 10 s_q = 1,\ldots,10 s q = 1 , … , 10 . We compare our approach with the exchangeable negative binomial mixture of beta–Bernoulli models [13 ] , using the same negative binomial hyperparameter specification as in our model. Figure 2 shows that the proposed method achieves comparable or improved predictive performance in both scenarios, highlighting the benefit of borrowing information across sites. Moreover, we recall that exchangeable feature allocation models of this type do not provide estimates or predictions for the number of shared species between sites. In the Supplementary Material, we provide additional results for experiments with varying numbers of sites Q Q Q , together with additional experiments evaluating the frequentist coverage of posterior credible intervals for the regression coefficients ζ \zeta ζ and dispersion parameters ϕ \phi ϕ .
Figure 2. Average MSE in predicting K q , s q ( n q ) K_{q,s_q}^{(n_q)} K q , s q ( n q ) for each s q = 1 , … , 10 s_q = 1,\ldots,10 s q = 1 , … , 10 for all q = 1 , … , Q q = 1,\ldots,Q q = 1 , … , Q for MOSAIC and the exchangeable negative binomial mixture of beta-Bernoulli (BB).
Global patterns in airborne fungal diversity
We analyze data from a global project on the biodiversity of fungi, the GSSP ([27 ] ; [1 ] ). Fungi are among the most diverse and ecologically important kingdoms of life, despite fungal diversity and its environmental drivers remaining poorly understood ([1 ] ). We consider samples in temperate and polar-continental climatic zones, for a total of Q = 34 Q = 34 Q = 34 sites in the Northern hemisphere, covering North America, Europe, Asia and the Arctic. In the Supplementary Material we show the map with the sampling locations. The data consist of 1,425 observations over 1 year with site-specific sample sizes n q n_q n q ranging between 12 and 103, with a median of
######## Additional details and results on fungal biodiversity application
The Supplementary Material reports the GSSP labels used in the following to denote the sampling locations, together with the corresponding sample sizes for all sites.
The total number of identified species is k = 15,352 k = 15{,}352 k = 15 , 352 . Data are very sparse, as more than half of the species (57%) have only been observed once. To account for climatic and latitudinal effects, we include latitude and mean annual temperature along with its quadratic term as covariates. In addition, weather covariates may influence variation in fungal communities [1 ] . Consequently, we also include the mean annual wind speed and the mean annual precipitation for each sampling area, leading to H = 6 H = 6 H = 6 columns of Z Z Z .
We fit MOSAIC using the MCMC specification in the simulation study. The posterior mean of the global number of species N N N is 16,574, with 95% credible interval (16,494, 16,659). This estimate exceeds the 15,352 observed species, suggesting that about 1,200 species remain undetected. tbl:1 reports estimates of the regression coefficients ζ \zeta ζ . The large negative intercept reflects the extreme sparsity of the data. Temperature and latitude emerge as the main drivers of the distribution of airborne fungi, consistent with [1 ] and previous studies on soil fungi [37 ] . Fungi follow a predictable latitudinal diversity gradient, with prevalence decreasing with distance from the Equator. The mean annual temperature coefficients reveal a hump-shaped relationship with temperature, where fungal activity increases with temperature up to an optimal point before declining at higher, physiologically stressful temperatures. This pattern is commonly described as a thermal performance curve [17 ] . The negative sign of the mean annual precipitation coefficient is consistent with rainfall washout, where fungal spores are scavenged from the air and deposited onto surfaces. Several studies have reported negative relationships between rainfall and airborne fungal abundance, see, e.g., [29 ] , with genera such as Cladosporium and Alternaria being
caption: Posterior means and 95% credible intervals for ζ \zeta ζ .
:::
particularly susceptible to rainfall-induced declines in airborne occurrence [18 ] .
To assess diversity in fungal composition we compute the β \beta β -diversity index proposed in Section 3.2. Figure 3 displays the posterior mean of the similarity 1 − β q r 1-\beta_{qr} 1 − β q r for all pairs of sites, with detectability parameter σ q = 0.6 \sigma_q = 0.6 σ q = 0.6 for all sites. Because the index is invariant to a global rescaling of the occupancy probabilities, the choice of σ q \sigma_q σ q leaves the overall structure essentially unchanged, as illustrated in the Supplementary Material. The results are compared to those obtained via the Jaccard and Ochiai similarity indices [25 ] . Although the overall patterns are broadly consistent, the proposed index reveals more complex and nuanced biodiversity patterns, particularly for sites with small sample sizes. For example, our measure identifies a small cluster of High Arctic and Arctic tundra locations, comprising CHA, ZAC, NUU, PRI and TOO, that is not visible in the Jaccard index. These sites are associated with cold-climate ecosystems and are geographically isolated, all having relatively small sample sizes n q n_q n q , highlighting the importance of borrowing information across locations in the estimation procedure.
Figure 3. Posterior mean of the proposed β \beta β -similarity for all sites, grouped by climatic zone, compared with Jaccard and Ochiai similarities.
In general, airborne fungal communities exhibit a pronounced biogeographic organization, with temperate European sites forming a highly connected cluster (from KLA to ZUR) and polar-continental locations in Europe grouping separately (from TUR to TAR). These patterns are consistent with evidence that fungal community composition is strongly shaped by climatic zone [1 ] . The high similarity among temperate European sites likely reflects shared climate and atmospheric factors, while Arctic and polar-continental sites appear to be more strongly shaped by environmental filtering and geographic isolation. For example, the marked dissimilarity of CAM (Sandgerði, Iceland) and CHA (Cambridge Bay, Canada) from most other locations is ecologically meaningful and consistent with Arctic aerobiology and microbial biogeography theory [7 ] . This likely reflects their atypical climatic settings, with Sandgerði influenced by strong wind exposure and North Atlantic air-mass transport and Cambridge Bay characterized by extreme continental isolation and sparse vegetation, both of which may shape local airborne fungal assemblages differently from typical polar-continental sites.
To further explore fungal biodiversity, we produce extrapolations of rarefaction curves for shared species between pairs of sites, C s q , s r ( n q , n r ) C_{s_q,s_r}^{(n_q,n_r)} C s q , s r ( n q , n r ) . Since most sites have different sample sizes, the one-dimensional summary curve taking the diagonal of the whole surface has limited utility here. We instead construct extrapolations of rarefaction curves by simulating from the posterior under two prediction sets: (i) a one-side test set, where we simulate additional observations only for the site with the smaller sample size and compute the number of shared species using the remaining observations from the other site; and (ii) a full test set, where we simulate additional observations from both sites. Assuming n q > n r n_q > n_r n q > n r , the size of the one-side test set is n q − n r n_q - n_r n q − n r . This approach makes full use of the available data, avoiding unnecessary discarding of observations. The results with test size s q = 1 , … , 30 s_q = 1, \ldots, 30 s q = 1 , … , 30 are shown in Figure 4 for three illustrative site pairs: a pair of temperate locations, Klagenfurt (KLA) and Zurich (ZUR); a pair of polar-continental locations, Nuuk (NUU) and Wrangel Island (PRI); and a cross-zone pair, Klagenfurt and Fetsund (FTS). The figure also displays the posterior density of β q r \beta_{qr} β q r for each pair.
Figure 4. Top panel: empirical site-specific rarefaction curves (dots), posterior mean (solid lines) and 95% credible intervals (shaded ribbons) for one-side test and test predictions across three sampling location pairs. Gray vertical lines indicate training and one-side test sets. Bottom panel: posterior densities of β q r \beta_{qr} β q r for the same three pairs.
The expected number of new shared species grows rather slowly when collecting additional balanced samples for the Nuuk–Wrangel Island pair, which exhibits the highest β \beta β -diversity among the three. As previously noted, Arctic locations tend to appear quite distinct, likely reflecting the combined effects of geographic isolation, sparse tundra vegetation, and limited regional spore sources. Growth is faster for the remaining two pairs, which comprise locations in Europe, even though Fetsund belongs to the polar-continental climatic zone, consistent with lower β \beta β -diversity. In the Supplementary Material we provide additional experiments assessing the model performance in predicting the number of shared species in out-of-sample experiments.
To explore the detectability component, we provide site-specific rarefaction curves for the latent number of species L q , n q L_{q,n_q} L q , n q . Figure 5 shows results for three sites under three values of σ ∈ { 0.05 , 0.5 , 0.95 } \sigma\in\{0.05, 0.5, 0.95\} σ ∈ { 0.05 , 0.5 , 0.95 } . These values span a range of detectability regimes: σ = 0.05 \sigma= 0.05 σ = 0.05 corresponds to almost no species being detectable, σ = 0.95 \sigma= 0.95 σ = 0.95 to near-perfect detectability, and σ = 0.5 \sigma= 0.5 σ = 0.5 to an intermediate case. The curves characterize the growth rate of the number of species based on the latent occupancy data W \mathbf{W} W , which has no directly observed counterpart. Across all three sites, the effect of detectability is clearly visible: the expected number of species increases substantially as detectability decreases, highlighting how undetected species can represent a large fraction of the underlying diversity. We stress that these are illustrative scenarios rather than inferences about detectability: as discussed in Remark 1, σ \sigma σ is not identified from the data, so each curve reports what the latent diversity would be under the corresponding assumption. In the Supplementary Material we provide analogous results for the rarefaction curves of the latent number of shared species based on the occupancy data, U n q , n r U_{n_q,n_r} U n q , n r .
Figure 5. Posterior means and 95% credible intervals for site-specific rarefaction curves for the latent number of species L q , n q L_{q,n_q} L q , n q for different values of σ \sigma σ .
Discussion
We have proposed a MOSAIC modeling framework, defining a novel partially exchangeable feature allocation model with imperfect detection. Our motivation is to define a principled statistical framework for inference on β \beta β -diversity and heterogeneity in community composition across sites. In order for this framework to be useful to ecologists, including in the framework of modern large scale biodiversity surveys, it is crucial for model fitting to be both efficient and scalable. For this reason, we have balanced ecological realism and computational efficiency and scalability in MOSAIC. Data and code to reproduce our analysis are available at https://github.com/federicastolf/MOSAIC .
To obtain analytic tractability, we have made some simplifying assumptions that are not entirely realistic in ecology. For example, unlike joint species distribution models (JSDMs), we have not modeled statistical dependence in co-occurrence across species. Modern versions of JSDMs typically rely on Bayesian factor models for high-dimensional multivariate binary outcome species occurrence data, and substantial challenges arise in implementing such models for highly diverse communities containing 1,000s to millions of species [24 ] . In addition, a key aspect that forms much of the focus of the MOSAIC framework is the ability to allow and predict new species discovery as sampling proceeds, which is currently lacking in JSDMs models with the exception of [36 ] . In future work, it will be of substantial interest to define new approaches bridging between JSDMs and feature allocation models such as MOSAIC.
An additional key consideration in conducting inference in MOSAIC and related models is the challenge of non-identifiability of detectability parameters. For other types of ecological occupancy models, detectability parameters can sometimes be inferred using targeted sampling designs, such as capture-recapture [30 ] . For ecological surveys using cyclone samplers or malaise traps and DNA meta-barcoding, such designs are not possible, as one cannot release the organisms. An important open problem is developing new designs that allow inference on detectability parameters.
Acknowledgments
This research was partially supported by the European Research Council under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 856506) and the National Science Foundation (IIS-2426762). The authors thank Otso Ovaskainen, Tomas Roslin, and other members of the Lifeplan project team for helpful discussions.
Supplementary materials
Proofs of theoretical results
####### Auxiliary results
Lemma S1. Let V ≥ 1 V \ge1 V ≥ 1 be an integer and let ϱ 1 , … , ϱ V \varrho_1,\ldots,\varrho_V ϱ 1 , … , ϱ V be independent and identically distributed random variables with values in ( 0 , 1 ) (0,1) ( 0 , 1 ) . Let G 1 , … , G V G_1,\ldots,G_V G 1 , … , G V be conditionally independent Bernoulli random variables given ( ϱ 1 , … , ϱ V ) (\varrho_1,\ldots,\varrho_V) ( ϱ 1 , … , ϱ V ) , with pr ( G j = 1 ∣ ϱ 1 , … , ϱ V ) = ϱ j \operatorname{pr}(G_j=1\mid\varrho_1,\ldots,\varrho_V)=\varrho_j pr ( G j = 1 ∣ ϱ 1 , … , ϱ V ) = ϱ j for j = 1 , … , V j=1,\ldots,V j = 1 , … , V . Then
∑ j = 1 V G j ∼ Binomial { V , E ( ϱ 1 ) } . \sum_{j=1}^{V}G_j \sim\operatorname{Binomial}\{V,E(\varrho_1)\}. j = 1 ∑ V G j ∼ Binomial { V , E ( ϱ 1 )} .
Proof. Consider the probability generating function G ( z ) = E ( z ∑ j = 1 V G j ) G(z)=E(z^{\sum_{j=1}^{V}G_j}) G ( z ) = E ( z ∑ j = 1 V G j ) . By the tower property and the conditional independence of the G j G_j G j ’s,
E ( z ∑ j = 1 V G j ) = E { E ( z ∑ j = 1 V G j ∣ ϱ 1 , … , ϱ V ) } = E { ∏ j = 1 V E ( z G j ∣ ϱ 1 , … , ϱ V ) } . E\left(z^{\sum_{j=1}^{V}G_j}\right)=E\left\{E\left(z^{\sum_{j=1}^{V}G_j}\mid\varrho_1,\ldots,\varrho_V\right)\right\}=E\left\{\prod_{j=1}^{V}E\left(z^{G_j}\mid\varrho_1,\ldots,\varrho_V\right)\right\}. E ( z ∑ j = 1 V G j ) = E { E ( z ∑ j = 1 V G j ∣ ϱ 1 , … , ϱ V ) } = E { j = 1 ∏ V E ( z G j ∣ ϱ 1 , … , ϱ V ) } .
Since E ( z G j ∣ ϱ 1 , … , ϱ V ) = 1 − ϱ j + ϱ j z E(z^{G_j}\mid\varrho_1,\ldots,\varrho_V)=1-\varrho_j+\varrho_jz E ( z G j ∣ ϱ 1 , … , ϱ V ) = 1 − ϱ j + ϱ j z depends on ϱ j \varrho_j ϱ j only, and the ϱ j \varrho_j ϱ j ’s are independent and identically distributed,
E ( z ∑ j = 1 V G j ) = ∏ j = 1 V E ( 1 − ϱ j + ϱ j z ) = { 1 − E ( ϱ 1 ) + E ( ϱ 1 ) z } V , E\left(z^{\sum_{j=1}^{V}G_j}\right)=\prod_{j=1}^{V}E(1-\varrho_j+\varrho_jz)=\{1-E(\varrho_1)+E(\varrho_1)z\}^{V}, E ( z ∑ j = 1 V G j ) = j = 1 ∏ V E ( 1 − ϱ j + ϱ j z ) = { 1 − E ( ϱ 1 ) + E ( ϱ 1 ) z } V ,
which is the probability generating function of a Binomial { V , E ( ϱ 1 ) } \operatorname{Binomial}\{V,E(\varrho_1)\} Binomial { V , E ( ϱ 1 )} random variable.
Lemma S2. If V ∼ Poisson ( μ ) V\sim\operatorname{Poisson}(\mu) V ∼ Poisson ( μ ) and X ∣ V ∼ Binomial ( V , p ) X\mid V\sim\operatorname{Binomial}(V,p) X ∣ V ∼ Binomial ( V , p ) , then X ∼ Poisson ( μ p ) X\sim\operatorname{Poisson}(\mu p) X ∼ Poisson ( μ p ) .
Proof. For any integer x ≥ 0 x\ge0 x ≥ 0 ,
pr ( X = x ) = ∑ v ≥ x ( v x ) p x ( 1 − p ) v − x μ v e − μ v ! = ( μ p ) x x ! e − μ ∑ v ≥ x { μ ( 1 − p ) } v − x ( v − x ) ! = ( μ p ) x x ! e − μ e μ ( 1 − p ) = ( μ p ) x x ! e − μ p , \begin{aligned}
\operatorname{pr}(X=x)=\sum_{v\ge x}\binom{v}{x}p^x(1-p)^{v-x}\frac{\mu^v e^{-\mu}}{v!}
=\frac{(\mu p)^x}{x!}e^{-\mu}\sum_{v\ge x}\frac{\{\mu(1-p)\}^{v-x}}{(v-x)!} \\
&=\frac{(\mu p)^x}{x!}e^{-\mu}e^{\mu(1-p)}
=\frac{(\mu p)^x}{x!}e^{-\mu p},
\end{aligned} pr ( X = x ) = v ≥ x ∑ ( x v ) p x ( 1 − p ) v − x v ! μ v e − μ = x ! ( μ p ) x e − μ v ≥ x ∑ ( v − x )! { μ ( 1 − p ) } v − x = x ! ( μ p ) x e − μ e μ ( 1 − p ) = x ! ( μ p ) x e − μ p ,
which is the probability mass function of a Poisson distribution with mean μ p \mu p μ p .
####### Proof of Theorem 1
The proof is a special case of Theorem 1 in [19 ] with n = 2 n=2 n = 2 , a 1 = − α q a_1=-\alpha_q a 1 = − α q , b 1 = σ q ( α q + θ q ) b_1=\sigma_q(\alpha_q+\theta_q) b 1 = σ q ( α q + θ q ) , a 2 = − α q + σ q ( α q + θ q ) a_2=-\alpha_q+\sigma_q(\alpha_q+\theta_q) a 2 = − α q + σ q ( α q + θ q ) and b 2 = ( 1 − σ q ) ( α q + θ q ) b_2=(1-\sigma_q)(\alpha_q+\theta_q) b 2 = ( 1 − σ q ) ( α q + θ q ) .
####### Proof of Proposition 1
Conditionally on N N N , the number of distinct species observed in the n q n_q n q samples collected at site q q q is
K q , n q = ∑ j = 1 N 1 ( ∑ i = 1 n q Y ~ i j q > 0 ) = ∑ j = 1 N G j q . K_{q,n_q} = \sum_{j=1}^{N} \mathbb{1}\left(\sum_{i=1}^{n_q}\widetilde{Y}_{ijq} > 0\right) = \sum_{j=1}^{N} G_{jq}. K q , n q = j = 1 ∑ N 1 ( i = 1 ∑ n q Y ij q > 0 ) = j = 1 ∑ N G j q .
where the G j q G_{jq} G j q are independent Bernoulli given the random probabilities
ϱ j q : = pr ( G j q = 1 ∣ π j q ) = 1 − pr ( ∑ i = 1 n q Y ~ i j q = 0 ∣ π j q ) = 1 − ( 1 − π j q ) n q . \varrho_{jq} := \operatorname{pr}(G_{jq}=1 \mid\pi_{jq}) = 1-\operatorname{pr}\left(\sum_{i=1}^{n_q}\widetilde{Y}_{ijq}=0 \mid\pi_{jq}\right) = 1-(1-\pi_{jq})^{n_q}. ϱ j q := pr ( G j q = 1 ∣ π j q ) = 1 − pr ( i = 1 ∑ n q Y ij q = 0 ∣ π j q ) = 1 − ( 1 − π j q ) n q .
The π j q \pi_{jq} π j q ’s are independent and identically distributed across j j j , and hence so are the ϱ j q \varrho_{jq} ϱ j q ’s, being the same measurable transformation of them. Lemma S1 then gives K q , n q ∣ N ∼ Binomial { N , E ( ϱ 1 q ) } K_{q,n_q}\mid N \sim\operatorname{Binomial}\{N,E(\varrho_{1q})\} K q , n q ∣ N ∼ Binomial { N , E ( ϱ 1 q )} with
E ( ϱ 1 q ) = 1 − E { ( 1 − π 1 q ) n q } = 1 − p 0 , q ( α q , θ q , n q ) = p 1 , q ( α q , θ q , n q ) . E(\varrho_{1q}) = 1-E\{(1-\pi_{1q})^{n_q}\} = 1-p_{0,q}(\alpha_q,\theta_q,n_q) = p_{1,q}(\alpha_q,\theta_q,n_q). E ( ϱ 1 q ) = 1 − E {( 1 − π 1 q ) n q } = 1 − p 0 , q ( α q , θ q , n q ) = p 1 , q ( α q , θ q , n q ) .
Since N ∼ Poisson ( λ ) N\sim\operatorname{Poisson}(\lambda) N ∼ Poisson ( λ ) , Lemma S2 yields K q , n q ∼ Poisson { λ p 1 , q ( α q , θ q , n q ) } K_{q,n_q}\sim\operatorname{Poisson}\{\lambda p_{1,q}(\alpha_q,\theta_q,n_q)\} K q , n q ∼ Poisson { λ p 1 , q ( α q , θ q , n q )} . Conditionally on N N N , the total number of distinct species observed in n n n samples is
K n = ∑ j = 1 N 1 ( ∑ q = 1 Q ∑ i = 1 n q Y ~ i j q > 0 ) = ∑ j = 1 N G j . K_n = \sum_{j=1}^{N} \mathbb{1}\left(\sum_{q=1}^{Q}\sum_{i=1}^{n_q}\widetilde{Y}_{ijq} > 0\right) = \sum_{j=1}^{N}G_j. K n = j = 1 ∑ N 1 ( q = 1 ∑ Q i = 1 ∑ n q Y ij q > 0 ) = j = 1 ∑ N G j .
where the G j G_j G j are independent Bernoulli given the random probabilities
ϱ j : = pr ( G j = 1 ∣ π j 1 , … , π j Q ) = 1 − pr ( ∑ q = 1 Q ∑ i = 1 n q Y ~ i j q = 0 ∣ π j 1 , … , π j Q ) = 1 − ∏ q = 1 Q ( 1 − π j q ) n q . \varrho_j := \operatorname{pr}(G_j=1\mid\pi_{j1},\ldots,\pi_{jQ}) = 1-\operatorname{pr}\left(\sum_{q=1}^{Q}\sum_{i=1}^{n_q}\widetilde{Y}_{ijq}=0\mid\pi_{j1},\ldots,\pi_{jQ}\right) = 1-\prod_{q=1}^{Q}(1-\pi_{jq})^{n_q}. ϱ j := pr ( G j = 1 ∣ π j 1 , … , π j Q ) = 1 − pr ( q = 1 ∑ Q i = 1 ∑ n q Y ij q = 0 ∣ π j 1 , … , π j Q ) = 1 − q = 1 ∏ Q ( 1 − π j q ) n q .
The vectors ( π j 1 , … , π j Q ) (\pi_{j1},\ldots,\pi_{jQ}) ( π j 1 , … , π j Q ) are independent and identically distributed across j j j , and hence so are the ϱ j \varrho_j ϱ j ’s. Lemma S1 then gives K n ∣ N ∼ Binomial { N , E ( ϱ 1 ) } K_n\mid N\sim\operatorname{Binomial}\{N,E(\varrho_1)\} K n ∣ N ∼ Binomial { N , E ( ϱ 1 )} with
E ( ϱ 1 ) = 1 − E { ∏ q = 1 Q ( 1 − π 1 q ) n q } = 1 − ∏ q = 1 Q E { ( 1 − π 1 q ) n q } = 1 − ∏ q = 1 Q p 0 , q ( α q , θ q , n q ) = p 1 ( α , θ , n ) , E(\varrho_1) = 1-E\left\{\prod_{q=1}^{Q}(1-\pi_{1q})^{n_q}\right\} = 1-\prod_{q=1}^{Q}E\{(1-\pi_{1q})^{n_q}\} = 1-\prod_{q=1}^{Q}p_{0,q}(\alpha_q,\theta_q,n_q) = p_1(\boldsymbol{\alpha},\boldsymbol{\theta},\boldsymbol{n}), E ( ϱ 1 ) = 1 − E { q = 1 ∏ Q ( 1 − π 1 q ) n q } = 1 − q = 1 ∏ Q E {( 1 − π 1 q ) n q } = 1 − q = 1 ∏ Q p 0 , q ( α q , θ q , n q ) = p 1 ( α , θ , n ) ,
where the second equality uses the independence of the occurrence probabilities across sites. Lemma S2 then gives K n ∼ Poisson { λ p 1 ( α , θ , n ) } K_n\sim\operatorname{Poisson}\{\lambda p_1(\boldsymbol{\alpha},\boldsymbol{\theta},\boldsymbol{n})\} K n ∼ Poisson { λ p 1 ( α , θ , n )} . For
L q , n q = ∑ j = 1 N 1 ( ∑ i = 1 n q W ~ i j q > 0 ) and L n = ∑ j = 1 N 1 ( ∑ q = 1 Q ∑ i = 1 n q W ~ i j q > 0 ) , L_{q,n_q} = \sum_{j=1}^{N}\mathbb{1}\left(\sum_{i=1}^{n_q}\widetilde{W}_{ijq}>0\right) \quad\text{and}\quad L_n = \sum_{j=1}^{N}\mathbb{1}\left(\sum_{q=1}^{Q}\sum_{i=1}^{n_q}\widetilde{W}_{ijq}>0\right), L q , n q = j = 1 ∑ N 1 ( i = 1 ∑ n q W ij q > 0 ) and L n = j = 1 ∑ N 1 ( q = 1 ∑ Q i = 1 ∑ n q W ij q > 0 ) ,
the proof proceeds in exactly the same way, with π j q \pi_{jq} π j q replaced by the occupancy probabilities η j q ∼ i n d Beta { − α q , σ q ( α q + θ q ) } \eta_{jq}\mathrel{\overset{\mathrm{ind}}{\sim}}\operatorname{Beta}\{-\alpha_q,\sigma_q(\alpha_q+\theta_q)\} η j q ∼ ind Beta { − α q , σ q ( α q + θ q )} , which are independent across j j j and across sites. Since σ q ( α q + θ q ) = α q + θ q ( σ ) \sigma_q(\alpha_q+\theta_q)=\alpha_q+\theta_q(\boldsymbol{\sigma}) σ q ( α q + θ q ) = α q + θ q ( σ ) , we have E { ( 1 − η 1 q ) n q } = p 0 , q ( α q , θ q ( σ ) , n q ) E\{(1-\eta_{1q})^{n_q}\}=p_{0,q}(\alpha_q,\theta_q(\boldsymbol{\sigma}),n_q) E {( 1 − η 1 q ) n q } = p 0 , q ( α q , θ q ( σ ) , n q ) , and it follows that L q , n q ∼ Poisson { λ p 1 , q ( α q , θ q ( σ ) , n q ) } L_{q,n_q}\sim\operatorname{Poisson}\{\lambda p_{1,q}(\alpha_q,\theta_q(\boldsymbol{\sigma}),n_q)\} L q , n q ∼ Poisson { λ p 1 , q ( α q , θ q ( σ ) , n q )} and L n ∼ Poisson { λ p 1 ( α , θ ( σ ) , n ) } L_n\sim\operatorname{Poisson}\{\lambda p_1(\boldsymbol{\alpha},\boldsymbol{\theta}(\boldsymbol{\sigma}),\boldsymbol{n})\} L n ∼ Poisson { λ p 1 ( α , θ ( σ ) , n )} .
####### Proof of Proposition 2
Conditionally on N N N , the number of species shared between two sites q q q and r r r , with q ≠ r q \ne r q = r , is
C n q , n r = ∑ j = 1 N G j , q r , G j , q r = 1 ( ∑ i = 1 n q Y ~ i j q > 0 ) 1 ( ∑ i = 1 n r Y ~ i j r > 0 ) , C_{n_q,n_r} = \sum_{j=1}^{N} G_{j,qr}, \qquad G_{j,qr} = \mathbb{1}\left(\sum_{i=1}^{n_q} \widetilde{Y}_{ijq} > 0\right)\mathbb{1}\left(\sum_{i=1}^{n_r} \widetilde{Y}_{ijr} > 0\right), C n q , n r = j = 1 ∑ N G j , q r , G j , q r = 1 ( i = 1 ∑ n q Y ij q > 0 ) 1 ( i = 1 ∑ n r Y ij r > 0 ) ,
where the G j , q r G_{j,qr} G j , q r are independent Bernoulli given the random probabilities
ϱ j , q r : = pr ( G j , q r = 1 ∣ π j q , π j r ) = { 1 − pr ( ∑ i = 1 n q Y ~ i j q = 0 ∣ π j q ) } { 1 − pr ( ∑ i = 1 n r Y ~ i j r = 0 ∣ π j r ) } = { 1 − ( 1 − π j q ) n q } { 1 − ( 1 − π j r ) n r } , \begin{aligned}
\varrho_{j,qr} := \operatorname{pr}(G_{j,qr} = 1 \mid\pi_{jq},\pi_{jr}) = \left\{1 - \operatorname{pr}\left(\sum_{i=1}^{n_q} \widetilde{Y}_{ijq} = 0 \mid\pi_{jq}\right)\right\}\left\{1 - \operatorname{pr}\left(\sum_{i=1}^{n_r} \widetilde{Y}_{ijr} = 0 \mid\pi_{jr}\right)\right\} \\
&= \left\{1 - (1-\pi_{jq})^{n_q}\right\}\left\{1 - (1-\pi_{jr})^{n_r}\right\},
\end{aligned} ϱ j , q r := pr ( G j , q r = 1 ∣ π j q , π j r ) = { 1 − pr ( i = 1 ∑ n q Y ij q = 0 ∣ π j q ) } { 1 − pr ( i = 1 ∑ n r Y ij r = 0 ∣ π j r ) } = { 1 − ( 1 − π j q ) n q } { 1 − ( 1 − π j r ) n r } ,
the factorization following from the conditional independence of the samples collected at the two sites. The pairs ( π j q , π j r ) (\pi_{jq},\pi_{jr}) ( π j q , π j r ) are independent and identically distributed across j j j , and hence so are the ϱ j , q r \varrho_{j,qr} ϱ j , q r ’s, so Lemma S1 gives C n q , n r ∣ N ∼ Binomial { N , E ( ϱ 1 , q r ) } C_{n_q,n_r} \mid N \sim\operatorname{Binomial}\{N,E(\varrho_{1,qr})\} C n q , n r ∣ N ∼ Binomial { N , E ( ϱ 1 , q r )} with
E ( ϱ 1 , q r ) = E [ { 1 − ( 1 − π 1 q ) n q } { 1 − ( 1 − π 1 r ) n r } ] = [ 1 − E { ( 1 − π 1 q ) n q } ] [ 1 − E { ( 1 − π 1 r ) n r } ] = p 1 , q ( α q , θ q , n q ) p 1 , r ( α r , θ r , n r ) , \begin{aligned}
E(\varrho_{1,qr}) = E\left[\{1-(1-\pi_{1q})^{n_q}\}\{1-(1-\pi_{1r})^{n_r}\}\right] = \left[1-E\{(1-\pi_{1q})^{n_q}\}\right]\left[1-E\{(1-\pi_{1r})^{n_r}\}\right] \\
&= p_{1,q}(\alpha_q,\theta_q,n_q)p_{1,r}(\alpha_r,\theta_r,n_r),
\end{aligned} E ( ϱ 1 , q r ) = E [ { 1 − ( 1 − π 1 q ) n q } { 1 − ( 1 − π 1 r ) n r } ] = [ 1 − E {( 1 − π 1 q ) n q } ] [ 1 − E {( 1 − π 1 r ) n r } ] = p 1 , q ( α q , θ q , n q ) p 1 , r ( α r , θ r , n r ) ,
where the second equality uses the independence of the occurrence probabilities across sites. Since N ∼ Poisson ( λ ) N \sim\operatorname{Poisson}(\lambda) N ∼ Poisson ( λ ) , Lemma S2 then yields C n q , n r ∼ Poisson { λ p 1 , q ( α q , θ q , n q ) p 1 , r ( α r , θ r , n r ) } C_{n_q,n_r} \sim\operatorname{Poisson}\{\lambda p_{1,q}(\alpha_q,\theta_q,n_q)p_{1,r}(\alpha_r,\theta_r,n_r)\} C n q , n r ∼ Poisson { λ p 1 , q ( α q , θ q , n q ) p 1 , r ( α r , θ r , n r )} . For the latent number of shared species, U n q , n r = ∑ j = 1 N 1 ( ∑ i = 1 n q W ~ i j q > 0 ) 1 ( ∑ i = 1 n r W ~ i j r > 0 ) U_{n_q,n_r}=\sum_{j=1}^{N}\mathbb{1}\left(\sum_{i=1}^{n_q}\widetilde{W}_{ijq}>0\right)\mathbb{1}\left(\sum_{i=1}^{n_r}\widetilde{W}_{ijr}>0\right) U n q , n r = ∑ j = 1 N 1 ( ∑ i = 1 n q W ij q > 0 ) 1 ( ∑ i = 1 n r W ij r > 0 ) , the proof proceeds in the same way with π j q , π j r \pi_{jq},\pi_{jr} π j q , π j r replaced by the occupancy probabilities η j q ∼ i n d Beta { − α q , σ q ( α q + θ q ) } \eta_{jq}\mathrel{\overset{\mathrm{ind}}{\sim}}\operatorname{Beta}\{-\alpha_q,\sigma_q(\alpha_q+\theta_q)\} η j q ∼ ind Beta { − α q , σ q ( α q + θ q )} and η j r ∼ i n d Beta { − α r , σ r ( α r + θ r ) } \eta_{jr}\mathrel{\overset{\mathrm{ind}}{\sim}}\operatorname{Beta}\{-\alpha_r,\sigma_r(\alpha_r+\theta_r)\} η j r ∼ ind Beta { − α r , σ r ( α r + θ r )} , which are independent across j j j and across sites. Since σ q ( α q + θ q ) = α q + θ q ( σ q ) \sigma_q(\alpha_q+\theta_q)=\alpha_q+\theta_q(\sigma_q) σ q ( α q + θ q ) = α q + θ q ( σ q ) , we have E { ( 1 − η 1 q ) n q } = p 0 , q ( α q , θ q ( σ q ) , n q ) E\{(1-\eta_{1q})^{n_q}\}=p_{0,q}(\alpha_q,\theta_q(\sigma_q),n_q) E {( 1 − η 1 q ) n q } = p 0 , q ( α q , θ q ( σ q ) , n q ) , giving
U n q , n r ∼ Poisson { λ p 1 , q ( α q , θ q ( σ q ) , n q ) p 1 , r ( α r , θ r ( σ r ) , n r ) } . U_{n_q,n_r} \sim\operatorname{Poisson}\{\lambda p_{1,q}(\alpha_q,\theta_q(\sigma_q),n_q)p_{1,r}(\alpha_r,\theta_r(\sigma_r),n_r)\}. U n q , n r ∼ Poisson { λ p 1 , q ( α q , θ q ( σ q ) , n q ) p 1 , r ( α r , θ r ( σ r ) , n r )} .
####### Proof of Theorem 3
Let A = { j ∈ { 1 , … , N } : ∑ q = 1 Q ∑ i = 1 n q Y ~ i j q = 0 } \mathcal{A}=\{j\in\{1,\ldots,N\}:\sum_{q=1}^{Q}\sum_{i=1}^{n_q}\widetilde{Y}_{ijq}=0\} A = { j ∈ { 1 , … , N } : ∑ q = 1 Q ∑ i = 1 n q Y ij q = 0 } be the set of species unrecorded at all sites, so that ∣ A ∣ = N − k |\mathcal{A}|=N-k ∣ A ∣ = N − k given Y Y Y . In the MOSAIC model the number of new species that would be discovered in s s s additional samples is
K s ( n ) = ∑ j ∈ A 1 ( ∑ q = 1 Q ∑ i = n q + 1 n q + s q Y ~ i j q > 0 ) = ∑ j ∈ A G j , K_s^{(n)}=\sum_{j\in\mathcal{A}}\mathbb{1}\left(\sum_{q=1}^{Q}\sum_{i=n_q+1}^{n_q+s_q}\widetilde{Y}_{ijq}>0\right)=\sum_{j\in\mathcal{A}}G_j, K s ( n ) = j ∈ A ∑ 1 q = 1 ∑ Q i = n q + 1 ∑ n q + s q Y ij q > 0 = j ∈ A ∑ G j ,
where the G j G_j G j ’s are independent Bernoulli given the random probabilities
ϱ j : = pr ( ∑ q = 1 Q ∑ i = n q + 1 n q + s q Y ~ i j q > 0 ∣ π j 1 , … , π j Q ) = 1 − ∏ q = 1 Q ( 1 − π j q ) s q . \varrho_j:=\operatorname{pr}\left(\sum_{q=1}^{Q}\sum_{i=n_q+1}^{n_q+s_q}\widetilde{Y}_{ijq}>0\mid\pi_{j1},\ldots,\pi_{jQ}\right)=1-\prod_{q=1}^{Q}(1-\pi_{jq})^{s_q}. ϱ j := pr q = 1 ∑ Q i = n q + 1 ∑ n q + s q Y ij q > 0 ∣ π j 1 , … , π j Q = 1 − q = 1 ∏ Q ( 1 − π j q ) s q .
By Theorem 2 in the main text, the occurrence probabilities of the previously unobserved species satisfy π j q ′ ∣ Y ∼ i n d Beta ( − α q , α q + θ q + n q ) \pi'_{jq}\mid Y\mathrel{\overset{\mathrm{ind}}{\sim}}\operatorname{Beta}(-\alpha_q,\alpha_q+\theta_q+n_q) π j q ′ ∣ Y ∼ ind Beta ( − α q , α q + θ q + n q ) , independently across species and sites, so that the ϱ j \varrho_j ϱ j ’s are independent and identically distributed across j ∈ A j \in\mathcal{A} j ∈ A given Y \mathbf{Y} Y , with
E ( ϱ j ∣ Y ) = 1 − ∏ q = 1 Q E { ( 1 − π j q ) s q ∣ Y } = 1 − ∏ q = 1 Q p 0 , q ( α q , θ q + n q , s q ) = p 1 ( α , θ + n , s ) . E(\varrho_j \mid\mathbf{Y}) = 1 - \prod_{q=1}^{Q} E\{(1-\pi_{jq})^{s_q} \mid\mathbf{Y}\} = 1 - \prod_{q=1}^{Q} p_{0,q}(\alpha_q,\theta_q+n_q,s_q) = p_1(\alpha,\theta+\mathbf{n},\mathbf{s}). E ( ϱ j ∣ Y ) = 1 − q = 1 ∏ Q E {( 1 − π j q ) s q ∣ Y } = 1 − q = 1 ∏ Q p 0 , q ( α q , θ q + n q , s q ) = p 1 ( α , θ + n , s ) .
Lemma S1 then gives K s ( n ) ∣ Y , N ∼ Binomial { N − k , p 1 ( α , θ + n , s ) } K_s^{(n)} \mid\mathbf{Y},N \sim\operatorname{Binomial}\{N-k,p_1(\alpha,\theta+\mathbf{n},\mathbf{s})\} K s ( n ) ∣ Y , N ∼ Binomial { N − k , p 1 ( α , θ + n , s )} . By the same theorem, N − k ∣ Y ∼ Poisson { λ p 0 ( α , θ , n ) } N-k \mid\mathbf{Y} \sim\operatorname{Poisson}\{\lambda p_0(\alpha,\theta,\mathbf{n})\} N − k ∣ Y ∼ Poisson { λ p 0 ( α , θ , n )} , so that marginalizing through Lemma S2 gives K s ( n ) ∣ Y ∼ Poisson { λ p 0 ( α , θ , n ) p 1 ( α , θ + n , s ) } K_s^{(n)} \mid\mathbf{Y} \sim\operatorname{Poisson}\{\lambda p_0(\alpha,\theta,\mathbf{n})p_1(\alpha,\theta+\mathbf{n},\mathbf{s})\} K s ( n ) ∣ Y ∼ Poisson { λ p 0 ( α , θ , n ) p 1 ( α , θ + n , s )} .
Let now A q = { j ∈ { 1 , … , N } : ∑ i = 1 n q Y ~ i j q = 0 } A_q=\{j\in\{1,\ldots,N\}:\sum_{i=1}^{n_q}\widetilde{Y}_{ijq}=0\} A q = { j ∈ { 1 , … , N } : ∑ i = 1 n q Y ij q = 0 } be the set of species not yet recorded at site q q q , so that ∣ A q ∣ = N − k q |A_q|=N-k_q ∣ A q ∣ = N − k q ; note that A q A_q A q contains species that have been recorded at some other site. The number of species new to site q q q that would be discovered in s q s_q s q additional samples is
K q , s q ( n q ) = ∑ j ∈ A q 1 ( ∑ i = n q + 1 n q + s q Y ~ i j q > 0 ) = ∑ j ∈ A q G j q . K_{q,s_q}^{(n_q)}=\sum_{j\in A_q}1\left(\sum_{i=n_q+1}^{n_q+s_q}\widetilde{Y}_{ijq}>0\right)=\sum_{j\in A_q}G_{jq}. K q , s q ( n q ) = j ∈ A q ∑ 1 i = n q + 1 ∑ n q + s q Y ij q > 0 = j ∈ A q ∑ G j q .
where the G j q G_{jq} G j q ’s are independent Bernoulli given the random probabilities ϱ j q : = 1 − ( 1 − π j q ) s q \varrho_{jq}:=1-(1-\pi_{jq})^{s_q} ϱ j q := 1 − ( 1 − π j q ) s q , which are independent and identically distributed across j ∈ A q j\in A_q j ∈ A q given Y \mathbf{Y} Y , again by Theorem 2, and satisfy
E ( ϱ j q ∣ Y ) = 1 − E { ( 1 − π j q ) s q ∣ Y } = 1 − p 0 , q ( α q , θ q + n q , s q ) = p 1 , q ( α q , θ q + n q , s q ) . E(\varrho_{jq}\mid\mathbf{Y})=1-E\{(1-\pi_{jq})^{s_q}\mid\mathbf{Y}\}=1-p_{0,q}(\alpha_q,\theta_q+n_q,s_q)=p_{1,q}(\alpha_q,\theta_q+n_q,s_q). E ( ϱ j q ∣ Y ) = 1 − E {( 1 − π j q ) s q ∣ Y } = 1 − p 0 , q ( α q , θ q + n q , s q ) = p 1 , q ( α q , θ q + n q , s q ) .
We decompose A q = A ∪ ( A q ∖ A ) A_q=A\cup(A_q\setminus A) A q = A ∪ ( A q ∖ A ) , where A q ∖ A A_q\setminus A A q ∖ A is the set of species recorded at some site other than q q q , whose cardinality k − k q k-k_q k − k q is known given Y \mathbf{Y} Y . Applying Lemma S1 to each of the two groups separately gives
K q , s q ( n q ) ∣ Y , N = d Binomial { N − k , p 1 , q ( α q , θ q + n q , s q ) } + Binomial { k − k q , p 1 , q ( α q , θ q + n q , s q ) } , K_{q,s_q}^{(n_q)}\mid\mathbf{Y},N \mathrel{\overset{d}{=}} \operatorname{Binomial}\{N-k,p_{1,q}(\alpha_q,\theta_q+n_q,s_q)\}
+\operatorname{Binomial}\{k-k_q,p_{1,q}(\alpha_q,\theta_q+n_q,s_q)\}, K q , s q ( n q ) ∣ Y , N = d Binomial { N − k , p 1 , q ( α q , θ q + n q , s q )} + Binomial { k − k q , p 1 , q ( α q , θ q + n q , s q )} ,
the two terms being independent, since they involve disjoint sets of species with independent occurrence probabilities. Marginalizing the first term over N − k ∣ Y ∼ Poisson { λ p 0 ( α , θ , n ) } N-k\mid\mathbf{Y}\sim\operatorname{Poisson}\{\lambda p_0(\alpha,\theta,\mathbf{n})\} N − k ∣ Y ∼ Poisson { λ p 0 ( α , θ , n )} through Lemma S2, we obtain
K q , s q ( n q ) ∣ Y = d Poisson { λ p 0 ( α , θ , n ) p 1 , q ( α q , θ q + n q , s q ) } + Binomial { k − k q , p 1 , q ( α q , θ q + n q , s q ) } . K_{q,s_q}^{(n_q)}\mid\mathbf{Y}\mathrel{\overset{d}{=}}\operatorname{Poisson}\{\lambda p_0(\alpha,\theta,\mathbf{n})p_{1,q}(\alpha_q,\theta_q+n_q,s_q)\}
+\operatorname{Binomial}\{k-k_q,p_{1,q}(\alpha_q,\theta_q+n_q,s_q)\}. K q , s q ( n q ) ∣ Y = d Poisson { λ p 0 ( α , θ , n ) p 1 , q ( α q , θ q + n q , s q )} + Binomial { k − k q , p 1 , q ( α q , θ q + n q , s q )} .
The latent quantities are obtained along the same lines, replacing the observed data Y \mathbf{Y} Y with the occupancies W \mathbf{W} W throughout. Let B = { j ∈ { 1 , … , N } : ∑ q = 1 Q ∑ i = 1 n q W ~ i j q = 0 } B=\{j\in\{1,\ldots,N\}:\sum_{q=1}^{Q}\sum_{i=1}^{n_q}\widetilde{W}_{ijq}=0\} B = { j ∈ { 1 , … , N } : ∑ q = 1 Q ∑ i = 1 n q W ij q = 0 } and B q = { j ∈ { 1 , … , N } : ∑ i = 1 n q W ~ i j q = 0 } B_q=\{j\in\{1,\ldots,N\}:\sum_{i=1}^{n_q}\widetilde{W}_{ijq}=0\} B q = { j ∈ { 1 , … , N } : ∑ i = 1 n q W ij q = 0 } be the sets of species absent from all samples and from those collected at site q q q , respectively, so that ∣ B ∣ = N − ℓ |B|=N-\ell ∣ B ∣ = N − ℓ and ∣ B q ∣ = N − ℓ q |B_q|=N-\ell_q ∣ B q ∣ = N − ℓ q , and set
L s ( n ) = ∑ j ∈ B 1 ( ∑ q = 1 Q ∑ i = n q + 1 n q + s q W ~ i j q > 0 ) , L q , s q ( n q ) = ∑ j ∈ B q 1 ( ∑ i = n q + 1 n q + s q W ~ i j q > 0 ) . L_s^{(n)}=\sum_{j\in B}1\left(\sum_{q=1}^{Q}\sum_{i=n_q+1}^{n_q+s_q}\widetilde{W}_{ijq}>0\right),\qquad
L_{q,s_q}^{(n_q)}=\sum_{j\in B_q}1\left(\sum_{i=n_q+1}^{n_q+s_q}\widetilde{W}_{ijq}>0\right). L s ( n ) = j ∈ B ∑ 1 q = 1 ∑ Q i = n q + 1 ∑ n q + s q W ij q > 0 , L q , s q ( n q ) = j ∈ B q ∑ 1 i = n q + 1 ∑ n q + s q W ij q > 0 .
Conditioning on W W W rather than on Y Y Y is essential here: a null entry of Y Y Y is ambiguous between absence and non-detection, whereas a null entry of W W W identifies absence, and only in the latter case does the occupancy probability of an unrecorded species admit the conjugate posterior η j q ∣ W ∼ i n d Beta { − α q , σ q ( α q + θ q ) + n q } \eta_{jq} \mid W \mathrel{\overset{\mathrm{ind}}{\sim}} \operatorname{Beta}\{-\alpha_q,\sigma_q(\alpha_q+\theta_q)+n_q\} η j q ∣ W ∼ ind Beta { − α q , σ q ( α q + θ q ) + n q } . Since σ q ( α q + θ q ) = α q + θ q ( σ q ) \sigma_q(\alpha_q+\theta_q)=\alpha_q+\theta_q(\sigma_q) σ q ( α q + θ q ) = α q + θ q ( σ q ) , the argument used for the occurrence probabilities applies verbatim with θ q \theta_q θ q replaced by θ q ( σ q ) \theta_q(\sigma_q) θ q ( σ q ) , and the number of species absent everywhere satisfies N − ℓ ∣ W ∼ Poisson { λ p 0 ( α , θ ( σ ) , n ) } N-\ell\mid W \sim\operatorname{Poisson}\{\lambda p_0(\boldsymbol{\alpha},\boldsymbol{\theta}(\boldsymbol{\sigma}),\boldsymbol{n})\} N − ℓ ∣ W ∼ Poisson { λ p 0 ( α , θ ( σ ) , n )} . Hence
L s ( n ) ∣ W ∼ Poisson { λ p 0 ( α , θ ( σ ) , n ) p 1 ( α , θ ( σ ) + n , s ) } , L_{s}^{(n)} \mid W \sim\operatorname{Poisson}\{\lambda p_0(\boldsymbol{\alpha},\boldsymbol{\theta}(\boldsymbol{\sigma}),\boldsymbol{n})p_1(\boldsymbol{\alpha},\boldsymbol{\theta}(\boldsymbol{\sigma})+\boldsymbol{n},\boldsymbol{s})\}, L s ( n ) ∣ W ∼ Poisson { λ p 0 ( α , θ ( σ ) , n ) p 1 ( α , θ ( σ ) + n , s )} ,
and, decomposing B q = B ∪ ( B q ∖ B ) \mathcal{B}_q=\mathcal{B}\cup(\mathcal{B}_q\setminus\mathcal{B}) B q = B ∪ ( B q ∖ B ) with ∣ B q ∖ B ∣ = ℓ − ℓ q |\mathcal{B}_q\setminus\mathcal{B}|=\ell-\ell_q ∣ B q ∖ B ∣ = ℓ − ℓ q known given W W W ,
L q , s q ( n q ) ∣ W = d Poisson { λ p 0 ( α , θ ( σ ) , n ) p 1 , q ( α q , θ q ( σ q ) + n q , s q ) } + Binomial { ℓ − ℓ q , p 1 , q ( α q , θ q ( σ q ) + n q , s q ) } , L_{q,s_q}^{(n_q)} \mid W \mathrel{\overset{d}{=}} \operatorname{Poisson}\{\lambda p_0(\boldsymbol{\alpha},\boldsymbol{\theta}(\boldsymbol{\sigma}),\boldsymbol{n})p_{1,q}(\alpha_q,\theta_q(\sigma_q)+n_q,s_q)\}
+\operatorname{Binomial}\{\ell-\ell_q,p_{1,q}(\alpha_q,\theta_q(\sigma_q)+n_q,s_q)\}, L q , s q ( n q ) ∣ W = d Poisson { λ p 0 ( α , θ ( σ ) , n ) p 1 , q ( α q , θ q ( σ q ) + n q , s q )} + Binomial { ℓ − ℓ q , p 1 , q ( α q , θ q ( σ q ) + n q , s q )} ,
with the two terms independent, as above.
####### Proof of Theorem 4
Let q ≠ r q\ne r q = r and define the three disjoint sets of species
A 00 = { j ∈ { 1 , … , N } : ∑ i = 1 n q Y ~ i j q = 0 , ∑ i = 1 n r Y ~ i j r = 0 } , \mathcal{A}_{00}=\left\{j\in\{1,\ldots,N\}:\sum_{i=1}^{n_q}\widetilde{Y}_{ijq}=0,\sum_{i=1}^{n_r}\widetilde{Y}_{ijr}=0\right\}, A 00 = { j ∈ { 1 , … , N } : i = 1 ∑ n q Y ij q = 0 , i = 1 ∑ n r Y ij r = 0 } ,
A q = { j ∈ { 1 , … , N } : ∑ i = 1 n q Y ~ i j q > 0 , ∑ i = 1 n r Y ~ i j r = 0 } , \mathcal{A}_q=\left\{j\in\{1,\ldots,N\}:\sum_{i=1}^{n_q}\widetilde{Y}_{ijq}>0,\sum_{i=1}^{n_r}\widetilde{Y}_{ijr}=0\right\}, A q = { j ∈ { 1 , … , N } : i = 1 ∑ n q Y ij q > 0 , i = 1 ∑ n r Y ij r = 0 } ,
A r = { j ∈ { 1 , … , N } : ∑ i = 1 n q Y ~ i j q = 0 , ∑ i = 1 n r Y ~ i j r > 0 } , \mathcal{A}_r=\left\{j\in\{1,\ldots,N\}:\sum_{i=1}^{n_q}\widetilde{Y}_{ijq}=0,\sum_{i=1}^{n_r}\widetilde{Y}_{ijr}>0\right\}, A r = { j ∈ { 1 , … , N } : i = 1 ∑ n q Y ij q = 0 , i = 1 ∑ n r Y ij r > 0 } ,
namely the species recorded at neither site, those recorded only at site q q q and those recorded only at site r r r . To indicate the detection of species j j j at site h ∈ { q , r } h\in\{q,r\} h ∈ { q , r } in the additional samples we use I j h = 1 ( ∑ i = n h + 1 n h + s h Y ~ i j h > 0 ) I_{jh}=\mathbf{1}\left(\sum_{i=n_h+1}^{n_h+s_h}\widetilde{Y}_{ijh}>0\right) I j h = 1 ( ∑ i = n h + 1 n h + s h Y ij h > 0 ) . In the MOSAIC model the number of new species shared between sites q q q and r r r that would be discovered in ( s q , s r ) (s_q,s_r) ( s q , s r ) additional samples is
C s q , s r ( n q , n r ) = ∑ j ∈ A 00 I j q I j r + ∑ j ∈ A q I j r + ∑ j ∈ A r I j q . (8) C_{s_q,s_r}^{(n_q,n_r)}=\sum_{j\in\mathcal{A}_{00}}I_{jq}I_{jr}+\sum_{j\in\mathcal{A}_q}I_{jr}+\sum_{j\in\mathcal{A}_r}I_{jq}. \tag*{(8)} C s q , s r ( n q , n r ) = j ∈ A 00 ∑ I j q I j r + j ∈ A q ∑ I j r + j ∈ A r ∑ I j q . (8)
For h ∈ { q , r } h\in\{q,r\} h ∈ { q , r } , the indicators I j h I_{jh} I j h are independent Bernoulli given the random probabilities ϱ j h : = 1 − ( 1 − π j h ) s h \varrho_{jh}:=1-(1-\pi_{jh})^{s_h} ϱ j h := 1 − ( 1 − π j h ) s h . By Theorem 2 in the main text, the occurrence probabilities of the previously unobserved species satisfy π j h ′ ∣ Y ∼ i n d Beta ( − α h , α h + θ h + n h ) \pi'_{jh}\mid Y\mathrel{\overset{\mathrm{ind}}{\sim}}\operatorname{Beta}(-\alpha_h,\alpha_h+\theta_h+n_h) π j h ′ ∣ Y ∼ ind Beta ( − α h , α h + θ h + n h ) , independently across species and sites, so that
E ( ϱ j h ∣ Y ) = 1 − E { ( 1 − π j h ′ ) s h ∣ Y } = 1 − p 0 , h ( α h , θ h + n h , s h ) = p 1 , h ( α h , θ h + n h , s h ) , E(\varrho_{jh}\mid Y)=1-E\{(1-\pi'_{jh})^{s_h}\mid Y\}=1-p_{0,h}(\alpha_h,\theta_h+n_h,s_h)=p_{1,h}(\alpha_h,\theta_h+n_h,s_h), E ( ϱ j h ∣ Y ) = 1 − E {( 1 − π j h ′ ) s h ∣ Y } = 1 − p 0 , h ( α h , θ h + n h , s h ) = p 1 , h ( α h , θ h + n h , s h ) ,
whereas the species in A 00 \mathcal{A}_{00} A 00 contribute with random success probability ϱ j q ϱ j r \varrho_{jq}\varrho_{jr} ϱ j q ϱ j r , whose expectation factorizes by the independence of π j q \pi_{jq} π j q and π j r \pi_{jr} π j r ,
E ( ϱ j q ϱ j r ∣ Y ) = p 1 , q ( α q , θ q + n q , s q ) p 1 , r ( α r , θ r + n r , s r ) . E(\varrho_{jq}\varrho_{jr}\mid Y)=p_{1,q}(\alpha_q,\theta_q+n_q,s_q)p_{1,r}(\alpha_r,\theta_r+n_r,s_r). E ( ϱ j q ϱ j r ∣ Y ) = p 1 , q ( α q , θ q + n q , s q ) p 1 , r ( α r , θ r + n r , s r ) .
Within each of the sets A 00 A_{00} A 00 , A q A_q A q and A r A_r A r these success probabilities are independent and identically distributed across j j j given Y \mathbf{Y} Y .
It remains to identify the cardinalities of the three sets. Recalling that k q k_q k q and k r k_r k r denote the number of species recorded at sites q q q and r r r , that c q , r c_{q,r} c q , r is the number of species recorded at both and that k q , r = k q + k r − c q , r k_{q,r}=k_q+k_r-c_{q,r} k q , r = k q + k r − c q , r , we have ∣ A q ∣ = k q − c q , r |A_q|=k_q-c_{q,r} ∣ A q ∣ = k q − c q , r and ∣ A r ∣ = k r − c q , r |A_r|=k_r-c_{q,r} ∣ A r ∣ = k r − c q , r , both known given Y \mathbf{Y} Y . As for A 00 A_{00} A 00 , we split it into the species that are unrecorded everywhere, which are N − k N-k N − k and hence random, and the species recorded at some site other than q q q and r r r , which are k − k q , r k-k_{q,r} k − k q , r and hence known given Y \mathbf{Y} Y . Applying Lemma S1 to each of the four groups separately, we obtain
C s q , s r ( n q , n r ) ∣ Y , N = d Binomial { N − k , p 1 , q ( α q , θ q + n q , s q ) p 1 , r ( α r , θ r + n r , s r ) } + Binomial { k − k q , r , p 1 , q ( α q , θ q + n q , s q ) p 1 , r ( α r , θ r + n r , s r ) } + Binomial { k q − c q , r , p 1 , r ( α r , θ r + n r , s r ) } + Binomial { k r − c q , r , p 1 , q ( α q , θ q + n q , s q ) } . C_{s_q,s_r}^{(n_q,n_r)} \mid\mathbf{Y},N \mathrel{\mathop{=}^{d}} \operatorname{Binomial}\left\{N-k,p_{1,q}(\alpha_q,\theta_q+n_q,s_q)p_{1,r}(\alpha_r,\theta_r+n_r,s_r)\right\}
\\
+ \operatorname{Binomial}\left\{k-k_{q,r},p_{1,q}(\alpha_q,\theta_q+n_q,s_q)p_{1,r}(\alpha_r,\theta_r+n_r,s_r)\right\}
\\
+ \operatorname{Binomial}\left\{k_q-c_{q,r},p_{1,r}(\alpha_r,\theta_r+n_r,s_r)\right\}
\\
+ \operatorname{Binomial}\left\{k_r-c_{q,r},p_{1,q}(\alpha_q,\theta_q+n_q,s_q)\right\}. C s q , s r ( n q , n r ) ∣ Y , N = d Binomial { N − k , p 1 , q ( α q , θ q + n q , s q ) p 1 , r ( α r , θ r + n r , s r ) } + Binomial { k − k q , r , p 1 , q ( α q , θ q + n q , s q ) p 1 , r ( α r , θ r + n r , s r ) } + Binomial { k q − c q , r , p 1 , r ( α r , θ r + n r , s r ) } + Binomial { k r − c q , r , p 1 , q ( α q , θ q + n q , s q ) } .
where the four terms are independent because they involve disjoint sets of species with independent occurrence probabilities. Finally, by Theorem 2 in the main text, N − k ∣ Y ∼ Poisson { λ p 0 ( α , θ , n ) } N-k\mid\mathbf{Y}\sim\operatorname{Poisson}\{\lambda p_0(\boldsymbol{\alpha},\boldsymbol{\theta},\mathbf{n})\} N − k ∣ Y ∼ Poisson { λ p 0 ( α , θ , n )} , so that marginalizing the first term through Lemma S2 gives
C s q , s r ( n q , n r ) ∣ Y = d Poisson { λ p 0 ( α , θ , n ) p 1 , q ( α q , θ q + n q , s q ) p 1 , r ( α r , θ r + n r , s r ) } + Binomial { k − k q , r , p 1 , q ( α q , θ q + n q , s q ) p 1 , r ( α r , θ r + n r , s r ) } + Binomial { k q − c q , r , p 1 , r ( α r , θ r + n r , s r ) } + Binomial { k r − c q , r , p 1 , q ( α q , θ q + n q , s q ) } . C_{s_q,s_r}^{(n_q,n_r)} \mid\mathbf{Y} \mathrel{\mathop{=}^{d}} \operatorname{Poisson}\left\{\lambda p_0(\boldsymbol{\alpha},\boldsymbol{\theta},\mathbf{n})p_{1,q}(\alpha_q,\theta_q+n_q,s_q)p_{1,r}(\alpha_r,\theta_r+n_r,s_r)\right\}
\\
+ \operatorname{Binomial}\left\{k-k_{q,r},p_{1,q}(\alpha_q,\theta_q+n_q,s_q)p_{1,r}(\alpha_r,\theta_r+n_r,s_r)\right\}
\\
+ \operatorname{Binomial}\left\{k_q-c_{q,r},p_{1,r}(\alpha_r,\theta_r+n_r,s_r)\right\}
\\
+ \operatorname{Binomial}\left\{k_r-c_{q,r},p_{1,q}(\alpha_q,\theta_q+n_q,s_q)\right\}. C s q , s r ( n q , n r ) ∣ Y = d Poisson { λ p 0 ( α , θ , n ) p 1 , q ( α q , θ q + n q , s q ) p 1 , r ( α r , θ r + n r , s r ) } + Binomial { k − k q , r , p 1 , q ( α q , θ q + n q , s q ) p 1 , r ( α r , θ r + n r , s r ) } + Binomial { k q − c q , r , p 1 , r ( α r , θ r + n r , s r ) } + Binomial { k r − c q , r , p 1 , q ( α q , θ q + n q , s q ) } .
For the latent number of new shared species U s q , s r ( n q , n r ) U_{s_q,s_r}^{(n_q,n_r)} U s q , s r ( n q , n r ) the proof proceeds along the same lines, replacing the observed data Y \mathbf{Y} Y with the occupancies W \mathbf{W} W throughout, as in the proof of Theorem 3. The three sets are now defined by absence rather than by non-detection, namely
B 00 = { j ∈ { 1 , … , N } : ∑ i = 1 n q W ~ i j q = 0 , ∑ i = 1 n r W ~ i j r = 0 } , B_{00}=\left\{j\in\{1,\ldots,N\}:\sum_{i=1}^{n_q}\widetilde{W}_{ijq}=0,\ \sum_{i=1}^{n_r}\widetilde{W}_{ijr}=0\right\}, B 00 = { j ∈ { 1 , … , N } : i = 1 ∑ n q W ij q = 0 , i = 1 ∑ n r W ij r = 0 } ,
B q = { j ∈ { 1 , … , N } : ∑ i = 1 n q W ~ i j q > 0 , ∑ i = 1 n r W ~ i j r = 0 } , B_q=\left\{j\in\{1,\ldots,N\}:\sum_{i=1}^{n_q}\widetilde{W}_{ijq}>0,\ \sum_{i=1}^{n_r}\widetilde{W}_{ijr}=0\right\}, B q = { j ∈ { 1 , … , N } : i = 1 ∑ n q W ij q > 0 , i = 1 ∑ n r W ij r = 0 } ,
B r = { j ∈ { 1 , … , N } : ∑ i = 1 n q W ~ i j q = 0 , ∑ i = 1 n r W ~ i j r > 0 } . B_r=\left\{j\in\{1,\ldots,N\}:\sum_{i=1}^{n_q}\widetilde{W}_{ijq}=0,\ \sum_{i=1}^{n_r}\widetilde{W}_{ijr}>0\right\}. B r = { j ∈ { 1 , … , N } : i = 1 ∑ n q W ij q = 0 , i = 1 ∑ n r W ij r > 0 } .
the target is (S1) with the indicators I ~ j h = 1 ( ∑ i = n h + 1 n h + s h W ~ i j h > 0 ) \widetilde{I}_{jh}=\mathbb{1}\left(\sum_{i=n_h+1}^{n_h+s_h}\widetilde{W}_{ijh}>0\right) I j h = 1 ( ∑ i = n h + 1 n h + s h W ij h > 0 ) in place of the I j h I_{jh} I j h ; and the occupancy probabilities of the species absent from site h h h satisfy η j h ∣ W ∼ i n d Beta { − α h , σ h ( α h + θ h ) + n h } \eta_{jh}\mid\mathbf{W}\mathrel{\mathop{\sim}^{\mathrm{ind}}}\operatorname{Beta}\{-\alpha_h,\sigma_h(\alpha_h+\theta_h)+n_h\} η j h ∣ W ∼ ind Beta { − α h , σ h ( α h + θ h ) + n h } . Since σ h ( α h + θ h ) = α h + θ h ( σ h ) \sigma_h(\alpha_h+\theta_h)=\alpha_h+\theta_h(\sigma_h) σ h ( α h + θ h ) = α h + θ h ( σ h ) , every occurrence of θ h \theta_h θ h is replaced by θ h ( σ h ) \theta_h(\sigma_h) θ h ( σ h ) , including in the Poisson mean, which becomes λ p 0 ( α , θ ( σ ) , n ) \lambda p_0(\boldsymbol{\alpha},\boldsymbol{\theta}(\boldsymbol{\sigma}),\mathbf{n}) λ p 0 ( α , θ ( σ ) , n ) because N − ℓ ∣ W ∼ Poisson { λ p 0 ( α , θ ( σ ) , n ) } N-\ell\mid\mathbf{W}\sim\operatorname{Poisson}\{\lambda p_0(\boldsymbol{\alpha},\boldsymbol{\theta}(\boldsymbol{\sigma}),\mathbf{n})\} N − ℓ ∣ W ∼ Poisson { λ p 0 ( α , θ ( σ ) , n )} . The four cardinalities become N − ℓ N-\ell N − ℓ , ℓ − ℓ q , r \ell-\ell_{q,r} ℓ − ℓ q , r , ℓ q − u q , r \ell_q-u_{q,r} ℓ q − u q , r and ℓ r − u q , r \ell_r-u_{q,r} ℓ r − u q , r , with ℓ q , r = ℓ q + ℓ r − u q , r \ell_{q,r}=\ell_q+\ell_r-u_{q,r} ℓ q , r = ℓ q + ℓ r − u q , r , all determined by W \mathbf{W} W .
Derivation of the pEFPF
In this section, we obtain the analytical formulation for the marginal distribution of the proposed approach reported in (5) in the main paper. Let M ~ j q = ∑ i = 1 n q Y ~ i j q \widetilde{M}_{jq} = \sum_{i=1}^{n_q} \widetilde{Y}_{ijq} M j q = ∑ i = 1 n q Y ij q , for j = 1 , … , N j = 1, \ldots, N j = 1 , … , N and q = 1 , … , Q q = 1, \ldots, Q q = 1 , … , Q , be the number of samples from site q q q in which species X ~ j \widetilde{X}_j X j is recorded, and let M ~ = ( M ~ j q ) \widetilde{M} = (\widetilde{M}_{jq}) M = ( M j q ) be the corresponding N × Q N \times Q N × Q matrix, with realization m ~ \widetilde{m} m . These extend the observed frequencies M M M of the main text to the complete species list, and the two coincide once the N − k N-k N − k all-zero rows of m ~ \widetilde{m} m are discarded, up to a permutation of the columns.
Conditionally on the parameters N N N , α \alpha α , θ \theta θ and on π = ( π j q : j = 1 , … , N ; q = 1 , … , Q ) \boldsymbol{\pi} = (\pi_{jq}: j = 1, \ldots, N; q = 1, \ldots, Q) π = ( π j q : j = 1 , … , N ; q = 1 , … , Q ) , the likelihood function for the event M ~ = m ~ \widetilde{M} = \widetilde{m} M = m is
L ( π , N ; m ~ ) = ∏ q = 1 Q ∏ j = 1 N π j q m ~ j q ( 1 − π j q ) n q − m ~ j q , \mathcal{L}(\boldsymbol{\pi},N;\widetilde{m}) = \prod_{q=1}^{Q}\prod_{j=1}^{N}\pi_{jq}^{\widetilde{m}_{jq}}(1-\pi_{jq})^{n_q-\widetilde{m}_{jq}}, L ( π , N ; m ) = q = 1 ∏ Q j = 1 ∏ N π j q m j q ( 1 − π j q ) n q − m j q ,
and integrating over π \boldsymbol{\pi} π we obtain the marginal probability
Pr ( M ~ = m ~ ) = ∏ q = 1 Q ∏ j = 1 N ∫ 0 1 π j q m ~ j q ( 1 − π j q ) n q − m ~ j q Γ ( θ q ) Γ ( θ q + α q ) Γ ( − α q ) π j q − α q − 1 ( 1 − π j q ) α q + θ q − 1 d π j q = ∏ q = 1 Q [ { Γ ( θ q ) Γ ( θ q + α q ) Γ ( − α q ) Γ ( θ q + n q ) } N ∏ j = 1 N Γ ( m ~ j q − α q ) Γ ( n q − m ~ j q + α q + θ q ) ] . \begin{aligned}
\Pr(\widetilde{M}=\widetilde{m}) &= \prod_{q=1}^{Q}\prod_{j=1}^{N}\int_{0}^{1}\pi_{jq}^{\widetilde{m}_{jq}}(1-\pi_{jq})^{n_q-\widetilde{m}_{jq}}\frac{\Gamma(\theta_q)}{\Gamma(\theta_q+\alpha_q)\Gamma(-\alpha_q)}\pi_{jq}^{-\alpha_q-1}(1-\pi_{jq})^{\alpha_q+\theta_q-1}\,\mathrm{d}\pi_{jq}\\
&= \prod_{q=1}^{Q}\left[\left\{\frac{\Gamma(\theta_q)}{\Gamma(\theta_q+\alpha_q)\Gamma(-\alpha_q)\Gamma(\theta_q+n_q)}\right\}^{N}\prod_{j=1}^{N}\Gamma(\widetilde{m}_{jq}-\alpha_q)\Gamma(n_q-\widetilde{m}_{jq}+\alpha_q+\theta_q)\right].
\end{aligned} Pr ( M = m ) = q = 1 ∏ Q j = 1 ∏ N ∫ 0 1 π j q m j q ( 1 − π j q ) n q − m j q Γ ( θ q + α q ) Γ ( − α q ) Γ ( θ q ) π j q − α q − 1 ( 1 − π j q ) α q + θ q − 1 d π j q = q = 1 ∏ Q [ { Γ ( θ q + α q ) Γ ( − α q ) Γ ( θ q + n q ) Γ ( θ q ) } N j = 1 ∏ N Γ ( m j q − α q ) Γ ( n q − m j q + α q + θ q ) ] .
The event ( M = m , K n = k ) (M=m,K_n=k) ( M = m , K n = k ) obtains when exactly k k k rows of m ~ \widetilde{m} m are nonzero and carry the observed frequencies m m m , the remaining N − k N-k N − k rows being identically zero. Since the species labels are exchangeable, the ( N k ) \binom{N}{k} ( k N ) choices of which rows are the nonzero ones all contribute the same probability, and it suffices to evaluate the display above at ( m ~ 1 q , … , m ~ N q ) = ( m 1 q , … , m k q , 0 , … , 0 ) (\widetilde{m}_{1q},\ldots,\widetilde{m}_{Nq})=(m_{1q},\ldots,m_{kq},0,\ldots,0) ( m 1 q , … , m N q ) = ( m 1 q , … , m k q , 0 , … , 0 ) for every q = 1 , … , Q q=1,\ldots,Q q = 1 , … , Q and multiply by ( N k ) \binom{N}{k} ( k N ) . Hence
Pr ( M = m , K n = k ∣ N , α , θ ) = ( N k ) ∏ q = 1 Q [ { Γ ( θ q ) Γ ( θ q + α q ) Γ ( − α q ) Γ ( θ q + n q ) } N × { Γ ( − α q ) Γ ( n q + θ q + α q ) } N − k × ∏ j = 1 k Γ ( m j q − α q ) Γ ( n q − m j q + α q + θ q ) ] . \begin{aligned}
\Pr(M=m,K_n=k\mid N,\alpha,\theta)
&=\binom{N}{k}\prod_{q=1}^{Q}\left[\left\{\frac{\Gamma(\theta_q)}{\Gamma(\theta_q+\alpha_q)\Gamma(-\alpha_q)\Gamma(\theta_q+n_q)}\right\}^{N}\right.\\
&\quad\left.\times\{\Gamma(-\alpha_q)\Gamma(n_q+\theta_q+\alpha_q)\}^{N-k}\right.\\
&\quad\left.\times\prod_{j=1}^{k}\Gamma(m_{jq}-\alpha_q)\Gamma(n_q-m_{jq}+\alpha_q+\theta_q)\right].
\end{aligned} Pr ( M = m , K n = k ∣ N , α , θ ) = ( k N ) q = 1 ∏ Q [ { Γ ( θ q + α q ) Γ ( − α q ) Γ ( θ q + n q ) Γ ( θ q ) } N × { Γ ( − α q ) Γ ( n q + θ q + α q ) } N − k × j = 1 ∏ k Γ ( m j q − α q ) Γ ( n q − m j q + α q + θ q ) ] .
which, expressing the gamma ratios through Pochhammer symbols, becomes
Pr ( M = m , K n = k ∣ N , α , θ ) = ( N k ) ∏ q = 1 Q [ { ( θ q + α q ) n q ( θ q ) n q } N { − α q ( θ q + α q ) n q } k × ∏ j = 1 k ( θ q + α q ) n q − m j q ( 1 − α q ) m j q − 1 ] . \begin{aligned}
\Pr(M=m,K_n=k\mid N,\alpha,\theta)
&=\binom{N}{k}\prod_{q=1}^{Q}\left[\left\{\frac{(\theta_q+\alpha_q)_{n_q}}{(\theta_q)_{n_q}}\right\}^{N}\left\{\frac{-\alpha_q}{(\theta_q+\alpha_q)_{n_q}}\right\}^{k}\right.\\
&\quad\left.\times\prod_{j=1}^{k}(\theta_q+\alpha_q)_{n_q-m_{jq}}(1-\alpha_q)_{m_{jq}-1}\right].
\end{aligned} Pr ( M = m , K n = k ∣ N , α , θ ) = ( k N ) q = 1 ∏ Q [ { ( θ q ) n q ( θ q + α q ) n q } N { ( θ q + α q ) n q − α q } k × j = 1 ∏ k ( θ q + α q ) n q − m j q ( 1 − α q ) m j q − 1 ] .
Finally, marginalizing with respect to N ∼ Poisson ( λ ) N \sim\operatorname{Poisson}(\lambda) N ∼ Poisson ( λ ) we obtain
pr ( M = m , K n = k ∣ λ , α , θ ) = ∑ N ≥ k e − λ λ N N ! ( N k ) ∏ q = 1 Q { ( θ q + α q ) n q ( θ q ) n q } N { − α q ( θ q + α q ) n q } k × ∏ j = 1 k ( θ q + α q ) n q − m j q ( 1 − α q ) m j q − 1 = e − λ λ k k ! ∑ N ≥ k λ N − k ( N − k ) ! ∏ q = 1 Q { ( θ q + α q ) n q ( θ q ) n q } N − k { − α q ( θ q ) n q } k × ∏ j = 1 k ( θ q + α q ) n q − m j q ( 1 − α q ) m j q − 1 = e − λ λ k k ! [ ∑ N ≥ k λ N − k ( N − k ) ! ∏ q = 1 Q { ( θ q + α q ) n q ( θ q ) n q } N − k ] × ∏ q = 1 Q [ { − α q ( θ q ) n q } k ∏ j = 1 k ( θ q + α q ) n q − m j q ( 1 − α q ) m j q − 1 ] = λ k k ! exp ( − λ { 1 − ∏ q = 1 Q { ( α q + θ q ) n q ( θ q ) n q } } ) × ∏ q = 1 Q [ { − α q ( θ q ) n q } k ∏ j = 1 k ( 1 − α q ) m j q − 1 ( α q + θ q ) n q − m j q ] . \begin{aligned}
\operatorname{pr}(M=m,K_n=k\mid\lambda,\alpha,\theta)
&=\sum_{N\geq k}e^{-\lambda}\frac{\lambda^N}{N!}\binom{N}{k}\prod_{q=1}^{Q}\left\{\frac{(\theta_q+\alpha_q)_{n_q}}{(\theta_q)_{n_q}}\right\}^{N}\left\{\frac{-\alpha_q}{(\theta_q+\alpha_q)_{n_q}}\right\}^{k}\\
&\quad\times\prod_{j=1}^{k}(\theta_q+\alpha_q)_{n_q-m_{jq}}(1-\alpha_q)_{m_{jq}-1}\\
&=\frac{e^{-\lambda}\lambda^k}{k!}\sum_{N\geq k}\frac{\lambda^{N-k}}{(N-k)!}\prod_{q=1}^{Q}\left\{\frac{(\theta_q+\alpha_q)_{n_q}}{(\theta_q)_{n_q}}\right\}^{N-k}\left\{\frac{-\alpha_q}{(\theta_q)_{n_q}}\right\}^{k}\\
&\quad\times\prod_{j=1}^{k}(\theta_q+\alpha_q)_{n_q-m_{jq}}(1-\alpha_q)_{m_{jq}-1}\\
&=\frac{e^{-\lambda}\lambda^k}{k!}\left[\sum_{N\geq k}\frac{\lambda^{N-k}}{(N-k)!}\prod_{q=1}^{Q}\left\{\frac{(\theta_q+\alpha_q)_{n_q}}{(\theta_q)_{n_q}}\right\}^{N-k}\right]\\
&\quad\times\prod_{q=1}^{Q}\left[\left\{\frac{-\alpha_q}{(\theta_q)_{n_q}}\right\}^{k}\prod_{j=1}^{k}(\theta_q+\alpha_q)_{n_q-m_{jq}}(1-\alpha_q)_{m_{jq}-1}\right]\\
&=\frac{\lambda^k}{k!}\exp\left(-\lambda\left\{1-\prod_{q=1}^{Q}\left\{\frac{(\alpha_q+\theta_q)_{n_q}}{(\theta_q)_{n_q}}\right\}\right\}\right)\\
&\quad\times\prod_{q=1}^{Q}\left[\left\{\frac{-\alpha_q}{(\theta_q)_{n_q}}\right\}^{k}\prod_{j=1}^{k}(1-\alpha_q)_{m_{jq}-1}(\alpha_q+\theta_q)_{n_q-m_{jq}}\right].
\end{aligned} pr ( M = m , K n = k ∣ λ , α , θ ) = N ≥ k ∑ e − λ N ! λ N ( k N ) q = 1 ∏ Q { ( θ q ) n q ( θ q + α q ) n q } N { ( θ q + α q ) n q − α q } k × j = 1 ∏ k ( θ q + α q ) n q − m j q ( 1 − α q ) m j q − 1 = k ! e − λ λ k N ≥ k ∑ ( N − k )! λ N − k q = 1 ∏ Q { ( θ q ) n q ( θ q + α q ) n q } N − k { ( θ q ) n q − α q } k × j = 1 ∏ k ( θ q + α q ) n q − m j q ( 1 − α q ) m j q − 1 = k ! e − λ λ k [ N ≥ k ∑ ( N − k )! λ N − k q = 1 ∏ Q { ( θ q ) n q ( θ q + α q ) n q } N − k ] × q = 1 ∏ Q [ { ( θ q ) n q − α q } k j = 1 ∏ k ( θ q + α q ) n q − m j q ( 1 − α q ) m j q − 1 ] = k ! λ k exp ( − λ { 1 − q = 1 ∏ Q { ( θ q ) n q ( α q + θ q ) n q } } ) × q = 1 ∏ Q [ { ( θ q ) n q − α q } k j = 1 ∏ k ( 1 − α q ) m j q − 1 ( α q + θ q ) n q − m j q ] .
Posterior computation
Posterior computation for the proposed MOSAIC model proceeds through a Markov chain Monte Carlo (MCMC) algorithm. The procedure is facilitated by the explicit relationship between the partially exchangeable BB and MOSAIC models established in Theorem 1, and by the availability of the closed-form pEFPF in (5) . Indeed, draws from the posterior distribution of ( α , θ ) (\alpha,\theta) ( α , θ ) can be obtained via any collapsed Metropolis–Hastings algorithm leveraging the marginal likelihood in (5) . Specifically, we obtain posterior samples of the regression coefficients ζ \zeta ζ and of the precision parameters ϕ \phi ϕ using Hamiltonian Monte Carlo implemented in the probabilistic programming language Stan [8 ] , and then map these samples through (6) to recover posterior draws of ( α , θ ) (\alpha,\theta) ( α , θ ) . We recall that the posterior of N N N has a closed-form expression, as stated in Theorem 2, so that no MCMC step is needed to obtain posterior draws of the number of unseen species. To sample the occupancy and detectability components η q \eta_q η q and γ q \gamma_q γ q , the sampler relies on a data augmentation strategy based on the augmented arrays W ~ \widetilde{W} W and T ~ \widetilde{T} T . Conditional on N N N and on these augmented data, the full conditional distributions of η j q \eta_{jq} η j q and γ j q \gamma_{jq} γ j q are beta by conjugacy, which enables straightforward Gibbs updates. The full sampling procedure is summarized in Algorithm 1, which is stated for the Poisson formulation with λ \lambda λ fixed, for the sake of simplicity. The extension to a random λ \lambda λ with a Gamma prior is straightforward
Algorithm 1 Posterior sampling algorithm for the MOSAIC model
Sample B B B draws of ζ \zeta ζ and ϕ \phi ϕ from their posterior through a collapsed HMC algorithm targeting the marginal likelihood (5).
Set θ ( b ) = ϕ ( b ) \theta^{(b)}=\phi^{(b)} θ ( b ) = ϕ ( b ) and α ( b ) = − μ ( b ) ϕ ( b ) \alpha^{(b)}=-\mu^{(b)}\phi^{(b)} α ( b ) = − μ ( b ) ϕ ( b ) , with μ ( b ) = 1 / ( 1 + e − Z ζ ( b ) ) \mu^{(b)}=1/(1+e^{-Z\zeta^{(b)}}) μ ( b ) = 1/ ( 1 + e − Z ζ ( b ) ) , for b = 1 , … , B b=1,\ldots,B b = 1 , … , B .
For b = 1 , … , B b=1,\ldots,B b = 1 , … , B , sample the number of unseen species N ′ ( b ) ∼ Poisson { λ p 0 ( α ( b ) , θ ( b ) , n ) } N'^{(b)}\sim\operatorname{Poisson}\{\lambda p_0(\alpha^{(b)},\theta^{(b)},\mathbf{n})\} N ′ ( b ) ∼ Poisson { λ p 0 ( α ( b ) , θ ( b ) , n )} and set N ( b ) = k + N ′ ( b ) N^{(b)}=k+N'^{(b)} N ( b ) = k + N ′ ( b ) .
For each draw b = 1 , … , B b=1,\ldots,B b = 1 , … , B and each site q = 1 , … , Q q=1,\ldots,Q q = 1 , … , Q (in parallel), suppressing the index b b b for readability and setting y ~ i j q = y i j q \tilde{y}_{ijq}=y_{ijq} y ~ ij q = y ij q for the observed species j = 1 , … , k j=1,\ldots,k j = 1 , … , k and y ~ i j q = 0 \tilde{y}_{ijq}=0 y ~ ij q = 0 for the unseen ones j = k + 1 , … , N j=k+1,\ldots,N j = k + 1 , … , N :
(i) for i = 1 , … , n q i=1,\ldots,n_q i = 1 , … , n q and j = 1 , … , N j=1,\ldots,N j = 1 , … , N , if y ~ i j q = 1 \tilde{y}_{ijq}=1 y ~ ij q = 1 then set W ~ i j q = T ~ i j q = 1 \widetilde{W}_{ijq}=\widetilde{T}_{ijq}=1 W ij q = T ij q = 1 , otherwise sample ( W ~ i j q , T ~ i j q ) (\widetilde{W}_{ijq},\widetilde{T}_{ijq}) ( W ij q , T ij q ) from the categorical distribution
( W ~ i j q , T ~ i j q ) = { ( 0 , 0 ) with probability proportional to ( 1 − η j q ) ( 1 − γ j q ) , ( 0 , 1 ) with probability proportional to ( 1 − η j q ) γ j q , ( 1 , 0 ) with probability proportional to η j q ( 1 − γ j q ) . (\widetilde{W}_{ijq},\widetilde{T}_{ijq})=
\begin{cases}
(0,0) & \text{with probability proportional to }(1-\eta_{jq})(1-\gamma_{jq}),\\
(0,1) & \text{with probability proportional to }(1-\eta_{jq})\gamma_{jq},\\
(1,0) & \text{with probability proportional to }\eta_{jq}(1-\gamma_{jq}).
\end{cases} ( W ij q , T ij q ) = ⎩ ⎨ ⎧ ( 0 , 0 ) ( 0 , 1 ) ( 1 , 0 ) with probability proportional to ( 1 − η j q ) ( 1 − γ j q ) , with probability proportional to ( 1 − η j q ) γ j q , with probability proportional to η j q ( 1 − γ j q ) .
(ii) for j = 1 , … , N j=1,\ldots,N j = 1 , … , N , sample
η j q ∣ − ∼ Beta { − α q + ∑ i = 1 n q W ~ i j q , σ q ( α q + θ q ) + n q − ∑ i = 1 n q W ~ i j q } , \eta_{jq}\mid-\sim\operatorname{Beta}\left\{-\alpha_q+\sum_{i=1}^{n_q}\widetilde{W}_{ijq},\ \sigma_q(\alpha_q+\theta_q)+n_q-\sum_{i=1}^{n_q}\widetilde{W}_{ijq}\right\}, η j q ∣ − ∼ Beta { − α q + i = 1 ∑ n q W ij q , σ q ( α q + θ q ) + n q − i = 1 ∑ n q W ij q } ,
γ j q ∣ − ∼ Beta { − α q + σ q ( α q + θ q ) + ∑ i = 1 n q T ~ i j q , ( 1 − σ q ) ( α q + θ q ) + n q − ∑ i = 1 n q T ~ i j q } . \gamma_{jq}\mid-\sim\operatorname{Beta}\left\{-\alpha_q+\sigma_q(\alpha_q+\theta_q)+\sum_{i=1}^{n_q}\widetilde{T}_{ijq},\ (1-\sigma_q)(\alpha_q+\theta_q)+n_q-\sum_{i=1}^{n_q}\widetilde{T}_{ijq}\right\}. γ j q ∣ − ∼ Beta { − α q + σ q ( α q + θ q ) + i = 1 ∑ n q T ij q , ( 1 − σ q ) ( α q + θ q ) + n q − i = 1 ∑ n q T ij q } .
and requires only two modifications: the marginal likelihood in step 1 and the sampling distribution of N ′ N' N ′ in step 3 are replaced by their negative binomial counterparts, reported in Section S.3.
The negative binomial extension
Extension of theoretical results
In this section we adapt the results of Section 3.1 of the main paper to the case in which the total number of species follows a negative binomial distribution, i.e. N ∼ NegBinomial ( r 0 , ν 0 ) N\sim\operatorname{NegBinomial}(r_0,\nu_0) N ∼ NegBinomial ( r 0 , ν 0 ) with size r 0 > 0 r_0>0 r 0 > 0 and mean ν 0 > 0 \nu_0>0 ν 0 > 0 . This is equivalent to assuming N ∼ Poisson ( λ ) N\sim\operatorname{Poisson}(\lambda) N ∼ Poisson ( λ ) and λ ∼ Gamma ( r 0 , r 0 / ν 0 ) \lambda\sim\operatorname{Gamma}(r_0,r_0/\nu_0) λ ∼ Gamma ( r 0 , r 0 / ν 0 ) . All the remaining assumptions of the MOSAIC model are left unchanged. The results below are obtained exactly as their Poisson counterparts, the only difference being that Lemma S2 is replaced by the following thinning property of the negative binomial distribution.
Lemma S3. If V ∼ NegBinomial ( r , μ ) V\sim\operatorname{NegBinomial}(r,\mu) V ∼ NegBinomial ( r , μ ) and X ∣ V ∼ Binomial ( V , p ) X\mid V\sim\operatorname{Binomial}(V,p) X ∣ V ∼ Binomial ( V , p ) , then X ∼ NegBinomial ( r , μ p ) X\sim\operatorname{NegBinomial}(r,\mu p) X ∼ NegBinomial ( r , μ p ) .
Proof. Let ϕ = μ / ( μ + r ) \phi=\mu/(\mu+r) ϕ = μ / ( μ + r ) , so that the probability generating function of V V V is E ( z V ) = { ( 1 − ϕ ) / ( 1 − ϕ z ) } r E(z^V)=\{(1-\phi)/(1-\phi z)\}^r E ( z V ) = {( 1 − ϕ ) / ( 1 − ϕ z ) } r . Then
E ( z X ) = E { E ( z X ∣ V ) } = E { ( 1 − p + p z ) V } = ( 1 − ϕ 1 − ϕ ( 1 − p ) − ϕ p z ) r = ( 1 − ϕ ′ 1 − ϕ ′ z ) r , E(z^X)=E\{E(z^X\mid V)\}=E\{(1-p+pz)^V\}=\left(\frac{1-\phi}{1-\phi(1-p)-\phi pz}\right)^r=\left(\frac{1-\phi'}{1-\phi'z}\right)^r, E ( z X ) = E { E ( z X ∣ V )} = E {( 1 − p + p z ) V } = ( 1 − ϕ ( 1 − p ) − ϕp z 1 − ϕ ) r = ( 1 − ϕ ′ z 1 − ϕ ′ ) r ,
with ϕ ′ = ϕ p / { 1 − ϕ ( 1 − p ) } \phi'=\phi p/\{1-\phi(1-p)\} ϕ ′ = ϕp / { 1 − ϕ ( 1 − p )} , since 1 − ϕ ′ = ( 1 − ϕ ) / { 1 − ϕ ( 1 − p ) } 1-\phi'=(1-\phi)/\{1-\phi(1-p)\} 1 − ϕ ′ = ( 1 − ϕ ) / { 1 − ϕ ( 1 − p )} . Hence X X X is negative binomial with size r r r and mean r ϕ ′ / ( 1 − ϕ ′ ) = r ϕ p / ( 1 − ϕ ) = μ p r\phi'/(1-\phi')=r\phi p/(1-\phi)=\mu p r ϕ ′ / ( 1 − ϕ ′ ) = r ϕp / ( 1 − ϕ ) = μ p .
We first state the analogue of Theorem 2 in the main paper, which is a special case of a more general result in [14 ] . Recall that N ′ = N − k N'=N-k N ′ = N − k denotes the number of species unseen given Y Y Y and that L ′ = N − ℓ L'=N-\ell L ′ = N − ℓ denotes the number of species never present given W W W .
Proposition S1. Let Y Y Y follow the negative binomial specification of the MOSAIC model with parameters ( r 0 , ν 0 , α , θ , σ ) (r_0,\nu_0,\alpha,\theta,\sigma) ( r 0 , ν 0 , α , θ , σ ) , then
N ′ ∣ Y ∼ NegBinomial ( r 0 + k , μ 0 ) , μ 0 = ( r 0 + k ) p 0 ( α , θ , n ) r 0 / ν 0 + p 1 ( α , θ , n ) . N'\mid Y\sim\operatorname{NegBinomial}(r_0+k,\mu_0),\qquad\mu_0=\frac{(r_0+k)p_0(\alpha,\theta,n)}{r_0/\nu_0+p_1(\alpha,\theta,n)}. N ′ ∣ Y ∼ NegBinomial ( r 0 + k , μ 0 ) , μ 0 = r 0 / ν 0 + p 1 ( α , θ , n ) ( r 0 + k ) p 0 ( α , θ , n ) .
whereas the posterior distributions of the occurrence probabilities π j q ∗ \pi^*_{jq} π j q ∗ and π j q ′ \pi'_{jq} π j q ′ are those given in Theorem 2, and are therefore unchanged.
Proposition S2 (A priori distinct species ). Under the MOSAIC model with N ∼ NegBinomial ( r 0 , ν 0 ) N\sim\operatorname{NegBinomial}(r_0,\nu_0) N ∼ NegBinomial ( r 0 , ν 0 ) , the number of distinct species observed at site q q q and across sites is distributed as
K q , n q ∼ NegBinomial { r 0 , ν 0 p 1 , q ( α q , θ q , n q ) } , K n ∼ NegBinomial { r 0 , ν 0 p 1 ( α , θ , n ) } , K_{q,n_q}\sim\operatorname{NegBinomial}\{r_0,\nu_0p_{1,q}(\alpha_q,\theta_q,n_q)\},\qquad K_n\sim\operatorname{NegBinomial}\{r_0,\nu_0p_1(\alpha,\theta,n)\}, K q , n q ∼ NegBinomial { r 0 , ν 0 p 1 , q ( α q , θ q , n q )} , K n ∼ NegBinomial { r 0 , ν 0 p 1 ( α , θ , n )} ,
for q = 1 , … , Q q=1,\ldots,Q q = 1 , … , Q , whereas the number of distinct species present at site q q q and across sites is distributed as
L q , n q ∼ NegBinomial { r 0 , ν 0 p 1 , q ( α q , θ q ( σ q ) , n q ) } , L n ∼ NegBinomial { r 0 , ν 0 p 1 ( α , θ ( σ ) , n ) } . L_{q,n_q}\sim\operatorname{NegBinomial}\{r_0,\nu_0p_{1,q}(\alpha_q,\theta_q(\sigma_q),n_q)\},\qquad L_n\sim\operatorname{NegBinomial}\{r_0,\nu_0p_1(\alpha,\theta(\sigma),n)\}. L q , n q ∼ NegBinomial { r 0 , ν 0 p 1 , q ( α q , θ q ( σ q ) , n q )} , L n ∼ NegBinomial { r 0 , ν 0 p 1 ( α , θ ( σ ) , n )} .
Proof. The proof of Proposition 1 shows that K q , n q ∣ N ∼ Binomial { N , p 1 , q ( α q , θ q , n q ) } K_{q,n_q}\mid N\sim\operatorname{Binomial}\{N,p_{1,q}(\alpha_q,\theta_q,n_q)\} K q , n q ∣ N ∼ Binomial { N , p 1 , q ( α q , θ q , n q )} and K n ∣ N ∼ Binomial { N , p 1 ( α , θ , n ) } K_n\mid N\sim\operatorname{Binomial}\{N,p_1(\alpha,\theta,n)\} K n ∣ N ∼ Binomial { N , p 1 ( α , θ , n )} , since that step does not involve the law of N N N . The statements then follow from Lemma S3 in place of Lemma S2. For the latent counterparts the same proof applies with θ q \theta_q θ q replaced by θ q ( σ q ) \theta_q(\sigma_q) θ q ( σ q ) .
The simple interpretation of the Poisson case carries over: E ( K n ) = ν 0 p 1 ( α , θ , n ) E(K_n)=\nu_0p_1(\alpha,\theta,n) E ( K n ) = ν 0 p 1 ( α , θ , n ) , with ν 0 \nu_0 ν 0 the expected total number of species and p 1 ( α , θ , n ) p_1(\alpha,\theta,n) p 1 ( α , θ , n ) the expected fraction of these that is observed under sampling effort n n n .
Proposition S3 (A priori shared species ). Under the MOSAIC model with N ∼ NegBinomial ( r 0 , ν 0 ) N\sim\operatorname{NegBinomial}(r_0,\nu_0) N ∼ NegBinomial ( r 0 , ν 0 ) , the number of species shared between sites q q q and r r r , with q ≠ r q\ne r q = r , is distributed as
C n q , n r ∼ NegBinomial { r 0 , ν 0 p 1 , q ( α q , θ q , n q ) p 1 , r ( α r , θ r , n r ) } , C_{n_q,n_r}\sim\operatorname{NegBinomial}\{r_0,\nu_0p_{1,q}(\alpha_q,\theta_q,n_q)p_{1,r}(\alpha_r,\theta_r,n_r)\}, C n q , n r ∼ NegBinomial { r 0 , ν 0 p 1 , q ( α q , θ q , n q ) p 1 , r ( α r , θ r , n r )} ,
whereas its latent counterpart is distributed as
U n q , n r ∼ NegBinomial { r 0 , ν 0 p 1 , q ( α q , θ q ( σ q ) , n q ) p 1 , r ( α r , θ r ( σ r ) , n r ) } . U_{n_q,n_r} \sim\operatorname{NegBinomial}\left\{r_0,\nu_0 p_{1,q}(\alpha_q,\theta_q(\sigma_q),n_q)p_{1,r}(\alpha_r,\theta_r(\sigma_r),n_r)\right\}. U n q , n r ∼ NegBinomial { r 0 , ν 0 p 1 , q ( α q , θ q ( σ q ) , n q ) p 1 , r ( α r , θ r ( σ r ) , n r ) } .
Proof. As in the proof of Proposition 2, C n q , n r ∣ N ∼ Binomial { N , p 1 , q ( α q , θ q , n q ) p 1 , r ( α r , θ r , n r ) } C_{n_q,n_r} \mid N \sim\operatorname{Binomial}\left\{N,p_{1,q}(\alpha_q,\theta_q,n_q)p_{1,r}(\alpha_r,\theta_r,n_r)\right\} C n q , n r ∣ N ∼ Binomial { N , p 1 , q ( α q , θ q , n q ) p 1 , r ( α r , θ r , n r ) } , and the statement follows from Lemma S3; the latent counterpart is obtained by replacing θ q \theta_q θ q and θ r \theta_r θ r with θ q ( σ q ) \theta_q(\sigma_q) θ q ( σ q ) and θ r ( σ r ) \theta_r(\sigma_r) θ r ( σ r ) respectively.
Proposition S4 (A posteriori distinct species). Under the MOSAIC model with N ∼ NegBinomial ( r 0 , ν 0 ) N \sim\operatorname{NegBinomial}(r_0,\nu_0) N ∼ NegBinomial ( r 0 , ν 0 ) , the number of new species discovered at site q q q and across sites satisfies
K q , s q ( n q ) ∣ Y ∼ d NegBinomial { r 0 + k , μ 0 p 1 , q ( α q , θ q + n q , s q ) } + Binomial { k − k q , p 1 , q ( α q , θ q + n q , s q ) } , ( q = 1 , … , Q ) , K_{q,s_q}^{(n_q)} \mid\mathbf{Y} \mathrel{\mathop{\sim}\limits^{d}} \operatorname{NegBinomial}\left\{r_0+k,\mu_0p_{1,q}(\alpha_q,\theta_q+n_q,s_q)\right\}
+\operatorname{Binomial}\left\{k-k_q,p_{1,q}(\alpha_q,\theta_q+n_q,s_q)\right\}, \qquad(q=1,\ldots,Q), K q , s q ( n q ) ∣ Y ∼ d NegBinomial { r 0 + k , μ 0 p 1 , q ( α q , θ q + n q , s q ) } + Binomial { k − k q , p 1 , q ( α q , θ q + n q , s q ) } , ( q = 1 , … , Q ) ,
K s ( n ) ∣ Y ∼ NegBinomial { r 0 + k , μ 0 p 1 ( α , θ + n , s ) } , K_s^{(n)} \mid\mathbf{Y} \sim\operatorname{NegBinomial}\left\{r_0+k,\mu_0p_1(\boldsymbol{\alpha},\boldsymbol{\theta}+\mathbf{n},\mathbf{s})\right\}, K s ( n ) ∣ Y ∼ NegBinomial { r 0 + k , μ 0 p 1 ( α , θ + n , s ) } ,
with μ 0 \mu_0 μ 0 as in Proposition S1, whereas their latent counterparts satisfy
L q , s q ( n q ) ∣ W ∼ d NegBinomial { r 0 + ℓ , μ 0 W p 1 , q ( α q , θ q ( σ q ) + n q , s q ) } + Binomial { ℓ − ℓ q , p 1 , q ( α q , θ q ( σ q ) + n q , s q ) } , ( q = 1 , … , Q ) , L_{q,s_q}^{(n_q)} \mid\mathbf{W} \mathrel{\mathop{\sim}\limits^{d}} \operatorname{NegBinomial}\left\{r_0+\ell,\mu_0^{W}p_{1,q}(\alpha_q,\theta_q(\sigma_q)+n_q,s_q)\right\}
+\operatorname{Binomial}\left\{\ell-\ell_q,p_{1,q}(\alpha_q,\theta_q(\sigma_q)+n_q,s_q)\right\}, \qquad(q=1,\ldots,Q), L q , s q ( n q ) ∣ W ∼ d NegBinomial { r 0 + ℓ , μ 0 W p 1 , q ( α q , θ q ( σ q ) + n q , s q ) } + Binomial { ℓ − ℓ q , p 1 , q ( α q , θ q ( σ q ) + n q , s q ) } , ( q = 1 , … , Q ) ,
L s ( n ) ∣ W ∼ NegBinomial { r 0 + ℓ , μ 0 W p 1 ( α , θ ( σ ) + n , s ) } , L_s^{(n)} \mid\mathbf{W} \sim\operatorname{NegBinomial}\left\{r_0+\ell,\mu_0^{W}p_1(\boldsymbol{\alpha},\boldsymbol{\theta}(\boldsymbol{\sigma})+\mathbf{n},\mathbf{s})\right\}, L s ( n ) ∣ W ∼ NegBinomial { r 0 + ℓ , μ 0 W p 1 ( α , θ ( σ ) + n , s ) } ,
with
μ 0 W = ( r 0 + ℓ ) p 0 ( α , θ ( σ ) , n ) r 0 / ν 0 + p 1 ( α , θ ( σ ) , n ) . \mu_0^{W}=\frac{(r_0+\ell)p_0(\boldsymbol{\alpha},\boldsymbol{\theta}(\boldsymbol{\sigma}),\mathbf{n})}{r_0/\nu_0+p_1(\boldsymbol{\alpha},\boldsymbol{\theta}(\boldsymbol{\sigma}),\mathbf{n})}. μ 0 W = r 0 / ν 0 + p 1 ( α , θ ( σ ) , n ) ( r 0 + ℓ ) p 0 ( α , θ ( σ ) , n ) .
Proof. The proof of Theorem 3 applies up to the point where the number of unseen species is integrated out: conditionally on N ′ N' N ′ one has K s ( n ) ∣ N ′ , Y ∼ Binomial { N ′ , p 1 ( α , θ + n , s ) } K_s^{(n)} \mid N',\mathbf{Y} \sim\operatorname{Binomial}\left\{N',p_1(\boldsymbol{\alpha},\boldsymbol{\theta}+\mathbf{n},\mathbf{s})\right\} K s ( n ) ∣ N ′ , Y ∼ Binomial { N ′ , p 1 ( α , θ + n , s ) } , and the species new to site q q q split into the N ′ N' N ′ species unrecorded everywhere and the k − k q k-k_q k − k q species recorded at some other site but not at q q q , each contributing with probability p 1 , q ( α q , θ q + n q , s q ) p_{1,q}(\alpha_q,\theta_q+n_q,s_q) p 1 , q ( α q , θ q + n q , s q ) . Since N ′ ∣ Y ∼ NegBinomial ( r 0 + k , μ 0 ) N' \mid\mathbf{Y} \sim\operatorname{NegBinomial}(r_0+k,\mu_0) N ′ ∣ Y ∼ NegBinomial ( r 0 + k , μ 0 ) by Proposition S1, applying Lemma S3 instead of Lemma S2 to the first group gives the stated expressions, the binomial term being unchanged because k − k q k-k_q k − k q is known given Y \mathbf{Y} Y . The latent counterparts are obtained analogously, by replacing Y \mathbf{Y} Y with W \mathbf{W} W , θ \boldsymbol{\theta} θ with θ ( σ ) \boldsymbol{\theta}(\boldsymbol{\sigma}) θ ( σ ) and ( k , k q , N ′ , μ 0 ) (k,k_q,N',\mu_0) ( k , k q , N ′ , μ 0 ) with ( ℓ , ℓ q , L ′ , μ 0 W ) (\ell,\ell_q,L',\mu_0^{W}) ( ℓ , ℓ q , L ′ , μ 0 W ) .
Proposition S5 (A posteriori shared species). Under the MOSAIC model with N ∼ NegBinomial ( r 0 , ν 0 ) N \sim\operatorname{NegBinomial}(r_0,\nu_0) N ∼ NegBinomial ( r 0 , ν 0 ) , the number of species shared between sites q q q and r r r , with q ≠ r q \ne r q = r , that would be discovered in ( s q , s r ) (s_q,s_r) ( s q , s r ) additional samples satisfies
C s q , s r ( n q , n r ) ∣ Y ∼ d NegBinomial { r 0 + k , μ 0 p 1 , q ( α q , θ q + n q , s q ) p 1 , r ( α r , θ r + n r , s r ) } + Binomial { k − k q , r , p 1 , q ( α q , θ q + n q , s q ) p 1 , r ( α r , θ r + n r , s r ) } + Binomial { k q − c q , r , p 1 , r ( α r , θ r + n r , s r ) } + Binomial { k r − c q , r , p 1 , q ( α q , θ q + n q , s q ) } , C_{s_q,s_r}^{(n_q,n_r)} \mid\mathbf{Y} \mathrel{\mathop{\sim}\limits^{d}} \operatorname{NegBinomial}\left\{r_0+k,\mu_0p_{1,q}(\alpha_q,\theta_q+n_q,s_q)p_{1,r}(\alpha_r,\theta_r+n_r,s_r)\right\}
+\operatorname{Binomial}\left\{k-k_{q,r},p_{1,q}(\alpha_q,\theta_q+n_q,s_q)p_{1,r}(\alpha_r,\theta_r+n_r,s_r)\right\}
+\operatorname{Binomial}\left\{k_q-c_{q,r},p_{1,r}(\alpha_r,\theta_r+n_r,s_r)\right\}
+\operatorname{Binomial}\left\{k_r-c_{q,r},p_{1,q}(\alpha_q,\theta_q+n_q,s_q)\right\}, C s q , s r ( n q , n r ) ∣ Y ∼ d NegBinomial { r 0 + k , μ 0 p 1 , q ( α q , θ q + n q , s q ) p 1 , r ( α r , θ r + n r , s r ) } + Binomial { k − k q , r , p 1 , q ( α q , θ q + n q , s q ) p 1 , r ( α r , θ r + n r , s r ) } + Binomial { k q − c q , r , p 1 , r ( α r , θ r + n r , s r ) } + Binomial { k r − c q , r , p 1 , q ( α q , θ q + n q , s q ) } ,
with μ 0 \mu_0 μ 0 as in Proposition S1, whereas its latent counterpart satisfies
U s q , s r ( n q , n r ) ∣ W = d NegBinomial { r 0 + ℓ , μ 0 W p 1 , q ( α q , θ q ( σ q ) + n q , s q ) p 1 , r ( α r , θ r ( σ r ) + n r , s r ) } + Binomial { ℓ − ℓ q , r , p 1 , q ( α q , θ q ( σ q ) + n q , s q ) p 1 , r ( α r , θ r ( σ r ) + n r , s r ) } + Binomial { ℓ q − u q , r , p 1 , r ( α r , θ r ( σ r ) + n r , s r ) } + Binomial { ℓ r − u q , r , p 1 , q ( α q , θ q ( σ q ) + n q , s q ) } . U^{(n_q,n_r)}_{s_q,s_r} \mid W \mathrel{\overset{d}{=}} \operatorname{NegBinomial}\left\{r_0+\ell,\mu_0^W p_{1,q}(\alpha_q,\theta_q(\sigma_q)+n_q,s_q)p_{1,r}(\alpha_r,\theta_r(\sigma_r)+n_r,s_r)\right\}
+ \operatorname{Binomial}\left\{\ell-\ell_{q,r},p_{1,q}(\alpha_q,\theta_q(\sigma_q)+n_q,s_q)p_{1,r}(\alpha_r,\theta_r(\sigma_r)+n_r,s_r)\right\}
+ \operatorname{Binomial}\left\{\ell_q-u_{q,r},p_{1,r}(\alpha_r,\theta_r(\sigma_r)+n_r,s_r)\right\}
+ \operatorname{Binomial}\left\{\ell_r-u_{q,r},p_{1,q}(\alpha_q,\theta_q(\sigma_q)+n_q,s_q)\right\}. U s q , s r ( n q , n r ) ∣ W = d NegBinomial { r 0 + ℓ , μ 0 W p 1 , q ( α q , θ q ( σ q ) + n q , s q ) p 1 , r ( α r , θ r ( σ r ) + n r , s r ) } + Binomial { ℓ − ℓ q , r , p 1 , q ( α q , θ q ( σ q ) + n q , s q ) p 1 , r ( α r , θ r ( σ r ) + n r , s r ) } + Binomial { ℓ q − u q , r , p 1 , r ( α r , θ r ( σ r ) + n r , s r ) } + Binomial { ℓ r − u q , r , p 1 , q ( α q , θ q ( σ q ) + n q , s q ) } .
with μ 0 W \mu_0^W μ 0 W as in Proposition S4.
Proof. Identical to the proof of Theorem 4: the four groups of species and the corresponding success probabilities are unaffected by the law of N N N , and only the first term, which involves the N ′ N' N ′ species unrecorded everywhere, requires integrating out N ′ N' N ′ . Since N ′ ∣ Y ∼ NegBinomial ( r 0 + k , μ 0 ) N' \mid Y \sim\operatorname{NegBinomial}(r_0+k,\mu_0) N ′ ∣ Y ∼ NegBinomial ( r 0 + k , μ 0 ) by Proposition S1, Lemma S3 replaces Lemma S2 in that step. The latent counterpart follows in the same way, with θ \boldsymbol{\theta} θ replaced by θ ( σ ) \boldsymbol{\theta}(\boldsymbol{\sigma}) θ ( σ ) and ( k , k q , k r , c q , r , k q , r , N ′ , μ 0 ) (k,k_q,k_r,c_{q,r},k_{q,r},N',\mu_0) ( k , k q , k r , c q , r , k q , r , N ′ , μ 0 ) replaced by ( ℓ , ℓ q , ℓ r , u q , r , ℓ q , r , L ′ , μ 0 W ) (\ell,\ell_q,\ell_r,u_{q,r},\ell_{q,r},L',\mu_0^W) ( ℓ , ℓ q , ℓ r , u q , r , ℓ q , r , L ′ , μ 0 W ) .
Derivation of the pEFPP
In this section, we obtain the analytical formulation for the marginal distribution of the proposed MOSAIC model under the assumption N ∼ NegBinomial ( r 0 , ν 0 ) N \sim\operatorname{NegBinomial}(r_0,\nu_0) N ∼ NegBinomial ( r 0 , ν 0 ) . Let π n ( m ; r 0 , ν 0 , θ , α ) = Pr ( M = m , K n = k ∣ r 0 , ν 0 , θ , α ) \pi_n(\boldsymbol{m};r_0,\nu_0,\boldsymbol{\theta},\boldsymbol{\alpha})=\Pr(M=m,K_n=k\mid r_0,\nu_0,\boldsymbol{\theta},\boldsymbol{\alpha}) π n ( m ; r 0 , ν 0 , θ , α ) = Pr ( M = m , K n = k ∣ r 0 , ν 0 , θ , α ) , then adapting the results presented in Section S.2, the pEFPP is
π n ( m ; r 0 , ν 0 , θ , α ) = ( k + r 0 − 1 k ) ( r 0 ν 0 + r 0 ) r 0 { 1 − ( ν 0 ν 0 + r 0 ) ∏ q = 1 Q ( θ q + α q ) n q ( θ q ) n q } − k − r 0 ( ν 0 ν 0 + r 0 ) k ∏ q = 1 Q { − α q ( θ q ) n q } k ∏ j = 1 k ( 1 − α q ) m j q − 1 ( α q + θ q ) n q − m j q . (9) \pi_n(\boldsymbol{m};r_0,\nu_0,\boldsymbol{\theta},\boldsymbol{\alpha})=
\binom{k+r_0-1}{k}
\left(\frac{r_0}{\nu_0+r_0}\right)^{r_0}
\left\{1-\left(\frac{\nu_0}{\nu_0+r_0}\right)\prod_{q=1}^{Q}\frac{(\theta_q+\alpha_q)_{n_q}}{(\theta_q)_{n_q}}\right\}^{-k-r_0}
\left(\frac{\nu_0}{\nu_0+r_0}\right)^k
\prod_{q=1}^{Q}\left\{\frac{-\alpha_q}{(\theta_q)_{n_q}}\right\}^{k}
\prod_{j=1}^{k}(1-\alpha_q)_{m_{jq}-1}(\alpha_q+\theta_q)_{n_q-m_{jq}}. \tag*{(9)} π n ( m ; r 0 , ν 0 , θ , α ) = ( k k + r 0 − 1 ) ( ν 0 + r 0 r 0 ) r 0 { 1 − ( ν 0 + r 0 ν 0 ) q = 1 ∏ Q ( θ q ) n q ( θ q + α q ) n q } − k − r 0 ( ν 0 + r 0 ν 0 ) k q = 1 ∏ Q { ( θ q ) n q − α q } k j = 1 ∏ k ( 1 − α q ) m j q − 1 ( α q + θ q ) n q − m j q . (9)
Proof. Adapting the calculation in Section S.2, given the following expression
π n ( m ; N , α , θ ) = ( N k ) ∏ q = 1 Q { ( θ q + α q ) n q ( θ q ) n q } N { − α q ( θ q + α q ) n q } k ∏ j = 1 k ( θ q + α q ) n q − m j q ( 1 − α q ) m j q − 1 , \pi_n(\boldsymbol{m};N,\boldsymbol{\alpha},\boldsymbol{\theta})=
\binom{N}{k}
\prod_{q=1}^{Q}\left\{\frac{(\theta_q+\alpha_q)_{n_q}}{(\theta_q)_{n_q}}\right\}^{N}
\left\{\frac{-\alpha_q}{(\theta_q+\alpha_q)_{n_q}}\right\}^{k}
\prod_{j=1}^{k}(\theta_q+\alpha_q)_{n_q-m_{jq}}(1-\alpha_q)_{m_{jq}-1}, π n ( m ; N , α , θ ) = ( k N ) q = 1 ∏ Q { ( θ q ) n q ( θ q + α q ) n q } N { ( θ q + α q ) n q − α q } k j = 1 ∏ k ( θ q + α q ) n q − m j q ( 1 − α q ) m j q − 1 ,
we need to marginalize with respect to N ∼ NegBinomial ( r 0 , ν 0 ) N \sim\operatorname{NegBinomial}(r_0,\nu_0) N ∼ NegBinomial ( r 0 , ν 0 ) , hence
π n ( m ; r 0 , ν 0 , α , θ ) = ∑ N ≥ k ( N + r 0 − 1 N ) ( ν 0 ν 0 + r 0 ) N ( r 0 r 0 + ν 0 ) r 0 ( N k ) ∏ q = 1 Q { ( θ q + α q ) n q ( θ q ) n q } N × { − α q ( θ q + α q ) n q } k ∏ j = 1 k ( θ q + α q ) n q − m j q ( 1 − α q ) m j q − 1 = 1 k ! ( r 0 − 1 ) ! [ ∏ q = 1 Q { − α q ( θ q + α q ) n q } k ∏ j = 1 k ( θ q + α q ) n q − m j q ( 1 − α q ) m j q − 1 ] × ( r 0 r 0 + ν 0 ) r 0 ∑ N ≥ k ( N + r 0 − 1 ) ! ( N − k ) ! ( ν 0 ν 0 + r 0 ) N ∏ q = 1 Q { ( θ q + α q ) n q ( θ q ) n q } N = 1 k ! ( r 0 − 1 ) ! [ ∏ q = 1 Q { − α q ( θ q + α q ) n q } k ∏ j = 1 k ( θ q + α q ) n q − m j q ( 1 − α q ) m j q − 1 ] × ( r 0 r 0 + ν 0 ) r 0 ∑ N ≥ 0 ( N + k + r 0 − 1 ) ! N ! ( ν 0 ν 0 + r 0 ) N + k ∏ q = 1 Q { ( θ q + α q ) n q ( θ q ) n q } N + k . \begin{aligned}
\pi_n(m;r_0,\nu_0,\boldsymbol{\alpha},\boldsymbol{\theta})={}&\sum_{N\geq k}\binom{N+r_0-1}{N}\left(\frac{\nu_0}{\nu_0+r_0}\right)^N\left(\frac{r_0}{r_0+\nu_0}\right)^{r_0}\binom{N}{k}\prod_{q=1}^{Q}\left\{\frac{(\theta_q+\alpha_q)_{n_q}}{(\theta_q)_{n_q}}\right\}^{N}\\
&\times\left\{\frac{-\alpha_q}{(\theta_q+\alpha_q)_{n_q}}\right\}^{k}\prod_{j=1}^{k}(\theta_q+\alpha_q)_{n_q-m_{jq}}(1-\alpha_q)_{m_{jq}-1}\\
={}&\frac{1}{k!(r_0-1)!}\left[\prod_{q=1}^{Q}\left\{\frac{-\alpha_q}{(\theta_q+\alpha_q)_{n_q}}\right\}^{k}\prod_{j=1}^{k}(\theta_q+\alpha_q)_{n_q-m_{jq}}(1-\alpha_q)_{m_{jq}-1}\right]\\
&\times\left(\frac{r_0}{r_0+\nu_0}\right)^{r_0}\sum_{N\geq k}\frac{(N+r_0-1)!}{(N-k)!}\left(\frac{\nu_0}{\nu_0+r_0}\right)^N\prod_{q=1}^{Q}\left\{\frac{(\theta_q+\alpha_q)_{n_q}}{(\theta_q)_{n_q}}\right\}^{N}\\
={}&\frac{1}{k!(r_0-1)!}\left[\prod_{q=1}^{Q}\left\{\frac{-\alpha_q}{(\theta_q+\alpha_q)_{n_q}}\right\}^{k}\prod_{j=1}^{k}(\theta_q+\alpha_q)_{n_q-m_{jq}}(1-\alpha_q)_{m_{jq}-1}\right]\\
&\times\left(\frac{r_0}{r_0+\nu_0}\right)^{r_0}\sum_{N\geq0}\frac{(N+k+r_0-1)!}{N!}\left(\frac{\nu_0}{\nu_0+r_0}\right)^{N+k}\prod_{q=1}^{Q}\left\{\frac{(\theta_q+\alpha_q)_{n_q}}{(\theta_q)_{n_q}}\right\}^{N+k}.
\end{aligned} π n ( m ; r 0 , ν 0 , α , θ ) = = = N ≥ k ∑ ( N N + r 0 − 1 ) ( ν 0 + r 0 ν 0 ) N ( r 0 + ν 0 r 0 ) r 0 ( k N ) q = 1 ∏ Q { ( θ q ) n q ( θ q + α q ) n q } N × { ( θ q + α q ) n q − α q } k j = 1 ∏ k ( θ q + α q ) n q − m j q ( 1 − α q ) m j q − 1 k ! ( r 0 − 1 )! 1 [ q = 1 ∏ Q { ( θ q + α q ) n q − α q } k j = 1 ∏ k ( θ q + α q ) n q − m j q ( 1 − α q ) m j q − 1 ] × ( r 0 + ν 0 r 0 ) r 0 N ≥ k ∑ ( N − k )! ( N + r 0 − 1 )! ( ν 0 + r 0 ν 0 ) N q = 1 ∏ Q { ( θ q ) n q ( θ q + α q ) n q } N k ! ( r 0 − 1 )! 1 [ q = 1 ∏ Q { ( θ q + α q ) n q − α q } k j = 1 ∏ k ( θ q + α q ) n q − m j q ( 1 − α q ) m j q − 1 ] × ( r 0 + ν 0 r 0 ) r 0 N ≥ 0 ∑ N ! ( N + k + r 0 − 1 )! ( ν 0 + r 0 ν 0 ) N + k q = 1 ∏ Q { ( θ q ) n q ( θ q + α q ) n q } N + k .
Since − α q > 0 -\alpha_q>0 − α q > 0 and α q + θ q > 0 \alpha_q+\theta_q>0 α q + θ q > 0 , we have 0 < ν 0 ν 0 + r 0 ∏ q = 1 Q ( θ q + α q ) n q ( θ q ) n q < 1 0<\frac{\nu_0}{\nu_0+r_0}\prod_{q=1}^{Q}\frac{(\theta_q+\alpha_q)_{n_q}}{(\theta_q)_{n_q}}<1 0 < ν 0 + r 0 ν 0 ∏ q = 1 Q ( θ q ) n q ( θ q + α q ) n q < 1 , thus we recognize the probability mass function of a negative binomial and the latter equation simplifies to
π n ( m ; r 0 , ν 0 , α , θ ) = ∏ q = 1 Q { − α q ( θ q + α q ) n q } k ∏ j = 1 k ( θ q + α q ) n q − m j q ( 1 − α q ) m j q − 1 [ ( k + r 0 − 1 ) ! k ! ( r 0 − 1 ) ! ] × ( r 0 r 0 + ν 0 ) r 0 ( ν 0 ν 0 + r 0 ) k ∏ q = 1 Q { ( θ q + α q ) n q ( θ q ) n q } k { 1 − ν 0 ν 0 + r 0 ∏ q = 1 Q ( θ q + α q ) n q ( θ q ) n q } − k − r 0 . \begin{aligned}
\pi_n(m;r_0,\nu_0,\boldsymbol{\alpha},\boldsymbol{\theta})={}&\prod_{q=1}^{Q}\left\{\frac{-\alpha_q}{(\theta_q+\alpha_q)_{n_q}}\right\}^{k}\prod_{j=1}^{k}(\theta_q+\alpha_q)_{n_q-m_{jq}}(1-\alpha_q)_{m_{jq}-1}\left[\frac{(k+r_0-1)!}{k!(r_0-1)!}\right]\\
&\times\left(\frac{r_0}{r_0+\nu_0}\right)^{r_0}\left(\frac{\nu_0}{\nu_0+r_0}\right)^{k}\prod_{q=1}^{Q}\left\{\frac{(\theta_q+\alpha_q)_{n_q}}{(\theta_q)_{n_q}}\right\}^{k}\left\{1-\frac{\nu_0}{\nu_0+r_0}\prod_{q=1}^{Q}\frac{(\theta_q+\alpha_q)_{n_q}}{(\theta_q)_{n_q}}\right\}^{-k-r_0}.
\end{aligned} π n ( m ; r 0 , ν 0 , α , θ ) = q = 1 ∏ Q { ( θ q + α q ) n q − α q } k j = 1 ∏ k ( θ q + α q ) n q − m j q ( 1 − α q ) m j q − 1 [ k ! ( r 0 − 1 )! ( k + r 0 − 1 )! ] × ( r 0 + ν 0 r 0 ) r 0 ( ν 0 + r 0 ν 0 ) k q = 1 ∏ Q { ( θ q ) n q ( θ q + α q ) n q } k { 1 − ν 0 + r 0 ν 0 q = 1 ∏ Q ( θ q ) n q ( θ q + α q ) n q } − k − r 0 .
Additional results for simulation study
Additional experiments
To assess the performance of the MOSAIC model in predicting K q , s q ( n q ) K_{q,s_q}^{(n_q)} K q , s q ( n q ) as a function of the number of sites, we simulate data under the same settings as scenario A in Section 4 of the main paper, considering different values for Q Q Q , namely Q ∈ { 5 , 8 , 10 , 15 , 20 , 25 , 30 } Q \in\{5, 8, 10, 15, 20, 25, 30\} Q ∈ { 5 , 8 , 10 , 15 , 20 , 25 , 30 } . Figure S1 shows the predictive performance compared with the exchangeable negative binomial mixture of beta Bernoulli. As Q Q Q increases, the improved predictive performance of MOSAIC relative to analyzing data from each site separately becomes increasingly apparent.
Figure S1. Average MSE in predicting K q , s q ( n q ) K_{q,s_q}^{(n_q)} K q , s q ( n q ) , averaged over l = 1 , … , s q l=1,\ldots,s_q l = 1 , … , s q , with s q = 10 s_q=10 s q = 10 , for the MOSAIC model and the exchangeable negative binomial mixture of beta-Bernoulli models (BB) across different values of Q Q Q .
To assess whether Bayesian posterior distributions are well calibrated, we evaluate the frequentist coverage of posterior credible intervals for the regression coefficients ζ \zeta ζ and dispersion parameters ϕ \phi ϕ . These parameters are directly interpretable in the MOSAIC framework - ζ \zeta ζ governs the mean structure, while ϕ \phi ϕ controls site-level dispersion - making reliable uncertainty quantification essential. We simulate the data under the MOSAIC model using the same parameter settings as in the main paper with Q = 15 Q=15 Q = 15 sites, but consider two different values for the mean of the negative binomial distribution from which N N N is drawn, namely 3000 and 5000. We generated 500 datasets and for each replicate compute 95% posterior credible intervals for each component of ζ \zeta ζ and ϕ \phi ϕ . Coverage is then estimated as the proportion of times the true parameter values are contained within these intervals. The results, reported in Table S1 , show coverage close to the nominal level for both parameters, with moderate variability across components.
Table S1. Coverage probabilities for ζ \zeta ζ and ϕ \phi ϕ under two values of ν 0 \nu_0 ν 0 .
######### Additional plots
Additional details and results on fungal biodiversity application
Additional details on fungi data
Figure S3. Map of sampling locations for the fungi data analysis colored by climatic zones.
Table S2. Labels of sampling locations and sample sizes for the fungi data analysis.
######### Additional results on fungal biodiversity application
We provide rarefaction curves for the latent total number of shared species U n q , n r U_{n_q,n_r} U n q , n r for pairs of sites with the same sample sizes. Figure S5 shows results for three pairs under three values of σ ∈ { 0.1 , 0.4 , 0.95 } \sigma\in\{0.1, 0.4, 0.95\} σ ∈ { 0.1 , 0.4 , 0.95 } for all sites. The results are analogous to those for site-specific rarefaction curves in the main paper: lower detectability yields notably higher expected shared species counts.
Figure S5. Posterior mean and 95% credible intervals for simplified rarefaction curves for the latent total number of shared species U n q , n r U_{n_q,n_r} U n q , n r for different values of σ \sigma σ .
Figure S4. Posterior mean of proposed β \beta β -similarity for all sites grouped by climatic zones for different values of σ q = σ ∈ { 0.3 , 0.6 , 0.9 } \sigma_q = \sigma\in\{0.3, 0.6, 0.9\} σ q = σ ∈ { 0.3 , 0.6 , 0.9 } for all q = 1 , … , 34 q = 1, \ldots, 34 q = 1 , … , 34 .
######### Additional experiments for fungal biodiversity application
We perform additional experiments to evaluate the out of sample predictive performance of the MOSAIC model for the number of shared species on the fungi data. We split the data into training and test sets, using 70% of the observations in each site for training and the remaining for test, so that each training set size is proportional to n q n_q n q . We fit MOSAIC using the same hyperparameters as in the main paper on the training data. Based on the model estimates, we then predict the number of shared species for pairs of sites in the test set, that is, C s q , s r ( n q , n r ) C_{s_q,s_r}^{(n_q,n_r)} C s q , s r ( n q , n r ) . Figure S6 displays the rarefaction surfaces for shared species on the test set along with the corresponding model-based predictions for three illustrative pairs of sites: a pair of temperate locations Hainich (HAI) - Santa Cruz (SAN); a pair of polar-continental locations Viikki (VII) - Terney (YAK); and a cross-zone pair Waldhäuser (BAV) - Fetsund (FTS). The test set sizes, summarized in Table S3 , vary across sites in proportion to their sample sizes, allowing evaluation of predictive performance across a range of sampling efforts. Viikki, a district in northeastern Helsinki, and Terney, a remote coastal site in the Sikhote-Alin region of the Russian Far East, are the pair with the fewest shared species. While both sites lie at similar latitudes, they differ markedly in vegetation and degree of human influence: Terney is characterized by mixed forests dominated by Korean pine and Mongolian oak under a climate influenced by the Sea of Japan, whereas Viikki is located within the urban research campus of the University of Helsinki. In contrast, the two European locations - Waldhäuser and Fetsund - share the highest number of species in common. The posterior mean closely follows the empirical surfaces for all pairs, indicating strong agreement between observed and predicted values.
Table S3. Training (n q n_q n q ) and test (s q s_q s q ) sample sizes for each site.
Figure S6. Empirical rarefaction surfaces for shared species in the test set, together with the corresponding posterior mean estimates of C s q , s r ( n q , n r ) C_{s_q,s_r}^{(n_q,n_r)} C s q , s r ( n q , n r ) .