betaFeedback

WorksPhysics and Astronomy

VIDE 2.0: Robust, Scalable Void Finding for Extremely Large Galaxy Surveys

Abstract

Current and future spectroscopic and photometric galaxy surveys will deliver unprecedented numbers of galaxies, with expected counts in the tens of millions. To answer the computational challenges posed by these surveys, in this paper we present VIDE 2.0, a significantly improved and enhanced void finding algorithm and analysis pipeline based on VIDE. Leveraging the parameter-free identification of voids via a Voronoi tessellation and watershed scheme, we implement a robust mechanism for handling survey boundaries and incompleteness without relying on vast numbers of boundary particles. This new method identifies edge galaxies and flags voids contaminated by their presence. The mechanism is fully deterministic, producing bit-for-bit reproducible catalogs from identical inputs. An automatic rule for setting the mask resolution renders the entire pipeline parameter-free. Combined with other performance enhancements to the underlying VIDE engine, we find significant (>100×>100\times) speedups and a dramatic reduction in memory usage compared to previous versions: a 12-million-galaxy DESI Y1-class mock catalog now completes in roughly 15 minutes on a single laptop, where VIDE 1.0 could not run a catalog of even 10610^6 galaxies on the same hardware. We are also able to identify voids existing closer to survey boundaries via deterministic per-void contamination flags, maximizing their statistical potential. We exercise the upgraded pipeline on two large mock suites (MultiDark-Patchy DR12 and AbacusSummit DESI Y1 ELG), demonstrating converged cosmic-variance covariances for the void number function, the void-galaxy cross-correlation, and the void ellipticity distribution – key ingredients for Stage IV void cosmology. VIDE 2.0 is now well suited to handle large numbers of galaxies in mock and real observations, and ready to tackle the challenges of future campaigns.

Introduction

The study of cosmic voids, the vast underdense regions that dominate the volume of the universe [59], has grown into a broad field encompassing many robust cosmological probes. Owing to their relatively simple evolution and pristine contents [53, 49, 64], voids provide complementary constraints on cosmological parameter space [9], a window into the evolution of low-mass galaxies [50], an opportunity to independently constrain neutrino mass [40, 60, 7], a pathway to testing gravity free from cluster systematics [66, 67, 65], as well as a laboratory for investigating galaxy bias [42] and alignment (W. D. D. et al. 2022), among many other strengths. Besides raw number counts [44, 35], many void properties are rich with information, such as their density profiles [25, 54, 14], the Alcock-Paczynski test [36], redshift space distortions [22], spin distributions [32], and cross-correlations with gravitational lensing shear [29, 37] and gamma-ray sources [18].

In P. M. Sutter et al. [55] we introduced a publicly available void catalog, drawn from the SDSS DR7 main sample [1]. Since then, additional public catalogs (e.g., Q. Mao et al. [39]; K. A. Douglass et al. [17]) have continued to enrich and enliven the void ecosystem. These catalogs have been analyzed in multiple subsequent studies, for example for cosmological constraints from measurements of the Alcock–Paczyński effect [57, 24] and the void size distributions [12, 51], with galaxy-void cross-correlations leading to precision constraints on the expansion geometry and growth of structure [23].

The first data release of DESI (DR1) [11] has inaugurated the era of Stage IV cosmological surveys, which will include observing campaigns with the Euclid space telescope [41], the SPHEREx mission [8], the Nancy Grace Roman Space Telescope (J. E. Schlieder et al. 2024), and the Chinese Space Station Survey [19]. These massive surveys will unlock unprecedented numbers of galaxies, typically tens of millions per survey. Mock observations and theoretical modeling suggest that voids in these surveys will provide extraordinary constraints through void number functions [13, 61], the void power spectrum [52], the void-lensing cross-correlation [68], the void-galaxy cross-correlation [26], and more.

To achieve these predicted results, void finders must provide reliable, well-understood void populations and statistics that allow robust tracing from void properties identified in galaxy surveys to the underlying dark matter distribution, and further to the fundamental cosmological parameters. Additionally, any void finder must grapple with the large volumes of data presented by existing and upcoming surveys while dealing with the vagaries of real-world data collection, including boundary mask handling and survey incompleteness.

Historically, void finders were developed for dark matter simulations (or halo or galaxy populations drawn from the dark matter), which provided tracer densities to arbitrary resolution while side-stepping observational effects (e.g., M. C. Neyrinck [43]). Meanwhile, pioneering efforts identified voids observationally with simple approximations from limited survey data (e.g., G. Kauffmann & A. P. Fairall (1991); F. Hoyle & M. S. Vogeley (2002)). In P. M. Sutter et al. [57] we introduced VIDE, the Void IDentification and Examination toolkit, a public void finder based on a heavily modified version of the ZOBOV algorithm employed in M. C. Neyrinck [43] that could handle realistic survey scenarios.

This engine, which discovered voids via a parameter-free tessellation scheme to build a density field from the given tracers, followed by a watershed transform to join adjacent regions into void basins, powered a raft of void discoveries in both simulations and observations, including many of the above references.

In addition to performance and reliability improvements, alongside extensions to handle arbitrary volumes and sparse density sampling, a key innovation behind VIDE compared to ZOBOV was its handling of survey boundaries, which involved flooding edges with vast numbers of boundary particles to constrain the tessellation. This approach preserved (as much as possible) the parameter-free spirit of void analysis, and it became the de facto method employed to produce other public catalogs (e.g., Q. Mao et al. [39]).

While this approach worked well for limited galaxy surveys, for large (e.g., Stage IV) surveys, this approach to boundary handling produces far more boundary particles than actual galaxies, requiring drastically more memory and compute for a typical tessellation; enough to lead to impractically long run-times, if not outright overwhelming typical system constraints.

This procedure introduced an element of stochasticity, as the algorithm randomly placed the boundary particles, which would lead to small catalog variations from run to run.

It also required an extra tunable parameter, the density of boundary particles, that propagated into distortions of the tessellation and thereby the resulting void catalog. Meanwhile, other void finders suitable for surveys have emerged, including approaches leveraging persistent homology [2], dynamical modeling (Óscar Monllor-Berges et al. 2025), spherical aggregation [45] and graph flows [58]. Several studies have compared and contrasted these differing algorithms (e.g., J. M. Colberg et al. [10]; D. Veyrat et al. [62]; K. A. Douglass et al. [17]; F. Zaidouni et al. [69]), which generally find broad agreement of key void properties (e.g., the location of density minima). However, some ambiguities persist, which must be resolved to fully unlock the potential of cosmic voids.

To answer the data volume challenges presented by modern and future surveys, and to provide a robust, flexible platform for analysis and comparison, in this paper we present VIDE 2.0, an upgraded and enhanced implementation of our original void-finding algorithm. This upgrade includes several performance enhancements, a new tessellation engine, and most crucially, a parameter-free, deterministic approach to boundary handling that preserves the scale-agnostic, purely morphological void definition in any survey configuration. This paper is organized as follows. In § 2 we detail the new boundary handling procedure. In § 3 we showcase the improved scalability and performance of the new scheme and the entire VIDE pipeline. In § 4 we elucidate how the new boundary handling approach impacts the underlying tessellation and resulting void catalogs (in particular emphasizing how the new void catalogs are more reliable). In § 5 we run the upgraded pipeline on two large mock suites and measure the void statistics and covariances that Stage IV void cosmology will require. We conclude in § 6 by providing general philosophies for approaching void catalogs and summarize forward-looking prospects and future work.

Parameter Free Survey Boundary Handling

The original procedure and its limitations

The primary strategy behind VIDE’s approach to void finding is to treat them as objects that are defined in a scale-agnostic way, only by the morphology of the tracer distribution. We implement this philosophy through a heavily modified version of ZOBOV (M. C. Neyrinck [43]), which begins with a tessellation step to transform a set of tracers (e.g., dark matter particles in a simulation or galaxies in a survey) into a density field. We then apply the watershed transform to that density field to segment it into a series of basins (which become the voids) separated by ridges (the void boundaries).

This tracer-agnostic approach offers many advantages, which are elucidated in more detail in P. M. Sutter et al. [57]. Primarily, VIDE is parameter-free. There is a single, optional tunable parameter that controls the threshold by which adjacent basins are merged into singular voids. In practice, adjusting this value has little impact, especially in galaxy surveys where tracer density is relatively sparse, as VIDE naturally produces a hierarchy of voids and sub-voids; adjusting the merging threshold merely changes the truncation point for the identification of low-contrast super-voids spanning large volumes.

As is usually the case, to make this ideal philosophy useful in realistic scenarios, such as in a galaxy catalog that results from a real survey, we must contend with the limitations of actual observational data. In particular, the presence of a survey mask alters the topology of the density field. It introduces artificial bounds, beyond which the density field is unknown, spatially varying radial survey completeness, and foreground objects insert punctures that must be accounted for.

Since we understand voids as basins in the density field, these effects necessarily alter the sizes, shapes, and distributions of identified voids ([56]). For example, masks preferentially eliminate large voids while preserving smaller ones. They also introduce a more subtle bias, as voids that happen to lie along the mask edge survive intact, while those perpendicular to the mask become truncated. This can impact analyses that require a fair sampling of void orientations (e.g., the Alcock–Paczyński effect).

To be fair, the same problems affect other void finding algorithms, since regardless of underlying philosophy voids are large, extended objects.

When we originally adapted VIDE for galaxy surveys, we developed a boundary handling scheme that was primarily designed to be as conservative and cautious as possible. The scheme was as follows:

  1. Convert a given survey mask to HEALPix format with a chosen NSIDE resolution.

  1. Identify the boundaries of that mask. This step is illustrated in Figure 1, and will be identical for our new implementation.

Figure 1. The first step in VIDE’s mask handling procedure, shown using a fiducial example drawn from the SDSS galaxy survey. Both panels overlay the input mask interior (gray) at its native resolution NSIDE=512\mathrm{NSIDE}=512 on the boundary contour identified by VIDE 2.0 (red) at NSIDE=128\mathrm{NSIDE}=128, the latter chosen here for presentation clarity. Top: full-sky Mollweide projection. Bottom: gnomonic zoom centered on the densest region of the survey.

  1. “Flood the zone” by placing boundary particles, uniformly at random with a given density along the radial rays defined by boundary pixel centers.

  1. Also place boundary particles uniformly at random along the maximum and minimum redshift caps of the survey volume.

  1. Proceed with the Voronoi tessellation, including the inserted boundary particles.

  1. Prior to the watershed step, remove any adjacency links to any galaxy that is itself adjacent to a boundary particle (i.e., in the local Voronoi tessellation sense), effectively preventing the voids from reaching mask edges.

In practice, this procedure was destructive: any galaxy adjacent to a boundary particle had its Voronoi volume set to effectively zero and all adjacency links severed, permanently erasing information from the density field. The resulting void catalog was thus shaped not only by the real galaxy distribution, but also by the particular random realization of boundary particles.

Since VIDE was originally developed in the context of the first void Alcock–Paczyński test, during post-processing the default void catalog produced by VIDE implemented additional filters, such as eliminating any void whose maximum extent (as defined relative to its volume-weighted center) was greater than the distance to the nearest boundary particle. This mitigated any mask-alignment biases. Note that VIDE also produced a parallel catalog of all voids, but this catalog also included voids directly adjacent to mask boundaries. These cases were valid watershed basins representing underdense regions, but with derived information, such as extent, shape, and center, that could be affected by the mask they abutted or intersected.

Although this approach worked at the time and, indeed, delivered reliable Alcock–Paczynski results and a bounty of other void information and follow-up studies, it suffers from three drawbacks.

The first is one of stochasticity. Since we place the boundary particles randomly, different runs of VIDE on the same dataset can produce slightly different void catalogs, as the underlying tessellation (i.e., topology) changes for each realization of boundary particles. Because of this, small pinholes in the mask may not get a sufficient density of boundary particles, leading VIDE to believe that these regions are fairly sampled when in reality they are not.

Require: Galaxy positions {gi}\{g_i\}, survey mask MM at resolution NsideN_{\mathrm{side}}, redshift bounds [zmin⁡,zmax⁡][z_{\min},z_{\max}]

Ensure: Set of flagged edge galaxies F\mathcal{F}

  1. Run Voronoi tessellation on {gi}\{g_i\} using VORO++

  1. Build KD-tree T\mathcal{T} from {gi}\{g_i\}

  1. Identify boundary pixels {pj}\{p_j\} from mask MM

  1. Compute minimum linear resolution δ\delta from NsideN_{\mathrm{side}} at the survey’s midpoint distance

  1. for each boundary pixel pjp_j do

  1. Define ray rjr_j along pixel center from zmin⁡z_{\min} to zmax⁡z_{\max}

  1. Divide rjr_j into redshift bins; refine angular sampling via HEALPix nesting to maintain spacing ≤δ\leq\delta

  1. for each sample point ss along rjr_j and any refined sub-rays do

  1. Query T\mathcal{T} for nearest galaxy gkg_k to ss

  1. F←F∪{gk}\mathcal{F} \leftarrow\mathcal{F} \cup\{g_k\}

  1. end for

  1. end for

  1. Sample low-zz and high-zz caps at spacing δ\delta; flag nearest galaxies

  1. Set Voronoi volumes of all g∈Fg \in\mathcal{F} to ∼0\sim0

  1. Proceed with watershed transform

Algorithm 1. VIDE 2.0 Edge Galaxy Flagging

The second is the introduction of another artificial parameter, the density (or equivalently nearest-neighbor distance) of boundary particles, which violates the spirit of attempting to keep the void finding approach flexible, universal, and as agnostic to the introduction of additional scales as possible.

Lastly, and most severely, large surveys covering substantial cosmological volumes required an enormous number of boundary particles to adequately close the survey domain. In many cases, these particles vastly outnumbered the galaxies in the survey proper, leading to a significant loss of scalability, performance, and ease of use.

For these reasons, for VIDE 2.0 we targeted the boundary handling step as the critical component for updating.

Edge-galaxy flagging in vide 2.0

Our new procedure, designed to alleviate these issues, is summarized in Algorithm 1.

As a first step, we replaced the underlying tessellation engine, the QHULL library (C. B. Barber et al. [6]), with VORO++ (C. H. Rycroft [47]). In addition to significant improvements in performance, scalability, and portability (to be demonstrated below), VORO++ provides a clearer relationship between the point set to be tessellated and the bounding volume containing those points. In particular, with QHULL if a particle is near a boundary its tessellation is ambiguous. In contrast, VORO++ explicitly includes the boundaries as labeled faces of a given particle’s Voronoi tessellation, if that boundary intersects its local volume. While this does not remove the ambiguity of the volumes associated with galaxies near the edge of the survey region, it provides a zero-cost, exact geometric determination of which galaxies have Voronoi cells clipped by the domain boundary — information that was fundamentally unavailable in the previous scheme, where edge proximity could only be inferred statistically from the random boundary particle distribution.

As for the angular survey bounds, in contrast to VIDE 1.0’s approach of flooding mask edges with boundary particles, for VIDE 2.0 we adopt an approach based on identification of galaxies near the edges, flagging them, and modifying the tessellation. This new approach maintains the spirit of treating voids purely as morphological entities, and the impact of the mask as necessarily morphological in nature as well.

This means that the user must clearly specify the survey mask in a way that VIDE can understand. To do this, in addition to the raw mask file the user must choose NSIDE, the desired HEALPix resolution for the internal representation of the mask within VIDE. We describe VIDE 2.0’s automatic rule for selecting NSIDE — which eliminates it as a free parameter — in § 2.3 below. For the present discussion we take it as given.

Once the mask properties are determined, the next step for VIDE 2.0 is to simply run the tessellation as-is. This “raw” tessellation contains mostly useful density field calculations (for regions far away from any survey boundaries) but also many pathological cases: Voronoi volumes that jump across gaps in the survey, volumes that connect to the cubic bounding box containing the survey volume, and so on.

To identify these pathological cases we drive rays down the pixel centers defining the mask edges. We implement a memory-efficient linear search to find all the nearest neighbor galaxies to any point on the line, down to a resolution threshold determined by the comoving linear separation of HEALPix pixels at the midpoint of the survey’s comoving distance range.

Figure 2 shows a cartoon representation of this nearest-neighbor searching. This approach of flagging along rays works for both pinholes (as depicted) and survey edges.

Cartoon depicting flagging technique

Figure 2. Cartoon depicting flagging technique. Shown is a Voronoi tessellation of a 2D grid of random points. The observer is at the lower left corner. A pinhole in the mask punches a radial ray through the survey, represented here as the green line. Galaxies “nearest” to this ray are flagged in red. We define “nearest” to mean any particle whose local Voronoi volume is intersected by the ray. The flagged galaxies have unreliable volumes and densities due to the presence of the pinhole.

At higher redshifts these rays can separate, allowing Voronoi volumes to potentially “spill” outside the survey bounds. To account for this we divide the survey volume into redshift bins and add higher-resolution rays using the nested refinement capabilities of HEALPix. We choose the nesting such that we maintain a consistent

linear separation between rays defined by NSIDE at the survey’s midpoint distance throughout its entire volume.

Finally, we sample a discrete set of points at the low- and high-zz redshift ends of the survey at the appropriate spacing so as to maintain the minimum linear resolution determined by NSIDE.

The net result of this procedure is equivalent to discovering which galaxies would have Voronoi volumes intersected by the outermost edges and internal holes of the survey, and whose volumes would “jump” across disparate regions of the survey. This makes their local density estimate unreliable and hence unusable for the watershed transform. Flagging complete, we then take any edge galaxy and set their Voronoi volumes to be (essentially) zero. When the watershed transform proceeds to find basins and join them into voids, these edge-galaxies act as infinitely-high walls that prevent the watershed from spilling outside the survey bounds.

Importantly, VIDE 2.0 distinguishes among three independent sources of edge contamination, each recorded with a distinct flag value: (1) galaxies whose Voronoi cells are geometrically clipped by the tessellation domain boundary, identified directly by VORO++†; (2) galaxies nearest the low- and high-redshift caps of the survey volume; and (3) galaxies nearest the angular mask edges, found via the ray-based search. This per-galaxy provenance information is stored in the output catalogs, enabling users to selectively filter based on the source of contamination — for example, retaining voids near redshift boundaries while excluding those affected by angular mask holes.

Automatic Selection of NSIDE Resolution

Up to this point we have taken the HEALPix resolution NSIDEN_{\mathrm{SIDE}} at which the mask is processed as given. In VIDE 1.0 this was a free parameter the user had to tune by hand, and in practice the choice was nonobvious. Two failure modes constrain it. Mode A is under-resolution: when the mask is coarser than the typical galaxy separation, real survey edges are smoothed over, some edge-adjacent galaxies escape flagging, and the clean and central catalogs are inflated by spurious voids that should have been flagged. Mode B is the opposite: when the mask is much finer than the data and the mask carries non-trivial internal topology (small interior pinholes, jagged outer edges, or both), each finer subdivision of NSIDEN_{\mathrm{SIDE}} resolves smaller mask features that drive volume-piercing rays through the survey interior, the watershed is over-segmented, and the clean and central populations collapse rapidly. Both modes are visible in Figure 3.

Figure 3. Number of voids identified by VIDE 2.0 as a function of HEALPix NSIDE, separated into the two trustworthy catalog tiers: clean voids (no edge-flagged member galaxies) and central voids (no edge-flagged members and macrocenter farther from any survey edge than the void’s maximum extent). Left: SDSS DR7, whose mask has native resolution NSIDE = 512. Right: a CMASS-like mock lightcone on the BOSS DR12 CMASS NGC footprint, whose mask has native resolution NSIDE = 128. Vertical dashed line: NSIDE chosen by the auto rule of Equation (3); vertical dotted line: input mask’s native NSIDE when distinct from the auto value. The shaded red band on the left panel is the Mode B regime where pinhole-piercing on the complex DR7 mask collapses both catalog tiers by roughly two decades.

The empirical safe zone lies between these two failure modes, bounded above by the mask’s intrinsic resolution and below by the data’s density. We make this concrete in two steps. The mask side is straightforward: read NSIDE,maskN_{\mathrm{SIDE,mask}} directly from the input HEALPix file. For the data side, we ask at what NSIDEN_{\mathrm{SIDE}} the volume-weighted mean linear pixel resolution matches the mean tracer separation. For a survey with comoving distance DC(z)D_{\mathrm{C}}(z) between bounds [zmin⁡,zmax⁡][z_{\min}, z_{\max}], this volume-weighted mean is

⟨rpix⟩V=θpix34DC4(zmax⁡)−DC4(zmin⁡)DC3(zmax⁡)−DC3(zmin⁡),(1)\langle r_{\mathrm{pix}}\rangle_{\mathrm{V}} = \theta_{\mathrm{pix}} \frac{3}{4} \frac{D_{\mathrm{C}}^{4}(z_{\max}) - D_{\mathrm{C}}^{4}(z_{\min})}{D_{\mathrm{C}}^{3}(z_{\max}) - D_{\mathrm{C}}^{3}(z_{\min})}, \tag*{(1)}

where θpix=4π/Npix\theta_{\mathrm{pix}} = \sqrt{4\pi/N_{\mathrm{pix}}} is the characteristic HEALPix pixel angle and Npix=12NSIDE2N_{\mathrm{pix}} = 12N_{\mathrm{SIDE}}^{2}. Setting ⟨rpix⟩V\langle r_{\mathrm{pix}}\rangle_{\mathrm{V}} equal to the mean tracer separation sˉ=(Vsurvey/Ntracers)1/3\bar{s} = (V_{\mathrm{survey}}/N_{\mathrm{tracers}})^{1/3} and rounding up to the nearest power of two yields the data-density-matched value

NSIDE,data=2⌈log⁡2(π/3/θ⋆)⌉,θ⋆=4sˉ3DC3(zmax⁡)−DC3(zmin⁡)DC4(zmax⁡)−DC4(zmin⁡).(2)\begin{aligned} N_{\mathrm{SIDE,data}} &= 2^{\left\lceil\log_{2}\left(\sqrt{\pi/3}/\theta^{\star}\right)\right\rceil}, \\ \theta^{\star} &= \frac{4\bar{s}}{3}\frac{D_{\mathrm{C}}^{3}(z_{\max}) - D_{\mathrm{C}}^{3}(z_{\min})}{D_{\mathrm{C}}^{4}(z_{\max}) - D_{\mathrm{C}}^{4}(z_{\min})}. \tag*{(2)} \end{aligned}

The auto rule is then simply

NSIDE,auto=min⁡(NSIDE,mask,NSIDE,data).(3)N_{\mathrm{SIDE,auto}} = \min(N_{\mathrm{SIDE,mask}}, N_{\mathrm{SIDE,data}}). \tag*{(3)}

Both terms are reported at run time so the user can confirm or override the selection. We use the volume-weighted form in Equation (1), rather than evaluating at the median redshift, because a median-zz choice would under-represent the larger-volume regions at high redshift where pixel sizes are largest and the danger of under-resolution is greatest. The volume weighting is the natural average for an observable — the void number function — that integrates over the whole volume.

For the SDSS DR7 example used throughout the present paper, the auto rule selects NSIDE,auto=64N_{\mathrm{SIDE,auto}} = 64 (NSIDE,mask=512N_{\mathrm{SIDE,mask}} = 512, NSIDE,data=64N_{\mathrm{SIDE,data}} = 64); the data side wins because the volume-weighted pixel resolution at NSIDE=64N_{\mathrm{SIDE}} = 64 already matches the mean tracer separation of 7.4 h−1 Mpc7.4\,h^{-1}\,\mathrm{Mpc}. With this rule in place, the only remaining input VIDE 2.0 requires from the user is the survey mask itself.

Demonstration on SDSS DR7

Throughout this paper we exercise VIDE on three datasets: the SDSS DR7 sample packaged with VIDE as an exemplar, used in this section, plus two Stage III/IV-scale mock catalogs introduced for the demonstrations of § 5. Their properties are summarized in Table 1. We chose DR7 as the boundary-handling exemplar because, while it is neither modern nor large by current standards, it has a particularly complicated mask — many internal holes and jagged outer edges — making it the ideal test case for the new procedures developed above, as well as small enough to easily visualize for demonstration purposes.

PropertyAbacusSummit DESI Y1 ELGMultiDark-Patchy DR12 NGC
SimulationAbacusSummitMultiDark Planck
Galaxy sampleDESI Y1 ELGBOSS LOWZ+CMASS LRG
FootprintDESI Y1 NGC+SGCBOSS DR12 NGC
Redshift range0.8≤z≤1.60.8 \le z \le1.60.2≤z≤0.750.2 \le z \le0.75
Sky fraction0.220.22 (∼9,100 deg2\sim9{,}100\ \mathrm{deg}^{2})0.180.18 (∼7,400 deg2\sim7{,}400\ \mathrm{deg}^{2})
Galaxies per realization1.21×1071.21 \times10^{7}9.2×1059.2 \times10^{5}
Mean tracer separation∼12h−1 Mpc\sim12 h^{-1}\ \mathrm{Mpc}∼17h−1 Mpc\sim17 h^{-1}\ \mathrm{Mpc}
Realizations used25200

Table 1. Mock suites used for the demonstrations of § 5. In the text we refer to them as the Abacus ELG and Patchy mocks.

Figure 4 highlights the differences between the VIDE 1.0 and VIDE 2.0 approaches to boundary handling in a slice through the DR7 volume-limited subsample at z=0z = 0–0.150.15.

Figure 4. Comparison between old and new survey boundary handling procedures. Shown is a slice taken from an example survey (in particular, the SDSS DR7 dataset packaged as an exemplar with VIDE). We deliberately have taken a slice through the survey to showcase both boundary edges and internal holes. Galaxy positions are converted to h−1 Mpch^{-1}\,\mathrm{Mpc} assuming a fiducial Λ\LambdaCDM cosmology. Left: VIDE 1.0 scheme, with galaxies nearest the slicing plane shown as black points, inserted boundary particles as yellow points, and galaxies adjacent to boundary particles (and hence removed from the watershed step) as red points. Right: VIDE 2.0 scheme, with galaxies nearest the slicing plane shown as black points, and flagged galaxies nearest to boundaries as red points.

In this example case, of the 164,647 original galaxies in the sample, VIDE flagged 42,017, or roughly 25% of them, as edge galaxies. This large fraction is due to the relative sparsity of the surveyed galaxies and the aforementioned complexity of the mask. This is roughly equivalent to the number of edge galaxies flagged in VIDE 1.0.

Performance & Scaling

The benefits to this new approach are manifest. For one, it is extremely lightweight. With no additional particles added, there is no additional cost to running VIDE with larger surveys, aside from the expected cost of dealing with greater numbers of galaxies.

Secondly, this method is fully deterministic. VIDE 1.0 had two sources of stochasticity: the random placement of boundary particles to flood the survey edges, and the variable sampling of pinholes when the mask resolution was insufficient. Two consecutive VIDE 1.0 runs on the same input therefore produced slightly different catalogs, with run-to-run scatter in core densities, density contrasts, and membership for voids near the survey edges. VIDE 2.0 places no random particles, and given identical mask, NSIDE, and tracer positions it identifies the same edge galaxies (Algorithm 1) before passing them to the (deterministic) voro++ tessellator. The pipeline is therefore reproducible at the bit level: re-running on the same input yields SHA-256 identical catalog files, and the single-domain and KD-tree-split configurations agree bit-for-bit at the recommended buffer width.

Combined with the auto-NSIDE rule of § 2.3, the boundary-handling step requires no further tuning: the user supplies only a tracer catalog and a mask.

Lastly, the new approach is much faster and exhibits better scaling behavior than the mask handling scheme in VIDE 1.0. The additional computational cost comes from the need to, essentially, cross-correlate every boundary pixel with every galaxy. However, we achieve significant speedups by placing the galaxies on a KD-tree, which makes the nearest-neighbor search at each reference point along a ray an O(log⁡n)\mathcal{O}(\log n) operation. We tested several schemes for searching along the ray, including a bisection search, but found that a simple linear stepping provided the most robust, memory- and compute-efficient approach. The bisection search remains available via a user-selectable option should it prove useful in other scenarios.

While the search for edge-adjacent galaxies does incur an additional computational cost, it relieves VIDE from having to consider a substantial number of boundary particles in the tessellation, which dominate the computational cost, and the watershed transform. In some cases, this made VIDE 1.0 outright unusable, with so many particles required that even memory-generous systems were overwhelmed.

Additionally, voro++ is much more lightweight than QHULL. Even though the latter is more widely used, it is much more general, meant for tessellations in any number of dimensions. In our experience, voro++ has proven to be roughly 30–50% faster and require significantly less memory in the Voronoi tessellation step.

To test the scaling behavior we ran the full pipeline on mock0, one realization of the AbacusSummit DESI Y1 ELG_LOP_complete mocks (the Abacus ELG mocks of Table 1). Mock0 contains 1.21×1071.21 \times10^{7} galaxies, and we also ran seven uniformly random subsamples of it down to 1.25×1051.25 \times10^{5} tracers. We left every parameter at its default so that the curves reflect what a user gets without any tuning: a KD-tree subdomain split with two domains, automatic thread allocation, and the auto-nside rule, which selected nside = 64, 128, and 256 for the small, intermediate, and large samples respectively. All runs used eight processes on an Apple M2 MacBook Air with 8 GB of memory.

Figure 5 shows the result. Wall time and peak memory both scale near-linearly in NN across two orders of magnitude in catalog size. The Voronoi tessellation dominates the wall clock budget at every size; edge flagging is the next-largest contributor, with visible bumps each time the auto-nside rule increases the mask resolution. The watershed and pruning steps together account for only about 2% of the wall time. jozov2 tracks an asymptotic memory footprint of ∼160\sim160 B per tracer, with the full process tree at ∼250\sim250 B per tracer. Tessellation aside, all components approach O(n)O(n) scaling at large NN once the KD-tree of galaxy positions is in place.

Scaling performance of VIDE 2.0

Figure 5. Scaling performance of VIDE 2.0 on the Abacus ELG mock0 catalog and uniform random subsamples of it, on an Apple M2 MacBook Air (8 GB). Left: wall time per pipeline stage versus tracer count. The dotted line marks O(n)\mathcal{O}(n) scaling. Right: peak resident memory of the full process tree and of each pipeline binary; the dotted line is the asymptotic 160 B per tracer set by the watershed step.

For comparison: VIDE 1.0 required roughly 10 minutes to process the SDSS DR7 sample of ∼1.6×105\sim1.6 \times10^{5} galaxies, while VIDE 2.0 runs the same problem in a handful of seconds. At the other end, the full 1.21×1071.21 \times10^{7}-galaxy Abacus ELG mock — a catalog two orders of magnitude larger than DR7 — completes in ∼15\sim15 minutes and peaks at ∼2.2\sim2.2 GiB on the same laptop. VIDE 1.0 could not run a catalog of even 10610^{6} galaxies on this hardware: it crashed from memory exhaustion after ∼30\sim30 minutes, well before the watershed step. CMASS-class samples (∼106\sim10^{6} galaxies) and DESI Y1 ELG-class samples (∼107\sim10^{7}) are therefore inaccessible to VIDE 1.0 on a laptop and tractable with VIDE 2.0 in minutes.

We also made several other changes as part of the upgrade.

Per-tracer weights.—VIDE 2.0 accepts an optional weights file as a sample-level input. After the Voronoi tessellation returns a per-tracer cell volume ViV_i, the watershed input volume is set to Viws=wi−1ViV_i^{\mathrm{ws}} = w_i^{-1}V_i. The mechanism is decoupled from edge flagging — edge-flagged tracers override the weight by forcing ViwsV_i^{\mathrm{ws}} to a near-zero wall value, so weights cannot interact with boundary handling — and an all-ones weights file produces a bit-identical catalog to an unweighted run.

Auto-resolved defaults.—In addition to the auto-nside rule of § 2.3, VIDE 2.0 auto-resolves the subdomain decomposition mode (a KD-tree split for observational data, a regular cubic split for simulations) and the per-subdomain buffer width (0.30 for the KD-tree mode, 0.10 for the regular grid). With two domains, the KD-tree split delivers a ∼3×\sim3\times speedup on the tessellation while reproducing the single-domain catalog bit-for-bit. Each auto-resolved choice is written to the run log and can be overridden explicitly.

Build modernization.—The original VIDE build relied on a hand-rolled setup.py that drove CMake through a custom build extension and rebuilt every internal C/C++ dependency at install time. On Python ≥3.12\ge3.12 that path is broken, since distutils has been removed from the standard library. VIDE 2.0 is built with scikit-build-core via pyproject.toml; we removed Boost, NetCDF, Eigen, and libSDF as build-time dependencies, copied the seven CosmoTool headers that VIDE actually uses directly into its source tree, and defaulted the remaining external libraries (HEALPix, cfitsio, GSL, voro++, HDF5) to system packages. A clean pip install vide now takes roughly two minutes on an Apple Silicon laptop with Homebrew.

Other improvements.—Internal pipeline I/O has moved from NetCDF to HDF5, giving roughly an order-of-magnitude speedup on the data-preparation stage. The per-tracer edge-flag field now carries a small integer recording which of three independent boundary-detection mechanisms (voro++’s native wall detection, the redshift-cap ray-march, the angular-mask ray-march) flagged each tracer, allowing downstream users to retain or exclude voids by edge type. A lightweight process-tree poller records per-stage peak resident memory in the run log alongside wall-clock timing, and a pytest-based regression harness compares end-to-end pipeline outputs against locked hash manifests on every commit.

Impacts on Void Catalogs

We first developed VIDE to apply watershed techniques to real galaxy surveys and to identify voids in a consistent, coherent fashion at scale in observations. For this reason, we took an exceptionally cautious approach to boundary handling, wishing only to include voids that were “far” from any survey edges, redshift boundaries, and internal holes.

The choices proved prudent, but now with over a decade of experience with voids in surveys, we can afford to take a more precise approach — we can get closer to the edge and still understand how the watershed algorithm is behaving and the kinds of voids it produces.

The combination of Voronoi tessellation and watershed algorithm explores the complex, irregular bounding surface of the voids, up to the ridge-lines defined by the spine of the cosmic web. Figure 6 shows a typical large void identified with VIDE 2.0. Voids this large were usually discarded by the conservative central cut of VIDE 1.0: in our SDSS DR7 sample the VIDE 2.0 clean catalog contains four times as many voids with Reff>25 h−1 MpcR_{\mathrm{eff}} > 25\ h^{-1}\ \mathrm{Mpc} as the VIDE 1.0 central catalog (32 versus 8; Figure 8).

Figure 6. A typical large void identified with VIDE 2.0, with the cosmic web providing the bounding volume for the complex shape of the interior void basin.

Figure 8. Comparison of void number functions between VIDE 1.0 and VIDE 2.0 catalogs in the SDSS DR7 galaxy sample. Catalogs are the central voids, which are determined to be as far away from the boundaries as possible and minimally biased for Alcock–Paczyński analysis, along with the full clean set from VIDE 2.0. The new boundary handling procedure eliminates the appearance of small, spurious voids near the boundaries, and allows for the persistence of large voids.

Indeed, we find significant overlap between void populations found with VIDE 1.0 and VIDE 2.0, especially in the central regions of the survey, far from any edges and holes, as shown in Figure 7. In this figure, we compare the central catalog of voids in VIDE 1.0 (which excludes any voids whose maximum extent from the macrocenter is farther than the nearest boundary particle) to the clean catalog of voids in VIDE 2.0, which excludes any void that explicitly contains a flagged edge galaxy as a member. Unlike the geometric heuristic employed in VIDE 1.0, which compared a void’s maximum extent to the distance to the nearest stochastically-placed boundary particle, this classification is exact: a void is contaminated if and only if it contains an edge galaxy, with no dependence on random realizations. This precision allows VIDE 2.0 to retain significantly more voids near survey boundaries while maintaining rigorous contamination control.

Figure 7. Comparison between VIDE 1.0 central voids (left) with VIDE 2.0 clean voids (right) for our fiducial test case. Inserted boundary particles and the edges of their Voronoi cells are shown in yellow, edge galaxies and their Voronoi cells in red, and regular galaxies in black. Note that this is a map of galaxies nearest the slicing plane, but for clarity we only display galaxies if they happen to lie within their corresponding Voronoi cell on this plane. In other words, this is not the Voronoi diagram of a 2D distribution of points, but a slice through the full 3D distribution. We overlay on this slice the identified voids, where we shade the Voronoi cells and member galaxies with random colors. The new boundary handling approach preserves voids in the central region of the survey, while preventing the formation of spurious small voids while also permitting the survival of large ones.

As efficient as the new mask handling procedure is, it does change the overall topology of a given survey volume. In a sense, VIDE’s understanding of the nature of the boundary is now different, which impacts the final population of voids. Figure 8 highlights these differences.

While overall VIDE 2.0 identifies fewer small voids, this is a feature, not a limitation. In VIDE 1.0, the randomly-placed boundary particles introduced spurious density fluctuations that the watershed transform interpreted as real voids, creating fictitious basins that do not correspond to genuine underdensities in the galaxy distribution. These artifacts preferentially appear at small scales, roughly on the order of the mean galaxy separation, where the boundary particle density is comparable to the tracer density. VIDE 2.0 eliminates this entire class of spurious detections, since no artificial particles enter the tessellation. We highlight this effect in Figure 9, which contrasts the much cleaner boundary management in VIDE 2.0.

Figure 9. Same as Figure 7 — VIDE 1.0 central voids (left) compared to VIDE 2.0 clean voids (right) — zoomed into the upper-right quadrant of the slice. Note the presence of a spurious small void near the edge at approximate coordinates (x=150x = 150, y=325y = 325) h−1h^{-1} Mpc in VIDE 1.0 due to boundary effects, which are handled more gracefully by VIDE 2.0. In addition, VIDE 2.0 allows many more voids to survive closer to the edge due to less aggressive filtering schemes.

In contrast, the aggressive cleaning approach in VIDE 1.0 removes many large voids, as they might introduce a bias since they are near edges and holes. With VIDE 2.0, we find excellent agreement at large void radii when we impose a similar severe restriction to mimic the central catalog behavior. Because the v2 reference point is an edge-flagged galaxy — inside the survey by definition — rather than a mock particle placed outside, the cut is slightly more restrictive, and most aggressive at the largest void radii. Relaxing to the clean catalog gives a much larger population across almost all sizes.

That said, VIDE 2.0 still preserves critical void statistics. For example, the radial density profile, which is equivalent to the void-galaxy cross-correlation [24], of stacked voids within given size bins (Figure 10) remains nearly identical for voids larger than 15 h−1h^{-1} Mpc. This is perhaps unsurprising. For these void sizes, most of the voids are away from the boundaries and are exactly the same in the two catalogs. They are still identified underdense regions in the galaxy distribution. For the smallest voids (5–15 h−1h^{-1} Mpc), whose radii approach the mean galaxy separation, the profiles from both versions are noisy and agree only within that noise.

Comparison of radial density profiles for stacks of various void sizes

Figure 10. Comparison of radial density profiles for stacks of various void sizes. Distances are scaled by the midpoint radius RmidR_{\mathrm{mid}} of each void-size bin. The vertical dotted line indicates the mean galaxy separation in the sample, ∼7.4\sim7.4 h−1h^{-1} Mpc. Note the excellent agreement for larger voids; the smallest voids, with radii near the mean galaxy separation, give noisy profiles in both versions.

For completeness we show the all catalog of voids identified in VIDE 2.0 in Figure 11. As expected, the universe is mostly void ([17]), with watershed basins filling up the entire survey volume. Despite the artificial high densities for any flagged galaxies, many underdense regions approach the survey boundaries, where they are truncated by the lack of data.

Figure 11. All voids, including edge-contaminated ones, identified with VIDE 2.0. Note that Voronoi cells near the boundaries are ambiguously determined, but the regions interior to them contain valid underdensities.

Since voids have many uses across many analyses, VIDE 2.0 classifies every basin at pruning time. A void is contaminated if any of its member galaxies is edge-flagged, and central if it is clean (i.e., not contaminated) and its maximum extent from the macrocenter is shorter than the distance to the nearest edge-flagged galaxy. The classification is deterministic, depending only on the per-tracer edge flags and the macrocenter geometry. The pipeline produces four standard catalogs:

  • all: every watershed basin. This catalog is appropriate when only the location of an underdensity matters, as in weak-lensing stacks or cross-correlations with diffuse high-energy backgrounds.

  • clean: voids with no edge-flagged member galaxies. Because these voids are bounded entirely by unflagged tracers, their core densities, density contrasts, and adjacency-derived properties carry no imprint of the survey geometry. Use it for void-galaxy cross-correlations, ellipticity distributions, and alignments with large-scale structure.

  • central: a strict subset of clean in which the void’s maximum extent fits inside the survey on every side. Use it for radial density profiles, void lensing, Alcock–Paczynski tests, and other measurements keyed to the macrocenter.

  • tree: the hierarchical merger of adjacent basins into super-voids. This is not a primary science catalog — it duplicates all — but it is useful for constructing void merger histories and for identifying broad, shallow super-voids.

On SDSS DR7 the clean and central views contain 137 and 78 voids; on the 1.21 × 10710^{7}-galaxy Abacus ELG mock0 the corresponding counts are 4.2 × 10410^{4} and 3.5 × 10410^{4}.

In addition to these catalog-level classifications, VIDE 2.0 stores per-galaxy edge flags indicating whether each tracer was flagged and, if so, the source of contamination (domain boundary, redshift cap, or angular mask edge). This granularity is new to VIDE 2.0 and enables analyses that were previously impossible; for example, retaining edge-adjacent voids for lensing cross-correlations while excluding specific flagged galaxies from local density estimates, or studying how different sources of survey incompleteness impact void properties independently.

Demonstration of Stage IV Readiness

In this section we run VIDE 2.0 on the two mock suites of Table 1, generating the kinds of derived statistics a Stage IV analysis would need. We do not attempt a cosmological measurement here; our goal is to show that the catalogs are ready for one.

We ran VIDE 2.0 over the first 200 realizations of the Patchy mocks [34, 46], each containing ∼9.2×105\sim9.2 \times10^{5} galaxies at z∈[0.2,0.75]z \in[0.2, 0.75].

We did the same for all 25 realizations of the Abacus ELG mocks [38, 16] (B. Hadzhiyska et al. 2022), each carrying 1.21×1071.21 \times10^{7} tracers at z∈[0.8,1.6]z \in[0.8, 1.6]. Both ensembles used the auto-resolved defaults of § 2.3 and the catalog-tier flags of § 4, and required no parameter retuning between the BOSS-class and DESI-class samples.

The lack of stochasticity has direct consequences for ensemble work. In VIDE 1.0, the random boundary particles gave every catalog some run-to-run scatter (§ 3). Across a mock ensemble, that scatter adds a finder-specific term to the measured covariance, on top of the cosmic variance we actually want to measure. Because the term is independent between realizations, it does not stall the convergence of the covariance estimate; instead it inflates the converged value, and no statistic built from one run per realization can separate it from cosmic variance. VIDE 2.0 removes the term entirely: re-running any realization reproduces its catalog bit for bit, so the scatter across realizations comes entirely from the realizations themselves.

Figure 12 shows how precisely the two ensembles pin down that cosmic variance. We bootstrap-resample NN realizations, with replacement, from the 200 Patchy and the 25 Abacus ELG realizations and recompute the relative cosmic-variance scatter σ/⟨n(>R)⟩\sigma/ \langle n(> R) \rangle at three representative radii. The central value is essentially flat in NN and we omit it; the top panel shows the relative uncertainty on the estimate. For Patchy it falls from ∼35%\sim35\% at N=5N = 5 to ∼5%\sim5\% for the full 200-realization ensemble, tracking to within ∼10%\sim10\% the 1/2(N−1)1/\sqrt{2(N - 1)} expected for the standard deviation of Gaussian-distributed counts. The 25 Abacus ELG realizations determine their cosmic-variance scatter to ∼10\sim10–13%13\%.

Figure 12. Convergence diagnostics for the cosmic-variance covariance. Top: bootstrap relative uncertainty on the diagonal estimator σ/⟨n(>R)⟩\sigma/\langle n(>R)\rangle, resampling NN realizations with replacement, as a function of NN, at three representative radii (R=15,30,50 h−1 MpcR = 15, 30, 50\,h^{-1}\,\mathrm{Mpc}). Solid lines with circle markers: Patchy (200 realizations); dashed lines with square markers: Abacus ELG (25 realizations). The estimator’s central value is essentially flat in NN and is omitted. The dotted line is the Gaussian expectation 1/2(N−1)1/\sqrt{2(N-1)} for a sample standard deviation, and the last point of each curve is the uncertainty for the full ensemble. Bottom: Hartlap precision-matrix bias factor f=(N−K−2)/(N−1)f = (N-K-2)/(N-1) ([28]) as a function of NN for several data-vector lengths KK, with operating-point annotations at the ensemble sizes used here (N=25N = 25 for Abacus ELG, N=200N = 200 for Patchy) and a joint-probe extrapolation (N=1000N = 1000). The horizontal dashed line is the conventional f=0.85f = 0.85 safe floor.

The bottom panel of Figure 12 translates this into a practical mock-count budget at fixed data-vector length KK, via the Hartlap precision-matrix bias factor f=(N−K−2)/(N−1)f = (N - K - 2)/(N - 1) [28]. The 200 Patchy realizations supply f≥0.85f \ge0.85 for K≤28K \le28, comfortable for any single-statistic void analysis. Joint-probe analyses concatenating multiple void summaries with galaxy clustering or lensing push KK to several tens: reaching f=0.85f = 0.85 at K=60K = 60 takes N≈400N \approx400, and N∼1000N \sim1000 gives f≈0.94f \approx0.94. At BOSS scale, 1000 realizations take roughly a day of laptop wall-time at the per-realization cost of § 3.

The Alcock–Paczyński test remains a prime target for capturing cosmological parameter information from cosmic voids in Stage IV surveys. From the data side, this requires the stacked void-galaxy cross-correlation measured along and across the line of sight, ξvg(r∥,r⊥)\xi_{vg}(r_{\parallel}, r_{\perp}), together with an ensemble-derived estimate of its covariance. As a first step, Figure 13 shows the monopole ξvg(r)\xi_{vg}(r) and its covariance, computed on the central voids of each ensemble that additionally satisfy a stricter shell-coverage cut: the macrocenter must lie at least 3Rv3R_v from any survey edge, so the analysis shell out to r/Rv=3r/R_v = 3 remains inside the survey for every stacked void. The left panels show the per-realization curves with the ensemble mean and ±1σ\pm1\sigma band; both ensembles show the familiar compensated void profile, with its depth and ridge location set by each sample’s tracer density and redshift. The middle panels are the correlation matrices Cij/CiiCjjC_{ij}/\sqrt{C_{ii}C_{jj}} of the 20-bin estimator. They show strong positive correlations: adjacent bins correlate at ≈0.6\approx0.6, and beyond r≈25 h−1 Mpcr \approx25\,h^{-1}\,\mathrm{Mpc} every pair of bins is positively correlated (mean 0.55 for Patchy and 0.69 for Abacus ELG) — expected from bin coupling when voids are large compared to the bin width, rather than from contamination. The right panels show the bootstrap relative uncertainty σ^/∣μ^∣\hat{\sigma}/|\hat{\mu}| on the ensemble mean as a function of NmocksN_{\mathrm{mocks}}. With all 200 Patchy realizations, about half the bins fall below 1% (median 0.8%); the exceptions are the bin nearest the zero crossing, where ∣μ^∣|\hat{\mu}| approaches zero, and the largest separations, which reach ∼5%\sim5\%. The 25 Abacus ELG realizations reach a median of 1.3%, and their curves are still falling at N=25N = 25. The Hartlap factors are f≈0.89f \approx0.89 for Patchy (K=20K = 20, N=200N = 200) and f≈0.13f \approx0.13 for Abacus ELG (K=20K = 20, N=25N = 25); the former is comfortably inside the validity regime, the latter satisfies the threshold N>K+2N > K + 2 but is small, and a downstream analysis on the Abacus ELG ensemble would either restrict KK or extend the ensemble further.

Figure 13. Void-galaxy monopole cross-correlation ξvg(r)\xi_{vg}(r) from both demonstration ensembles: Patchy (top, 200 realizations) and Abacus ELG (bottom, 25 realizations). Both measurements use the central voids whose macrocenter lies at least 3R3R from any survey edge. Left: per-mock curves (light grey) with the ensemble mean and ±1σ\pm1\sigma cosmic-variance band. Middle: correlation matrix Cij/CiiCjjC_{ij}/\sqrt{C_{ii}C_{jj}} of the 20-bin estimator. Right: bootstrap estimate of σ^/∣μ^∣\hat{\sigma}/|\hat{\mu}| on the ensemble-mean estimator versus NmocksN_{\mathrm{mocks}} (Nboot=200N_{\mathrm{boot}}=200 resamples per ensemble size), color-coded by separation rr.

Watershed voids also carry shape information. As an example, we apply the same analysis to the void ellipticity distribution P(ϵ)P(\epsilon), computed from the shape-tensor eigendecomposition of each clean void. Figure 14 shows the result, using the same three-panel layout as Figure 13 but with 10 bins. Both ensembles peak near ϵ≈0.12\epsilon\approx0.12 with a tail extending to ϵ∼0.30\epsilon\sim0.30. For Patchy the correlation matrix is diagonal-dominant with mild anti-correlation between adjacent bins (mean −0.10-0.10), as expected for a normalized histogram, where an excess in one bin must be offset elsewhere; the Abacus ELG matrix shows the same pattern but is noisier, as expected from only 25 realizations. We include P(ϵ)P(\epsilon) because spherical-overdensity finders assume every void is a sphere, so they cannot produce this statistic at all; we can measure it only because the watershed approach defines each void by its actual basin geometry.

Figure 14. Ensemble void ellipticity distribution P(ϵ)P(\epsilon) computed from each void’s shape-tensor eigendecomposition, restricted to clean voids. Top: Patchy (200 realizations). Bottom: Abacus ELG (25 realizations). Panels and color coding as in Figure 13, on a 10-bin estimator.

We can also examine what VIDE 2.0 produces on Stage IV-class catalogs using only its default settings. Figure 15 shows the stacked galaxy density profile δ(r/Rv)\delta(r/R_v) in three void-radius bins, around the central voids of the Abacus ELG mock0 realization whose macrocenter lies at least 3Rv3R_v from any survey edge. Pair counts are estimated against the Abacus ELG random catalog (1.83×1071.83 \times10^{7} points across NGC + SGC), and errors come from a 10-fold spatial jackknife. The profiles recover the universal void shape of N. Hamaus et al.

Figure 15. Stacked galaxy density profile δ(r/Rv)\delta(r/R_v) around the central voids of the Abacus ELG mock0 realization (1.21×1071.21 \times10^7 galaxies, fsky=0.22f_{\mathrm{sky}} = 0.22, z∈[0.8,1.6]z \in[0.8,1.6]) whose macrocenter lies at least 3Rv3R_v from any survey edge, in three bins of void radius. Bands are 10-fold spatial-jackknife errors, mostly smaller than the markers. Pair counts are computed against the Abacus ELG random catalog (1.83×1071.83 \times10^7 points) projected to comoving Cartesian coordinates with Ωm=0.315\Omega_m = 0.315.

(2014a) : an interior underdensity that deepens with void size (minimum δ≈−0.54\delta\approx-0.54 to −0.76-0.76), a zero crossing at r/Rv≈0.75r/R_v \approx0.75–0.780.78, and a compensation overdensity near r/Rv≈1.1r/R_v \approx1.1 whose amplitude decreases monotonically with void size, from +0.37+0.37 for Rv∈[17,22] h−1 MpcR_v \in[17,22]\,h^{-1}\,\mathrm{Mpc} to +0.14+0.14 for Rv∈[28,50] h−1 MpcR_v \in[28,50]\,h^{-1}\,\mathrm{Mpc}, before returning to the survey mean by r/Rv≈2r/R_v \approx2. The amplitude–RvR_v relation is in quantitative agreement with the BOSS DR11 CMASS stacked-watershed measurements of N. Hamaus et al. [24].

In absolute units, the ensemble-mean cross-correlation ξvg(r)\xi_{vg}(r) over all 25 Abacus ELG realizations (bottom row of Figure 13; 17,857 ±\pm 126 voids per realization under the same cut) reaches ξvg≈−0.59\xi_{vg} \approx-0.59 at r≈6 h−1 Mpcr \approx6\,h^{-1}\,\mathrm{Mpc}, crosses zero near r≈18 h−1 Mpcr \approx18\,h^{-1}\,\mathrm{Mpc}, peaks in compensation at ξvg≈+0.19\xi_{vg} \approx+0.19 near r≈26 h−1 Mpcr \approx26\,h^{-1}\,\mathrm{Mpc}, and falls below 0.01 by r≈60 h−1 Mpcr \approx60\,h^{-1}\,\mathrm{Mpc}. The scatter between realizations is small: beyond the innermost bin, σ≲0.005\sigma\lesssim0.005 everywhere, under 1% of the trough depth and about 2% of the compensation peak.

Lastly, we can compare the three catalogs directly. Figure 16 overlays the cumulative number function n(>R)n(>R) of clean voids on a single axis for the observed SDSS DR7 sample (a single catalog), the Patchy ensemble mean, and the Abacus ELG ensemble mean. Note that the two ensemble curves cross at R≈34 h−1 MpcR \approx34\,h^{-1}\,\mathrm{Mpc}: the Abacus ELG mocks produce ∼2\sim2–3×3\times more small voids than Patchy at R≲20 h−1 MpcR \lesssim20\,h^{-1}\,\mathrm{Mpc}, but Patchy extends to larger RR with ∼25×\sim25\times more voids by R=60 h−1 MpcR = 60\,h^{-1}\,\mathrm{Mpc}. Three effects drive this. The dominant one is tracer density: watershed voids are built on the Voronoi tessellation of the tracers, so the characteristic void size scales with the mean tracer separation sˉ=ng−1/3\bar{s} = n_g^{-1/3}. Patchy and Abacus ELG differ in sˉ\bar{s} by a factor of ∼1.4\sim1.4 (sˉPatchy≈17 h−1 Mpc\bar{s}_{\mathrm{Patchy}} \approx17\,h^{-1}\,\mathrm{Mpc}, sˉAbacus≈12 h−1 Mpc\bar{s}_{\mathrm{Abacus}} \approx12\,h^{-1}\,\mathrm{Mpc}), so the sparser Patchy catalog produces correspondingly larger Voronoi cells and watershed basins. In each sample the smallest voids are a few times the mean tracer separation, the standard ZOBOV resolution scaling (M. C. Neyrinck [43]; P. M. Sutter et al. [57]). Two subdominant effects push in the same direction: cosmic structure is further along at Patchy’s median z≈0.5z \approx0.5, where the linear growth factor is ∼40%\sim40\% larger than at the Abacus ELG mocks’ z≈1.2z \approx1.2, and the more strongly biased BOSS LRGs (b≈2b \approx2) avoid void interiors more cleanly than the less biased ELGs (b≈1.3b \approx1.3–1.51.5). A controlled test would subsample the Abacus ELG catalogs down to Patchy’s tracer density and rerun; we expect the small-RR surplus to disappear and the crossover to migrate toward larger RR.

Figure 16. Cumulative void number function n(>R)n(>R) for clean voids across the three catalog products demonstrated in this work: a single observed SDSS DR7 sample, the 200-realization Patchy ensemble mean, and the 25-realization Abacus ELG ensemble mean. The two ensemble curves cross at R≈34 h−1 MpcR \approx34\,h^{-1}\,\mathrm{Mpc}.

A cosmological measurement is beyond the scope of this paper. A defensible Alcock–Paczyński analysis on Stage IV data requires a theoretical template mapping cosmological parameters onto ξvg(r∥,r⊥)\xi_{vg}(r_{\parallel},r_{\perp}) calibrated against cosmology mocks, treatment of void selection bias under the chosen tracer and cut definition (A. Pisani et al. [44]; N. Hamaus et al. [22]), and marginalization over nuisance parameters within a mock-validated likelihood. The contribution of this section is to show that our deterministic void finder yields ensemble covariances that converge, and that it delivers the multivariate per-void inputs that void-based cosmological studies can adopt without reprocessing.

Conclusions

We have presented VIDE 2.0, an overhaul and upgrade to the VIDE void-finding and analysis package. We primarily focused on improving and clarifying the relationships among Voronoi tessellations, watershed algorithms, and the survey mask. In addition to providing a rigorous procedure for mapping a survey’s boundary into the void finding algorithm, we have also significantly improved the performance and reliability of the VIDE engine. In particular, VIDE 2.0 enjoys orders-of-magnitude speedups, significantly less memory usage, and near-optimal scalability with respect to tracer number.

Void analysis will play a critical role in current and upcoming large galaxy surveys, and with these upgrades in place VIDE 2.0 is now poised to handle the large quantities of data produced by these surveys. Additionally, VIDE is now fully parameter-free, aside from user specification of the mask, allowing our approach to identify voids in any survey configuration and arrangement with maximum flexibility.

The approach of the VAST void-finding engine [17] employs a related strategy to survey edge-finding, wherein they flag any galaxy whose local Voronoi polyhedron extends past the survey boundary, and then assign an infinite density to those galaxies prior to the watershed step. However, to determine which voids are contaminated by the edge, VAST allows the user to specify what percentage of any void’s surface area must include edge galaxies. We adopt a simpler, parameter-free approach, wherein we mark any void that includes even a single edge-flagged galaxy as contaminated. However, any voids that are merely adjacent to edge galaxies remain, since the interior edges of Voronoi cells of those edge galaxies are secure.

The survey mask can have dramatic impacts on the derived void catalog, since voids are large, extended objects that often intersect survey boundaries. A clean, precise understanding of the mask, as viewed through the same morphological lens as the watershed basins themselves, is crucial for properly navigating the implications of a resulting void catalog.

As discussed in § 2.3, the choice of NSIDE has two failure modes that constrain it from above and below. We addressed this by introducing an automatic rule that selects NSIDE by matching the mean tracer separation in a volume-weighted sense; for our fiducial DR7 example the rule selects NSIDE = 64.

The goal of VIDE 2.0 is to allow the user maximum freedom to use the produced void catalogs for their desired purposes. In the tradition established with the release of VIDE 1.0, the void finding package (housed at http://www.cosmicvoids.net) includes many helpful Python-based analysis tools and methods, which allow the user to quickly load void catalogs, member galaxies, and more, as well as filter the catalogs to suit desired purposes and perform analyses as they wish. Users can select from all voids (including ones that are contaminated by galaxies near the survey edge), clean voids free from edge effects, and central voids that are as bias-free as possible. Users may also choose to build an entire (sub-)void hierarchy.

We have made available the visualization routines used to create the new slice plots in this paper in the analysis package. All analysis routines are fully backwards-compatible with void catalogs produced by VIDE 1.0.

In addition to the crucial boundary handling updates discussed in the present paper, the public code repository1 details many other updates, fixes, and quality-of-life improvements, such as more efficient memory and storage handling, code reorganization and cleanup, a modern interface for working with and manipulating void datasets, enhanced capabilities for managing survey incompleteness and weighting schemes, and much more. These details are discussed in the documentation made available as part of the public repository.

This is the first major upgrade to VIDE in over a decade. It maintains the parameter-free spirit that first animated the conversion of VIDE into a fully-featured void analysis suite. Void catalogs produced by VIDE 2.0 are also fully deterministic: given the same input galaxy catalog and survey mask, the output is bit-for-bit identical across runs. This is in contrast to VIDE 1.0, where stochastic boundary particle placement meant that void catalogs, particularly near survey edges, varied between runs, complicating reproducibility studies and covariance estimation from mock catalogs. Furthermore, VIDE 2.0 stores per-galaxy edge provenance flags, enabling downstream analyses to apply custom contamination criteria without re-running the pipeline. Combined with the richer void property estimates now computed during pruning (ellipticity, eigenvalues, density profiles), the catalogs produced by VIDE 2.0 are not only more complete but carry substantially more metadata per void, reducing the need for post-hoc reprocessing.

Our analysis of two large mock suites — 200 MultiDark-Patchy DR12 NGC realizations at BOSS scale, and 25 AbacusSummit DESI Y1 ELG realizations at DESI Y1 scale — produced converged cosmic-variance covariances for the void number function, the void-galaxy cross-correlation ξvg(r)\xi_{vg}(r), and the void ellipticity distribution P(ϵ)P(\epsilon). Because VIDE 2.0 is deterministic, none of the measured scatter comes from the void finder itself.

The full ensemble work was completed on commodity laptop hardware under modest process parallelism, with no parameter retuning between the BOSS-class and DESI-class samples. Together with the single-laptop scaling of § 3, this makes Stage IV void analyses, and the Alcock–Paczyński test in particular, practical on modest computing resources.

Finally, VIDE maintains a long history of use, from a variety of simulations to multiple surveys, providing the community with a rich history of understanding as seen through the lens of this algorithm. Follow-up papers will use the upgraded pipeline to make actual cosmological measurements. Here we have shown that everything leading up to those measurements, from deterministic catalog production to ensemble-level covariance estimation, can be done on commodity hardware.

Acknowledgments

PMS owes an enormous debt of gratitude to Guilhem Lavaux, Alice Pisani, Nico Hamaus, and all the other void enthusiasts who maintained VIDE over the years. We thank Giovanni Verza for implementing per-tracer weights in VIDE. BDW acknowledges the support of the Simons Foundation through the Simons Collaboration on “Learning the Universe”.

Software: VIDE (P. M. Sutter et al. [57]), VORO++ (C. H. Rycroft [47]), HEALPix (K. M. Górski et al. [20]), healpy (A. Zonca et al. [70]), NumPy (C. R. Harris et al. [27]), SciPy (P. Virtanen et al. [63]), Astropy (Astropy Collaboration et al. [5, 4, 3]), Matplotlib (J. D. Hunter [31]), h5py, HDF5, GSL, CFITSIO.

References

  1. [1]Abazajian, K. N., Adelman-McCarthy, J. K., Agüeros, M. A., et al. 2009, Astrophysical Journal, Supplement Series, 182, 543, doi: 10.1088/0067-0049/182/2/543DOI
  2. [2]Aggarwal, M., Tonegawa, M., Appleby, S., Park, C., & Periwal, V. 2024, The Astrophysical Journal, 975, 95, doi: 10.3847/1538-4357/ad7f5d
  3. [3]Astropy Collaboration, Price-Whelan, A. M., Lim, P. L., et al. 2022, ApJ, 935, 167, doi: 10.3847/1538-4357/ac7c74DOI
  4. [4]Astropy Collaboration, Price-Whelan, A. M., Sipőcz, B. M., et al. 2018, AJ, 156, 123, doi: 10.3847/1538-3881/aabc4f
  5. [5]Astropy Collaboration, Robitaille, T. P., Tollerud, E. J., et al. 2013, A&A, 558, A33, doi: 10.1051/0004-6361/201322068
  6. [6]Barber, C. B., Dobkin, D. P., Huhdanpaa, H., et al. 2013, ascl, ascl:1304.016. https://ui.adsabs.harvard.edu/abs/2013ascl.soft04016B/abstract
  7. [7]Bayer, A. E., Liu, J., Kreisch, C. D., & Pisani, A. 2024, PhRvD, 110, L061305, doi: 10.1103/PhysRevD.110.L061305DOI
  8. [8]Bock, J. J., Aboobaker, A. M., Adamo, J., et al. 2026, ApJ, 999, 139, doi: 10.3847/1538-4357/ae2be2DOI
  9. [9]Bromley, B. C., & Geller, M. J. 2025, Journal of Cosmology and Astroparticle Physics, 2025, 011, doi: 10.1088/1475-7516/2025/05/011DOI
  10. [10]Colberg, J. M., Pearce, F., Foster, C., et al. 2008, Monthly Notices of the Royal Astronomical Society, 387, 933, doi: 10.1111/j.1365-2966.2008.13307.xDOI
  11. [11]Collaboration, D., Abdul-Karim, M., Adame, A. G., et al. 2025, arXiv, 46, arXiv:2503.14745, doi: 10.48550/ARXIV.2503.14745
  12. [12]Contarini, S., Pisani, A., Hamaus, N., et al. 2023, The Astrophysical Journal, 953, 46, doi: 10.3847/1538-4357/acde54DOI
  13. [13]Contarini, S., Verza, G., Pisani, A., et al. 2022, Astronomy and Astrophysics, 667, A162, doi: 10.1051/0004-6361/202244095DOI
  14. [14]Curtis, O., McDonough, B., & Brainerd, T. G. 2025, ApJ, 985, 244, doi: 10.3847/1538-4357/ADCEBFDOI
  15. [16]DESI Collaboration. 2025, JCAP, 2025, 008, doi: 10.1088/1475-7516/2025/09/008
  16. [17]Douglass, K. A., Veyrat, D., & BenZvi, S. 2023, The Astrophysical Journal Supplement Series, 265, 7, doi: 10.3847/1538-4365/ACABCFDOI
  17. [18]Furniss, A., Amador, J. N., Hervet, O., et al. 2025, ApJL, 980, L15, doi: 10.3847/2041-8213/ADAE9DDOI
  18. [19]Gong, Y., Miao, H., Zhou, X., et al. 2025, Science China: Physics, Mechanics and Astronomy, 68, 280402, doi: 10.1007/s11433-025-2646-2DOI
  19. [20]Górski, K. M., Hivon, E., Banday, A. J., et al. 2005, ApJ, 622, 759, doi: 10.1086/427976DOI
  20. [22]Hamaus, N., Cousinou, M.-C., Pisani, A., et al. 2017, Journal of Cosmology and Astroparticle Physics, 2017, 014, doi: 10.1088/1475-7516/2017/07/014DOI
  21. [23]Hamaus, N., Pisani, A., Choi, J.-A., et al. 2020, JCAP, 2020, 023, doi: 10.1088/1475-7516/2020/12/023DOI
  22. [24]Hamaus, N., Pisani, A., Sutter, P. M., et al. 2016, Physical Review Letters, 117, 091302, doi: 10.1103/PhysRevLett.117.091302DOI
  23. [25]Hamaus, N., Sutter, P. M., & Wandelt, B. D. 2014a, Physical Review Letters, 112, 251302, doi: 10.1103/PhysRevLett.112.251302DOI
  24. [26]Hamaus, N., Wandelt, B. D., Sutter, P., Lavaux, G., & Warren, M. S. 2014b, Physical Review Letters, 112, 041304, doi: 10.1103/physrevlett.112.041304DOI
  25. [27]Harris, C. R., Millman, K. J., van der Walt, S. J., et al. 2020, Nature, 585, 357, doi: 10.1038/s41586-020-2649-2DOI
  26. [28]Hartlap, J., Simon, P., & Schneider, P. 2007, A&A, 464, 399, doi: 10.1051/0004-6361:20066170DOI
  27. [29]Hossen, M. R., Ema, S. A., Bolejko, K., & Lewis, G. F. 2022, Monthly Notices of the Royal Astronomical Society, 513, 5575, doi: 10.1093/mnras/stac1247DOI
  28. [31]Hunter, J. D. 2007, Computing in Science and Engineering, 9, 90, doi: 10.1109/MCSE.2007.55DOI
  29. [32]Kang, G., & Lee, J. 2025, JCAP, 2025, 011, doi: 10.1088/1475-7516/2025/06/011DOI
  30. [34]Kitaura, F.-S., Rodríguez-Torres, S., Chuang, C.-H., et al. 2016, MNRAS, 456, 4156, doi: 10.1093/mnras/stw2826
  31. [35]Kreisch, C. D., Pisani, A., Villasecusa-Navarro, F., et al. 2022, The Astrophysical Journal, 935, 100, doi: 10.3847/1538-4357/ac7d4bDOI
  32. [36]Lavaux, G., & Wandelt, B. D. 2012, The Astrophysical Journal, 754, 109, doi: 10.1088/0004-637X/754/2/109DOI
  33. [37]Maggiore, L., Contarini, S., Giocoli, C., & Moscardini, L. 2025, A&A, 701, A55, doi: 10.1051/0004-6361/202554968DOI
  34. [38]Maksimova, N. A., Garrison, L. H., Eisenstein, D. J., et al. 2021, MNRAS, 508, 4017, doi: 10.1093/mnras/stab2484DOI
  35. [39]Mao, Q., Berlind, A. A., Scherrer, R. J., et al. 2016, The Astrophysical Journal, 835, 161, doi: 10.3847/1538-4357/835/2/161DOI
  36. [40]Massara, E., Villaescusa-Navarro, F., Viel, M., & Sutter, P. 2015, Journal of Cosmology and Astroparticle Physics, 2015, 018, doi: 10.1088/1475-7516/2015/11/018DOI
  37. [41]Mellier, Y., Abdurro’uf, Barroso, J. A. A., et al. 2025, Astronomy and Astrophysics, 697, A1, doi: 10.1051/0004-6361/202450810DOI
  38. [42]Montero-Dorta, A. D., Balaguera-Antolínez, A., Alfaro, I. G., et al. 2025, A&A, 703, A58, doi: 10.1051/0004-6361/202555221DOI
  39. [43]Neyrinck, M. C. 2008, Monthly Notices of the Royal Astronomical Society, 386, 2101, doi: 10.1111/j.1365-2966.2008.13180.xDOI
  40. [44]Pisani, A., Sutter, P. M., Hamaus, N., et al. 2015, Physical Review D - Particles, Fields, Gravitation and Cosmology, 92, 083531, doi: 10.1103/PhysRevD.92.083531DOI
  41. [45]Rincon, H., Benzvi, S., Douglass, K., et al. 2025, ApJ, 982, 38, doi: 10.3847/1538-4357/ADB559DOI
  42. [46]Rodríguez-Torres, S. A., Chuang, C.-H., Prada, F., et al. 2016, MNRAS, 460, 1173, doi: 10.1093/mnras/stw1014
  43. [47]Rycroft, C. H. 2009, Chaos, 19, 041111, doi: 10.1063/1.3215722DOI
  44. [49]Schuster, N., Hamaus, N., Pisani, A., Dolag, K., & Weller, J. 2025, arXiv, arXiv:2509.07092, doi: 10.48550/ARXIV.2509.07092arxiv.org/abs/2509.07092
  45. [50]Shojaei, M. R., Tavasoli, S., & Ghafour, P. 2025, arXiv, arXiv:2506.06711, doi: 10.48550/ARXIV.2506.06711
  46. [51]Song, Y., Gong, Y., Zhou, X., et al. 2025, MNRAS, 540, 2853, doi: 10.1093/MNRAS/STAF918
  47. [52]Song, Y., Xiong, Q., Gong, Y., et al. 2024, ApJ, 976, 244, doi: 10.3847/1538-4357/AD8DE9DOI
  48. [53]Sutter, P. M., Elahi, P., Falck, B., et al. 2014a, Monthly Notices of the Royal Astronomical Society, 445, 1235, doi: 10.1093/mnras/stu1845DOI
  49. [54]Sutter, P. M., Lavaux, G., Hamaus, N., et al. 2014b, Monthly Notices of the Royal Astronomical Society, 442, 462, doi: 10.1093/mnras/stu893DOI
  50. [55]Sutter, P. M., Lavaux, G., Wandelt, B. D., & Weinberg, D. H. 2012, Astrophysical Journal, 761, 44, doi: 10.1088/0004-637X/761/1/44DOI
  51. [56]Sutter, P. M., Lavaux, G., Wandelt, B. D., et al. 2014c, Monthly Notices of the Royal Astronomical Society, 442, 3127, doi: 10.1093/mnras/stu1094DOI
  52. [57]Sutter, P. M., Lavaux, G., Hamaus, N., et al. 2015, Astronomy and Computing, 9, 1, doi: 10.1016/j.ascom.2014.10.002DOI
  53. [58]Thiele, L. 2026, The Open Journal of Astrophysics, 9, 60855, doi: 10.33232/001c.160855DOI
  54. [59]Thompson, L. A., & Gregory, S. A. 2011, arXiv e-prints. https://arxiv.org/abs/1109.1268
  55. [60]Verza, G., Carbone, C., Pisani, A., & Renzi, A. 2023, JCAP, 2023, 044, doi: 10.1088/1475-7516/2023/12/044DOI
  56. [61]Verza, G., G. Pisani, A., et al. 2025, ApJ, 993, 227, doi: 10.3847/1538-4357/AE07D9DOI
  57. [62]Veyrat, D., Douglass, K. A., & BenZvi, S. 2023, ApJ, 958, 59, doi: 10.3847/1538-4357/ACF4F5DOI
  58. [63]Virtanen, P., Gommers, R., Oliphant, T. E., et al. 2020, Nature Methods, 17, 261, doi: 10.1038/s41592-019-0686-2DOI
  59. [64]Wang, B. Y., & Pisani, A. 2024, ApJL, 970, L32, doi: 10.3847/2041-8213/ad5ffeDOI
  60. [65]Williams, M. J., Macpherson, H. J., Wiltshire, D. L., & Stevens, C. 2025, Monthly Notices of the Royal Astronomical Society, 536, 2645, doi: 10.1093/mnras/stae2737DOI
  61. [66]Wilson, C., & Bean, R. 2021, Physical Review D, 104, 023512, doi: 10.1103/PhysRevD.104.023512DOI
  62. [67]Wilson, C., & Bean, R. 2023, Physical Review D, 107, 124008, doi: 10.1103/PhysRevD.107.124008DOI
  63. [68]Xiong, Q., Gong, Y., Yan, J., et al. 2025, arXiv, arXiv:2511.04099, doi: 10.48550/arXiv.2511.04099arxiv.org/abs/2511.04099
  64. [69]Zaidouni, F., Veyrat, D., Douglass, K. A., & BenZvi, S. 2025, The Astrophysical Journal, 978, 3, doi: 10.3847/1538-4357/ad9585DOI
  65. [70]Zonca, A., Singer, L., Lenz, D., et al. 2019, Journal of Open Source Software, 4, 1298, doi: 10.21105/joss.01298DOI

Paper details

vlnc:W_v9jrram2
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents