ODIN: Comparison of Source Extractor and Tractor Source Detection for Lyman Alpha Emitting Galaxy Selection at z=3.1

Introduction

Lyman Alpha Emitting Galaxies (LAEs) are star-forming galaxies as seen by their strong Lyα emission at a rest-frame wavelength of 121.6 nm [28]. Studies show that LAEs tend to be young and compact with low line-of-sight dust content [10, 13, 31], allowing Lyα photons to escape with less absorption and scattering before being observed. Therefore, LAEs can be efficiently identified and used to study the large-scale structure of the Universe and galaxy evolution over time [10, 15].

Since the first discoveries of LAEs in the 1990s [29] (Hu & McMahon 1996), there have been a multitude of surveys focused on finding these high-$z$ galaxies [9, 6, 27, 11, 21]. These studies have revealed some differences in measured LAE properties; however, different definitions and LAE selection methods employed by these earlier studies may have contributed to these results, highlighting the need for a more systematic investigation.

Earlier studies such as the previously mentioned Hu & McMahon (1996) and Pascarelle et al. [29], as well as the Hawaii Survey (Cowie & Hu [5]), used manual and visual inspection techniques to identify their LAEs because automated source detection had not yet been commonly utilized. Once the Large Area Lyman Alpha (LALA) survey (Rhoads et al. [33]) employed Source Extractor (SE), automated source detection became standard in high-$z$ LAE surveys.

SE was released in 1996, the same year the first LAEs were discovered, and quickly gained traction due to its fast, flexible, and robust nature (Bertin & Arnouts 1996). SE became one of the primary tools for source detection in high-$z$ galaxy surveys, including the Subaru Deep Field (SDF) survey (Ouchi et al. [25]), the Multiwavelength survey by Yale-Chile (MUSYC) (Gawiser et al. [10]), and the Lyman Alpha Galaxies in the Epoch of Reionization (LAGER) survey (Zheng et al. [42]). SE works by estimating the background level of an image, then uses pixel thresholding to detect and flag sources. After carefully deblending any merged objects, SE employs aperture-based photometry and returns a catalog of all sources and their resulting positions, shapes, fluxes, and quality flags.

The Tractor (Tr) is another prominent source detection method used in large multi-band imaging surveys (Lang et al. [19]). Released in 2016, Tr has gained wide-spread attention for its ability to handle complex large-area datasets and is now a core tool in surveys like the Dark Energy Spectroscopic Instrument (DESI) Legacy Survey (Dey et al. [6]). Tr measures photometry by forward modeling based on the local point spread function (PSF) for each source in an image. Model parameters are optimized to best match the image data, and forced photometry is applied to the other bands using positions and shapes derived from the detection image. This results in a catalog that contains PSF-aware flux measurements, as well as source positions, shapes, and associated uncertainties.

As SE and Tr become more widely used tools in the search for high-$z$ galaxies, the One-hundred-deg$^2$ DECam Imaging in narrow-bands (ODIN; Lee et al. [21]) survey offers a valuable opportunity to directly compare these two source detection methods. ODIN is a NOIRLab survey which utilizes the Dark Energy Camera (DECam; Flaugher et al. [8]) on the 4m Víctor M. Blanco telescope at the Cerro Tololo Inter-American Observatory (CTIO) in Chile. With DECam, ODIN collects LAE samples at redshifts of 2.4, 3.1, and 4.5 using custom narrow-band filters that correspond to each redshift (Firestone et al. [7]). This range allows ODIN to look into the evolution of galaxies during what is known to be the peak epoch of star formation in the Universe, Cosmic Noon. In addition to deep narrow-band images, ODIN uses archival grizy broad-band images from the Hyper Supreme-Cam (HSC; Aihara et al. [1]) as well as $u$-band images from the CFHT Large Area $u$-band Deep Survey (CLAUDS; Sawicki et al. [34]). With this data, ODIN is expected to discover over 100,000 LAEs across 7 fields and 3 redshift bins, making it one of the largest LAE surveys to date (Lee et al. [21]).

Both SE and Tr are applicable to ODIN and future LAE surveys. In this paper, we analyze the difference between SE and Tr generated catalogs and their respective photometry for ODIN. Understanding the differences that arise from these selection methods will not only allow ODIN to create a more robust sample of LAEs, but it will also improve the accuracy of survey comparisons by clarifying methodological biases in source detection and measurement.

In § 2, we summarize the ODIN imaging data, SE and Tr processing, and LAE selection methods. In § 3, we present a comparison of the resulting catalogs, including photometric differences and evidence for potential contamination. In § 4, we discuss the implications of these results for ODIN LAE selection and propose an additional LAE selection criterion, followed by conclusions and future work in § 5. In this work, we focus on selected LAEs for $z = 3.1$ in the 9 deg$^2$ ODIN-COSMOS field as this is ODIN’s most well-studied redshift/field as of this publication. Throughout this work, we assume a ΛCDM cosmology with $h = 0.7$, $\Omega_m = 0.27$, and $\Omega_\Lambda= 0.73$. All magnitudes in this paper are in the AB magnitude system (Oke [24]).

Data

Images

ODIN combines N501 observations with broad-band images for LAE selection in the 9 deg$^2$ COSMOS field. Both SE and Tr source catalogs utilize narrow-band data. ODIN’s N501 filter was custom-made with a central wavelength of $501 \pm3.8\,\mathrm{nm}$ to detect LAEs at redshifts of 3.1, with a redshift interval of $\Delta z = 0.059$ corresponding to the full-width-at-half-maximum (FWHM) of the filter transmission (Lee et al. [21]).

Archival grizy broad-band images used in the SE and Tr source catalogs come from the second data release of the Hyper Supreme-Cam Subaru Strategic Program (HSC-SSP) on the 8.2m Subaru telescope located in Hawaii (Aihara et al. [1]). The HSC-SSP broad-band data include three layers in the COSMOS field: Wide, Deep, and Ultradeep. Both the SE and Tr catalogs have been created using a combination of the Deep and Ultradeep layers. In addition to archival grizy broad-band images, both SE and Tr catalogs use u-band images from The CFHT Large Area U-band Deep Survey (CLAUDS; Sawicki et al. [34]) using the MegaCam mosaic imager on the Canadian-France-Hawaii Telescope (CFHT). This u-band data was created to match the depth of the Deep and Ultradeep layers in the COSMOS field with a median depth of 27.7 mag (Sawicki et al. [34]).

SE Source Catalog

For SE, the N501 detection image is a full-depth coadd of dithered DECam exposures processed with the NOIRLab DECam Community Pipeline (CP; Valdes et al. [38]; Valdes [37]). The COSMOS N501 coadd consists of 72 individual exposures and reaches a nearly uniform median depth of approximately 25.4 AB mag. At the individual-exposure level, the CP flags bad CCD pixels, saturated and bleed-trail pixels, cosmic rays, and other streaks, while satellite trails are additionally identified manually and masked. Flagged and masked pixels are excluded when constructing the final coadd. The broad-band measurement images consist of HSC-SSP coadded grizy images and a CLAUDS u-band image. The HSC-SSP science images, data-quality maps, and RMS maps are reprojected using the DECam CP to match the tangent point and pixel scale of the ODIN data.

We run SE in dual-image mode, using the N501 coadd for detection and the ugrizy broad-band images for measurement. All images are convolved with a 1 arcsecond FWHM Gaussian filter (FILTERING = Y), approximately matched to the average narrow-band seeing. We adopt a detection threshold of 0.95$\sigma$ (in units of the pre-convolution rms) and a minimum area of 1 pixel (DETECT THRESH = 0.95; DETECT MINAREA = 1), which maximize the detection of faint point sources while maintaining a low contamination rate ($\lesssim1\%$, see Ramakrishnan et al. [32], for details). Photometry is extracted in 12 apertures with diameters of 1, 1.5, 1.75, 2, 2.25, 2.75, 3, 3.5, 4, 4.5, 5, and 7 arcseconds.

We additionally impose a half-light radius cut ($R_{50} < 1.38^{\prime\prime}$) and use two N501 “half-stacks,” each constructed from 36 exposures with the DECam CP, to identify remaining spurious sources. Sources are detected in the full-depth coadd and their fluxes are independently measured in each half-stack; strongly inconsistent measurements between the two are identified as likely spurious and removed (see Firestone et al. [7] for further details).

Tr Source Catalog

Tr is run as part of the Legacytype reduction pipeline (Lang et al. [20]). Source detection, morphological type classification, and measurements of fluxes, positions, and galaxy shapes are performed on all overlapping images jointly. The pipeline processes one “brick” of 0.25 × 0.25 deg$^{2}$ at a time. Source detection proceeds by merging detections in any of the three narrow-band filters, with detection performed as described in Lang & Hogg [18]. Source fitting is carried out by nonlinear least-squares optimization: the code finds model parameters including source positions, fluxes, and galaxy shapes such that the PSF-convolved model predictions best match all the observed images that overlap. Sources that have flux in only one of the narrow-band filters will end up with fit fluxes that are consistent with zero in the other bands. After source positions and galaxy shapes have been measured in the narrow-band images, they are applied to the broad-band images via forced photometry: the source profiles are held fixed and only the broad-band fluxes are fit so that the PSF-convolved model predictions best match the observed images.

Catalog Processing

Our LAE samples are created by processing either the SE or Tr source catalog through a selection pipeline. For SE, we use the methodology described by Firestone et al. [7], while the Tr catalog is processed using the same pipeline adjusted to accommodate the differences in catalog structure. Both catalogs undergo multiple corrections and selection criteria, as described below, to guarantee robust LAE samples.

We first consider the scattering and absorption of light as it passes through dust in the Milky Way and makes its way to Earth. This effect causes sources to appear dimmer and redder than they truly are. Therefore, Galactic dust corrections are applied to sources found in both the SE and Tractor catalogs using the 8′ resolution reddening map (SFD) from Schlegel et al. [35] and Equation (1);

Here, $\mathrm{flux}_{\mathrm{corr}}$ is the Galactic dust-corrected flux, $\mathrm{flux}_{\mathrm{obs}}$ is the observed flux, $k$ is the extinction coefficient for the corresponding filter, and $E(B-V)$ is the SFD reddening value for each source. The filter-specific extinction coefficients, $k$, are given in Firestone et al. [7]. In addition, we assign a magnitude of 40 to sources with flux density values that have behavior indistinguishable from zero ($\leq3.6 \times10^{-7}\ \mu$Jy) to flag them as extremely faint objects. We choose to assign a magnitude of 40 to these objects because this magnitude does not overlap with the primary distribution of magnitudes, which peaks around 25 mag.

When running the pipeline with our SE catalog, we also apply aperture corrections to improve the accuracy of our photometric measurements while limiting noise in our data. Our SE catalog includes flux density measurements for 12 different aperture diameters. As a compromise between noise and aperture loss, we use the 2.0 arcsecond aperture for each source’s flux measurement. However, this approach underestimates the total flux density of our sources, since smaller apertures do not capture all of a source’s light. To compensate, we compute the ratio of the median fractional flux in the chosen aperture (2.0 arcsecond) to the largest aperture (5.0 arcsecond) for bright, unsaturated point sources. We then use this fractional flux measurement as a correction factor to adjust the flux density upward to more accurately reflect a source’s total brightness, while maintaining low levels of noise (for more details, see Firestone et al. [7]).

For both cases, we then apply star masking to exclude sources contaminated by bright stars. We use starmasks from the HSC-SSP $g$-band images (Coupon et al. [4]) to remove saturated pixels and regions affected by CCD blooming. This allows us to minimize contamination and maintain the reliability of our photometry.

We then implement data quality cuts to ensure that our data are reliable and reduce contamination from low-quality detections that are not removed by star masking. We first require the narrow-band signal to noise ratio ($\mathrm{S/N}_{\mathrm{NB}}$) to be above 5 in both SE and Tr. The remaining data quality cuts differ between SE and Tr catalogs due to their differences in diagnostic flags and metadata. SE quality cuts are primarily image-based and focus on the reliability of detections at a pixel level. We require FLAGS $< 4$, allowing sources affected by neighboring objects or de-blending while excluding sources with more severe extraction issues, such as pixel saturation (Bertin, E. & Arnouts, S. 1996). We additionally require IMAFLAGS_ISO $= 0$, indicating that none of the pixels associated with a source are flagged in the external flag map. We also ensure that flux measurements are non-zero to remove sources with unavailable or invalid photometry and include a half-light radius cut to exclude large, extended sources. In contrast, Tr quality cuts use both metadata from source model fitting and image-space quality flags. Tr uses the flag BRICK_PRIMARY to ensure that objects are within the brick boundary, as well as NOBS_N501 to enforce a minimum number of observations for the narrow-band filter. Other quality criteria include the image-space flag MASKBITS, for which we require MASKBITS $= 0$ to exclude sources associated with flagged regions, including bright and medium-bright Gaia stars, SGA galaxies, and saturated pixels. We additionally require ALLMASK_N501 $= 0$ to specifically exclude sources affected by bad or saturated pixels in the N501 imaging. Similar image-quality filtering is applied in the SE catalog through the IMAFLAGS_ISO and FLAGS cuts described above. Tr also produces a FRACFLUX metric for each catalog source, describing the fraction of flux contributed by neighboring sources within the fitting region and providing a measure of source blending. For the Tr catalog, we apply constraints on FRACFLUX_N501 to limit the contribution of flux from neighboring sources and help remove heavily blended objects with potentially unreliable photometry. We also require FORCED_MEAN_FLUX_IVAR_BB $> 0$, where FORCED_MEAN_FLUX_IVAR_BB is the inverse variance associated with the forced mean broad-band flux measurement. This requirement ensures that sources have a valid broad-band flux measurement with an associated uncertainty, removing sources for which broad-band photometry is unavailable or invalid. A complete list of these cuts and thresholds is summarized in Table 1.

SE	Tr
S/N$_{\mathrm{NB}}$ $\ge5$	S/N$_{\mathrm{NB}}$ $\ge5$
IMAFLAGS_ISO $= 0$	BRICK_PRIMARY $=$ True
FLAGS $< 4$	MASKBITS $= 0$
$f_\nu\ne0$	ALLMASK_N501 $= 0$
$R_{50} < 1.38^{\prime\prime}$	NOBS_N501 $> 10$
	$0 \le$ FRACFLUX_N501 $\le0.5$
	FORCED_MEAN_FLUX_IVAR_BB $> 0$

We note that the SE and Tr catalogs cover the same area within the ODIN-COSMOS field, allowing their source populations to be directly compared without normalization for differences in survey footprint.

LAE Selection

The final step in our pipeline is to compile our LAE sample based on specific selection criteria. The criteria for LAE classification primarily rely on the hybrid-weighted double-broad-band continuum estimation technique introduced in Firestone et al. [7]. This technique estimates the continuum emission at the effective wavelength of the narrow-band filter by combining measurements from the two broad-band filters that bracket it. For $N501$, the $g$ and $r$-band magnitudes are linearly weighted to construct a synthetic continuum magnitude, denoted $gr$, following

This continuum estimate is then compared to the narrow-band magnitude to compute the narrow-band excess ($gr - N501$), which quantifies the strength of Ly$\alpha$ emission above the underlying continuum. If the narrow-band excess of a source exceeds the minimum narrow-band excess cut,

(corresponding to a rest-frame Ly$\alpha$ equivalent width of 20 Å), and has a significance greater than $3\sigma$, the source may be classified as an LAE candidate. We also require sources to have a broadband flux consistent with the limit expected for a pure emission-line (infinite-EW) object, thereby reducing contamination from spurious narrow-band detections. In addition to these criteria, we require our LAEs to have a narrow-band magnitude $\ge20$ mag and be brighter than the median $5\sigma$ depth of the narrow-band image to remove likely interlopers.

Once we apply Galactic dust extinction corrections, aperture corrections, star masking, data quality cuts, and our LAE selection criteria, we are left with two clean samples of LAEs for study; the SE LAE catalog, containing 5,354 sources, and the Tr LAE catalog, containing 12,521 sources.

Results

Photometric Differences

To understand the origin of the differences in LAE selection between SE and Tr, we first examine how the two methods measure photometry for the source catalogs created in § 2.4 i.e., before LAE selection. By comparing flux measurements across all detected objects in the narrow-band and broad-band filters, we isolate systematic differences in flux estimation and uncertainty modeling that are present at the source catalog level. Examining these differences provides essential context for interpreting LAE selection behavior observed between the two detection methods.

We separate the SE Source Catalog and Tr Source Catalog into three sub-samples: sources detected by both SE and Tr (70% of the SE Source Catalog, 43% of the Tr Source Catalog), sources detected only by SE (30% of the SE Source Catalog), and sources detected only by Tr (57% of the Tr Source Catalog). Direct photometric comparisons are possible only for sources detected by both SE and Tr, while sources detected only by either SE or Tr are discussed separately. For median flux values of each of these categories, see Table 2.

	SE Photometry		Tr Photometry
	SE Only	SE and Tr	Tr Only	SE and Tr
$N501$	0.986	0.619	0.179	0.715
$g$-band	0	0.511	0.140	0.573
$r$-band	0	0.762	0.216	0.849

For sources detected by both SE and Tr, we compare their flux density measurements in the narrow-band and broad-band filters to assess the level of agreement between the two methods. In particular, we focus on the $N501$ narrow-band and the $g$ and $r$ bands, as these filters are used to measure narrow-band excess, which is critical in our selection of LAEs. Figure 1 shows SE versus Tr flux density measurements for sources detected by both, with points colored by differential flux significance,

Image not extracted

Because SE and Tr measure fluxes from the same underlying imaging, their measurements are not statistically independent. We therefore use $S_{\Delta f}$ as a metric for comparing flux differences relative to the reported uncertainties rather than as a formal statistical significance. This representation highlights both the overall agreement between the pipelines and the subset of sources for which the measured fluxes differ substantially relative to their uncertainties.

We find good agreement between SE and Tr flux density measurements across all filters, with the majority of sources lying close to the 1:1 relation, particularly at low differential flux significance and high flux. In the $N501$ narrow-band, most sources cluster tightly around the 1:1 line, though a population of objects show significant deviations at higher fluxes. In this regime, Tr tends to measure higher flux densities than SE, indicating a systematic, method-dependent offset at high measured flux densities. We observe similar trends in the $g$ and $r$ broad-bands, along with more scatter and pronounced asymmetric departures from the 1:1 relation, consistent with stronger method-dependent differences in broad-band flux estimation.

To further investigate these photometric differences between SE and Tr, we now compare sources detected only by one of the two methods. Similar to SE and Tr sources, we focus on the $N501$, $g$, and $r$ filters and present the median flux values of each filter and category in Table 2.

For sources detected only by Tr, we have found that the median flux density values across all filters are lower than those of the sources detected by both SE and Tr. For sources detected only by SE, the median $N501$ flux density is higher than that of sources detected by both SE and Tr, while the median flux densities in all broad-band filters are consistent with zero. Only 9% of SE-only sources exhibit positive $g$-band flux, indicating extremely weak or undetected continuum emission. This behavior suggests that SE preferentially recovers narrow-band-dominated sources with very weak continuum, likely reflecting a higher vulnerability to narrow-band-only spurious detections. The majority of these objects are removed by our current LAE selection. However, a small fraction remains in the final LAE samples. We address these remaining contaminants by introducing an additional LAE selection criterion in § 4.3.

Categories of LAEs

We then separate our SE and Tr LAE samples into five different categories. Comparing the samples in these categories allows us to understand the differences between SE and Tr source detection methods.

We separate our SE and Tr LAE catalogs according to how they were identified in SE and Tr to investigate the systematic differences between the two methods. The five categories include:

• SE-only LAEs: Objects missing from the Tr catalog but selected as LAEs in the SE catalog.

• Tr-only LAEs: Objects missing from the SE catalog but selected as LAEs in the Tr catalog.

• SE LAEs: In both the SE and Tr source catalogs but selected as LAEs only in SE.

• Tr LAEs: In both the SE and Tr source catalogs but selected as LAEs only in Tr.

• SE+Tr LAEs: In both the SE and Tr source catalogs and selected as LAEs in both catalogs.

The LAEs in these categories come from the LAEs in SE Source Catalog or the LAEs in Tr Source Catalog, and are assigned to a category depending on if they are also found in the broader SE Source Catalog, Tr Source Catalog, or both. To illustrate how each category is derived, we present a Venn diagram as well as a pie chart with relative population sizes for each category in Figure 2.

Image not extracted

The SE-only LAEs category contains the smallest population of objects, comprising only 3% of the total LAEs included in this analysis. SE-only LAEs are those found exclusively in the SE Source Catalog with photometric measurements available only from SE. The Tr-only LAEs represent a much greater portion, which constitutes about 47% of the LAEs analyzed. These objects appear only in the Tr Source Catalog, and therefore only have data derived from Tr.

The SE LAEs and Tr LAEs categories yield 5% and 14%, respectively, of the total LAEs. Finally, the SE+Tr LAEs category makes up a significant portion of the LAEs used in this analysis, accounting for 31% of the sample. Objects in these last three categories have photometric data from both source detection methods.

By examining these five categories, we perform a detailed analysis of the differences between LAEs selected by SE and Tr in the COSMOS field at $z = 3.1$. Each category offers a distinct perspective on the systematic discrepancies and photometric uncertainties introduced by the differing source detection methods.

LAE Selection Cuts

For our LAE categories, we first examine why SE LAEs and Tr LAEs were not selected as LAEs by the other detection method. These two categories are unique in that they contain objects with photometry measured by both SE and Tr while being classified as an LAE by only one of the detection methods, making them the only samples for which we can directly evaluate the reason they do not pass our selection criteria in one method.

For each object in these categories, we test whether it satisfies each individual selection criterion described in § 2.5 to better understand why they were not classified as LAEs by both SE and Tr. Because objects may fail multiple criteria simultaneously, Table 3 reports the fraction of objects in each category that fail each criterion when considered individually, regardless of whether they also fail other cuts. We find that the narrow-band excess (NB Excess) and narrow-band excess significance (NB Excess $3\sigma$) cuts dominate the differences in LAE classification between SE and Tr, while all other criteria affect only a small fraction of sources.

SE LAEs		Tr LAEs
Primary:	0.0%	zero flux:	0.0%
Maskbits:	3.5%	flags:	0.5%
Allmask:	0.0%	imaflags:	1.5%
NOBS:	20.5%	S/N:	1.0%
Fractional Flux:	8.1%
S/N:	3.2%
Ivar:	4.7%
NB Excess:	50.8%	NB Excess:	79.2%
NB Excess $3\sigma$:	32.9%	NB Excess $3\sigma$:	48.6%
Inf EW:	0.8%	Inf EW:	0.3%
NB Mag:	0.0%	NB Mag:	0.0%
NB Depth:	1.5%	Flux Radius:	3.9%
		NB Depth:	0.5%
		Halfstacks:	4.8%

We also examine the overlap between objects rejected by the NB Excess and NB Excess $3\sigma$ criteria. Among SE LAEs, 11% of sources fail both cuts, while 40% fail only the NB Excess cut and 22% fail only the NB Excess $3\sigma$ cut. In contrast, for Tr LAEs, 43% of objects fail both criteria, with 36% failing only the NB Excess cut and just 6% failing only the NB Excess $3\sigma$ cut. This asymmetric failure pattern indicates that the two pipelines reject each other’s LAE candidates for somewhat different reasons. Tr’s rejection of SE LAEs is split between objects that fail due to narrow-band excess and those that fail due to narrow-band excess significance, perhaps reflecting how Tr’s forward-modeling affects fluxes and uncertainties in different ways for different sources. In contrast, SE’s rejection of Tr LAEs is primarily the result of reduced narrow-band excess measurements, with narrow-band excess significance failures largely accompanying the weaker excess rather than acting as an independent source of rejection.

Narrow-band Excess

Narrow-band excess, defined as the magnitude difference between the continuum and Ly$\alpha$ emission ($gr - N501$), is one of the most important criteria for identifying an LAE. A narrow-band excess greater than 0.83 mags indicates that the Ly$\alpha$ emission is sufficiently strong to be distinguished from the continuum, making the source a potential LAE candidate. We have also found in § 3.3 that our narrow-band excess criterion is the most prominent cut in our selection. Therefore, we analyze the different narrow-band excess distributions in each LAE category for both SE and Tr measurements as well as the potential reasons behind the differences.

Figure 3 shows the distributions of narrow-band excess measurements for each LAE category, with the SE LAEs, Tr LAEs, and SE+Tr LAEs illustrating the measurement differences between SE and Tr for the same set of objects. For the SE+Tr LAEs, we find that both the median narrow-band excess and the overall distribution are higher in Tr, and an object-by-object comparison shows that Tr generally measures a higher narrow-band excess than SE. This trend can also be seen in the Tr LAEs category, with many Tr-measured objects just above our narrow-band excess cut while their SE measurement counterparts primarily sit below, aligning with our results from § 3.3. The SE LAEs category exhibit a more skewed distribution in their Tr measurements, with roughly half of the objects lying below the selection cut, while the minimum and maximum measurements extend to substantially more extreme narrow-band excesses than seen in the other LAE categories. Finally, although the SE-only LAEs and Tr-only LAEs do not allow for direct cross-method comparison, their median narrow-band excess values fall between the corresponding detection method medians of LAEs they identify in other categories: SE-only LAEs lie between the SE measurements of the SE LAEs and SE+Tr LAEs, and Tr-only LAEs lie between the Tractor measurements of the Tr LAEs and SE+Tr LAEs.

Image not extracted

These results indicate that differences between SE and Tr LAE selections are strongly influenced by systematic offsets in measured narrow-band excess. Tr measurements are generally higher, leading many Tr LAEs to lie just above the narrow-band excess cut while their SE counterparts fall below. At the same time, the presence of highly skewed and extreme Tr measurements in the SE LAEs demonstrates that method-dependent outliers also contribute to selection differences. Given that the narrow-band excess is defined by the relative difference between continuum and narrow-band measurements, we next investigate which component of the photometry drives this systematic difference.

In Figure 4 we present the difference in narrow-band excess between SE and Tr ($\Delta\mathrm{NBex}$) as a function of the difference in estimated continuum magnitude ($\Delta gr$) for SE+Tr LAEs. Both differences are calculated as the Tr measurement minus the corresponding SE measurement. Points are colored by the difference in narrow-band magnitude ($\Delta\mathrm{NB}$), calculated in the same manner.

Image not extracted

A strong, approximately linear correlation is observed between $\Delta\mathrm{NBex}$ and $\Delta gr$, indicating that systematic differences in continuum estimation dominate the observed narrow-band excess offsets between SE and Tr. The color gradient reveals that while narrow-band magnitudes generally track continuum differences between the two methods, they do not exhibit a strong correlation with the narrow-band excess itself. These trends can also be seen for SE LAEs and Tr LAEs, and overall, these results demonstrate that differences in narrow-band excess measurements are primarily driven by method-dependent differences in the broad-band photometry used to estimate the continuum.

Equivalent Width

Differences in the photometric measurements produced by SE and Tr can affect not only which sources satisfy our LAE selection criteria, but also the inferred properties of the resulting LAE samples. Comparing the equivalent-width distributions of the SE and Tr LAE samples therefore provides an additional way to assess how differences between the two photometric methods influence LAE selection and the properties of the resulting catalogs.

The equivalent width (EW) of a spectral line quantifies its strength relative to the underlying continuum. It is defined as the width of a rectangle, with height equal to the continuum level, that has the same integrated area as the spectral feature. Therefore, we use EW to characterize our LAE samples, as it provides a measure of Ly$\alpha$ emission strength relative to the continuum and offers insight into the underlying galaxy properties. For our EW distributions, we follow the methodologies of Venemans et al. [39] and Guaita et al. [13]. We define the observed rest-frame Ly$\alpha$ equivalent width as

where

In Equation 6, $Q_{\rm NB}$ and $Q_{ab}$ represent the fraction of continuum flux transmitted after absorption by the Ly$\alpha$ forest in the narrow-band ($N501$) and the double-broad-band continuum estimation ($gr$), respectively. We compute $Q$ for a given filter as

where $c$ is the speed of light and $T(\lambda)$ is the filter transmission as a function of wavelength. $\tau_{\rm eff}(\lambda)$ is the effective optical depth resulting from the attenuation of continuum emission blueward of Ly$\alpha$ (ie. the Ly$\alpha$ forest [30, 22, 39]), as defined below.

and

In Equation 7, $w_{\rm BB}$ is the weight assigned to the broad-band, BB, in which the Ly$\alpha$ emission line resides. For $z = 3.1$ LAEs, this broad-band corresponds to the $g$-band. Because the exact Ly$\alpha$ wavelength is not known for individual LAEs, we use the narrow-band transmission curve as a proxy for the expected redshift distribution. DESI spectroscopic confirmations of a subset of ODIN LAEs have revealed that this is a realistic assumption (see White et al. [40]). We then compute $T_{\rm EL,filt}$ as a weighted average of the filter transmission over this distribution. $\lambda_{\rm EL}$ is the wavelength of the emission line, which is set to 501 nm, the effective wavelength of the $N501$ narrow-band filter [21].

We then fit the EW distributions to characterize the overall distribution of equivalent widths and to determine whether the LAE samples selected by SE and Tractor exhibit systematic differences. Previous studies have found that LAE EW distributions are reasonably described by exponential or Gaussian functional forms (e.g. Gronwall et al. [12]; Ouchi et al. [26]; Nilsson, K. K. et al. [23]; Guaita et al. [13]; Zheng et al. [41]; Kerutt, J. et al. [17]). We therefore use these distributions as empirical models for the expected shape of the LAE EW distribution. We fit both an exponential distribution (Equation (10)) and a Gaussian distribution (Equation (11)), as shown below.

Here, $N$ is the number of LAEs per EW bin, $C$ is a normalization constant from the fit, $EW$ is the rest-frame Ly$\alpha$ equivalent width ($EW_{\mathrm{obs}}/(1+z)$), and $w_0$ and $\sigma_{\mathrm{gauss}}$ are the characteristic scale parameters of the exponential and Gaussian distributions, respectively. Applying these fits to the EW distributions of the SE LAE Catalog and Tr LAE Catalog (Figure 5) reveals two markedly different distributions, with the corresponding fit parameters listed in Table 4.

Image not extracted

Sample (Size)	Parameter	$\chi^2_\nu$	$f_{\mathrm{cont}}$
SE (5354)	$w_0 = 67.5 \pm1$ Å	2.05	0.15
	$\sigma= 91.0 \pm1$ Å	4.22	0.253
Tr (12521)	$w_0 = 41.4 \pm0.6$ Å	9.23	0.268
	$\sigma= 48.7 \pm0.5$ Å	14.5	0.364
Tr w/o Tr-only LAEs (6135)	$w_0 = 67.7 \pm1$ Å	3.35	0.216
	$\sigma= 90.5 \pm1$ Å	5.92	0.322
SE (New Criterion) (4853)	$w_0 = 72.1 \pm1$ Å	1.63	0.137
	$\sigma= 96.7 \pm1$ Å	3.47	0.234
Tr (New Criterion) (9905)	$w_0 = 51.9 \pm0.8$ Å	6.57	0.248
	$\sigma= 66.2 \pm0.8$ Å	11.0	0.357

While neither model provides an exceptional fit, the Tr LAE Catalog shows significantly poorer agreement with both the exponential and Gaussian models, driven by a pronounced excess of low-EW sources. Since continuum-only contaminants do not exhibit strong Ly$\alpha$ emission, they are expected to populate the low-EW regime, suggesting that the observed excess arises from contamination by these sources. Assuming the fitted model represents the true intrinsic EW distribution of genuine LAEs, the Tr LAE Catalog also exhibits a substantially larger estimated contamination fraction than the SE LAE Catalog. Upon further analysis, we find that the Tr-only LAEs category is primarily responsible for shifting the EW distribution of the Tr LAE Catalog toward lower values. Removing the Tr-only LAEs produces an EW distribution that is nearly identical to that of the SE LAE Catalog, and correspondingly reduces the estimated contamination fraction. However, despite yielding nearly identical best-fit parameters to the SE LAE Catalog, the contamination estimate remains higher, indicating that the Tr LAE Catalog without the Tr-only LAEs still exhibits a greater positive deviation from the fitted model.

Discussion

High-EW Sources

The differences observed between the SE and Tr LAE samples provide insight into how source detection and photometric methods can influence the properties of selected LAE populations. At the same time, similarities between the two samples can help distinguish features intrinsic to the LAE population from those introduced by the detection method.

We find that both the SE LAE Catalog and Tr LAE Catalog contain a similar fraction of high-EW sources, with ∼ 11% of each sample exhibiting EW values greater than 240 Å. LAEs with EW values in this range are either inflated due to noisy measurements caused by spurious narrow-band detections [7], or the result of intrinsically extreme systems, such as young, metal-poor stellar populations [16]; however, their true nature requires robust spectroscopic measurements and well-understood ISM behavior.

The agreement of this fraction across both methods, despite their systematic differences in photometry and selection, suggests that this high-EW population is unlikely to be solely a product of the detection technique. This argues against the 11% being dominated by spurious narrow-band detections which should have different rates in each method. But it remains a question whether EW greater than 240 Å is primarily caused by noisy EW measurements or a genuine feature of the underlying LAE population. As a result, this subset of LAEs represents an interesting target for future work aimed at distinguishing between these possibilities.

Source Blending and Deblending

Another possible explanation for the differences between the SE and Tr LAE catalogs is their treatment of blended sources. Because Tr uses forward modeling, it may be better able to distinguish neighboring sources that SE detects as a single blended object. In cases where an LAE is blended with a non-LAE source, the additional continuum emission could reduce the measured EW, potentially causing the LAE to fail our SE selection criteria.

To investigate whether differences in source blending contribute to the discrepancies between the catalogs, we searched for neighboring sources within 2 arcseconds using the Tr Source Catalog. We identified approximately 1,400 LAEs in the Tr LAE Catalog and 500 LAEs in the SE LAE Catalog with nearby Tr sources. We then compared the SE EW distributions of LAEs with and without nearby sources to determine whether the presence of neighboring objects systematically affects the measured photometry. We find no clear differences between these distributions, suggesting that nearby sources do not have a significant systematic effect on the measured SE EWs. These results suggest that differences in source blending are unlikely to be a dominant contributor to the discrepancies between the SE and Tr LAE catalogs.

New LAE Selection Criterion

One of the primary advantages of narrow-band surveys is their ability to identify high-redshift galaxies whose continua are too faint to be detected reliably in broad-band imaging. At $z \sim3$, the observed spectrum of a galaxy is strongly affected by neutral hydrogen along the line of sight. Photons with wavelengths shorter than the Lyman limit (rest-frame 91.2 nm) are absorbed almost completely, while wavelengths between the Lyman limit and Lyα (rest-frame 121.6 nm) are partially absorbed by intervening hydrogen clouds, producing the Lyα forest. As a result, the continuum blueward of Lyα is significantly suppressed relative to redder wavelengths, creating a sharp spectral break.

For LAEs at $z \sim3.1$, the $u$-band primarily samples the strongly suppressed region near and below the Lyman limit, while the $g$-band primarily samples wavelengths between the Lyman limit and Lyα, where the continuum is less strongly suppressed by the Lyα forest. We therefore expect the $u$-band flux to generally be lower than the $g$-band flux. While this provides a physically motivated way to distinguish true high-redshift LAEs from low-redshift interlopers or continuum-dominated sources, care must be taken when incorporating a Lyman-break requirement into the LAE selection process. Requiring a clearly detected Lyman break could exclude genuine LAEs whose continua are too faint to produce a significant $u$-band dropout, thereby reducing one of the primary advantages of narrow-band selection.

Figure 6 presents the stacked median spectral energy distributions of the SE+Tr LAEs and a subset of Tr-only LAEs that do not have the expected Lyman-break behavior. While the SE+Tr LAEs exhibit the expected suppression of the continuum blueward of Ly$\alpha$, these Tr-only LAEs display markedly different behavior, including a suspicious depression in the $g$-band relative to the $u$-band. This behavior is inconsistent with the expected broad-band colors of true LAEs and motivates the use of a Lyman-break-based selection criterion.

Image not extracted

Rather than requiring that the $g$-band flux simply exceeds the $u$-band flux, we instead require consistency with the expected Lyman break corresponding to $u-g \ge1$, while accounting for the photometric uncertainties. Sources are rejected only if their measured fluxes are inconsistent with this threshold beyond the level expected from photometric noise. We therefore introduce the following selection criterion:

Figure 7 illustrates the motivation for this criterion by comparing the distribution of the $u$- and $g$-band significance for the SE+Tr LAEs and Tr-only LAEs. A strict requirement of $u-g > 1$, equivalent to $f_u < 0.4f_g$, would reject all sources with negative significance, including many for which the measured fluxes remain consistent with this threshold within the photometric uncertainties. The proposed criterion instead rejects only sources with significance less than $-1$, preserving objects whose apparent deviation from the expected Lyman break is consistent with noise.

Image not extracted

Applying this criterion to the full LAE catalogs removes 501 sources from the SE LAE Catalog and 2,616 sources from the Tr LAE Catalog, 1,818 of which belong to the Tr-only LAE category ($\sim28\%$ of all *Tr-only LAEs). This further supports the conclusion that $Tr$ is more prone to including contaminant sources than $SE$.

The impact of this criterion on the EW distributions is shown in Table 4. For both the SE LAE Catalog and Tr LAE Catalog, the best-fit exponential ($w_0$) and Gaussian ($\sigma_{\mathrm{gauss}}$) parameters increase relative to the original catalogs, indicating the preferential removal of low-EW sources. The quality of both fits also improves, with the reduced $\chi_\nu^2$ values decreasing for both the exponential and Gaussian models. Likewise, the estimated contamination fractions decrease for both catalogs. Although the improvement is more pronounced for the SE LAE Catalog, the updated Tr LAE Catalog remains more contaminated than the SE LAE Catalog and does not achieve the level of agreement obtained by excluding the Tr-only LAEs. This suggests that the proposed criterion removes a substantial fraction of contaminant sources while preserving the majority of genuine LAEs, but does not fully account for the excess contamination present in the Tr-selected sample.

Application to Other LAE Surveys

Although this criterion was developed for the ODIN N501 survey, the same methodology can be applied to other narrow-band LAE surveys by selecting broad-band filters that lie on either side of the Lyman break at the appropriate redshift. For the ODIN N673 survey ($z \sim4.5$), the Lyman break shifts between the $g$- and $r$-bands, making an analogous criterion based on the requirement that $f_r > f_g$ appropriate. As with the N501 survey, this comparison should account for the combined photometric uncertainties to avoid rejecting genuine LAEs whose continua are too faint to produce a statistically significant dropout.

For the ODIN N419 survey ($z \sim2.4$), the Lyman break lies blueward of the available broad-band coverage and therefore cannot be directly measured. However, the same approach can still be used to identify suspicious sources with unexpectedly high $u$-band fluxes relative to the $g$-band. Although this does not provide a direct Lyman-break test, it offers an additional diagnostic for identifying likely low-redshift interlopers within the LAE sample.

More generally, this method can be applied to any LAE survey with broad-band observations spanning both sides of the expected Lyman break. Surveys with multiple broad-band filters are particularly well suited to this approach because they can use two broad-band filters to estimate the continuum ([7]) while simultaneously performing an independent Lyman-break check. Rather than requiring the detection of a Lyman break, which may exclude continuum-faint LAEs, this approach instead identifies sources that exhibit an obvious lack of the expected Lyman-break signature.

Conclusions and Future Work

In this paper, we compared SE and Tr source detection methods for the identification and photometric characterization of LAEs in the 9 deg$^2$ ODIN-COSMOS field at $z = 3.1$. Our main conclusions are summarized below.

1. Comparison of source catalog photometry shows that SE and Tr generally agree across most flux ranges, though systematic differences emerge at both the high and low-flux extremes. At higher flux densities, Tr tends to measure systematically larger flux densities than SE, while at lower flux levels the detection methods diverge in which sources they detect. These disagreements are most apparent for narrow-band-dominated sources with weak continuum emission, suggesting that differences in source detection sensitivity and uncertainty estimation contribute to the observed asymmetries in LAE selection between the two methods.

2. We find that differences between the SE and Tr LAE catalogs are primarily driven by systematic offsets in narrow-band excess measurements, with Tr generally measuring higher values and selecting a number of LAE candidates that is nearly double the amount of SE. This behavior is driven by differences in continuum estimation, where Tr tends to measure lower flux density in the broad-bands — particularly the $g$ and $r$ filters that are used in estimating narrow-band excess — leading to larger narrow-band excess values.

3. Analysis of equivalent width (EW) distributions further implies that the Tr LAE Catalog contains a significant excess of low-EW sources, resulting in poorer agreement with both exponential and Gaussian models. This excess is largely driven by the Tr-only LAEs population which makes up half the Tr LAE Catalog, indicating that Tr is more prone to including continuum-dominated sources that are not detected by SE. Removing these sources brings the Tr EW distribution into close agreement with that of SE and lowers the overall reduced $\chi^2$ values, indicating a reduction in contamination. However, the estimated continuum fraction still remains larger than that of the SE LAE Catalog.

4. We also find that 11% of both the SE LAE Catalog and Tr LAE Catalog have EW measurements greater than 240 Å, which is the speculated range for LAEs composed of young, metal-poor stellar populations [16]. This suggests a high EW-tail is not unique to one detection method and may therefore represent a promising population for future study.

5. We introduce a new selection criterion based on the expected Lyman break behavior of high-redshift galaxies at $z \sim3$, where absorption from intervening hydrogen in the Ly$\alpha$ forest suppresses continuum emission blueward of the Lyman limit, producing a $u-g$ color consistent with a Lyman break. By requiring consistency with this expected color while accounting for the photometric uncertainties, the criterion most strongly impacts the Tr-only LAEs category within the Tr LAE Catalog, removing $\sim28\%$ of these sources. The resulting improvement in the EW distribution, including lower contamination fractions and better agreement with the fitted models, supports the conclusion that Tr includes a larger fraction of contaminants and demonstrates the usefulness of incorporating expected Lyman break behavior into LAE selection.

Taken together, these results indicate that the SE LAE Catalog provides a cleaner sample of $z \sim3.1$ LAEs than the Tr LAE Catalog for the ODIN survey. Although Tr offers several advantages through its forward-modeling approach, our implementation produces a larger population of continuum-only contaminants that broadens the EW distribution and increases the inferred contamination fraction. While removing the Tr-only LAEs and applying our additional Lyman break consistency criterion both improve the Tr-selected sample, the resulting catalog still does not achieve the level of performance obtained with the SE LAE Catalog. For this reason, we conclude that SE is the preferred source detection and photometry method for LAE selection in ODIN.

These conclusions are specific to the ODIN imaging and photometric pipeline used in this work and should not be interpreted as demonstrating that SE is generally preferable to Tr for LAE selection. Targeted spectroscopic follow-up of sources will ultimately be necessary to determine their true nature and directly measure the purity and completeness of each LAE sample.

Acknowledgments

This work is based on observations at Cerro Tololo Inter-American Observatory, NSF’s NOIRLab (Prop. ID 2020B-0201; PI: K.-S. Lee), which is managed by the Association of Universities for Research in Astronomy under a cooperative agreement with the National Science Foundation. BT, EG, and NF acknowledge support from NSF grant AST-2206222. NF acknowledges support from the NSF Graduate Research Fellowship Program under Grant No. DGE-2330066. KSL acknowledges financial support from the National Science Foundation under grant Nos. AST-2206705, AST-2408359, and from the Ross-Lynn Purdue Research Foundations. HSH acknowledges support from the National Research Foundation of Korea (NRF) funded by the Korea government (MSIT; RS-2026-25482692) and the Global-LAMP Program funded by the Ministry of Education (RS-2023-00301976). LG gratefully acknowledges financial support FONDECYT regular project number 1230591, from ANID - MILENIO - NCN2024_112, from the ANID BASAL project FB210003.