Tabular Foundation Models Dramatically Outperform Bespoke Photometric Redshift Estimators

Estimating galaxy redshifts (a proxy for distance) from imaging data alone, also known as photometric redshifts (photo-$z$'s), is essential for studies of extragalactic astrophysics and cosmology. However, data-driven estimators rely on spectroscopic training datasets, which are scarce and systematically biased toward bright and easily observed galaxies. In this work, we show that tabular foundation models (TFMs), whose synthetic pretraining amounts to a learned prior over regression problems, can outperform bespoke estimators using only in-context learning. Evaluating on a widely used benchmark dataset, we find that TFMs predicting full probability distributions in a single forward pass systematically outperform bespoke estimators across both representative and biased training data regimes, matching or exceeding baselines trained on over $40{,}000$ galaxies with just $1{,}000$ training context examples. We also show that the TFM advantage is largest when little to no calibration data are available, and that it persists even when the bespoke baselines are supplemented with up to $10{,}000$ representative calibration spectra. This shows that TFMs offer a sample-efficient framework for upcoming imaging-only astronomical surveys, where observational selection bias can be mitigated simply by including a small representative calibration sample in context.