TopoField: Technical Report for ECCV 2026 SceneFun3D Challenge

Introduction

SceneFun3D [1] asks a model to locate nine categories of functional interaction elements, such as knobs, buttons, handles, and sockets, in complete indoor laser scans. Each prediction requires a category, a point mask, and a confidence score.

These elements occupy only a small portion of their host objects, and adjacent elements can share material, color, and local geometry. The task therefore requires not only recognizing an interactable object, but also recovering the spatial extent of the part that affords an action. Related efforts emphasize fine-grained video understanding [2], region-focused visual representations [3], and entity-level alignment in composed retrieval [4, 5, 6].

Two coupled difficulties arise after a point-wise semantic predictor such as VOLT [7]. Small and partly occluded parts make per-point probabilities noisy: a high threshold truncates weak boundaries, whereas a low threshold can merge the target with its host or a neighbor. In addition, the predictor returns a continuous probability field, while the benchmark evaluates discrete masks sorted by confidence. Even with reasonable semantics, proposal construction and ordering can therefore cause a large field-to-instance error.

We quantify this gap on 30 validation scenes containing 561 evaluable instances. The examined pool has an IoU-0.5-valid candidate for 67.2% of the instances, and top-1 coverage is 20.0%. These complementary statistics characterize candidate-pool coverage and the overlap achieved by top-ranked candidates.

We present TopoField, whose method follows the two stages in Figure 1. Stage 1, TTA-based Semantic Consensus, applies the shared predictor to 13 test-time forward passes over 12 distinct rotation–scale outcomes. Each fragment retains a processed-point index that scatters its softmax output into a common point-by-class array; accumulated probabilities are divided by coverage count and the independent caches are renormalized and fused. Stage 2, Geometry-aware Instance Decoding, selects a class-specific operator from argmax connectivity, threshold-support connectivity, or a seeded graph basin. The basin branch separates high-confidence cores from broader supports, builds a symmetric radius-pruned 32-nearest-neighbor graph, and assigns support points to seed components by accumulated graph-path length. Candidates are then scored and suppressed class by class in the original scan point order.

Our contributions are as follows.

• We formulate the gap between point-wise semantics and the benchmark’s ranked instance-set output, and quantify it through complementary any-candidate and top-$k$ coverage statistics.

• We organize the system as two explicit stages that mirror its implementation: index-aligned test-time semantic consensus and geometry-aware instance decoding. The first stage makes point identity and cache fusion explicit; the second exposes core selection, graph construction, geodesic seed assignment, and candidate ranking.

• The parameter-free decoder achieves 28.286% AP, 41.383% AP$_{50}$, and 47.558% AP$_{25}$.

Related Work

3D instance segmentation. PointGroup [8] groups point-wise semantics with center offsets in two coordinate spaces, while SoftGroup [9] reduces hard-label error propagation with soft semantic grouping. Query-based methods such as Mask3D [10] and SPFormer [11] learn instance or superpoint queries and directly predict masks. These methods jointly learn representation, grouping, and refinement [12, 13]. In visual tracking, complementary approaches explore window-based attention [14], correlative masked modeling [15], and query-gated feature selection [16]. TopoField converts a fixed semantic predictor’s output into a ranked instance set through deterministic grouping and confidence-based ranking.

Functional interaction element segmentation. SceneFun3D [1] extends indoor scene understanding from complete objects to functional parts such as handles, knobs, and buttons. These parts are small, often partly occluded, and geometrically similar to their neighbors [17, 18]. Related work on composed video retrieval models shared and differential semantics [19] and calibrates directional anchors in composed representations [20]. VOLT [7] supplies the point-wise semantics in our system; TopoField constructs and ranks instance proposals from these semantic predictions. In video tracking, semantic consistency strengthens point correspondence [21], while temporally coherent object flow supports multiframe object association [22].

Graph-based region partitioning. Watershed-like methods propagate labels from seeds under a local adjacency structure [23]. A sparse point graph constrains propagation to 3D neighborhoods and prevents unreachable regions from borrowing labels across empty space [24, 25, 26]. Graph-structured relations also support heterogeneous nearest-neighbor retrieval [27] and collaborative motion reasoning in multi-object tracking [28]. TopoField applies this idea to a class-conditioned support: cores supply seeds, a radius-pruned symmetric kNN graph defines paths, and multi-source shortest paths assign support points to seeds. Connected components first isolate disconnected supports; geodesic propagation then handles connected supports with several cores.

Test-time augmentation. Test-time augmentation reduces variance by transforming the input and aggregating the corresponding outputs [29, 30, 31, 32]. For per-point prediction, point identity must be preserved across augmentation and test-time voxelization. Stage 1 therefore scatters probabilities through retained point indices, normalizes by coverage, and fuses the caches before Stage 2 consumes the consensus field. Complementary evidence is also integrated through coupled multimodal state-space models [33]. Egocentric video understanding systems further explore category-conditioned inference and test-time consistency [34], as well as capability-based reasoning routes [35].

Noise-aware visual matching. Reliable visual matching depends on identifying task-relevant evidence and estimating the confidence of candidate associations. Research on noisy correspondence studies these issues in image-text retrieval, where training pairs can contain incorrect semantic associations. Huang et al. [36] introduce the Noisy Correspondence Rectifier, which partitions training pairs according to their estimated reliability and adaptively rectifies their correspondence through co-teaching. Qin et al. [37] develop an evidential learning framework that models correspondence uncertainty and adjusts negative-sample learning through a robust dynamic hinge loss. For composed image retrieval [38], ConeSep [39] localizes noisy correspondences through geometric fidelity estimation and performs boundary-based targeted unlearning. Air-Know [40] integrates expert-prior arbitration with knowledge internalization to guide robust representation learning, while INTENT [41] combines visual invariant composition with discriminative learning to address cross-modal correspondence noise and modality-inherent interference. Together, these approaches highlight evidence selection and confidence estimation as central components of reliable matching. In TopoField, these considerations motivate the use of semantic consensus, confident core selection, and class-aware candidate scoring for constructing ranked instance predictions.

Method

TopoField is a deterministic decoder after a point-wise predictor. Stage 1 stabilizes the probability field and resolves index misalignment between augmented inputs; Stage 2 converts the field into instance masks and scores without updating predictor parameters.

Problem formulation

Given a scan with $N$ points, the predictor produces $P_i(c)$ for point $i$ and class $c \in\{0,\ldots,9\}$, where class 0 is background and classes 1, …, 9 are target categories.

The decoder returns

where $M_\ell$ uses the original point order and $s_\ell$ orders candidates. Only foreground instances are emitted.

Stage 1: TTA-based Semantic Consensus

Augmented forward passes. The released profile runs 13 forward passes. Its first twelve are the Cartesian product of four vertical-axis rotations, $\rho\in\{0,\frac{1}{2},1,\frac{3}{2}\}$, and three scales, $s \in\{0.95,1.00,1.05\}$. The thirteenth is a double-axis flip. Flipping both horizontal axes equals a 180-degree rotation, so the passes contain 12 distinct spatial transforms.

Index-aligned accumulation. For pass $a$ and voxelized fragment point $j$, let $\pi_a(j)$ be its retained processed-point index. Test-time voxelization can place several fragments under the same index. We accumulate softmax probabilities in a common $N \times10$ array

and divide by the number of contributions to each processed point:

Retained processed-point indices align predictions across coordinate and normal transformations, enabling direct accumulation in the common point array.

Cache fusion. Each pass writes an independent cache. For positive weights $w_a$ and $W = \sum_a w_a$, we combine the caches and normalize each row:

All 13 passes receive equal weights. Since index 12 duplicates index 2 geometrically, their combined weight is 2/13.

Stage 2: Geometry-aware Instance Decoding

Class-conditioned route. The class-conditioned decoder selects among argmax connectivity, foreground-threshold connectivity, and a seeded graph basin for each category. The basin branch in Stage 2 of Figure 1 uses confident cores and graph connectivity to partition support regions.

(a) Class-wise support and core. For each foreground class $c$, let

Given support threshold $\tau_s$ and core threshold $\tau_h$, with $\tau_h\geq\tau_s$, define

The support retains weak boundary evidence while the core supplies reliable seeds. The representative recipe uses $\tau_s=0.15$, $\tau_h=0.40$, at least five support points, and at least three core points.

(b) Sparse 3D graph. We build a graph $G_c=(S_c,E_c)$ only over the support points. Let $\mathcal{N}_k(i)$ be the $k$-nearest-neighbor set of $i$ in $S_c$. We take the union of outgoing and incoming neighbor relations and remove long edges:

The implementation uses $k=32$ and weight $w_{ij}=\|p_i-p_j\|_2$, bounded below by $10^{-12}$. The resulting sparse, symmetric graph constrains propagation to local 3D neighborhoods.

(c) Multi-source geodesic assignment. Connected components of $G_c$ are formed first. Inside each support component, the core points induce seed components $Q_{cjm}$. If a support component contains more than one seed component, we run the multi-source Dijkstra step from all seeds and assign each support point to the seed with the smallest accumulated graph distance:

The resulting candidate is

With one seed component, the whole support component forms a single candidate.

(d) Candidate ranking and suppression. For a materialized candidate $M$, the primary score is the mean consensus probability of its class:

The decoder also supports quantile and hybrid pooling, and a class-wise size factor,

where $r_c$ is a reference point count and $\alpha_c = 0$ recovers the unscaled score. Exact duplicate masks keep the highest-scoring copy. For overlapping same-class candidates, soft NMS uses

Candidates are filtered and suppressed class by class, and the final scene list is returned in original point order.

Implementation details

VOLT predicts at a 1 cm voxel resolution from XYZ, RGB, and normals. It has 12 transformer blocks, 768-dimensional embeddings, and 12 heads, initialized from ScanNet200 weights. Training uses AdamW ($3 \times10^{-4}$, weight decay 0.05), label-smoothed cross-entropy, multiclass Lovász loss, and EMA (0.999). Test inference uses frozen probability caches without gradient updates. Unless stated otherwise, Stage 1 is fixed while Stage 2 is varied.

Experiments

Setup

SceneFun3D [1] contains 160 training, 30 validation, and 30 test scenes over nine categories. We train VOLT with XYZ, RGB, and normals at 1 cm. Stage 1 is used only at inference. The official metric is AP over IoU 0.50–0.95, with AP50 and AP25 also reported. $R_{\mathrm{any}}$ is the fraction of ground-truth instances whose candidate pool contains an IoU-0.5-valid mask; $R_k$ restricts this test to the class-wise top-$k$ candidates.

Candidate diagnosis

Using fixed epoch-80 EMA 13-TTA caches, Table 1 shows a 47.2-point gap between any-candidate and top-1 coverage. The two measurements distinguish candidate-pool coverage from top-ranked mask quality and provide complementary criteria for evaluating Stage 2.

Diagnostic	Value
Foreground-point precision / recall	68.4 / 60.1
$R_{\mathrm{any}}$@0.5	67.2
$R_1$@0.5	20.0
$R_3$@0.5	38.3
$R_5$@0.5	46.3

Validation and test results

Table Table 2 reports validation results for successive decoder configurations. Support/core selection gives the largest improvement, and graph geodesic partitioning further separates adjacent candidates. Class-conditioned decoding raises AP from 37.60 to 38.45 on the development split and from 41.86 to 42.31 on the holdout split, reaching 39.67 AP across all 30 validation scenes.

Configuration	Development (20)	Holdout (10)	Full validation (30)
Baseline	35.21	39.47	36.58
+ Dual-threshold	36.89	41.02	38.14
+ Graph geodesic	37.60	41.86	38.92
+ Class-conditioned	38.45	42.31	39.67

Table Table 3 reports official submissions from two EMA checkpoints with robust and aggressive post-processing. Each checkpoint uses its associated saved seed configuration. The epoch-100 aggressive configuration achieves 28.286% AP, 41.383% AP₅₀, and 47.558% AP₂₅.

EMA epoch	Post-processing	AP	AP₅₀	AP₂₅
80	Robust	27.919	41.373	45.878
80	Aggressive	27.640	41.103	46.364
100	Robust	28.270	40.847	46.761
100	Aggressive	28.286	41.383	47.558

Discussion

Decoder components. The successive configurations in Table 2 increase development-split AP by 1.68 points with support/core selection, 0.71 points with graph geodesic partitioning, and 0.85 points with class-conditioned decoding. Together, these components raise full-validation AP from 36.58 to 39.67.

Method behavior. The two stages perform complementary operations. Stage 1 aggregates index-aligned point predictions into a semantic consensus field. Stage 2 constructs and ranks instance masks using class-conditioned connectivity and graph-geodesic partitioning. Support/core selection retains boundary evidence and supplies confident seeds, while geodesic assignment partitions multi-core support regions according to their 3D connectivity.

Conclusion

We presented TopoField, a two-stage deterministic decoder for functional interaction element segmentation. Stage 1 turns multiple test-time forward passes into a stable semantic field by retaining processed-point indices, accumulating softmax probabilities in a common point array, normalizing repeated fragment contributions by coverage, and fusing independent caches. Stage 2 uses class-conditioned instance decoding; its seeded graph-basin branch separates a high-confidence core from a broader support region, constrains propagation with a symmetric radius-pruned point graph, and assigns support points to seed components by graph-geodesic distance. The validation analysis characterizes candidate coverage and ranking, and the combined decoder achieves 39.67% validation AP and 28.286% official-test AP. The decoder adds no learnable parameters to the point-wise predictor.