betaFeedback

WorksComputer Science

Beyond Pattern Matching in 3D MLLMs: Part-Level Functional Reasoning for Generalizable Recognition

Abstract

3D MLLMs recognize objects via pattern matching, which fails when geometrically similar objects require functional reasoning to disambiguate (e.g., dresser vs. wardrobe). Our per-sample Geometric Confusion Score (GCS) exposes this failure: baseline accuracy is 33.7% on ModelNet40's most ambiguous 20% versus 84.0% on its least ambiguous 20%. We identify a function-as-annotation pattern, in which functional language decorates a known identity instead of supporting inference, and hypothesize that reversing this information flow should concentrate improvements on the most ambiguous samples. PFIR (Part-level Functional Inference for Recognition) reverses that flow. Built on PartNeXt part segmentation, its data pipeline produces 507K QA pairs with functional CoTs (geometry →\to function →\to identity), and its progressive curriculum culminates in dual-objective GRPO: strict functional reasoning and descriptive grounding for Objaverse's diverse distribution. The two RL objectives yield nearly orthogonal task vectors (cos⁡=0.056\cos = 0.056), enabling composition via Task Arithmetic. Under a strict dual OOD protocol (Objaverse →\to ModelNet40; XYZRGB →\to XYZ-only), PFIR raises zero-shot accuracy from 60.5% to 66.7%, with gains concentrated 2.6×2.6\times more on geometrically ambiguous samples—supporting part-level functional reasoning as an effective route to more generalizable 3D recognition. Code: https://github.com/this233/pfir-3d.

Paper details

vlnc:W_xq25sf7k
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
arXiv nonexclusive license
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…