betaFeedback

WorksComputer Science

ReflectScene:MLLM-Guided Closed-Loop Diffusion for 3D Indoor Scene Generation

Abstract

Text-conditioned 3D indoor scene generation requires layouts satisfying geometric, functional, and semantic constraints. However, existing diffusion methods rely primarily on static textual conditions, limiting their ability to assess evolving scenes and translate semantic discrepancies into executable object-level corrections. MLLM-based post-hoc refinement provides semantic feedback but remains decoupled from denoising, potentially disrupting scene coherence. We propose ReflectScene, an MLLM-guided closed-loop diffusion framework. First, we introduce Hierarchical Prior-Grounded Scene Understanding that aligns structured layouts with multi-view visual evidence, to interpret object-specific context and scene global semantics via a frozen multi-modal large language model (MLLM). Second, we propose MLLM-Guided Semantic Reward Evaluation to distill MLLM's diagnoses and assessments on possible candidate corrections into a lightweight Reward Predictor, enabling correction decision making with reduced computation burden. Finally, Decision-Conditioned Trajectory-Consistent Diffusion is proposed to integrate selected corrections through transient conditioning and persistent clean-estimate projection, correcting object layout in the generated scene while preserving non-target arrangements throughout denoising. Experiments on 3D-FRONT demonstrate improved functional consistency and semantic alignment while maintaining competitive generation quality and diversity.

Introduction

Text-conditioned 3D indoor scene generation aims to synthesize realistic furniture layouts from natural language, with applications in virtual reality, game development, and embodied AI [9, 18, 30]. Beyond geometric plausibility, realistic scenes must satisfy functional and semantic requirements, including meaningful spatial relationships and constraints. textual dependencies [18, 39].

As illustrated in Figure 1, recent learning-based approaches have achieved promising results in indoor scene synthesis. DiffuScene [28] employs denoising diffusion to jointly model furniture attributes, while InstructScene [19] incorporates semantic graph priors for controllable generation. These methods primarily rely on static textual conditions or predefined structural priors, and present limited ability to reassess intermediate denoising states and correct emerging semantic discrepancies. Consequently, generated layouts may remain geometrically plausible while failing to satisfy the functional and contextual intent of the input description.

Figure 1. Motivation of ReflectScene. (A) One-shot diffusion relies on fixed text conditioning, lacking feedback to correct unsatisfied relations. (B) Post-hoc repair acts after generation and may disrupt existing layouts. (C) ReflectScene converts step-wise semantic assessments into executable object-level corrections, buffering and sequentially executing decisions every 50 steps to guide subsequent denoising while preserving non-target arrangements.

Meanwhile, multimodal foundation models provide rich semantic and functional knowledge for scene understanding and reasoning [3, 15, 40]. Function2Scene [29] combines geometric measurements, LLM reasoning, and VLM assessment, while SceneReVis [38] employs vision-grounded diagnosis and executable actions for iterative scene refinement. Despite these advances, semantic reasoning and diffusion-based generation remain largely decoupled: static conditions cannot respond to evolving semantic discrepancies, while post-hoc corrections are not inherently propagated through subsequent denoising. Bridging this gap requires reliable intermediate scene understanding, context-aware evaluation of candidate corrections, and persistent integration of accepted object-level decisions into the generative trajectory, without unnecessarily disrupting satisfactory scene content. Moreover, repeatedly invoking large MLLMs to evaluate multiple candidates introduces substantial computational overhead.

In this paper, we present ReflectScene, an MLLM-guided closed-loop diffusion framework that integrates semantic understanding, reward evaluation, object-level decision-making, and corrective generation into a single reverse-diffusion trajectory. To achieve this goal, it requires faithfully representing the evolving layout, efficiently evaluating context-dependent candidate corrections, and integrating selected corrections into the reverse-diffusion trajectory without disrupting satisfactory scene content. To represent the intermediate layout, we develop Hierarchical Prior-Grounded Scene Understanding, which combines structural-semantic alignment, multi-view visual supervision, and geometry-aware metric learning to produce object-specific context and global scene representations. To make candidate evaluation tractable, we introduce MLLM-guided Semantic Reward Evaluation and Object-Level Decision, which distills a frozen MLLM evaluator into a lightweight Reward Predictor that estimates prompt alignment, functional coherence, and semantic consistency while retaining candidate-specific scene understanding. To maintain the effect of selected corrections during generation, we develop Decision-conditioned Trajectory-consistent Diffusion, which feeds accepted decisions back into the reverse-diffusion trajectory through transient guidance and persistent correction constraints, while preserving non-target attributes. These components are interleaved at denoising steps, and corrections accumulated within each buffering interval are then executed sequentially, enabling progressive refinement along a single reverse-diffusion trajectory.

Our main contributions are summarized as follows:

  • We introduce ReflectScene that integrates semantic assessment and executable object-level corrections into a single reverse-diffusion trajectory, bridging the gap between static conditional generation and decoupled post-hoc refinement.

  • We develop hierarchical scene understanding and MLLM-guided reward mechanism that combines structural, semantic, and metric representations with reward distillation, enabling efficient evaluation and context-aware selection of geometrically feasible corrections.

  • We propose a Decision-conditioned Trajectory-consistent Diffusion mechanism that combines transient decision conditioning and decision-effect consistency learning to complete selected corrections while preserving non-target scene content.

Related Work

3D Indoor Scene Synthesis

3D indoor scene synthesis aims to generate plausible furniture arrangements with semantic coherence and spatial consistency. Early approaches primarily focus on learning object-centric scene distributions. For example, ATISS [22] models indoor scenes as unordered object sets using autoregressive transformers, while DiffuScene [28] employs denoising diffusion to jointly generate object semantics, spatial attributes, and geometric features. Such generative representations [13, 36] enable flexible modeling of diverse furniture configurations and scene compositions.

Recent studies [10, 20, 37] further incorporate structural priors to improve layout plausibility and controllability. For instance, InstructScene [19] introduces semantic graph priors to capture object relationships, while MMGDreamer [35] employs mixed-modality graphs for geometry-controllable generation. SemLayoutDiff [26] further incorporates architectural constraints into semantic layout diffusion. These advances [6, 7, 24, 34] enhance the modeling of object interactions and spatial dependencies. Nevertheless, these approaches largely rely on learned distributions or predefined priors, with limited explicit feedback on evolving scene configurations.

Feedback-Guided Scene Refinement

Improving spatial and functional plausibility has motivated various guidance and refinement strategies. One direction incorporates explicit physical constraints into generation. For example, PhyScene [33] introduces physics-and interactivity-based guidance into diffusion sampling to reduce collisions and improve spatial feasibility. Such guidance[16, 25] provides explicit objectives for encouraging physically consistent scene configurations.

Another direction explores reasoning-driven scene refinement through iterative feedback. For instance, SDesc3D[8] combines multi-view structural priors and functionality-aware layout grounding with iterative reflection-rectification to resolve spatial inconsistencies in scenes generated from short descriptions. Similarly, SceneReVis [38] employs vision-grounded diagnosis and corrective actions for iterative scene refinement, emphasizing the role of contextual reasoning in layout optimization[17]. However, converting iterative feedback into persistent object-level corrections without disrupting unaffected scene content remains challenging.

Multimodal Priors for 3D Understanding and Generation

Pretrained multimodal models offer rich semantic knowledge, motivating efforts[14, 23, 27, 32] to connect 3D representations with language models. For example, PointLLM [31] aligns point-cloud features with language representations for 3D object understanding, while 3D-LLaVA [5] introduces a dedicated 3D connector for scene-level multimodal reasoning. These approaches[1, 2, 4] facilitate the integration of geometric information with pretrained semantic knowledge.

Beyond understanding, pretrained multimodal priors have also been explored for generative modeling. For instance, RepFusion [21] incorporates MLLM priors into diffusion-based image generation through dynamic conditioning on evolving visual representations, highlighting the potential of state-dependent multimodal guidance. However, translating multimodal semantic assessments into precise and executable geometric corrections remains underexplored.

Method

ReflectScene formulates text-conditioned 3D scene generation as a closed-loop process, in which intermediate layouts are interpreted, corrected and fed back into the same reverse-diffusion trajectory. To achieve this goal, ReflectScene is composed of three interconnected components: Hierarchical Prior-Grounded Scene Understanding, MLLM-guided Semantic Reward Evaluation and Object-Level Decision, and Decision-Conditioned Trajectory-Consistent Diffusion.

As illustrated in Figure 2, given a text prompt and an initial noise state, the diffusion model progressively generates a scene layout. At each denoising step checkpoint, the Hierarchical Prior-Grounded Scene Understanding module

Figure 2. Overview of ReflectScene. ReflectScene integrates hierarchical scene understanding and MLLM-guided decisions into object-centric diffusion. At each denoising step, intermediate clean estimates are assessed to select object-level actions, providing transient conditioning, while buffered non-STOP actions are executed every 50 steps to update a persistent correction state. This state is projected onto subsequent clean estimates before DDPM posterior sampling, with decision-effect consistency learning promoting target recovery and non-target preservation.

constructs structured object-specific context representations and global scene representation of the intermediate layout. These hierarchical representations are then passed to MLLM-guided Semantic Reward Evaluation and Object-Level Decision to evaluate candidate corrections and select an object-level correction. The Decision-conditioned Trajectory-consistent Diffusion module then incorporates the selected decision into subsequent denoising while preserving accepted corrections along the reverse-diffusion trajectory.

Hierarchical Prior-Grounded Scene Understanding

At each selected denoising step, ReflectScene evaluates the intermediate layout before selecting an object-level correction, requiring a representation that jointly captures object semantics and geometry and the semantic priors needed for functional evaluation. To this end, we introduce a Hierarchical Prior-Grounded Scene Understanding module, which first constructs a set of hierarchical representations of the intermediate layout and then adapts it to a frozen MLLM through complementary semantic and metric feature paths. The resulting representation supports candidate-specific reward evaluation and object-level decision making. Moreover, a multi-view alignment pretraining scheme is introduced for facilitating the adaptation of scene representations to MLLM’s hidden space.

Hierarchical Scene Representation. For each object ii, we combine its category embedding E(ci)\mathbf{E}(c_i), geometric attributes gi\mathbf{g}_i, and encoded diffusion timestep PE(t)\mathbf{PE}(t) to obtain an initial object feature,

hi=WcE(ci)+Wggi+WtPE(t)∈R128,(1)\mathbf{h}_i = \mathbf{W}_c\mathbf{E}(c_i) + \mathbf{W}_g\mathbf{g}_i + \mathbf{W}_t\mathbf{PE}(t) \in\mathbb{R}^{128}, \tag*{(1)}

where Wc\mathbf{W}_c, Wg\mathbf{W}_g, and Wt\mathbf{W}_t denote three independent linear projections applied on E(ci)\mathbf{E}(c_i), gi\mathbf{g}_i and PE(t)\mathbf{PE}(t).

Meanwhile, for each object pair (i,j)(i,j), we construct a pairwise relation descriptor from their geometric attributes:

uij=[Δpij,dijxz,bi,bj,sin⁡Δθij,cos⁡Δθij](2)\mathbf{u}_{ij} = [\Delta\mathbf{p}_{ij}, d_{ij}^{xz}, \mathbf{b}_i, \mathbf{b}_j, \sin\Delta\theta_{ij}, \cos\Delta\theta_{ij}] \tag*{(2)}

where Δpij\Delta\mathbf{p}_{ij} is the 3D relative displacement, dijxzd_{ij}^{xz} is the horizontal distance, bi,bj∈R3\mathbf{b}_i,\mathbf{b}_j \in\mathbb{R}^{3} are the object sizes, and Δθij=θj−θi\Delta\theta_{ij} = \theta_j - \theta_i is their relative yaw angle.

The short-range context for each object zishort\mathbf{z}_i^{\mathrm{short}} is aggregated from these pairwise descriptors by using masked attention:

zishort=Attn⁡(hi,{MLP⁡r(uij)}j=1Nk,Minbr),(3)\mathbf{z}_i^{\mathrm{short}} = \operatorname{Attn}\left(\mathbf{h}_i,\{\operatorname{MLP}_r(\mathbf{u}_{ij})\}_{j=1}^{N_k},\mathbf{M}_i^{\mathrm{nbr}}\right), \tag*{(3)}

where Minbr\mathbf{M}_i^{\mathrm{nbr}} masks all objects except the up to eight nearest neighbors within a 2-meter radius of object ii.

To capture long-range interactions and scene-level context, we prepend a learnable global query qg\mathbf{q}_g to the sequence of object features and process the sequence with a Transformer:

({zilong}i=1Nk,zglobal)=ψ(qg,{[hi∥zishort]}i=1Nc)(4)\left(\left\{z_i^{\mathrm{long}}\right\}_{i=1}^{N_k},z^{\mathrm{global}}\right)=\psi\left(q_g,\left\{\left[h_i\Vert z_i^{\mathrm{short}}\right]\right\}_{i=1}^{N_c}\right) \tag*{(4)}

where ψ(⋅)\psi(\cdot) is a two-layer Transformer with hidden dimension 128, and [⋅∥⋅][\cdot\Vert\cdot] denotes feature concatenation. The Transformer outputs object-specific long-range features {zilong}\left\{z_i^{\mathrm{long}}\right\} and the global scene representation zglobalz^{\mathrm{global}}

Prior-Grounded Scene Understanding. To incorporate the hierarchical scene representation into the frozen MLLM, we project the obtained global scene feature zglobalz^{\mathrm{global}} and object-level long-range features {zilong}i=1Nk\left\{z_i^{\mathrm{long}}\right\}_{i=1}^{N_k}, and append the resulting continuous scene tokens to the prompt embeddings. Let Etxt(ps)E_{\mathrm{txt}}(p_s) denote the prompt token embeddings obtained from the MLLM’s input embedding layer. The joint text-scene sequence is constructed as Tsem=[Etxt(ps);Wszglobal;{Wszilong}i=1Nk]T_{\mathrm{sem}}=[E_{\mathrm{txt}}(p_s);W_s z^{\mathrm{global}};\{W_s z_i^{\mathrm{long}}\}_{i=1}^{N_k}], where WsW_s is a learnable projection that maps the scene features into the 2048-dimensional hidden space of the MLLM. After providing this combined sequence to a frozen MLLM, we extract the final-layer hidden states at the positions of the appended scene tokens as

(hgL,{hiL}i=1Nk)=MLLM⁡ϕ,hidden(L)(Tsem)(5)\left(h_g^L,\left\{h_i^L\right\}_{i=1}^{N_k}\right)=\operatorname{MLLM}_{\phi,\mathrm{hidden}}^{(L)}\left(T_{\mathrm{sem}}\right) \tag*{(5)}

where LL denotes the final MLLM layer. The resulting hgLh_g^L and hiLh_i^L are the MLLM-adapted global scene feature and the corresponding MLLM-adapted feature for object ii.

To further encode object geometry in the MLLM feature space, we introduce a Metric Adapter that transforms the geometric attributes gig_i into a geometry-aware query:

qim=Wmgi+MLP⁡m(gi)(6)q_i^m=W_m g_i+\operatorname{MLP}_m(g_i) \tag*{(6)}

where WmW_m and MLP⁡m\operatorname{MLP}_m are learnable branches whose outputs have the same dimensionality as the MLLM hidden states. To incorporate the information encoded in the semantic token sequence, we cache the keys and values produced by the first layer of the frozen MLLM:

(Ksem,Vsem)=KV⁡ϕ1frozen(Tsem),(K_{\mathrm{sem}},V_{\mathrm{sem}})=\operatorname{KV}_{\phi_1}^{\mathrm{frozen}}(T_{\mathrm{sem}}),
him=CrossAttn⁡ϕ1frozen(qim,sg⁡(Ksem),sg⁡(Vsem))(7)h_i^m=\operatorname{CrossAttn}_{\phi_1}^{\mathrm{frozen}}\left(q_i^m,\operatorname{sg}(K_{\mathrm{sem}}),\operatorname{sg}(V_{\mathrm{sem}})\right) \tag*{(7)}

where sg⁡(⋅)\operatorname{sg}(\cdot) denotes the stop-gradient operation. The resulting metric feature himh_i^m captures the geometry of object ii through interaction with the MLLM-adapted scene context.

Overall, the scene understanding module produces a hierarchical representation by combining object-level semantic and metric features with global scene semantics, which is formulated as

({(hiL,him)}i=1Nk,hgL)=U(Sk,ps).(8)\left(\left\{\left(h_i^L,h_i^m\right)\right\}_{i=1}^{N_k},h_g^L\right)=\mathcal{U}(S_k,p_s). \tag*{(8)}

The resulting features serve as the hierarchical scene representations for subsequent reward evaluation and object-level decision making.

Multi-view Alignment Pretraining. To provide visual supervision for the structured scene representation during training, we render each scene from a set V\mathcal{V} of top-down and oblique viewpoints. Each rendered image IvI^v is processed by a frozen MLLM visual encoder Evis(⋅)E_{\mathrm{vis}}(\cdot) to obtain a spatial feature map. Instance-mask-weighted pooling then extracts an object-aligned visual feature:

viv=MaskPool⁡(Evis(Iv),Miv),v∈V,(9)\mathbf{v}_i^v = \operatorname{MaskPool}\left(E_{\mathrm{vis}}(I^v),\mathbf{M}_i^v\right), \qquad v \in\mathcal{V}, \tag*{(9)}

where Miv\mathbf{M}_i^v denotes the instance mask of object ii in view v∈top,obliquev \in\mathrm{top},\mathrm{oblique}.

We align the object-level SGR feature ziSGR\mathbf{z}_i^{\mathrm{SGR}} with its corresponding visual feature viv\mathbf{v}_i^v using a cosine similarity loss and an object-level contrastive loss:

Lvis=Lcos(zilong,viv)+λLNCE(zilong,viv),(10)\mathcal{L}_{\mathrm{vis}} = \mathcal{L}_{\mathrm{cos}}\left(\mathbf{z}_i^{\mathrm{long}},\mathbf{v}_i^v\right) + \lambda\mathcal{L}_{\mathrm{NCE}}\left(\mathbf{z}_i^{\mathrm{long}},\mathbf{v}_i^v\right), \tag*{(10)}

where both losses are evaluated over matched object-view pairs. This alignment is introduced for providing visual supervision during training, while rendered views are not required for online scene understanding.

MLLM-guided Semantic Reward Evaluation and Object-level Decision

While MLLM is capable of diagnosing the intermediate layout at each denoising step, repeatedly invoking it would incur substantial inference cost. To mitigate this issue, we introduce an MLLM-guided Semantic Reward Evaluation and Object-Level Decision module, which distills the MLLM’s candidate-level evaluations into a lightweight Reward Predictor. During inference, the predictor estimates utilities of candidate corrections and selects the correction to guide subsequent denoising.

MLLM-Based Candidate Correction Generation. At denoising step tt, we provide the hierarchical scene representation Ut\mathcal{U}_t and the text prompt psp_s to an MLLM, which generates a textual description Tt\mathcal{T}_t of object-level corrections:

Tt=MLLM⁡(Ut,ps).(11)\mathcal{T}_t = \operatorname{MLLM}(\mathcal{U}_t,p_s). \tag*{(11)}

We parse Tt\mathcal{T}_t into a structured set of NtN_t candidate actions,

Atprop={ait}i=1Nt,ait=(idi,Mi,yi,tar),(12)\mathcal{A}_t^{\mathrm{prop}} = \left\{a_i^t\right\}_{i=1}^{N_t}, \qquad a_i^t = (id_i,\mathbf{M}_i,y_{i,\mathrm{tar}}), \tag*{(12)}

where idiid_i identifies the target object, Mi\mathbf{M}_i specifies the attributes to be modified, and yi,tary_{i,\mathrm{tar}} denotes their target values. In this work, we mainly consider two correction types, horizontal translation and yaw rotation. For the object attributes (x,y,ψ)(x,y,\psi), the corresponding masks are Mi=[1,1,0]\mathbf{M}_i = [1,1,0] and Mi=[0,0,1]\mathbf{M}_i = [0,0,1], respectively. To filter out geometrically invalid proposals, we temporarily apply each candidate action to the current layout and discard it if the resulting configuration violates any scene-level geometric validity constraint.

Semantic Reward Prediction and Object-level Decision. We rank the feasible candidate corrections in Atprop\mathcal{A}_t^{\mathrm{prop}} using a lightweight Reward Predictor. The predictor approximates the MLLM-based evaluation of each candidate along three dimensions: Prompt Alignment (P), Functional Coherence (F), and Semantic Consistency (S).

For each feasible action aa targeting object ii, we decode the projected clean estimate x^0(t)\hat{\mathbf{x}}_0^{(t)} to obtain the current scene St\mathcal{S}_t and apply aa to construct a hypothetical candidate scene St(a)\mathcal{S}_t^{(a)}. We then extract the hierarchical representations of the reference and candidate scenes, Utref=U(St,ps)\mathcal{U}_t^{\mathrm{ref}} = \mathcal{U}(\mathcal{S}_t,p_s) and Ut(a)=U(St(a),ps)\mathcal{U}_t^{(a)} = \mathcal{U}(\mathcal{S}_t^{(a)},p_s). Given these representations, the cached prompt feature qps\mathbf{q}_{p_s}, and the action encoding ι(a)\iota(a), the Reward Predictor {ηd(⋅)}\{\eta_d(\cdot)\} outputs a score distribution for each evaluation dimension,

{π^d(a)}d∈{P,F,S}={ηd(Utref,Ut(a),qps,ι(a))},(13)\left\{\hat{\pi}_d^{(a)}\right\}_{d\in\{\mathrm{P},\mathrm{F},\mathrm{S}\}} = \left\{\eta_d\left(\mathcal{U}_t^{\mathrm{ref}},\mathcal{U}_t^{(a)},\mathbf{q}_{p_s},\iota(a)\right)\right\}, \tag*{(13)}

where the action encoding ι(a)\iota(a) specifies the target object, correction type, and proposed target attributes. Each π^d(a)\hat{\pi}_d^{(a)} is a probability distribution over the five valid ordinal levels {0,1,2,3,4}\{0,1,2,3,4\} and an additional Unknown class. The predicted classes and conditional expected scores are then normalized and weighted aggregated with weights of 3:1:13:1:1 to calculate a predicted utility score u^a\hat{u}_a.

Among these candidates, we select the action with the highest utility after penalizing the magnitude of the proposed correction,

Dt=arg⁡max⁡a∈Atprop u^a−λcCt(a)(14)\mathcal{D}_t = \underset{a\in\mathcal{A}_t^{\mathrm{prop}}}{\arg\max}\ \hat{u}_a - \lambda_c C_t(a) \tag*{(14)}

where Ct(a)C_t(a) is the normalized modification cost and λc\lambda_c controls its penalty.

We encode each selected decision with the Decision Encoder to obtain a transient conditioning signal for subsequent denoising,

cttrans=Edec(Dt).(15)\mathbf{c}_t^{\mathrm{trans}} = E_{\mathrm{dec}}(\mathcal{D}_t). \tag*{(15)}

This signal conveys the selected correction to the diffusion model at the following denoising steps.

At the same time, we collect all accepted corrections within each denoising window of length τb\tau_b in chronological order:

Bk=Buffer⁡τb({Dt}t∈Wk),(16)\mathcal{B}_k = \operatorname{Buffer}_{\tau_b}\left(\left\{\mathcal{D}_t\right\}_{t\in\mathcal{W}_k}\right), \tag*{(16)}

where Wk\mathcal{W}_k denotes the kk-th denoising window.

After each buffering interval Wk\mathcal{W}_k of τb\tau_b denoising steps, the buffered actions are sequentially applied to update a persistent correction state, initialized as P0=∅\mathcal{P}_0 = \varnothing. For each corrected object, the state stores its persistent ID, attribute mask, and corresponding target values for position and yaw. The state update is formulated as:

Pk+1=Update⁡seq(Pk,Bk),(17)\mathcal{P}_{k+1} = \operatorname{Update}_{\mathrm{seq}}(\mathcal{P}_k, \mathcal{B}_k), \tag*{(17)}

where Update⁡seq\operatorname{Update}_{\mathrm{seq}} sequentially applies buffered actions in chronological order, preserving previously stored targets while allowing later actions to overwrite the same attributes.

MLLM-supervised Reward Distillation. During training, a frozen MLLM evaluator EE provides P/F/S score labels, including Unknown, and pairwise preferences among feasible candidates. The Reward Predictor fηf_\eta learns to reproduce individual scores and relative candidate rankings through:

Lpref=Lrank+λtieEtie[(u^a−u^b)2],\mathcal{L}_{\mathrm{pref}} = \mathcal{L}_{\mathrm{rank}} + \lambda_{\mathrm{tie}}\mathbb{E}_{\mathrm{tie}}\left[(\hat{u}_a-\hat{u}_b)^2\right],
LR=Lscorerows+Lscoregroups+λprefLpref+λontoLontology,Lscore=LCE+Lreg+λKLLKL,(18)\begin{aligned} \mathcal{L}_{R} &= \mathcal{L}_{\mathrm{score}}^{\mathrm{rows}} + \mathcal{L}_{\mathrm{score}}^{\mathrm{groups}} \\ &\quad+ \lambda_{\mathrm{pref}}\mathcal{L}_{\mathrm{pref}} + \lambda_{\mathrm{onto}}\mathcal{L}_{\mathrm{ontology}},\\ \mathcal{L}_{\mathrm{score}} &= \mathcal{L}_{\mathrm{CE}} + \mathcal{L}_{\mathrm{reg}} + \lambda_{\mathrm{KL}}\mathcal{L}_{\mathrm{KL}}, \tag*{(18)} \end{aligned}

where score losses supervise individual and group-level evaluations, ranking and tie losses transfer candidate preferences, and the ontology loss regularizes cross-dimensional consistency. The λ\lambda terms control the relative contributions of each objective. Only fηf_\eta is optimized, while EE and U\mathcal{U} remain frozen. At inference, EE is removed and fηf_\eta evaluates candidate corrections.

Decision-conditioned Trajectory-consistent Diffusion

To learn localized correction, we construct decision supervision through single-object perturbations. Given a clean layout x0∗x_0^*, we randomly select one object ii and perturb its horizontal position or yaw, obtaining x0cx_0^c while leaving other attributes unchanged. The training decision and its encoded condition are:

yi,tar∗=mτ⊙yi∗+(1−mτ)⊙yic,y_{i,\mathrm{tar}}^* = \mathbf{m}_{\tau} \odot y_i^* + (1-\mathbf{m}_{\tau}) \odot y_i^c,
Dtrain=(idi,mi,yi,tar∗),(19)\mathcal{D}^{\mathrm{train}} = (\mathrm{id}_i,\mathbf{m}_i,y_{i,\mathrm{tar}}^*), \tag*{(19)}
ctrain=Edec(Dtrain),\mathbf{c}^{\mathrm{train}} = E_{\mathrm{dec}}(\mathcal{D}^{\mathrm{train}}),

where yi=[pix,piz,θi]\mathbf{y}_i = [p_i^x,p_i^z,\theta_i], and mτ\mathbf{m}_{\tau} selects the perturbed attributes. yi∗\mathbf{y}_i^* and yic\mathbf{y}_i^c denote the clean and corrupted attributes, respectively.

We retain a frozen baseline denoiser vθ0v_{\theta_0}, initialized from the same pretrained weights as the trainable denoiser vθv_\theta. The corrupted layout x0cx_0^c is forward-diffused with a shared noise sample to obtain xtcx_t^c. Both denoisers receive the same noised input, timestep, prompt, and noise realization. The baseline receives no decision condition, whereas the trainable denoiser is conditioned on the encoded training decision. Their clean estimates are reconstructed as:

x^0base=αˉtxtc−1−αˉtvθ0(xtc,t,ps,c∅),\hat{x}_0^{\mathrm{base}} = \sqrt{\bar{\alpha}_t}x_t^c - \sqrt{1-\bar{\alpha}_t}v_{\theta_0}(x_t^c,t,p_s,c_{\varnothing}),
x^0dec=αˉtxtc−1−αˉtvθ(xtc,t,ps,Edec(Dtrain)).(20)\hat{x}_0^{\mathrm{dec}} = \sqrt{\bar{\alpha}_t}x_t^c - \sqrt{1-\bar{\alpha}_t}v_{\theta}(x_t^c,t,p_s,E_{\mathrm{dec}}(\mathcal{D}^{\mathrm{train}})). \tag*{(20)}

The denoiser and Decision Encoder are optimized with:

LDCD=Ldiffclean+λtargetLtarget+λpreserveLpreserve,\mathcal{L}_{\mathrm{DCD}} = \mathcal{L}_{\mathrm{diff}}^{\mathrm{clean}} + \lambda_{\mathrm{target}}\mathcal{L}_{\mathrm{target}} + \lambda_{\mathrm{preserve}}\mathcal{L}_{\mathrm{preserve}},
Ltarget=∥M⊙(x^0dec−x0∗)∥1,\mathcal{L}_{\mathrm{target}} = \left\|\mathbf{M}\odot(\hat{x}_0^{\mathrm{dec}}-x_0^*)\right\|_1,
Lpreserve=∥(1−M)⊙(x^0dec−sg⁡(x^0base))∥1,(21)\mathcal{L}_{\mathrm{preserve}} = \left\|(1-\mathbf{M})\odot\left(\hat{x}_0^{\mathrm{dec}}-\operatorname{sg}(\hat{x}_0^{\mathrm{base}})\right)\right\|_1, \tag*{(21)}

where Ldiffclean\mathcal{L}_{\mathrm{diff}}^{\mathrm{clean}} preserves generation capability, and M\mathbf{M} selects the perturbed attributes. The target loss encourages recovery of the clean attributes, while the preservation loss distills baseline behavior on unselected attributes. The two coefficients control the relative contributions of correction and preservation; sg⁡\operatorname{sg} denotes stop-gradient.

At inference, selected decisions are accumulated through buffered sequential execution to update the persistent correction state Pk\mathcal{P}_k. At each reverse-diffusion step, we enforce the accumulated targets by softly projecting the predicted clean attributes:

y~i=W(y^i+ρMˉi,k⊙W(yˉi,tar(k)−y^i)),(22)\tilde{\mathbf{y}}_i = \mathcal{W}\left(\hat{\mathbf{y}}_i+\rho\bar{\mathbf{M}}_{i,k}\odot\mathcal{W}\left(\bar{\mathbf{y}}_{i,\mathrm{tar}}^{(k)}-\hat{\mathbf{y}}_i\right)\right), \tag*{(22)}

where y^i=[p^ix,p^iz,θ^i]\hat{\mathbf{y}}_i = [\hat{p}_i^x,\hat{p}_i^z,\hat{\theta}_i] is extracted from the predicted clean estimate, and Mˉi,k\bar{\mathbf{M}}_{i,k} and yˉi,tar(k)\bar{\mathbf{y}}_{i,\mathrm{tar}}^{(k)} denote the accumulated attribute mask and target retrieved from Pk\mathcal{P}_k, respectively. ρ∈(0,1]\rho\in(0,1] controls the projection strength, and W\mathcal{W} wraps only the yaw component while leaving position unchanged.

The projected attributes are composed back into the original clean estimate, with yaw re-encoded using sine and cosine:

x~0(t)=Mklayout⊙x0,proj(t)+(1−Mklayout)⊙x^0(t),(23)\tilde{x}_0^{(t)} = \mathbf{M}_k^{\mathrm{layout}}\odot x_{0,\mathrm{proj}}^{(t)} + (1-\mathbf{M}_k^{\mathrm{layout}})\odot\hat{x}_0^{(t)}, \tag*{(23)}

where Mklayout\mathbf{M}_k^{\mathrm{layout}} expands the persistent attribute masks to the corresponding layout channels, including both sine and cosine channels for yaw. The resulting x0,proj(t)x_{0,\mathrm{proj}}^{(t)} contains the projected target attributes at the selected channels. It is then incorporated into the reverse diffusion process:

xt−1=DDPM⁡(xt,x~0(t),t),(24)x_{t-1} = \operatorname{DDPM}(x_t,\tilde{x}_0^{(t)},t), \tag*{(24)}

where DDPM denotes the standard posterior sampling operator, taking the current noisy state, projected clean estimate, and diffusion timestep as inputs to produce the next denoising state. By repeatedly enforcing accumulated corrections along the same reverse diffusion trajectory, this mechanism maintains accepted target modifications while preserving non-target scene attributes. Further training and posterior sampling details are provided in the supplementary material.

Experiment

Datasets and Implementation Details

Datasets, Rendering, and Evaluation. We use 3D-FRONT [11] (4,110/1,068/576 train/val/test scenes) and

3D-FUTURE [12] (33 furniture categories). The test set contains 192 scenes per room type (Bedroom, Dining, Living) and 1,500 prompts; comparisons are category-wise, while ablations and capability evaluations use all 576 scenes. Following M3DLayout [37], all methods share identical assets and rendering settings. Evaluation covers distributional fidelity (FID/KID), text-scene alignment (CLIP-Score), distribution similarity (SCA; 50% indicates similarity), spatial overlap (PIoU), and physical-semantic consistency (PSA).

Implementation Details. ReflectScene uses M3DLayout on 2×2 \times RTX 5090 GPUs. Its 512-channel 1D U-Net (120 slots; frozen BERT) trains for 28K epochs (Adam; batch 256; lr 2×10−42 \times10^{-4}; 1K steps). Scene understanding uses a frozen 3B MLLM with two-stage multi-view training. The P evaluation head and Reward Predictor are trained for 2K/2.4K updates (batch 64; lr 10−310^{-3}), using MLLM P/F/S scores and preference supervision. DCD fine-tunes the denoiser and Decision Encoder against an identically initialized frozen baseline (2.56K updates; AdamW; batch 64; lr 10−5/10−410^{-5}/10^{-4}; clean/target/preservation loss weights 1.0/0.4/0.2). Inference performs per-step selection and executes buffered actions every 50 steps via persistent clean-estimate projection. Further details are provided in the supplementary material.

Comparisons with Existing Methods

Quantitative Results. We reproduce and retrain competing methods using public implementations under identical data splits and rendering protocols. As shown in Table 1, ReflectScene achieves the best FID, PSA, and CLIP-Score across all room types, reducing FID by 6.0%, 15.0%, and 9.9% over M3DLayout, respectively. It also achieves the lowest KID on Bedroom and Dining Room while maintaining competitive SCA and PIoU, demonstrating improved semantic and functional coherence without compromising spatial plausibility.

MethodVenueBedroomDining RoomLiving Room
FID↓KID↓SCA%PSA↑CLIP-Score↑PIoU↓FID↓KID↓SCA%PSA↑CLIP-Score↑PIoU↓FID↓KID↓SCA%PSA↑CLIP-Score↑PIoU↓
DiffuScene[28]CVPR 2024118.113.5362.2974.20.24570.77144.9013.1983.3761.50.21230.84123.949.5479.2967.60.23520.62
InstructScene[19]ICLR 2024131.478.8971.5769.70.22820.86143.0217.9891.2260.80.21180.82127.9710.4283.9162.40.22370.47
M3DLayout[37]CVPR 2026115.635.0164.6273.90.24630.78138.1012.8478.4666.40.20340.74122.789.4877.9571.30.23660.31
ReflectSceneOurs108.702.9762.3182.50.24660.71117.389.2883.3374.70.21590.77110.629.6371.2878.10.25460.44

Table 1. Quantitative comparison of different methods on Bedroom, Dining Room, and Living Room scenes.

Qualitative Results. As shown in Figure 3, ReflectScene produces more semantically aligned and functionally coherent layouts across all three room types, better preserving prompt-specified furniture arrangements than competing methods. The rendered scenes and 3D bounding boxes further demonstrate improved text-scene alignment and spatial organization.

Figure 3. Qualitative comparison. Across three room types, ReflectScene better preserves prompt-specified furniture arrangements and functional spatial relationships, producing more coherent layouts. Top: rendered scenes; bottom: 3D bounding-box visualizations.

Ablation Study

Main Ablation Study. Table 2 reports the results of progressively removing the three components to assess their contributions, evaluating all variants on the full test set. Removing Reward Evaluation degrades performance, increasing FID from 69.37 to 71.89 and reducing PSA from 78.4 to 75.3. Further removing Scene Understanding causes additional declines in generation quality and semantic alignLatency is measured for evaluating five fixed candidates per state item, with request features cached. Both methods exclude upstream scene understanding (U) computation. Preference accuracy should be interpreted together with preference coverage.

VariantFID↓\downarrowKID↓\downarrowSCA%PSA↑\uparrowCLIP-Score↑\uparrowPIoU↓\downarrow
Ours69.375.2374.3878.40.24250.64
w/o (Reward Evaluation)71.896.3676.5275.30.24120.65
w/o (Reward Evaluation + Scene Understanding)75.056.7077.3771.10.23950.71
w/o (Reward Evaluation + Scene Understanding + DCD)80.229.7578.9166.70.21890.73

Table 2. Progressive ablation study of the three components in ReflectScene.

ment. Removing DCD leads to the largest degradation, with FID rising to 80.22, PSA dropping to 66.7, and CLIP-Score falling to 0.2189. Meanwhile, PIoU increases from 0.64 to 0.73, indicating more severe object overlaps, while SCA moves further from the ideal 50%. These results confirm the complementary roles of the three components in semantic understanding, corrective decision-making, and preserving corrections throughout denoising.

Reward Predictor. As shown in Table 3, on an independent acceptance set of 40 novel scenes, 93 valid state tasks, and 1,488 questions, the Reward Predictor achieves 98.98% task-preference accuracy, 97.46% P-balanced accuracy, and 100% preference coverage. It reduces median candidate-evaluation latency from 506.24 ms to 9.66 ms (52.4×\times faster), while supplementary results confirm high P/F/S distillation fidelity with 98.20% candidate preference agreement. These results validate that MLLM-based semantic evaluation can be effectively distilled into a lightweight predictor, preserving reliable candidate-level decisions while substantially reducing evaluation overhead.

MetricFrozen MLLM Evaluator (Teacher)Reward Predictor (Student)
Task-Preference Accuracy (↑\uparrow)99.24%98.98%
Preference Coverage (↑\uparrow)99.90%100%
P-Balanced Accuracy (↑\uparrow)98.26%97.46%
Missing-Evidence Unknown Recall (↑\uparrow)100%99.73%
Known-Score Preservation Rate (↑\uparrow)100%100%
Median Candidate-Evaluation Latency (↓\downarrow)506.24 ms9.66 ms

Table 3. Teacher–student comparison for candidate-level reward evaluation on an independent acceptance set of 40 novel scenes, 93 valid state tasks, and 1,488 questions.

SGR Adapter and Multi-view Alignment. As shown in Table 4, integrating the frozen MLLM improves semantic, relational, and functional understanding over Adapter-only, demonstrating the value of contextual reasoning. As shown in Table 5, the SGR Adapter maintains above 91% Macro-F1 and below 1.4 cm XZ error across DDPM stages, while visual matching recovers from 59.33% to 73.16%, approaching the clean-layout result of 73.26%. Additional supplementary evaluations confirm its fine-grained understanding, cross-seed stability, and counterfactual robustness, validating reliable scene understanding and visual grounding throughout denoising.

MetricAdapter-onlyAdapter + Frozen MLLM
Category Macro-F1 (↑\uparrow)84.58%96.68%
Distance Rel. F1 (↑\uparrow)82.25%94.07%
Adjacency Rel. F1 (↑\uparrow)71.92%82.82%
Weak Functional Role F1 (↑\uparrow)90.17%99.11%
Room Category Accuracy (↑\uparrow)86.77%90.77%
Activity Binary F1 (↑\uparrow)93.81%98.73%
XZ Center MAE (↓\downarrow)1.57 cm1.21 cm
Yaw MAE (↓\downarrow)0.98°0.91°

Table 4. Comparison of adapter-only tuning and adapter tuning with a frozen MLLM for scene understanding.

StateCat. F1 (↑\uparrow)Dist. F1 (↑\uparrow)XZ MAE (↓\downarrow)Visual Top-1 (↑\uparrow)
Clean98.28%95.48%1.15 cm73.26%
Perturbed98.04%95.36%1.23 cm70.34%
DDPM Early94.43%93.29%1.36 cm59.33%
DDPM Middle95.65%92.30%1.22 cm70.51%
DDPM Late95.21%91.43%1.14 cm73.16%
Overall–––70.38%

Table 5. Scene understanding and object-visual alignment across diffusion states. Cat. F1 and Dist. F1 denote category and distance-relation Macro-F1, respectively; XZ MAE denotes the XZ center mean absolute error, and Visual Top-1 denotes object-visual matching Top-1 accuracy.

Models with Different Capabilities

Table 6 compares different reasoning strategies and MLLM backbones to investigate their impact on scene generation. Adapter-only and Geometric Algorithm achieve comparable performance to Qwen2-VL-2B, suggesting that bypassing contextual MLLM reasoning or relying on geometric heuristics provides limited benefits. Increasing backbone capability to Qwen2.5-VL-3B and 7B yields substantial improvements across all metrics, highlighting the importance of effective semantic and functional reasoning. However, scaling further to Qwen3-VL-32B brings only marginal gains over 7B, indicating diminishing returns as stronger reasoning capabilities approach the demands of structured indoor scene generation. This suggests that effective semantic reasoning, rather than model scale alone, is crucial for improving scene generation.

ModelFID↓\downarrowKID↓\downarrowSCA%PSA↑\uparrowCLIP-Score↑\uparrowPIoU↓\downarrow
Adapter-only76.0410.9583.3665.80.20240.77
Geometric Algorithm78.339.5281.2969.10.21160.74
Qwen2-VL-2B78.669.4382.7170.20.21330.74
Qwen2.5-VL-3B69.375.2374.3878.40.24250.64
Qwen2.5-VL-7B63.713.5770.1283.10.26100.58
Qwen3-VL-32B62.983.2469.8684.50.26710.56

Table 6. Quantitative comparison of different scene reasoning strategies and MLLM backbones.

Conclusion

We presented ReflectScene, an MLLM-guided closed-loop diffusion framework integrating hierarchical scene understanding, reward-driven decisions, and decision-conditioned denoising to guide scene evolution while preserving corrections. Experiments on 3D-FRONT show improved functional consistency and semantic alignment with competitive generation quality and diversity.

LLM Usage Statement

During the preparation of this work the author(s) used Chat-GPT in order to check grammar mistakes. After using this tool/service, the author(s) reviewed and edited the content as needed and take(s) full responsibility for the content of the published article.

References

  1. [1]Daichi Azuma, Taiki Miyanishi, Shuhei Kurita, and Motoaki Kawanabe. Scanqa: 3d question answering for spatial scene understanding. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19107–19117. IEEE, 2022. 3arxiv.org/abs/2112.10482
  2. [2]Dave Zhenyu Chen, Angel X Chang, and Matthias Nießner. Scanrefer: 3d object localization in rgb-d scans using natural language. In European conference on computer vision, pages 202–221. Springer, 2020. 3arxiv.org/abs/1912.08830
  3. [3]Sijin Chen, Xin Chen, Chi Zhang, Mingsheng Li, Gang Yu, Hao Fei, Hongyuan Zhu, Jiayuan Fan, and Tao Chen. L13da: Visual interactive instruction tuning for omni-3d understanding, reasoning, and planning. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 26418–26428. IEEE, 2024. 2arxiv.org/abs/2311.18651
  4. [4]Zhenyu Chen, Ali Gholami, Matthias Nießner, and Angel X Chang. Scan2cap: Context-aware dense captioning in rgb-d scans. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3193–3203, 2021. 3
  5. [5]Jiajun Deng, Tianyu He, Li Jiang, Tianyu Wang, Feras Dayoub, and Ian Reid. 3d-llava: Towards generalist 3d llms with omni superpoint transformer. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3772–3782. IEEE, 2025. 3arxiv.org/abs/2501.01163
  6. [6]Wei Deng, Mengshi Qi, and Huadong Ma. Global-local tree search in vlms for 3d indoor scene generation. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8975–8984. IEEE, 2025. 2
  7. [7]Haoran Feng, Yifan Niu, Zehuan Huang, Yang-Tian Sun, Chunchao Guo, Yuxin Peng, and Lu Sheng. Repurposing 3d generative model for autoregressive layout generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3231–3243, 2026. 2arxiv.org/abs/2604.16299
  8. [8]Jie Feng, Jiawei Shen, Junjia Huang, Junpeng Zhang, Mingtao Feng, Weisheng Dong, and Guanbin Li. Sdesc3d: Towards layout-aware 3d indoor scene generation from short descriptions. arXiv preprint arXiv:2604.01972, 2026. 3arxiv.org/abs/2604.01972
  9. [9]Weixi Feng, Wanrong Zhu, Tsu-jui Fu, Varun Jamini, Arjun Akula, Xuehai He, Sugato Basu, Xin Eric Wang, and William Yang Wang. Layoutgpt: Compositional visual planning and generation with large language models. Advances in neural information processing systems, 36:18225–18250, 2023. 1arxiv.org/abs/2305.15393
  10. [10]Weitao Feng, Hang Zhou, Jing Liao, Li Cheng, and Wenbo Zhou. Casagt: cuboid arrangement and scene assembly for interior design. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 29173–29182. IEEE, 2025. 2arxiv.org/abs/2504.19478
  11. [11]Huan Fu, Rongfei Jia, Lin Gao, Ling-Xiao Zhang, Jiaming Wang, Cao Li, Qixun Zeng, Chengyue Sun, Rongfei Jia, Binqiang Zhao, and Hao Zhang. 3d-front: 3d furnished rooms with layouts and semantics. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 10933–10942, 2021. 6arxiv.org/abs/2011.09127
  12. [12]Huan Fu, Rongfei Jia, Lin Gao, Mingming Gong, Binqiang Zhao, Stephen J. Maybank, and Dacheng Tao. 3D-FUTURE: 3D Furniture Shape with TextURE. International Journal of Computer Vision, 129(12):3313–3337, 2021. 7arxiv.org/abs/2009.09633
  13. [13]Siyuan Huang, Zan Wang, Puhao Li, Baoxiong Jia, Tengyu Liu, Yixin Zhu, Wei Liang, and Song-Chun Zhu. Diffusion-based generation, optimization, and planning in 3d scenes. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16750–16761. IEEE, 2023. 2arxiv.org/abs/2301.06015
  14. [14]Krishna Murthy Jatavallabhula, Alihusein Kuwajerwala, Qiao Gu, Mohd Omama, Tao Chen, Alaa Malalouf, Shuang Li, Ganesh Iyer, Soroush Saryazdi, Nikhil Keetha, et al. Conceptfusion: Open-set multimodal 3d mapping. arXiv preprint arXiv:2302.07241, 2023. 3
  15. [15]Baoxiong Jia, Yixin Chen, Huangyue Yu, Yan Wang, Xuesong Niu, Tengyu Liu, Qing Li, and Siyuan Huang. Sceneservese: Scaling 3d vision-language learning for grounded scene understanding. In European Conference on Computer Vision, pages 289–310. Springer, 2024. 2arxiv.org/abs/2401.09340
  16. [16]Haiyan Jiang, Deyu Zhang, Dongdong Weng, Weitao Song, and Henry Been-Lin Duh. Hog-layout: Hierarchical 3d scene generation, optimization and editing via vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 31586–31596, 2026. 3arxiv.org/abs/2604.10772
  17. [17]Zhengfei Kuang, Rui Lin, Long Zhao, Gordon Wetzstein, Saining Xie, and Sanghyun Woo. Vulcan: Tool-augmented multi agents for iterative 3d object arrangement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23763–23773, 2026. 3arxiv.org/abs/2512.22351
  18. [18]Manyi Li, Akshay Gadi Patil, Kai Xu, Siddhartha Chaudhuri, Owais Khan, Ariel Shamir, Changhe Tu, Baoxuan Chen, Daniel Cohen-Or, and Hao Zhang. Grains: Generative recursive autoencoders for indoor scenes. ACM Transactions on Graphics (TOG), 38(2):1–16, 2019. 1, 2
  19. [19]Chenguo Lin and Yadong Ma. Instructscene: Instruction-driven 3d indoor scene synthesis with semantic graph prior. In International Conference on Learning Representations, pages 25687–25718, 2024. 2, 7
  20. [20]Shuhan Xiao, Biru Cao, and Junling Zhuang. Network: Conditional 3d furnished house layout generation through latent heterogeneous graph diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 27093–27103, 2026. 2
  21. [21]Xichen Pan, Aashu Singh, Satya Narayan Shukla, Xiangjun Fan, Shlok Kumar Mishra, and Saining Xie. Repfunion: Leveraging multimodal priors for denoising in representation space. arXiv preprint arXiv:2606.14700, 2026. 3arxiv.org/abs/2606.14700
  22. [22]Despoina Paschalidou, Amlan Kar, Maria Shugrina, Karsten Kreis, Andreas Geiger, and Sanja Fidler. Autoregressive transformers for indoor scene synthesis. Advances in neural information processing systems, 34:12013–12026, 2021. 2arxiv.org/abs/2110.03675
  23. [23]Songyun Peng, Kyle Genova, Chiyu Jiang, Andrea Tagliasacchi, Marc Pollefeys, and Thomas Funkhouser. Openscene: 3d scene understanding with open vocabularies. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 815–824. IEEE, 2023. 3
  24. [24]Xingjian Ran, Yixuan Li, Linning Xu, Mulin Yu, and Bo Dai. Direct numerical layout generation for 3d indoor scene synthesis via spatial reasoning. In Advances in Neural Information Processing Systems, 2025. 2arxiv.org/abs/2506.05341
  25. [25]Fan-Yun Sun, Weiyu Liu, Siyi Gu, Dylan Lim, Goutam Bhat, Federico Tombari, Manling Li, Nick Haber, and Jiajun Wu. Layoultvlm: Differentiable optimization of 3d layout via vision-language models. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 29469–29478. IEEE, 2025. 3
  26. [26]Xiaohou Sun, Divyam Goel, and Angel X Chang. Semlay-outdiff: Semantic layout generation with diffusion model for indoor scene synthesis. In 2026 International Conference on 3D Vision (3DV), pages 1545–1555. IEEE, 2026. 2
  27. [27]Ayça Takmaz, Elisabetta Fedele, Robert W Sumner, Marc Pollefeys, Federico Tombari, and Francis Engelmann. Openmask3d: Open-vocabulary 3d instance segmentation. arXiv preprint arXiv:2306.13631, 2023. 3arxiv.org/abs/2306.13631
  28. [28]Jiapeng Tang, Yinyu Nie, Lev Markhasin, Angela Dai, Justus Thies, and Matthias Nießner. Diffuscene: Denoising diffusion models for generative indoor scene synthesis. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20507–20518. IEEE, 2024. 2, 7arxiv.org/abs/2303.14207
  29. [29]Ruiqi Wang, Qimin Chen, Daniel Ritchie, Angel X Chang, Manolis Savva, Kai Wang, and Hao Zhang. Function2scene: 3d indoor scene layout from functional specifications. arXiv preprint arXiv:2605.30819, 3, 2026. 2arxiv.org/abs/2605.30819
  30. [30]Xinpeng Wang, Chandan Yeshwanth, and Matthias Nießner. Sceneformer: Indoor scene generation with transformers. In 2021 International conference on 3D vision (3DV), pages 106–115. IEEE, 2021. 1arxiv.org/abs/2012.09793
  31. [31]Runsen Xu, Xiaolong Wang, Tai Wang, Yilun Chen, Jiangmiao Pang, and Dahua Lin. Pointllm: Empowering large language models to understand point clouds. In European Conference on Computer Vision, pages 131–147. Springer, 2024. 3arxiv.org/abs/2308.16911
  32. [32]Le Xue, Mingfei Gao, Chen Xing, Roberto Martín-Martín, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese. Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1179–1189. IEEE, 2023. 3arxiv.org/abs/2212.05171
  33. [33]Yandan Yang, Baoxiong Jia, Peiyuan Zhi, and Siyuan Huang. Physcene: Physically interactable 3d scene synthesis for embodied ai. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16262–16272. IEEE, 2024. 3arxiv.org/abs/2404.09465
  34. [34]Yue Yang, Fan-Yun Sun, Luca Weihs, Eli VanderBilt, Alvaro Herrasti, Winston Han, Jiajun Wu, Nick Haber, Ranjay Krishna, Lingjie Liu, et al. Holodeck: Language guided generation of 3d embodied ai environments. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16277–16287. IEEE, 2024. 2arxiv.org/abs/2312.09067
  35. [35]Zhifei Yang, Keyang Lu, Chao Zhang, Jiaxing Qi, Hanqi Jiang, Ruifei Ma, Shenglin Yin, Yifan Xu, Mingzhe Xing, Zhen Xiao, et al. Mmgdreamer: Mixed-modality graph for geometry-controllable 3d indoor scene generation. In Proceedings of the AAAI conference on artificial intelligence, pages 9391–9399, 2025. 2arxiv.org/abs/2502.05874
  36. [36]Guangyao Zhai, Evin Pinar Örnek, Shun-Cheng Wu, Yan Di, Federico Tombari, Nassir Navab, and Benjamin Busam. Commonsense: Generating commonsense 3d indoor scenes with scene graph diffusion. Advances in Neural Information Processing Systems, 36:30026–30038, 2023. 2arxiv.org/abs/2305.16283
  37. [37]Yiheng Zhang, Zhuojian Cai, Mingdao Wang, Meitong Guo, Tianxiao Li, Li Lin, and Yuwang Wang. M3DLayout: A multi-source dataset of 3D indoor layouts and structured descriptions for 3D generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 34217–34226, 2026. 2, 7arxiv.org/abs/2509.23728
  38. [38]Yang Zhao, Shizhao Sun, Meisheng Zhang, Yingdong Shi, Xubo Yang, and Jiang Bian. Scenerevis: A self-reflective vision-grounded framework for 3d indoor scene synthesis via multi-turn rl. arXiv preprint arXiv:2602.09432, 2026. 2, 3DOI
  39. [39]Yang Zhou, Zachary While, and Evangelos Kalogerakis. Scenegraphnet: Neural message passing for 3d indoor scene augmentation. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 7383–7391. IEEE, 2019. 2arxiv.org/abs/1907.11308
  40. [40]Ziyu Zhu, Xiaojian Ma, Yixin Chen, Zhidong Deng, Siyuan Huang, and Qing Li. 3d-vista: Pre-trained transformer for 3d vision and text alignment. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 2899–2909. IEEE, 2023. 2arxiv.org/abs/2308.04352

Paper details

vlnc:W_bzqutrdg
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 9

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Chat-GPT

View statement
Looking for related papers…
Contents