betaFeedback

WorksComputer Science

Learning to Edit IFC Building Models: Verifiable Checkers as the Training Signal for Dependency-Aware Editing Agents

Abstract

Editing a building information model in the Industry Foundation Classes (IFC) schema requires propagating changes to dependent elements while preserving unrelated entities. Existing language-model agents for IFC editing use prompted models without learning from edit outcomes. Target-file comparisons do not specify which alternative edits are allowed or which unrelated entities must remain unchanged. We present IfcGym, a CodeAct environment in which a language model edits IFC through executable code and four checkers verify validity relative to the input model, intent, dependency and preservation against a task specification. Their conjunction is strict success. IfcEdit distils verified teacher episodes into a Qwen3.5-9B student through supervised fine-tuning (SFT), then applies Group Relative Policy Optimization (GRPO) with strict success as the reward. On IfcEditBench, 1,990 editing tasks on 13 held-out real projects, the final student reaches 86.78% strict success, from 79.15% after SFT. The paired confidence interval for the difference from its teacher includes zero; the same recipe on a 27B model reaches 91.76% and exceeds the teacher. On the public benchmarks BIM-Edit and BIBIMBAP the final student scores 55.19 and 54.50. On IfcEditBench, the students outperform all prompted models except the teacher under matched tool access; both evaluated frameworks also score lower. Rewarding all four checkers yields 4.83 points more strict success than rewarding intent and validity alone.

Introduction

Building information modelling (BIM) uses digital representations of built assets to support information management and collaboration [3, 57]. The Industry Foundation Classes (IFC) standard defines an open, vendor-neutral data schema for representing and exchanging this information [20]; an IFC model represents a building as typed, interrelated entity instances. Design revisions, code-compliance checking and repair [7], and coordination across disciplines require changes to existing models. Such changes rarely affect an isolated element. Moving a wall may require moving its openings, doors and windows and updating the boundaries of adjacent spaces; deleting a door may also require removing its opening. We study dependency-aware editing: carrying out a natural-language instruction, propagating the change to its specified dependents, and preserving unrelated entities.

Dependency-aware editing requires reasoning over the model’s entity graph. Several edit sequences can satisfy an instruction, while a geometrically plausible result can still leave an opening detached, a space boundary inconsistent or an unrelated entity altered. Existing approaches encode building dependencies in a manually specified topology graph [60] or learn to resolve clashes through a small set of rigid moves on one model [21]. On an unseen building, the agent must identify which entities an instruction affects and how to edit them while preserving their relationships.

Large language models (LLMs) support BIM question answering and retrieval [34, 50, 66], and can access a model through code. CodeAct [59] and IFC tool servers [11, 42] provide interfaces for inspecting and editing a model. These IFC agents rely on prompted models and do not train on their edit outcomes. The benchmarks BIM-Edit [43] and BIBIMBAP [29] expose failures involving dependent elements and integrity preservation. Learning from these failures requires a success criterion that checks the requested modification, its dependent changes and the entities that must remain unchanged. A single target file does not directly specify this distinction or the range of acceptable solutions.

We make these editing requirements an executable training objective. IfcGym represents each task by a specification of its targets, allowed changes, intended modification and dependency rules. Four checkers evaluate validity, intent, dependency and preservation; their conjunction defines strict success. The same criterion selects successful teacher demonstrations, rewards the agent during reinforcement learning and evaluates its final edits. Different edit sequences can pass when they satisfy the specification; no target file is needed.

Figure 1. Data construction and learning in IfcGym. (a) Synthetic buildings supply verified teacher demonstrations; real IFC projects supply IfcEditBench tasks. (b) Bar heights encode square-root SFT counts; green circle areas encode test counts. Outer blue and inner green numbers give counts; lower panels show within-split reference/tier proportions and counts. (c) Teacher distillation, strict-reward GRPO and benchmark evaluation.

IfcEdit uses this supervision to train an open language model for dependency-aware editing. It first distils verified teacher episodes into Qwen3.5-9B through supervised fine-tuning (SFT) [18, 39, 65], then improves the policy through Group Relative Policy Optimization (GRPO) with strict success as the reward [10, 14, 52, 64]. The SFT corpus contains 19,141 verified episodes on 3,059 synthetic buildings with geometry and naming conventions drawn from real exports. Evaluation tests transfer to real buildings through IfcEditBench, comprising 1,990 tasks on 13 projects excluded from training, and to the public BIM-Edit and BIBIMBAP benchmarks. Figure 1 connects synthetic training and real-project test construction in (a), the resulting data composition in (b), and agent training and evaluation in (c).

We collect verified demonstrations across synthetic buildings with varied layouts and export conventions. To distinguish what the students learn from what the tools provide, we give prompted and trained models the same editing operations. We then compare each student before training, after SFT and after GRPO to measure the gains from the two stages.

On IfcEditBench, Qwen3.5-9B improves from 16.08% strict success when prompted to 79.15% after SFT and 86.78% after GRPO; the best prompted model other than the teacher with the same CodeAct tools reaches 67.99%, and the teacher 88.04%. The largest gaps occur on edits that must propagate to dependent elements, which the other prompted models and evaluated IFC frameworks often fail to complete. The paired gain from GRPO is 7.75 points, comparable in magnitude to the 9.15-point gain from applying the same SFT recipe to the larger 27B model. Applying GRPO to that larger student raises strict success further, to 91.76%.

Improvements also transfer to the public benchmarks: the final 9B student scores 55.19 on BIM-Edit and 54.50 on BIBIMBAP, improving over its SFT start on both. In the reward ablation, the four-gate reward improves strict success by 4.83 points over rewarding intent and validity alone.

The contributions are:

  1. IfcGym turns requested edits, dependency propagation and preservation into executable supervision, providing a common criterion for constructing verified training data, rewarding edits and evaluating outcomes.

  1. IfcEdit combines teacher distillation and GRPO to train

open models for dependency-aware editing, with gains beyond SFT at both 9B and 27B scales.

  1. Evaluation on IfcEditBench and two public benchmarks measures transfer from synthetic buildings to real projects and compares the students with their teacher and other prompted models under matched tool access. Paired uncertainty estimates and reward-term and data-scale ablations quantify these differences.

Related work

Language-model agents and benchmarks for IFC models

Language-model work in BIM includes question answering and information retrieval over IFC [17, 66], domain-adapted models and benchmarks [34], and generation from text or sketches [6]. Tool-based agents also give prompted models direct access to IFC files. MCP4IFC [42] exposes a tool server for scene queries and attribute edits. Its best configuration answers 83.1% of 65 scene-querying questions; edits requiring geometry, spatial context or element dependencies were not tested. IFC-Agent [11] adds schema-guided reasoning, isolation and versioning around IfcOpenShell, with evaluations centred on retrieval and generation. KIR [30] proposes a typed intermediate representation and dependency-closure diffs for agent-authored buildings, without an LLM experiment. These systems do not train the model on the outcome of its edits.

BIM-Edit [43] and BIBIMBAP [29] evaluate editing more directly. BIM-Edit compares geometry, semantics and topology with one target file per task and reports low strict completion across models. BIBIMBAP covers create, read, update and delete (CRUD) operations with executable per-task tests. Its best model reaches 50.2% overall, with 81.1% for Create and 18.9% for Delete; integrity preservation is the hardest criterion across models. Our setting makes the allowed changes and dependency rules explicit in a task specification. Edits pass when they satisfy those constraints and preserve entities outside the allowed set. We adopt BIM-Edit’s CodeAct protocol and use both benchmarks as external tests (§ 4.2); the training reward comes from our entity-graph checkers (§ 3.3.3).

Modifying building and CAD models under constraints

Wu et al. [60] formalise human alteration strategies with an LLM, apply them through a rule-coded topology graph, and re-check compliance with a commercial model checker. Their adjustment mechanism reduces the sampling cost of feasible alterations, with manually coded dependencies and evaluation on one building. IFCDecoupling [36] addresses safe modification of shared references on the IFC graph. For clash resolution, a PPO agent in a Gym wrapper uses eight rigid move actions and a scalar clash reward, requiring thousands to tens of thousands of steps on a single IFC house [21]. Text2BIM [6] repairs models during generation, and defect-repair systems [33] produce textual suggestions.

Language models also generate and edit parametric CAD through sketch-and-extrude programs [61], text-to-CAD generation [26] and CAD-kernel tools [41]. LLM4CAD-Editor [55] and B-repLer [38] evaluate language-guided edits through parsing or validity and geometric fidelity. Both report high parsing or validity rates alongside lower fidelity for more functional edits. These metrics do not directly measure preservation of unmentioned entities. City Editing [37] evaluates prompted agents under dependencies and constraints in urban geospatial models. ALAS [13] limits repair to a verified neighbourhood and rolls back failures, while trace-guided CAD repair [9] bounds edits with a dependency graph and uses a vision-language model to judge preservation from renders. IfcGym checks the edited IFC entities directly after transactional execution. IfcEdit learns how to propagate changes from instructions, with evaluation on held-out projects.

Training tool agents with verifiable rewards

IfcEdit builds on reasoning interleaved with tool calls [58, 63], executable code actions [59], and interfaces and benchmarks for software and service agents [22]. Reinforcement learning with verifiable rewards [16, 64] has been applied to strategic tool use [10], while trajectory distillation transfers agent behaviour to smaller models [39]. Ma et al. [40] find that the gains of tool-use reinforcement learning can arise from reasoning rather than from tool use itself.

In text-to-CAD, ToolCAD [14] combines SFT and GRPO with a learned outcome reward. Its 7B and 8B models achieve 61.8–63.9% success, within a few points of prompted GPT-4o. Embodied CAD [35] reports little improvement from GRPO over SFT for parametric assembly modelling. Ishigaki-IDS [24] applies validator-gated GRPO to specification drafting on Ishigaki-Bench [25]. Its 8B model exceeds the frontier baselines in validator pass rate while a content metric remains flat. In floor-plan generation [32], verifiable rewards improve constraint adherence, but connectivity-only rewards permit reward hacking [54]. We combine four specification-based checkers in the reward and test the contribution of dependency and preservation checks against an intent-and-validity reward (§ 4.2.1).

Method

We formulate dependency-aware IFC editing as code generation under a verifiable specification (§ 3.2). IfcGym executes and checks the agent’s edits (§ 3.3), supports task and data construction (§ 3.4), and supplies the training signal for IfcEdit (§ 3.5).

The distillation corpus and the reinforcement-learning pool

Task specification and the four checkers

Problem formulation

An editing task is a triple (M,q,σ)(M, q, \sigma): an IFC model MM, a graph of typed entity instances linked by attribute references and by relationship entities (the subtypes of IfcRelationship); a natural-language instruction qq; and a specification σ=(T,A,ι,Δ)\sigma= (T, A, \iota, \Delta) that the agent never sees. The specification and the checkers operate on products (IfcProduct: elements, openings, spaces, storeys, building and site), keyed by their GlobalId; placement, geometry and property-set entities are folded into the product that owns them, so a change to any of them counts as a change of that product (quantity sets are not compared). TT is the set of target products the instruction refers to, A⊇TA \supseteq T the set of products allowed to change, ι\iota the intent (the type of change and its parameters, for example a translation vector, a property value or a new height) and Δ={(dj,ρj)}\Delta= \{(d_j, \rho_j)\} the dependents that must follow the target, each with the rule ρj\rho_j it obeys: an opening and its fill translate with their host wall and stay hosted, a door’s opening is removed with the door, a space face meets the wall that was moved. The generator fixes the editing policy for each task family and states it in both the specification and the instruction. For example, IFC allows an opening to remain after its door is removed; a task can require either removal or retention of that opening. The agent produces a final model M′M'. Four binary checkers, each a function of (M,M′,σ)(M, M', \sigma), judge the outcome: validity vv, intent ii, dependency dd and preservation pp (§ 3.3.3). We define strict success as their conjunction,

s(M,M′,σ)=v⋅i⋅d⋅p,(1)s(M, M', \sigma) = v \cdot i \cdot d \cdot p, \tag*{(1)}

which is the held-out metric and reinforcement reward. Because σ\sigma specifies constraints rather than a target file, any M′M' satisfying them passes, regardless of the API-call sequence.

IfcGym

IfcGym supports teacher-episode collection, student training and evaluation through a transactional sandbox, element-local helper tools and the four checkers.

Sandbox

The agent has a single tool, execute_ifc_code, following BIM-Edit’s CodeAct interface [43]. Inside the code passed to it, the agent calls the helper package h (Table 1) or IfcOpenShell directly, and closes an editing step with commit(); a call without commit() only inspects the model. The tool runs Python in a subprocess with the model loaded through IfcOpenShell [28]. It returns standard output and the value assigned to result; changes persist only when the code calls commit(). Let Mk\mathcal{M}_k denote the model after the kk-th commit. Each call cc uses a fresh namespace and starts from the last committed state:

GroupHelpers
Inspection (read-only)inspect, find, unit_scale, world_matrix, wall_axis_world, profile_world_polygon, wall_length, space_area, check_model
Element-local editstranslate, move_wall_end, stretch_wall_end, shift_vertices_on_plane, shift_profile_vertices, shift_boundary_curve, set_extrusion_depth, set_wall_thickness, rotate, set_property, set_attribute, delete, connect_walls, private
Creationadd_wall, add_slab, add_column, add_space, add_opening_fill

Table 1. The helper package h: 28 helpers in three groups.

Mk+1={c(Mk)if c calls commit(),Mkotherwise,M0=M.(2)\mathcal{M}_{k+1} = \begin{cases} c(\mathcal{M}_k) & \text{if } c \text{ calls commit()},\\ \mathcal{M}_k & \text{otherwise}, \end{cases} \qquad\mathcal{M}_0 = \mathcal{M}. \tag*{(2)}

Uncommitted changes are discarded. Each commit writes a numbered snapshot, allowing replay and rollback. The final model is the last commit, M′=MKM' = \mathcal{M}_K, whether the episode ends with an answer or budget exhaustion. The worker has separate user, mount, process and network namespaces, with access restricted to its working directory, helper package and input model. Episodes allow 20 tool-call rounds and terminate when the model answers without a tool call or exhausts that budget.

Helper tools

The helper package h is documented in the system prompt (Table 1); the full signatures are listed in Appendix § 10.1. Inspection returns an entity’s placement chain, representation, openings and fills, wall connections, space boundaries, property sets and shared sub-entities. These structured results reduce the exploratory calls needed to recover relationships from unfamiliar IFC attributes.

Edit helpers are element-local: they translate a placement, stretch a wall end, move body vertices or a space-boundary surface, set a property, or delete a product and clean its relationships. Shared sub-entities are copied before modification so that changing a shared point does not unintentionally move another wall. The agent still selects which dependent products to edit, which ends to move and the direction of each change.

Gated checkers and the strict reward

The checkers read σ\sigma, MM and M′M'. Each returns a binary verdict; all must pass for strict success.

The specification
Validity

The checker parses M′M' and runs the IfcOpenShell schema validator (attribute types and cardinalities; EXPRESS where-rules are not evaluated) on MM and M′M'; an error already reported on MM is excluded, so the gate tests that the edit introduced no new schema error. The body geometry of every product the edit touched must rebuild in the IfcOpenShell geometry kernel, and a cycle in the decomposition graph (IfcRelAggregates, IfcRelNests) fails the gate, because geometry generation does not terminate on such a model. Validity here is limited to these schema, geometry and cycle tests. Full IFC or model-view conformance, structural consistency and regulatory compliance are outside their scope.

Intent The checker tests the requested change on the targets according to ι\iota: translation within tolerance, an attribute or property value, deletion, a dimension change, creation of the requested class at the specified location, or a retrieval answer in the final message. Leaving a requested edit undone fails this check and is also recorded by an under-edit indicator.

Dependency Each dependent djd_j in Δ\Delta must satisfy its rule ρj\rho_j in M′M', such as translating with the target, retaining its host, being removed or retained, or meeting or remaining joined to a moved wall. The rules test the resulting model. A product placed relative to the moved wall, as an opening is in many exports, translates with it automatically. The delete helper removes the deleted product’s relationships; the agent must separately remove the opening of a deleted door. A space face, a space-boundary surface or a joined wall end has its own geometry and must be edited by the agent; the tiers of Table 2 are ordered by how many such explicit dependent edits a task requires.

Dependency tierWhat must followReferenceHow the target is named
Tier 0attribute or property of one element; only the target may changeDirectby its GlobalId
Tier 1translate a wall with openings, delete a door, change a wall height, create an element; openings and fills translate with the wall and stay hosted; by the task’s policy a deleted door’s opening is removed and its host wall staysSpatialby geometry relative to a named entity: the nearest parallel wall north of a wall, the window on the storey above
Tier 2translate a wall that bounds spaces; as tier 1, and each bounded space’s face on the wall side meets the moved wall, its space boundary moving with itTopologicalthrough model relationships: the space bounded by a given set of walls, the wall that hosts a given door
Tier 3translate a wall joined to other walls; as tier 2, and each joined wall stays joined to the moved wallPathalong a route through the building: after entering a space through a given door, the wall on your left
Viewpointas seen from a standpoint: the wall facing you when looking in through a given window
OperationWhat the task asks forTarget elementWhat is targeted
Createa new wall, column, slab, space, door or window (with its opening)Wall, slab, columnIfcWall (including IfcWallStandardCase), IfcSlab and IfcColumn, each with its own body and contained in a storey; a wall also carries openings, joints and space boundaries
Updatemove, resize, rotate, re-host or re-attribute an existing element; its dependents must followDoor, window, openingan opening is the void cut into a host wall; a door or window is the fill hosted by an opening and moves with its wall
Deleteremove an element with its relationships; its dependents are removed or kept by ruleSpacean IfcSpace aggregated into a storey; space tasks are limited to interior rooms whose space boundaries to their walls are explicit
Retrieveanswer a question (count, length, area, volume, identifier); the model must stay unchangedProperty, quantitya value in an element’s property set, or a quantity derived from an element (length, area, volume, count): the targets of attribute updates and of quantity take-off
Compoundtwo of the above in one instruction

Table 2. Task taxonomy: the dependency tier (what the task’s policy requires to follow the target; everything outside the allowed set is protected by the preservation gate), how the instruction names its target, what it asks for, and the target elements. The two lists of each block are independent; rows do not correspond across them.

Preservation An entity-level diff between MM and M′M', in which a product counts as modified when its attributes, property sets, representation or placement change, yields the set Θ\Theta of entities added, removed or modified and the sets R+R^{+} and R−R^{-} of relationships added and removed. With A=T∪A∪{dj}∪N\mathcal{A}=T\cup A\cup\{d_j\}\cup N, where NN holds the new entities of the classes a creation task expects, preservation holds when nothing outside A\mathcal{A} changed:

p=1[Θ∖A=∅]⋅1[∀(a,b)∈R+∪R−:a∈A∨b∈A](3)p = \mathbf{1}\left[\Theta\setminus\mathcal{A}=\emptyset\right]\cdot\mathbf{1}\left[\forall(a,b)\in R^{+}\cup R^{-}:a\in\mathcal{A}\lor b\in\mathcal{A}\right] \tag*{(3)}

The second factor detects relationship removals between entities unrelated to the task, such as a space boundary or wall connection two rooms away. An entity-only diff misses these changes. Relationships with one end in A\mathcal{A} are governed by the listed dependency rules. If an allowed product has no dependency rule, its relationship to the target may change. When a wall that bounds a space is moved at tier one, the space is in A\mathcal{A} but not in Δ\Delta, so removing the space boundary between them passes pp; at tier two the space is in Δ\Delta with the rule that its face meets the moved wall and its boundary follows, so the removed boundary fails dd. A connection

Algorithm 1

Figure 1 (PDF p. 5). One task, from draw to training episode. T\mathcal{T} denotes the teacher, C\mathcal{C} the four checkers and ss strict success. Every gate discards on failure.

removed between two walls elsewhere on the storey fails pp at every tier.

The reinforcement reward is strict success from Eq. (1): r(τ)=sr(\tau)=s for an episode τ\tau with final commit M′M'. Different edit sequences receive the same reward when they satisfy the specification.

Tasks and data

For SFT, the data-construction pipeline instantiates a task on a synthetic building, verifies the task, paraphrases its prompt under guards and collects a teacher solution (Algorithm 1). Only episodes that pass strict success enter the supervised corpus. The separate test-data branch in Figure 1(a) constructs IfcEditBench tasks on held-out real IFC projects and applies the task-verification gates; the corpus and test split are detailed in § 3.4.4.

Synthetic buildings

Training buildings are synthesised from CubiCasa5K floor plans [23]. Each plan becomes a complete IFC model with standard walls (axis and extruded profile), wall joints, openings with doors and windows, spaces whose footprints meet the wall faces, space boundaries, stacked storeys with slabs and property sets; a share of the buildings tiles several plans along a corridor around a vertical core. The model is exported in a randomised style that imitates real authoring tools: metres or millimetres; IFC2X3, IFC4 or IFC4X3; rectangular or polyline profiles; Revit- or ArchiCAD-like names; one to four storeys, up to eight in the tiled buildings; wall and space bodies as extrusions, faceted Breps, clipped extrusions or face sets in the shares found in a survey of real exports, which included one of the test projects (Appendix § 8); and,

One task, from draw to training episode

Figure 1 (PDF p. 5, second).

in a fifth of the buildings, the sparse relation set of a Revit export without wall connections and space boundaries.

A survey of the wall openings in the real projects that are not used for evaluation showed that real exports almost never use the plan-profile extrusion of the first generator version: openings are elevation profiles extruded across the wall, rotated or polyline profiles, Brep or face-set bodies, and doors and windows carry shared type geometry through mapped items. A second generator mode draws these opening and fill styles per building with the surveyed shares, so that the training distribution covers the geometry the student meets on real files.

Task generation

Two generators derive tasks and their specifications σ\sigma from the building model. The cascade generator covers four dependency tiers (Table 2). Higher tiers require more dependents to follow the target; preservation protects everything outside the allowed set. Dependents are read from the model’s explicit relationships (IfcRelVoidsElement, IfcRelFillsElement, IfcRelSpaceBoundary, IfcRelConnectsPathElements). When an export lacks them, the generator does not infer them from geometry: a wall move on such a model is generated at the tier its explicit relationships support, and the grid templates that need a bounded space are skipped on that model. The verdict therefore depends on the export: the same wall move is a tier-two task in a project with space boundaries and a tier-one task in a project without them, where a bounded space is protected by the preservation gate instead of being required to follow. At tiers one and two, the dependents that the tier does not list may be left inconsistent without failing the task. Quantity sets are not compared by the checkers and are not recomputed by the helpers. A wall move at the deepest tier is also generated at tiers one and two, where space or joint changes are allowed but not required.

The grid generator covers creation, update, deletion and retrieval for walls, doors, windows, columns, slabs and spaces. Instructions identify targets directly by identifier, topologically through relationships, spatially through relative geometry, or along a path through the building. It also produces basic, geometric, topological and numeric retrieval, quantity take-off, compound instructions and operationally defined conceptual questions. The input building is a derived copy with a few columns added and spaces removed to provide locations for creation tasks. Each task is described by its dependency tier, reference style, operation and target elements. The tier depends on the operation and the target’s surroundings: retrieval and property edits have no dependents, most creation, deletion and update tasks occupy the next tier, and wall moves affecting spaces or connected walls occupy the deeper tiers.

Prompt paraphrase and gates

Generated tasks first pass static lint, a scripted oracle and a reference verifier. Lint checks identifiers and parameters. The oracle requires an independently scripted solution to pass all four checkers and a deliberately wrong solution to fail. The reference verifier checks that the instruction identifies its targets uniquely. Tasks with failing oracles are excluded as generator or checker defects.

Kimi K3 [27] paraphrases templated prompts in registers ranging from terse engineering requests to detailed client instructions. The generator retains the specification; the model changes the wording. Guards require every identifier, number and unit to remain verbatim, prohibit new identifiers, preserve retrieval answer formats and bound the rewrite length. Opposed directional terms, including left/right, north/south, clockwise/counter-clockwise, above/below and farthest/nearest, must retain their meaning. A failed rewrite is retried once with the violation specified, then replaced by the original template if it fails again. A separate model call checks semantic fidelity, including whether identifiers still refer to the same elements.

Corpus and splits

The supervised corpus contains 19,141 strictly successful teacher episodes spanning 3,059 synthetic buildings under Eq. (1). Buildings are counted by distinct identifiers in the final SFT training split. The separate synthetic validation split contains 2,270 episodes from 340 other buildings. Wall updates form the largest element–operation group in Figure 1. Real buildings are reserved for development and testing. The test set, IfcEditBench, contains 1,990 tasks from 13 held-out real projects in ifc-bench [17], excluded from training, pool selection and checkpoint selection. Tasks are drawn according to the training corpus’s element–operation distribution, with door, window and opening kinds over-sampled, at least 30 tasks per kind where available, and buildings balanced within each kind; 1,500 tasks cover all families and a supplement of 490 tasks covers doors, windows and openings (the fill tasks of the appendix). All pass the gates in § 3.4.3.

Training

IfcEdit has two training stages: imitation of verified teacher episodes (§ 3.5.1), followed by GRPO on the resulting student’s own rollouts (§ 3.5.2). The second stage draws tasks from a synthetic pool, executes the rollouts in IfcGym and uses strict success as the reward. Distillation [18, 39] exposes the student to teacher-visited states; on-policy training evaluates the trajectories produced by the student under the same success criterion.

Teacher distillation

Kimi K3 [27], used in thinking mode, receives the natural-language task instruction, access to the initial IFC model through the student’s tools, and an additional edit recipe in its system prompt. The recipe provides reusable procedures for editing elements, propagating dependent changes and resolving references. For example, it explains how a wall move should update touching wall ends, adjacent space footprints and space boundaries. The teacher resolves the task’s targets and parameters from the instruction and the model; the scripted oracle solution is not supplied. Collection allows 40 tool-call rounds, twice the student’s budget, with no online checker feedback. After each episode, the four checkers evaluate its final outcome, and only strictly successful episodes are retained. Exception-raising calls are removed from the trace. The student’s prompt retains the task instruction and tool documentation but omits the additional recipe. Let yy be the resulting message sequence of reasoning, tool calls and tool returns. The set G(y)\mathcal{G}(y) contains the positions generated by the model (reasoning and calls) and excludes the environment’s tool returns. The student minimises the negative log-likelihood of the generated positions,

LSFT(θ)=−∑(q,y)∈D∑t∈G(y)log⁡πθ(yt∣y<t,q),(4)\mathcal{L}_{\mathrm{SFT}}(\theta) = - \sum_{(q,y)\in\mathcal{D}} \sum_{t\in\mathcal{G}(y)} \log\pi_\theta(y_t \mid y_{<t},q), \tag*{(4)}

which supervises tool selection, argument construction and stopping throughout the episode. The student, Qwen3.5-9B [48, 62], is trained with low-rank adapters [19] on all linear layers of the language model (rank 64, scaling 128), learning rate 10−410^{-4} with cosine decay, three epochs, batch size 16 and sequences up to 32,768 tokens. We use the final checkpoint without selection. The same recipe trains the Qwen3.5-27B student of § 4.

GRPO on the strict reward

The second stage applies Group Relative Policy Optimization (GRPO) [10, 52] to the SFT student with the strict reward as the only signal. For a task xx the policy samples a group of GG episodes {τg}\{\tau_g\} in the sandbox under the evaluation protocol, each scored r(τg)=sr(\tau_g)=s by the checkers. advantage is the group-standardised reward,

Ag=r(τg)−μrσr,μr=mean⁡g′r(τg′),σr=std⁡g′r(τg′)(5)A_g = \frac{r(\tau_g)-\mu_r}{\sigma_r}, \qquad\mu_r = \operatorname{mean}_{g'} r(\tau_{g'}), \qquad\sigma_r = \operatorname{std}_{g'} r(\tau_{g'}) \tag*{(5)}

so no value network is required, and the policy maximises the clipped surrogate of PPO [51] with a KL penalty towards the SFT policy πref\pi_{\mathrm{ref}},

J(θ)=E[min⁡(wgAg,clip⁡(wg,1−ϵ,1+ϵ)Ag)]−βDKL[πθ∥πref],(6)\mathcal{J}(\theta) = \mathbb{E}\left[\min\left(w_g A_g,\operatorname{clip}(w_g,1-\epsilon,1+\epsilon)A_g\right)\right] - \beta D_{\mathrm{KL}}\left[\pi_\theta\parallel\pi_{\mathrm{ref}}\right], \tag*{(6)}

with wg=πθ(τg∣x)πθold(τg∣x)w_g = \frac{\pi_\theta(\tau_g \mid x)}{\pi_{\theta_{\mathrm{old}}}(\tau_g \mid x)} over the model-generated tokens only. Groups whose episodes all pass or all fail provide no reward-based contrast. Task selection and online filtering favour groups with mixed outcomes.

Task pool Tasks are drawn only from synthetic buildings and filtered offline with the starting policy. Each is attempted four times at sampling temperature one. Tasks solved on every attempt are discarded; those solved one to three times are retained. Unsolved tasks are capped at a fifth of the retained pool to limit groups with uniformly failing rollouts.

Optimisation Each step samples 64 prompts with G=8G = 8 rollouts per prompt under the evaluation protocol (budget of 20 rounds, no checker feedback) and updates the adapters in mini-batches of 8 prompts at learning rate 10−510^{-5} in the verl framework [53]. Online group filtering [64] drops every group whose eight rewards are equal and replaces it with new prompts until the step holds 64 mixed groups, and the KL coefficient β=10−2\beta= 10^{-2} keeps the policy close to the SFT start. The adapters are merged into the rollout weights after every step, so the policy that collects episodes is always the current one. Training runs for 100 steps; the final checkpoint is merged into the base weights and served like the SFT student.

Experiments

Experimental settings

Test sets IfcEditBench, the test set of § 3.4.4, holds 1,990 tasks grouped by the taxonomy in Table 2. BIM-Edit [43] provides 324 instructions, scored with our calibrated re-implementation of its evaluator. Appendix G describes its calibration and protocol reproduction. BIBIMBAP [29] provides 100 instructions with the authors’ executable per-task tests.

Models Kimi K3 in thinking mode supplies the teacher episodes (§ 3.5.1). The SFT student is Qwen3.5-9B trained with LoRA on the complete supervised corpus, including the second generator’s opening and fill styles. The final student adds 100 GRPO steps on a synthetic task pool that includes these styles. The 27B student uses the same SFT data and recipe on Qwen3.5-27B to assess model size; untrained Qwen3.5-27B with the same tools provides its starting point.

Baselines Prompted comparisons use the students’ CodeAct protocol, tools and turn budget without checker feedback: DeepSeek-V4-Flash [5] on all test sets, Gemma 4 31B on IfcEditBench, and Kimi K3, the teacher itself, on IfcEditBench and BIBIMBAP. Two published frameworks are evaluated on IfcEditBench with DeepSeek-V4-Flash: the released MCP4IFC tool server [42] and our re-implementation of IFC-Agent [11]. We also reproduce BIM-Edit’s prompted protocol with Gemma 4 31B to assess our harness and scorer, as detailed in Appendix G.

Protocol Models are served in bfloat16 with vLLM [31], or with SGLang [67] for Gemma 4 31B and Kimi K3, using common flags, temperature 0, thinking enabled and at most 16,384 output tokens per turn; Kimi K3 runs at its highest reasoning effort. The budget is 20 tool-call rounds, or 25 for BIBIMBAP as prescribed by that benchmark. Unless stated otherwise, models receive no checker feedback. Every model uses a 65,536-token context window, except Gemma 4 31B at 131,072 tokens; an episode that exceeds it counts as a failure. We report strict success in percent on IfcEditBench and the respective benchmark scores for external evaluation. Paired differences carry 95% cluster-bootstrap intervals [8], using target-element clusters stratified by building and 10,000 resamples. A pair includes only tasks for which both models returned an outcome; an episode without one counts as a failure in the table and is excluded from the pair.

Implementation Training uses verl [53], with its supervised trainer for SFT and hybrid engine for GRPO. The latter interleaves generation, sandbox execution and policy updates. Data construction, training and evaluation ran on a cluster of four nodes, each with eight H100 80 GB GPUs.

Main result

GRPO improves the 9B student and narrows its gap to the 27B SFT model. On the 1,990 IfcEditBench tasks, the final student reaches 86.78% compared with 79.15% for its SFT start, 88.29% for the 27B SFT student, and 59.35% and 16.08% for the untrained models Table 3. By instruction reference, the final student resolves 86.9% of direct, 86.2% of spatial and 90.7% of topological references, and 77.1% of the 70 path references, the hardest group for every model but the 27B GRPO student. By operation, GRPO lifts the SFT student most on creations, from 73.0% to 87.3%, and on compound edits, from 48.2% to 60.0%, while deletions already reach 90.3% after SFT. The prompted models show a different profile: DeepSeek-V4-Flash resolves 81.9% of topological references but completes 51.7% of deletions and 32.9% of compound edits, and the two published frameworks stay below 13% on creations and on path references. The 27B GRPO student reaches 91.76% and leads the students in every column except spatial and topological references. The 27B SFT student leads on spatial references; the two tie on topological references. The teacher exceeds the 27B GRPO student on spatial references and compound edits.

Model and trainingAll 1,990Direct 1,503Spatial 217Topol. 193Path 70Create 355Update 1,019Delete 331Retr. 200Comp. 85
Prompted models
DeepSeek-V4-Flash [5]61.1156.976.081.950.061.162.451.782.032.9
Gemma 4 31B [12]67.9966.174.783.444.360.864.788.275.042.4
Kimi K3 [27]88.0486.694.092.785.789.988.188.589.574.1
Qwen3.5-9B [48]16.0815.418.923.82.99.315.615.735.07.1
Qwen3.5-27B [48]59.3556.265.977.755.754.654.977.973.527.1
Published frameworks
MCP4IFC [42]39.4039.739.249.710.012.436.264.469.522.4
IFC-Agent [11]13.6211.610.635.28.60.014.20.063.00.0
IfcEdit (ours)
Qwen3.5-9B, SFT79.1578.783.481.970.073.079.790.382.048.2
Qwen3.5-9B, SFT + GRPO86.7886.986.290.777.187.386.794.086.060.0
Qwen3.5-27B, SFT88.2987.689.992.785.785.489.494.388.064.7
Qwen3.5-27B, SFT + GRPO91.7692.287.692.791.491.892.297.391.065.9

Table 3. IfcEditBench, 1,990 tasks: strict success (%) overall, by instruction reference (the seven viewpoint tasks count in All only) and by operation. Blocks: prompted models, the two published frameworks run with the DeepSeek-V4-Flash backbone, and the IfcEdit students. Bold: best among the IfcEdit students.

The paired comparisons in Figure 2 distinguish model size from the reinforcement stage. Applying the same SFT recipe and data to the larger model gives +9.15 points, with interval [7.42, 11.05]; the difference is positive in 12 of the 13 projects and zero in the remaining project. GRPO adds +7.75 points [5.79, 9.47]. The larger SFT model retains a lead of +1.51 points [−0.05, 3.48] over the final student, while the door, window and opening difference is −0.26 [−2.99, 2.41]; both intervals include zero. Improvements are prominent for spaces and for walls, slabs and columns; the size and reinforcement contrasts on property and quantity tasks are not clearly separated from zero. GRPO on the 27B SFT student, with the same recipe, reaches 91.76% (Table 3); it adds +3.42 [1.99, 4.71] over its SFT start.

Figure 2. Paired strict-success differences and 95% cluster-bootstrap intervals for student size, GRPO on each student and the remaining size gap, overall and by target element. Identifier retrieval and compound tasks count in the overall group only; pair counts exclude tasks without an outcome.

Figure 3 splits the results in Table 3 by the dependency tiers defined in Table 2. Apart from the untrained smaller model and the teacher, the prompted models solve 76.5% to 85.8% of the tier-0 tasks, which touch one element, but 13.0% to 38.0% of the tier-2 tasks, where a translated wall must carry its space boundaries, and the two frameworks fall to 2.0% and 0.0% there. The teacher solves 92.5% of the tier-0 and 74.0% of the tier-2 tasks. Every trained student stays at or above 75% on all four tiers; the 27B GRPO student reaches 100.0% and 99.0% on tiers 2 and 3. The largest differences between trained and prompted models occur on these dependent edits. The values of both figures are tabulated in Appendix H.

Figure 3. Strict success (%) by dependency tier for prompted models, published frameworks and trained students. Tiers 2 and 3 hold the wall translations whose bounded spaces or joined walls must follow.

The checker breakdown (Table 4) shows higher intent, dependency and preservation pass rates after GRPO. Validity is already high and changes little, while intent pass rates differ most across models. The larger student after GRPO has the highest intent pass rate of all students.

Model and trainingValid.IntentDep.Pres.
Prompted models
DeepSeek-V4-Flash [5]96.068.380.496.4
Gemma 4 31B [12]96.577.590.393.0
Kimi K3 [27]99.491.895.796.4
Qwen3.5-9B [48]97.819.556.797.7
Qwen3.5-27B [48]98.865.680.995.7
Published frameworks
MCP4IFC [42]85.150.376.088.5
IFC-Agent [11]90.324.456.175.8
IfcEdit (ours)
Qwen3.5-9B, SFT99.280.992.696.8
Qwen3.5-9B, SFT + GRPO99.187.495.698.0
Qwen3.5-27B, SFT98.789.696.797.3
Qwen3.5-27B, SFT + GRPO98.892.598.097.5

Table 4. IfcEditBench: share of the 1,990 tasks (%) that pass each checker individually, for prompted models, published frameworks and trained students. Strict success requires all four. Bold: best among the IfcEdit students.

Prompted and framework references. The trained students outperform every prompted comparison other than the teacher under the shared tool interface. DeepSeek-V4-Flash reaches 61.11% strict success, about two points above the un-trained larger model and 25.68 points [23.12, 28.10] below the final student. Gemma 4 31B reaches 67.99%, leaving a gap of 18.79 points [16.35, 20.90]. The teacher, Kimi K3, reaches 88.04% under the student’s budget of 20 rounds, 1.16 points [−0.73, 3.36] above the final student and 3.72 points [1.78, 5.59] below the 27B GRPO student; the interval includes zero for the 9B comparison and excludes zero for the 27B comparison. Twenty-eight of the teacher’s episodes exceeded the context window and count as failures. The published frameworks score lower with the same DeepSeek-V4-Flash backbone: 39.40% for MCP4IFC and 13.62% for IFC-Agent, which averages 130 tool calls per task. Neither provides a dependency-aware edit primitive. MCP4IFC’s reported result is the better of two tool-output caps; context overflow at the 65,536-token limit affects 5% of tasks with the tighter cap and 27% with the looser cap. The frameworks differ in both tool interface and execution constraints; the prompted-model comparisons hold the interface fixed to assess training.

External benchmarks

The trained students improve over their starting models on both external benchmarks. On BIBIMBAP (Table 5), the

ModelDirectSpatialTopol.NumericConcept.CreateReadUpdateDeleteAll
Prompted models
Claude Opus 4.6 [1]†65.8363.8953.0638.3330.0081.147.361.518.950.22
GPT-5.4 [47]†62.1542.0820.5626.675.0056.022.450.420.031.29
Gemini 3 Flash [15]†65.4240.9721.1120.006.6745.620.646.737.830.83
GPT-5.2 [45]†52.5034.5835.425.0025.0058.918.251.126.730.50
GPT-5 Mini [46]†50.9425.8325.830.0025.0031.210.955.543.325.00
GPT-4.1 [44]†38.3330.0031.250.0020.0029.49.060.036.623.92
DeepSeek-V4-Flash [5]68.3355.8353.3335.0025.0076.741.873.313.347.50
Kimi K3 [27]77.5067.0875.0035.0060.0091.760.071.136.762.92
Qwen3.5-9B [48]45.0040.0026.6725.0015.0016.729.162.216.730.33
Qwen3.5-27B [48]66.2560.4266.6735.0035.0073.349.171.126.752.67
IfcEdit (ours)
Qwen3.5-9B, SFT77.5057.0865.8330.0030.0091.741.868.933.352.08
Qwen3.5-9B, SFT + GRPO79.1760.0073.3330.0030.00100.041.873.336.754.50
Qwen3.5-27B, SFT76.6766.6775.8330.0050.00100.052.775.630.059.83
Qwen3.5-27B, SFT + GRPO79.1771.6773.3345.0050.00100.060.075.630.063.83

Table 5. BIBIMBAP, 100 tasks: score (%) by category, operation and overall. Rows marked † are reported by the benchmark [29]. Blocks distinguish prompted models and IfcEdit students; bold marks the best value among the rows we evaluated.

final student scores 54.50, its SFT start 52.08, the larger SFT student 59.83 and the untrained larger model 52.67; the final and larger students exceed the published Claude Opus 4.6 score of 50.2% [29]. The teacher, Kimi K3, scores 62.92 there, 8.42 points [1.42, 15.75] above the final student and 0.92 points [−8.00, 6.00] below the 27B GRPO student, with an interval that includes zero. On BIM-Edit (Table 6), the final student scores 55.19, its SFT start 50.46, the larger SFT student 54.76 and the untrained larger model 35.34. GRPO adds +4.73 points [2.77, 6.70], similar to the size effect of +4.30 [2.11, 6.57]. DeepSeek-V4-Flash scores 47.50 on BIBIMBAP and 41.63 on BIM-Edit. Our BIM-Edit scorer differs from the official evaluator used for published rows, so those cross-block comparisons remain approximate; Appendix G reports the reproduction.

ModelFinalGeom.Sem.Topo.
Prompted models
Gemini 3 Flash [15]†49.4868.8741.8137.77
Qwen 3.6 Plus [49]†47.6359.2135.9347.77
Claude Sonnet 4.6 [2]†45.3155.9033.8546.19
GPT-5.4 Pro [47]†43.9450.1438.6942.97
DeepSeek V3.2 [4]†43.2150.9734.8043.86
GPT-5.4 Mini [47]†39.7945.1838.8035.38
Gemma 4 31B [12]†37.5444.9429.9437.75
DeepSeek-V4-Flash [5]41.6346.7432.5645.60
Qwen3.5-9B [48]21.5310.7229.4024.46
Qwen3.5-27B [48]35.3437.2131.6837.14
IfcEdit (ours)
Qwen3.5-9B, SFT50.4661.0336.0654.29
Qwen3.5-9B, SFT + GRPO55.1971.5834.8859.12
Qwen3.5-27B, SFT54.7669.9533.6660.68
Qwen3.5-27B, SFT + GRPO56.9475.5032.8262.50

Table 6. BIM-Edit, 324 tasks: average final, geometry, semantics and topology scores. Rows marked † are reported by the benchmark [43] with the official scorer; the other rows use our calibrated re-implementation. Blocks distinguish prompted models and IfcEdit students; bold marks the best value among the rows we evaluated.

The 27B GRPO student scores 63.83 on BIBIMBAP and 56.94 on BIM-Edit, against 59.83 and 54.76 for its SFT start.

Expert evaluation A practitioner reviewed 30 IfcEdit-Bench edits of the final student: 15 strict passes and 15 failures under the checkers. The review used before-and-after views with the changed elements highlighted. The same practitioner also reviewed 20 blind pairs on the public benchmarks (Figure 4); one pair was left unanswered because the benchmark instruction contains a malformed coordinate. The expert judged 21 of the edits and marked 9 as not judgeable from the views; on the 21, the verdict agrees with the four-gate verdict in 19 cases, rejecting 10 of the 11 checker failures and accepting 9 of the 10 passes. In the blind pairs the final student was preferred in 10 and tied as acceptable in 7, against 2 for the other output; against the released GPT-5.4 Pro outputs on BIM-Edit it was preferred in 4 of 5 pairs. Where a side was preferred, the choice agrees with the sign of the benchmark score in 10 of 12 pairs.

Figure 4. Expert evaluation by one practitioner. (a) Verdict on 30 IfcEditBench edits of the final student, 15 strict passes and 15 failures under the four checkers, judged from before-and-after views. (b) Blind pairwise preference between the final student and another output on the public benchmarks.

Figure 5 compares six creation, deletion and room-level edits from the final student, its SFT start and the other evaluated systems on the two benchmarks. Further IfcEditBench edits and benchmark pairs are shown in Appendix L.

Six public-benchmark edits across comparison models and trained students

Figure 5. Six public-benchmark edits (columns) across comparison models and trained students (rows). Section views mark additions in green, removals in red using original geometry, and modifications in orange. Labels give BIM-Edit scores under the re-implemented evaluator or the number of BIBIMBAP checks passed.

Ablations

Figure 6 compares reward terms, supervised data scale and evaluation-time settings.

Figure 6. Ablations of the 9B student as paired differences in strict success on IfcEditBench with 95% cluster-bootstrap intervals: the intent-and-validity reward against the four-gate reward, both after GRPO; SFT on nested fractions of the corpus against SFT on the full corpus; and the final student with a doubled turn budget or with checker feedback against its standard evaluation.

Reward terms The ablation repeats GRPO with reward 1 when intent and validity pass and 0 otherwise, logging the other gates without rewarding them. The intent-and-validity policy reaches 82.01% on IfcEditBench, 52.14 on BIM-Edit and 57.75 on BIBIMBAP; its paired differences from the four-gate policy are −4.83 [−6.65, −3.10], −3.06 [−4.89, −1.29] and +3.25 [−0.25, 7.42]. Adding dependency and preservation to the reward improves IfcEditBench performance.

Data scale Much of the IfcEditBench performance is obtained from a fraction of the supervised corpus. The ablation uses nested random quarters with the same trainer seed. The 25%, 50% and 75% students reach 75.38%, 78.39% and 79.35% on IfcEditBench, paired differences from the full-corpus student of −3.72 [−5.58, −1.82], −0.70 [−2.52, 1.23], +0.20 [−1.46, 1.92]. On BIM-Edit they score 47.41, 49.24 and 52.94 against 50.46, differences of −3.05 [−5.50, −0.63], −1.22 [−3.60, 1.15], +2.48 [−0.07, 5.05]; on BIBIMBAP 47.00, 51.67 and 51.58 against 52.08, differences of −5.08 [−11.08, 0.58], −0.42 [−6.33, 5.58], −0.50 [−6.42, 5.33].

Budget and feedback Doubling the budget to 40 rounds leaves the final student at 86.38% on IfcEditBench, a paired difference of −0.40 [−1.84, 1.56] from its standard evaluation; returning the four checker verdicts after each executed step gives 87.29%, +0.40 [−1.14, 2.09].

Training steps Figure 7 follows both GRPO runs through their exported checkpoints. For the 27B student, IfcEditBench strict success rises from 88.29% to 90.20% at step 25 and 91.46% at step 50, then stays between 91.76% and 92.01%. For the 9B student it rises from 79.15% to 83.57% at step 25 and 85.43% at step 50, reaches 85.93% at step 75 and 86.78% at step 100. Most of the GRPO gain occurs in the first half of training at both scales. Appendix § 11.0.0.0.1 lists every exported 27B checkpoint with its benchmark scores.

Figure 7. GRPO on the 27B and 9B SFT students: strict success (%) on the 1,990 IfcEditBench tasks at the exported checkpoints. Step 0 is the SFT student (square); step 100 the final GRPO student. Bars give 95% cluster-bootstrap intervals.

Discussion

The trained students complete dependent edits, such as moving a wall with its space boundaries and joined walls, more often than the other prompted models. The small student approaches its teacher’s performance with the same tools and a fraction of its parameters. An in-house deployment could use such a model for routine edits while keeping project data local. For tasks with an executable specification, the four checkers could also control whether an authoring tool accepts an edit: validity, intent, dependency and preservation must all pass. Free-form instructions would first need to be converted into specifications.

Engineers can inspect and extend the domain rules in the task generator and checkers [56], while the policy learns to satisfy them through the tools. Executable specifications allow training tasks to be generated on synthetic buildings and the same checks to evaluate edits on real projects, without target files. Applying this approach to other structured artefacts, such as parametric CAD assemblies, structural analysis models, product lifecycle data or schema-bound databases, would require task generators with checkable specifications and a sandbox for the agent’s code.

Strict success measures compliance with the task specification. Validity checks for new schema errors and geometry failures relative to the input; dependent elements are checked against the rules supplied by the generator and the relationships present in the export. Broader conformance and consistency require additional checks. Coverage of real export geometry depends on the synthetic styles used for training. Projects and generation seeds are held out, but task templates are shared between training and test, leaving transfer to unseen templates untested. The comparisons include two frontier models, one of them the teacher, one open model and two published frameworks on one backbone. The agent edits the element and property kinds represented in its training data. Extending the dependency rules to relations missing from the current specifications and evaluating on held-out templates would test a wider range of edits.

Conclusion

We train IFC editing agents against task specifications that check the requested change, its dependents and the preservation of unrelated entities. IfcGym provides a replayable sandbox and four checkers for validity, intent, dependency and preservation. Their conjunction selects training episodes, supplies the reinforcement reward and measures strict success without comparing with a target file.

With tool access held fixed on IfcEditBench, verified teacher distillation followed by GRPO brings the small open model close to its teacher’s performance; the same recipe takes the larger student beyond the teacher. The gain from reinforcement is comparable to the gain from increasing student size. Both students improve over their starting models on the public benchmarks and outperform the other prompted models and evaluated frameworks on IfcEditBench.

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant No. 52568010).

Code and data availability

The IfcGym environment and the IfcEdit code (agent, evaluation runners, training recipe) are released under the Apache License.2 IfcEditBench is available with the real projects under their original licences (Creative Commons Attribution, MIT, GPL and the KIT terms of use, as listed with each project) and with the task specifications under Creative Commons Attribution.3 The synthetic training buildings, the generated tasks, the supervised corpus and the reinforcement task pool are available under Creative Commons Attribution-NonCommercial, as the floor plans derive from CubiCasa5K.4 BIM-Edit and BIBIMBAP are distributed by their authors.

References

The test projects

IfcEditBench draws its 1,990 tasks from 13 real projects of the ifc-bench collection [17], none of which appears in training, pool selection or checkpoint selection; the seven other projects of the collection on disk served as development projects. Tables 7 and 8 list, for every test project, the schema, length unit and authoring tool recorded in the file header, the element counts, the representation types of the wall bodies, the relationship classes the generators read, and the number of tasks by dependency tier. The projects span three schema versions, three length units, Revit, ArchiCAD and a Synchro re-export, and wall bodies from plain extrusions to clipped extrusions, faceted Breps and polygonal face sets; six projects carry neither wall connections nor space boundaries, two carry wall connections only, and one (a Synchro re-export) carries no openings, fills or spaces at all. The cascade templates read a project’s arc.ifc directly (603 tasks); the grid templates read a derived copy, arc_v4.ifc, that differs from it only by three to six added columns and one or two removed spaces, which provide locations and targets for creation tasks (1,387 tasks).

ProjectSchemaUnitAuthoring toolProductsWallsDoorsWindowsOpeningsSlabsColumnsSpacesStoreys
4351IFC2X3ftRevit Architecture 201144143173326065
city_house_munichIFC4X3mRevit 2025293831613331901915
dental_clinicIFC2X3mRevit Architecture 20113,2981,08025458403302694
digital_hubIFC4mRevit 20191,02617864472442362643
duplexIFC2X3mRevit Architecture 201129557142450210214
fantasy_hotel_1IFC4mRevit 20241,0825325731384939264
fantasy_hotel_2IFC4mmRevit 20241,571118721902857886206
fantasy_office_building_1IFC4mRevit 20251,8466434421201834445
fantasy_office_building_3IFC4mRevit 20241,460223751673041720876
molioIFC2X3mmArchiCAD 254,192189119119245531411710
samuel_macalisterIFC4mmRevit 201452448161733123146
schependomlaanIFC2X3mmSynchro re-export3,50888020525902792301
smiley_westIFC4mArchiCAD 201,27228117080280120201405
All 1320,8083,2971,0811,0922,16769830182774

Table 7. The 13 test projects (a): schema, length unit and authoring tool from the file header, and element counts in the original arc.ifc.

Tasks and relationships The relationships present in each file determine which dependency tiers the generators can construct (Section C, Table 8). Tier-2 tasks, which require bounded spaces to follow a moved wall, exist only in the five projects with space boundaries; tier-3 tasks, which require joined walls to stay joined, in five of the seven projects with path connections, and in city_house_munich, which exports connections but no boundaries, all 37 deep-tier tasks are joint tasks without a space rule. The six projects without either relationship class, and samuel_macalister, which has no deep-tier task, contribute tier-0 and tier-1 tasks only: attribute and property edits, deletions, creations, retrievals and the fill-supplement tasks on doors, windows and openings. By reference style, 815 tasks name the target by identifier, 217 by relative geometry, 193 through relationships, 70 along a route and 7 from a viewpoint; 85 are compound instructions and the 603 cascade tasks use templated direct references.

ProjectWall bodiesConnectionsBoundariesVoidsFillsTasks T0/T1/T2/T3Total
4351S 58, C 37, M 565142322022/153/16/16207
city_house_munichS 53, C 479333332622/174/0/37233
dental_clinicS 96, C 2, M 12,1033,12440330223/184/32/16255
digital_hubS 81, F 190024410914/69/0/083
duplexS 91, C 7, M 282265503818/166/15/14213
fantasy_hotel_1S 64, F 36001389815/68/0/083
fantasy_hotel_2S 92, F 80028526217/65/0/082
fantasy_office_building_1S 91, F 9001207224/64/0/088
fantasy_office_building_3S 27, F 730030424222/61/0/083
molioB 52, – 482821,97324523831/136/6/0173
samuel_macalisterS 54, C 35, B 10750332419/116/0/0135
schependomlaanB 100000034/67/0/0101
smiley_westS 96, I 43691,68928025020/186/31/17254
All 13S 56, B 30, F 7, C 33,0697,1932,1671,681281/1,509/100/1001,990

Table 8. The 13 test projects (b): share of walls by Body representation type (S swept solid, C clipping result, B faceted Brep, F polygonal face set, M surface model, I mapped item, “–” axis only), counts of path connections, space boundaries, voids and fills relationships, and IfcEditBench tasks by dependency tier.

Sources and licences The projects come from the public ifc-bench collection; their individual licences, recorded with each project, are CC BY 4.0 (city_house_munich, dental_clinic, duplex, molio, schependomlaan), MIT (digital_hub and the four fantasy projects of TUM student exercises), GPL v3 (4351 and samuel_macalister, from the BIMserver test files) and the KIT terms of unrestricted use with a citation request (smiley_west). The IfcEditBench release retains each project’s original licence.

The export-style surveys Two surveys set the export-style shares of the synthetic training buildings. The body-style shares (extrusion, faceted Brep, clipped extrusion, face set, and the share of sparse exports) were read in October 2026 from 33 real models on disk: the seven development projects of ifc-bench, the two BIBIMBAP models and 24 BIM-Edit models, and also schependomlaan, which at that time was the only test project on disk; its 880 Brep walls contributed to the Brep share. No task, instruction or checker output of that project entered training; the share of Brep wall bodies in the synthetic buildings reflects, in part, this project’s walls. The opening and fill survey (placement parent, body representation of doors, windows and openings, extrusion direction of opening profiles) used the seven development projects only, and none of the 13 test projects.

The distillation corpus and the reinforcement-learning pool

Size and provenance The supervised corpus holds 19,141 training episodes on 3,059 synthetic buildings and 2,270 validation episodes on 340 other buildings; no building appears in both splits, and no real project appears in either. Every episode was produced by Kimi K3 in thinking mode with the edit recipe of the main text and no checker feedback during the episode. Of the 24,276 teacher episodes that were run, 23,266 (95.8%) passed all four checkers. Removing duplicate task solutions and episodes longer than the sequence limit (fewer than sixty) leaves 21,411 episodes across the training and validation splits. Of the 21,099 task prompts with a paraphrase record, 20,612 (97.7%) were paraphrased; the remainder kept the template wording after the rewrite failed the guards twice. The 284 templates instantiate all task families of the main text (Table 9): wall updates, including the three cascade tiers of wall moves, are the largest group, followed by retrieval, slab updates, quantity take-off, property edits and compound instructions. Each of the element kinds that IfcEditBench over-samples (doors, windows and openings) is represented by several hundred episodes per operation.

Target elementCreateUpdateDeleteRetrieveTotal
Wall3364,581296–5,213
Slab6631,204382–2,249
Column503609452–1,564
Door530253740–1,523
Window551755625–1,931
Opening–216206–422
Space384781349–1,514
Property set–1,074––1,074
Quantity take-off–––1,1181,118
Retrieval (identifiers, geometry, topology)–––1,4241,424
Compound instructionstwo operations each1,043
Other–66––66
Total2,9679,5393,0502,54219,141

Table 9. Composition of the training split of the distillation corpus (episodes) by target element and operation. Property edits, quantity take-off, retrieval and compound instructions are listed as their own rows; “other” collects the few updates of elements outside the main kinds.

Table 9. Composition of the training split of the distillation corpus (episodes) by target element and operation. Property edits, quantity take-off, retrieval and compound instructions are listed as their own rows; “other” collects the few updates of elements outside the main kinds.

Tiers and reference styles By dependency tier the training split holds 3,536 episodes at tier 0, 13,205 at tier 1 and 1,200 each at tiers 2 and 3; the two deepest tiers are wall moves whose neighbouring walls or bounding spaces must follow. Among the grid-generator templates that vary the way an instruction refers to its target, 6,308 episodes name the target directly by identifier, 2,534 through a relationship, 2,450 through relative geometry, 645 along a path through the building and 111 from a viewpoint; the remaining 7,093 episodes come from the cascade, compound and single-form templates.

Buildings The 3,059 training buildings cover the export styles of the generator: 2,045 are modelled in metres and 1,014 in millimetres; 736 are exported as IFC2X3, 1,321 as IFC4 and 1,002 as IFC4X3; 602 carry the sparse relation set of a Revit-like export; 338 tile several plans along a corridor. Wall and space bodies are plain extrusions in 1,424 buildings, faceted Breps in 595, face sets in 649 and clipped extrusions in 391. Of the 3,000 buildings of the main generator, 1,090 have one storey, 825 two, 453 three, 452 four and the 180 remaining tiled buildings five to eight; 3,399 of the 3,400 generated buildings appear in the two splits. The 400 buildings of the second opening and fill generator (through-wall opening extrusions, rotated and polyline profiles, mapped door and window geometry) contribute 1,160 training and 129 validation episodes on door, window, opening and quantity tasks.

Episodes An episode is the message sequence of system prompt, task instruction, the teacher’s reasoning and tool calls and the sandbox returns. The training episodes have a median of five assistant turns (mean 5.9, 90th percentile 8, maximum 31) and five tool calls (mean 6.2, 90th percentile 10, maximum 39); 959 episodes use at most two calls, 9,119 three to five, 7,553 six to ten and 1,510 more than ten. 15,440 episodes end with exactly one commit(), 1,128 commit two or more times, and 2,573 retrieval and quantity episodes answer without a commit. Measured with the student’s tokenizer, an episode has a median length of 5,972 tokens (mean 7,216, 90th percentile 12,809, 99th percentile 22,901, maximum 31,278), of which the generated part, the reasoning, the tool calls and the final answer, is a median of 1,766 tokens (mean 2,646, 99th percentile 14,680), 37% of all tokens; the sandbox returns and the fixed system prompt make up the rest. The teacher’s reasoning is 42% of the generated text. The validation split has the same distributions.

The reinforcement-learning pool The GRPO pool holds 5,895 tasks on 2,456 synthetic buildings: 1,095 at tier 0, 4,428 at tier 1, 165 at tier 2 and 207 at tier 3; by family, 1,829 creation, 1,302 update, 578 deletion, 849 retrieval and quantity, 426 compound, 664 cascade wall-move, 246 reference-resolution tasks and one spatial-reference task. 3,860 of them are tasks of the supervised corpus and 2,035 were generated for the pool and never shown to the teacher; 5,817 carry paraphrased prompts. Every task was attempted four times by the SFT student at temperature one before training: 458 were never solved, 689 once, 1,408 twice and 3,340 three times; tasks solved on all four attempts were discarded. This selection favours tasks that produce mixed outcomes in a group of eight rollouts.

Task specification and the four checkers

All tolerances are in metres; model units are converted with the file’s unit scale (IfcOpenShell calculate_unit_scale), and meshes are generated in world coordinates with opening subtraction disabled.

The specification

A task record holds the identifier, project, template, dependency tier, instruction and input file, and the specification σ\sigma in four fields: target, the GlobalIds the instruction is about (for a creation task the storey the new element goes to); allowed, the GlobalIds that may change without counting as side effects; intent, the predicate of the intent checker with its parameters; and dependents, a list of (GlobalId, rule) pairs for the dependency checker, with rule-specific parameters such as the move vector or the class of an expected new element. Creation tasks add allowed_new_classes, the classes whose new instances are expected. The first example is a tier-2 wall move in a Revit 2011 export; the two connected walls are allowed to change but carry no joint rule, because tier two relaxes joints. The second is a door creation.

{"task_id": "dental_clinic-T2-move-0122-s2", "project": "dental_clinic",
 "template": "T2-translate-wall-bounding-spaces", "tier": 2,
 "prompt": "Move the wall with GlobalId 3NhaIUfh12PAdrGa$S3xpR by 0.75 m in the -x direction of
    the project coordinate system. Its openings and the doors/windows in them must move with it and
    stay hosted, and the spaces it bounds and their space boundaries must follow the new wall face.
    Walls connected to it may be adjusted as needed. Do not change anything else.",
 "target": ["3NhaIUfh12PAdrGa$S3xpR"],
 "allowed": ["3NhaIUfh12PAdrGa$S3xpR", "3eM8WbY_59RR5TDWs3xrFu", "0ClPCUC7jCQRnj1dhupMRJ",
    "3Egy$LBd99weR14rW4bge1", "3NhaIUfh12PAdrGa$S3xqK"],
 "intent": {"type": "translate", "guid": "3NhaIUfh12PAdrGa$S3xpR", "vector": [-0.75, 0.0, 0.0], "tol": 0.02},
 "dependents": [
    {"guid": "3eM8WbY_59RR5TDWs3xrFu", "rule": "translate_with", "vector": [-0.75, 0.0, 0.0]},
    {"guid": "3eM8WbY_59RR5TDWs3xrFu", "rule": "stays_hosted"},
    {"guid": "0ClPCUC7jCQRnj1dhupMRJ", "rule": "space_meets_wall", "wall": "3NhaIUfh12PAdrGa$S3xpR",
     "vector": [-0.75, 0.0, 0.0]},
    {"guid": "0ClPCUC7jCQRnj1dhupMRJ", "rule": "boundary_on_wall", "wall": "3NhaIUfh12PAdrGa$S3xpR",
     "vector": [-0.75, 0.0, 0.0]}],
 "meta": {"n_openings": 1, "n_spaces": 1, "n_connected": 2}, "ifc": ".../dental_clinic/arc.ifc"}
{"task_id": "4351-C-door-0174-s2", "project": "4351", "template": "C-create-door", "tier": 1,
 "prompt": "Add a door 0.9 m wide and 2.1 m high to the wall with GlobalId 0S4TGKDX5FsvETM2IopwXc,
    anywhere along the wall. The door must be hosted by the wall through an opening
    (IfcRelVoidsElement / IfcRelFillsElement). Do not change anything else.",
 "target": ["0S4TGKDX5FsvETM2IopwXc"], "allowed": ["0S4TGKDX5FsvETM2IopwXc"],
 "allowed_new_classes": ["IfcOpeningElement", "IfcDoor"],
 "intent": {"type": "create_fill", "host": "0S4TGKDX5FsvETM2IopwXc", "kind": "door",
    "width": 0.9, "height": 2.1, "tol": 0.03},
 "dependents": [], "ifc": ".../4351/arc.ifc"}

Validity

The checker opens M′\mathcal{M}' (an unparsable file fails every gate) and applies three tests. (i) A cycle in the decomposition graph: following IfcRelAggregates and IfcRelNests from every part, no object may reach itself. The test runs first because the geometry kernel does not terminate on such a model. A prompted model produced this failure by aggregating a storey into itself. (ii) New schema errors: the IfcOpenShell validator is run on M\mathcal{M} and on M′\mathcal{M}' without EXPRESS rules, which checks that no abstract entity is instantiated and that every attribute value has the declared type, enumeration member, aggregate element type and cardinality, including inverse cardinalities; each logged line is one error, and the gate fails if M′\mathcal{M}' has an error line that M\mathcal{M} does not. (iii) Geometry: for every product the edit touched (added or modified, Section C.5) that has a representation, the IfcOpenShell geometry kernel must build its shape without raising. The gate does not evaluate EXPRESS where-rules, geometric plausibility such as clashes or overlaps, the geometry of untouched products, or anything already wrong in the input export.

Intent

Table 10 lists the intent types with their test and tolerance. Displacements are measured on mesh centroids in world coordinates, falling back to the placement origin when a mesh is unavailable. The task files set the tolerances explicitly; the table gives the values used in IfcEditBench. A compound intent is a list of parts that must all pass. A retrieval intent is judged on the last JSON object of the final message that carries a value, global_id or global_ids key. An intent type unknown to the checker fails.

IntentTestTolerance
translatecentroid displacement equals the requested vector0.02 m
set_attribute, set_propertythe attribute, or the property in the named property set, equals the value (numbers: relative tolerance)exact; 0
delete, delete_allthe product (every listed product) no longer exists–
resize_heightmesh height equals the requested height; where given, the top or bottom elevation as well0.02 m
wall_axis (shorten, rotate)axis end points match the requested points in either order, and the body follows the axis: both ends moved like the nearest axis end (parallel case) or the body’s principal direction aligned with the new axis with its length kept0.03 m per end; 0.1 m body
set_thickness, set_thickness_sideprofile and body thickness across the axis; for the one-sided variant the fixed face stays in place0.02 m
bbox, create_wall_bboxbounding box of the (new) product equals the requested box0.05 m
create_fillexactly one new door or window, hosted through an opening that voids the host wall, with the requested width and height, within the wall’s length, not overlapping another opening; sill and plan centre where given0.03 m
create_wall, create_columnnew product with the requested axis end points (sum of end errors), thickness and height, or plan centre, size and height; contained in the requested storey0.03 m
create_slab, create_space, space_footprintfootprint polygon within 5 % of the requested area in symmetric difference (3 % for an edited space footprint) plus 0.05–0.1 m2^2; thickness, height or top elevation; storey0.03–0.1 m
rehost_fill, opening_width, create_slab_openingthe fill’s opening voids exactly the new host at the requested position; the opening’s extent along the wall and its centre; a new opening in a slab with the requested plan box through the slab’s thickness0.02–0.05 m
answeridentifier: exact; identifier set: equal; number: within the larger of an absolute and a relative tolerance (areas 1 %, quantities 2 %, counts exact)as stated

Table 10. Intent types: what is tested on the target products of M′\mathcal{M}' and the tolerance used in IfcEditBench.

Dependency

Every dependent carries one rule; the gate is the conjunction over all dependents, and a rule the checker does not know fails. Table 11 lists the rules with their tests, and the number of dependents using each rule in IfcEditBench. Rules that compare with the input are vacuous when the input lacks the evidence: a space that did not meet the wall in M\mathcal{M}, or a space boundary without evaluable connection geometry, passes its rule with the verdict “not applicable”. Rules test outcomes, so an opening that moves with its wall through its relative placement passes translate_with without an explicit edit.

RuleTestUses
translate_withthe dependent’s centroid displacement equals the target’s move vector (0.02 m)958
stays_hostedthe opening still voids a wall and, if it had a fill in M\mathcal{M}, still has one614
removed, presentthe dependent no longer exists; still exists487, 32
space_meets_wallalong the move axis, the surface face of the space mesh (restricted to the wall’s extent) lies within 0.02 m of a face of the moved wall, as it did in M\mathcal{M}374
boundary_on_wallevery space boundary between the space and the wall with connection geometry, transformed through the space’s placement, lies within 0.02 m of the wall’s centre or faces along the move axis, as it did in M\mathcal{M}374
unchangedthe dependent is neither removed nor modified (no attribute, property, placement or geometry change)353
joined_tothe surface gap between the dependent wall and the moved wall is no larger than in M\mathcal{M} plus 0.02 m; the gap is zero when the solids interpenetrate, else the minimum point-to-triangle distance between surface samples278
space_boundary_with, space_boundary_new, new_space_boundsthe space’s boundaries include the element; a new element of the class has a boundary to the space; a new space has a boundary to the element183, 26, 13
new_in_storey, in_storeyevery new element of the class is contained in the storey; the element’s only container is the storey137, 36
connected_new, rel_connects, not_connecteda new element of the class is connected to the wall; the two elements are connected by a connection relationship; they are not89, 24, 10

Table 11. Dependency rules: the test on the dependent in M′\mathcal{M}', and the number of dependents using the rule across the 1,990 tasks of IfcEditBench.

####### Preservation

The diff compares two graphs built from M\mathcal{M} and M′\mathcal{M}'. A node is an IfcProduct instance with a GlobalId: elements, openings, spaces, storeys, building and site. Its content is the class; every scalar direct attribute except GlobalId, OwnerHistory, Tag, Description and LongName; every property of its property sets, flattened as “set.property” (type-level sets inherited, quantity sets not included); a content hash of its representation subtree; and a content hash of its placement subtree. The hashes follow every referenced entity to depth 24 and ignore STEP identifiers and GlobalIds. Re-serialising equivalent geometry in a different structure therefore counts as a modification. The placement hash follows relative-placement chains, so products placed relative to a moved wall also count as modified. The generator includes openings and fills in the allowed set for this reason. An edge is a relationship instance keyed by class and end points: voids (wall to opening), fills (opening to fill), containment (structure to element), aggregation (object to part), path and element connections, space boundaries (space to element; all subtypes under one class name), nesting and port connections, and type and material assignments (element to the assigned type or material name). Property definitions are folded into the node and are not edges. A node is modified when its class, attributes, representation hash or placement hash differ; edges are added or removed by key.

With A\mathcal{A} as in the main text (target, allowed, listed dependents, and new instances of the expected classes), the gate fails on any added, removed or modified node outside A\mathcal{A} and on any added or removed edge whose two ends are both outside A\mathcal{A} (an assignment edge counts as having one end outside). An edge with one end in A\mathcal{A} is exempt from the preservation check. Removing the moved wall’s voids relationship or its boundary to a listed space therefore passes preservation; the dependency rules stays_hosted and boundary_on_wall detect those removals. Conversely, the removal of a space boundary between an unlisted space and an unlisted wall changes no node, since boundaries are not products, and is caught only by the edge condition. The gate applies no geometric tolerance; the tolerances of Table 10 and Table 11 belong to the other two checkers.

####### Automatic and explicit dependent changes

Table 12 separates what follows from the model when the agent edits the target from what the agent must do. The helpers edit one product each; the only propagation they perform is the one IFC itself performs through relative placement, and the relationship clean-up of a deletion. Before any edit, a helper gives the product a private copy of a sub-entity it shares with another product (a placement, a profile, a point, a property set), so that an edit of one wall does not move or re-label another.

ChangeHow it happensTested by
openings and fills placed relative to the moved wallfollow through the placement chain when translate or rotate edits the wall’s own placement; inspect reports whether an opening or fill is placed relative to its walltranslate_with
openings and fills with an absolute placementthe agent translates eachtranslate_with
relationships of a deleted productdelete removes the product with its voids, fills, containment, aggregation, connection, boundary, type, material and property relationships and repairs geometry shared with survivors; it does not delete the opening of a deleted door or the openings of a deleted wallremoved, stays_hosted
space footprint on the moved wall facethe agent moves the vertices on that face (shift_profile_vertices, shift_vertices_on_plane)space_meets_wall
space-boundary surface on the moved wallthe agent moves the boundary curve (shift_boundary_curve); it is placed relative to the space and never follows the wallboundary_on_wall
joined wall endsthe agent lengthens or shortens the touching end along its own axis (move_wall_end, stretch_wall_end)joined_to
containment, connections and boundaries of a new elementadd_wall, add_slab, add_column contain the element in the storey and add_space aggregates the space; connections and boundaries are separate steps (connect_walls, IfcOpenShell API)new_in_storey, connected_new, space_boundary_new

Table 12. Dependent changes that follow from the model or from a helper, and those the agent must make explicitly; the right column names the checker rule that tests the outcome.

Exports without explicit relationships

Both generators derive dependents from the relationships in the file and never infer one from geometry. The cascade generator reads the spaces a wall bounds from its space boundaries and the walls it is joined to from its path connections, and assigns the tier from what it finds: a wall with openings but without boundaries or connections yields a tier-1 move whose dependents are the openings and fills; each bounded space raises the task to tier two and adds the two space rules; each connected wall raises it to tier three and adds the joint rule. The grid generator indexes connections and boundaries the same way and uses geometry only to phrase references, to pick the two walls a new wall abuts, and to test touching ends; a template that needs a bounded space, such as a wall identified by its inward side or a space adjacency question, skips a wall without boundaries. Table 8 shows the consequence: deep-tier tasks exist only in the projects that export the relationships.

Passing and failing edits

Table 13 gives, for each gate, one edit that passes and one that fails, for a tier-three wall move with an opening, a door and a joined wall, and for the door creation of the second example above.

GatePassesFails
Validitythe wall’s placement is edited; the validator logs the same lines as before and the wall, its opening and door rebuilda predefined type set to a value outside its enumeration (new validator line); a profile replaced by a two-point polyline (the wall no longer rebuilds); a storey aggregated into itself (cycle)
Intentthe wall’s centroid moved by (−0.25,0,0)(-0.25, 0, 0) within 0.020.02 m; exactly one new door, hosted through an opening of the named wall, 0.90×2.100.90 \times2.10 m within 0.030.03 m, inside the wall’s lengththe wall moved by +0.25+0.25 m; two doors created; a property written to a set whose name carries a trailing space, so the lookup finds nothing
Dependencyopening and door moved by the same vector and still hosted; the joined wall’s end stretched so that its surface gap to the moved wall is no larger than beforethe joined wall left in place (gap 0.250.25 m); the door deleted instead of moved; at tier two, the space footprint or boundary curve left 0.250.25 m from the new wall face
Preservationonly the wall, its opening and door, the listed spaces and joined walls appear in the diff, and every changed relationship has an end among thema profile shared with a second wall edited in place, so that wall’s geometry hash changes; the storey placement moved, so every element counts as modified; a new space created in a wall-creation task; a containment relationship of an unrelated element removed

Table 13. One passing and one failing edit per gate, on a wall move by −0.25-0.25 m in xx with an opening, a door and a joined wall, and on a door creation.

The helper package

The agent can call the helpers in h from execute_ifc_code. Table 14 gives their arguments and effects.

HelperArgumentsReturns or effect
Inspection read-only functions of the model
inspectentity, [psets], [related_to]placement chain, representation summary, openings and fills, connected walls and connection types, space boundaries and bounded spaces, shared sub-entities, property sets
find[class], [storey], [name], [near, radius], [type]the matching elements
unit_scale, world_matrix, wall_axis_world, profile_world_polygonmodel or productmetres per model unit; world placement; wall axis and footprint profile in world coordinates
wall_length, space_areawall or spaceaxis length; net floor area
check_modelpath of the input modelnew schema errors; entities added, removed and modified against the input
Element-local edits one product each; shared sub-entities are copied before they are changed
translateproduct, vectormoves one placement; children placed relative to it follow
move_wall_end, stretch_wall_endwall, end, vector or deltamoves or lengthens one end of a wall along its axis
shift.vertices.on.plane, shift_profile_verticesproduct, plane, vector, [within]moves every vertex of the body (extrusion, Brep, polyline) or of the footprint profile that lies on a world plane
shift_boundary_curvespace boundary, [plane], vectormoves a space-boundary surface
set_extrusion_depth, set_wall_thickness, rotateproduct, valueheight; thickness about the axis; rotation about a world point
set_property, set_attributeproduct, name, valuewrites a private property set or an attribute
delete, connect_walls, privateproduct(s)removes a product with its relationships cleaned; joins two walls; makes a shared sub-entity private
Creation new products on a storey or in a wall; the storey relationship is chosen by class
add_wall, add_slab, add_columnstorey, geometrya new element with its own placement and body, contained in the storey through IfcRelContainedInSpatialStructure
add_spacestorey, footprint, heighta new IfcSpace aggregated into the storey through IfcRelAggregates, as the spatial structure prescribes
add_opening_fillwall, position, size, kindan opening and its door or window hosted by the wall

Table 14. The helper package h with its arguments and effects, grouped by inspection, element-local editing and creation. Bracketed arguments are optional. Helpers act on the specified products; propagation to other dependents is selected by the agent.

Reproduction of training and evaluation

Software and hardware The sandbox, the checkers, the task generators and the BIM-Edit scorer run on Python 3.12 with IfcOpenShell 0.8.5 [28] and NumPy 2.5; the same interpreter image executes the agent’s code in every episode, so the helper package and the checkers see one IfcOpenShell version throughout. Training uses verl 0.9.0 [53], whose hybrid engine serves rollouts with vLLM 0.24 [31]; the students are evaluated with the same vLLM build in bfloat16. DeepSeek-V4-Flash is served with the vLLM build released for that model and Kimi K3 with SGLang [67], both at a 65,536-token context window (Section F lists the per-run settings). Model weights are the public releases of Qwen3.5-9B, Qwen3.5-27B, Gemma 3 31B, DeepSeek-V4-Flash and Kimi K3. All runs used nodes with eight H100 80 GB GPUs: SFT on one node, 9B GRPO on four nodes, the intent-and-validity GRPO and the 27B GRPO on two nodes, Kimi K3 inference on four nodes.

Prompts The student’s system prompt documents the single tool execute_ifc_code and the 28 helpers of the package h with their arguments and effects (Section D); a budget note is appended to the tool return when two rounds remain. The teacher receives, in addition, an edit recipe of about 12,000 characters: step-by-step procedures for a wall move (axis end points, faces, connected walls, openings, bounded spaces and their boundaries), for re-hosting a fill, for deletions with relationship clean-up and for resolving spatial, topological and path references. The student learns from the resulting traces, with the recipe omitted from its prompt. Both prompts and the recipe are released verbatim with the code.

Determinism Evaluation decodes at temperature 0; Section F lists the evaluation behind every row.

Training hyperparameters Both student sizes use the training settings in Table 15. Settings are fixed independently of the test set. SFT uses the final checkpoint, and GRPO runs for a fixed number of steps with the task pool and online group filter described in the main text. The adapters are merged into the rollout weights after every step so that the sampling policy and the trained policy coincide.

StageSettingValue
SFTadaptersLoRA on all linear layers, rank 64, scaling 128
SFToptimiserlearning rate 10−410^{-4}, cosine decay, 3 epochs, batch size 16
SFTsequence lengthup to 32,768 tokens; final checkpoint used without selection
GRPOpoolsynthetic tasks, prefiltered offline with 4 attempts of the SFT student at temperature 1; unsolved tasks capped at one fifth
GRPOsampling64 prompts per step, G=8G = 8 rollouts each, evaluation protocol (20 rounds, no checker feedback)
GRPOfilteringgroups with eight equal rewards dropped and resampled until 64 mixed groups
GRPOoptimisermini-batches of 8 prompts, learning rate 10−510^{-5}, KL coefficient β=10−2\beta= 10^{-2} towards the SFT policy
GRPOschedule100 steps, adapters merged into the rollout weights after every step

Table 15. Training hyperparameters of the two stages.

Evaluation runs and run-to-run variation

Four evaluations supply the reported IfcEditBench scores: I, the students and the untrained 27B model of the main table; II, the prompted and framework references; III, the ablations and the untrained 9B model; IV, the 27B GRPO checkpoints; the 9B GRPO checkpoints have their own evaluations. Table 16 identifies the weights and evaluation behind each reported score. We use one IfcEditBench score per model throughout the paper and report its external-benchmark scores from the same evaluation. All evaluations use the protocol of the main text (temperature 0, 16,384 output tokens per turn, budget 20, or 25 on BIBIMBAP, no checker feedback) unless the table says otherwise. A task without an outcome counts as a failure in the overall score and is dropped from paired comparisons.

RowWeightsEvaluationSettings beyond the protocolStrict
Qwen3.5-9B (untrained)public Qwen3.5-9BIII–16.08
Qwen3.5-27B (untrained)public Qwen3.5-27BI–59.35
Qwen3.5-9B, SFTSFT, final checkpoint (3,588 steps)I–79.15
Qwen3.5-9B, SFT + GRPOGRPO step 100 from the SFT weightsI–86.78
Qwen3.5-27B, SFTSFT, final checkpointI–88.29
Qwen3.5-27B, SFT + GRPOGRPO step 100 from the 27B SFT weightsIV–91.76
DeepSeek-V4-Flashpublic weights, vLLM, tensor parallel 4II65,536-token context61.11
Gemma 4 31Bpublic weights, SGLangII131,072-token context67.99
Kimi K3public weights, SGLang on four nodesII65,536-token context, reasoning effort high88.04
MCP4IFCDeepSeek-V4-Flash backbone, 39 toolsIItool output capped at 8,000 characters39.40
MCP4IFC, looser capsameIIcap 60,000 characters33.42
IFC-AgentDeepSeek-V4-Flash backboneIIown execution loop13.62
Ablations and curves
reward on intent and validity onlyGRPO step 100 with the intent-and-validity rewardIII–82.01
SFT on 25 / 50 / 75 % of the corpusSFT on nested subsetsIII–75.38, 78.39, 79.35
final student, budget 40as the final studentIII; fill tasks separatebudget 4086.38
final student with verifieras the final studentIII; fill tasks separatechecker feedback after every step87.29
9B GRPO steps 25, 50, 75exported checkpoints of the 9B GRPO runseparate–83.57, 85.43, 85.93
27B GRPO steps 10–75exported checkpoints of the 27B GRPO runIV–90.15–92.01

Table 16. Correspondence of the reported IfcEditBench rows to weights and evaluations (I–IV as defined in the text; every evaluation covers the 1,500 held-out and the 490 fill tasks), with each model’s IfcEditBench strict success (%).

Training runs Table 17 lists the training runs behind the reported students. The 9B GRPO run was restarted once from its step-65 checkpoint with a fresh optimiser state; the 27B run was not interrupted.

RunStartDataComputeServed checkpoint
9B SFTQwen3.5-9B19,141 episodes (3 epochs, 3,588 steps)8 H100final step, adapters merged
27B SFTQwen3.5-27Bsame8 H100final step, merged
9B SFT on 25 / 50 / 75 %Qwen3.5-9Bnested subsets of 4,785 / 9,570 / 14,356 episodes8 H100final step, merged
9B GRPO (final student)9B SFTpool of 5,895 tasks32 H100, 18 h; restarted once from step 65step 100, merged; steps 25, 50, 75 exported
9B GRPO, intent-and-validity reward9B SFTsame pool16 H100, 23 hstep 100, merged
27B GRPO27B SFTsame pool16 H100, 49 hstep 100, merged; steps 10–75 exported

Table 17. Training runs: starting weights, data, compute and the checkpoint that was served.

####### Reproduction of the BIM-Edit protocol and scorer

The official BIM-Edit evaluator is not public. We re-implemented it from the benchmark paper and the evaluator defaults shipped with the released runs, and fitted the choices the paper leaves open to released per-task scores before any model of ours was scored.

Scoring formula For a task with input model M\mathcal{M}, ground truth M⋆\mathcal{M}^{\star} and prediction M′\mathcal{M}^{\prime}, the scorer builds the two edit graphs diff⁡(M,M⋆)\operatorname{diff}(\mathcal{M},\mathcal{M}^{\star}) and diff⁡(M,M′)\operatorname{diff}(\mathcal{M},\mathcal{M}^{\prime}) with the graph of Section C (products as nodes, relationship instances as edges). The live nodes of each side are its added and modified products; each receives an area-weighted surface point cloud of max⁡(64,min⁡(4096,⌊16384/n⌋))\max(64,\min(4096,\lfloor16384/n\rfloor)) points for nn live nodes. Live nodes of the two sides are matched by Hungarian assignment on the Monte-Carlo intersection over union of their oriented bounding boxes, with identical GlobalIds matched at once, pairs of different IFC classes excluded, and a pair kept only above an overlap of 0.05. For creation and update tasks, geometry is exp⁡(−5d/D)\exp(-5d/D), where dd is the symmetric median nearest-neighbour distance between the pooled ground-truth and pooled prediction clouds and DD the diagonal of their joint bounding box; semantics averages over the ground-truth live nodes one half for a class match plus one half times the share of the node’s attributes and property values (tolerance 5% on numbers) that the matched prediction node reproduces, with unmatched nodes scoring zero; topology is 0.3F1node+0.7F1rel0.3F^{\mathrm{node}}_{1}+0.7F^{\mathrm{rel}}_{1}, the F1F_1 scores of the edited-node sets and of the added and removed relationship tuples over six relationship classes (voids, fills, containment, aggregation, path connection, space boundary). For deletion tasks, geometry is one when every ground-truth-removed product is absent from the prediction and zero otherwise, and semantics takes the complementary value, the choice that fitted the released scores best. The final score is the mean of the three axes; an unparsable prediction scores zero on all of them, and on anchored tasks a prediction that drops the pinned target scores zero.

Fit Eight choices were left open by the benchmark description: the alignment of the two sides (edit graphs or GlobalIds), the attribute set for semantics (all or task-relevant), the box type for matching (oriented or axis-aligned), class-matched pairs only or not, zero score without any match or not, the delete-semantics convention, the overlap threshold (0.05 or 0.2) and the relationship-class set (four, six, eight or eleven classes), together with the anchored pin; all 1,024 combinations were enumerated. The fixed constants (chamfer scale 5, node weight 0.3, tolerance 0.05, sample counts) are the released evaluator defaults. The objective was the mean absolute difference between our and the official final score over the 324 tasks, averaged over two released runs with edited files, Claude Sonnet 4.6 and Gemini 3.0 Flash; the two other released runs with edited files, GPT-5.4 Pro and GPT-5.4 Mini, were held out and used only to report the error. The best configuration (edit-graph alignment, all attributes, oriented boxes, class-matched pairs, no zeroing, present-is-one delete semantics, threshold 0.05, six relationship classes, pin on) reaches a mean absolute error of 20.8 points on the calibration runs and is the configuration used for every BIM-Edit score in this paper. Table 18 gives the error per run: on the two held-out runs the per-task mean absolute error on the final score is 20 points, the share of tasks within 5 points is 35–41 %, and the run mean is reproduced within 2.2 points.

RunRoleMAE finalMAE geom.MAE sem.MAE topo.Within 5Mean ours / officialBias
Claude Sonnet 4.6calibration18.426.227.020.40.4642.06 / 45.31−3.26
Gemini 3.0 Flashcalibration23.227.633.938.40.2050.48 / 49.48+1.00
GPT-5.4 Proheld-out20.329.928.523.80.3545.26 / 43.94+1.32
GPT-5.4 Miniheld-out20.127.527.421.50.4137.57 / 39.78−2.21
All four20.527.829.226.00.3643.84 / 44.63−0.79

Table 18. Agreement of the re-implemented scorer with the official per-task scores on the four released runs with edited files (324 tasks each; points on a 0–100 scale). Calibration runs were used to select the configuration; held-out runs were not. Within 5 = share of tasks whose final score agrees within 5 points; bias = ours minus official on the run mean.

Per-operation bias Table 19 splits the agreement by operation over the four released runs. For creations and deletions, the final-score error is 9–14 points, the per-task correlation is 0.7–0.8 for creations and for deletions on three of the four runs, and the run means agree within 1.3 points. Updates have a per-task error of 39 points and a correlation of at most 0.24. Their semantics bias (−22 points) and topology bias (+24 points) partly cancel in the final score (−2.3 points). On deletions the geometry and semantics errors are mirror images (−20 and +20 points) and cancel exactly, since both axes flip together whenever a removed product survives. All our models use this scorer. It reproduces run means within about two points, but update scores per task and update/deletion scores per axis cannot be compared directly with published values.

OperationOfficialOursMAErrGeom.Sem.Topo.
Create25.9727.1414.30.75–0.78+13.3−8.8−1.0
Delete50.8349.598.6−0.12–0.79−19.9+20.1−3.9
Update57.0954.8038.6−0.02–0.24−8.3−22.1+23.6

Table 19. Per-operation agreement with the official scores, pooled over the four released runs (432 tasks per operation; points). Bias = ours minus official on the mean; rr = per-task Pearson correlation of the final score (range over the four runs).

The Gemma 4 31B reproduction Our own run of Gemma 4 31B under the BIM-Edit protocol (Table 20) has similar runtime behaviour to the published run but a lower final score. Execution, budget-exhaustion and crash rates differ by less than two points, and tool-call counts are similar. The benchmark releases per-operation scores for its Gemma run but not its edited files, so the two runs are compared by operation only, and this model is not part of the calibration. The published and our final scores are 0.58 against 0.86 for creations, 53.97 against 46.84 for deletions and 58.08 against 43.96 for updates. The differences follow the measured scorer bias: lower geometry and higher semantics on deletions, lower semantics and higher topology on updates. This comparison combines scorer bias with differences between the two runs. The operation ordering and episode-completion rates agree with the published run.

Table 20. Gemma 4 31B, prompted CodeAct, under the BIM-Edit protocol: our harness and re-implemented scorer against the published row; 95% bootstrap interval on our final score.

Gemma 4 31B, prompted CodeActOursPublished
Final score30.55 [27.27, 33.31]37.54
Create0.90.6
Delete46.854.0
Update44.058.1
Episodes executed (%)84.085.19
Budget exhausted (%)10.28.3
Crashed (%)5.94.6
Tool calls per task9.578.57

Table 20.

####### Results by dependency tier and by target element

Tier 1 accounts for most of the test set (1,509 of 1,990 tasks) and therefore largely determines the overall score (Table 21); tiers 2 and 3 are the 200 wall moves whose bounding spaces or joined walls must follow. The prompted models lose most of their success between tier 0 and tier 2: DeepSeek-V4-Flash falls from 85.8 to 13.0, Qwen3.5-27B from 76.5 to 25.0 and Gemma 4 31B from 80.4 to 38.0, and the two published frameworks reach 2.0 and 0.0 at tier 2. Kimi K3, the teacher, declines more gently (92.5 to 74.0). The trained students maintain higher success on these dependent edits: the 9B student after SFT scores 75.0 and 80.0 at tiers 2 and 3, after GRPO 82.0 and 89.0, and the 27B students 93.0 and 96.0 after SFT and 100.0 and 99.0 after GRPO. Training produces its largest advantage over prompting on the deeper dependency tiers.

Table 21. Strict success (%) on IfcEditBench by dependency tier for prompted models, frameworks and trained students (tier sizes 281, 1,509, 100, 100).

ModelTier 0Tier 1Tier 2Tier 3All
DeepSeek-V4-Flash85.862.013.027.061.11
Gemma 4 31B80.468.238.060.067.99
Kimi K392.589.374.070.088.04
Qwen3.5-9B42.012.51.013.016.08
Qwen3.5-27B76.560.625.027.059.35
MCP4IFC75.436.82.015.039.40
IFC-Agent56.67.30.02.013.62
Qwen3.5-9B, SFT86.578.075.080.079.15
Qwen3.5-9B, SFT + GRPO89.086.582.089.086.78
Qwen3.5-27B, SFT91.186.993.096.088.29
Qwen3.5-27B, SFT + GRPO93.690.4100.099.091.76

Table 21.

Table 22 gives paired differences by target-element group with their 95% cluster-bootstrap intervals. Identifier retrieval and compound tasks contribute to the overall group only, as in the main-text figure. The two scale and training effects are of similar size overall (+9.15+9.15 for the larger student, +7.75+7.75 for GRPO on the 9B student) and both are largest on walls, slabs, columns and spaces; on property and quantity tasks neither interval excludes zero, and this group is where the 9B GRPO student trails the 27B SFT student most clearly (+4.71+4.71 [0.59, 8.88]). Between those two models, the interval for the overall difference of +1.51+1.51 includes zero. On door, window and opening tasks the point estimate is slightly negative and its interval also includes zero. The 27B GRPO student exceeds its SFT start by 3.42 points overall, by 4.73 on walls, slabs and columns, 2.98 on doors, windows and openings and 1.94 on spaces, and by 0.00 on property and quantity tasks.

Table 22. Paired differences in strict success (points) with 95% cluster-bootstrap intervals and the number of paired tasks, per target-element group.

Group27B SFT − 9B SFT9B SFT + GRPO − 9B SFT27B SFT − 9B SFT + GRPO27B SFT + GRPO − 27B SFT
All tasks+9.15 [7.42, 11.05] (1989)+7.75 [5.79, 9.47] (1988)+1.51 [-0.05, 3.48] (1987)+3.42 [1.99, 4.71] (1,989)
Wall, slab, column+10.81 [7.68, 13.92] (740)+7.98 [4.48, 11.04] (739)+2.84 [0.13, 5.93] (739)+4.73 [2.72, 6.67] (740)
Door, window, opening+8.30 [5.38, 11.18] (771)+8.68 [6.00, 11.46] (772)−0.26 [−2.99, 2.41] (771)+2.98 [0.77, 5.33] (771)
Space+10.68 [3.06, 17.86] (103)+10.68 [3.00, 18.00] (103)+0.00 [−5.71, 5.88] (103)+1.94 [−5.45, 9.28] (103)
Property, quantity+2.92 [−1.16, 7.14] (171)−1.18 [−5.68, 3.47] (170)+4.71 [0.59, 8.88] (170)+0.00 [−2.99, 3.01] (171)

Table 22.

The 27B GRPO run at every exported checkpoint

Table 23 lists every exported checkpoint of the 27B GRPO run together with the SFT start (step 0) and the final student (step 100), on IfcEditBench and on both public benchmarks; the main text plots steps 0, 25, 50, 75 and 100 on IfcEditBench only. On IfcEditBench most of the gain arrives in the first ten steps (88.29 to 90.15) and the curve then climbs slowly to 92.01 at step 75 before settling at 91.76; no checkpoint falls below the SFT start. BIM-Edit moves from 54.76 to between 56.94 and 59.51 from step 10 onwards without a clear trend across checkpoints. BIBIMBAP, with 100 tasks, varies between 57.25 and 65.83 from checkpoint to checkpoint; this spread is comparable to the differences between models in the main-text table, limiting the precision of rankings based on their point estimates.

GRPO stepIfcEditBenchBIM-EditBIBIMBAP
088.2954.7659.83
1090.1558.0065.83
2090.2558.6859.17
2590.2058.7863.50
3090.3557.0563.00
4090.8059.5164.50
5091.4658.2364.33
7592.0159.4957.25
10091.7656.9463.83

Table 23. The 27B GRPO run: strict success (%) on IfcEditBench, BIM-Edit final score and BIBIMBAP overall score at every exported checkpoint (step 0 = the SFT start, step 100 = the final 27B GRPO student).

Runtime outcomes

Table 24 reports runtime outcomes for models evaluated with the shared CodeAct protocol. Executed denotes normal termination with a final answer, regardless of whether a commit was made or the checkers passed; budget and crash denote those termination conditions, and other collects the remaining outcomes. Budget exhaustion and under-editing are the dominant failure modes of the untrained open models: Qwen3.5-9B exhausts the budget on 44.6% of the tasks and leaves 70.3% under-edited, DeepSeek-V4-Flash and Qwen3.5-27B on about a fifth each, while crashes stay below five percent for every model. Training removes most of both: the 9B student after SFT exhausts the budget on 7.1% and under-edits 7.9%, after GRPO 5.2% and 4.4%, and the 27B students reach the execution rate of Gemma 4 31B and Kimi K3 (95.0–98.2%) with fewer tool calls per task (6.86–7.58 against 7.37–9.10). The untrained models that fail also call tools more often (12–15 calls per task), consistent with repeated inspection without a commit. Under-edit is recorded separately when a non-retrieval task fails intent with no entities changed. Both under-edit and outcome rates use all tasks as their denominator.

ModelExecutedBudgetCrashOtherUnder-editCalls / task
DeepSeek-V4-Flash75.220.44.30.222.313.67
Gemma 4 31B95.54.20.30.12.77.37
Kimi K395.02.31.41.43.99.10
Qwen3.5-9B53.544.61.40.570.315.22
Qwen3.5-27B76.920.82.30.026.312.06
Qwen3.5-9B, SFT91.67.11.00.37.98.35
Qwen3.5-9B, SFT + GRPO94.55.20.20.24.48.07
Qwen3.5-27B, SFT96.12.90.80.32.97.58
Qwen3.5-27B, SFT + GRPO98.21.50.30.01.06.86

Table 24. Runtime outcomes (%) on IfcEditBench and mean tool calls per task; the two published frameworks use their own execution loops and are omitted.

Expert evaluation

Sampling The 30 IfcEditBench items are edits of the final 9B student from its main-text evaluation on the 1,500 held-out tasks, restricted to editing families (creation, deletion, update, compound and the cascade tiers; no retrieval or quantity task) and to tasks whose edited file exists. With a fixed random seed, 15 strict passes were drawn by a quota over task families (creations, deletions, updates, compound instructions, spatial references and the cascade tiers) and 15 failures by the first failing gate in the order validity, preservation, dependency, intent (one, three, five and six items), with at most one failure per project and template. The 20 benchmark pairs put the final student next to another system on the same task: on BIM-Edit against the released GPT-5.4 Pro outputs (five pairs) and the 9B SFT start (five), on BIBIMBAP against the 9B SFT start (five) and the 27B SFT student (five). Within each BIM-Edit group, pairs were drawn so that the benchmark score favours the final student in three, the other system in one, and is equal in one control pair, with at most one task per element–operation combination; within each BIBIMBAP group, four pairs differ by at least 0.25 with at most one favouring the other system, plus one control pair with equal scores; all our outputs come from the main-text evaluation.

Material and blinding The rater worked through a static web page built from the shuffled items. An IfcEditBench item shows the instruction, a “before” view (isometric and plan section, with the elements the instruction refers to dashed) and an “after” view (added elements green, removed red, modified orange), the list of changed products with class, name and storey, and the list of changed relationships. The rater answers whether the edited model is an acceptable execution of the instruction (yes, no, or cannot judge from this material), ticks reasons for a rejection (requested change missing or wrong, dependents not handled, side effects, invalid model, other), states a confidence (low, medium, high) and may comment. A pair item shows the input model and the two outputs as Output A and Output B in random order, with their change lists; the rater chooses the better output, a tie with both acceptable or a tie with both unacceptable, and rates each output’s acceptability. The page carries no checker verdict, benchmark score or model name; the mapping from items to systems and verdicts was kept apart from it. The rater is one BIM practitioner, recorded under the label R07; no further personal information was collected.

Results on the 30 edits Table 25 lists every item. The rater judged 21 items and marked 9 as not judgeable, every one of them at low confidence; the recurring reason is that a verdict needed a number the views do not show (a slab’s thickness or elevation, an opening’s width) or a reference element that the section box does not identify (the nearest column, the lowest slab, the exterior wall). Of the 21 judged items, 19 agree with the four-gate verdict. In item A05, the checkers accept a door in the host wall named by the specification. The rater interprets “the wall on your right” as the opposite wall of the route and rejects that placement. In item A22 the checkers reject a tier-three wall move because the actual also shifted two vertices of a basement space whose footprint abutted the moved wall; the export carries no space boundaries, so the generator had no relationship from which to list the space as a dependent, and the rater found the edit acceptable. A specification with that relationship would make the space a required dependent instead of a protected entity.

ItemProjectTemplate (tier)CheckersRaterConf.Reason
A01fantasy_hotel_2compound of three edits (1)fail: intent (slab elevation off by 0.1 m)cannot judgelowelevations after the edit not shown
A02city_house_munichset wall height, spatial reference (1)passyeshigh
A03fantasy_hotel_2delete slab and its containment (1)fail: validity, preservation (5,314 side effects)nohighside effects, dependents, invalid: 679 elements changed
A04fantasy_office_building_1set wall height (1)passyesmedium
A05smiley_westcreate door, path reference (1)passnomediumdoor placed in the wall on the other side of the route
A06moliore-host window (1)fail: intent (file unchanged)nohighrequested change missing
A07duplexmove wall with openings (1)fail: intent, dependency (nothing moved)nohighrequested change missing
A08fantasy_hotel_1compound of two edits (1)fail: intent (column at the wrong position)nohighrequested change wrong
A09fantasy_office_building_3compound of three edits (1)passcannot judgelowslab thickness and top not shown; view occluded
A10moliodelete column, spatial reference (1)passcannot judgelowreference wall not identifiable in the view
A11city_house_munichwiden opening (1)fail: intent (width 1.00 m instead of 1.20 m)cannot judgelowwidths not shown
A12duplexcreate slab under a room (1)passcannot judgelowthickness and elevation not shown
A13dental_clinicmove wall bounding spaces (2)passyesmedium
A14smiley_westmove joined wall (3)fail: intent, dependency (unchanged)nohighrequested change missing
A15schependomlaandelete column (1)passyeshigh
A16moliocreate wall (1)passyesmedium
A17digital_hubcreate slab on a space footprint (1)fail: intent (footprint in local instead of world coordinates)nohighslab near the origin, not under the space
A18fantasy_hotel_2create slab on columns (1)fail: intent (footprint)cannot judgelowslab outside the section box; no dimensions
A19moliomove column towards the exterior wall (1)passcannot judgelowmove confirmed, nearest exterior wall not identifiable
A20smiley_westdelete wall, viewpoint reference (1)fail: intent, dependency, preservation (wrong wall)nomediumrequested change wrong, side effects
A21samuel_macalisterdelete the lowest slab, spatial reference (1)fail: intent, preservation (wrong slab)cannot judgelowelevations not shown
A22city_house_munichmove joined wall (3)fail: preservation (one unlisted space modified)yesmediumwall, window, joined walls and adjoining space all consistent
A23fantasy_hotel_2resize slab (1)passyeshigh
A24schependomlaancompound of two edits (1)fail: intent, dependency (no connection relationship)nohighrequested change wrong, dependents
A25smiley_westmove wall bounding spaces (2)fail: intent, dependency (unchanged)nohighrequested change missing
A26fantasy_hotel_1set space height (1)passyesmedium
A274351move wall with openings (1)fail: intent, dependency (unchanged)nohighrequested change missing
A28smiley_westre-host window (1)passyesmedium
A29city_house_munichmove joined wall (3)passcannot judgelowopening change outside the pictured area
A30moliodelete space (1)passyeshigh

Table 25. The 30 IfcEditBench items of the expert evaluation: task template and tier, the four-gate verdict with the failing gates, the rater’s verdict and confidence, and the rater’s reason, or the reason an item was not judgeable.

Results on the 20 pairs Table 26 lists every pair with the benchmark scores of both outputs and the rater’s choice. One pair was left unanswered because the benchmark instruction contains a malformed coordinate; of the 19 answered pairs the final student was preferred in 10, tied as acceptable in 7, and the other output preferred in 2. Where a side was preferred, the choice agrees with the sign of the benchmark score in 10 of 12 pairs; in the two others the rater preferred the output that made the requested change in full (a window raised together with its opening, pair B34) or made a change at all (pair B50, where the released GPT-5.4 Pro output leaves the model unchanged but carries the higher official score). In two BIM-Edit pairs against GPT-5.4 the rater found both outputs unacceptable and still expressed a preference for the student’s.

PairBenchmarkTaskOtherScore student / otherPreferenceAcceptable (student / other) and note
B31BIM-Editcreate door between two roomsGPT-5.4 Pro0.65 / 0.00studentno / no; door overshoots the wall end
B32BIBIMBAProtate wall by a quarter turn9B SFT0.67 / 0.33tieyes / yes; different pivots, both legitimate
B33BIBIMBAPdelete columns unrelated to a slab9B SFT0.50 / 0.00studentyes / unsure
B34BIBIMBAPraise window by 0.5 m27B SFT0.33 / 0.0027B SFTno / yes; student raised the opening only
B35BIM-Editdelete wall, spatial reference9B SFT0.58 / 0.58tieyes / yes; identical results
B36BIBIMBAPmove column by 3 m9B SFT1.00 / 1.00tieyes / yes
B37BIBIMBAPremove the wall touching two walls27B SFT0.50 / 0.00tieyes / yes; two walls satisfy the condition
B38BIBIMBAPdelete opening27B SFT0.50 / 0.00studentyes / no; other made no change
B39BIM-Editdelete room, topological referenceGPT-5.4 Pro0.67 / 0.67tieyes / yes
B40BIBIMBAPenclose a room9B SFT1.00 / 0.00studentyes / no
B41BIBIMBAProtate wall by a quarter turn27B SFT0.67 / 1.00tieyes / yes
B42BIBIMBAProtate parallel wall by 45 degrees27B SFT1.00 / 1.00tieyes / yes
B43BIBIMBAProtate parallel wall by 45 degrees9B SFT1.00 / 0.67studentyes / no; other unchanged
B44BIM-Editcreate wall, topological reference9B SFT0.37 / 0.00studentyes / no
B45BIM-Editcreate slab at given coordinates9B SFT0.84 / 0.30unansweredmalformed coordinate in the instruction
B46BIM-Editdelete wall, spatial reference9B SFT0.67 / 0.36studentyes / no
B47BIM-Editupdate window, topological referenceGPT-5.4 Pro0.55 / 0.00studentno / no; window kept in the wrong storey
B48BIM-Editcreate wall with four connections9B SFT0.00 / 0.479B SFTno / no; other partial, student unchanged
B49BIM-Editcreate window, spatial referenceGPT-5.4 Pro0.58 / 0.00studentunsure / no
B50BIM-Editupdate wall, spatial referenceGPT-5.4 Pro0.42 / 0.81studentno / yes; GPT-5.4 output unchanged

Table 26. Table 26. The 20 benchmark pairs: task, the other system, the benchmark score of the final student and of the other output (BIM-Edit: our scorer for our outputs and the official score for GPT-5.4 Pro; BIBIMBAP: the benchmark’s score), the rater’s preference and the acceptability of each output (final student / other).

Further visual examples

Figure 8 shows the before-and-after section views used in the expert evaluation. The four passing examples are: a door created in a wall that the instruction names by a route through the building (a), a window re-hosted to the parallel wall on the other side of its room (b), a wall move that drags the footprints of the spaces it bounds (c) and a wall move at the deepest tier, where the joined walls, their openings and the spaces all follow (d). The four failures show what each checker rejects: a slab created with the wrong footprint fails intent (e); a wall move that the student did not carry out (eleven inspection calls, no commit) fails intent and dependency (f); a wall move that also alters a space outside the allowed set fails preservation (g); and a slab deletion whose removal cascaded through geometry shared with other products fails validity as well as preservation (h). § 11.0.0.2.§ 1 audits the relationships and attributes behind four of these verdicts.

Figure 8. IfcEditBench edits of the final student: before and after each edit (green = added, red = removed, orange = modified, dashed = elements the instruction refers to) with the instruction and the four-gate verdict. (a)–(d) strict passes at tiers 1 to 3; (e)–(h) checker failures with the failing gates.

Relationship and attribute audit of four cases

The entity and relationship diffs explain the checker verdicts in Figure 8. (b) Window re-hosted: two products modified, the window and its opening; one relationship removed and one added, the opening’s voids relationship moving from the old wall to the new one; the intent test finds the opening voiding exactly the new host, its plan centre within 0.000 m of the requested point and its sill at the requested height. (c) and (d) Wall moves at tiers two and three: the wall’s centroid moved by the requested vector within 0.000 m; the opening and the fill translated with it through their relative placement and the opening still voids the wall; at tier two the three bounded spaces’ faces and their space-boundary curves lie on the moved wall within 0.02 m, as they did before; at tier three the surface gap between each joined wall and the moved wall is no larger after the move than before; no product outside the allowed set changed. (g) Tier-three move rejected: the same tests pass, but the diff lists one modified product outside the allowed set, a basement space whose footprint the student shifted by the move vector because it abutted the wall face; the Revit 2025 export carries no space boundaries, so the generator had no relationship from which to list that space as a dependent. (h) Slab deletion rejected: the instruction asks to delete a footing slab and to remove it from the storey’s containment relationship, that is, to drop one member of the relationship instance rather than the instance itself. The student’s commit removed the slab, but the deletion cascaded through geometry entities that this millimetre Revit export shares between products: 273 products removed, 406 modified, 4,638 relationship instances dropped (openings and containment on other storeys among them; 5,314 of these changes lie outside the allowed set) and 674 new schema errors from placements and faces left without their referents. Although the student’s final message says that only the slab was removed and the relationship kept, the diff records damage to shared geometry and unrelated products. The rater rejected the edit for these side effects.

Figure 9 shows six public-benchmark tasks with the final student’s output next to another system’s output. On BIM-Edit, the student places a door in the wall between two named rooms where the released GPT-5.4 Pro output scores 0.00 (a), deletes a room with the same partial score as GPT-5.4 Pro (b) and, like its 9B SFT start, deletes the wall on the left after walking into a room with a partial score (c). These partial scores reflect BIM-Edit’s separate measures of geometry, semantics and topology. On BIBIMBAP, the final student rotates a wall by a quarter turn (d) and deletes the columns unrelated to a slab (e) where the SFT start scores 0.33 and 0.00, and in (f) the final student raises only the opening of a window by half a metre, scored 0.33, where the 27B SFT student raises window and opening together but scores 0.00.

Figure 9. Public-benchmark tasks: the input model, the final student’s output and another system’s output (the released GPT-5.4 Pro outputs, the 9B SFT start or the 27B SFT student), with the benchmark score of each output where the benchmark defines one. (a) BIM-Edit: create a door in the wall which is connecting the room with the ID 2UsXAbj6n0fwEWqdkVuHvh to the room [...]. input; GPT-5.4 release 0.00; final student 0.65. (b) BIM-Edit: delete the room which is connecting the room with the ID 2UsXAbj6n0fwEWqdkVuHvh to the room with the ID [...]. input; final student 0.67; GPT-5.4 release 0.67. (c) BIM-Edit: delete the wall that is on the left hand side after walking into the room with the ID [...]. input; 9B SFT start 0.58; final student 0.58. (d) BIBIMBAP: Rotate the wall with id 22hyxvAPr65PFt9WZfHSP3 by +90 degree around the z axis. input; 9B SFT start 0.33; final student 0.67. (e) BIBIMBAP: Delete the columns that have no relationship with the slab with the id 2IIWDJ6bzEPxU4iXeUX9Z5. input; 9B SFT start 0.00; final student 0.50. (f) BIBIMBAP: Increase the height by 0.5 meters of the window with id 11kJIqz$n2Jf_DfJV1SDbS. input; final student 0.33; 27B SFT student 0.00.

References

  1. [1]Anthropic. Claude Opus 4.6 system card. https://www.anthropic.com/system-cards, 2026.
  2. [2]Anthropic. Claude Sonnet 4.6 system card. https://www.anthropic.com/system-cards, 2026.
  3. [3]André Borrmann, Markus König, Christian Koch, and Jakob Beetz, editors. Building Information Modeling: Technology Foundations and Industry Practice. Springer International Publishing, Cham, 2018.
  4. [4]DeepSeek-AI. DeepSeek-V3.2: Pushing the frontier of open large language models. arXiv preprint arXiv:2512.02556, 2025.arxiv.org/abs/2512.02556
  5. [5]DeepSeek-AI. DeepSeek-V4: Towards highly efficient million-token context intelligence. arXiv preprint arXiv:2606.19348, 2026.arxiv.org/abs/2606.19348
  6. [6]Changyu Du, Sebastian Esser, Stavros Nousias, and André Borrmann. Text2BIM: Generating building models using a large language model-based multiagent framework. Journal of Computing in Civil Engineering, 40(2):04025142, 2026.arxiv.org/abs/2408.08054
  7. [7]C. Eastman, Jae-Min Lee, Yeon-Suk Jeong, and Jin-Kook Lee. Automatic rule-based checking of building designs. Automation in Construction, 18(8):1011–1033, 2009.DOI
  8. [8]Bradley Efron and R. J. Tibshirani. An Introduction to the Bootstrap. Chapman and Hall/CRC, New York, 1994.
  9. [9]Fengxiao Fan, Jingzhe Ni, Fan Sang, Xiaolong Yin, Yu Liu, Ruofeng Tong, Min Tang, and Peng Du. TraceCAD: Trace-guided repair for agentic CAD generation. arXiv preprint arXiv:2608.03062, 2026.arxiv.org/abs/2608.03062
  10. [10]Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong. ReTool: Reinforcement learning for strategic tool use in LLMs. In The Fourteenth International Conference on Learning Representations (ICLR 2026), 2026.arxiv.org/abs/2504.11536
  11. [11]Yan Gao, Fuji Hu, Chengzhang Chai, Yiwei Weng, and Haijiang Li. Multi-agent framework for schema-guided reasoning and tool-augmented interaction with IFC models. Automation in Construction, 186:106888, 2026.DOI
  12. [12]Gemma Team. Gemma 4 technical report. arXiv preprint arXiv:2607.02770, 2026.
  13. [13]Longling Geng and Edward Y. Chang. ALAS: Transactional and dynamic multi-agent LLM planning. arXiv preprint arXiv:2511.03094, 2025.arxiv.org/abs/2511.03094
  14. [14]Yifei Gong, Xing Wu, Wenda Liu, and Kang Tu. TOOLCAD: Exploring tool-using large language models in text-to-CAD generation with reinforcement learning. In Findings of the Association for Computational Linguistics: ACL 2026, pages 23161–23188, 2026.
  15. [15]Google DeepMind. Gemini 3 Flash model card. https://deepmind.google/models/model-cards/gemini-3-flash/, 2025.
  16. [16]Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning. Nature, 645:633–638, 2025.DOI
  17. [17]Sylvain Hellin, Suhyung Jang, Stefan Fuchs, Stavros Nousias, and André Borrmann. BIM information extraction through LLM-based adaptive exploration. arXiv preprint arXiv:2605.01698, 2026.arxiv.org/abs/2605.01698
  18. [18]Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.arxiv.org/abs/1503.02531
  19. [19]Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuntian Li, Shen Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In The Tenth International Conference on Learning Representations (ICLR 2022), 2022.arxiv.org/abs/2106.09685
  20. [20]International Organization for Standardization. ISO 16739-1:2024 – industry foundation classes (IFC) for data sharing in the construction and facility management industries — part 1: Data schema. International Standard, 2nd edition, 2024.
  21. [21]Yuye Jiang, Changyu Du, Jiabin Wu, Stavros Nousias, and André Borrmann. Towards automated BIM conflict resolution using reinforcement learning. In EG-ICE 2025: International Workshop on Intelligent Computing in Engineering, pages 97–107. University of Strathclyde Publishing, 2025.
  22. [22]Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. SWE-bench: Can language models resolve real-world GitHub issues? In The Twelfth International Conference on Learning Representations (ICLR 2024), 2024.arxiv.org/abs/2310.06770
  23. [23]Ahti Kalervo, Juha Ylioinas, Markus Häkkiö, Antti Karhu, and Juho Kannala. CubiCas5K: A dataset and an improved multi-task model for floorplan image analysis. In Image Analysis: 21st Scandinavian Conference, SCIA 2019, Proceedings (Lecture Notes in Computer Science), pages 28–40. Springer, 2019.arxiv.org/abs/1904.01920
  24. [24]Ryo Kanazawa, Koyo Hidaka, Teppei Miyamoto, Takayuki Kato, Tomoki Ando, Chenguang Wang, Dayuan Jiang, Naofumi Fujita, Shuhei Saitoh, Atomu Kondo, Koki Arakawa, and Daiho Nishioka. Ishigaki-IDS: An open-weight verifier-aware model for information delivery specification drafting in building information modeling. arXiv preprint arXiv:2606.08545, 2026.arxiv.org/abs/2606.08545
  25. [25]Ryo Kanazawa, Koyo Hidaka, Teppei Miyamoto, Takayuki Kato, Tomoki Ando, Chenguang Wang, Dayuan Jiang, Naofumi Fujita, Shuhei Saitoh, Atomu Kondo, Koki Arakawa, and Daiho Nishioka. Ishigaki-IDS-Bench: A benchmark for generating information delivery specification from BIM information requirements. In Proceedings of the 35th ACM International Conference on Information and Knowledge Management (CIKM 2026), pages 8926–8930. ACM, 2026.arxiv.org/abs/2605.22079
  26. [26]Mohammad Sadil Khan, Sankalp Sinha, Talha Uddin Sheikh, Didier Stricker, Sk Aziz Ali, and Muhammad Zeshan Afzal. Text2CAD: Generating sequential CAD designs from beginner-to-expert level text prompts. In Advances in Neural Information Processing Systems 37 (NeurIPS 2024), pages 7552–7579, 2024.arxiv.org/abs/2409.17106
  27. [27]Kimi Team. Kimi K3: Open frontier intelligence. arXiv preprint arXiv:2607.24653, 2026.
  28. [28]Thomas Krijnen and IfcOpenShell contributors. IfcOpenShell: The open source IFC toolkit and geometry engine. Software, version 0.8.5. https://ifcopenshell.org, 2026. LGPL-3.0; accessed 2026-11-03.
  29. [29]Clemens Kujat, Bharathi Kannan Nithyanantham, Stefan Telgmann, Tobias Sesterhenn, Jörn Plönnings, Stefan Lüdtke, and Christian Bartelt. BIBIMAP: A benchmark for instructional BIM-based automated programming. In Proceedings of the 2026 European Conference on Computing in Construction (EC3 2026), Corfu, Greece, 2026.DOI
  30. [30]Dmitry Kuklev. A building as a repository: KIR, a typed intermediate representation for agent-authored building information models. arXiv preprint arXiv:2609.14578, 2026.arxiv.org/abs/2609.14578
  31. [31]Woosuk Kwon, Zuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with PagedAttention. In Proceedings of the 29th Symposium on Operating Systems Principles (SOSP 2023), pages 611–626. ACM, 2023.arxiv.org/abs/2309.06180
  32. [32]Luis Lara, Aristides Milios, Zhi Hao Luo, Aditya Sharma, Ge Ya Luo, Christopher Beckham, Florian Golemo, and Christopher Pal. Generative floor plan design with LLMs via reinforcement learning with verifiable rewards. In Findings of the Association for Computational Linguistics: ACL 2026, pages 26612–26627, 2026.
  33. [33]Jia-Rui Lin, Yunhong Cai, Xiang-Rui Ni, and Peng Pan. Intelligent identification and correction of property-related design defects in BIM via large language models. Automation in Construction, 192:107218, 2026.DOI
  34. [34]Jia-Rui Lin, Yun-Hong Cai, Xiang-Rui Ni, Shaojie Zhou, and Peng Pan. Qwen-BIM: developing large language model for BIM-based design with domain-specific benchmark and dataset. arXiv preprint arXiv:2602.20812, 2026.arxiv.org/abs/2602.20812
  35. [35]Fumin Liu, Haoyu Zhou, Fei Hao, and Lin Yang. Embodied CAD: Solver-grounded LLM agents for parametric B-Rep assembly modeling. arXiv preprint arXiv:2606.31252, 2026.arxiv.org/abs/2606.31252
  36. [36]Han Liu, Jisheng Ma, Ge Gao, and Ming Gu. Decoupling IFC models for reliable modification of shared references. Automation in Construction, 181:106670, 2026.DOI
  37. [37]Rui Liu, Steven Jige Quan, Zhong-Ren Peng, Zijun Yao, Han Wang, Pengyang Wang, Kunpeng Liu, Yanjie Fu, and Dongjie Wang. City editing: Hierarchical agentic execution for dependency-aware urban geospatial modification. arXiv preprint arXiv:2602.19326, 2026.arxiv.org/abs/2602.19326
  38. [38]Yilin Liu, Niladri Shekhar Dutt, Changjian Li, and Niloy J. Mitra. B-repLer: Language-guided editing of CAD models. In SIGGRAPH 2026 Conference Papers, pages 1–12. ACM, 2026.arxiv.org/abs/2508.10201
  39. [39]Yuanjie Lyu, Chengyu Wang, Jun Huang, and Tong Xu. Student-centered distillation narrows the agentic gap between small and large LLMs. In Proceedings of the 43rd International Conference on Machine Learning (ICML 2026), pages 84006–84029, 2026.arxiv.org/abs/2509.14257
  40. [40]Yan Ma, Weiyu Zhang, Tianle Li, Linge Du, Xuyang Shen, and Pengfei Liu. What does vision tool-use reinforcement learning really learn? disentangling tool-induced and intrinsic effects for crop-and-zoom. In Proceedings of the 43rd International Conference on Machine Learning (ICML 2026), pages 85238–85261, 2026.arxiv.org/abs/2602.01334
  41. [41]Dimitrios Mallis, Ahmet Serda Karadeniz, Sebastian Cavada, Danila Rukhovich, Niki Foteinopoulou, Kseniya Cherenkova, Anis Kacem, and Djamila Aouada. CAD-assistant: Tool-augmented VLLMs as generic CAD task solvers. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), pages 7284–7294, 2025.
  42. [42]Bharathi Kannan Nithyanantham, Tobias Sesterhenn, Ashwin Nedungadi, Sergio Peral Garijo, Janis Zenkner, Christian Bartelt, and Stefan Lüdtke. MCP4IFC: IFC-based building design using large language models. arXiv preprint arXiv:2511.05533, 2025.arxiv.org/abs/2511.05533
  43. [43]Bharathi Kannan Nithyanantham, Clemens Kujat, Tobias Sesterhenn, Stefan Telgmann, Ashwin Nedungadi, Jörn Plönnings, Christian Bartelt, and Stefan Lüdtke. BIM-Edit: Benchmarking large language models for IFC-based building information modeling. arXiv preprint arXiv:2606.20146, 2026.arxiv.org/abs/2606.20146
  44. [44]OpenAI. Introducing GPT-4.1 in the API. https://openai.com/index/gpt-4-1/, 2025.
  45. [45]OpenAI. GPT-5 system card update: GPT-5.2. https://openai.com/index/gpt-5-system-card-update-gpt-5-2, 2025.
  46. [46]OpenAI. GPT-5 system card. arXiv preprint arXiv:2601.03267, 2026.
  47. [47]OpenAI. GPT-5.4 Thinking system card. https://openai.com/index/gpt-5-4-thinking-system-card/, 2026.
  48. [48]Qwen Team. Qwen3.5: Towards native multimodal agents. Blog post and model cards, https://qwen.ai/blog?id=qwen3.5, 2026.
  49. [49]Qwen Team. Qwen3.6-Plus. Alibaba Cloud, https://qwen.ai/blog, 2026.
  50. [50]Abdullahi Saka, Ridwan Taiwo, Nurudeen Saka, Babatunde Abiodun Salami, Saheed Ajayi, Kabiru Akande, and Hadi Kazemi. GPT models in construction industry: Opportunities, limitations, and a use case validation. Developments in the Built Environment, 17:100300, 2024.DOI
  51. [51]John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.arxiv.org/abs/1707.06347
  52. [52]Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Juncxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024.arxiv.org/abs/2402.03300
  53. [53]Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Yang, Yanghua Peng, Haibin Lin, and Chuan Wu. HybridFlow: A flexible and efficient RLHF framework. In Proceedings of the Twentieth European Conference on Computer Systems (EuroSys 2025), pages 1279–1297. ACM, 2025.DOI
  54. [54]Joar Skalse, Nikolaus H. R. Howe, Dmitrii Krasheninnikov, and David Krueger. Defining and characterizing reward gaming. In Advances in Neural Information Processing Systems 35 (NeurIPS 2022), pages 9460–9471, 2022.arxiv.org/abs/2209.13085
  55. [55]Yuewan Sun and Zhenghui Sha. LLM4CAD-Editor: An intent-aware large language model framework for multi-level computer-aided design editing. arXiv preprint arXiv:2606.20607, 2026.arxiv.org/abs/2606.20607
  56. [56]Bowen Wang, Liangzhi Li, Yuta Nakashima, and Hajime Nagahara. Learning bottleneck concepts in image classification. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10962–10971. IEEE, 2023.arxiv.org/abs/2304.10131
  57. [57]Bowen Wang, Jiaxin Zhang, Ran Zhang, Yunqin Li, Liangzhi Li, and Yuta Nakashima. Improving facade parsing with vision transformers and line integration. Advanced Engineering Informatics, 60:102463, 2024.arxiv.org/abs/2309.15523
  58. [58]Bowen Wang, Zhouqiang Jiang, Yasuaki Susumu, Shotaro Miwa, Tianwei Chen, and Yuta Nakashima. Taming the untamed: Graph-based knowledge retrieval and reasoning for MLLMs to conquer the unknown. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), pages 4732–4742. IEEE, 2025.arxiv.org/abs/2506.17589
  59. [59]Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. Executable code actions elicit better LLM agents. In Proceedings of the 41st International Conference on Machine Learning (ICML 2024), pages 50208–50232, 2024.arxiv.org/abs/2402.01030
  60. [60]Jiabin Wu, Stefan Fuchs, Pieter Pauwels, and André Borrmann. Automated building component alterations driven by LLM-formalized human strategies to achieve code compliance. Automation in Construction, 190:107148, 2026.DOI
  61. [61]Rundi Wu, Chang Xiao, and Changxi Zheng. DeepCAD: A deep generative network for computer-aided design models. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 6752–6762, 2021.arxiv.org/abs/2105.09492
  62. [62]An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025.arxiv.org/abs/2505.09388
  63. [63]Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations (ICLR 2023), 2023.arxiv.org/abs/2210.03629
  64. [64]Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. DAPO: An open-source LLM reinforcement learning system at scale. In Advances in Neural Information Processing Systems 38 (NeurIPS 2025), pages 125532–125554, 2025.arxiv.org/abs/2503.14476
  65. [65]Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. STaR: Bootstrapping reasoning with reasoning. In Advances in Neural Information Processing Systems 35 (NeurIPS 2022), pages 15476–15488, 2022.arxiv.org/abs/2203.14465
  66. [66]Junwen Zheng and Martin Fischer. Dynamic prompt-based virtual assistant framework for BIM information search. Automation in Construction, 155:105067, 2023.
  67. [67]Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozirakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, and Ying Sheng. SGLang: Efficient execution of structured language model programs. In Advances in Neural Information Processing Systems, 2024.arxiv.org/abs/2312.07104

Paper details

vlnc:W_ppbtktdd
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents