WorksComputer Science
VisFlow: Interpretable Flow Matching for Visibility Reconstruction, Characterization, and Recognition
Abstract
Methods for learning continuous representations, such as latent manifolds, are fundamental and ubiquitous in modern machine learning. Yet many geometric objects combine continuous shape information with discrete combinatorial structure. This combination presents a fundamental challenge for representation learning because small geometric perturbations can induce abrupt changes in the underlying discrete representation. As a result, the smoothness assumptions that make continuous representations effective no longer hold. Polygons provide a canonical example. Their vertex locations vary continuously, but infinitesimal changes in geometry can dramatically alter the associated visibility graph. Existing methods address the inverse mapping from visibility graphs to polygon geometry using vertex-, triangulation-, or signed-distance-function (SDF)-based representations. Vertex- and triangulation-based approaches construct polygons through explicit geometric operations. However, they are less effective at modeling the relationship between geometry and visibility. SDF-based approaches capture this relationship more naturally. Their intermediate states, however, do not correspond to explicit stages of polygon construction. We present VisFlow, a novel, interpretable flow formulation that combines continuous SDF representations with explicit polygon evolution. We evaluate VisFlow on Visibility Reconstruction and Visibility Characterization. Visibility Reconstruction generates one polygon consistent with a visibility graph (G), while Visibility Characterization characterizes all such polygons. VisFlow maintains 20% more visibility relations on Visibility Reconstruction and improves coverage by 30% on Visibility Characterization compared to state-of-the-art methods.
Introduction
Learning the mapping from a graph to its geometric realization arises in problems such as molecular conformation generation, spatial layout reconstruction, and mesh generation. Existing approaches often use continuous geometric attributes, such as coordinates or distances, alongside graph connectivity to represent the underlying geometry. Local perturbations produce small changes in these attributes, keeping nearby geometric realizations close in the representation space. The resulting continuity forms a smooth geometric manifold that learning models can exploit. Purely discrete graph representations lack an analogous continuous structure. Small geometric perturbations can change several edge relations simultaneously, causing similar realizations to map to substantially different graph representations. Therefore, learning the inverse mapping from a discrete graph to its geometric realization is more difficult because nearby geometries are not necessarily nearby in the representation space.
In this paper, we study one such discrete representation: the visibility graph of a polygon. Two polygon vertices are connected by an edge if the line segment joining them lies entirely within the polygon. Small perturbations of the polygon can change several visibility relations simultaneously. Prior work has explored triangulation, vertex-based, and signed distance function (SDF) representations for learning polygon realizations from visibility graphs. Triangulation and vertex-based representations explicitly model geometric construction. These representations have shown promising results in mesh generation, where continuous geometric information is available. However, they struggle to preserve the target visibility structure when conditioned only on a discrete visibility graph. A visibility graph contains no distance information between vertices. In contrast, SDF representations provide a continuous distance representation of the polygon and capture the relationship between polygon geometry and visibility more effectively. Existing SDF-based methods start from Gaussian noise and perform diffusion directly in grid space. Their generative process therefore does not explicitly model polygon formation. Our work asks whether the construction-based formulation of geometric representations can be incorporated into SDF-based generative modeling while retaining the strong relationship between SDFs and visibility?
We present VisFlow, a geometrically meaningful flow-matching model that combines a continuous SDF representation with explicit polygon evolution. Standard flow matching typically learns trajectories from an unstructured source distribution, such as Gaussian noise, to the target data distribution. The intermediate states along such trajectories are not tied to stages of polygon formation. Figure 1 illustrates the contrast between such unstructured intermediate states and the explicit polygon evolution used in VisFlow. VisFlow instead learns an SDF latent representation from polygon trajectories that evolve from an initial triangle to the target polygon. The latent representation is trained so that the intermediate polygon states lie along approximately linear paths. We then use rectified flow to learn the transformation from the initial triangle latent to the target polygon latent, conditioned on the visibility graph. We additionally learn a differentiable Visibility Prediction Network that provides visibility supervision during training and guides optimization of the generated polygon toward the target visibility graph during inference. We further apply the same trajectory-supervision formulation to protein structure generation from contact graphs, demonstrating its use with a different underlying structural evolution process.
Figure 1. Rectified Flow vs polygon evolution. Top: SDF rectified flow starts from Gaussian noise and produces intermediate grid states that are not explicitly tied to stages of polygon formation. Bottom: Our formulation starts from a triangle and evolves through intermediate polygon states toward the final polygon.
Our contributions are as follows:
We present VisFlow, a geometrically meaningful flow-matching model that combines a continuous SDF representation with explicit polygon evolution for visibility-conditioned polygon generation. We also introduce a Visibility Prediction Network that provides dif-
ferentiable visibility supervision during training and enables visibility-guided optimization during inference. VisFlow improves F1-score by 20% on Visibility Reconstruction and improves coverage by 30% on Visibility Characterization compared with state-of-the-art methods.
We extend the trajectory-supervision formulation of VisFlow to protein structure generation from contact graphs.
We present a new dataset for Visibility Reconstruction with 22% greater shape diversity than the state-of-the-art dataset.
Related Work
We categorize related work into three directions: (i) visibility graph reconstruction, recognition, and characterization, (ii) interpretable flow matching, and (iii) polygon propagation methods.
Visibility Graph Reconstruction, Recognition, and Characterization: Prior work has developed reconstruction and recognition algorithms for restricted polygon classes, including staircase, tower, anchor, spiral, terrain, and convex fan polygons [1, 8, 5, 10, 29]. However, the general visibility-graph recognition problem remains open [11]. More recently, learning-based methods have modeled the relationship between polygon geometry and visibility. VisDiff [21] showed that diffusion through an intermediate SDF representation better preserves visibility graphs than GNN- and mesh-based alternatives [9, 7, 18]. However, its coarse grid-based SDF restricts representable polygon geometry, and its intermediate diffusion states do not correspond to a geometry-driven process of polygon formation. We address these limitations using a continuous SDF representation and explicit polygon-evolution supervision.
Flow Matching: Flow matching learns a continuous-time vector field that transports samples from a source distribution to the data distribution [19]. Rectified Flow [20], a flow-matching formulation, encourages approximately straight source-to-target trajectories. Standard formulations typically use unstructured source distributions such as Gaussian noise, so their intermediate states are not tied to the evolution of the generated structure. Recent work has explored giving flow trajectories physical meaning. Pivi et al. [24] use transitions between equilibrium states of an Ising system, while Singh et al. [30] combine flow matching with partially known ODE/PDE dynamics. These approaches incorporate the physical process by changing the dynamics that the flow is trained to follow. In contrast, we use a meaningful geometric evolution to supervise the latent space without replacing the efficient source-to-target transport used for generation by flow matching.
Polygon Propagation Approaches: Polygon propagation defines controlled geometric shape evolution. Grassfire propagation moves the polygon boundary inward along SDF normals [4], while Bruckstein et al. [6] update vertices based on their neighbors. Learning-based methods have also used polygon evolution for shape fitting: Deep Snake [23] iteratively deforms a polygon toward an object boundary, while Deep Extreme Level Set Evolution [32] learns implicit-contour evolution for segmentation. In these methods, the target shape is given and evolution is used to fit its boundary. In contrast, we start from a visibility graph and use polygon evolution to supervise generation of a polygon consistent with the input visibility graph.
Problem Formulation
We focus on simple (no boundary crossing), simply connected polygons (i.e., without holes). Let denote the vertex coordinates of a polygon , and let denote its visibility graph represented as an adjacency matrix. The function computes the visibility graph of . A visibility graph is valid if there exists at least one polygon such that .
Visibility Reconstruction: Given a valid visibility graph , generate a polygon such that .
Visibility Characterization: given a valid visibility graph , generate all simple polygons such that .
Visibility Recognition: Given a graph , determine whether there exists a polygon such that .
Figure 2. Visibility graph problems. Reconstruction finds a polygon consistent with a given visibility graph, characterization finds all polygons corresponding to the same visibility graph (The red circle shows the perturbations but still the same visibility graph), and recognition determines whether an arbitrary graph is a valid visibility graph.
Method
We now detail the architecture of VisFlow. The central idea is to encode the signed distance function (SDF) of a polygon into spatially localized latent features and learn a latent representation in which stages of polygon formation follow approximately linear trajectories. We construct these trajectories by applying grassfire evolution [4] to progressively collapse each polygon to a triangle and then reversing the resulting trajectory. During sampling, we draw an initial triangle and evolve its latent representation to generate a polygon conditioned on the target visibility graph. VisFlow consists of three main components: an SDF Latent Generator that learns spatially localized SDF features and the corresponding polygon-formation trajectories, a Visibility Prediction Network that provides differentiable visibility supervision, and a Flow Network that models polygon generation conditioned on the target visibility graph. We train the components sequentially: first the Visibility Prediction Network, then the SDF Latent Generator with the visibility network frozen, and finally the Flow Network with both networks frozen. Figure 3 provides an overview of the complete architecture. Grassfire evolution [4] and the visibility formulation motivating our Visibility Prediction Network are detailed in Appendix § 7.1.
Figure 3. Overview of VisFlow. Left: The Flow Network evolves an initial triangle latent into a target polygon latent using a time-conditioned U-Net conditioned on the target visibility graph. Right: The SDF Latent Generator encodes a continuous SDF into spatially localized latent features and decodes polygon vertices conditioned on the visibility graph. The Visibility Prediction Network provides differentiable visibility supervision. Inference: Given a target visibility graph , we sample and encode an initial triangle, evolve its latent using the learned flow, decode the final latent into a polygon, and refine its vertices using the Visibility Prediction Network to better match .
SDF Latent Generator
The SDF Latent Generator learns a compact representation of the continuous SDF while preserving the progression of polygon formation. We use a continuous SDF to avoid dependence on a fixed grid resolution. Direct generative modeling of the continuous SDF would require the flow model to operate over dense spatial SDF samples at every integration step, resulting in high computational and memory costs. We therefore encode the SDF into a compact latent representation. We define polygon-formation trajectories using grassfire flow [4]. The inward evolution is stopped when only a triangle remains, the simplest non-degenerate polygonal state. We then reverse the trajectory so that polygon generation proceeds from the triangle to the complete polygon. We normalize trajectory time to , with at the triangle and at the complete polygon. The resulting grassfire trajectories depend on polygon geometry and therefore follow different deformation paths across shapes. A common latent parameterization is needed to represent these trajectories consistently across polygons. We therefore learn a latent representation in which the trajectories follow approximately straight paths between their endpoint latents. Intermediate grassfire states supervise points along each latent path so that linearly interpolated latents decode to the corresponding stages of polygon formation when conditioned on their visibility graphs.
The straight-line latent trajectories are learned using an encoder–decoder architecture. The encoder represents the input SDF using point samples. We adopt the PatchNets formulation (Tretschk et al., 2020) and represent the encoded SDF using spatially localized patch latents over . PointNet [25] maps each SDF sample to a feature representation, which is aggregated within its corresponding spatial patch to produce the patch latent. A positional embedding of each patch center encodes its spatial location, and a Transformer encoder exchanges information across patches to produce a context-aware SDF representation. The decoder uses learnable vertex queries to recover polygon vertex locations. The visibility graph is processed to produce one visibility embedding per vertex, which is concatenated with the corresponding vertex query. A Transformer decoder then cross-attends the resulting vertex representations to the context-aware patch latents and predicts the polygon vertex locations. Additional architecture details are provided in Appendix § 7.2.
We train the SDF Latent Generator using reconstruction, alignment, and regularization losses to preserve polygon geometry and visibility while encouraging linear latent trajectories. The overall objective is
The alignment loss is
The reconstruction loss combines geometric and visibility supervision. The coordinate loss measures the mean squared error between predicted and ground-truth vertices at the start, end, and intermediate states decoded from . Each intermediate state is conditioned on the visibility graph of the corresponding ground-truth state so that the linearly interpolated latents decode to the corresponding stages of polygon formation. Coordinate reconstruction alone does not enforce the desired visibility relations, so we additionally use , defined as the binary cross-entropy between the ground-truth visibility graph and the prediction from the frozen Visibility Prediction Network. We next use the alignment loss to encourage the encoded grassfire states to follow the linear path between the endpoint latents. The loss encourages the encoded grassfire state to match the corresponding point on this path, while prevents the endpoint latents from being updated by the alignment objective. Since the alignment loss does not control the magnitude of the latent representation, penalizes the mean squared magnitude of the start, end, and encoded intermediate latents. We set and in all experiments.
Visibility Prediction Network
The Visibility Prediction Network provides a differentiable approximation of polygon visibility. Exact visibility is non-differentiable with respect to polygon vertices, preventing direct visibility supervision during training and gradient-based refinement during inference. We therefore learn a differentiable predictor based on the visibility predicates defined in the preliminaries (Appendix § 7.1).
The network first maps each polygon vertex to a positional embedding. For each candidate segment between vertices and , the endpoint embeddings are concatenated to form a candidate-segment representation. The intersection branch compares this representation with each non-incident polygon edge and uses an MLP to predict pairwise intersection probabilities. A max operation aggregates these probabilities to obtain , since an intersection with any non-incident edge invalidates visibility. The in-cone branch combines the candidate-segment representation with the neighboring edge representations at each endpoint and uses an MLP to predict the corresponding in-cone probabilities. The two endpoint probabilities are multiplied to obtain , since the in-cone condition must hold at both endpoints. The final visibility probability is computed as . Both branches are trained using binary cross-entropy with their corresponding ground-truth predicate labels.
The pretrained Visibility Prediction Network is frozen during SDF Latent Generator training and used to compute , allowing visibility gradients to propagate to the predicted polygon vertices. The same network is used during inference for gradient-based refinement toward the target visibility graph. Table 6 reports the effect of the differentiable visibility loss, Table 4 compares the predicate-based architecture with a standard Transformer-based visibility predictor, and Figure 9 shows the complete architecture.
| Acc | Prec | Rec | F1 | |
| (a) Visibility Verifier Network | ||||
| (b) Transformer Architecture |
Table 4. Vertex Prediction comparison: Acc: Accuracy, Prec: Precision, Rec: Recall, F1: F1-Score.
Figure 9. Architecture of the Visibility Prediction Network. Polygon vertices are first mapped to positional embeddings by a shared vertex encoder. For a candidate segment , the In-Cone Branch combines the candidate-segment representation with the neighboring polygon edges at each endpoint and predicts an in-cone probability for both endpoints. The two endpoint probabilities are multiplied to obtain . The Intersection Branch compares the candidate segment with each non-incident polygon boundary edge and predicts pairwise intersection probabilities, which are aggregated using a max operation to obtain . The branch outputs are combined as to obtain the final visibility probability for .
| Acc | Prec | Rec | F1 | |
| (a) MSE Loss | 0.87 0.05 | 0.81 0.04 | 0.75 0.03 | 0.77 0.03 |
| (b) MSE + Visibility Loss | 0.89 0.02 | 0.84 0.04 | 0.82 0.04 | 0.83 0.04 |
Table 6. Loss Ablation Study: Acc: Accuracy, Prec: Precision, Rec: Recall, F1: F1-Score.
Flow Network
The Flow Network models the transition from an initial triangle latent to a target polygon latent conditioned on the target visibility graph. We obtain the endpoint latents and by encoding the SDFs of the corresponding trajectory endpoints with the pretrained SDF Latent Generator, which remains frozen during Flow Network training. The approximately linear latent trajectories allow us to model these transitions using rectified flow (Liu et al. [20]). At time , we construct the linearly interpolated latent as
with target velocity . We sample and train the Flow Network with the rectified-flow objective
The Flow Network predicts the velocity from , time , and the target visibility graph . We parameterize the velocity field using a time-conditioned U-Net following Latent Diffusion (Rombach et al. [26]). A separate visibility encoder produces one visibility embedding per vertex, which conditions the U-Net through cross-attention. Additional architecture details are provided in Appendix § 7.3.
Sampling. Given a target visibility graph , we sample initial triangles from the valid coordinate range and construct their continuous SDFs. The SDF Latent Generator encodes each triangle SDF to obtain an initial latent. We then integrate the learned velocity field using Euler integration while conditioning on . The resulting target latent is decoded by the SDF Latent Generator to obtain a polygon. The triangle initialization and approximately linear latent trajectories encourage the intermediate states to follow the corresponding stages of polygon formation. The decoded polygon may not exactly satisfy , so we refine its vertex locations using binary cross-entropy between the predicted visibility graph and through the frozen Visibility Prediction Network. We additionally apply binary cross-entropy to predicted intersections between non-incident polygon edges, using zero as the target to penalize self-intersections. The visibility and intersection losses are summed and propagated through the frozen Visibility Prediction Network to optimize the polygon vertices.
Results
In this section, we evaluate VisFlow on Visibility Reconstruction, Visibility Characterization, Visibility Recognition, and contact-graph protein structure reconstruction. We first introduce the baselines and evaluation metrics, followed by the results for each problem.
Baselines: We compare VisFlow with Mesh-Generation (Lionar et al. [18]), C-VAE (Harvey et al. [12]), GNN (Cui & Wei [9]), and VisDiff (Mahesh et al. [21]) on Visibility Reconstruction. VisDiff is also used as the baseline for Visibility Characterization and Visibility Recognition. All methods are trained on the training split of our dataset and evaluated on its test split. Details of the dataset generation and splits are provided in the appendix 7.10.
Metrics: We evaluate VisFlow using the metrics defined in VisDiff [21]. The Visibility Reconstruction problem is evaluated by comparing the visibility graphs of generated and ground-truth polygons as a binary classification task over vertex pairs. We report accuracy, precision, recall, and F1-score, with F1-score used as the primary metric because of class imbalance. The Visibility Characterization problem requires measuring how broadly the space of polygons satisfying a target visibility graph is covered. Since this space cannot be enumerated exhaustively, we use neighbourhood exploration as a proxy for coverage. We draw a root sample from the source distribution and generate its corresponding polygon conditioned on . We then explore the neighbourhood of using breadth-first search up to depth with branching factor . Each child sample is obtained by applying scheduled perturbations to its parent and is used to generate a polygon conditioned on . A node is retained only if its visibility F1-score with respect to exceeds and its polygon is sufficiently distinct from all previously accepted polygons under the distance threshold . We measure distinctness by aligning the polygons for rotation and translation using Procrustes alignment [28], uniformly sampling their boundaries, and computing the Chamfer distance between the sampled points. Coverage is defined as the fraction of explored nodes that are retained. We assess diversity by varying . Figure 4 illustrates the coverage metric.
Figure 4. Coverage: (a) A learned visibility-graph reconstruction network generates polygon from source state . Child states are formed as , where . (b) A candidate polygon is accepted if and for every . Here, is the F1 score between the visibility graph of and the target visibility graph, and is the Chamfer distance between the two polygons. Coverage is the ratio of expanded nodes to total nodes, .
Visibility Reconstruction
We evaluate VisFlow on Visibility Reconstruction against the baseline methods. We generate polygon samples per generative approach and select the polygon whose predicted visibility graph achieves the highest F1-score with respect to the target graph, while the GNN baseline produces a single polygon. Table 1 shows that VisFlow outperforms all baselines. Table 5 further reports its performance on the original VisDiff dataset, and Figure 10 provides a qualitative comparison. VisDiff exhibits a substantial performance decline on our 50-vertex dataset compared with its original 25-vertex dataset because its coarse SDF resolution limits faithful representation of polygon geometry at higher vertex counts, whereas the continuous SDF representation in VisFlow preserves the geometry. We next examine the source of the performance gains. Table 7 shows that inference-time optimization provides additional improvement, while the flow model already outperforms VisDiff and the other baselines without optimization. We then compare VisFlow with continuous SDF latent diffusion (Cont-VisDiff) in Table 9 to separate the effects of polygon propagation and the continuous SDF latent representation. Polygon propagation provides only a small additional improvement on Visibility Reconstruction, indicating that the primary performance gain comes from the continuous SDF representation.
Table 5. Baseline comparison VisDiff Dataset: Acc: Accuracy, Prec: Precision, Rec: Recall, F1: F1-Score.
Figure 10. Visibility Reconstruction Qualitative Results: (a) GT, (b) VisFlow + Optimization, (c) VisDiff, (d) GNN, (e) Mesh, (f) C-VAE. The labels below show the F-1 Score comparison with the GT Visibility Graph.
| Acc | Prec | Rec | F1 | |
| (a) VisFlow | 0.89 0.02 | 0.84 0.04 | 0.82 0.04 | 0.83 0.04 |
| (b) VisFlow + Optimization | 0.94 0.03 | 0.91 0.05 | 0.91 0.05 | 0.91 0.05 |
| (c) Optimization | 0.88 0.04 | 0.81 0.07 | 0.88 0.04 | 0.83 0.06 |
Table 7. Approach Ablation Study Reconstruction Problem: Acc: Accuracy, Prec: Precision, Rec: Recall, F1: F1-Score.
| Acc | Prec | Rec | F1 | |
| (a) Cont-VisDiff | 0.88 0.02 | 0.82 0.03 | 0.83 0.03 | 0.82 0.03 |
| (b) VisFlow | 0.89 0.02 | 0.84 0.04 | 0.82 0.04 | 0.83 0.04 |
| (c) Cont-VisDiff + Optimization | 0.94 0.02 | 0.91 0.03 | 0.91 0.03 | 0.91 0.03 |
| (d) VisFlow + Optimization | 0.94 0.03 | 0.91 0.05 | 0.91 0.05 | 0.91 0.05 |
Table 9. Approach Ablation Study: Acc: Accuracy, Prec: Precision, Rec: Recall, F1: F1-Score.
Visibility Characterization
We evaluate VisFlow on Visibility Characterization by measuring coverage under local source-space perturbations. We sample a root noise vector with variance for VisDiff and root triangle parameters from the valid parameter ranges for VisFlow. Child samples are generated by perturbing their parent samples with Gaussian noise, with set to 10% of the corresponding sampling scale.
| Acc ↑ | Prec ↑ | Rec ↑ | F1 ↑ | |
| (a) C-VAE ([12]) | 0.84 ± 0.04 | 0.78 ± 0.04 | 0.69 ± 0.03 | 0.72 ± 0.03 |
| (b) GNN ([9]) | 0.86 ± 0.06 | 0.86 ± 0.06 | 0.69 ± 0.05 | 0.73 ± 0.06 |
| (c) Mesh ((Lionар et al., 2025)) | 0.84 ± 0.06 | 0.82 ± 0.11 | 0.69 ± 0.05 | 0.71 ± 0.05 |
| (d) VisDiff ([21]) | 0.86 ± 0.04 | 0.80 ± 0.04 | 0.74 ± 0.03 | 0.76 ± 0.03 |
| (e) VisFlow + Optimization (Ours) | 0.94 ± 0.03 | 0.91 ± 0.05 | 0.91 ± 0.05 | 0.91 ± 0.05 |
caption: Table 1: Baseline comparison: Acc: Accuracy, Prec: Precision, Rec: Recall, F1: F1-Score.
:::
We use , , , and , with chosen from the minimum and maximum diversity observed among realizations of the same visibility graph in our dataset. Table 2 shows that VisFlow achieves substantially higher coverage than VisDiff, particularly at larger and . We next analyze the source of this improvement. Table 8 shows that inference-time optimization provides additional gains, while VisFlow already exceeds VisDiff without optimization. We then compare VisFlow with continuous SDF latent diffusion in Table 10 to isolate the effect of polygon propagation. Perturbations of the latent-diffusion source produce few distinct polygons satisfying the target visibility constraint, whereas perturbations of the triangle source in VisFlow produce more polygons satisfying both acceptance criteria. These results indicate that polygon propagation is the primary source of the coverage improvement on Visibility Characterization, while the Visibility Reconstruction results show that the continuous SDF representation primarily improves visibility reconstruction.
| F1 Threshold | Distance Threshold | VisDiff ([21]) | VisFlow + Optimization |
| 0.70 | 0.01 | 0.89 ± 0.28 | 1.00 ± 0.00 |
| 0.70 | 0.05 | 0.68 ± 0.29 | 1.00 ± 0.01 |
| 0.80 | 0.01 | 0.03 ± 0.16 | 0.95 ± 0.17 |
| 0.80 | 0.05 | 0.00 ± 0.00 | 0.95 ± 0.19 |
Table 2. Coverage performance (↑) under varying F1 threshold and distance threshold : each value represents the fraction of nodes expanded by the method relative to the total number of nodes in the BFS search tree.
| F1 Threshold | Distance Threshold | VisFlow + Optimization | VisFlow |
| 0.70 | 0.01 | 1.00 0.00 | 1.00 0.00 |
| 0.70 | 0.05 | 1.00 0.01 | 0.76 0.21 |
| 0.80 | 0.01 | 0.95 0.17 | 0.51 0.47 |
| 0.80 | 0.05 | 0.95 0.19 | 0.36 0.35 |
Table 8. Approach Ablation Study Characterization Problem: each value represents the fraction of nodes expanded by the method relative to the total number of nodes in the BFS search tree.
| F1 Threshold | Distance Threshold | Cont-VisDiff | VisFlow |
| 0.70 | 0.01 | 0.05 0.10 | 1.00 0.00 |
| 0.70 | 0.05 | 0.01 0.00 | 0.76 0.21 |
| 0.80 | 0.01 | 0.05 0.10 | 0.51 0.47 |
| 0.80 | 0.05 | 0.01 0.00 | 0.36 0.35 |
Table 10. Coverage performance () under varying F1 threshold and distance threshold : each value represents the fraction of nodes expanded by the method relative to the total number of nodes in the BFS search tree.
Visibility Recognition
We evaluate VisFlow on Visibility Recognition. Following VisDiff, we construct the test set by adding holes to half of the polygons in the test split, producing visibility graphs that serve as proxies for invalid inputs. Polygons are generated using the same procedure as in Visibility Reconstruction, and an input graph is classified as valid if the visibility F1-score of the selected polygon exceeds threshold TT. Figure 5 compares VisFlow with VisDiff. VisDiff classifies all inputs as invalid, whereas VisFlow achieves 80% recognition accuracy at a threshold near its mean F1-score on Visibility Reconstruction. These results show that the improved reconstruction performance on Visibility Reconstruction also translates to Visibility Recognition.
Figure 5. Recognition Performance: Accuracy vs F1 Threshold
Contact-Graph Reconstruction
We next evaluate whether the trajectory-supervision formulation of VisFlow extends beyond polygon reconstruction to protein contact-graph reconstruction. A protein contact graph is a binary adjacency matrix indicating whether residue pairs are spatially close. Since the polygon-formation trajectory does not represent protein conformational dynamics, we use targeted molecular dynamics (Schlitter et al., [27]) trajectories and reverse them to obtain random-coil-to-native reconstruction paths. We replace the polygon-specific encoder and decoder with a contact-graph encoder and a 3D coordinate decoder. The model is trained on ProteinNet (AlQuraishi, 2019) using Kabsch-aligned mean-squared error (Kabsch, [14]) together with a contact-graph reconstruction loss. We evaluate on FragFold150 (Kosciolek & Jones, 2014) by comparing the contact graph induced by the reconstructed structure with the target graph using accuracy, precision, recall, and F1-score. Table 3 shows that VisFlow substantially outperforms the baselines. We omit VisDiff and MeshGPT because their architectures are specialized for 2D polygon and mesh representations and are not directly applicable to 3D protein contact-graph reconstruction. Figure 6 shows qualitative samples generated by VisFlow, and Appendix § 7.6 provides architecture details.
Table 3. Contact-Graph Reconstruction Performance Fragfold 150 Dataset: Acc: Accuracy, Prec: Precision, Rec: Recall, F1: F1-Score.
Figure 6. Contact-Graph Reconstruction. Ground-truth protein structures are shown in (a) and (c), with their corresponding VisFlow reconstructions shown in (b) and (d), respectively. The labels report the contact-graph similarity between each ground-truth structure and its reconstruction.
Conclusion
We presented VisFlow, an interpretable flow-matching model that combines continuous SDF representations with explicit polygon evolution for learning geometric realizations from visibility graphs. We show that the continuous SDF representation drives improvements in reconstruction accuracy, while trajectory-supervised polygon evolution leads to substantial gains in coverage and diversity of valid realizations. We introduce a new visibility-graph reconstruction dataset with 22% greater shape diversity than the state-of-the-art dataset, on which VisFlow improves F1-score by 20% on Visibility Reconstruction and coverage by 30% on Visibility Characterization. We further show that the same trajectory-supervision formulation can be applied to protein structure reconstruction from contact graphs. However, our patch-based SDF representation currently assumes that shape information is distributed uniformly across fixed spatial patches. Future work will relax this assumption by introducing the patch radius as an additional parameter, following the original PatchNets (Tretsch et al., [31]) formulation. We also plan to investigate contact-graph realizations as additional structural information for de novo protein generation with AlphaFold (Jumper et al., [13]).
Appendix
We first provide preliminary for our approach in 7.1. Next, we present the architectural details of the SDF Latent Generator in Section 7.2 and the Flow Network in Section 7.3. We then present the architecture and evaluation of the Visibility Prediction Network in Section 7.4. Next, we present additional Visibility Reconstruction results on the original VisDiff dataset in Section 7.5, followed by architectural and training details for contact-graph reconstruction in Section 7.6. We present ablation studies of the training losses and model components in Section 7.7. In Section 7.8, we compare VisFlow with continuous SDF latent diffusion to isolate the contribution of polygon propagation. We then compare the vertex prediction architectures of VisFlow and VisDiff in Section 7.9. Finally, we analyze the statistics and diversity of our generated dataset in Section 7.10.
Preliminaries
Grassfire Evolution: Grassfire defines a deterministic inward evolution of the polygon boundary in which vertices move along local SDF-normal directions. An edge-collapse event occurs as two neighboring vertices meet and the intervening edge shrinks to zero length, reducing the number of polygon vertices. Figure 7 illustrates the evolution and edge-collapse process.
Visibility Formulation: Two polygon vertices are visible if the segment connecting them lies entirely within the polygon. Visibility decomposes into an in-cone condition at both endpoints and an intersection condition with non-incident polygon edges (o’Rourke, 1998). Visibility between vertices and is defined as
where indicates whether lies within the valid visibility cone at both endpoints, and indicates whether it intersects any non-incident polygon edge. Figure 8 illustrates both predicates. The Visibility Prediction Network follows the same decomposition.
Figure 7. Grassfire trajectory for polygon formation. Ground-truth trajectories are constructed by applying grassfire flow to collapse the polygon along local SDF-normal directions until only a triangle remains. The arrows indicate the direction of vertex motion. The yellow vertices mark an edge-collapse event, in which two neighboring vertices meet as the edge between them shrinks to zero length, after which the polygon continues with one fewer vertex. For training, we reverse the trajectory so that polygon formation proceeds from the triangle to the complete polygon. The intermediate grassfire states supervise the corresponding points along the latent trajectory.
SDF Latent Generator Architecture
In this section, we present the architectural details fo SDF Ltent Generator. We represent each polygon using a spatially structured SDF latent representation. The domain is uniformly partitioned into a grid, yielding spatially localized patches. Each SDF sample is first mapped to a feature representation using PointNet (Qi et al., [25]) and assigned to the patch whose center is nearest to the sample location. Within each patch, the sample features are aggregated using a Gaussian-decayed inverse-distance-weighted average based on their distance to the patch center, producing one latent vector per patch. We use a fixed uniform partition rather than adaptive patch placement to maintain a consistent spatial decomposition across shapes and avoid the additional complexity of adaptive patch optimization (Tretschk et al., 2020). The polygon decoder uses learnable vertex queries to recover the polygon vertex locations from the patch latents. The visibility graph is represented using its upper-triangular entries and processed by an MLP that produces one visibility embedding for each vertex. Each visibility embedding is concatenated with the corresponding vertex query, and the resulting query representations are processed by a Transformer decoder that cross-attends to the patch latents to predict the polygon vertex locations.
Figure 8. Visibility predicates. (a) Two polygon vertices can be visible only if their connecting segment lies within the in-cone region at both endpoints. At each endpoint, the two incident polygon edges define an angular region, and the portion of this region lying inside the polygon forms the local in-cone region, shown in blue. Green arrows indicate that the segment from the other vertex enters through the in-cone region, while red arrows indicate that it enters from outside. Thus, fails at both endpoints, satisfies the condition only at , and satisfies it at both endpoints. (b) The intersection predicate determines whether a candidate segment intersects a non-incident polygon edge.
Flow Network Architecture
In this section, we present the architectural details fo Flow Network. The flow network operates on a spatially structured SDF latent representation consisting of 36 patch latents associated with a uniform partition of the domain . The patch latents are arranged according to their spatial locations to form a latent feature map. We parameterize the velocity field using a time-conditioned U-Net following the architecture of Latent Diffusion Rombach et al. [26].
The target visibility graph is represented by its upper-triangular entries. A visibility encoder processes this global graph representation using an MLP and produces one visibility embedding for each vertex. The resulting per-vertex visibility embeddings condition the U-Net through cross-attention, allowing the predicted velocity field to depend on both the current latent state and the target visibility graph.
Visibility Prediction Network
In this section, we present the detailed architecture of the Visibility Prediction Network and evaluate its performance on ground-truth polygons from the test split of our dataset. Figure 9 illustrates the network architecture. We assess its ability to recover visibility relationships through a comparison with a conventional transformer architecture in Table 4. The results show that our network perfectly reconstructs the visibility graphs of the ground-truth polygons in the test set and substantially outperforms the transformer baseline.
Visibility Reconstruction
In this section, we additionally evaluate VisFlow on the original VisDiff dataset [21] and compare its performance with existing baselines. Table 5 shows the comparison between VisFlow and baselines. VisFlow outperforms all baselines showing that the improvements of VisFlow are also retained on the original VisDiff dataset.
Contact-Graph Reconstruction
In this section, we present the architecture details for the contact-graph reconstruction problem. We use the same trajectory-supervised latent-flow formulation as VisFlow, while replacing the polygon-specific components with a protein SDF encoder, a backbone decoder, and a contact-graph encoder. We define the protein contact graph as a binary adjacency matrix over residues, where if the C atoms of residues and are within 8 Å, and otherwise.
Protein backbones are represented using continuous SDF samples computed from the N, C, and C atoms. The SDF encoder uses patch latent tokens of dimension 128, forming a latent representation. The backbone decoder predicts N/C/C coordinates for proteins containing up to 270 residues and is conditioned on both the amino-acid sequence and the target contact graph. We use a maximum length of 270 residues because it covers the proteins in the training and evaluation datasets while maintaining a fixed-size representation. The contact graph is represented as a padded matrix and encoded row-wise using an MLP followed by a six-layer Transformer. The sequence embeddings and graph features are jointly processed by a six-layer Transformer decoder with hidden dimension 128 and eight attention heads.
The SDF latent representation is trained using the same loss formulation as in polygon generation, with a reconstruction loss , alignment loss , and latent regularization loss . For protein reconstruction, consists of a Kabsch-aligned backbone coordinate reconstruction loss [14] and a contact-graph consistency loss . Given the predicted C coordinates , we compute the pairwise distances as
and define the corresponding contact logits as
We define as a weighted binary cross-entropy loss between the predicted contacts and the target contact graph for residue pairs satisfying , which excludes short-range sequence contacts and focuses the loss on long-range structural relationships. We use weights 0.05 for contacts and 0.95 for non-contacts. The reconstruction loss is
and the overall latent objective is
We set , , and .
The SDF encoder and backbone decoder are frozen once the latent representation has been trained. We then train the contact-graph-conditioned Rectified Flow using the same formulation as in polygon generation, with the target contact graph providing the conditioning signal to the latent U-Net. The Rectified Flow model is trained using AdamW with a learning rate of and weight decay of , and inference is performed using 60 Euler steps.
Ablation Study
In this section, we present an ablation study of the training loss on the Visibility Reconstruction problem. We also examine the contribution of each module in VisFlow to performance on Visibility Reconstruction and Visibility Characterization. Table 6 reports the effect of adding the visibility loss on Visibility Reconstruction. The results show that incorporating the visibility loss substantially improves reconstruction performance. Table 7 and Table 8 report the effects of adding each module on Visibility Reconstruction and Visibility Characterization, respectively. The results show that incorporating optimization into VisFlow improves performance on both problems.
How does VisFlow compare with latent diffusion?
In this section, we examine whether VisFlow offers advantages over a conventional latent diffusion model operating on continuous SDF latents. We compare VisFlow with Cont-VisDiff, which performs diffusion in the SDF latent space without polygon propagation. We evaluate both methods on Visibility Reconstruction and Visibility Characterization. Table 9 presents the results on Visibility Reconstruction. The results show that polygon propagation provides no meaningful improvement in reconstruction performance. Table 10 presents the corresponding results on Visibility Characterization. Here, VisFlow achieves substantial gains over Cont-VisDiff. The findings highlight the contribution of polygon propagation to characterization by enabling more effective exploration of distinct polygons consistent with a target visibility graph.
Vertex Prediction Architecture
In this section, we compare the vertex prediction architectures of VisFlow and VisDiff. We provide each architecture with the ground-truth SDF and the corresponding visibility graph to assess its ability to recover polygon vertices. Table 11 presents the comparison. The results show that VisFlow recovers polygons more reliably than VisDiff even when both methods receive ground-truth SDFs. The comparison highlights the limitations of VisDiff’s coarse SDF resolution for preserving the geometric detail required for accurate vertex recovery, while the continuous SDF representation in VisFlow supports more accurate polygon reconstruction.
| Acc ↑ | Prec ↑ | Rec ↑ | F1 ↑ | |
| (a) VisDiff [21] | 0.84 ± 0.05 | 0.78 ± 0.04 | 0.72 ± 0.03 | 0.74 ± 0.03 |
| (b) Cont-VisDiff | 0.88 ± 0.02 | 0.81 ± 0.04 | 0.82 ± 0.03 | 0.82 ± 0.03 |
| (c) VisFlow | 0.89 ± 0.02 | 0.82 ± 0.04 | 0.83 ± 0.04 | 0.83 ± 0.04 |
Table 11. Vertex Prediction comparison: Acc: Accuracy, Prec: Precision, Rec: Recall, F1: F1-Score.
Dataset Generation
Earlier datasets contain limited variation in polygon concavity. VisDiff broadens the range of concave geometries, but its coarse grid-based SDF limits the geometries that can be represented accurately. We therefore construct a dataset with greater geometric and visibility-graph diversity. We generate a large pool of random polygons using the 2-opt move algorithm [3], normalize them to , order their vertices counterclockwise, and canonicalize the ordering by choosing the bottom-left vertex as the first vertex. We compute their visibility graphs and select 10,000 polygons with unique visibility graphs, sampling across link-diameter values to cover a broad range of concavities. For each polygon, we generate 20 additional instances through vertex perturbations that preserve its visibility graph, providing geometrically distinct realizations of the same visibility structure for the one-to-many Visibility Characterization problem. The test set uses visibility graphs excluded from training and sampled from the remaining polygon pool, ensuring evaluation on unseen visibility structures. We analyze the generated dataset using link diameter and visibility-graph edge density to characterize its geometric and structural variation. Figure 11 shows the training distributions of these statistics after resampling across link-diameter values. We further compare the range of polygon concavities in our dataset with existing datasets using link diameter. As shown in Table 12, our dataset has a larger average link diameter and covers a wider range of link-diameter values than the compared datasets. In particular, the maximum link diameter increases from 9.0 in VisDiff to 11.0 in our dataset, while the higher average link diameter indicates greater representation of highly concave polygons. These statistics reflect the broader range of polygon structures included in our dataset.
Table 12. Real-world dataset comparison showing the diversity (in link diameter) between our dataset and standard benchmarks.
Figure 11. Dataset Statistics: Distribution of (a) polygon link diameter and (b) visibility-graph edge density across polygons in the VisFlow after resampling based on link diameter.
References
- [1]James Abello, Ömer Egecioglu, and Krishna Kumar. Visibility graphs of staircase polygons and the weak bruhat order, from visibility graphs to maximal chains. Discrete & Computational Geometry, 14(3):331–358, 1995.DOI
- [3]Thomas Auer and Martin Held. Rpg-heuristics for the generation of random polygons. In Proc. 8th Canada Conf. Comput. Geom. Ottawa, Canada, pp. 38–44. Citeseer, 1996.
- [4]Harry Blum. A transformation for extracting new descriptors of shape. In Weiant Wathen-Dunn (ed.), Models for the Perception of Speech and Visual Form, pp. 362–380. MIT Press, Cambridge, MA, 1967.
- [5]Hossein Boomari and Alireza Zarei. Visibility graphs of anchor polygons. In Topics in Theoretical Computer Science: The First IFIP WG 1.8 International Conference, TTCS 2015, Tehran, Iran, August 26-28, 2015, Revised Selected Papers I, pp. 72–89. Springer, 2016.
- [6]Alfred M Bruckstein, Guillermo Sapiro, and Doron Shaked. Evolutions of planar polygons. International Journal of Pattern Recognition and Artificial Intelligence, 9(06):991–1014, 1995.DOI
- [7]Yiwen Chen, Tong He, Di Huang, Weicai Ye, Sijin Chen, Jiaxiang Tang, Xin Chen, Zhongang Cai, Lei Yang, Gang Yu, Guosheng Lin, and Chi Zhang. Meshanything: Artist-created mesh generation with autoregressive transformers, 2024. URL https://arxiv.org/abs/2406.10163.
- [8]Paul Colley, Anna Lubiw, and Jeremy Spinrad. Visibility graphs of towers. Computational Geometry, 7(3):161–172, 1997.DOI
- [9]Guanyu Cui and Zhewei Wei. Mgnn: Graph neural networks inspired by distance geometry problem, 2023. URL https://arxiv.org/abs/2201.12994.
- [10]Hazel Everett and Derek G. Corneil. Recognizing visibility graphs of spiral polygons. Journal of Algorithms, 11(1):1–26, 1990.DOI
- [11]Subir K Ghosh and Partha P Goswami. Unsolved problems in visibility graphs of points, segments, and polygons. ACM Computing Surveys (CSUR), 46(2):1–29, 2013.arxiv.org/abs/1012.5187
- [12]William Harvey, Saeid Naderiparizi, and Frank Wood. Conditional image generation by conditioning variational auto-encoders, 2022. URL https://arxiv.org/abs/2102.12037.
- [13]John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. nature, 596(7873):583–589, 2021.DOI
- [14]Wolfgang Kabsch. A solution for the best rotation to relate two sets of vectors. Foundations of Crystallography, 32(5):922–923, 1976.
- [16]Yann LeCun. The MNIST database of handwritten digits. http://yann.lecun.com/exdb/mnist/, 1998.
- [17]Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740–755. Springer, 2014.arxiv.org/abs/1405.0312
- [18]Stefan Lionar, Jiabin Liang, and Gim Hee Lee. Treemeshgpt: Artistic mesh generation with autoregressive tree sequencing. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 26608–26617. IEEE, 2025.arxiv.org/abs/2503.11629
- [19]Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022.arxiv.org/abs/2210.02747
- [20]Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003, 2022.arxiv.org/abs/2209.03003
- [21]Rahul Moorthy Mahesh, Jun-Jee Chao, and Volkan Isler. Visdiff: Sdf-guided polygon generation for visibility reconstruction, characterization and recognition. Advances in Neural Information Processing Systems, 38:66549–66580, 2026.
- [23]Sida Peng, Wen Jiang, Huajin Pi, Xiuli Li, Hujun Bao, and Xiaowei Zhou. Deep snake for real-time instance segmentation. In 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp. 8530–8539. IEEE, 2020.arxiv.org/abs/2001.01629
- [24]Francesco Pivi, Simone Gazza, Davide Evangelista, Roberto Amadini, and Maurizio Gabbrielli. On the flow matching interpretability. arXiv preprint arXiv:2510.21210, 2025.arxiv.org/abs/2510.21210
- [25]Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652–660, 2017.arxiv.org/abs/1612.00593
- [26]Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In 2022 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp. 10674–10685. ieee, 2022.arxiv.org/abs/2112.10752
- [27]Jürgen Schlitter, Michael Engels, and Peter Krüger. Targeted molecular dynamics: a new approach for searching pathways of conformational transitions. Journal of molecular graphics, 12(2):84–89, 1994.pubmed.ncbi.nlm.nih.gov/7918256
- [28]Peter H Schönemann. A generalized solution of the orthogonal procrustes problem. Psychometrika, 31(1):1–10, 1966.DOI
- [29]André C Silva. On Visibility Graphs of Convex Fans and Terrains. arXiv preprint arXiv:2001.06436, 2020.arxiv.org/abs/2001.06436
- [30]Gurjeet Sangra Singh, Frantzeska Lavda, Giangiacomo Mercatali, and Alexandros Kalousis. Variational grey-box dynamics matching. In Emtiyaz Khan, Yingzhen Li, Arno Solin, and Aaditya Ramasdas (eds.), Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, volume 300 of Proceedings of Machine Learning Research, pp. 2782–2790. PMLR, 02–05 May 2026. URL https://proceedings.mlr.press/v300/singh26a.html.
- [31]Edgar Tretsch, Ayush Tewari, Vladislav Golyanik, Michael Zollhöfer, Carsten Stoll, and Christian Theobalt. Patchnets: Patch-based generalizable deep implicit 3d shape representations. In European Conference on Computer Vision, pp. 293–309. Springer, 2020.
- [32]Zian Wang, David Acuna, Huan Ling, Amlan Kar, and Sanja Fidler. Object instance annotation with deep extreme level set evolution. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7492–7500. IEEE, 2019.DOI