WorksAgricultural and Biological Sciences
From the possible to the optimal: differentiable optimization of gene regulatory networks
Abstract
Understanding the space of ``the actual and the possible'' regulatory architectures capable of realizing biological functions is one of the major challenges in biology. Optimization-based approaches can rationalize regulatory mechanisms from functional objectives and constraints, but their computational cost has limited them to finding individual solutions. Here, we introduce jaxgrn, a framework that combines differentiable programming with stochastic simulations of gene regulatory networks (GRNs) to optimize thousands of networks end-to-end on GPUs. Using morphogen-driven embryonic patterning as a case study, we optimize networks for domain formation and positional information, while varying constraints on input noise, boundary precision and scaling with system size. While recovering known archetypes for scaling, we find that varying the constraints organizes optimal solutions into qualitatively distinct regimes. When optimizing domain formation, we observe phenotypic convergence across a continuum of regulatory architectures, ranging from input-driven to feedback-driven networks. When optimizing positional information, solutions are initially phenotypically diverse despite moderate variation in regulatory architecture; imposing scaling drives both phenotypic and regulatory convergence of the optimized networks onto a few recurring regulatory architectures. Our systematic approach opens the door for variational systems biology in which changing functional constraints reshape the landscape of possible phenotypes and regulatory architectures.
Introduction
The origins of biological structures (from proteins to gene networks, organs and body-plan) remain one of the most fundamental questions of biology [1, 2]. While we know that Darwinian evolution is the engine of biological changes, there is much debate about how directional it is, and the question remains whether we can predict anything about biology. While some authors (such as Gould) have insisted on the contingent aspects of evolution [3], suggesting it is essentially unpredictable, there are also many examples at all scales of convergent evolution, suggesting that there are indeed regularities that can be rationalized and predicted [4]. Furthermore, the variations accessible to evolution are themselves highly structured: developmental and genotype-to-phenotype mappings can strongly bias which phenotypes are more readily produced than others [5, 6]. Evolutionary dynamics can themselves restrict phenotypic variation to lower-dimensional manifolds [7]. Convergence could therefore reflect not only selection for similar functions, but also constraints on the space of accessible solutions [8].
A classical (but debated) approach is to frame evolution as an optimization problem. Focusing on cellular processing, if the environmental and internal constraints on the one hand, and the cellular function on the other hand can be formalized and identified, it should indeed be possible to infer biological mechanisms from first principles [9, 10]. Such a perspective has already led to major insights into biology, for instance on the structural correspondence between biological networks and the ones independently designed by engineers [11]. Can we then characterize not only individual solutions to a given biological function, but the space of possible solutions and how it is shaped by different constraints? In other words, rather than a single optimum, we seek to characterize the ensemble of solutions compatible with a given function and set of constraints [12, 13].
There are a few classical examples of evolutionary prediction through gradient descent, e.g. the derivation of a plausible evolutionary pathway from a photolayer of cells to a proper eye from purely physical considerations [14]. However, given the wide range of the “actual and the possible” [15], computational methods are necessary to explore the optimization of complex phenotypes. Examples include directed network enumeration [16, 17], evolutionary simulations [18, 19, 20], neural network optimization to match gene network behavior [21, 22] or Monte Carlo/annealing simulations exploring how specific gene network structures naturally emerge to solve specific problems [23]. These approaches have led to predictive and actionable insights into biology, e.g. in silico evolution of discrimination models for immunology suggested an Adaptive Kinetic Proofreading framework [24], validated experimentally and eventually leading to the development of new cancer immunotherapies [25].
However, existing methods for gene network optimization remain limited in their ability to characterize the diversity of solutions to a given biological function. Previous approaches have identified individual solutions or explored relatively small sets of networks, leaving higher-level questions about the structure of solution spaces and their dependence on functional constraints largely unanswered. Recent advances in machine learning offer opportunities to overcome these limitations. GPU-based simulations combined with differentiable programming have enabled efficient optimization of artificial neural networks with billions of parameters [26], and these approaches are increasingly being extended to biological models [27], including detailed biophysical neuron models [28] and gene networks driving the morphogenesis of growing cell clusters [29].
Here, we introduce jaxgrn, a high-throughput framework that combines differentiable programming with stochastic simulations to optimize large ensembles of gene regulatory networks for specific biological functions. Using morphogen-driven embryonic patterning as a case study, we optimize networks for domain formation and positional information under varying constraints on input noise, boundary precision and scaling with system size. We show that constraints qualitatively reshape the distribution of optimized solutions, producing regimes ranging from phenotypic diversity to convergence in both phenotypic and regulatory properties. More generally, our results establish differentiable optimization as a practical approach for systematically investigating how functional constraints structure the space of possible regulatory architectures and phenotypes.
Results
A JAX-based optimization pipeline for spatial stochastic patterning models of gene networks
Our goal is to optimize families of gene networks for various properties through end-to-end automatic differentiation. Our implementation is organized as a modular pipeline separating (i) model definition, (ii) stochastic simulation, (iii) loss evaluation and (iv) gradient-based optimization. For a given gene-network parameter set , we generate an ensemble of stochastic gene expression patterns. The simulated profiles are then mapped to scalar objectives and constraint penalties and the resulting scalar loss is minimized using gradient descent. The framework is modular at the level of parameterization, allowing different constrained or transformed representations of model parameters (e.g. bounded variables, masked couplings or sign-constrained interactions) without changing the simulator or loss definitions.
Figure 1A introduces both our biochemical model and the specific system we are considering. We generate gene expression patterns for an ensemble of simulated embryos on which to compute the optimization objectives. We simulate morphogen-driven reaction-diffusion dynamics in a spatial stochastic framework along a one-dimensional axis (biologically corresponding to the anterior-posterior (AP) axis). We consider morphogen inputs and network genes. We model gene expression via a network of genes, driven by external morphogen inputs, that evolves through regulation, decay, and diffusion between neighboring nuclei. Following [23], gene regulation is captured by a Monod-Wyman-Changeux (MWC) framework, in which each gene’s activity is a sigmoidal function of a regulation potential combining weighted, saturable contributions from morphogens and other network genes, offering a more interpretable and flexible alternative to standard Hill functions. The optimized parameters are the strengths and dissociation constants of all morphogen-gene and gene-gene interactions, together with gene-specific activation thresholds , i.e. 52 parameters for four genes and two morphogen inputs, and 60 for three inputs. We include both extrinsic variability (embryo-to-embryo fluctuations of the morphogen inputs, hereafter “input noise”) and intrinsic variability due to the stochasticity of biochemical reactions, modeled by a chemical Langevin approximation of the underlying chemical master equation [30]. For any given network, we integrate the corresponding stochastic differential equations over an ensemble of embryos, using the explicit Euler-Maruyama method [31], with a ReLU projection applied at each step to enforce non-negative expression levels (a small numerical correction for rare negative excursions due to noise) while using a straight-through estimator [32] to preserve differentiability. More mathematical and implementation details are given in Methods (Sec. 4) and in the Supplement.
A key design choice is to treat the entire workflow as a single differentiable computation graph, enabling end-to-end optimization by automatic differentiation of regulatory parameters and, where desired, of dynamical parameters such as diffusion coefficients and decay rates (both held fixed in the case studies below, see Methods, Sec. 4.1). We implement this framework in JAX [33], using just-in-time compilation and automatic vectorization to simulate large ensembles of stochastic embryos in parallel on GPU. This makes it practical to optimize stochastic dynamical systems directly from simulation-derived statistics and, in our setting, yields substantially faster convergence than gradient-free alternatives for the same model class and objective [23].
Parameter updates are performed using the Adam optimizer [34], which minimizes a problem-specific loss function (Methods, Sec. 4.2 and Supplement) using gradients obtained by automatic differentiation through the simulation pipeline. At each optimization step, noise sources are resampled to prevent overfitting to a fixed stochastic realization and to encourage robust solutions. This implements an effective stochastic gradient descent in parameter space.
The pipeline is driven by a command-line interface that fully specifies the model class, noise settings and objective/constraint configuration, enabling reproducible sweeps across thousands of optimization runs. Each run produces a self-contained output folder containing the full configuration, model checkpoints, scalar logs and standardized diagnostic plots, facilitating systematic hyperparameter exploration and benchmarking across model classes and objective choices.
The full simulation-to-optimization pipeline and the main JAX transformations used are summarized in Figure 1B.
Figure 1. jaxgrn: differentiable simulation and optimization of morphogen-driven GRN patterning. A: Externally specified morphogen profiles drive a network of genes whose regulation is computed locally using an MWC model for gene regulation (Eq. (2)) and converted into a production rate. Gene products then evolve in 1D space (along the AP axis) and time according to discretized reaction-diffusion dynamics with decay and diffusive coupling between neighboring nuclei (Eq. (1)). Stochasticity is incorporated at two levels: embryo-to-embryo extrinsic variability is applied directly to morphogen inputs, while intrinsic fluctuations are introduced during time integration through a chemical Langevin approximation including production, degradation and diffusion hopping noise. Simulating an ensemble of embryos produces spatiotemporal expression patterns and their statistics (means and covariances), which are used to evaluate optimization objectives and biological constraints. B: At each training step, a parameter set (regulatory parameters and, optionally, physical parameters such as and , which are held fixed in this study) is used to generate an ensemble of stochastic gene expression trajectories by forward simulation. The resulting spatiotemporal profiles are mapped to a scalar loss consisting of a primary objective term together with biological constraints and regularization. Gradients of the total loss with respect to are computed by automatic differentiation through the simulator and loss evaluation, enabling gradient-based parameter updates. Noise sources are resampled across iterations to favor robust solutions and independent initializations can be optimized in parallel to recover diverse ensembles of optimal solutions.
Below, we study the optimization of three phenotypes as case studies: two primary ones, and a secondary one (i.e. contingent on the primary ones). The two primary objectives optimized respectively are domain definition (similar to Hox gene patterning) [19] and positional information [23]. More details are given in each section; both of those phenotypes were previously explored using Monte Carlo-based or evolutionary optimizations, but our framework allows for a much more extensive exploration of solutions. As a secondary objective, we are interested in the problem of scale-invariant patterning. Despite fluctuations in length and size of embryos, it has been observed both in vertebrates and invertebrates that gene expression patterns in development typically scale with size [35, 36, 37], a property that also generalizes to stem cell systems and gastruloids [38, 39]. This has been most recently quantified in Drosophila embryos, for the gap gene system: while the embryonic lengths fluctuate by , the corresponding gap gene pattern scales almost perfectly [40], despite the fact that the morphogenetic inputs do not [36]. This suggests that scaling emerges through network interaction, and offers a good case study of the emergence of regulatory strategies implementing complex phenotypes.
Scaling in a minimal domain-formation problem
The anterior-posterior axis of metazoans is organized into discrete regions, with sharp boundaries, specifying local identity (head, thorax, types of vertebrae...). Remarkably, those regions are defined by genes belonging to the Hox family, expressed in discrete, well-defined domains along the AP axis. The problem of gene network evolution optimizing such discrete organization has been previously studied using in silico evolution of gene networks, using a mutual-information metric that optimizes regional identities (high number of well-defined regions with specific genes expressed, see Methods, Sec. 4.3) [19]. As a first case study, we revisit this problem by optimizing domain formation across embryos with fluctuating lengths .
We fixed the number of genes to 4, so that optimizing mutual information between position and gene expression following [19] should give rise to 4 regions of equal size, where only one of the four genes is expressed. The challenge is to simultaneously optimize this mutual information with scale-invariance, i.e. consistency of domain positions in relative coordinates across embryo sizes. As our objective we use the mutual information in relative coordinates between position and cell-fate identity to reward domain formation of equal size [19] (Methods, Sec. 4.3). Because this objective is evaluated in relative coordinates and pooled over embryos of different lengths, it implicitly promotes scaling: boundaries that shift in relative coordinates with blur fate assignments across the ensemble and lower the mutual information.
We optimized GRN models under the control of two opposing non-scaling morphogen inputs: an anterior gradient with absolute decay length and a posterior gradient with absolute decay length , as defined in Eq. (4), so that neither gradient scales with length. In this minimal domain-formation problem, morphogen decay lengths are fixed (not optimized), and we set the input noise to zero (), retaining intrinsic expression noise and embryo-length variability. Optimizing the mutual-information objective alone, however, rarely yields fully scaling four-gene solutions (Supplementary Figures). We therefore add an explicit scaling constraint based on the mean absolute deviation (MAD), which measures the average deviation of individual-embryo expression profiles from the ensemble mean profile in relative coordinates, and is thus small when profiles of embryos of different lengths superimpose (Methods, Sec. 4.5).
Figure 2 shows an illustrative example of a fully scaling GRN, optimized through our framework, displaying high precision and scale-invariant boundaries. Panel A shows the profiles of both input morphogens and downstream 4 genes after optimization, illustrating the four well-separated domains along the relative position. Figure 2B displays the corresponding interaction matrix strength between genes (see details on its computation in Methods, Sec. 4.6; red corresponds to activation and blue to repression; darker colors correspond to stronger contributions at maximal expression). Of note, only three of the four genes are significantly regulated by both morphogens, and we observe an interaction pattern in which all genes self-activate and mutually repress.

Figure 2. Example of a fully scaling four-gene GRN. Network optimized for domain formation (mutual-information objective) with the scaling constraint. A: Mean (bold) and single-embryo (light) expression profiles of the optimized network versus relative position; fixed morphogen inputs at reference length in background. B: Effective contributions of each regulator (morphogen or gene) on target gene : negligible (, grey), weak (, light) or strong (, dark); red, activation; blue, repression. Unlike the other genes, G2 is barely regulated by the inputs but strongly repressed by all other genes. C: Relative boundary positions versus embryo length , with linear fits. D: Standard deviation of boundary positions versus mean position (bootstrapped errors); red curve, expected precision for non-scaling boundaries anchored to the poles (Supplement). E: Single-embryo profiles colored by (left) and scaling deviation estimated from random subsets of embryos, extrapolated to (right; Supplement). F, G: Positional information and fraction of information about absolute position for single genes and the joint pattern. Error bars in G are bootstrapped.
Figure 2C–G provide quantitative measures of scaling properties. Figure 2C shows the relative positions of the domain boundaries as a function of embryo length. Lines are nearly flat: residual slopes are small but nonzero, corresponding to boundary shifts below one cell over the range of simulated lengths, within our scaling criteria (Methods, Sec. 4.5). More quantitatively, Figure 2D shows the standard deviation of boundary positions shown in C, as a function of the mean boundary position, with typical values around of the embryonic length (to be compared to the variation in embryonic length we impose). For further comparison, we also show an estimate of precision (red line in Figure 2D) based on the limiting case of non-scaling anchored boundaries (this estimate depends on relative position and vanishes at and , since the poles are well defined irrespective of embryo length). So the optimized standard deviation is well below this natural limit, as expected for scaling solutions.
Finally, we use an information-based measurement, the deviation from scale invariance , which quantifies how much positional information is gained by knowing the length of the embryo (see Methods, Sec. 4.5 for definition, corresponds to perfect scaling). Figure 2E shows gene expression profiles as a function of relative position for each gene and a range of embryo lengths, as well as estimates of . Visually, we see that rescaled profiles collapse on one another irrespective of the length, indicating scaling, which is quantitatively confirmed by the very low for all four genes. The joint pattern carries about bits of positional information (Figure 2F), and Figure 2G quantifies the fraction of information about absolute position for each gene and shows that it is smaller than both for individual genes and for the joint distribution (thus satisfying the scaling criteria defined in Methods, Sec. 4.5). Thus, this example illustrates that our optimization method allows for the optimization of scaling domain solutions using information-based criteria, with clear, quantitative measurements of information content, position and scaling.
A statistical analysis of domain-scaling solutions
Having established that our framework can find working solutions, we now test if we can generate extensive families of solutions, with multiple scenarios and enough statistical power to disentangle the effect of multiple constraints on the optimization of domain scaling.
Morphogen shapes
We first tested the influence of the shapes of the morphogen inputs. We considered various symmetric and asymmetric scenarios with different combinations of and to vary gradient steepness and the range of possible positional readouts (Figure 3A), and optimized 200 GRNs per input combination with the scaling constraint. We then categorized the optimization outcomes: GRNs in which no pattern formed, or for which optimization did not converge, are

Figure 3. Optimization of scaling GRNs under different non-scaling input scenarios in a minimal domain-formation problem. Panels B–D show results obtained with the scaling constraint; corresponding results without it are shown in Supplementary Figures. A: Symmetric (S, top) and asymmetric (A, bottom) opposing input scenarios, with decay lengths and in units of . B: Optimization outcomes by scaling performance and number of expressed genes. C: Positional information , mean boundary precision, (median over genes) and mean boundary shift for fully and partially scaling GRNs. Mean ± standard deviation across networks; dashed lines, scaling thresholds. D: Signed-score interaction matrix of the 88 fully scaling four-gene GRNs, genes ordered anterior to posterior. E: Perturbation-averaged versus for the 107 fully scaling four-gene GRNs (19 optimized without and 88 with the scaling constraint), with marginal histograms. Error bars, standard error across perturbations; dashed lines, scaling thresholds. Green and red stars, robust (Figure 2) and non-robust (Supplementary Figures) examples.
distinguished from models that scale either fully or partially. Partially scaling GRNs have at least one gene but not all satisfying the scaling criteria (Methods, Sec. 4.5).
Depending on the input scenario, between 20 and 55% of networks fully scale, with either three or four expressed genes, with intermediate-steepness scenarios A2 and S2 yielding the highest fractions of fully scaling models (Figure 3B). This result is intuitive: we expect scaling to be most difficult far from the poles, where morphogen levels are low, and S2 corresponds to the case where both morphogens are still present in the center, with substantial relative variation with respect to one another. Positional information, boundary precision and scaling metrics ( and boundary shift , Methods, Sec. 4.5) are summarized in Figure 3C. There is a clear difference between partially scaling and scaling solutions as expected but, interestingly, for the scaling solutions and boundaries, we do not see much difference between input scenarios.
For comparison, we also performed 200 optimizations per input scenario with the mutual-information objective alone (Supplementary Figures). Adding the scaling constraint increased the overall fraction of fully scaling solutions from 31% to 39%, and the fraction of fully scaling four-gene solutions from 2% to 9%.
Phenotypic and regulatory convergence
Among fully scaling solutions, we observe limited phenotypic variability, i.e. all fully optimized patterns present at least 3 domains of equal size (as expected from the loss function used [19]). Such convergence in phenotypes (i.e. pattern) raises the question of the underlying genotypes (i.e. parameters) and corresponding regulatory strategies (i.e. correlated interaction signs and strengths). We computed a signed-score interaction matrix from the 88 fully scaling four-gene solutions obtained by optimizing mutual information with an explicit scaling constraint, pooling all five input scenarios and ordering the genes from anterior to posterior (Supplement); this matrix is shown in Figure 3D. The pattern we find is remarkably simple and intuitive. There is first a graded regulation pattern by the inputs (anterior to the left), with the leftmost gene (G1) strongly activated by the anterior morphogen (input 1) and weakly repressed by the posterior one (input 2). The second gene from the left (G2) is moderately activated by input 1 and only marginally regulated by input 2. We observe a symmetrical situation for the two rightmost (posterior) genes. All genes further self-activate, more strongly for genes at the boundaries, and neighboring genes repress one another. Mutual repression between neighboring genes is known to sharpen boundaries and improve their precision [41], and interactions among target genes can increase the information they jointly transmit about their inputs [42]; but our high-throughput approach allows for the establishment of more general rules, with subtle effects on the magnitude of interactions as a function of position. Notice also that, while average interactions are well defined, the example shown in Figure 2 presents both typical features (neighboring genes strongly repressing) and exceptions to these rules (regulation of Gene 2), showing that, even for relatively simple problems, there is strong variability in the solutions found.
Exploring regulatory strategies through perturbation analysis
Having established general features of scaling solutions, we can also use our framework to study which aspects of the networks found matter the most for the scaling behavior. Inspired by biology, we followed a targeted mutation approach with two classes: (i) single gene knockouts (KO) for each of the four genes and (ii) regulatory interaction perturbations, selectively removing contributions from the matrix by setting selected classes of interaction strengths to zero. As in a laboratory knockout, all other parameters are kept at their optimized values, without re-optimization [23]. Specifically, we removed auto-regulation, nearest-neighbor (NN) interactions on all network interactions, and also tested networks retaining only auto-regulation, only NN interactions, or both, the latter removing all next-nearest-neighbor and longer-range interactions (Methods, Sec. 4.6). These perturbations assess the relative roles of morphogen inputs versus network interactions in establishing scale-invariant boundaries.
We first examined the example GRN from Figure 2 (details in Supplementary Figures). Scaling in this network is robust to both removal of genes and removal of interactions, with relative domain positions largely preserved. This suggests that scaling is input-driven, particularly through the regulation of genes 1, 3 and 4. In the Supplement, we show how this can occur: thresholds defined by linear combinations of the logarithms of the opposing morphogen concentrations can specify boundaries at fixed relative positions as embryo length varies, consistent with earlier work on combinatorial decoding of opposing gradients [43, 44].
We contrasted this with another GRN from scenario A1 with weaker morphogen contributions but preserved scale invariance (Supplement). There, the central genes are significantly less robust to perturbations, as their boundary scaling depends more on network interactions than morphogen inputs. In particular, perturbations that remove nearest-neighbor regulatory interactions are the most detrimental to scaling. This is reminiscent of previously proposed architectures based on mutual repression between neighboring domains [41].
To test whether these behaviors are general features of scaling solutions to the domain formation problem, we repeated the perturbation analysis across all 107 fully scaling four-gene GRNs, pooled across the five input scenarios and across optimizations with and without the scaling constraint (Figure 3E). For each GRN and perturbation type, we computed the fraction of information about absolute position (near zero for scale invariance) and the mean boundary shift , and then averaged over perturbations. We see a continuum of solutions, with strong correlations between the two metrics: when a perturbation breaks scaling, it does so by making boundaries shift systematically with embryo length, which in turn makes the pattern informative about absolute position. Robust GRNs, such as the input-driven example of Figure 2 (green star), still satisfy both scaling criteria on average after perturbation, whereas non-robust GRNs, such as the network-driven example from scenario A1 (red star), violate both. This establishes that, even if we found multiple scaling solutions with almost exactly the same phenotype, there is degeneracy not only in the genotype space (strength of interactions), but also in the higher-level regulatory space (overall strategy underlying scaling).
Optimizing positional information
We now move to a problem with a more complex function where the space of phenotypes should be much more degenerate. Inspired by studies of the Drosophila gap gene system [45, 46], we generate gene networks optimizing mutual information between position and overall gene expression , i.e. positional information (Methods, Sec. 4.4). This contrasts with the previous section in the sense that we no longer require a single gene to be expressed at any given position: rather, we expect that combinations of genes encode local information, allowing for a cell to decode its position from such combinations. Again, we consider optimization of gene networks with four genes (by analogy with the four main gap genes hb, Kr, kni and gt), and consider optimization in response to three input genes: one anterior (similar to bicoid), one posterior (similar to caudal), and one expressed at both ends (similar to the terminal torso system); following Drosophila nomenclature, these are labeled as maternal inputs A, P and T in the figures. To match the Drosophila case, we further introduced extrinsic fluctuations in the inputs (Methods, Sec. 4.1).
Of note, this optimization requires special care in the definition of the loss function. The standard mutual information estimator involves a Gaussian-mixture marginal entropy that lacks a closed-form expression [47]. To obtain a closed-form, differentiable approximation of positional information suitable for gradient-based optimization, we adapt an existing second-order entropy expansion for Gaussian mixtures [47] to serve as a training objective within our end-to-end differentiable pipeline. In detail, we rely on multivariate Gaussian approximations to fit the average gene profiles, their covariance, and Taylor-expand the marginal entropy [47] that depends only on the average of gene profiles and the covariances (Methods, Sec. 4.4). Those can be fitted through differentiable functions, thus providing us with estimates of mutual information that are fully differentiable.
Using this differentiable positional information approximation , we optimize GRN models via gradient descent. Following [23], we impose biologically informed sign constraints on regulatory couplings and “resource constraints” mimicking what is observed in Drosophila (Methods, Secs. 4.1 and 4.2).
We tested optimization scenarios with different biologically relevant constraints, optimizing 200 models for each scenario from random initializations. We found that our optimization pipeline reliably discovered high positional information solutions (about 4.4 bits on average, Figure 4A, comparable to the 4.2 bits found in the Drosophila gap gene system [46, 23]), with many different patterns (a sample is shown in Figure 4B). To quantify the diversity of solutions, we computed pairwise permutation-invariant distances between patterns (Methods, Sec. 4.6). Pairwise distance matrices (Figure 4B) reveal structured regions of low pairwise distance, forming large clusters along the diagonal that frequently correspond to symmetric patterns (for example profile e in Figure 4B).

Figure 4. Without embryo-length variability, a constraint promoting uniform positional error reshapes precision while preserving pattern diversity. Networks optimized and evaluated without embryo-length variability (all embryos of length ), with ; 200 initializations per scenario. A: Positional information , mean positional error , mean boundary precision and without () and with () the uniformity constraint. Mean ± standard deviation across converged networks ( bits; convergence percentages in legend). B: Pairwise distance matrices of converged networks (left), clustered independently for each scenario. Vertical lines in the distance matrices indicate representative patterns shown on the right. These examples illustrate the qualitative diversity of spatial expression profiles obtained across independent optimization runs. C: Positional error along the AP axis: mean across converged networks (solid, ; dashed, ) ± standard deviation (shading, hatching).
Changing constraints modulates the nature and diversity of accessible solutions
From this diverse baseline of network solutions optimizing high information content, we can study how various constraints can be added to shape the space of the solutions. Adding a precision-related constraint such as uniformity of the positional error (the precision with which position can be decoded from gene expression, Methods, Sec. 4.4), quantified by its coefficient of variation along the AP axis, reshapes the high-information solution manifold. With such a constraint, we consistently obtained GRN models with more uniform positional error while maintaining comparably high positional information (Figure 4A,C). We note a trade-off, with improved uniformity accompanied by a modest reduction in mean positional information and an increase in mean positional error. Substantial heterogeneity in pattern space persisted across independently optimized high-information solutions, and the distributions of pairwise distances overlap across optimization scenarios, with comparable means and spread (Supplementary Figures). This suggests that the positional-error uniformity constraint restricts the set of high-information solutions but does not collapse it in pattern space.
We also varied the noise level in the inputs (Supplementary Figures). We found that pairwise distances decreased with increasing input noise during optimization, consistent with an implicit regularization effect that compressed the accessible high-performing manifold in pattern space, but without changing it qualitatively. At the GRN level, increasing optimization noise systematically altered the regulatory architecture, from more feedforward to more feedback-based architectures. These interaction changes paralleled the improved robustness to input noise, consistent with internal gene–gene coupling reducing effective sensitivity to fluctuations in maternal inputs.
From these various case studies, we conclude that the differentiable approximation of positional information provides a stable objective for gradient-based optimization through spatial stochastic simulations. Our high-throughput optimizations allow us to show how additional constraints can modulate the space of possible solutions found, with more feedback-based regulation when input noise is increased.
Pattern diversity is reduced among fully scaling high-positional-information networks
The persistence of pattern diversity under the positional-error uniformity constraint and input-noise variations raises the question of whether such degeneracy survives a qualitatively different systems-level constraint: scale invariance. We combined the positional-information objective with the MAD scaling constraint introduced above (), with or without an additional constraint on the uniformity of positional error (Figure 5A; see Methods, Secs. 4.4 and 4.5). Note that in contrast with the domain-formation problem, simply optimizing positional information over an ensemble of embryos of variable length is not sufficient to obtain scaling solutions: the pressure towards scale invariance carried by the objective is weaker and must be reinforced explicitly
Figure 5. With embryo-length variability, pattern diversity is reduced and regulatory structure converges among fully scaling high-positional-information GRNs. Networks optimized at and evaluated at with 5% length variability; 200 initializations per scenario. A: Positional information and median (over genes) for four scenarios: and optimized without length variability (“no L var.”), and optimized with length variability (“L var.”); all evaluated with length variability. Mean standard deviation across networks. B: Pairwise distance matrices per scenario, grouped by scaling performance: fully (green), partially (orange) or not scaling (red). C: Cumulative distributions of pairwise distances for the scenarios in B (colors as in A; dashed, with uniformity constraint) and for pooled fully scaling networks (black). Inset: mean , with bootstrap standard errors from resampling networks within scenarios. D: matrix of fully scaling GRNs pooled across MAD scenarios (; top band, scenario). White boxes, main pattern clusters a–c, with representative profiles (cluster medoids). E: Center: pairwise Manhattan distances between regulatory topology vectors within cluster a (; Methods, § 4.6); white boxes, two largest subclusters. Left and right: their mean regulatory matrices (red, activation; blue, repression). Genes ordered anterior to posterior.
by the scaling constraint (Supplementary Figures). Since training at lower input noise yielded more fully scaling solutions (Supplementary Figures), networks in this section were optimized at and evaluated at . For each scenario, we found that about 20% of solutions were fully scaling. We also checked that networks optimized retain high positional information and comparable positional precision, while substantially reducing the scaling deviation , the fraction of information about absolute position and the mean boundary shift (Supplementary Figures). This shows that scale invariance can
be selected directly without losing patterning performance: all four scenarios reach about 4 bits of positional information, while only MAD-optimized networks bring the median below threshold Figure 5A.
However, in contrast to the positional-error uniformity constraint and input-noise variations considered above, the scaling constraint strongly restricts the accessible pattern space, selecting a narrow subset within the broader high-information manifold. In particular, matrices show stronger clustering and reduced pairwise distances, with cumulative distributions systematically shifted toward much smaller values (see Supplementary Figures for comparisons of scaling metrics under all scenarios). When pooled across scenarios, the fully scaling GRNs (green sub-squares in Figure 5B) occupy a more restricted region of pattern space than the full set of optimized solutions, with the lowest mean pairwise distance of all ensembles (Figure 5C, black, and inset). Their pairwise distances reveal only a few correlated clusters, most of which correspond to approximately symmetric expression patterns. The very dominant cluster a (50 networks, Figure 5D) bears resemblance to the sequential , and posterior domains sandwiched between two regions, observed along the Drosophila AP axis, with moderate overlaps between the four genes. We note that these patterns were already present in the positional-information optimization results without the scaling constraint (Supplementary Figures).
Regulatory convergence for scaling with high-positional information
To examine whether the phenotypic convergence in pattern space reflects a corresponding regulatory convergence in network space, we characterized the interactions of fully scaling GRNs within each pattern cluster (Figure 5E). Pairwise Manhattan distances between topology vectors (Methods, Sec. 4.6) reveal only a small number of tight subclusters, indicating that only a few distinct regulatory strategies are obtained. The dominant subcluster within each pattern group accounts for the majority of solutions and, in all cases, is characterized by weak repressive interactions between neighboring genes and strong repressive interactions between non-adjacent genes along the AP axis. Scaling thus acts as a strong filter on interactions, selecting a specific class of underlying regulatory mechanisms. This architecture corresponds to the “alternating cushions” scheme previously proposed for the Drosophila gap gene system [48, 49].
We show one representative fully scaling GRN in Figure 6 from cluster a identified in Figure 5D, together with its associated precision and scaling metrics. To get a better sense of how scaling works, we further performed perturbation analyses on representative scaling networks obtained (Figure 6 and see another example in the Supplementary Figures). Perturbation analysis showed that these repressions between non-adjacent genes are important for maintaining both precision and scale invariance: for instance, knocking out (resp. ) causes (resp. ) to expand into the vacated domain; this expression is still relatively precise (Figure 6H) but no longer scales (Figure 6F). The more central the gene, the larger the impact: for instance removal of the central strongly perturbs both precision and scaling for all genes, while removal of is much less impactful on both properties (Figure 6F–H). In the Supplement, we further tested other perturbations, e.g. showing that while a minimal level of diffusion is necessary, diffusion plays no significant role in maintaining either precision or scale invariance in these networks.

Figure 6. Example of a fully scaling high-positional-information GRN. Network optimized for positional information with scaling and positional-error uniformity constraints (). A–D: As Figure 2A–D; A additionally shows the positional error along the AP axis. E: As Figure 2G. F: Single-gene knockouts (KO): single-embryo profiles for embryos of different lengths, with mean expression. G: Removal of all gene–gene interactions (No net., , hatched), profiles as in F; other network perturbations in Supplementary Figures. H: Mean positional error and joint for wild type (WT) and the perturbations in F and G. Dashed lines, the reference positional error of one cell and the scaling threshold of 2%; error bars, standard deviations.
Discussion
We propose a forward, predictive approach by leveraging modern machine learning tools to perform systematic optimizations of models of gene networks, under various constraints. We focused on embryonic patterning under the control of static morphogens as a case study, and optimized complex properties by gradient descent, such as scaling or positional information. Over multiple rounds, our approach produced thousands of solutions, with different properties depending on the constraints, which allows us to perform a detailed analysis of both the solutions themselves and their statistics.
At a granular level, we optimized networks for scaling, under multiple primary objectives, and, remarkably, recovered regulatory strategies related to three previously proposed archetypes: input-based decoding [43, 44], mutual repression between neighboring domains [41], and the second-neighbor-based “alternating cushions” architecture, with weak repression between adjacent domains and strong repression between next-adjacent ones [48, 49]. While this validates the tool for enumerating plausible mechanisms, the diversity of solutions found rather suggests a continuous space of solutions, where the optimized networks tinker and combine several archetypes to get a scaling solution. Here we imposed the length variability measured in Drosophila; for such small variations, scaling only needs to hold to linear order in , which many architectures can achieve. We expect larger size variations to require scaling beyond linear order and thus to restrict solutions to even fewer strategies, a prediction directly testable within our framework.
Thus, beyond the specific cases studied, our high-throughput approach allows us to systematically probe the space of solutions for complex biological problems, shedding new light on some long-debated questions on the “actual” and the possible [15]. We first observe a rather complex genotype-to-phenotype mapping, best illustrated with the domain scaling problem, where the exact same pattern of four equally sized scaling domains is realized through a continuum of underlying network architectures in network space. Depending on the problem, we also get a spectrum of properties on the family of solutions themselves. When optimizing for positional information only, we get a zoo of solutions, with very comparable positional information, thus suggesting a highly diverse space of solutions for that problem specifically, consistent with previous work [23]. Conversely, when optimizing for scaling and information simultaneously, we see a collapse onto a few solutions. Interestingly, the dominant pattern found in the latter case was also observed when optimizing for positional information only, and gradually becomes more frequent as we increase the number of constraints such as precision. It thus seems that adding constraints does not simply improve or degrade performance, rather they reshape the space of accessible solutions.
This naturally raises the question of why some specific solutions are obtained, especially in the latter regime where only very few emerge from gradient descent. The simplest explanation is that the space of optimal solutions itself shrinks as more constraints are added, so that our optimization simply reflects this restriction. Alternatively, as the problem becomes more constrained (e.g. from information only to information + scaling), directions in network space might themselves become restricted, as previously observed with evolutionary approaches on simpler problems [50, 20], and more generally proposed from the emergence of low-dimensional or “soft” modes in evolved biological systems [7, 51]. Only a few solutions might then be easily reachable by gradient descent, while others remain isolated or difficult to access, somewhat reminiscent of the “survival of the flattest” mechanism [52].
We did not disentangle the biases of gradient descent from the intrinsic structure of the solution space. Gradient-free Monte Carlo [23] and evolutionary [19] approaches could in principle serve as comparisons, but they lack the throughput needed to sample thousands of networks, and they have their own, different biases. This raises the broader question of the inductive biases of our optimization approach, which might or might not be shared with actual evolution. Such biases could arise, for instance, from combining multiple constraints into a single loss rather than using a Pareto-like optimization [53]. A bias toward simpler or more symmetric solutions during optimization has already been reported in related contexts [54, 55], and has been related to the idea from complexity theory that “simple” patterns (in the Kolmogorov complexity sense) are, surprisingly, more probable [56]. Here, symmetric patterns are already common among high-information solutions but come to dominate once scaling is imposed, suggesting that such intrinsic biases interact non-trivially with the geometry of the constrained solution space. These questions could be explored directly within our framework by varying the optimization procedure or its initial conditions. Notably, recent large-scale simulations of artificial neural networks have also shown increasingly complex relationships between initial conditions and convergence time as problems become harder, including a fractal phase transition [57]. Whether similar phenomena occur for GRNs remains an open question.
Lastly, the coexistence of multiple regimes in our optimizations, with many solutions on the one hand vs few solutions on the other hand is very reminiscent of the evolutionary question raised in the introduction of contingency vs convergence. The size and diversity of network spaces have been long studied through the lens of robustness and evolvability [58, 59, 1] or through enumeration of circuits performing specific tasks [60, 16]. While these approaches were typically limited to small networks and/or discrete phenotypes, our framework complements and extends them: it optimizes a continuous loss (e.g. information) through stochastic spatial simulations, can sample thousands of solutions for a given set of constraints, and track how the ensemble of solutions reorganizes. In our simulations, one can further continuously move from one regime to the other simply by adding more constraints. This does not tell us how real evolution behaves, but shows that our proposed framework can turn the old contingency vs convergence debate into a series of testable questions at scale: given a biological function and a set of constraints, how large, connected and diverse is the space of solutions? More broadly, differentiable programming may therefore provide the basis for a variational systems biology in which the possible, and not only the actual, becomes an object of systematic study.
Methods
Model and stochastic simulations
We simulate morphogen-driven reaction-diffusion dynamics along a one-dimensional anterior-posterior (AP) axis for an ensemble of embryos, with externally specified morphogen inputs and network genes interacting through regulatory feedback. For each gene ,
where and are the vectors of gene and morphogen levels, the decay rate, the diffusion coefficient and the production term. We integrate a spatially discretized version of (1) on nuclear volumes separated by a distance , with reflecting boundaries and effective hopping rate .
Gene regulation is modeled with a Monod-Wyman-Changeux (MWC) framework rather than a phenomenological Hill function, so that each interaction is described by two interpretable parameters: a signed regulatory strength and an effective dissociation constant . The activity of gene is a sigmoid of a regulation potential combining saturating morphogen and network contributions,
where sets gene-specific activation thresholds and the superscripts distinguish morphogen and network gene contributions. When prior knowledge is available about interaction signs, we optionally fix the sign of selected entries of and/or while learning their magnitudes. The production term includes a basal floor, .
Stochasticity enters at two levels. Embryo-to-embryo extrinsic variability is applied multiplicatively to the inputs: for each embryo , input and site ,
with generating global amplitude and local fluctuations, drawn once per embryo and held fixed along the trajectory; we write for their common value when , calibrated against measured Bicoid variability (Supplement). Intrinsic expression noise is added during integration through a chemical Langevin approximation with production, degradation and diffusive-hopping contributions [30, 61, 23]. Equations are integrated by explicit Euler-Maruyama with a ReLU projection enforcing non-negative expression; differentiability is preserved with a straight-through estimator (Supplement).
Morphogens have fixed physical decay lengths (reported in units of the reference length ), so that in relative coordinates their profiles
depend explicitly on embryo length and are not scale invariant; terminal inputs expressed at both poles are built as the sum of two such gradients (Supplement).
Embryo lengths are drawn , . All embryos share the same grid of relative positions, so a length change rescales the spacing, with : size therefore enters both through the input profiles (Eq. (4)) and through the effective hopping rate . Although the framework supports optimization of dynamical parameters, diffusion and decay are here held fixed and identical across genes, so that (bounded reparameterizations, Supplement). Noise sources and embryo lengths are resampled at every optimization step, and all settings are listed in Table 1. Following [23], the posteriormost 7% of embryo length is excluded from objectives and position averages in the positional-information problem, as wild-type gap genes are not expressed there.
| Nuclei | 90 [62] | |
| Reference spacing | 8.5 [23] | |
| Length variability | 5% [40] | |
| Diffusion | 0.4 [23] | |
| Decay rate | 1 (sets the time unit, ) | |
| Basal production | ||
| Intrinsic noise | ||
| Input noise | 0 (domain formation) / [0, 0.05, 0.10, 0.15] (positional information) | |
| Embryos per step | 100 | |
| Time step | 0.01 | |
| Measurement time | 15 decay times | |
| Optimizer | Adam [34] | learning rate 0.1, 5000 steps |
| Runs per scenario | 200 random initializations | |
| Analysis ensemble | 500 embryos |
Table 1. Simulation and optimization settings.
Loss structure
We minimize by gradient descent a loss combining a primary objective with biological constraints and regularization,
evaluated on the simulated profiles , where is the primary objective, either penalizing an error directly or written with a minus sign to maximize a metric (e.g. positional information, ). Constraints are soft inequality penalties [63], inactive when a model output falls within a biologically motivated range and growing smoothly once it is violated: for a quantity with admissible threshold ,
with symmetric penalties for two-sided ranges. Normalization by renders constraints dimensionless and comparable. Two constraints are used in the positional-information problem: a global resource-utilization budget, defined as expression averaged over genes and spatial positions at , with an upper bound of as measured in gap gene data [45], and a minimum peak-expression target of 0.9 per gene. The regularization term consists of an penalty on regulatory parameters, supplemented in the domain-formation problem by a cosine-similarity penalty on near-duplicate rows of and to discourage different genes from converging to identical regulatory structures (Supplement). A constraint may alternatively be applied as a multiplicative gate on the primary objective rather than as an additive penalty; we use this only for the scaling constraint, described below.
Loss function for domain formation
Here, position is a discrete variable with values and cell-fate identity a discrete variable with values . Following [19], we define per embryo the transformed expression , which limits the dominance of highly expressed genes, and convert it into a categorical distribution over identities,
so that identity depends on relative rather than absolute expression. These conditionals are averaged across embryos at each relative position, , and combined with a uniform prior into the joint . The domain-formation loss is the logarithmic distance from the maximal achievable position-identity information [19],
where reaches its maximum for sharply separated domains of equal extent and vanishes when identities are uniformly mixed, so that minimizing drives toward its maximum.
Differentiable positional information
Positional information is the mutual information between the expression vector , measured at a fixed readout time, and position :
where denotes entropy and spatial averages are discrete sums. Throughout, denotes this quantity, expressed in bits, and is evaluated for all optimized models irrespective of the objective used. Following [46, 23], the conditional distribution at fixed position is approximated by a multivariate Gaussian with and the ensemble mean and covariance, giving . The marginal is then a Gaussian mixture whose entropy has no closed form, as it involves the logarithm of a sum of Gaussians. We approximate it by a second-order expansion [47], Taylor expanding about the conditional mean of each mixture component and averaging under the local Gaussian:
where the second term is a curvature correction controlled by the local covariance. Eq. 10 yields a differentiable approximation of computed entirely from simulated ensemble means and covariances, through which gradients are propagated together with the simulation pipeline itself. Precision is quantified by the positional error [46]
with derivatives taken by central differences in relative coordinates and reported in cells, one cell being one nuclear spacing. The positional-error uniformity constraint minimizes the coefficient of variation along the axis, which has no natural target value.
Scale invariance
Scale invariance is optimized explicitly through a constraint on the mean absolute deviation (MAD) of expression profiles, which measures profile alignment in relative coordinates as the simulated embryonic length varies,
with the ensemble mean profile (excluding the posterior cutoff region in the positional-information problem). The corresponding penalty is given by Eq. 6 with and . It is applied as a multiplicative reward gate on the primary objective [53, 23],
so that the gate equals two when the constraint is satisfied and decays toward one as it is violated, thereby favoring alignment. In the positional-information problem, is negative, so the gate simply becomes . In the domain-formation problem, the gate acts on the logarithmic loss of (8).
After optimization, boundaries are detected on individual gene expression profiles (Supplement), and expressed in relative coordinates , which are length-independent under perfect scaling (Fig. 2C). We compute their embryo-to-embryo standard deviation , compared to the value expected for non-scaling boundaries anchored to the poles (Supplement), and the systematic boundary shift , obtained by linear regression of against embryo length and expressed in cells over the 5–95% range of simulated lengths. Following [40], we also quantify the deviation from scale invariance
which measures how much positional information is gained by additionally knowing embryo length, so that means length cannot be inferred from the profiles, a necessary condition for scaling. It is estimated under a Gaussian approximation on embryos binned by length and corrected for finite-sample bias by extrapolating subsampled estimates to infinitely many embryos, with bootstrapped errors and shuffled controls (example in Fig. 2E; Supplement). For the domain-formation problem, a GRN is classified as fully scaling if all expressed genes satisfy bits, and cell. For the positional-information problem, we use slightly relaxed thresholds to account for the added input noise: all four genes must satisfy bits, and cells. These thresholds remain consistent with the range of scaling metrics measured experimentally in the Drosophila gap gene system [40]. A GRN is classified as partially scaling if at least one, but not all, of the genes satisfies the corresponding criteria. Runs are classified as not converged when the loss plateaus well above the ensemble mean loss, and as forming no pattern when the profiles carry less than 3 bits of positional information.
Network analysis
Following [23], each interaction between regulator (gene or morphogen) and target gene is summarized by its effective contribution at maximal expression, , classified as negligible (), weak () or strong (), with and ; signed-score matrices summarizing sign consistency across an ensemble are defined in the Supplement. Regulatory topologies are compared by discretizing each into five signed categories using the same thresholds, reordering genes by first half-maximum crossing and concatenating the discretized input-to-gene and gene-to-gene matrices into one topology vector per GRN, which are then clustered by Manhattan distance and average linkage (Supplement). Optimized patterns are compared across runs through a permutation-invariant distance between mean profiles at ,
minimized over permutations of the gene labels, which are arbitrary across independent optimizations; we set following [23] and reorder matrices by hierarchical clustering (average linkage). Finally, optimized networks are perturbed by single-gene knockouts and by setting selected classes of entries of to zero: removing auto-regulation, nearest-neighbor (NN) interactions or all gene–gene interactions, or retaining only auto-regulation, only NN interactions, or both. The last condition removes all non-adjacent interactions, i.e. both next-nearest-neighbor (NNN) and longer-range interactions, with neighbors defined by ordering genes by first half-maximum crossing. Perturbed networks are then re-simulated without re-optimization and re-analyzed with the same metrics (Supplement).
Code availability
The jaxgrn framework and the configuration files used to generate all figures will be released as open-source software upon journal publication. A preliminary version is available from the authors on request.
Acknowledgments
We acknowledge the use of AI-based writing tools for light language enhancement. This work was supported by Institut Pasteur, the French National Research Agency (ANR-10-LABX-73 “Revive” and ANR-16-CONV-0005 “INCEP-TION”), the CFM Foundation for Research, and the inIdEx FORMULA project at Université Paris Cité (ANR-18-IDEX-0001). This research was enabled in part by support provided by Calcul Québec (calculquebec.ca) and the Digital Research Alliance of Canada (alliancecan.ca).
References
- [1]A. Wagner, Arrival of the Fittest: How Nature Innovates. Current, 2015.
- [2]D. H. Erwin, The Origins of the New: Novelty and Innovation in the History of Life, Culture, and Technology. Princeton University Press, 2026.DOI
- [3]S. J. Gould, Wonderful Life: The Burgess Shale and the Nature of History. W. W. Norton & Company, 1989.
- [4]J. B. Losos, “Convergence, adaptation, and constraint,” Evolution, vol. 65, no. 7, pp. 1827–1840, 2011.DOI
- [5]T. Uller, A. P. Moczek, R. A. Watson, P. M. Brakefield, and K. N. Laland, “Developmental bias and evolution: A regulatory network perspective,” Genetics, vol. 209, no. 4, pp. 949–966, 2018.DOI
- [6]K. Kaneko, “Evolution of robustness to noise and mutation in gene expression dynamics,” PLoS ONE, vol. 2, no. 5, p. e434, 2007.DOI
- [7]C. Furusawa and K. Kaneko, “Formation of dominant mode by evolution in biological systems,” Physical Review E, vol. 97, no. 4, p. 042410, 2018.DOI
- [8]Y. Schaerli, A. Jiménez, J. M. Duarte, L. Mihajlovic, J. Renggli, M. Isalan, J. Sharpe, and A. Wagner, “Synthetic circuits reveal how mechanisms of gene regulatory networks constrain evolution,” Molecular Systems Biology, vol. 14, no. 9, p. MSB178102, 2018.DOI
- [9]W. Bialek, Biophysics: Searching for Principles. Princeton University Press, 2012.
- [10]G. Tkačik, C. G. Callan, and W. Bialek, “Information flow and optimization in transcriptional regulation,” Proceedings of the National Academy of Sciences, vol. 105, no. 34, pp. 12265–12270, 2008.DOI
- [11]S. Mangan and U. Alon, “Structure and function of the feed-forward loop network motif,” Proceedings of the National Academy of Sciences, vol. 100, no. 21, pp. 11980–11985, 2003.pubmed.ncbi.nlm.nih.gov/14530388
- [12]W. F. Młynarski, M. Hledík, T. R. Sokolowski, and G. Tkačik, “Statistical analysis and optimality of neural systems,” Neuron, vol. 109, no. 7, pp. 1227–1241, 2021.DOI
- [13]D. B. Brückner and G. Tkačik, “Marr’s three levels for embryonic development: information, dynamical systems, gene networks,” PRX Life, vol. 4, no. 1, p. 017001, 2026.
- [14]D.-E. Nilsson and S. Pelger, “A pessimistic estimate of the time required for an eye to evolve,” Proceedings of the Royal Society B: Biological Sciences, vol. 256, no. 1345, pp. 53–58, 1994.pubmed.ncbi.nlm.nih.gov/8008757
- [15]F. Jacob, “Evolution and tinkering,” Science, vol. 196, no. 4295, pp. 1161–1166, 1977.pubmed.ncbi.nlm.nih.gov/860134
- [16]J. Cotterell and J. Sharpe, “An atlas of gene regulatory networks reveals multiple three-gene mechanisms for interpreting morphogen gradients,” Molecular Systems Biology, vol. 6, p. 425, 2010.DOI
- [17]Y. Schaerli, A. Munteanu, M. Gili, J. Cotterell, J. Sharpe, and M. Isalan, “A unified design space of synthetic stripe-forming networks,” Nature Communications, vol. 5, no. 1, p. 4905, 2014.DOI
- [18]P. François and V. Hakim, “Design of genetic networks with specified functions by evolution in silico,” Proceedings of the National Academy of Sciences, vol. 101, no. 2, pp. 580–585, 2004.
- [19]P. François and E. D. Siggia, “Predicting embryonic patterning using mutual entropy fitness and in silico evolution,” Development, vol. 137, no. 14, pp. 2385–2395, 2010.DOI
- [20]H. Booth and Z. Hadjivasiliou, “Gene network organization, mutation, and selection collectively drive developmental pattern evolvability and predictability,” PRX Life, vol. 3, no. 3, p. 033023, 2025.
- [21]T. W. Hiscock, “Adapting machine-learning algorithms to design gene circuits,” BMC Bioinformatics, vol. 20, no. 1, p. 214, 2019.
- [22]J. Shen, F. Liu, Y. Tu, and C. Tang, “Finding gene network topologies for given biological function with recurrent neural network,” Nature Communications, vol. 12, no. 1, p. 3125, 2021.DOI
- [23]T. R. Sokolowski, T. Gregor, W. Bialek, and G. Tkačik, “Deriving a genetic regulatory network from an optimization principle,” Proceedings of the National Academy of Sciences, vol. 122, no. 1, p. e2402925121, 2025.DOI
- [24]J.-B. Lalanne and P. François, “Principles of adaptive sorting revealed by in silico evolution,” Physical Review Letters, vol. 110, no. 21, p. 218102, 2013.DOI
- [25]T. Kondo, F. X. P. Bourassa, S. Achar, J. DuSold, P. F. Céspedes, M. Ando, A. Dwivedi, J. Moraly, C. Chien, S. Majgödl, et al., “Engineering TCR-controlled fuzzy logic into CAR T cells enhances therapeutic specificity,” Cell, vol. 188, no. 9, pp. 2372–2389, 2025.DOI
- [26]J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei, “Scaling laws for neural language models,” arXiv preprint arXiv:2001.08361, 2020.arxiv.org/abs/2001.08361
- [27]M. AlQuraishi and P. K. Sorger, “Differentiable biology: using deep learning for biophysics-based and data-driven modeling of molecular mechanisms,” Nature Methods, vol. 18, no. 10, pp. 1169–1180, 2021.DOI
- [28]M. Deistler, K. L. Kadhim, M. Pals, J. Beck, Z. Huang, M. Gloeckler, J. K. Lappalainen, C. Schröder, P. Berens, P. J. Gonçalves, and J. H. Macke, “Jaxley: differentiable simulation enables large-scale training of detailed biophysical models of neural dynamics,” Nature Methods, vol. 22, no. 12, pp. 2649–2657, 2025.DOI
- [29]R. Deshpande, F. Mottes, A.-D. Vlad, M. P. Brenner, and A. Dal Co, “Engineering morphogenesis of cell clusters with differentiable programming,” Nature Computational Science, vol. 5, no. 10, pp. 875–883, 2025.DOI
- [30]D. T. Gillespie, “The chemical Langevin equation,” The Journal of Chemical Physics, vol. 113, no. 1, pp. 297–306, 2000.DOI
- [31]P. E. Kloeden and E. Platen, Numerical Solution of Stochastic Differential Equations. Berlin, Heidelberg: Springer, 1992.
- [32]Y. Bengio, N. Léonard, and A. Courville, “Estimating or propagating gradients through stochastic neurons for conditional computation,” arXiv preprint arXiv:1308.3432, 2013.arxiv.org/abs/1308.3432
- [33]J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, P. Szoke, J. VanderPlas, S. Wanderman-Milne, and Q. Zhang, “JAX: Composable transformations of Python+NumPy programs,” 2018. Version 0.3.13.
- [34]D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in International Conference on Learning Representations (ICLR), 2015.arxiv.org/abs/1412.6980
- [35]J. Cooke, “Scale of body pattern adjusts to available cell number in amphibian embryos,” Nature, vol. 290, no. 5809, pp. 775–778, 1981.pubmed.ncbi.nlm.nih.gov/7219562
- [36]T. Gregor, W. Bialek, R. R. de Ruyter van Steveninck, D. W. Tank, and E. F. Wieschaus, “Diffusion and scaling during early embryonic pattern formation,” Proceedings of the National Academy of Sciences, vol. 102, no. 51, pp. 18403–18407, 2005.pubmed.ncbi.nlm.nih.gov/16352710
- [37]D. Ben-Zvi, B.-Z. Shilo, A. Fainsod, and N. Barkai, “Scaling of the BMP activation gradient in Xenopus embryos,” Nature, vol. 453, no. 7199, pp. 1205–1211, 2008.
- [38]V. M. Lauschke, C. D. Tsiairis, P. François, and A. Aulehla, “Scaling of embryonic patterning based on phase-gradient encoding,” Nature, vol. 493, no. 7430, pp. 101–105, 2013.DOI
- [39]M. Merle, L. Friedman, C. Chureau, A. Shoushtarizadeh, and T. Gregor, “Precise and scalable self-organization in mammalian pseudo-embryos,” Nature Structural & Molecular Biology, vol. 31, no. 6, pp. 896–902, 2024.DOI
- [40]M. Nikolić, V. Antonetti, F. Liu, G. Muhaxheri, M. D. Petkova, M. Scheeler, E. M. Smith, W. Bialek, and T. Gregor, “Scale invariance in early embryonic development,” Proceedings of the National Academy of Sciences, vol. 121, no. 46, p. e2403265121, 2024.pubmed.ncbi.nlm.nih.gov/38235065
- [41]T. R. Sokolowski, T. Erdmann, and P. R. ten Wolde, “Mutual repression enhances the steepness and precision of gene expression boundaries,” PLOS Computational Biology, vol. 8, no. 8, p. e1002654, 2012.DOI
- [42]A. M. Walczak, G. Tkačik, and W. Bialek, “Optimizing information flow in small genetic networks. II. feed-forward interactions,” Physical Review E, vol. 81, no. 4, p. 041905, 2010.pubmed.ncbi.nlm.nih.gov/20481751
- [43]P. McHale, W.-J. Rappel, and H. Levine, “Embryonic pattern scaling achieved by oppositely directed morphogen gradients,” Physical Biology, vol. 3, no. 2, p. 107, 2006.DOI
- [44]J. Shen, F. Liu, and C. Tang, “Scaling dictates the decoder structure,” Science Bulletin, vol. 67, no. 14, pp. 1486–1495, 2022.DOI
- [45]M. D. Petkova, G. Tkačik, W. Bialek, E. F. Wieschaus, and T. Gregor, “Optimal decoding of cellular identities in a genetic network,” Cell, vol. 176, no. 4, pp. 844–855.e15, 2019.DOI
- [46]J. O. Dubuis, G. Tkačik, E. F. Wieschaus, T. Gregor, and W. Bialek, “Positional information, in bits,” Proceedings of the National Academy of Sciences, vol. 110, no. 41, pp. 16301–16308, 2013.
- [47]M. F. Huber, T. Bailey, H. Durrant-Whyte, and U. D. Hanebeck, “On entropy approximation for Gaussian mixture random vectors,” Proceedings of the IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, pp. 181–188, 2008.DOI
- [48]S. Vakulenko, Manu, J. Reinitz, and O. Radulescu, “Size regulation in the segmentation of Drosophila: Interacting interfaces between localized domains of gene expression ensure robust spatial patterning,” Physical Review Letters, vol. 103, no. 16, p. 168102, 2009.pubmed.ncbi.nlm.nih.gov/19911861
- [49]J. Jaeger, M. Blagov, D. Kosman, K. N. Kozlov, Manu, E. Myasnikova, S. Surkova, C. E. V. Vanario-Alonso, M. Samsonova, D. H. Sharp, and J. Reinitz, “Dynamical analysis of regulatory interactions in the gap gene system of Drosophila melanogaster,” Genetics, vol. 167, no. 4, pp. 1721–1737, 2004.pubmed.ncbi.nlm.nih.gov/15342511
- [50]P. François, V. Hakim, and E. D. Siggia, “Deriving structure from evolution: metazoan segmentation,” Molecular Systems Biology, vol. 3, p. 154, 2007.pubmed.ncbi.nlm.nih.gov/18091725
- [51]K. Husain and A. Murugan, “Physical constraints on epistasis,” Molecular Biology and Evolution, vol. 37, no. 10, pp. 2865–2874, 2020.DOI
- [52]C. O. Wilke, J. L. Wang, C. Ofria, R. E. Lenski, and C. Adami, “Evolution of digital organisms at high mutation rates leads to survival of the flattest,” Nature, vol. 412, no. 6844, pp. 331–333, 2001.pubmed.ncbi.nlm.nih.gov/11460163
- [53]A. Warmflash, P. François, and E. D. Siggia, “Pareto evolution of gene networks: an algorithm to optimize multiple fitness objectives,” Physical Biology, vol. 9, no. 5, p. 056001, 2012.DOI
- [54]I. G. Johnston, K. Dingle, S. F. Greenbury, C. Q. Camargo, J. P. K. Doye, S. E. Ahnert, and A. A. Louis, “Symmetry and simplicity spontaneously emerge from the algorithmic nature of evolution,” Proceedings of the National Academy of Sciences, vol. 119, no. 11, p. e2113883119, 2022.DOI
- [55]V. Mochulska and P. François, “Generative epigenetic landscapes map the topology and topography of cell fates,” Proceedings of the National Academy of Sciences, vol. 122, no. 50, p. e2514508122, 2025.DOI
- [56]K. Dingle, C. Q. Camargo, and A. A. Louis, “Input–output maps are strongly biased towards simple outputs,” Nature Communications, vol. 9, no. 1, p. 761, 2018.DOI
- [57]J. Lai, A. Bao, J. Quinn, and W. Gilpin, “Fractal basins trap latent reasoning,” arXiv preprint arXiv:2609.04963, 2026.arxiv.org/abs/2609.04963
- [58]S. Ciliberti, O. C. Martin, and A. Wagner, “Robustness can evolve gradually in complex regulatory gene networks with varying topology,” PLoS computational biology, vol. 3, no. 2, p. e15, 2007.pubmed.ncbi.nlm.nih.gov/17274682
- [59]A. Wagner, “Robustness and evolvability in living systems,” in Robustness and evolvability in living systems, Princeton university press, 2013.DOI
- [60]W. Ma, A. Trusina, H. El-Samad, W. A. Lim, and C. Tang, “Defining network topologies that can achieve biochemical adaptation,” Cell, vol. 138, no. 4, pp. 760–773, 2009.
- [61]H. de Jong, “Modeling and simulation of genetic regulatory systems: a literature review,” Journal of Computational Biology, vol. 9, no. 1, pp. 67–103, 2002.
- [62]L. McGough, H. Casademunt, M. Nikolić, Z. Aridor, M. D. Petkova, T. Gregor, and W. Bialek, “Finding the last bits of positional information,” PRX Life, vol. 2, no. 1, p. 013016, 2024.DOI
- [63]J. Nocedal and S. J. Wright, Numerical Optimization, New York: Springer, 2nd ed., 2006.

