betaFeedback

WorksComputer Science

Bayesian Optimal Experimental Design for Constitutive Model Calibration

Abstract

Computational simulation is increasingly relied upon for high-consequence engineering decisions, and a foundational element to solid mechanics simulations, such as finite element analysis (FEA), is a credible constitutive or material model. Calibration of these complex models is an essential step; however, the selection, calibration and validation of material models is often a discrete, multi-stage process that is decoupled from material characterization activities, which means the data collected does not always align with the data that is needed. To address this issue, an integrated workflow for delivering an enhanced characterization and calibration procedure (Interlaced Characterization and Calibration (ICC)) is introduced. This framework leverages Bayesian optimal experimental design (BOED) to select the optimal load path for a cruciform specimen in order to collect the most informative data for model calibration. The critical first piece of algorithm development is to demonstrate the active experimental design for a fast model with simulated data. For this demonstration, a material point simulator that models a plane stress elastoplastic material subject to bi-axial loading was chosen. The ICC framework is demonstrated on two exemplar problems in which BOED is used to determine which load step to take, e.g., in which direction to increment the strain, at each iteration of the characterization and calibration cycle. Calibration results from data obtained by adaptively selecting the load path within the ICC algorithm are compared to results from data generated under two naive static load paths that were chosen a priori based on human intuition. In these exemplar problems, data generated in an adaptive setting resulted in calibrated model parameters with reduced measures of uncertainty compared to the static settings.

This article has been authored by an employee of National Technology & Engineering Solutions of Sandia, LLC under Contract No. DE-NA0003525 with the U.S. Department of Energy (DOE). The employee owns all right, title and interest in and to the article and is solely responsible for its contents. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this article or allow others to do so, for United States Government purposes. The DOE will provide public access to these results of federally sponsored research at the DOE Public Access Plan https://www.energy.gov/downloads/doe-public-access-plan.

Introduction

Material constitutive models—mathematical representations of the complex mechanical behavior of materials under various loading conditions—are critical to solid mechanics simulations like finite element analysis (FEA). These models contain material-dependent parameters that must be determined in order to reliably simulate material behavior. These unknown parameters are obtained by first collecting experimental data, i.e., material characterization, and then using the data to drive an inverse problem for parameter estimation, i.e., calibration.

The process of material characterization and model calibration has evolved over the years. Tensile dog bone samples, which develop a homogeneous state of stress, allow constitutive models to be calibrated analytically. Alternative specimens and test methods, such as notched tension, shear, torsion, interrupted or reversed loading, etc., can be employed to access other stress states, but require advanced calibration methods such as finite element model updating (FEMU) [1]. Additionally, Digital Image Correlation (DIC) delivers full-field kinematic data (e.g., strain on the surface of a test specimen [2]) and can be used with FEMU and other advanced calibration methods like the Virtual Fields Method (VFM) [1, 3–10].

More often than not, material characterization and model calibration are performed deterministically, which provides a single set of optimal parameters. Alternatively, a stochastic approach to model calibration, such as Bayesian inference [11], produces a probability distribution over the unknown model parameters. Under this paradigm, posterior summary metrics such as parameter expected values, variances and credible intervals are useful in characterizing parameter uncertainty, which can then be propagated to the model output. A description of uncertainty is valuable in engineering settings as it can be used to make confident design decisions, especially in high-consequence scenarios. There is a growing body of work in fields of science and engineering that use Bayesian inference for model calibration [12–20].

Despite these advances in material characterization and model calibration, the two processes are typically decoupled from each other, with each being performed sequentially. Only after a separate validation effort is performed can calibration results be evaluated. If the results are unsatisfactory, a new model must be selected, potentially necessitating another experimental campaign [10, 21–29], which is a time-consuming and costly process.

The expense of this decoupled approach to characterization and calibration may be alleviated with a smart selection, or design, of experiments. The overarching goal when selecting an experimental design is to maximize the information content in the data with respect to the goals of the experiment. If the goal is to estimate model parameters (as in this work), then information content of the data may be measured by the resulting uncertainty of the inferred model parameters.

The experimental design may be constructed under several different settings. A static experimental design is one such that the characterization experiments are determined prior to seeing any data and that does not adapt as data is collected. At the most fundamental level, human intuition is used to determine the experimental design ahead of data collection, which is the approach traditionally taken and is prone to result in data not optimal for the model calibration problem at hand. Static designs may also be obtained under more structured guidelines. The full factorial design first introduced by Fisher [30] constructs a design matrix containing all possible combinations of discrete design settings. Although this design approach is comprehensive, the cost of executing the design matrix grows exponentially as the number of design variables and possible values they can assume increases. A reduced design matrix may be obtained with a fractional factorial design [31], which uses a selective subset from the full factorial design matrix while limiting the loss in critical information. These combinatorial approaches are in general not feasible for calibration of constitutive models due to the number of design settings involved and the time and cost requirements associated with running each experiment. Thus, when the experimental design is constructed in a static manner, it is susceptible to result in the inefficient use of limited resources and/or result in sub-optimal data for calibration.

In lieu of determining the experimental design matrix up front, an adaptive design—which re-evaluates the design periodically based on the observed data thus far—is appealing for engineering applications in order to maximize the information content in the data while making the best use of limited resources.

Bayesian optimal experimental design (BOED), first introduced by Lindley [32], provides a principled way to adaptively choose an experimental design under uncertainty as it is based on Bayesian inference, which naturally incorporates any a priori knowledge of the parameters into the decision-making process as well as uncertainties in the data and model form and any numerical error.2 An optimal design of experiments is one that provides the most informative data, and in an ideal scenario, only data that is both necessary and sufficient (i.e., all the data and only the data that is needed) for calibration is collected. In this work, BOED is the paradigm used to adaptively determine the optimal experimental design.

There are several different ways an optimal experimental design (OED) can be specified for constitutive model calibration. Two contrasting approaches involve either determining the optimal load path for a specimen of a given geometry [33, 34] (the focus of this work), or optimizing the specimen geometry for a given load path [35–37]. Both objectives of optimizing the load path and specimen geometry were explored in [38]. Additionally, OED is often used for sensor placement when there are limited physical sensors (such as accelerometers, strain gauges or thermocouples) [39, 40]. All approaches, however, have the same goal of determining the OED for the purposes of the experiment.

This work utilizes adaptive BOED in a new Interlaced Characterization and Calibration (ICC) framework for material characterization and constitutive model calibration that interlaces the two processes in a quasi real-time feedback loop. In this adaptive experimental design framework, the load path placed on a specimen is determined in situ with model calibration in order to leverage information from previous load steps when determining future load steps. To the authors’ knowledge, this is the first realization of ICC for load path optimization of a specimen. With the ICC approach, both the outcome (i.e., parameter uncertainty) as well as the process (i.e., time and monetary efficiency) of characterization and calibration are improved.

The remainder of this paper is organized as follows: First, the components of the characterization and calibration framework are presented (§ 2), and then a background of the constitutive model, statistical methods, experimental design tools and other details of the framework are described in detail (§ 3). Next, the framework is demonstrated on two exemplar problems: The first exemplar focuses on yield anisotropy (§ 4), while the second exemplar additionally considers isotropic hardening (§ 5). In summary, this work addresses issues with the current decoupled approach to material characterization and model calibration by interlacing the two processes, resulting in improved calibration with reduced uncertainty.

Framework

The overarching goal of the ICC framework is to optimize the load path a specimen of a given geometry is subjected to in order to collect the most informative data for model calibration. In the ICC framework, an initial load is placed on the specimen, experimental data is collected and Bayesian calibration is performed to estimate a probability distribution of the unknown model parameters. Then, given a set of possible next load steps, BOED is used to determine which one to take using the expected information gain (EIG) as a measure of how informative each step is predicted to be. The experiment is actively driven through the load path with the highest EIG, and the characterization-calibration process is repeated for a predetermined number of load steps, which is just one of many possible ending criteria.

The ICC framework could be applied to numerous experimental configurations, but the target in this work is a cruciform specimen in a bi-axial load frame. As a starting point, a computationally cheap material point simulator (MPS), which represents the center of a cruciform specimen, was used for framework development and testing. The process is illustrated in Figure 1 for the MPS. Starting with plot (a) of Figure 1, the material point is driven along a load path by applying a strain increment along either of the two axes shown while the other axis is held fixed (displacement = 0), and data is collected; in (b), Bayesian inference is performed, which updates prior knowledge of the unknown parameters by conditioning on the available data to obtain a posterior understanding of the parameters; in (c), BOED is used to determine the load step that will be the most informative for calibration. The process continues in a feedback loop for a given number of load steps. Prior to entering the feedback loop, the load path space is reduced to a finite load path tree in (d), and surrogate models are constructed to replace the MPS, which is discussed in detail in § 3.5. Since the material under consideration in this work is an aluminum alloy, the load path tree also provides a way for path-dependence to be incorporated into the surrogates, which is important when considering plasticity. The implementation of the ICC framework utilizing the methods described in § 3 is detailed in Algorithm 1 in Appendix § 8.

Image not extracted

Figure 1. The ICC framework for a material point under bi-axial loading. An initial load step is taken by applying a strain increment along one of the two axes while keeping the other axis fixed (a). Given the collected data from (a), prior knowledge about the parameters θ\theta is updated via Bayesian inference (b). Given the two possible next load steps (applying a strain increment on either of the two axes), BOED is used to determined the next step (c). Prior to entering the feedback loop, the load path space is reduced to a finite load path tree, and surrogate models are constructed to replace the material point simulator (d).

The ICC approach has similar goals as the experimental design protocol proposed by Villarreal et al. [33], wherein a reinforcement learning algorithm was used for the design of experiments in order to calibrate a history-dependent constitutive model. A key difference is that Villarreal et al. leveraged a pre-trained policy to guide the experimental actions, whereas the ICC framework continuously updates the load path in situ during the experiment. Additionally, Villarreal et al. used a Kalman filter to obtain an estimate of the information gain corresponding to each experimental action, as opposed to the EIG used in the current work. Thus, the tools used within the ICC approach for experimental design are fundamentally different.

Methods

In this section, the different components that make up the framework are described in detail. First, the MPS and elastoplastic material model used in this work—Hill48 anisotropic yield criterion with two different isotropic hardening functions—are detailed along with an introduction to the two exemplar problems (§ 3.1). Next, Bayesian inference for material model calibration (§ 3.2) as well as the theory of BOED and the EIG calculation (§ 3.3) are presented. The load path tree that reduces the infinite-dimensional space of possible load paths to a manageable number is described (§ 3.5). Finally, the surrogate models that are constructed at each node of the load path tree are discussed (§ 3.6).

Material Point Simulator

To begin, a description of the constitutive model to be calibrated is presented. In the case of the MPS, it is assumed that the material point is subjected to a known total strain history in two directions, ε11\varepsilon_{11} and ε22\varepsilon_{22}, under a plane stress assumption. The model is taken to be temperature and rate independent, leaving the out-of-plane strain ε33\varepsilon_{33} and the stresses in the in-plane directions, σ11\sigma_{11} and σ22\sigma_{22}, as the unknowns to be calculated, along with any internal state variables.

The material response is taken to be elastoplastic with isotropic hardening and anisotropic yield with the constitutive response given via

σij=Cijkl(εkl−εklp),(1)\sigma_{ij}=C_{ijkl}\left(\varepsilon_{kl}-\varepsilon^{p}_{kl}\right), \tag*{(1)}

where CijklC_{ijkl} and εklp\varepsilon^{p}_{kl} are the elastic stiffness tensor (assumed isotropic) and plastic strain, respectively. To describe the plastic response, the yield function gg is introduced as

g=ϕ(σij)−σˉ(κ),(2)g=\phi\left(\sigma_{ij}\right)-\bar{\sigma}\left(\kappa\right), \tag*{(2)}

where ϕ\phi and σˉ\bar{\sigma} are the effective and flow stresses describing the shape and size of the yield surface, respectively. The flow stress is a function of the isotropic hardening variable κ\kappa. If g<0g<0, the material response is elastic while the condition g=0g=0 indicates inelastic plastic deformation.

For the hardening behavior, a combined linear and Voce type expression is used such that

σˉ(κ)=σy+hˉκ+A(1−exp⁡(−nκ)),(3)\bar{\sigma}\left(\kappa\right)=\sigma_y+\bar{h}\kappa+A\left(1-\exp\left(-n\kappa\right)\right), \tag*{(3)}

where σy\sigma_y, hˉ\bar{h}, AA and nn are the constant initial yield stress, linear hardening modulus, exponential modulus and the exponential fitting coefficient, respectively.

With respect to the shape of the yield surface, Hill’s effective stress [41] is utilized. This expression, however, is further simplified by two assumptions: the material is in a state of plane stress, and a purely bi-axial stress-state is taken to result from loading, leaving the remaining shear stresses as zero. These assumptions produce only two non-zero stresses, enabling the Hill effective stress to be reduced to

ϕ2(σij)=F(σ22)2+G(σ11)2+H(σ11−σ22)2.(4)\phi^{2}\left(\sigma_{ij}\right)=F\left(\sigma_{22}\right)^{2}+G\left(\sigma_{11}\right)^{2}+H\left(\sigma_{11}-\sigma_{22}\right)^{2}. \tag*{(4)}

Lastly, using an associative approach, the flow rule is given by

ε˙γγp=κ˙∂ϕ∂σγγ,γ=1,2ε˙33p=−(ε˙11p+ε˙22p),(5)\begin{aligned} \dot{\varepsilon}^{p}_{\gamma\gamma}&=\dot{\kappa}\frac{\partial\phi}{\partial\sigma_{\gamma\gamma}},\qquad\gamma=1,2 \\ \dot{\varepsilon}^{p}_{33}&=-\left(\dot{\varepsilon}^{p}_{11}+\dot{\varepsilon}^{p}_{22}\right), \tag*{(5)} \end{aligned}

with plasticity incompressibility (ε˙kkp=0\dot{\varepsilon}^{p}_{kk}=0) being used to arrive at the expression for the out-of-plane plastic strain.3

With a prescribed in-plane strain history, the in-plane stresses, out-of-plane strain, isotropic hardening variable and plastic strains all need to be determined, leading to a solution variable set of

x:=[σ11,σ22,ε33,κ,ε11p,ε22p,ε33p].x:=\left[\sigma_{11},\sigma_{22},\varepsilon_{33},\kappa,\varepsilon^{p}_{11},\varepsilon^{p}_{22},\varepsilon^{p}_{33}\right].

The updated state may be found using a classical return mapping algorithm [42] to solve Hooke’s Law, yield consistency condition and plastic flow rule. Details of that procedure are left to the cited reference.

Exemplar Problems. In this section, two exemplar problems that explore different combinations of material phenomenology are introduced, both of which use the MPS as the forward model, and their results are contained in § 4 and § 5. In Exemplar 1, the model parameters being inferred from the data, which are referred to as unknown parameters, are the anisotropic yield parameters FF and GG from (4), and the model parameter vector is θ:=[F,G]T\boldsymbol{\theta}:=[F,G]^{T} with all other parameters held fixed at the values presented in Table 1. In Exemplar 2, the initial yield stress σy\sigma_y as well as the isotropic hardening variables AA and nn from (3) are also inferred so that θ:=[F,G,σy,A,n]T\boldsymbol{\theta}:=[F,G,\sigma_y,A,n]^{T}.

ParameterExemplar 1Exemplar 2Bounds
EE70 GPa70 GPa–
ν\nu0.30.3–
FFUnknownUnknown[0.3, 0.7]
GGUnknownUnknown[0.3, 0.7]
HH0.50.5–
σy\sigma_y200 MPaUnknown[50, 500] MPa
hˉ\bar{h}200 MPa0 MPa–
AA200 MPaUnknown[10, 400] MPa
nn20Unknown[1 × 10−610^{-6}, 100]

Table 1. Parameters in Exemplars 1 and 2.

These two exemplars were selected to study the ability of the ICC algorithm to effectively drive calibration under different material responses. Exemplar 1 focuses solely on the anisotropy of the yield surface in order to observe the impact of direction dependence as probed by the bi-axial load frame; Exemplar 2 introduces plastic hardening into the calibration problem. Thus, two distinct material responses may be found with potentially competing calibration needs that pose a challenge for the ICC algorithm. As such, both responses are considered in this work.

Parameter Cases. The data used for inference was simulated with the MPS using an assumed true parameter vector θtrue\boldsymbol{\theta}^{true} with added Gaussian noise. In Exemplar 1, calibration was performed for four different parameter cases, each having a unique θtrue\boldsymbol{\theta}^{true} that is detailed in Table 2. These cases were chosen to evaluate the sensitivity of the load path selection algorithm to the level of yield anisotropy in the model. The four cases are labeled according to the FF and GG values, where F=G=0.5F=G=0.5 represents an isotropic (J2) yield surface. Also included in the table are the corresponding values of the yield multiplier parameters, R11=(G+H)−1R_{11}=(G+H)^{-1} and R22=(F+H)−1R_{22}=(F+H)^{-1}. 4 Parameter HH was held fixed at 0.5 in all four cases. Parameters FF and GG were chosen to have 1) values of equal and opposite distance from 0.5, 2) one value equal to 0.5, 3) both values greater than 0.5 and 4) one value above and one value below 0.5 with unequal distances away from 0.5. These selections are meant to test algorithmic performance given different parametric and response contributions.

Case No.Case DescriptionFFGGR11R_{11}R22R_{22}R33R_{33}
1Equal and Opposite0.550.451.050.951.00
21 Unchanged0.600.501.000.910.91
3Both > 0.50.600.600.910.910.82
4Strong Anisotropy0.690.431.070.840.89

Table 2. Four different parameter cases studied in Exemplar 1 with varying levels of yield anisotropy.

Two parameter cases were studied in Exemplar 2, and the θtrue\boldsymbol{\theta}^{true} used for data generation in each case is detailed in Table 3. In Case 5, a stronger emphasis was placed on the initial yield stress by setting σy\sigma_y to a value three times that of the exponential modulus AA, which describes the Voce component of hardening. In contrast, Case 6 put a stronger emphasis on hardening by setting AA to a value three times that of σy\sigma_y. The linear component of hardening was removed in this second exemplar by setting hˉ=0\bar{h}=0 so that the effects of the trade-off in the components contributing to the total stress (yield and hardening) could be observed.

Case No.EmphasisFFGGσy\sigma_y (MPa)AA (MPa)nn
5Yield0.550.4530010020
6Hardening0.550.4510030020

Table 3. Two different parameter cases studied in Exemplar 2, which emphasize the initial yield or Voce component of the hardening behavior.

Bayesian Inference

Model calibration is performed within the Bayesian paradigm for inference, which provides estimates of parameter uncertainty; the reader is referred to Gelman et al. [11] for a thorough introduction to Bayesian inference. The goal in Bayesian inference is to define and update the knowledge of unknown (or uncertain) quantities by conditioning on available information. In the context of model calibration, the unknown quantities are the forward model parameters θ∈Θ⊂RD\boldsymbol{\theta}\in\Theta\subset\mathbb{R}^{D} (see Table 1), where DD is the dimensionality of the parameter space Θ\Theta.

Before any data are observed, the uncertainty about the model parameters is modeled through a probability density function (PDF), which is the prior distribution (or "prior" for short) with probability density π(θ)\pi(\boldsymbol{\theta}); prior belief about the parameter ranges and regions of highest probability can be incorporated in this modeling step.

Data for the calibration of constitutive models often takes the form of one-dimensional force-extension curves or field data (e.g., displacement or strain on the surface of a test specimen). The data being used for calibration in this work in the context of the MPS are the in-plane stresses. Once this data y∈Y⊂RNobs\boldsymbol{y}\in\mathcal{Y}\subset\mathbb{R}^{N_{\mathrm{obs}}}, where NobsN_{\mathrm{obs}} (number of observations) is the dimensionality of the data space Y\mathcal{Y}, becomes available, the prior is updated to a PDF known as the posterior distribution (or "posterior") π(θ∣y)\pi(\boldsymbol{\theta}\mid\boldsymbol{y}) by conditioning on the available information. The probability distribution over the data is f(y∣θ)f(\boldsymbol{y}\mid\boldsymbol{\theta}), and for fixed y\boldsymbol{y}, it is considered as a function of θ\boldsymbol{\theta} and is called the likelihood function, which quantifies the likelihood of observing the data y\boldsymbol{y} given parameters θ\boldsymbol{\theta}. The posterior is obtained via Bayes’ rule, which is derived from the definition of conditional probability,

π(θ∣y)=f(y∣θ)π(θ)∫Θf(y∣θ)π(θ) dθ∝f(y∣θ)π(θ).(6)\pi(\boldsymbol{\theta}\mid\boldsymbol{y})=\frac{f(\boldsymbol{y}\mid\boldsymbol{\theta})\pi(\boldsymbol{\theta})}{\int_{\Theta}f(\boldsymbol{y}\mid\boldsymbol{\theta})\pi(\boldsymbol{\theta})\,\mathrm{d}\boldsymbol{\theta}}\propto f(\boldsymbol{y}\mid\boldsymbol{\theta})\pi(\boldsymbol{\theta}). \tag*{(6)}

The term in the denominator of the posterior is called the model evidence (also known as the marginal likelihood). Since θ\boldsymbol{\theta} is integrated out ∫Θf(y∣θ)π(θ)dθ\int_{\Theta}f(\boldsymbol{y}\mid\boldsymbol{\theta})\pi(\boldsymbol{\theta})\mathrm{d}\boldsymbol{\theta}, this term is only dependent on the observed data, and for fixed y\boldsymbol{y}, it is considered a constant. Therefore, the posterior can be written as being proportional to the product of the likelihood and prior as in (6). This form is convenient since direct computation of the marginal likelihood is in general not possible.

An important consideration in Bayesian inference is how to recover uncertainty information when posterior functionals and summaries such as the posterior means and probabilities are not analytically tractable. While the use of numerical integration may be a valid option to obtain these quantities for a parameter space Θ\Theta of low dimension (i.e., D≤4D\leq4) [13], its use may lead to large numerical errors in higher dimensions. Thus, the use of alternative methods for approximating these quantities is needed.

A commonly used method for obtaining posterior summaries is Markov Chain Monte Carlo (MCMC) simulation [44–48]. The high accuracy of an MCMC sampling-based approach is achieved at a high computational cost and is too expensive for the quasi real-time characterization and calibration setting in which the ICC framework will eventually operate (additional comments in Appendix A.1). Therefore, a highly efficient, non-sampling technique is better suited to recover posterior uncertainties in the ICC framework. One such method is Laplace’s approximation, which makes use of the second-order Taylor series expansion of the posterior centered about the maximum a posteriori (MAP) probability estimate, providing a Gaussian approximation of the posterior. The MAP estimate is obtained by finding the θ\theta that maximizes the log of the posterior, or equivalently, minimizes its negative. Define J(θ):=−log⁡π(θ∣y)J(\theta) := -\log\pi(\theta\mid\mathbf{y}), then the MAP estimate is

θ^=arg⁡min⁡θ∈Θ J(θ).(7)\hat{\theta} = \underset{\theta\in\Theta}{\arg\min}\, J(\theta). \tag*{(7)}

The Laplace approximation for the posterior is a normal distribution centered at θ^\hat{\theta} with a covariance matrix determined by the inverse of the Hessian of J(θ)J(\theta) at the MAP estimate, ΣL=H(θ^)−1\Sigma^{L} = H(\hat{\theta})^{-1} [49]. The approximation of the posterior is written as

π~L(θ∣y)=ND(θ^,ΣL)≈π(θ∣y).(8)\tilde{\pi}^{L}(\theta\mid\mathbf{y}) = \mathcal{N}_{D}(\hat{\theta}, \Sigma^{L}) \approx\pi(\theta\mid\mathbf{y}). \tag*{(8)}

The most straightforward approach to computing the Hessian is through a second-order finite difference approximation about J(θ^)J(\hat{\theta}). A primary benefit of using the Laplace approximation for the posterior is the low computational cost compared to MCMC simulation. After the ICC algorithm has completed and all the data has been collected, an MCMC simulation may optionally be used to obtain an approximation to the posterior given all the data—although, this step was not performed in the presented work.

Bayesian Optimal Experimental Design

The following section discusses BOED modeling steps and considerations within the scope of this work; the reader is referred to Ryan [50] for a comprehensive introduction and review of BOED algorithms and concepts. Additionally, the prior works of Long et al. [51], Beck et al. [52] and Huan & Marzouk [53] detail theoretical frameworks in line with what is presented here.

The first step of BOED is to define some metric by which to measure the information content of an experimental design. Define ξ\xi as a single experiment and ξ\boldsymbol{\xi} as the experimental design that may be comprised of one or more experiments. Given a design belonging to the space of all possible designs ξ∈Ξ\boldsymbol{\xi} \in\Xi, experimental data y\mathbf{y} and model parameters θ\boldsymbol{\theta}, a utility function U(ξ,y,θ)U(\boldsymbol{\xi}, \mathbf{y}, \boldsymbol{\theta}) is used to quantify the relationship between the design and the resulting information that is obtained from it. Since the outcome y\mathbf{y} of performing the experiment(s) in design ξ\boldsymbol{\xi} is unknown prior to performing the experiment(s), in practice, the expectation of the utility over the marginal distribution of all possible outcomes and the parameters Ey,θ∣ξ[U(ξ,y,θ)]\mathbb{E}_{\mathbf{y},\boldsymbol{\theta} \mid\boldsymbol{\xi}}[U(\boldsymbol{\xi}, \mathbf{y}, \boldsymbol{\theta})] is computed. The optimal design ξ∗\boldsymbol{\xi}^{*} is the one that maximizes the expected utility,

ξ∗=arg⁡max⁡ξ∈Ξ Ey,θ∣ξ[U(ξ,y,θ)].(9)\boldsymbol{\xi}^{*} = \underset{\boldsymbol{\xi} \in\Xi}{\arg\max}\,\mathbb{E}_{\mathbf{y},\boldsymbol{\theta} \mid\boldsymbol{\xi}}[U(\boldsymbol{\xi}, \mathbf{y}, \boldsymbol{\theta})]. \tag*{(9)}

There are a number of ways to formulate the utility function, and the decision should reflect the overarching goal of the experiment and inference. A thorough description of various utility functions is contained in [54]. A common choice for the expected utility function (and the one used in this work) when the interest is in maximizing the information content in the data with respect to parameter uncertainty is the expected Kullback-Leibler (KL) divergence. The KL divergence provides a measure of the statistical distance between one probability distribution q(θ)q(\boldsymbol{\theta}) and a second reference probability distribution p(θ)p(\boldsymbol{\theta}),

DKL(p(θ)∥q(θ))=∫Θp(θ)log⁡(p(θ)q(θ)) dθ.(10)D_{\mathrm{KL}}(p(\boldsymbol{\theta}) \mathbin{\|} q(\boldsymbol{\theta})) = \int_{\Theta} p(\boldsymbol{\theta}) \log\left(\frac{p(\boldsymbol{\theta})}{q(\boldsymbol{\theta})}\right) \,\mathrm{d}\boldsymbol{\theta}. \tag*{(10)}

In the Bayesian setting, the two distributions of interest are the prior π(θ)\pi(\boldsymbol{\theta}) and the posterior π(θ∣y,ξ)\pi(\boldsymbol{\theta} \mid\mathbf{y}, \boldsymbol{\xi}) after performing the experiment(s) in ξ\boldsymbol{\xi} and observing y\mathbf{y}. The optimal design is the one that has the greatest expected KL divergence, which means it results in a posterior distribution that has the greatest statistical distance from the prior and therefore contains the most information about the parameters. The expected

Bayesian Optimal Experimental Design

KL divergence is also commonly referred to as expected information gain (EIG).5 Replacing p(θ)p(\theta) and q(θ)q(\theta) in (10) with π(θ∣y,ξ)\pi(\theta\mid\mathbf{y}, \xi) and π(θ)\pi(\theta), respectively, and taking the expectation over the marginal distribution of all possible outcomes, the EIG is written as

EIG⁡(ξ)=Ey∣ξ[DKL(π(θ∣y,ξ) ∥ π(θ))](11)\operatorname{EIG}(\xi) = \mathbb{E}_{\mathbf{y} \mid\xi}\left[D_{KL}\left(\pi(\theta\mid\mathbf{y},\xi) \,\|\, \pi(\theta)\right)\right] \tag*{(11)}
=∫Y∫Θπ(θ∣y,ξ)f(y∣ξ)[log⁡π(θ∣y,ξ)−log⁡π(θ)]dθdy(12)= \int_{\mathcal{Y}}\int_{\Theta}\pi(\theta\mid\mathbf{y},\xi)f(\mathbf{y} \mid\xi)\left[\log\pi(\theta\mid\mathbf{y},\xi)-\log\pi(\theta)\right]\mathrm{d}\theta\mathrm{d}\mathbf{y} \tag*{(12)}
=∫Y∫Θf(y∣θ,ξ)π(θ)[log⁡f(y∣θ,ξ)−log⁡f(y∣ξ)]dθdy(13)= \int_{\mathcal{Y}}\int_{\Theta}f(\mathbf{y} \mid\theta,\xi)\pi(\theta)\left[\log f(\mathbf{y} \mid\theta,\xi)-\log f(\mathbf{y} \mid\xi)\right]\mathrm{d}\theta\mathrm{d}\mathbf{y} \tag*{(13)}
=Eθ[DKL(f(y∣θ,ξ) ∥ f(y∣ξ))],(14)= \mathbb{E}_{\theta}\left[D_{KL}\left(f(\mathbf{y} \mid\theta,\xi) \,\|\, f(\mathbf{y} \mid\xi)\right)\right], \tag*{(14)}

where f(y∣ξ)f(\mathbf{y} \mid\xi) is the evidence.

Using Bayes’ rule (6), the EIG can equivalently be written as the expectation of the KL divergence between the posterior and prior (11a) & (11b) or between the likelihood and evidence (11c) & (11d). In both cases, numerical methods are needed in order to obtain an estimate of the EIG since typically neither formulation can be expressed in a closed form.

In this work, a nested Monte Carlo estimator is used to obtain an approximation of the EIG using the form found in (11c) since it is straightforward to obtain samples from the prior and data distributions,

EIG⁡^MC(ξ)=1N∑n=1Nlog⁡f(yn∣θn,0,ξ)1M∑m=1Mf(yn∣θn,m,ξ),θn,m∼π(θ),yn∼f(y∣θn,0,ξ).(15)\widehat{\operatorname{EIG}}_{\mathrm{MC}}(\xi) = \frac{1}{N}\sum_{n=1}^{N}\log\frac{f(\mathbf{y}_n \mid\theta_{n,0},\xi)}{\frac{1}{M}\sum_{m=1}^{M}f(\mathbf{y}_n \mid\theta_{n,m},\xi)}, \qquad\theta_{n,m}\sim\pi(\theta), \quad\mathbf{y}_n\sim f(\mathbf{y}\mid\theta_{n,0},\xi). \tag*{(15)}

The approximation in (12) uses samples from the data distribution and prior to evaluate the inner and outer sum of the Monte Carlo estimator. The quality of the estimate is largely controlled by NN and MM, the number of samples used to evaluate the outer and inner sums, which control the estimator variance and bias, respectively [55]. Additional comments on the nested MC estimator can be found in Appendix A.2.

Recall from Sec. 1 that BOED is especially appealing in an adaptive setting in which previously collected data is incorporated into the design selection process.6 In an adaptive setting, the EIG for the ttht^{\mathrm{th}} experiment in ξ\xi, EIG⁡(ξt)\operatorname{EIG}(\xi_t), t=1,…,Tt=1,\ldots,T, (T is the total number of experiments in ξ\xi) is computed based on the the data collected from the past t−1t-1 experiments. More details of the adaptive BOED approach can be found in Appendix A.3

When the EIG is calculated for only the next design point ξt\xi_t without considering all subsequent design points ξt+1:T\xi_{t+1:T}, it is known as a myopic (or greedy) method, where a locally optimal decision is made at each step of the algorithm. While locally optimal decisions may result in the globally optimal experimental design ξ\xi (or, in this application, load path from beginning to end), this outcome is not guaranteed. At a minimum, the myopic algorithm is expected to yield improvement over a static design. The decision to use this myopic decision-making process was one of practicality, as the ICC framework will eventually be used in a quasi real-time setting. Since the number of design points grows exponentially with each subsequent load step (illustrated in Figure 2 in § 3.5), so would the computational cost of calculating the EIG for all paths to the leaves of the tree. In addition to this, in reality, it is difficult to hold a sample in an exact position for an indefinite amount of time while these computations are taking place. For these reasons, a myopic approach is utilized.

A simplified finite load-path tree with four possible load paths

Figure 2. A simplified load path tree showing four possible load paths (A-D), each consisting of two load steps.

In summary, an adaptive, myopic BOED algorithm is used within the ICC framework to optimize the load path of a specimen in order to gain the most informative data for calibration. This section is concluded with a summary of the notation used for the experiments in this work: Each load step applied to a specimen corresponds to a single experiment ξt\xi_t, and an experimental design consists of the full load path a specimen takes ξ={ξt}t=1T\boldsymbol{\xi} = \{\xi_t\}_{t=1}^{T}, where TT is the number of load steps in the load path. At load step tt, data yt\boldsymbol{y}_t is collected. The optimal experimental design in this context describes the optimal load path design.

Finite Load Path Tree

The finite load path tree that reduces the infinite-dimensional load path space of material deformation to a discrete space is introduced in this section. The graph structure in this network, depicted in Figure 2, is that of a binary tree, as each parent node has two children. This graph stands in for a collection of load steps that involve bi-axial loading of a specimen. The branches could take many forms, but for now, the load path tree is restricted to two children per node, which correspond to applying a strain increment along either the x1x_1 or x2x_2 direction (ε11\varepsilon_{11} or ε22\varepsilon_{22}) of the material point while holding the alternate axis fixed. Other, more complex, branch options may include both positive and negative strains along the x1x_1 and x2x_2 directions (i.e., node children ε11\varepsilon_{11}, ε22\varepsilon_{22}, −ε11-\varepsilon_{11} and −ε22-\varepsilon_{22}), as well as combinations of strains applied in both directions simultaneously (i.e., node children ε11ε22\varepsilon_{11}\varepsilon_{22}, −(ε11ε22)-(\varepsilon_{11}\varepsilon_{22}), −ε11ε22-\varepsilon_{11}\varepsilon_{22} and −ε22ε11-\varepsilon_{22}\varepsilon_{11}) etc. The selection of two children per node is a simplified starting point and is not necessarily guaranteed to be the optimal tree structure.

As the target application is an aluminum alloy that can fail around 0.15 mm/mm strain (depending on the material orientation) [56], the total strain imposed on the material point throughout the duration of the experiment (from the undeformed state to the end of the final load step) was chosen to be within the plastic regime of deformation away from failure. The total strain was segmented equally across load steps, with the number of steps chosen to keep the computational expense of the EIG (15) reasonable (in combination with the selection of 2 children per node). In Exemplar 1, the total strain was ε=0.05\varepsilon= 0.05 mm/mm, segmented into 5 load steps yielding Δε=0.01\Delta\varepsilon= 0.01 mm/mm strain per load step. In Exemplar 2, a total strain of ε=0.14\varepsilon= 0.14 mm/mm was split into 7 load steps with Δε=0.02\Delta\varepsilon= 0.02 mm/mm strain per load step. In both exemplars, there were 100 simulated pseudotime increments per load step. In Exemplar 1, QoIs σ11\sigma_{11} and σ22\sigma_{22} were stored at the end of each load step (corresponding to each node) and in Exemplar 2, at 3 equally-spaced strain increments between nodes. The strain increment was increased in Exemplar 2 in order to have sufficient data in the plastic region for calibrating the hardening parameters, and the number of measurement points per load step was increased to 3 in order to ensure there was enough data to mitigate issues with identifiability of the parameters. Note that optimizing the strain increment (as well as the form of the branches) is a subject of future work.

The BOED workflow begins at the root node, where an initial deformation direction may either be chosen by the analyst or the algorithm. If the EIG (15) is used to initiate the design, the EIG estimate is based entirely on the prior modeling and the forward model since there is not yet any data available. Therefore, the estimate is only as good as these two components are. Poor prior modeling and, likewise, a forward model that suffers from a high level of model-form error may lead to a poor EIG estimate. Alternatively, engineering judgement may be used to determine the first step. Both options were explored in this work: The initial step was determined ahead of time in Exemplar 1 and via the EIG estimate in Exemplar 2.

As an example, if the analyst chooses to initially deform the specimen by a set amount of strain along the left branch (ξ1=ε11\xi_1 = \varepsilon_{11}), it would result in a load path that leads to the state represented by node 1. The experimental data y1\mathbf{y}_1 is then collected to produce the observations at load step 1, and Bayesian calibration is performed to compute the posterior distribution of the model parameters conditioned on the available data π(θ∣y1,ξ1)\pi(\theta\mid\mathbf{y}_1, \xi_1). An instance of the BOED step selection problem is then solved: Given the current information about the unknown parameters, which step is the most desirable for calibration, resulting in the greatest reduction in posterior uncertainty—the one leading to node 3 or node 4 (ξ2=ε11\xi_2 = \varepsilon_{11} or ξ2=ε22\xi_2 = \varepsilon_{22})? If one were to follow the series of load steps leading to node 4, the full load path would consist of T=2T = 2 load steps—the first leading to node 1 and the second leading to node 4—and would be written as ξ=[ξ1,ξ2]=[ε11,ε22]\boldsymbol{\xi} = [\xi_1, \xi_2] = [\varepsilon_{11}, \varepsilon_{22}]. This process continues until an exit criterion is met, such as progressing through a predetermined number of load steps TT.

To aid in the description of the load path tree, the strains and in-plane stress curves from the MPS for load paths A and B from Figure 2 are plotted in Figure 3 vs. simulation pseudotime using parameter values from Case 1 in Table 2 (Figure 3a and Figure 3b) and Case 5 in Table 3 (Figure 3c and Figure 3d). If a single measurement is taken at the end of each load step as in Exemplar 1, then there are four measured data points, which are represented by black diamonds. The measured data are the two in-plane stresses that are measured at each node—represented by open circles—in the load path tree. If multiple measurements are taken during each load step as in Exemplar 2, then there are multiple measured data points between each node with the final measurement coinciding with the node. The stress curves for the experiments represented by load path A (ξ=[ξ1,ξ2]=[ε11,ε11]\boldsymbol{\xi} = [\xi_1, \xi_2] = [\varepsilon_{11}, \varepsilon_{11}]) resemble traditional uniaxial stress-strain curves, except that here ε22\varepsilon_{22} is fixed at zero, which causes the development of the σ22\sigma_{22} stress. The reversal in the direction of applied strain for load path B (ξ=[ξ1,ξ2]=[ε11,ε22]\boldsymbol{\xi} = [\xi_1, \xi_2] = [\varepsilon_{11}, \varepsilon_{22}]) produces striking changes in the in-plane stresses, and the exponential nature of the hardening behavior is also easily observed. For clarity, the applied strains are also shown in Figure 3.

Image not extracted

(a)

Image not extracted

(b)

Image not extracted

(c)

Image not extracted

(d)

Figure 3. The left axes of the plots show the resulting in-plane stresses from following load path AA (ξ=[ε11,ε11]\xi= [\varepsilon_{11}, \varepsilon_{11}]) (left column) and BB (ξ=[ε11,ε22]\xi= [\varepsilon_{11}, \varepsilon_{22}]) (right column) in Figure 2 using parameter values from Case 1 in Table 2 (top row) and Case 5 in Table 3 (bottom row). Locations of the measured data points are shown with black diamonds and tree nodes with open circles. The right axes of the plots show the associated strain paths for each case. The stresses and strain paths are a) load path A, Case 1 in Exemplar 1, b) load path B, Case 1 in Exemplar 1, c) load path A, Case 5 in Exemplar 2, and d) load path B, Case 5 in Exemplar 2.

Surrogate Model Construction

Even with the fast MPS, the computational cost of obtaining the Monte Carlo estimate of the EIG in (15) is very expensive, requiring the model to be evaluated for a number of parameter samples on the order of 10610^{6} or greater. The need to make the ICC algorithm as efficient as possible—of paramount importance for its future application in quasi-real-time characterization experiments and calibration—is a strong motivator for the use of surrogate models to replace the forward model. Not only do surrogates allow for rapid parameter inference, they also make it possible to expeditiously evaluate the EIG in order to determine the next load step to take.

The means chosen in this work for dealing with the history dependence inherent in elastoplastic models is to remove it from consideration by building individual surrogates u~\tilde{u} for each measured QoI, which are the in-plane stresses.7 For Exemplar 1, the surrogates were built for every node in the load path tree in Figure 2, other than the root node (which represents the undeformed material point). For Exemplar 2, the surrogates were built for the measured QoIs at the specified intermediate pseudotime increments between nodes (as well as at the nodes). The surrogate models approximately map constitutive model parameters to each QoI that can, in principle, be computed from experimental measurements.

Training data for the surrogates was generated by first obtaining samples of the unknown model parameters. For this task, Halton samples [57], which consist of a deterministic sequence of prime numbers to produce a space-filling design, were used. For Exemplar 1, 200 training samples were generated within the bounds specified for FF and GG in Table 1 while keeping all other parameters fixed at the values recorded in the table for Exemplar 1. Likewise, in Exemplar 2, 500 training samples were generated within the bounds specified for FF, GG, σy\sigma_y, AA and nn, keeping all other parameters fixed at the recorded values. Each parameter sample was used as input to the MPS to produce corresponding output, which was the in-plane stresses. The resulting collection of input-output pairs constitute the training and test data for the surrogates.

The surrogates built for both exemplar problems generally exhibited low error compared to the MPS. The mean absolute percentage error (MAPE) of the surrogates was calculated from 1,000 test samples generated with a Halton sequence. The surrogates in Exemplar 1 had a MAPE ranging from 2×10−4%2 \times10^{-4}\% to 3×10−4%3 \times10^{-4}\% at each node and for each QoI and those from Exemplar 2 had a MAPE ranging from 0.01%0.01\% to .2%.2\%.

The surrogate model used in this work is a Gaussian process (GP), which can be used to describe probability distributions over functions. A GP is fully characterized by a mean function μ(x)\mu(x) and covariance function k(x,x′)k(x,x') (xx being a random variable), which describes the strength of correlation between inputs as a function of the distance between the points. In this work, an anisotropic squared exponential kernel was used. The reader is guided to [58] for a thorough introduction to GPs.

A notable benefit of using GPs is that they are a stochastic process, which means they inherently provide a measure of uncertainty of the surrogate representation of the model. The uncertainty surrounding the surrogate model may then be incorporated into the inference infrastructure. However, in this work, for the sake of simplicity, the mean of the GP is used in the inference without accounting for the added uncertainty of the surrogate.

Exemplar 1

In this section, Exemplar 1 is presented and discussed. The statistical modeling choices, data generation process and algorithmic settings within the ICC framework are discussed (§ 4.1). Results are presented (§ 4.2) with an analysis of the adaptive load path selection (§ 4.2.1) and a comparison of the parameter inference from the adaptive setting via the ICC framework vs. two naive static design settings. Comparisons are made between the posterior summaries (§ 4.2.2) and propagated uncertainty (§ 4.2.3).

Problem Setup

The first exemplar problem, introduced in § 3.1, considers the anisotropic yield parameters FF and GG in (4) as unknown, with other parameters fixed as noted in Table 1, and the parameter vector is defined as θ=[F,G]T\boldsymbol{\theta} = [F,G]^{T}. This section is used to describe the statistical modeling decisions and other algorithm settings specific to this exemplar problem.

The parameters were constrained to take on values within the bounds specified for the surrogate training data (Table 1) by choosing a truncated normal (TN) prior probability distribution that enforces this constraint,

π(θd)=TN(θd;μ(θd),δ(θd)2,a(θd),b(θd)),d=1,…,D.(16)\pi(\theta_d) = TN\left(\theta_d;\mu_{(\theta_d)},\delta^2_{(\theta_d)},a_{(\theta_d)},b_{(\theta_d)}\right), \qquad d=1,\ldots,D. \tag*{(16)}

The distribution means μ\mu were chosen to yield an expected value within the bounds, and the variances δ2\delta^2 were chosen to be moderately diffuse. The support of the distribution, defined by aa (the lower bound) and bb (the upper bound), was chosen to be the same as that used for the surrogate training. All distribution parameters are detailed in Table 4. These prior choices not only restrict the range of values the parameters can take, but also reflect the belief that the parameters are less likely to take on values exactly at the bounds.

Parameterμθ\mu_\thetaδθ2\delta^2_\thetaaabb
F0.510.30.7
G0.510.30.7

Table 4. Means μθ\mu_\theta, variances δθ2\delta^2_\theta and bounds aa and bb for the truncated normal prior model (13) used in Exemplar 1.

Next, synthetic experimental results were modeled to reflect the assumption that the observed data is equal to the model output with added Gaussian error ee,

y=u~(θ)+e,e∼NNobs(0,Ψ).(17)\mathbf{y} = \tilde{\mathbf{u}}(\boldsymbol{\theta}) + \mathbf{e}, \qquad\mathbf{e} \sim N_{N_{\mathrm{obs}}}(\mathbf{0},\mathbf{\Psi}). \tag*{(17)}

The expression in (17) may be written as a multivariate normal model centered at the surrogate replacement of the forward model evaluated at θ\boldsymbol{\theta}, u~(θ)\tilde{\mathbf{u}}(\boldsymbol{\theta}), with a covariance matrix Ψ\mathbf{\Psi} of size Nobs×NobsN_{\mathrm{obs}} \times N_{\mathrm{obs}},

f(y∣θ,ξ)=NNobs(y;u~(θ),Ψ).(18)f(\mathbf{y}\mid\boldsymbol{\theta},\boldsymbol{\xi}) = N_{N_{\mathrm{obs}}}\left(\mathbf{y};\tilde{\mathbf{u}}(\boldsymbol{\theta}),\mathbf{\Psi}\right). \tag*{(18)}

The observation noise variance is assumed to be independent and identically distributed (i.i.d.) such that Ψ=ψ2INobs\mathbf{\Psi} = \psi^2\mathbf{I}_{N_{\mathrm{obs}}}, where INobs\mathbf{I}_{N_{\mathrm{obs}}} is an Nobs×NobsN_{\mathrm{obs}}\times N_{\mathrm{obs}} identity matrix and ψ2\psi^2 is the observation noise variance—assumed to be known and fixed at ψ2=10MPa2\psi^2 = 10\mathrm{MPa}^2. The noise level was chosen to be on the same order as that seen in experimental data.

In (17) and (18), u~(θ)\tilde{\mathbf{u}}(\boldsymbol{\theta}) represents the output of the surrogate model, which is the in-plane stresses [σ11,σ22][\sigma_{11},\sigma_{22}], with θ\boldsymbol{\theta} as input. Since a single measurement is collected at the end of each load step in Exemplar 1 (Figure 3), the observations occur at strains corresponding to the nodes in the load path tree. The number of experimental observations NobsN_{\mathrm{obs}} increases with each step in the algorithm as more load steps are added to the load path. Experimental data was simulated by running the MPS at assumed true parameter values θtrue\boldsymbol{\theta}^{true} (detailed in Table 2) with added Gaussian error according to (17) for a specified load path ξ\boldsymbol{\xi}.

Moving on to algorithmic settings for the ICC framework and calculation of the EIG (15), in this first exemplar problem, MM and NN were chosen to be 1×1021 \times10^2 and 1×1041 \times10^4, respectively, and the initial load step before any data was collected was set to be ξ1=ϵ11\boldsymbol{\xi}_1 = \boldsymbol{\epsilon}_{11}. The algorithm was carried out to a total of T=5T=5 load steps, with each load step applying a 0.01 mm/mm strain increment, so that the total strain was 0.05 mm/mm at the end of step 5. After the initial load step, the EIG was used within the BOED setup described in Section 3.3 to determine all subsequent load steps in the load path [ξ2,ξ3,ξ4,ξ5][\boldsymbol{\xi}_2,\boldsymbol{\xi}_3,\boldsymbol{\xi}_4,\boldsymbol{\xi}_5].

Results

The ICC algorithm was run for the four different parameter cases detailed in Table 2. For all four cases, calibration was performed in an adaptive design setting in which the optimal load path was chosen within the ICC framework as well as in two static design settings chosen based on human intuition a priori.8 The two static cases include one in which the load path was ε11\varepsilon_{11} for each step in the algorithm and, alternatively, one in which the path was ε22\varepsilon_{22} for each step. In total, 12 different combinations (4 parameter cases run under 3 design settings) were studied in Exemplar 1. Since there is randomness present in the data generation process as well as in the EIG estimation, 100 repeat trials were performed for each of the 12 combinations with different instances of noisy data in each trial. Repetition served to make meaningful comparisons in the parameter inference between the adaptive and static designs in an average sense. The repeat trials also serve to establish further confidence in the chosen MM and NN values for the EIG estimate in (15). Load paths that result in similar parameter inference over many trials indicate the bias and variance of the estimator are sufficiently small and that the optimal load path is in fact being chosen given the data. Since the data varies from trial to trial due to different instances of the random noise, there may be more than one optimal load path chosen among the different trials. Therefore, when assessing the success of the algorithm, the focus is on the resulting parameter inference and not necessarily on the load path itself. On average, each step in the algorithm took 26 seconds in Exemplar 1, which included the time needed to calculate the EIG as well as to perform the inference.

Optimal Load Path Selection

Details of the load step selection for Case 1—the equal and opposite case in which F=0.55F = 0.55 and G=0.45G = 0.45—are presented in Figure 4 and Figure 5. Figure 4 shows the decision tree for steps 1–5 starting at load step 1 after initially applying an ε11\varepsilon_{11} strain increment (Figure 2). Node numbers are shown that correspond to the same nodes as indicated in Figure 2. Not shown in the tree is the top node (node 0) at the undeformed state nor the right portion of the tree that follows an initial load step of applying an ε22\varepsilon_{22} strain increment—node 2 and all that follow. In the tree diagram, the mean EIG over the 100 trials at each node is shown in green, and the percent of the 100 trials that went to each node is shown in purple.

Image not extracted

Figure 4. Summary of the load path selection and EIG calculations for Case 1: F=0.55F=0.55 and G=0.45G=0.45. Node numbers correspond to those in Figure 2 extended to 5 load steps. The mean EIG over the 100 trials at each node is shown in green, and the percent of trials that went to each node is shown in purple. The most popular path, chosen for 99% of the trials, is emphasized with darkened arrows. Blue arrows indicate a ε11\varepsilon_{11} load step and red arrows indicate a ε22\varepsilon_{22} load step.

Image not extracted

Figure 5. Box plots show the distributions of EIG estimates for each design option over the 99 trials that chose the predominant alternating load path for steps 2-5 (ξ1=ε11\xi_1=\varepsilon_{11} was pre-selected ahead of the ICC algorithm initiation). The box limits are defined by Q1 and Q3, and whiskers extend to the minimum and maximum values. The medians are denoted by a horizontal line inside each box and the means by a diamond marker. The plots are shown are for Case 1: F=0.55F=0.55 and G=0.45G=0.45. The EIG box plots for ξ=ε11\xi=\varepsilon_{11} are shown in blue and in red for ξ=ε22\xi=\varepsilon_{22}. Box plots for each design option are offset for visual clarity.

At step 1, after the initial ε11\varepsilon_{11} load step was specified, the corresponding data was generated, and inference was performed on the unknown parameters to update the prior to the posterior, π(θ∣y1,ξ1)∝π(y1∣θ,ξ1)π(θ)\pi(\boldsymbol{\theta}\mid\boldsymbol{y}_1,\xi_1) \propto\pi(\boldsymbol{y}_1\mid\boldsymbol{\theta},\xi_1)\pi(\boldsymbol{\theta}), where ξ1=ε11\xi_1 = \varepsilon_{11}. The EIG was then calculated for each of the two load path options, ε11\varepsilon_{11} to go to node 3 or ε22\varepsilon_{22} to go to node 4. In all 100 trials, the optimal load step was determined to be ε22\varepsilon_{22} to go to node 4 at the end of step 2, as this load step had the greater EIG in all trials (mean value of 1.15 over the 100 trials) compared to a ε11\varepsilon_{11} load step (mean EIG value of 1.03 over the 100 trials). At load step 3, in 99% of the trials, load path ε11\varepsilon_{11} to go to node 9 was determined to be the optimal load step with an average EIG over all trials of 0.60, while 1 trial chose load step ε22\varepsilon_{22} to go to node 10. This process continued for 5 steps. In the end, the most popular path, chosen for 99% of the trials, was the alternating load path ξ=[ε11,ε22,ε11,ε22,ε11]\boldsymbol{\xi} = [\varepsilon_{11}, \varepsilon_{22},\varepsilon_{11},\varepsilon_{22},\varepsilon_{11}], which is emphasized with the darkened arrows.

Figure 5 contains box plots summarizing the distributions of the estimated EIG of each design for steps 2–5 of the predominant alternating path shown in Figure 4. The box bounds are defined by the lower (Q1) and upper (Q3) quartiles of the distributions with whiskers extending to the minimum and maximum EIG estimates of the 99 trials that chose this path. The distribution medians are denoted by a horizontal line within each box and the means by a diamond marker. The EIG noticeably decreases with each step as the possible gain in information from the collection of additional data decreases. There is clear division in the EIG distributions for each load step option at steps 2 and 4, resulting in unanimous step selection at these steps, which was load step ε22\varepsilon_{22}. At step 3, overlap in the EIG distributions resulted in 1 path choosing ε22\varepsilon_{22} and the other 99 choosing ε11\varepsilon_{11}. There was again overlap in the distributions at load step 5, although all trials chose ε11\varepsilon_{11}. Variation in the optimal path selection among different trials may be due in part to the data, as previously mentioned, which varied randomly from trial to trial, in addition to the presence of the EIG estimator bias and variance, which can be reduced with greater computational resources.

Table 5 shows the results of the path selection in the adaptive load path design for the 100 different repeat trials for Case 1 as well as for the other 3 cases in Exemplar 1. For Cases 2 and 3, the algorithm chose an alternating design for all 100 trials. In Cases 1 and 4, the alternating design was selected as the optimal one for 99 of the 100 trials. In the singular trial that did not strictly alternate, the ε22\varepsilon_{22} step was repeated once; otherwise, the design choice alternated. Thus, even in the presence of noisy data and randomness in the EIG estimation, an alternating design was clearly preferable for all cases.

Case No.Optimal Load PathPercent
1[ε11, ε22, ε11, ε22, ε11][\varepsilon_{11},\ \varepsilon_{22},\ \varepsilon_{11},\ \varepsilon_{22},\ \varepsilon_{11}]99%
[ε11, ε22, ε22, ε11, ε22][\varepsilon_{11},\ \varepsilon_{22},\ \varepsilon_{22},\ \varepsilon_{11},\ \varepsilon_{22}]1%
2[ε11, ε22, ε11, ε22, ε11][\varepsilon_{11},\ \varepsilon_{22},\ \varepsilon_{11},\ \varepsilon_{22},\ \varepsilon_{11}]100%
3[ε11, ε22, ε11, ε22, ε11][\varepsilon_{11},\ \varepsilon_{22},\ \varepsilon_{11},\ \varepsilon_{22},\ \varepsilon_{11}]100%
4[ε11, ε22, ε11, ε22, ε11][\varepsilon_{11},\ \varepsilon_{22},\ \varepsilon_{11},\ \varepsilon_{22},\ \varepsilon_{11}]99%
4[ε11, ε22, ε22, ε11, ε22][\varepsilon_{11},\ \varepsilon_{22},\ \varepsilon_{22},\ \varepsilon_{11},\ \varepsilon_{22}]1%

Table 5. Results of the design selection in Exemplar 1 for 4 different parameter cases. Percentages are shown over 100 trials.

Posterior Summaries

Functionals of the posterior, such as expected values Eθ∣y,ξ\mathbb{E}_{\theta\mid y,\xi}, variances Vθ∣y,ξ\mathbb{V}_{\theta\mid y,\xi} and credible intervals (CI) can be used for parameter estimation and to summarize uncertainty. These values are discussed here for Case 1, while complete results for all four cases are detailed in Appendix B for completeness.

Box plots showing the distribution of marginal posterior summaries at each step (t=1,…,5t=1,\ldots,5) over the 100 repeat trials are plotted in Figure 6 for Case 1 for the adaptive design as well as the two static designs.

Figure 6. Box plots are shown which describe the distribution of marginal posterior summaries over the 100 trials for Case 1 at each load step for the adaptive design and two static designs. True parameter values are shown with black dashed lines (Table 2). Box plots for the expected value Eθ∣y,ξ\mathrm{E}_{\theta\mid y,\xi} (left column) and variance Vθ∣y,ξ\mathrm{V}_{\theta\mid y,\xi} (right column) are shown for parameter FF (top row) and parameter GG (bottom row). The boxes are bounded by Q1 and Q3 and whiskers extend to the minimum and maximum values in the distribution. The median and mean values are shown with a horizontal line and diamond marker, respectively, for each box plot. The adaptive design results are shown in green, the ϵ11\epsilon_{11} design results are shown in blue and the ϵ22\epsilon_{22} design results in red. Box plots for each design are offset for visual clarity.

The static ε22\varepsilon_{22} load path produced an expected value for parameter FF, on average, that was closest to the true value (as shown by the mean) with the lowest variability from trial to trial (as shown by the box and whiskers) (Figure 6a), as well as the lowest uncertainty about FF at the final load step (Figure 6b). Likewise, the static ε11\varepsilon_{11} load path yielded data that provided an expected value for parameter GG, on average, that was closest to the true value at the final step with the lowest variability among the trials (Figure 6c), as well as the least amount of uncertainty (Figure 6d). Similar results were obtained for the other three cases as well (see Table B.1 in Appendix B).

While marginal summaries are useful for understanding the uncertainty and estimating the quality of each individual parameter, the interest of this work is to obtain the optimal joint calibration of the parameters that has the lowest combined uncertainty over all parameters. Two different scalar measures of multi-dimensional dispersion are the generalized variance and the total variance. The generalized variance is the determinant of the covariance matrix, det⁡(Ψ)\det(\Psi), and is proportional to the volume of space occupied by the distribution through its square root. This metric reflects parameter correlations and takes on smaller values in the presence of high parameter correlation and higher values when there is little to no parameter correlation. High diffusivity in the distribution, and likewise uncertainty, is indicated by large values. The total variance is obtained through the trace of the covariance matrix, trace⁡(Ψ)\operatorname{trace}(\Psi). As its name suggests, by summing all the diagonal components of variance, the total variance provides a measure of the total amount of variation in the distribution.

Figure 7 shows box plots of the generalized (7a) and total (7b) variance of the posterior for the adaptive and static designs at each step in the algorithm for Case 1. The plots summarize the distribution of results that were obtained over the 100 repeat trials. Both the generalized variance and total variance decreased with every step in the algorithm—corresponding to reduced parameter uncertainty as more data was collected. The adaptive design had a smaller variance by both metrics on average over the 100 trials than either of the static designs at each step. Again, similar results were obtained for the other three cases as well (see Table B.2 in Appendix B).

Figure 7. Box plots of the generalized (a) and total (b) variance for Case 1: F=0.55F = 0.55, G=0.45G = 0.45. The plots summarize the distributions over the 100 repeat trials. The boxes are bounded by Q1 and Q3 and have whiskers that extend to the minimum and maximum values. The median values are indicated with a horizontal line in the box, and the mean values are shown with diamond markers. Box plots for each design are offset for visual clarity.

It has been established that the overall parameter uncertainty was lower in the adaptive design for Exemplar 1 compared to the two static designs. However, if the lower uncertainty is accompanied by a greater bias in parameter expected values, then the inference results in a posterior distribution that places greater probability (confidence) in parameter values that are farther from the true values—an undesirable scenario. A useful metric in comparing how close a point xx is to a given distribution in multivariate space, often used for outlier detection, is the Mahalanobis distance (MD) [59],

MD⁡(x,μ)=((x−μ)TΨ−1(x−μ))0.5,(19)\operatorname{MD}(x,\mu)=\left((x-\mu)^{T}\Psi^{-1}(x-\mu)\right)^{0.5}, \tag*{(19)}

where μ\mu and Ψ\Psi are the mean and covariance of the distribution and xx is the point from which the distance is being calculated. If the variables are uncorrelated, the axes of the distribution are orthogonal to each other, and the Mahalanobis distance is equivalent to the Euclidean distance. However, in the case where two or more parameters are correlated, the axes are no longer orthogonal, and the Mahalanobis distance takes this correlation into account. A lower Mahalanobis distance corresponds to the point residing in a region of higher probability in the distribution. The average Mahalanobis distance of the true parameter values

MD⁡(θtrue)\operatorname{MD}(\theta^{true}) from the posterior distribution at the final step was 1.25 in the adaptive design and 1.26 for both the ε11\varepsilon_{11} and ε22\varepsilon_{22} static designs in Case 1. Table B.2 in Appendix B reports MD⁡(θtrue)\operatorname{MD}(\theta^{true}) for all cases, and in summary, MD⁡(θtrue)\operatorname{MD}(\theta^{true}) was less in the adaptive settings than either of the static settings in three of the four presented cases, meaning θtrue\theta^{true} sat in a region of higher probability, on average, when the load path was adaptively chosen. In Case 4, the static ε11\varepsilon_{11} design had the lowest MD, followed by the adaptive design and then finally the ε22\varepsilon_{22} design.

These scalar metrics reveal that when the load path is adaptively chosen within the ICC framework, the calibrated parameters have a lower combined uncertainty, and the posterior distribution places a higher probability on the true values (in most cases) when compared to the static designs. While the advantage of the adaptive design for this exemplar problem is demonstrably small (i.e., a marginally lower uncertainty and MD), these results show the success of the algorithm in actively obtaining data that is more informative for parameter calibration than the selected static designs.

Propagation of Uncertainty to Stress-Strain Space

The end goal of obtaining accurate parameter estimates with known levels of uncertainty is to be able to propagate the parameter uncertainty through the model in the application of interest. Here, the parameter uncertainty is propagated through the MPS for demonstrative purposes and compared for each of the design settings. One hundred samples were drawn from the posterior distribution θ~∼π(θ∣y,ξ)\tilde{\theta} \sim\pi(\theta\mid y, \xi) for each of the three experimental designs (adaptive and both static designs) for Case 1 and propagated to the model output space by running the MPS at the sampled parameters u(θ~)=[σ11,σ22]\mathbf{u}(\tilde{\theta}) = [\sigma_{11}, \sigma_{22}].

In Figure 8, 95% CIs for computed in-plane stress values from the 100 draws are shown along with the MPS output at the true parameter values u(θtrue)\mathbf{u}(\theta^{true}). The left column of plots shows the 95% CI for the trial in each design setting that had the least posterior total variance, and the right column of plots shows the 95% CI for the trial that had the greatest posterior total variance. Plots are shown for both the minimum and maximum total variance in order to illustrate the best and worst case scenario within the 100 trials for each design setting. For all three design settings, there is little difference in the CIs for the two bounding scenarios. There are instances where the true response sits outside the 95% CIs, most notably in Figure 8c and 8e. For the adaptive design, both the minimum and maximum total variance came from trials with an optimal design of ξ∗=[ε11,ε22,ε11,ε22,ε11]\xi^{*} = [\varepsilon_{11}, \varepsilon_{22}, \varepsilon_{11}, \varepsilon_{22}, \varepsilon_{11}]. For all three experimental designs, the low level of posterior uncertainty in the FF and GG parameters translates to low uncertainty in the stress-strain space, as evidenced by the narrow credible intervals. Thus, for this simplified exemplar, the marginally reduced uncertainty in the parameters obtained with the adaptive design did not translate to a noticeable reduction in the uncertainty of the final QOI (namely, the stress response) when compared to the static designs.

Figure 8. 95% credible intervals for the posterior draws θ~∼π(θ∣y,ξ)\tilde{\theta} \sim\pi(\theta\mid y,\xi) were calculated point-wise for Exemplar 1, Case 1 and are indicated the the blue (σ11\sigma_{11}) and red (σ22\sigma_{22}) dashed lines. The model output from the true parameter values is plotted with a black line. The left column of plots show the 95% CI for the trial in each design setting that had a posterior distribution with the least total variance among all trials. The right column of plots show 95% CI for the trial in each design setting that had the greatest total variance. Plots (a) and (b) show results for the adaptive design, plots (c) and (d) show results for the static ε11\varepsilon_{11} design and plots (e) and (f) for the static ε22\varepsilon_{22} design.

Exemplar 2

In this section, details of the statistical modeling choices and algorithmic settings are discussed for Exemplar 2 (Sec. 5.1) and results are presented (Sec. 5.2). The adaptive load paths chosen from within the ICC framework are discussed (Sec. 5.2.1) and the resulting parameter inference is compared to two naive static design settings via posterior summaries (§ 5.2.2) and propagated uncertainty (§ 5.2.3).

Problem Setup

In the second exemplar, introduced in § 3.1, the calibration was made more complex by adding the yield stress σy\sigma_y and the hardening parameters, AA and nn, (3) to the unknown parameter vector θ=[F,G,σy,A,n]T\boldsymbol{\theta} = [F,G,\sigma_y,A,n]^{T}.

As in Exemplar 1, the parameters were represented with a truncated normal probability model (13) in order to constrain the values they could take to be in agreement with the bounds used for building the surrogates (Table 1). The means, variances and bounds of the priors on FF and GG were the same as those used in Exemplar 1. The prior means for the yield stress σy\sigma_y and hardening parameters AA and nn were chosen to be within the defined bounds, and variances were chosen that were moderately diffuse. The upper and lower bounds (aa and bb) of the distribution were the same as those used in the surrogate builds (§ 3.6). Prior means, variances and bounds are detailed in Table 6. The model for the data was chosen to be the same as that used in Exemplar 1 (15), and the observation noise variance was assumed to be known at ψ2=10MPa2\psi^2 = 10\mathrm{MPa}^2.

Parameterμθ\mu_\thetaδθ2\delta_\theta^2aabb
FF0.510.30.7
GG0.510.30.7
AA200100010400
nn501001×10−61 \times10^{-6}100
σy\sigma_y250100050500

Table 6. Means μθ\mu_\theta, variances δθ2\delta_\theta^2 and bounds aa and bb for the truncated normal prior model (13) used in Exemplar 2.

Exemplar 2, which had 5 unknown parameters and exercised two different phenomenologies (yield and hardening), presented more complexities than Exemplar 1, which only had 2 unknown parameters and exercised only one phenomenology (yield). Specifically, local modes in the posterior made it more difficult to find θ^\hat{\boldsymbol{\theta}} (7), which was overcome by increasing the number of restarts (initializations from random locations) in the minimization. Parameter uncertainties were greater, leading to a higher susceptibility to arithmetic underflow (Appendix § 8.§ 2) in the initial steps of the algorithm. To overcome the occurrence of underflow, MM in the EIG estimate was increased to 1×1041 \times10^4, which was the same value as NN. Finally, in order to mitigate problems with parameter identifiability, the number of load steps was increased to T=7T = 7, with each load step containing a 0.02 mm/mm strain increment, so that the total strain was 0.14 mm/mm at the end of step 7, and data points were collected at three equally-spaced locations between each load step. Thus, three times more data was included compared to Exemplar 1. In total, these modifications allowed the ICC framework to be successful for the more complicated material model of Exemplar 2, as discussed in the following sections. The final deviation from the algorithmic setup in Exemplar 1 is that in Exemplar 2 the initial load step was determined by calculating the EIG, as opposed to Exemplar 1, in which the initial load step was pre-determined to be ξ1=ϵ11\xi_1 = \epsilon_{11}. In Exemplar 2, the time to complete each step in the algorithm (EIG calculation plus inference) was just under 3 minutes on average.

Results

The two parameter cases for Exemplar 2 detailed in § 3.1 were used in an adaptive setting in which the optimal load path was chosen by calculating the EIG, as well as in two static design settings—one in which the load path was decided a priori to be ϵ11\epsilon_{11} each step in the algorithm and, alternatively, one in which the path was ϵ22\epsilon_{22} for each step (reflecting traditional approaches based on human intuition). In total, 6 different combinations (2 parameter cases run under 3 design settings) were studied in this second exemplar. As in Exemplar 1, 100 repeat trials were performed for each of the 6 combinations.

Optimal Load Path

Table 7 shows the results of the path selection for the adaptive design for the 100 different repeat trials for both parameter cases. Unlike in Exemplar 1, there was not a single dominant load path in either case. Instead, four paths were all relatively likely (chosen 14–35% of the time), and a fifth path was chosen a few times in each case (2–4% of the time). Additionally, while the designs were predominately alternating (similar to Exemplar 1), most paths contained at least one repeat load step, typically near the end of the load path.

Case No.Optimal Load PathPercentEF∣y,ξE_{F \mid y,\xi}EG∣y,ξE_{G \mid y,\xi}EA∣y,ξE_{A \mid y,\xi}En∣y,ξE_{n \mid y,\xi}Eσy∣y,ξE_{\sigma_y \mid y,\xi}Gen. Var. (1 × 10−910^{-9})
5[ε11\varepsilon_{11}, ε22\varepsilon_{22}, ε11\varepsilon_{11}, ε22\varepsilon_{22}, ε11\varepsilon_{11}, ε22\varepsilon_{22}, ε22\varepsilon_{22}]32%0.5500.44910120.12991.19
5[ε22\varepsilon_{22}, ε11\varepsilon_{11}, ε22\varepsilon_{22}, ε11\varepsilon_{11}, ε11\varepsilon_{11}, ε22\varepsilon_{22}, ε22\varepsilon_{22}]31%0.5500.45010120.42991.22
5[ε11\varepsilon_{11}, ε22\varepsilon_{22}, ε11\varepsilon_{11}, ε22\varepsilon_{22}, ε22\varepsilon_{22}, ε11\varepsilon_{11}, ε11\varepsilon_{11}]19%0.5510.45299.620.53001.11
5[ε22\varepsilon_{22}, ε11\varepsilon_{11}, ε22\varepsilon_{22}, ε11\varepsilon_{11}, ε22\varepsilon_{22}, ε11\varepsilon_{11}, ε11\varepsilon_{11}]14%0.5470.44698.519.43011.30
5[ε11\varepsilon_{11}, ε22\varepsilon_{22}, ε11\varepsilon_{11}, ε22\varepsilon_{22}, ε11\varepsilon_{11}, ε11\varepsilon_{11}, ε22\varepsilon_{22}]4%0.5490.45410020.33001.15
5[ε11\varepsilon_{11}, ε11\varepsilon_{11}, ε11\varepsilon_{11}, ε11\varepsilon_{11}, ε11\varepsilon_{11}, ε11\varepsilon_{11}, ε11\varepsilon_{11}]NA0.5500.46010120.3301160
5[ε22\varepsilon_{22}, ε22\varepsilon_{22}, ε22\varepsilon_{22}, ε22\varepsilon_{22}, ε22\varepsilon_{22}, ε22\varepsilon_{22}, ε22\varepsilon_{22}]NA0.5450.45099.620.0299116
6[ε11\varepsilon_{11}, ε22\varepsilon_{22}, ε11\varepsilon_{11}, ε22\varepsilon_{22}, ε11\varepsilon_{11}, ε22\varepsilon_{22}, ε22\varepsilon_{22}]35%0.5500.45030020.01000.351
6[ε22\varepsilon_{22}, ε11\varepsilon_{11}, ε22\varepsilon_{22}, ε11\varepsilon_{11}, ε22\varepsilon_{22}, ε11\varepsilon_{11}, ε11\varepsilon_{11}]27%0.5470.44829920.099.60.370
6[ε11\varepsilon_{11}, ε22\varepsilon_{22}, ε11\varepsilon_{11}, ε22\varepsilon_{22}, ε11\varepsilon_{11}, ε22\varepsilon_{22}, ε11\varepsilon_{11}]22%0.5520.45030019.91010.353
6[ε22\varepsilon_{22}, ε11\varepsilon_{11}, ε22\varepsilon_{22}, ε11\varepsilon_{11}, ε22\varepsilon_{22}, ε11\varepsilon_{11}, ε22\varepsilon_{22}]14%0.5510.45130119.81000.390
6[ε11\varepsilon_{11}, ε22\varepsilon_{22}, ε11\varepsilon_{11}, ε22\varepsilon_{22}, ε11\varepsilon_{11}, ε11\varepsilon_{11}, ε22\varepsilon_{22}]2%0.5580.45930220.11020.340
6[ε11\varepsilon_{11}, ε11\varepsilon_{11}, ε11\varepsilon_{11}, ε11\varepsilon_{11}, ε11\varepsilon_{11}, ε11\varepsilon_{11}, ε11\varepsilon_{11}]NA0.5500.50331020.71036.68×10−46.68 \times10^{-4}
6[ε22\varepsilon_{22}, ε22\varepsilon_{22}, ε22\varepsilon_{22}, ε22\varepsilon_{22}, ε22\varepsilon_{22}, ε22\varepsilon_{22}, ε22\varepsilon_{22}]NA0.5420.45029719.998.766.1

Table 7. Results of the design selection in Exemplar 2 for parameter Cases 5 and 6 are tabulated as well as the percentage of trials (out of 100) that chose each load path. Marginal posterior expected values and generalized variances are recorded after the final load step for each optimal load path chosen. Reported values are averages over the trials that chose each load path. Below the dashed lines are the results from each of the two static designs (ε11\varepsilon_{11} and ε22\varepsilon_{22}) for each case.

Figure. 9—shown for the most popular design of Case 5—reveals that at some steps, the distributions of EIG estimates for each of the load step options were very similar, meaning that each load step option was expected to provide nearly the same amount of information. Indeed, as shown in Table 7, the marginal expected values and generalized variance were similar for all five load paths for both cases—indicating that similar information content was obtained from the various optimal load paths.

The higher number of optimal load paths for Exemplar 2 compared to Exemplar 1 may be caused by having more unknown parameters and activating two different phenomenologies. The different instances of noisy data that were used in each trial and the inherent variation in the EIG estimation (see § 3.3) may combine with complex parameter interactions to propel the BOED algorithm to select one path over another similar path.

In an actual physical experiment, only a single load path would be selected in situ, and this multitude of different optimal paths may not be noticed unless multiple physical experiments were performed. However, as shown in the following sections (as well as in Table 7, where the values from the static designs are recorded below the dashed lines) the paths generated adaptively within the ICC framework produce similar parameter estimates that, notably, are better on the whole than parameter estimates obtained through the static designs.

Posterior Summaries

Box plots showing the distribution of marginal posterior summaries at each step (t=1,⋯ ,7t = 1,\cdots,7) over the 100 repeat trials are plotted in Figure 10 for parameter Case 5 and are tabulated for both Cases 5 and 6 at the final load step in Table C.1. The plots reveal that the static ε22\varepsilon_{22} load path was suboptimal for inference on parameter FF, as seen by an expected value that is demonstrably further from the true value than either of the other two design settings (with much greater variability over the 100 trials) as well as a higher uncertainty (Figure 10a & Figure 10b). Likewise, the static ε11\varepsilon_{11} design was sub-optimal for inference on parameter GG compared to the other two settings (Figure 10c & Figure 10d). The adaptive load path yielded inference on parameter σy\sigma_y with an expected value consistently closer to the true value at the final load step than either of the static designs as well as a significantly lower amount of uncertainty (Figure 10e & Figure 10f). Expected values among all three designs were similar for parameters AA and nn, with AA having lower uncertainty in the adaptive setting (Figure 10g-Figure 10j).

Figure 10. Box plots are shown which describe the distribution of marginal posterior summaries over the 100 trials for Case 5 at each load step for the adaptive design and two static designs. True parameter values (Table 3) are shown with black dashed lines. Box plots for the expected value Eθ∣y,ξE_{\theta\mid y,\xi} (left column) and variance Vθ∣y,ξV_{\theta\mid y,\xi} (right column) are shown for each parameter. The boxes are bounded by Q1 and Q3 and whiskers extend to the minimum and maximum values in the distribution. The median and mean values are shown with a horizontal line and diamond marker, respectively, for each box plot. The adaptive design results are shown in green, the ϵ11\epsilon_{11} design results are shown in blue and the ϵ22\epsilon_{22} design results in red. Box plots for each design are offset for visual clarity.

Figure 9. Box plots show the distributions of the EIG estimates at steps 1-7. The represented distributions are over the 31 trials that chose the most popular path in Case 5. Box bounds indicate the upper and lower quartiles and whiskers extend to the minimum and maximum values. The median and mean values are shown with a horizontal line in the box and a diamond marker, respectively. The plots for ξ=ε11\xi= \varepsilon_{11} are shown in blue and in red for ξ=ε22\xi= \varepsilon_{22}. Box plots for each load step option are offset for visual clarity.

Figure 13 (PDF p. 24).

Figure 11 shows box plots of the generalized (11a) and total (11b) variance of the posterior for the adaptive and static designs at each step in the algorithm for Case 5. The plots summarize the distribution of results that were obtained over the 100 repeat trials. The adaptive design had a significantly smaller variance by both metrics than either of the static designs. Similar results were obtained for Case 6 (see Table C.3 in Appendix C).

Figure 11. Box plots of the generalized (a) and total (b) variance for Case 5: F=0.55F = 0.55, G=0.45G = 0.45, A=100A = 100, n=20n = 20, σy=300\sigma_y = 300. The plots summarize the distributions over the 100 repeat trials. The boxes are bounded by Q1 and Q3 and have whiskers that extend to the minimum and maximum values. The median values are indicated with a horizontal line in the box, and the mean values are shown with diamond markers. Box plots for each design are offset for visual clarity.

The most significant difference in Case 6 compared to Case 5 in the adaptive setting was seen in the inference for parameters σy\sigma_y and AA. Since the true values of AA and σy\sigma_y changed significantly, the coefficient of variation (Cv) (a standardized and unitless measure of variation) is referred to for a reliable comparison of parameter variability. Recall that Case 5 had a lower value of AA and a higher value of σy\sigma_y (A=100A = 100 MPa and σy=300\sigma_y = 300 MPa) compared to Case 6 (A=300A = 300 MPa and σy=100\sigma_y = 100 MPa). The coefficients of variation followed the opposite trends, with the Cv for AA being higher than the Cv for σy\sigma_y for Case 5 (CvA=0.022\mathrm{Cv}_A = 0.022 and Cvσy=0.010\mathrm{Cv}_{\sigma_y} = 0.010) and the Cv for AA being lower than the Cv for σy\sigma_y for Case 6 (CvA=0.009\mathrm{Cv}_A = 0.009 and Cvσy=0.026\mathrm{Cv}_{\sigma_y} = 0.026). These results suggest that the dominant phenomenology (e.g., yield in Case 5 and hardening in Case 6) is more readily identified with a lower amount of normalized variability compared to the non-dominant contributor.

A second key difference is that parameter inference was especially difficult in Case 6 for the static design settings, and not all parameters were successfully inferred. This is reflected in the considerable variances reported in Table 7 and Tables C.1 & C.3 in Appendix C. In particular, some of the credible intervals for FF and GG go outside the bounds that were set and yield values that are not physically possible.9 Even if the trials that were unsuccessful are removed from consideration in the average values (Tables C.2 & C.4 in Appendix C), the 95% CIs of GG still fall outside the given bounds and variances are still large. Overall, Case 6 proved to be a much more difficult setting to perform inference with a static design. In contrast, the adaptive design reliably provided parameter estimates in close agreement with the true values with low uncertainty (Table C.1 in Appendix C).

The average MD of the true parameter values from the posterior distribution at the final step is also shown in Table C.3. The adaptive design yielded a smaller MD than both static designs in both cases. Not only did the adaptive design provide parameter inference with lower uncertainty in Exemplar 2, it also yielded a posterior distribution that represented θtrue\theta^{true} better than either of the static designs.

Taken together, these results reinforce the advantages of the in situ feedback loop and BOED algorithm over traditional static designs chosen based on human intuition a priori. By ensuring that the necessary data is collected, the adaptive designs are able to calibrate material models in situations when static designs fail completely, and they provide greater accuracy and precision in the parameter values overall.

Propagation of Uncertainty to Stress-Strain Space

One hundred samples from the posterior distribution were drawn and propagated to the model output space as was done in Exemplar 1 (§ 4.2.3). In Figure 12, 95% credible intervals for computed in-plane stress values from the 100 draws are shown. The left column of plots shows the 95% CI for the trial in each design setting that had the least posterior total variance, and the right column of plots shows the 95% CI for the trial that had the greatest posterior total variance. For the adaptive design, the minimum total variance came from a trial with an optimal design of ξ∗=[ε11,ε22,ε11,ε22,ε22,ε11,ε11]\xi^{*} = [\varepsilon_{11}, \varepsilon_{22}, \varepsilon_{11}, \varepsilon_{22}, \varepsilon_{22}, \varepsilon_{11}, \varepsilon_{11}], and the maximum total variance came from a trial with an optimal design of ξ∗=[ε22,ε11,ε22,ε11,ε11,ε22,ε22]\xi^{*} = [\varepsilon_{22}, \varepsilon_{11}, \varepsilon_{22}, \varepsilon_{11}, \varepsilon_{11}, \varepsilon_{22}, \varepsilon_{22}]. At a high level, all three designs provided credible intervals that generally encompassed the true response.

Image not extracted

Figure 12. 95% credible intervals for the posterior draws θ~∼π(θ∣y,ξ)\tilde{\theta} \sim\pi(\theta\mid y, \xi) were calculated point-wise for Exemplar 2, Case 5 and are indicated the the blue (σ11\sigma_{11}) and red (σ22\sigma_{22}) dashed lines. The model output from the true parameter values is plotted with a black line. The left column of plots show the 95% CI for the trial that had a posterior distribution with the least total variance among all trials for each design setting. The right column of plots show 95% CI for the trial that had the greatest total variance. Plots (a) and (b) show results for the adaptive design, plots (c) and (d) show results for the static ε11\varepsilon_{11} design and plots (e) and (f) for the static ε22\varepsilon_{22} design. Box plots for each design are offset for visual clarity.

The 95% CIs for Exemplar 2 Case 6 are shown in Fig. C.1 in Appendix C. The difficulty of the static designs to perform reliable inference for this case is reflected in the wide CIs for the trials that had the maximum total variance. In summary, for Case 5, the reduced uncertainty in the material model parameters obtained with the adaptive design did not translate to a noticeable reduction in the uncertainty of the final QOI; however, in Case 6, the reduced parameter uncertainty did so translate. Thus, the adaptive design still has a great potential to reduce the uncertainty of the final QOI for more realistic applications that move beyond the material point simulator and utilize real experimental data. Such investigations are the subject for future work.

Conclusions

This paper presented a first demonstration of the ICC framework for a constitutive model calibration problem in which the load path of a material point—representing the center of a cruciform specimen—was optimized with respect to parameter uncertainty. The adaptive design of the load path was segmented into the selection of individual load steps of a specified increment of strain along a specified axis of the material point, and the optimal path was determined by calculating the EIG of each candidate load step. The benefits of choosing the load path in an adaptive manner within the framework as opposed to choosing a static load path were demonstrated with two exemplar problems of varying complexity.

In both exemplars, the adaptive load path designs yielded posterior distributions with lower uncertainty than in static settings on average when the uncertainty was considered holistically over all parameters. This result was supported by the total and generalized variances reported for the designs at each step in the algorithm. There was also evidence that the adaptive designs on average lead to posterior distributions that were more reliable in capturing the true parameter values. This greater reliability was supported by a lower MD(θtrue)MD(\theta^{true}) on average for the adaptive designs, which indicates that the true parameter values resided in regions of higher probability for the adaptive designs compared to the static designs.

In both exemplar problems, an alternating load path was clearly preferred in the adaptive designs. This alternating tendency was likely due to the yield anisotropy, which benefits from probing different directions in order to obtain estimates of the parameters. However, an in depth analysis of the contributing factors for the predominantly chosen load paths was not performed. The optimal load path is specific to the load path tree structure, which in this work was a binary tree where each parent node had two children nodes corresponding to applying a positive increment of strain along one of two directions (ϵ11\epsilon_{11} or ϵ22\epsilon_{22}). Any alternative tree structure would likely result in optimal load paths which deviate from what was observed in this work. Although the binary tree structure provided a simple starting point, the ICC framework is not restricted to this tree structure and is easily adaptable to alternatives. Additional children could be introduced at each node in the tree, which could include compression or simultaneous strain increments along the two axes, for example. In such a case, the ICC framework would operate in the same manner, with the exception that the EIG would need to be calculated for any additional design options.

This initial effort was the significant first step of a greater objective to demonstrate the ICC framework on a cruciform specimen being actively controlled in a bi-axial load frame in real-time with the calibration of FEA models using full-field DIC data. With a run-time of less than 3 minutes per step in Exemplar 2 (the more complicated problem), which includes the time required for the EIG calculation plus inference, the efficiency of the developed algorithm makes it feasible to be used in a quasi real time scenario. The importance of time efficiency is emphasized for future live demonstrations of the ICC framework which introduce lab time considerations as well as anticipated challenges with holding a sample completely still in the load frame while the algorithm is running. Unwanted sample movement during the experiment may result in boundary conditions and/or material responses (e.g. creep) that are unaccounted for in the material model, thus increasing any model form error that may be present and negatively impacting the results of the parameter inference. The 100 repeat trials that were performed in this work were utilized for demonstrative purposes only and will not be a part of future applications. With live experiments, the ICC algorithm will be run once, and a single specimen will be tested to collect data for calibration.

Acknowledgements

This work was supported by the Laboratory Directed Research and Development program at Sandia National Laboratories, a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

This paper describes objective technical results and analysis. Any subjective views or opinions that might be expressed in the paper do not necessarily represent the views of the U.S. Department of Energy or the United States Government.

Algorithmic Supplementary Material

Posterior Approximation with MCMC

Markov Chain Monte Carlo (MCMC) is a commonly used method for approximating the posterior distribution and obtaining posterior summaries such as parameter expected values, variances and credible intervals. In this approach, a Markov chain is constructed through sampling such that its stationary distribution matches the posterior distribution of interest. The trade-off for the high-accuracy of this method is computational time, as it can require millions of forward model evaluations. In the setting of material model calibration, MCMC simulations may be applied for computationally cheap models, but they become prohibitively expensive for high-fidelity constitutive models and simulations (such as FEA). There exist a number of ways to improve simulation efficiency, which include the use of conjugate priors [60, 61], advanced sampling techniques [64, 65] [62, 63], or using a surrogate model u~\tilde{u} in place of the expensive forward model uu. However, even with these adjustments, an MCMC simulation would be too expensive for the quasi real-time characterization and calibration setting in which the ICC framework will eventually operate.

Approximation of the Expected Information Gain (EIG)

A nested Monte Carlo estimator is used in this work to approximate the EIG,

EIG^MC(ξ)=1N∑n=1Nlog⁡f(yn∣θn,0,ξ)1M∑m=1Mf(yn∣θn,m,ξ),θn,m∼π(θ),yn∼f(y∣θn,0,ξ),(20)\widehat{\mathrm{EIG}}_{\mathrm{MC}}(\boldsymbol{\xi}) = \frac{1}{N} \sum_{n=1}^{N} \log\frac{f(\boldsymbol{y}_n \mid\boldsymbol{\theta}_{n,0}, \boldsymbol{\xi})}{\frac{1}{M}\sum_{m=1}^{M} f(\boldsymbol{y}_n \mid\boldsymbol{\theta}_{n,m}, \boldsymbol{\xi})}, \qquad\boldsymbol{\theta}_{n,m} \sim\pi(\boldsymbol{\theta}), \qquad\boldsymbol{y}_n \sim f(\boldsymbol{y} \mid\boldsymbol{\theta}_{n,0}, \boldsymbol{\xi}), \tag*{(20)}

where samples from the data distribution and the prior are used to evaluate the inner and outer sum of the Monte Carlo estimator. NN and MM determine the number of samples that are used to evaluate the outer and inner sums, and the estimator variance and bias scale inversely with the magnitude of NN and MM, respectively [55]. The estimator requires a total of N(1+M)N(1+M) model evaluations to calculate the EIG, so NN and MM must be chosen judiciously to balance the estimator quality and computational cost. Using the same θn,:\boldsymbol{\theta}_{n,:} parameter samples for every iteration of the outer loop reduces the number of model evaluations down to (N+M)(N+M). While this simplification may introduce some additional bias, some studies have shown that minimal levels of error are incurred [66] [53]; thus, using the same inner samples for each iteration of the outer loop is the approach taken in this work. Some alternative approaches include calculating the evidence with Laplace’s approximation [51, 55] or using a linear approximation of the forward model to improve the efficiency of EIG estimation [67].

In some cases, a numerical issue known as arithmetic underflow may be encountered when computing the inner loop in (15). In particular, arithmetic underflow may occur if the distribution of the evidence is narrow (i.e., low variability), and the prior distribution is not sufficiently centered on the posterior distribution. This situation may lead to a scenario where the calculated likelihood of all data samples yn\boldsymbol{y}_n are below machine precision, resulting in an approximation for the evidence term equal to 00. Arithmetic underflow is especially likely for small values of MM, so a simple way to avoid it is to increase the value of MM or to use the same parameter samples for the inner and outer loops [66], possibly at the cost of a greater bias in the estimate. Underflow can also be handled by using importance sampling with a biasing distribution other than the prior. One approach is to define the biasing distribution as the Laplace approximation to the posterior for each outer sample [68] [52]. Similarly, in [69] the biasing distribution was calculated with a self-normalized approximation to the posterior. In this work, underflow was avoided by choosing MM to be sufficiently large.

Adaptive BOED

In cases, such as in this work, in which multiple experiments are to be performed ξ=[ξ1,…,ξT]\boldsymbol{\xi} = [\xi_1, \ldots, \xi_T], where TT is the number of experiments, an adaptive BOED determines the optimal experimental design by incorporating previously collected data into the design selection process.

Following closely the theory laid out in [70], assume the TT experiments yield data y1,…,yT\boldsymbol{y}_1, \ldots, \boldsymbol{y}_T. Under the assumption of conditional independence of the data given the unknown parameters and the designs, the following relationship for the joint distribution over the data and parameters holds,

π(y1:T,θ∣ξ1:T)=π(θ)∏t=1Tf(yt∣θ,ξt).(21)\pi(\mathbf{y}_{1:T},\theta\mid\boldsymbol{\xi}_{1:T})=\pi(\theta)\prod_{t=1}^{T}f(\mathbf{y}_t\mid\theta,\boldsymbol{\xi}_t). \tag*{(21)}

This relationship is leveraged when calculating the EIG for the ttht^{\mathrm{th}} experiment, ξt\boldsymbol{\xi}_t, which takes into account data obtained from experiments 1:t−11:t-1. For all t>1t>1, π(θ)\pi(\theta) in (15) is replaced by the posterior conditional on all previous experiments and collected data, π(θ∣y1:t−1,ξ1:t−1)\pi(\theta\mid\mathbf{y}_{1:t-1},\boldsymbol{\xi}_{1:t-1}), and samples from the data distribution are drawn for the ttht^{\mathrm{th}} experiment, yn∼f(yt∣θ,ξt)\mathbf{y}_n\sim f(\mathbf{y}_t\mid\theta,\boldsymbol{\xi}_t). Thus, the joint distribution of the data and parameters given the designs at the ttht^{\mathrm{th}} step is written as

π(y1:t,θ∣ξ1:t)=π(θ)f(y1∣θ,ξ1)f(y2∣θ,ξ2)⋯f(yt−1∣θ,ξt−1)f(yt∣θ,ξt)=π(θ∣y1:t−1,ξ1:t−1)f(yt∣θ,ξt).(22)\begin{aligned} \pi(\mathbf{y}_{1:t},\theta\mid\boldsymbol{\xi}_{1:t})&=\pi(\theta)f(\mathbf{y}_1\mid\theta,\boldsymbol{\xi}_1)f(\mathbf{y}_2\mid\theta,\boldsymbol{\xi}_2)\cdots f(\mathbf{y}_{t-1}\mid\theta,\boldsymbol{\xi}_{t-1})f(\mathbf{y}_t\mid\theta,\boldsymbol{\xi}_t)\\ &=\pi(\theta\mid\mathbf{y}_{1:t-1},\boldsymbol{\xi}_{1:t-1})f(\mathbf{y}_t\mid\theta,\boldsymbol{\xi}_t). \tag*{(22)} \end{aligned}

ICC Implementation

The pseudocode for the proposed algorithm is presented in this section to outline the steps of the ICC framework. To initiate the procedure, surrogates are pre-built offline ahead of entering the feedback loop (Figure 1 and § 3.6). The characterization and calibration cycle is then initiated by determining the initial experimental design (load step). Algorithm 1 assumes the initial load step is determined by calculating the EIG, as in Exemplar 2 (§ 5), although the first load step may alternatively be pre-determined, as was done in Exemplar 1 (§ 4). At each decision point t=1,…,Tt=1,\ldots,T, the next load step in the load path is determined by calculating the EIG (15) for each proposed load step ξt∈[ε11,ε22]\boldsymbol{\xi}_t\in[\varepsilon_{11},\varepsilon_{22}]. To do this, parameter samples θn,0\boldsymbol{\theta}_{n,0} are drawn from the prior for the outer loop of the EIG approximation and data yn\mathbf{y}_n is simulated from the data distribution. In the ICC implementation, the same outer parameter samples were used for each proposed load step (ε11\varepsilon_{11} or ε22\varepsilon_{22}). Parameter samples θn,m\boldsymbol{\theta}_{n,m} are then drawn for the inner loop of the EIG approximation. The same inner loop samples were used for each iteration of the outer loop n=1,…,Nn=1,\ldots,N (Sec. Appendix A.2) and for each proposed load step. The load step that has the maximum EIG is chosen, and data yt\mathbf{y}_t is generated. The posterior is approximated and becomes the prior for the EIG approximation in the next step (A.2) and (A.3). After a pre-determined number of load steps, inference on the parameters is complete and summaries of the resulting posterior distribution are used to describe parameter uncertainty.

Algorithm 1 ICC Framework.

1: Input: pre-build surrogates offline, u~(⋅)\tilde{u}(\cdot). 2: Set q(θ)=π(θ)q(\theta)=\pi(\theta). 3: for t=1:Tt=1:T do 4: &nbsp;&nbsp;&nbsp;&nbsp;for i=1:length⁡(Ξ)i=1:\operatorname{length}(\Xi) do 5: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;for n=1:Nn=1:N do 6: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;if i==1i==1 then 7: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Draw parameter samples θn,0∼q(θ)\theta_{n,0}\sim q(\theta) for the outer loop of the EIG approximation. 8: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;if n==1n==1 then 9: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;for m=1:Mm=1:M do 10: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Draw parameter samples θn,m∼q(θ)\theta_{n,m}\sim q(\theta) for the inner loop of the EIG approximation. 11: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;end for 12: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;else 13: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Set θn,:=θ1,:\theta_{n,:}=\theta_{1,:} 14: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;end if 15: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;end if 16: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Simulate data sample yny_n from the data distribution yn∼f(y∣θn,0,ξi)y_n\sim f(y\mid\theta_{n,0},\xi_i). 17: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;end for 18: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Estimate EIG^MC(ξi)\widehat{EIG}_{MC}(\xi_i) via (15). 19: &nbsp;&nbsp;&nbsp;&nbsp;end for 20: &nbsp;&nbsp;&nbsp;&nbsp;ξt=arg⁡max⁡ξ∈ΞEIG^MC(ξ)\xi_t=\arg\max_{\xi\in\Xi}\widehat{EIG}_{MC}(\xi). 21: &nbsp;&nbsp;&nbsp;&nbsp;Produce data yty_t. 22: &nbsp;&nbsp;&nbsp;&nbsp;Calculate π(θ∣y1:t,ξ1:t)∝f(y1:t∣θ,ξ1:t)π(θ)\pi(\theta\mid y_{1:t},\xi_{1:t})\propto f(y_{1:t}\mid\theta,\xi_{1:t})\pi(\theta) via (6) and (8). 23: &nbsp;&nbsp;&nbsp;&nbsp;Set q(θ)=π(θ∣y1:t,ξ1:t)q(\theta)=\pi(\theta\mid y_{1:t},\xi_{1:t}). 24: end for 25: Output: Approximation of the posterior π(θ∣y1:T,ξ1:T)\pi(\theta\mid y_{1:T},\xi_{1:T}).

Exemplar 1 Supplementary Material

The complete results for all 12 parameter and design settings after the final load step are recorded in Table B.1. Along with the true parameter values, the posterior summaries averaged over all 100 repeat trials are presented. Expected values Eθ∣y,ξE_{\theta\mid y,\xi}, posterior variances Vθ∣y,ξV_{\theta\mid y,\xi}, equal-tailed 95% credible intervals (CI)—calculated from quantiles of the posterior—the mean absolute percentage error of the MAP estimate MAPE⁡(θ^)\operatorname{MAPE}(\hat{\theta}) and the coefficient of variation of each parameter (Cv) are recorded. The MAPE⁡(θ^)\operatorname{MAPE}(\hat{\theta}) was obtained by calculating the normalized difference between the true and estimated parameter values, MAPE⁡(θ^)=(∣θdtrue−θ^d∣/θdtrue)×100\operatorname{MAPE}(\hat{\theta}) = (|\theta^{true}_{d}-\hat{\theta}_{d}|/\theta^{true}_{d}) \times100, d=1,…,Dd = 1,\ldots,D. The coefficient of variation is a standardized and unitless measure of variation, defined by the ratio of the standard deviation and the mean. It is useful in comparing the degree of variation among different parameters, especially when they have magnitudes on different scales as in Exemplar 2. In all 4 parameter cases as well as for the adaptive and both static designs, the MAPE⁡(θ^)\operatorname{MAPE}(\hat{\theta}) was less than a quarter of a percent, variances were on the order of 1×10−41 \times10^{-4} to 1×10−51 \times10^{-5}, and the credible intervals contained the true values. The coefficient of variation was nearly equal for FF and GG in all cases in the adaptive setting, notably greater for FF than GG in the static ε11\varepsilon_{11} setting and vice versa in the ε22\varepsilon_{22} setting. There was no notable difference in the marginal summaries among the different parameter cases.

Case No.DesignParameterθtrue\theta^{true}Eθ∣y,ξE_{\theta\mid y,\xi}Vθ∣y,ξV_{\theta\mid y,\xi}95% CIMAPE⁡(θ^)\operatorname{MAPE}(\hat{\theta})Cv
1AdaptiveFF0.550.54966.75×10−56.75 \times10^{-5}(0.534, 0.566)0.0640.015
1GG0.450.45044.57×10−54.57 \times10^{-5}(0.437, 0.464)0.0810.015
1Static ε11\varepsilon_{11}FF0.550.55041.11×10−41.11 \times10^{-4}(0.530, 0.571)0.0760.019
1GG0.450.44993.94×10−53.94 \times10^{-5}(0.438, 0.462)0.0200.014
1Static ε22\varepsilon_{22}FF0.550.55035.32×10−55.32 \times10^{-5}(0.536, 0.565)0.0460.013
1GG0.450.44999.03×10−59.03 \times10^{-5}(0.431, 0.469)0.0260.021
2AdaptiveFF0.600.59968.38×10−58.38 \times10^{-5}(0.582, 0.618)0.0700.015
2GG0.500.50055.69×10−55.69 \times10^{-5}(0.486, 0.515)0.0900.015
2Static ε11\varepsilon_{11}FF0.600.60051.44×10−41.44 \times10^{-4}(0.577, 0.624)0.0810.020
2GG0.500.49994.82×10−54.82 \times10^{-5}(0.486, 0.514)0.0200.014
2Static ε22\varepsilon_{22}FF0.600.60036.43×10−56.43 \times10^{-5}(0.585, 0.616)0.0470.013
2GG0.500.49991.18×10−41.18 \times10^{-4}(0.479, 0.521)0.0220.022
3AdaptiveFF0.600.59969.04×10−59.04 \times10^{-5}(0.581, 0.618)0.0710.016
3GG0.600.60058.01×10−58.01 \times10^{-5}(0.583, 0.618)0.0880.015
3Static ε11\varepsilon_{11}FF0.600.60051.68×10−41.68 \times10^{-4}(0.575, 0.626)0.0900.022
3GG0.600.59996.71×10−56.71 \times10^{-5}(0.584, 0.616)0.0170.014
3Static ε22\varepsilon_{22}FF0.600.60036.72×10−56.72 \times10^{-5}(0.584, 0.616)0.0510.014
3GG0.600.59991.67×10−41.67 \times10^{-4}(0.575, 0.625)0.0240.022
4AdaptiveFF0.690.68861.05×10−41.05 \times10^{-4}(0.668, 0.709)0.2110.015
4GG0.430.43084.65×10−54.65 \times10^{-5}(0.417, 0.444)0.1840.016
4Static ε11\varepsilon_{11}FF0.690.68851.73×10−41.73 \times10^{-4}(0.663, 0.714)0.2210.019
4GG0.430.43053.91×10−53.91 \times10^{-5}(0.418, 0.443)0.1280.015
4Static ε22\varepsilon_{22}FF0.690.68958.17×10−58.17 \times10^{-5}(0.672, 0.707)0.0700.013
4GG0.430.43051.02×10−41.02 \times10^{-4}(0.411, 0.450)0.1140.024

Table B.1. Posterior summaries at the final step for all 12 parameter and design combinations in Exemplar 1. The posterior expected value (Eθ∣y,ξE_{\theta\mid y,\xi}), variance (Vθ∣y,ξV_{\theta\mid y,\xi}), 95% credible intervals (CI), MAPE of the MAP probability estimate and coefficient of variation (Cv) were averaged over all 100 trials.

Table B.2 shows the scalar posterior summaries of uncertainty with the total and generalized variance. The Mahalanobis distance (MD) of θtrue\theta^{true}, which measures the distance of the true parameter values from the posterior distribution, is also included. Values reported were averaged over all 100 repeat trials and were calculated from the posterior distribution after the final load step. Similar to the results for Case 1 shown in Figure 7, in all cases, both the total and generalized variance are smaller for the adaptive design compared to either of the static designs, showing that a load path determined adaptively provides a greater reduction in uncertainty in a holistic sense when all parameters are considered simultaneously.

Case No.DesignTot. Var. (1×10−41 \times10^{-4})Gen. Var. (1×10−91 \times10^{-9})MD(θtrue\theta^{true})
1Adaptive1.131.721.25
Static ε11\varepsilon_{11}1.432.371.26
Static ε22\varepsilon_{22}1.512.211.26
2Adaptive1.412.841.24
Static ε11\varepsilon_{11}1.933.761.26
Static ε22\varepsilon_{22}1.824.001.26
3Adaptive1.714.581.24
Static ε11\varepsilon_{11}2.346.381.26
Static ε22\varepsilon_{22}2.346.381.26
4Adaptive1.522.921.17
Static ε11\varepsilon_{11}2.123.691.13
Static ε22\varepsilon_{22}1.844.361.19

Table B.2. Scalar posterior summaries for Exemplar 1. Values reported are average over 100 trials.

Exemplar 2 Supplementary Material

Table C.1 contains marginal posterior summaries for Exemplar 2, where θ=[F,G,A,n,σy]T\boldsymbol{\theta} = [F,G,A,n,\sigma_y]^T. Posterior expected values (Eθ∣y,ξ\mathbb{E}_{\boldsymbol{\theta}\mid y,\boldsymbol{\xi}}), variances (Vθ∣y,ξ\mathbb{V}_{\boldsymbol{\theta}\mid y,\boldsymbol{\xi}}), 95% credible intervals (CIs), the mean absolute percentage error of the MAP estimate (MAPE⁡(θ^)\operatorname{MAPE}(\hat{\boldsymbol{\theta}})) and the coefficient of variation (Cv) were averaged over the 100 repeat trials for each parameter case and design option and were calculated after the final load step. In Case 5, θtrue\boldsymbol{\theta}^{true} is contained within the 95% CIs for both the adaptive and static designs. The marginal variances reveal that the parameter uncertainty was lower in the adaptive setting than the static ones for all parameters except for FF when compared to the ε11\varepsilon_{11} design and GG when compared to the ε22\varepsilon_{22} design—where the static designs shown a slightly lower average uncertainty. The most significant difference was in σy\sigma_y, where the adaptive design on average had a variance of 6.98 and the static designs had variances of 247.59 and 180.90. In Case 6, θtrue\boldsymbol{\theta}^{true} is contained within the 95% CIs for both the adaptive and static designs, but in the static designs, some of the parameter 95% CIs went outside the bounds defined in Table 1. Some trials for the static ε11\varepsilon_{11} design in Case 6 were unsuccessful in identifying the model parameters given the data. Marginal posterior values with the unsuccessful trials removed from the average values are reported in Table C.2.

Case No.DesignParameterθtrue\boldsymbol{\theta}^{true}Eθ∣y,ξ\mathbb{E}_{\boldsymbol{\theta}\mid y,\boldsymbol{\xi}}Vθ∣y,ξ\mathbb{V}_{\boldsymbol{\theta}\mid y,\boldsymbol{\xi}}95% CIMAPE⁡(θ^)\operatorname{MAPE}(\hat{\boldsymbol{\theta}})Cv
5AdaptiveFF0.550.54963.46×10−53.46 \times10^{-5}(0.538, 0.561)0.0750.011
5AdaptiveGG0.450.44992.69×10−52.69 \times10^{-5}(0.440, 0.460)0.0260.012
5AdaptiveAA100.0100.2234.96(95.86, 104.59)0.0220.022
5Adaptivenn20.020.171.56(17.73, 22.62)0.8660.062
5Adaptiveσy\sigma_y300.0299.656.98(294.48, 304.83)0.1150.009
5Static ε11\varepsilon_{11}FF0.550.55011.63×10−51.63 \times10^{-5}(0.542, 0.558)0.0250.007
5Static ε11\varepsilon_{11}GG0.450.46025.18×10−35.18 \times10^{-3}(0.321, 0.599)2.2770.157
5Static ε11\varepsilon_{11}AA100.0100.8624.23(91.28, 110.45)0.8620.049
5Static ε11\varepsilon_{11}nn20.020.251.97(17.52, 22.99)1.2610.069
5Static ε11\varepsilon_{11}σy\sigma_y300.0301.27247.59(270.68, 331.85)0.4230.052
5Static ε22\varepsilon_{22}FF0.550.54544.33×10−34.33 \times10^{-3}(0.417, 0.673)0.8310.122
5Static ε22\varepsilon_{22}GG0.450.44961.29×10−51.29 \times10^{-5}(0.443, 0.457)0.0960.008
5Static ε22\varepsilon_{22}AA100.099.6318.36(91.26, 107.99)0.3720.043
5Static ε22\varepsilon_{22}nn20.020.011.90(17.31, 22.70)0.0320.069
5Static ε22\varepsilon_{22}σy\sigma_y300.0298.60180.90(272.50, 324.70)0.4670.045
6AdaptiveFF0.550.55014.72×10−54.72 \times10^{-5}(0.537, 0.564)0.0250.012
6AdaptiveGG0.450.44973.87×10−53.87 \times10^{-5}(0.438, 0.462)0.0650.014
6AdaptiveAA300.0300.0288.30(294.38, 305.68)0.0090.010
6Adaptivenn20.019.950.22(19.04, 20.86)0.2610.023
6Adaptiveσy\sigma_y100.0100.136.87(95.00, 105.27)0.1340.026
6Static ε11\varepsilon_{11}FF0.550.54964.05×10−44.05 \times10^{-4}(0.534, 0.565)0.0710.014
6Static ε11\varepsilon_{11}GG0.450.50344.48(−0.431, 1.438)11.8780.755
6Static ε11\varepsilon_{11}AA300.0309.601.59×1051.59 \times10^{5}(129.77, 489.44)3.2000.270
6Static ε11\varepsilon_{11}nn20.020.74844.60(7.68, 33.80)3.7190.292
6Static ε11\varepsilon_{11}σy\sigma_y100.0102.579.71×1039.71 \times10^{3}(54.68, 150.47)2.5730.223
6Static ε22\varepsilon_{22}FF0.550.54211.09×10−21.09 \times10^{-2}(0.357, 0.727)1.440.193
6Static ε22\varepsilon_{22}GG0.450.44961.79×10−51.79 \times10^{-5}(0.441, 0.458)0.0840.009
6Static ε22\varepsilon_{22}AA300.0297.31446.59(260.50, 334.13)0.8960.065
6Static ε22\varepsilon_{22}nn20.019.862.21(17.21, 22.50)1.2740.068
6Static ε22\varepsilon_{22}σy\sigma_y100.098.7352.06(86.11, 111.34)1.2740.068

Table C.1. Posterior summaries at the final step for all 6 parameter and design combinations in Exemplar 2. The posterior expected value (Eθ∣y,ξ\mathbb{E}_{\boldsymbol{\theta}\mid y,\boldsymbol{\xi}}), variance (Vθ∣y,ξ\mathbb{V}_{\boldsymbol{\theta}\mid y,\boldsymbol{\xi}}), 95% credible intervals (CI), MAPE of the MAP probability estimate and coefficient of variation (Cv) were averaged over all 100 trials.

Table C.3 shows the scalar posterior summaries of uncertainty with the total and generalized variance and the Mahalanobis distance (MD) of θtrue\boldsymbol{\theta}^{true}. Values reported were averaged over all 100 repeat trials and were calculated from the posterior distribution after the final load step. Scalar posterior summaries with the unsuccessful trials in the Case 6 static ε11\varepsilon_{11} design removed are reported in Table C.4.

Case No.DesignParameter θtrue\theta^{true}Eθ∣y,ξE_{\theta\mid y,\xi}Vθ∣y,ξV_{\theta\mid y,\xi}95% CIMAPE(θ^)(\hat{\theta})CVC_V
6Static ε11\varepsilon_{11}FF0.550.54962.16×10−52.16 \times10^{-5}(0.541, 0.559)0.0690.008
6Static ε11\varepsilon_{11}GG0.450.49531.52×10−21.52 \times10^{-2}(0.285, 0.706)10.0720.229
6Static ε11\varepsilon_{11}AA300.0307.87636.26(264.60, 351.15)2.6240.072
6Static ε11\varepsilon_{11}nn20.020.633.26(17.50, 23.76)3.1580.077
6Static ε11\varepsilon_{11}σy\sigma_y100.0102.1170.22(87.41, 116.80)2.1050.074

Table C.2. Average Case 6 marginal posterior summaries for the ε11\varepsilon_{11} static design with the unsuccessful trials removed.

Case No.DesignTot. Var.Gen. Var. (1×10−91 \times10^{-9})MD⁡(θtrue)\operatorname{MD}(\theta^{true})
5Adaptive13.501.202.14
5Static ε11\varepsilon_{11}273.80160.003.00
5Static ε22\varepsilon_{22}201.17116.242.75
6Adaptive15.390.362.21
6Static ε11\varepsilon_{11}170,0136.68×1046.68 \times10^{4}5.13
6Static ε22\varepsilon_{22}500.8766.125.73

Table C.3. 1D posterior summaries for Exemplar 2. Values reported are average over 100 trials.

Case No.DesignTot. Var.Gen. Var. (1×10−91 \times10^{-9})MD⁡(θtrue)\operatorname{MD}(\theta^{true})
6Static ε11\varepsilon_{11}709.76150.575.00

Table C.4. Average Case 6 1D posterior summaries for the ε11\varepsilon_{11} static design with the unsuccessful trials removed.

Posterior uncertainty was propagated to the model output in Case 6 from Exemplar 2 and 95% CIs are plotted in Fig. C.1. The left column of plots show CIs for the trial in each design setting with the lowest amount of total variance, and the right column of plots show the trial with the highest level of total variance. For the adaptive design, the minimum total variance came from a trial with an optimal design of ξ∗=[ε11,ε22,ε11,ε22,ε11,ε22,ε11]\xi^{*} = [\varepsilon_{11}, \varepsilon_{22}, \varepsilon_{11}, \varepsilon_{22}, \varepsilon_{11}, \varepsilon_{22}, \varepsilon_{11}], and the maximum total variance came from a trial with an optimal design of ξ∗=[ε22,ε11,ε22,ε11,ε22,ε11,ε11]\xi^{*} = [\varepsilon_{22}, \varepsilon_{11}, \varepsilon_{22}, \varepsilon_{11}, \varepsilon_{22}, \varepsilon_{11}, \varepsilon_{11}]. The narrow CIs in the adaptive design for both the minimum and maximum case indicate that the adaptive design consistently resulted in parameter inference with a low degree of uncertainty across the 100 trials. In contrast, the CIs in the static settings varied greatly from the minimum to the maximum total variance trials, indicating that reliable parameter inference was not consistent across the 100 trials.

Image not extracted

Figure C.1. 95% credible intervals for the posterior draws θ~∼π(θ∣y,ξ)\tilde{\theta} \sim\pi(\theta\mid y, \xi) were calculated point-wise for Exemplar 2, Case 6 and are indicated the the blue (σ11\sigma_{11}) and red (σ22\sigma_{22}) dashed lines. The model output from the true parameter values is plotted with a black line. The left column of plots show the 95% CI for the trial that had a posterior distribution with the least total variance among all trials for each design setting. The right column of plots show 95% CI for the trial that had the greatest total variance. Plots (a) and (b) show results for the adaptive design, plots (c) and (d) show results for the static ϵ11\epsilon_{11} design and plots (e) and (f) for the static ϵ22\epsilon_{22} design.

References

  1. [64]Luke Tierney and Antonietta Mira. Some adaptive monte carlo methods for bayesian inference. Statistics in medicine, 18(17-18):2507–2515, 1999.
  2. [65]Peter J Green and Antonietta Mira. Delayed rejection in reversible jump metropolis–hastings. Biometrika, 88(4):1035–1053, 2001.DOI
  3. [66]Xun Huan. Accelerated Bayesian experimental design for chemical kinetic models. PhD thesis, Massachusetts Institute of Technology, 2010.
  4. [67]Joshua Eckels, Collin B Whittaker, Benjamin Jorns, and Alex Gorodetsky. Optimal experimental design to learn reduced-fidelity models for porous electrosprays. In AIAA SCITECH 2023 Forum, page 0066, 2023.DOI
  5. [68]Takashi Goda, Tomohiko Hironaka, and Takeru Iwamoto. Multilevel monte carlo estimation of expected information gains. Stochastic Analysis and Applications, 38(4):581–600, 2020.DOI
  6. [69]Chi Feng et al. Optimal Bayesian experimental design in the presence of model error. PhD thesis, Massachusetts Institute of Technology, 2015.
  7. [70]Adam Foster, Martin Jankowiak, Elias Bingham, Paul Horsfall, Yee Whye Teh, Thomas Rainforth, and Noah Goodman. Variational bayesian optimal experimental design. Advances in Neural Information Processing Systems, 32, 2019.arxiv.org/abs/1903.05480

Paper details

vlnc:W_25xath4e
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 6

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents