betaFeedback

WorksEngineering

HARDCORE: H-field and power loss estimation for arbitrary waveforms with residual, dilated convolutional neural networks in ferrite cores

Abstract

The MagNet Challenge 2023 calls upon competitors to develop data-driven models for the material-specific, waveform-agnostic estimation of steady-state power losses in toroidal ferrite cores. The following HARDCORE (H-field and power loss estimation for Arbitrary waveforms with Residual, Dilated convolutional neural networks in ferrite COREs) approach shows that a residual convolutional neural network with physics-informed extensions can serve this task efficiently when trained on observational data beforehand. One key solution element is an intermediate model layer which first reconstructs the bh curve and then estimates the power losses based on the curve's area rendering the proposed topology physically interpretable. In addition, emphasis was placed on expert-based feature engineering and information-rich inputs in order to enable a lean model architecture. A model is trained from scratch for each material, while the topology remains the same. A Pareto-style trade-off between model size and estimation accuracy is demonstrated, which yields an optimum at as low as 1755 parameters and down to below 8,% for the 95-th percentile of the relative error for the worst-case material with sufficient samples.

Introduction

The MagNet Challenge 2023 is tackled with a material-agnostic residual convolutional neural network (CNN) topology with physics-informed extensions in order to leverage domain knowledge. Topological design decisions are dictated by peculiarities found in the data sets and by the overall goal of maximum estimation accuracy at minimum model sizes.

The topology’s central idea is the calculation of the area within the bhbh polygon based on a preceding hh sequence estimate, see Figure 3. The area within the polygon formed by the sequences b,h∈R1024b, h \in\mathbb{R}^{1024} can be calculated using the shoelace formula or surveyor’s area formula [1]. The shoelace method assigns a trapezoid to each edge of the polygon as depicted in Figure 1. The area of these trapezoids is defined according to shoelace either with a positive or negative sign, according to the hysteresis direction. The negative areas compensate for the parts of positive trapezoids that extend beyond the boundaries of the polygon. Provided that the polygon is shifted into the first quadrant by some offsets hosh_{\mathrm{os}} and bosb_{\mathrm{os}}, the power loss in W m−3\mathrm{W\,m^{-3}} caused by magnetic hysteresis effects can be computed with the frequency ff, M=1024M = 1024, and circular padding by

Visualization of the shoelace formula applied to a $bh$ polygon

Figure 1. Visualization of the shoelace formula applied to a bhbh polygon.

Image not extracted

Figure 3. Overview of the physics-inspired HARDCORE modeling toolchain.

p^hyst=f⋅12∑i=0M−1bi(hi−1−hi+1).(1)\hat{p}_{\mathrm{hyst}} = f \cdot\frac{1}{2} \sum_{i=0}^{M-1} b_i(h_{i-1}-h_{i+1}). \tag*{(1)}

When applying (1) on the given sequences b,h∈RMb,h \in\mathbb{R}^{M} with M=1024M = 1024, it becomes evident that the calculated area does not equal the provided loss measurements exactly. Figure 2 shows the discrepancy with respect to the provided scalar loss pp for all materials. The relative error ranges up to over 7 % for certain materials (e.g. 3F4, N49, D, E). Consequently, if merely an hh-predicting model was to be identified, the lower bound on the rel. error would be significantly elevated by this circumstance alone.

Relative error histograms

Figure 2. Relative error (p^hyst−p)/p(\hat{p}_{\mathrm{hyst}}-p)/p histogram between provided scalar pp and p^hyst\hat{p}_{\mathrm{hyst}} calculated from the likewise provided bhbh polygon area.

Since the power losses calculated from neither the ground truth bhbh curve area (assuming ideal knowledge on the hh sequence) nor the estimated area (h^\hat{h} reconstructed via a CNN) do perfectly match the provided loss measurement values (targets), an additional residual correction mechanism is added to compensate for this. A high-level view on the proposed residual, physics-inspired modeling toolchain is depicted in Figure 3, which is coined the HARDCORE approach (H-field and power loss estimation for Arbitrary waveforms with Residual, Dilated convolutional neural networks in ferrite COREs), and its details are discussed in the following.

Model description

A residual CNN with physics-informed extensions is utilized for all materials. Such a CNN is trained for each material from scratch. Yet, the topology is unaltered across materials, signal waveforms, or other input data particularities.

One-dimensional CNNs for h-estimation

A 1D CNN is the fundamental building block in this contribution, which consists of multiple trainable kernels or filters per layer slided over the multi-dimensional input sequence in order to produce an activation on the following layer [2]. These activations denote the convolution (more precisely, the cross-correlation) between the learnable kernels and the previous layer’s activation (or input sequence). In this stateless architecture, circular padding ensures that subsequent activation maps are of equal size. Circular padding can be utilized here instead of the common zero-padding as sequences denote complete periods of the bb and hh curve during steady state. Moreover, a kernel does not need to read strictly adjacent samples in a sequence at each point in time, but might use a dilated view, where samples with several samples in between are used. The dilated, temporal CNN update equation for the ii-th filter’s activation ai(l)[k]a_i^{(l)}[k] at time kk and layer ll with the learnable coefficients Wi∈RA×κ\mathbf{W}_i \in\mathbb{R}^{A \times\kappa} applied on AA previous layer’s filters, an uneven kernel size of κ∈{2x+1:x∈N0}\kappa\in\{2x+1:x \in\mathbb{N}_0\}, and the dilation factor δ\delta reads

ai(l)[k]=∑p=0A−1∑j=−(κ−1)/2(κ−1)/2Wi;(p,j)⋅ap(l−1)[k+jδ].(2)a_i^{(l)}[k] = \sum_{p=0}^{A-1} \sum_{j=-(\kappa-1)/2}^{(\kappa-1)/2} W_{i;(p,j)} \cdot a_p^{(l-1)}[k+j\delta]. \tag*{(2)}

Since the task at hand does not require causality of CNN estimates along the time domain (losses are to be estimated from single bb sequences), the sliding operation can be efficiently parallelized, and sequential processing happens merely along the CNN’s depth. All 1D CNN layers are accompanied by weight normalization [3]. A conceptual representation of the 1D CNN for estimating h^\hat{h} is visualized in Figure 6 (left part).

Image not extracted

Figure 6. The residual 1D CNN topology is shown while applied on time series and scalar features, which also contain engineered features from Table I.

Feature engineering

The term feature engineering encompasses all preprocessing, normalization, and derivation of additional features in an observational data set. The input data contains the frequency ff, the temperature TT, the measured losses pp as well as the 1024 sample points for the bb and hh waveforms. Especially the creation of new features that correlate as much as possible with the target variable (here, the hh curve or the scalar power loss pp) is an important part of most machine learning (ML) frameworks [4].

Normalization

1) Normalization: As is typical in neural network training, all input and target features have to be normalized beforehand. All scalar and time series features are divided by their maximum absolute value that occurs in the material-specific data set, with the exception of the temperature and the frequency, which will be divided by 75 ∘C75\,^\circ\mathrm{C} and 150 kHz150\,\mathrm{kHz}, respectively, regardless of the material. Moreover, for an accurate hh estimate, it was found to be of paramount importance to normalize each bb and hh curve again on a per-profile base in dependence not only on the ℓ∞\ell_\infty norm of ∣b∣|b|, but also on the maximum absolute bb and hh appearing in the entire material-specific data set. The latter two values are denoted blim⁡b_{\lim} and hlim⁡h_{\lim}, and can be understood as material-specific scaling constants. In particular, the per-profile normalized bb and hh curves for a certain sample read

bn=bmax⁡k∣b[k]∣,hn=hhlim⁡⋅blim⁡max⁡k∣b[k]∣,(3)b_{\mathrm{n}} = \frac{b}{\max_k |b[k]|}, \qquad h_{\mathrm{n}} = \frac{h}{h_{\lim}} \cdot\frac{b_{\lim}}{\max_k |b[k]|}, \tag*{(3)}

with hlim⁡=max⁡i,k∣hi[k]∣h_{\lim} = \max_{i,k}|h_i[k]|, blim⁡=max⁡i,k∣bi[k]∣b_{\lim} = \max_{i,k}|b_i[k]|, and ii being the sample index in the entire material-specific data set. Then, bnb_{\mathrm{n}} is added to the set of input time series features, and hnh_{\mathrm{n}} is the target variable for the hh estimation task.

The bnb_{\mathrm{n}} over hnh_{\mathrm{n}} curves are displayed in Figure 4, which underlines how the polygon area becomes roughly unified (no large area difference between samples). In the following, all features that get in touch with the model are normalized values without any further notational indication.

Image not extracted

Figure 4. Exemplary samples of the normalized bb and hh curves.

Time series features (feature engineering #1)

2) Time series features (feature engineering #1): As discussed in Sec. II-A, 1D CNNs build the core of the implemented model. The inputs to the CNNs are the (per-profile) normalized magnetic flux density bnb_{\mathrm{n}} and the corresponding first and second order derivatives (b˙n\dot{b}_{\mathrm{n}} and b¨n\ddot{b}_{\mathrm{n}}) as time series. In a macroscopic measurement circuit context, bb corresponds to the applied magnetizing voltage throughout the measurement process of the data. Accordingly, b¨\ddot{b} represents the voltage slew rate during the commutation of the switches in the test setup. Consequently, the second derivative allows to detect switching events and to characterize them according to their maximum slew rate. Figure 5 shows, that the sinusoidal waveform (green) is generated without any fast transient switching behaviour, probably with a linear signal source. The nonsinusoidal examples show typical switching behaviour with different voltage slew rates during the single transitions and voltage overshoots as well as ringing. The second derivative of bb informs the ML model about switching transition events and how fast changes in time are.

Image not extracted

Figure 5. Magnetic flux density examples and their first and second order derivatives for a sinusoidal, triangular and one unclassified waveform with a circuit-based interpretation in terms of their proportionality to magnetic flux, voltage and the voltage slew rate.

Scalar features (feature engineering #2)

Although sequence-based CNNs take up the main share of the ML model size, scalar environmental variables also have a considerable impact on hh and pp. While the temperature TT is passed to the model unaltered (but normalized), the frequency is presented by its logarithm ln⁡(f)\ln(f). The sample time 1/f1/f is passed directly to the model. Furthermore, some bb-derived scalar features are also passed to the model to feed in a priori knowledge. For example, the peak-to-peak magnetic flux Δb\Delta b as well as the mean absolute time derivative ∣b˙∣|\dot{b}| are directly fed into the network. Each waveform is automatically classified into ”sine”, ”triangular”, ”trapezoidal”, and ”other” by consulting the form and crest factors, as well as some Fourier coefficients. The waveform classification is presented to the model by one hot encoding (OHE). A summary of all expert-driven input features is presented in Table I.

Time series featuresScalar features
mag. flux densitybbtemperatureTT
per-profile norm.bnb_nsample time1/f1/f
1st derivativeb˙n\dot{b}_nlog-frequencyln⁡(f)\ln(f)
2nd derivativeb¨n\ddot{b}_npeak2peakΔb\Delta b
tan-tan-btan⁡(0.9⋅tan⁡(bn))\tan(0.9 \cdot\tan(b_n))log peak2peakln⁡(Δb)\ln(\Delta b)
mean abs dbdt∣b˙∣|\dot{b}|
log mean abs dbdtln⁡(∣b˙∣)\ln(|\dot{b}|)
waveform (OHE)

Table I. UTILIZED INPUT FEATURES.

Residual correction and overall topology

The model topology comprises multiple branches that end in the scalar power loss estimate p^\hat{p}. An overview is sketched in Figure 6. Two main branches can be identified: an hh-predictor

and a wrapping pp-predictor. The hh-predictor utilizes both time series and scalar features with CNNs and multilayer perceptrons (MLPs), and estimates the full hh sequence. The pp-predictor predicts pp, on the other hand, and leverages the predicted magnetic field strength h^\hat{h} with the shoe-lace formula and a scaling factor. The latter accounts for the losses inexplicable by the bhbh-curve (recall Figure 2), and is predicted by a MLP that utilizes the scalar feature set only.

The hh-predictor merges time series and scalar feature information by the broadcasted addition of its MLP output to a part of the first CNN layer output. This effectively considers the MLP-transformed scalar features as bias term to the time-series-based CNN structure.

On the merged feature set, two further 1D CNN layers follow that end the transformation in a 1024-element sequence. The per-profile scaled bnb_n sequence from the set of input time series is element-wise added to this newly obtained estimation (residual connection). This results in the CNN model to merely learn the difference between hnh_n and bnb_n [5]. Eventually, this sequence becomes the hh estimation h^n\hat{h}_n when the sequence’s average along the time domain, that is, across all 1024 elements, is subtracted from each element. This is a physics-informed intervention in order to ensure a bias-free hh estimate h^\hat{h} after denormalization. Note that all such operations are still end-to-end differentiable with an automatic differentiation framework such as PyTorch [6, 7].

Since the resulting h^\hat{h} can only be trained to be as close as possible to the provided hh sequence, which is not leading to the correct pp ground truth (cf. Figure 2), another MLP is branched off the scalar input feature set, and denotes the start of the pp-predictor. This MLP inputs two hidden layers and concludes with a single output neuron. This neuron’s activation, however, is not p^\hat{p} but rather an area scaling factor s∈[−1,1]s \in[-1,1] to be embedded in the shoelace formula (1) with

p^=f⋅(0.5+(0.1⋅s))∑i=0M−1bi(h^i−1−h^i+1).(4)\hat{p}=f\cdot(0.5+(0.1\cdot s))\sum_{i=0}^{M-1}b_i\left(\hat{h}_{i-1}-\hat{h}_{i+1}\right). \tag*{(4)}

Consequently, the pp-predictor branch can alter the shoelace formula result by up to 10 % in positive and negative direction. The pp-predictor can be justified physically, when referring to Figure 2 again. As the comparison shows, the hysteresis loss represents the total loss within a variation of −10-10 % to +5+5 %. The positive deviation (p^hyst>p\hat{p}_{\mathrm{hyst}}>p) indicates some measurement discrepancy between the measured loss and the given bb and hh curves. For parts of the negative deviations, a physical explanation can be found in eddy current losses, related to a high dielectric constant and non-zero conductivity. Due to the small thickness of the used toroidal cores and the limited excitation frequency, eddy current losses are assumed to be of minor effect.

The physical interpretability of the intermediate estimate h^\hat{h} is a key advantage of the HARDCORE approach: First, it enables utilizing full hh time series simulation frameworks (e.g., time domain FEM solvers). Secondly, for future designs of magnetic components with arbitrary shapes it becomes indispensable to accurately take into account also geometrical parameters of the core. This is only possible by distinguishing between the magnetic hysteresis and the (di-)electric losses.

Training cost functions

The training process involves two cost functions for a training data set with size NN: First, the hh estimation accuracy, which is assessed with the mean squared error (MSE) as

LMSE,H=1NM∑n=0N−1∑i=0M−1(h^i,n−hi,n)2.(5)\mathcal{L}_{\mathrm{MSE,H}} = \frac{1}{NM}\sum_{n=0}^{N-1}\sum_{i=0}^{M-1}(\hat{h}_{i,n}-h_{i,n})^2. \tag*{(5)}

Second, the power loss estimation accuracy is to be gauged. Despite the relative error being the competition’s evaluation metric, the mean squared logarithmic error (MSLE) is selected

LMSLE,P=1N∑n=0N−1(ln⁡p^n−ln⁡pn)2(6)\mathcal{L}_{\mathrm{MSLE,P}} = \frac{1}{N}\sum_{n=0}^{N-1}(\ln\hat{p}_n-\ln p_n)^2 \tag*{(6)}

in order to not overemphasize samples with a relatively low power loss [8]. As LMSLE,P\mathcal{L}_{\mathrm{MSLE,P}} also depends on h^\hat{h} through (4), the question arises, how both cost functions are to be weighted. In this contribution, a scheduled weighting is applied with

Ltotal=αLMSLE,P+(1−α)LMSE,H,(7)\mathcal{L}_{\mathrm{total}} = \alpha\mathcal{L}_{\mathrm{MSLE,P}} + (1-\alpha)\mathcal{L}_{\mathrm{MSE,H}}, \tag*{(7)}

where α=(β⋅iepoch)/Kepoch\alpha=(\beta\cdot i_{\mathrm{epoch}})/K_{\mathrm{epoch}} with β∈[0,1]\beta\in[0,1] denoting a hyperparameter scaling factor, KepochK_{\mathrm{epoch}} being the number of training epochs, and iepoch∈{0,1,…,Kepoch−1}i_{\mathrm{epoch}}\in\{0,1,\ldots,K_{\mathrm{epoch}}-1\} representing the current epoch index. The scheduled weighting ensures that the model focuses on h^\hat{h} in the beginning of the training, where more information is available. Later though, the model shall draw most of its attention to the power loss estimate, possibly at the expense of the hh estimation accuracy. A training example for material A is depicted in Figure 7.

Image not extracted

Figure 7. Exemplary training and validation loss curve for material A, seed 0 and fold 3.

Hyperparameters, Pareto front and results

The proposed topology features several degrees of freedom in form of hyperparameters. An important aspect is the model size, which is defined by the number of hidden layers and neurons in each layer. A simple trial-and-error investigation is can provide fast insights into the performance degradation that comes with fewer model parameters. In Figure 8, several particularly selected model topologies are illustrated against their achieved relative error versus the inherited model size. The scatter in each quantile is due to different random number generator seeds and folds during a stratified 4-fold cross-validation. Topology variations are denoted by the amount of neurons in certain hidden layers. In addition, the largest topology has an increased kernel size with κ=17\kappa= 17, and the smallest topology has the second CNN hidden layer removed entirely (the green layer in Figure 6).

Image not extracted

Figure 8. Pareto front for the evaluation materials (A, B, C, D and E) showing model size (amount of parameters) vs. relative error of the power loss estimation.

A slight degradation gradient is evident as of 5 k parameters for materials A and C, whereas for the other materials the trend is visible only when removing the second hidden layer. Overall, the material performance scales strictly with the amount of training data available. Since fewer model parameters are a critical aspect, the chosen final model has 1755 parameters, which is at an optimal trade-off point on the Pareto front.

In Table II, the model size of a corresponding PyTorch model file dumped to disk as just-in-time (jit) compilation is reported. The exemplary bhbh-curve and hh-curve estimation is shown in Figure 9. Reported error rates come from the best seed out of five during a four-fold cross-validation (β=1\beta= 1, Kepochs=5000K_{\mathrm{epochs}} = 5000, Nesterov Adam optimizer). It shows effectively that any material can be modeled with the same topology at high accuracy as long as a critical training data set size is available (which is not the case for material D, see available training data in Table II). The final model is already a trade-off between model size and accuracy, such that in case one of the two criteria can be softened, the other can be further improved.

MaterialParametersTraining dataModel sizeRelative error AverageRelative error 95-th quantile
A1755243243.13 kB2.34 %6.20 %
B1755740043.13 kB1.10 %2.68 %
C1755535743.13 kB1.46 %3.70 %
D175558043.13 kB7.03 %25.76 %
E1755201343.13 kB2.51 %7.10 %

Table II. FINAL MODEL DELIVERY OVERVIEW

Image not extracted

Figure 9. Exemplary ground truth vs. estimated bhbh curve comparison.

The final model delivery is trained on all training data samples (no repetitions with different seeds), and with Kepoch=10000K_{\mathrm{epoch}} = 10000, δ=4\delta= 4, κ=9\kappa= 9. This final topology features a CNN with 12(TanH) →\rightarrow 8(TanH) →\rightarrow 1 (linear) kernels, a MLP with 11(TanH) neurons, and a pp-predictor MLP with 8(TanH) →\rightarrow 1(TanH) neurons.

Conclusion

A material-agnostic CNN topology for efficient steady-state power loss estimation in ferrite cores is presented. Since the topology remains unaltered across materials and waveforms at a steadily high accuracy, the proposed model can be considered universally applicable to plenty of materials. As long as sufficient samples of a material are available (roughly, 2000), the relative error on the 95-th quantile remains below 8 %. Thus, the contributed method is proposed to become a standard way of training data-driven models for power magnetics.

References

  1. [1]B. Braden, “The surveyor’s area formula,” The College Mathematics Journal, vol. 17, no. 4, pp. 326–337, 1986.DOI
  2. [2]A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” in Advances in Neural Information and Processing Systems (NIPS), vol. 60, no. 6, 2012, pp. 84–90. [Online]. Available: https://doi.org/10.1145/3065386
  3. [3]T. Salimans and D. P. Kingma, “Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks,” ArXiv e-prints: 1602.07868 [cs.LG], 2016. [Online]. Available: http://arxiv.org/abs/1602.07868
  4. [4]P. Domingos, “A few useful things to know about machine learning,” Communications of the ACM, vol. 55, no. 10, p. 78, 2012. [Online]. Available: http://dl.acm.org/citation.cfm?doid=2347736.2347755DOI
  5. [5]S. Bai, J. Z. Kolter, and V. Koltun, “An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling,” 2018. [Online]. Available: https://arxiv.org/abs/1803.01271
  6. [6]A. Paszke, S. Gross et al., “PyTorch: An Imperative Style, High-Performance Deep Learning Library,” in Advances in Neural Information Processing Systems 32. Curran Associates, Inc., 2019, pp. 8024–8035.arxiv.org/abs/1912.01703
  7. [7]A. G. Baydin, B. A. Pearlmutter et al., “Automatic differentiation in machine learning: A survey,” Journal of Machine Learning Research, vol. 18, pp. 1–43, 2018. [Online]. Available: https://arxiv.org/abs/1502.05767
  8. [8]C. Tofallis, “A better measure of relative prediction accuracy for model selection and model estimation,” Journal of the Operational Research Society, vol. 66, pp. 1532–1362, 2015.

Paper details

vlnc:W_f7maf6vy
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Engineering

Not recorded for this paper

View statement
Looking for related papers…
Contents