betaFeedback

WorksComputer Science

A Temporal Convolutional Network with Multi-Head Attention Model: A Surrogate for Parallel DEVS Semiconductor Fab Simulation

Abstract

Semiconductor factories process many kinds of wafers organized into lots of various sizes and configurations. Different kinds of machines, each with its own operations, process wafers for hours to days, and the wafers undergo many incremental fabrication steps (with re-entry) and flows spanning days to weeks across connected machines. Fast and accurate predictions of throughput (THTH) and turnaround time (TATA) dynamics for individual machines and their combinations are essential for factory operations. To use machine learning methods for this purpose, this paper presents a Temporal Convolutional Network with Multi-head Attention (TCNA) architecture. It uses simulated datasets from a benchmark Parallel DEVS (PDEVS) semiconductor fabrication factory model. The factory processes configurable wafer lots with uniform and sinusoidal wafer arrival profiles and may undergo preventive maintenance. The wafer lot size, arrival profile, and factory maintenance policy are used as static covariates. TCNA is applied to a selected set of simulated discrete-event throughput and turnaround time trajectories. The THTH and TATA time series predicted by TCNA, evaluated with the common MAPEMAPE, MSEMSE, and R2R^2 measures, show comparable or higher accuracy relative to other model architectures, with similar parameter counts and epoch times; the TCNA model size and execution time are similar to those of other architectures such as the Temporal Fusion Transformer. Although time series predictions are less accurate than high-resolution simulations, they can be deployed at a much lower computational cost. The TCNA achieves orders-of-magnitude speedup over PDEVS simulation with an acceptable loss of accuracy, enabling frequent product scheduling, capacity planning, and maintenance planning.

Introduction

As stochastic, event-oriented systems, manufacturing factories involve many machines operating in mixed sequential and concurrent settings. In some factory segments, machines have only feedforward relationships, while in others they also have feedback relationships. Semiconductor factories have complicated dynamics and must operate under stringent time constraints on what machines do individually and collectively. Machines with different capabilities must accept a variety of wafer lots in different configurations and process them in different settings to produce different kinds of chips.

A key characteristic of semiconductor fabrication is the long duration from the time raw wafers enter the factory until chips are shipped. Even though manufacturing chips typically takes several weeks to a few months, individual machines are expected to operate efficiently, given their workload capacity and resource availability (wafer lots and machines), at time scales of hours to a few days. Therefore, every machine in the process must meet time constraints determined in part by its upstream and downstream machines. Two essential metrics for manufacturing are throughput (the rate at which wafers are processed) and turnaround time (the total time required for raw wafers to become chips).

Accurate and fast predictions of throughput and turnaround time for individual machines, and therefore for the whole factory, are of the utmost importance for efficient, cost-effective, and resilient factory operation. Over many years, models have been built using physics-based concepts, mathematical theories, methods, and tools. These models represent the structure and behavior of dynamical systems so that they can simulate the dynamics of components and their relationships with high accuracy and precision. Simulations of these models are integral to the development and operation of semiconductor manufacturing factories.

In recent years, researchers and practitioners have turned to machine learning (ML) as a means to understand factory dynamics by analyzing the large amounts of data routinely collected from individual machines. The high degree of automation used in smart manufacturing for individual machines and process flows allows high-resolution data collection at scale (e.g., [1, 2, 3]). However, such data sets are closely guarded and are therefore not available to other researchers. Instead, suitable simulation models can be developed and simulated to obtain synthetic data sets, as is done in this research. A benchmark model of a semiconductor fabrication factory [4] is used to develop discrete-event single-stage and multi-stage models. These models accept wafers of different types, configurations, and lot sizes, generated with uniform or sinusoidal arrival patterns. Machines can go offline (undergoing repair) for a finite period and then return to operation. The throughput and turnaround time trajectories are discrete-event and have a continuous time base. Wafers enter and exit machines at arbitrary time instants because the machines have stochastic dynamics. Such data sets can be converted to discrete-time trajectories for use in ML models [5].

This paper presents a Temporal Convolutional Network with Multi-head Attention (TCNA), a variant of the TCN architecture, for constructing time series models from simulation data sets obtained for a discrete-event semiconductor manufacturing factory model [5, 6]. A multi-head attention layer is used to obtain contextual information from historical values for predicting future values. The TCNA model predicts future values by taking into account the context of historical values along with the effects of static variables identified for the factory model, and it accurately reproduces the simulated throughput and turnaround time values.

The simulations are developed and evaluated under low, medium, and high wafer volumes and different operational settings (e.g., machines that are temporarily unavailable). The prediction accuracy of TCNA is compared with that of the Autoformer [7], FEDformer [8], LSTM [9], N-BEATS [10], TCN [11], and TFT [12] models. Owing to the simplicity of the TCNA architecture, our efficiency analysis shows that TCNA needs fewer parameters and less memory than the other attention- and transformer-based models. TCNA predicts the THT_H and TAT_A time trajectories with comparable or better mean absolute percentage error (MAPEMAPE), mean squared error (MSEMSE), and R2R^2 score values.

Semiconductor Manufacturing Factory Simulation Models

The single-stage and multi-stage semiconductor factory models used in this research are developed using the Parallel Discrete Event System Specification (PDEVS) formalism [13] and simulated using the DEVS-Suite simulator [14]. An 8-stage cascade MiniFab factory model, composed of single-stage components, generates the simulation scenarios detailed in § 5. Each single-stage model has diffusion, implantation, and lithography segments, as defined for the benchmark Intel semiconductor factory called MiniFab [4]. The diffusion stage receives raw wafers from three generators and partially processed wafers from the lithography stage. The implantation stage receives partially processed wafers from the diffusion and lithography stages. The lithography stage receives partially processed wafers from the implantation and diffusion stages. In the last step, the lithography stage completes the processing of the wafers received from the diffusion stage. The interval between two (or more) wafers arriving at (or leaving) a stage may range from minutes to multiple hours, which is a distinct characteristic of semiconductor factories.

The diffusion model has a dispatcher with a queue and two machines that independently perform the same loading, processing, unloading, and transportation of wafer batches. Similarly, the implantation and lithography models consist of components that receive, process, and forward wafers to other machines, each operating under specific timing constraints. Single-stage factories are connected in a cascade fashion to form multi-stage semiconductor fabrication factories. Wafers are organized into different batch configurations and processed through a six-step feedforward and feedback process encompassing the diffusion, implantation, and lithography stages. Discrete-event data from machines and dispatchers are collected by transducers without disrupting their operations. The time resolution for loading, processing, unloading, and transporting wafers is in minutes, with 10% stochasticity. The 8-stage factory model is created by cascading eight single-stage factories, where the output of each upstream stage becomes the input of the downstream stage. The detailed design, simulation, data collection, and analysis of the PDEVS semiconductor fabrication factory models are described in [5]. These models are extended and simulated for multivariate time series analyses in [6, 15].

Related Work

Discrete-event simulations (DES) are widely used to generate synthetic data when real-world data are limited or inaccessible [16]. For example, the single-stage MiniFab model has more than 70 simple and compound variable types (e.g., queues) [5]. Compared with data sets such as pedestrian counts and wind farms [17], semiconductor factory data have unique characteristics. As discussed in [6], simulation models play a vital role in time series analysis for manufacturing. Such synthetic data are strongly event-driven, requiring temporal ML models to consider contextual information for effective predictions [5]. In the domain of time series prediction, [18] describes a hybrid model combining recurrent neural networks (RNNs) and genetic algorithms (GAs) to predict electricity consumption, and [19] applies deep learning to traffic flow forecasting. Compared with the electricity and traffic domains, semiconductor manufacturing involves intricate wafer fabrication processes [20, 21], and its time series capture complex process flows influenced by multiple feedforward and feedback loops. Transformer- and convolution-based architectures have gained prominence for time series forecasting [15, 22]. Temporal Fusion Transformers (TFT) [12] effectively model static covariates, such as repair modes in semiconductor processes, and use multi-head attention to capture contextual dependencies. Although transformer models excel at long-term forecasting [23], their computational efficiency can be improved with convolutional operations [24]. Lightweight models such as SparseTSF [25] and SAMformer [26] demonstrate efficient adaptations of transformers, and ModernTCN [27] and iTransformer [28] further optimize temporal feature extraction. For discrete-event time series, where both local transitions and global periodicity are critical, combining TCN and attention mechanisms can enhance temporal modeling. Unlike [29], which integrates TCN and attention in parallel, our architecture applies attention to the output of a TCN layer and feeds the result into a second TCN layer for richer feature extraction. In addition, gating mechanisms, inspired by [30], dynamically control the flow of information and enhance predictive accuracy.

Approach

As mentioned in § 2, the THTH and TATA series of a manufacturing factory are long (30,000 to 40,000 time steps), and forecasting them also depends on the static covariates associated with each series. Therefore, the problem can be formulated as follows:

Given historical observations X={xt−L,…,xt}∈RL×NX = \{x_{t-L}, \ldots, x_t\} \in\mathbb{R}^{L \times N}, where NN is the number of variates and LL is the look-back window size, and a set of static covariates S={s1,…,sn}S = \{s_1, \ldots, s_n\}, n∈Nn \in\mathbb{N}, the aim is to predict MM future time steps Y={yt+1,…,yt+M}∈RM×NY = \{y_{t+1}, \ldots, y_{t+M}\} \in\mathbb{R}^{M \times N}.

Discrete-event-based forecasting: Figure 3 and Figure 4 highlight the importance of contextual dependencies in these time series. The THTH and TATA time series exhibit non-stationary, monotonically increasing trends, with sharp variations influenced by historical values. In semiconductor fabrication, the intervals during which a machine is unavailable depend on factors such as usage, maintenance, and repair time. This local context can be expressed as

Throughput prediction comparison plots for a medium lot size

Figure 3. THTH Prediction Comparison for Medium Lot Size

Turnaround time prediction comparison plots for a medium lot size

Figure 4. TATA Prediction Comparison for Medium Lot Size

Local Context=fl(Xt−Δ:t),\text{Local Context} = f_l(X_{t-\Delta:t}),

where Xt−Δ:tX_{t-\Delta:t} is the sequence of observations over the past Δ\Delta time steps. In the same manufacturing system, the frequency of breakdowns may depend on seasonal trends, such as increased usage during peak production or reduced maintenance frequency over long intervals. This global context can be expressed as

Global Context=fg(X),\text{Global Context} = f_g(X),

where XX captures a long history of observations. Static factors such as machine type or operating policies significantly affect both contexts and are modeled as static covariates S={s1,…,sn}S = \{s_1, \ldots, s_n\}, n∈Nn \in\mathbb{N}. Events occur only at specific times (e.g., machine breakdowns and maintenance events); hence, observations are non-uniform over time, and their spacing is critical. Therefore, given historical observations XX, static covariates SS, and a target prediction horizon MM, the objective is to find a function gg that minimizes the error L(Y^,Y)\mathcal{L}(\hat{Y},Y):

Y^=g(fg(X),fl(Xt−Δ:t),S).\hat{Y} = g(f_g(X), f_l(X_{t-\Delta:t}), S).

The loss function is the mean squared error:

L(Y^,Y)=1M∑i=1M∥y^t+i−yt+i∥2,\mathcal{L}(\hat{Y},Y) = \frac{1}{M}\sum_{i=1}^{M}\lVert\hat{y}_{t+i} - y_{t+i}\rVert^2,

where ∥⋅∥2\lVert\cdot\rVert^2 denotes the squared error between the predictions y^\hat{y} and the true values yy. A simple architecture that captures sharp changes in the time series while retaining historical context serves this goal. The throughput and turnaround time obtained from the manufacturing simulation can be modeled as

yt=f(xt,xt−1,…,xt−L;s1,s2,…,sn)+ϵty_t = f(x_t,x_{t-1},\ldots,x_{t-L};s_1,s_2,\ldots,s_n) + \epsilon_t

where ϵt\epsilon_t is a noise term. A rolling forecast is used to predict future values yty_t iteratively, appending each prediction to the forecast sequence.

TCN with Attention

TCNA combines dilated causal convolutions [11], which capture local temporal patterns such as step-to-step increments, with multi-head self-attention [31], which relates all positions of the look-back window and thus captures the trend across time. The architecture is shown in Figure 1 and is organized in four stages: the input embedding layer, the input TCN layer, the attention and gating layer, and the output layer. In contrast to residual formulations, which predict the change of the target from one step to the next, TCNA predicts the target values directly.

TCN with Multi-head Attention (TCNA) Model Architecture

Figure 1. TCN with Multi-head Attention (TCNA) Model Architecture. The dashed future-time branch is an optional input for known-future (dynamic) covariates; it is not used in the experiments of this paper (see § 7).

Input Embedding Layer: Let X∈RL×cX \in\mathbb{R}^{L \times c} be the past window, which holds the cc dynamic input channels of the LL most recent observations; for the univariate THTH or TATA series considered here, these channels contain the scaled target values. Let hh be the static embedding width. The static covariates ss (lot size, wafer types, wafer-arrival profile, and repair type) are embedded by a dense layer of width hh, semb=ReLU⁡(Wss+bs)∈Rhs_{emb} = \operatorname{ReLU}(W_s s + b_s) \in\mathbb{R}^{h}, repeated LL times, and concatenated with the past window:

Xcomb=[X∥1Lsemb]∈RL×(c+h),X_{comb} = [X \Vert\mathbf{1}_L s_{emb}] \in\mathbb{R}^{L \times(c+h)},

where ∥\Vert denotes concatenation and 1L\mathbf{1}_L is a length-LL vector of ones. Every time step therefore carries the configuration of the factory that produced it.

Input TCN Layer: A TCN with hh filters, kernel size κ\kappa, and dilations (1,2,4)(1,2,4) or (1,2,4,8)(1,2,4,8) is applied to XcombX_{comb} with causal padding, residual and skip connections, and returned sequences, so that the output keeps the sequence length LL. Let uu index positions in the input window, jj the output filters, dd a dilation factor, and ii the kernel offsets. With input vector xcomb,u\mathbf{x}_{comb,u} and filter vector wi,j\mathbf{w}_{i,j}, each causal convolution computes

hj(u)=ReLU⁡(∑i=0κ−1wi,jTxcomb,u−id+bj),j=1,…,h,h_j(u) = \operatorname{ReLU}\left(\sum_{i=0}^{\kappa-1}\mathbf{w}_{i,j}^{\mathsf{T}}\mathbf{x}_{comb,u-id} + b_j\right), \quad j=1,\ldots,h,

where hj(u)h_j(u) is the output of filter jj at position uu and bjb_j is its bias; causal padding supplies zeros for unavailable input positions, so each output uses only present and past inputs. Stacking dilations widens the receptive field of the layer. Writing fTCNf_{TCN} for this transformation, the output is XTCN=fTCN(Xcomb)∈RL×hX_{TCN}=f_{TCN}(X_{comb})\in\mathbb{R}^{L\times h}.

Attention and Gating Layer: The TCN output is layer-normalized [32] and passed through multi-head self-attention with nhn_h heads. Let a=1,…,nha=1,\ldots,n_h index the heads, dkd_k be the key dimension, and QaQ_a, KaK_a, and VaV_a be the query, key, and value projections of the normalized sequence. Then

Attention⁡a=softmax⁡(QaKaTdk)Va,a=1,…,nh,\operatorname{Attention}_a=\operatorname{softmax}\left(\frac{Q_aK_a^{\mathsf{T}}}{\sqrt{d_k}}\right)V_a,\quad a=1,\ldots,n_h,

where the softmax normalizes each row of attention scores to sum to one. The concatenated heads are projected back to width hh and added to the TCN output through a residual connection,

Z=XTCN+MHA⁡(LN⁡(XTCN)),Z=X_{TCN}+\operatorname{MHA}(\operatorname{LN}(X_{TCN})),

where LN denotes layer normalization and MHA multi-head attention. A sigmoid gate then scales each element of this representation:

G=σ(ZWG+bG),Xgated=G⊙Z∈RL×h,G=\sigma(ZW_G+b_G),\quad X_{gated}=G\odot Z\in\mathbb{R}^{L\times h},

where σ\sigma is the logistic sigmoid and ⊙\odot the element-wise (Hadamard) product. Each entry of ZZ is thus multiplied by its own gate value in (0,1)(0,1), so the model can suppress positions and features that do not inform the forecast. This multiplicative gate is related to gated linear units [33] and to the gating of recurrent networks [30], but here the representation is scaled by a sigmoid projection of itself.

Output Layer: A second TCN with k′k' filters and dilations (1,2)(1,2) or (1,2,4)(1,2,4) summarizes the gated sequence into its last step, z′∈Rk′z'\in\mathbb{R}^{k'}. This vector is concatenated with the static embedding sembs_{emb} and mapped by two dense layers to the predicted target values:

y^=WoReLU⁡(Wu[z′∥semb]+bu)+bo∈RH,\hat{y}=W_o\operatorname{ReLU}(W_u[z'\mathbin{\Vert}s_{emb}]+b_u)+b_o\in\mathbb{R}^{H},

where H=MH=M is the number of forecast steps (H=1H=1 for the one-step-ahead forecasts reported in § 5). Concatenating sembs_{emb} again at the output lets the factory configuration act directly on the prediction as well as through the encoded sequence. For known-future inputs, the dashed branch of Figure 1 adds a future-time embedding femb=ReLU⁡(Wft++bf)∈Rff_{emb}=\operatorname{ReLU}(W_f t^+ + b_f)\in\mathbb{R}^{f} of the times t+t^+ of the HH steps to be predicted, which is concatenated as [z′∥semb∥femb][z'\mathbin{\Vert}s_{emb}\mathbin{\Vert}f_{emb}]; this branch is not used in the present experiments.

Local and global temporal structure: The convolutional and attention stages capture the trajectory at different scales. A causal convolution combines each position with a fixed set of preceding positions through weights shared across the window, so it extracts local patterns of the target, such as step-to-step increments and the short bursts of completions of a batch, and stacking dilations widens this neighborhood. Self-attention instead weights every pair of positions in the window by the similarity of their representations, so each position can draw on any other position in a single layer, independently of their distance and of the receptive field. It can thereby relate the current state of the factory to the global course of the window, such as the start-up level or the rise towards a plateau. The residual connection in ZZ keeps the local convolutional features and adds this window-wide context, and the gate selects, per position and feature, how much of the combined representation reaches the output layer.

Computational Efficiency

To determine the efficiency of the model, we focus on the dominant terms in the computational cost of each component. For the input TCN layer with input Xcomb∈RL×(c+h)X_{comb} \in\mathbb{R}^{L \times(c+h)}, where LL is the look-back window and c+hc+h is the number of input features, the computation for NN stacked residual blocks with hh filters and kernel size κ\kappa can be approximated as O(N⋅L⋅κ⋅h⋅(c+h))\mathcal{O}(N \cdot L \cdot\kappa\cdot h \cdot(c+h)). The time complexity of multi-head attention with nhn_h heads of key dimension dkd_k can be approximated as O(nh⋅(L2⋅dk+L⋅h⋅dk))\mathcal{O}(n_h \cdot(L^2 \cdot d_k + L \cdot h \cdot d_k)). The total time complexity of the architecture can therefore be approximated as

O(N⋅L⋅κ⋅h⋅(c+h))+O(nh⋅(L2⋅dk+L⋅h⋅dk)).\mathcal{O}(N \cdot L \cdot\kappa\cdot h \cdot(c+h)) + \mathcal{O}(n_h \cdot(L^2 \cdot d_k + L \cdot h \cdot d_k)).

Here, the local context is handled by the TCN layers, which leverage dilated convolutions for efficient local feature extraction, and the global context is captured by multi-head attention, which models relationships among all time steps. TCN layers are memory-efficient owing to convolutional sparsity, whereas attention layers use O(L2)\mathcal{O}(L^2) memory; hence, the look-back window length LL strongly affects the overall efficiency of the model.

Experiments

We conducted experiments with the TCNA model and other temporal ML models (see Table 2) to predict THTH (measured per minute) and TATA (measured in minutes) time series for the benchmark MiniFab model, with time tt defined in minutes. The THTH and TATA time series are approximations of their simulated discrete-event trajectory counterparts. Note that the scales of throughput and turnaround time differ by a factor of 10610^6. The MAPEMAPE, MSEMSE, and R2R^2 values of TCNA and the other models are computed against the ground-truth PDEVS simulated data sets. The experiments also evaluate the efficiency of the TCNA model (in terms of the number of parameters and maximum memory) relative to the other temporal ML models. An ablation study shows the contribution of each TCNA component (see Table 2 and Table 5).

DatasetScenarioMetricsLSTMTCNTFTTCNAFEDformerAutoformerN-BEATS
ThroughputAR2R^2 Score0.67540.98330.99980.99990.99840.99970.9990
ThroughputAMSE1.22E-074.00E-085.67E-104.07E-105.22E-091.14E-093.32E-09
ThroughputAMAPE32.59%4.30%0.54%0.86%3.28%1.52%1.90%
ThroughputBR2R^2 Score0.68100.99770.99980.99980.99830.99960.9989
ThroughputBMSE1.20E-066.07E-095.77E-106.43E-105.22E-091.15E-093.35E-09
ThroughputBMAPE32.56%2.05%0.57%0.95%3.36%1.54%1.98%
ThroughputCR2R^2 Score0.60600.99900.99970.99990.99800.99960.9989
ThroughputCMSE1.21E-062.45E-095.30E-102.20E-105.21E-091.14E-092.92E-09
ThroughputCMAPE34.88%1.50%0.60%0.83%3.67%1.66%2.05%
ThroughputDR2R^2 Score0.63200.99800.99980.99990.99810.99960.9989
ThroughputDMSE1.21E-064.24E-095.50E-102.33E-105.22E-091.13E-093.08E-09
ThroughputDMAPE38.69%1.97%0.68%0.96%3.82%1.72%2.29%
ThroughputER2R^2 Score0.60100.99900.99980.99990.99800.99960.9989
ThroughputEMSE1.22E-062.48E-095.37E-101.97E-105.21E-091.13E-092.91E-09
ThroughputEMAPE33.89%1.53%0.55%0.81%3.62%1.64%1.96%
ThroughputFR2R^2 Score0.62300.99720.99980.99990.99810.99960.9989
ThroughputFMSE1.22E-066.22E-095.37E-102.13E-105.22E-091.14E-093.01E-09
ThroughputFMAPE37.92%2.18%0.56%0.83%3.56%1.62%1.99%
Turnaround TimeAR2R^2 Score0.91440.97300.99960.99910.99770.99930.9983
Turnaround TimeAMSE3.67E+061.32E+062.11E+044.43E+049.98E+042.97E+047.26E+04
Turnaround TimeAMAPE10.72%6.51%1.10%0.89%2.20%0.92%1.34%
Turnaround TimeBR2R^2 Score0.91330.96960.99950.99920.99770.99930.9983
Turnaround TimeBMSE3.67E+061.41E+061.99E+043.58E+049.93E+042.93E+047.20E+04
Turnaround TimeBMAPE10.88%7.09%1.10%0.86%2.21%0.92%1.36%
Turnaround TimeCR2R^2 Score0.91210.96560.99950.99930.99760.99930.9982
Turnaround TimeCMSE3.47E+061.44E+061.88E+043.00E+049.74E+042.80E+047.07E+04
Turnaround TimeCMAPE10.32%7.43%1.07%0.90%2.29%0.95%1.46%
Turnaround TimeDR2R^2 Score0.91370.96390.99960.99930.99760.99930.9982
Turnaround TimeDMSE3.50E+061.54E+061.91E+042.93E+049.84E+042.87E+047.16E+04
Turnaround TimeDMAPE10.07%7.66%1.08%0.81%2.25%0.93%1.41%
Turnaround TimeER2R^2 Score0.91380.96740.99960.99910.99770.99930.9983
Turnaround TimeEMSE3.57E+061.45E+061.93E+044.05E+049.81E+042.86E+047.08E+04
Turnaround TimeEMAPE10.15%7.22%1.07%0.97%2.24%0.93%1.45%
Turnaround TimeFR2R^2 Score0.91450.96610.99950.99910.99760.99930.9985
Turnaround TimeFMSE3.50E+061.48E+061.92E+044.14E+049.78E+042.84E+046.90E+04
Turnaround TimeFMAPE9.91%7.35%1.06%0.92%2.25%0.94%1.35%

Table 2. Model Performance Comparison: Average Values for Scenarios A–F

MethodMSEMSER2R^2 ScoreMAPEMAPE
TCNA2.60E-100.99950.86%
No Attention5.02E-090.99843.21%
No Initial TCN2.60E-070.917021.94%
No Gating Mechanism1.56E-090.99951.17%

Table 5. Common Measurements for Ablation Study

Experimental Setup

The temporal ML models (Autoformer, FEDformer, LSTM, N-BEATS, TCN, TCNA, and TFT) were trained and tested using the MiniFab simulation data sets. The MiniFab model processes wafer lots containing three wafer types, called Pa, Pb, and Tw, which arrive at 8-, 12-, and 24-hour intervals, respectively. Each wafer type is produced by its own generator, and the generated wafers follow either uniform or sinusoidal arrival patterns. The wafers are combined into five different kinds of batches before going through the first processing step in the diffusion and implantation stages. In contrast, the machines belonging to the eight lithography stages can process only one wafer at a time. Every machine can be configured to undergo one or more repairs during its use. A repair (during which the machine is under maintenance or offline) is triggered either according to the mean time between failures (MTBF) or after a certain number of lots have been processed. The wafer types, repair type, and lot size are used as static covariates for training the ML models.

This configuration leads to the six scenarios in which the MiniFab model is simulated, as shown in Table 1. The lot size is the total number of Pa, Pb, and Tw wafers used in the manufacturing process. Lot sizes are divided into three groups: small (60 to 108), medium (120 to 156), and large (168 to 204). Nine lot size and wafer type configurations were used for training in each scenario, totaling 54 time series. Similarly, three different batch configurations were used for testing in each scenario, totaling 18 time series. The models are thus trained on one set of configurations and tested on unseen wafer batch configurations. Although the time needed for the MiniFab to process all wafers differs among the small, medium, and large configurations, each of the 72 throughput and turnaround time series is set to 30,000 values spanning simulated 21.83 days.

ScenarioRepair TypeWafer Generation
ANo RepairSinusoidal
BNo RepairUniform
CMTBFUniform
DProcessing StepsUniform
EMTBFSinusoidal
FProcessing StepsSinusoidal

Table 1. Simulation Scenarios for Wafer Manufacturing

Prediction Measurements

The prediction accuracies of TCNA, TFT, TCN, Autoformer, FEDformer, N-BEATS, and LSTM are evaluated for throughput (THTH) and turnaround time (TATA) using input embeddings and static covariates. The R2R^2, MSEMSE, and MAPEMAPE metrics are summarized in Table 2, and the MAPEMAPE values are visualized in Figure 2. For throughput, TFT achieves the lowest MAPEMAPE, followed by TCNA, whereas TCNA attains the best MSEMSE and R2R^2, with TFT ranking second; for turnaround time, this trend is reversed. For the medium lot size configuration (Pa = 72, Pb = 45, Tw = 27), Figure 3 and Figure 4 show that TCNA and TFT most closely track the MiniFab simulation trajectories. In contrast, LSTM and TCN do not capture the correct directional changes in throughput, while FEDformer, Autoformer, and N-BEATS follow the trends with reduced accuracy. All models use a 10-minute look-back window to predict one-step-ahead values, a design choice aimed at achieving the high short-horizon accuracy needed in semiconductor fabrication factories.

Radar charts showing MAPE values for throughput and turnaround time across six forecasting models

Figure 2. MAPE Values for an 8-Stage Cascade Semiconductor MiniFab Factory

Further analysis highlights the impact of lot size on prediction behavior. For small lot sizes (Figure 5 and Figure 7), shorter processing horizons and abrupt transitions increase sensitivity to local temporal dynamics, leading to delayed or oscillatory predictions from LSTM, TCN, Autoformer, and FEDformer, particularly near the first non-zero event (Figure 9 and Figure 10). As lot size increases (Figure 6 and Figure 8), the THTH and TATA trajectories become smoother and extend over longer time horizons, reducing short-term volatility but introducing cumulative drift, especially for turnaround time. Across all lot size regimes, TCNA maintains stable alignment with the PDEVS trajectories by capturing both local transitions and long-range dependencies, resulting in more consistent performance and reduced sensitivity to changes in batch configuration.

TH Prediction Comparison for Small Lot Size

Figure 5. THTH Prediction Comparison for Small Lot Size

TH Prediction Comparison for Large Lot Size

Figure 6. THTH Prediction Comparison for Large Lot Size

TA prediction comparison for small lot size

Figure 7. TATA Prediction Comparison for Small Lot Size

TA prediction comparison for large lot size

Figure 8. TATA Prediction Comparison for Large Lot Size

TH prediction around the first non-zero value for medium lot size

Figure 9. THTH Prediction Around the First Non-Zero Value for Medium Lot Size

TA prediction around the first non-zero value for medium lot size

Figure 10. TATA Prediction Around the First Non-Zero Value for Medium Lot Size

Table 3 reports the paired tt-statistics and Kolmogorov–Smirnov (KS) statistics for the throughput (THTH) and turnaround time (TATA) predictions in Scenario E for the small (Pa = 3, Pb = 42, Tw = 15; lot size = 60), medium (Pa = 72, Pb = 45, Tw = 27; lot size = 144), and large (Pa =150= 150, Pb=24P_b = 24, Tw=18T_w = 18; lot size =192= 192) lot sizes. Scenario E (repair type == MTBF, wafer generation == sinusoidal) has the most complex dynamics of all the scenarios (see Table 1). The tt-statistics reveal substantial variability across models and configurations, particularly for throughput, where large magnitudes indicate statistically significant differences between the model predictions and the PDEVS reference. Models such as LSTM and FEDformer consistently exhibit large positive or negative tt-values across lot sizes, suggesting systematic bias in their predictions. In contrast, TCNA produces tt-statistics among the closest to zero in most configurations (e.g., 10.1 for medium-lot throughput, and 50.4 and 68.8 for small- and large-lot turnaround time), although not for large-lot throughput; this indicates a generally smaller mean deviation from the simulated trajectories. The KS statistics provide complementary insight into distributional similarity. For both throughput and turnaround time, TCNA, TFT, and Autoformer generally achieve lower KS values than LSTM and TCN, particularly in the medium and large lot size configurations. This indicates closer alignment between the predicted and simulated value distributions. Notably, the KS statistics are markedly lower for the medium and large lot sizes than for the small lot size, reflecting smoother system dynamics and improved distributional agreement for most models. Overall, the combined tt-test and KS-test results indicate that, while several models capture global trends, TCNA achieves consistently strong agreement with the PDEVS reference in both mean behavior and distributional characteristics across lot size regimes.

MetricsLot SizeLSTMTCNTFTTCNAFEDformerAutoformerN-BEATS
t-statistic
ThroughputSmall3.37E+022.14E+021.84E+029.79E+011.02E+03-2.08E+028.96E+01
ThroughputMedium-1.68E+031.59E+02-2.12E+021.01E+018.02E+02-1.51E+021.73E+02
ThroughputLarge-4.07E+02-1.20E+021.43E+021.76E+028.42E+02-1.57E+021.22E+02
Turnaround TimeSmall1.12E+031.75E+03-4.95E+025.04E+014.72E+02-1.42E+024.15E+02
Turnaround TimeMedium4.79E+02-1.17E+002.96E+027.70E+014.42E+02-1.32E+02-1.79E+02
Turnaround TimeLarge-6.78E+02-1.79E+03-2.52E+026.88E+014.58E+02-1.36E+023.45E+02
KS-statistic
ThroughputSmall6.33E-015.94E-015.67E-015.67E-015.94E-015.68E-015.67E-01
ThroughputMedium5.54E-011.09E-014.02E-024.05E-026.63E-023.98E-028.60E-02
ThroughputLarge1.27E-015.02E-025.25E-025.25E-026.45E-025.20E-027.89E-02
Turnaround TimeSmall7.12E-016.53E-015.96E-015.69E-015.98E-015.69E-016.07E-01
Turnaround TimeMedium1.87E-011.07E-014.36E-021.12E-015.43E-025.01E-025.21E-02
Turnaround TimeLarge2.22E-013.23E-015.83E-026.40E-026.21E-025.46E-021.05E-01

Table 3. Statistical Analysis for Scenario E

Early- and Late-Stage Trajectory Predictions of Throughput and Turnaround Time

Although throughput (THTH) and turnaround time (TATA) exhibit similar trends at a macroscopic level, their early- and late-stage prediction behavior reveals important differences among the temporal ML models. Figures Figure 9 and Figure 10 illustrate model predictions around the first non-zero event, which corresponds to the time when the first wafer exits the MiniFab. This transition represents a sharp regime change that tests a model’s ability to capture local temporal dynamics near a zero-valued boundary. While most models achieve strong aggregate performance in terms of

MAPEMAPE, MSEMSE, and R2R^2 (see Table 2), the early-stage behavior exposes inaccuracies that are not reflected in global metrics. As shown in Table 4,1 Several models exhibit large percentage deviations at the initial prediction step, particularly for throughput. LSTM shows extremely high percentage differences (379% to 1932%), TFT, TCNA, and Autoformer deviate by approximately 87% to 155% for all lot sizes, and the transformer-based models without

TimestampLot SizePDEVS ValuesLSTMTCNTFTTCNAFEDformerAutoformerN-BEATS
InitialThroughputSmall1.14E-04378.70%31.71%96.02%94.38%34.08%155.16%3.21%
InitialThroughputMedium1.10E-041931.99%28.55%87.50%97.23%32.03%100.16%23.99%
InitialThroughputLarge1.12E-04896.10%12.20%95.65%92.52%27.96%107.36%32.96%
InitialTurnaround TimeSmall8.75E+0313.28%0.66%3.47%1.85%28.10%49.26%20.54%
InitialTurnaround TimeMedium9.09E+038.06%11.29%0.92%0.49%28.05%47.30%16.87%
InitialTurnaround TimeLarge8.87E+0323.57%36.75%3.10%2.62%3.23%1.29%2.09%
Early HorizonThroughputSmall1.14E-04315.60%13.63%97.48%90.70%62.52%24.59%51.14%
Early HorizonThroughputMedium1.10E-041906.65%10.21%85.49%96.10%26.74%140.99%43.12%
Early HorizonThroughputLarge1.12E-04874.67%1.60%99.34%91.60%63.40%24.94%52.07%
Early HorizonTurnaround TimeSmall8.75E+0311.91%8.32%3.22%3.38%3.27%1.30%11.28%
Early HorizonTurnaround TimeMedium9.09E+036.44%7.22%0.83%0.81%3.15%1.25%7.13%
Early HorizonTurnaround TimeLarge8.87E+0322.39%38.94%2.52%1.64%3.23%1.29%11.34%
FinalThroughputSmall3.17E-0310.19%3.28%1.76%1.49%6.20%0.45%2.71%
FinalThroughputMedium4.31E-0329.82%3.32%0.12%0.75%2.39%0.09%3.29%
FinalThroughputLarge4.60E-033.53%0.50%0.84%1.48%3.18%0.06%2.54%
FinalTurnaround TimeSmall1.34E+0411.88%7.47%1.13%1.43%3.21%1.04%3.79%
FinalTurnaround TimeMedium1.93E+049.07%3.29%1.64%5.58%2.19%0.36%0.14%
FinalTurnaround TimeLarge1.89E+044.94%12.55%0.59%2.45%2.19%0.28%4.29%

Table 4. Percentage Difference Between PDEVS and Model Predictions at Initial, Early-Horizon, and Final Time Steps for Throughput and Turnaround Time

convolutional structure (Autoformer and FEDformer) exhibit significant variability. Notably, some models predict negative initial throughput values, which are physically infeasible in a manufacturing context and arise from unconstrained regression near zero-valued targets.

To further examine short-horizon stabilization effects, Table 4 also reports prediction errors after a look-back window of 10 time steps, shown in the Early Horizon rows. These rows quantify the percentage difference between the model predictions and the PDEVS values after the first 10 minutes of production. For most models, and particularly for FEDformer and Autoformer on turnaround time, this short historical context reduces the error relative to the initial step alone. However, large discrepancies remain for the throughput predictions of LSTM (875% to 1907%) and, to a lesser extent, TFT and TCNA (approximately 85% to 99%), whereas TCN attains the lowest early-horizon throughput errors (1.6% to 13.6%). For turnaround time, TCNA exhibits low percentage differences at both the initial step and the early horizon (0.49% to 3.38%) across all lot sizes, demonstrating rapid stabilization. The final-value comparisons further show that most models converge toward the PDEVS reference by the end of processing, which can mask early-stage inaccuracies if only terminal performance is considered. Overall, these results highlight the importance of jointly evaluating initial, short-horizon, and final prediction behavior, especially in semiconductor manufacturing applications, where early prediction accuracy is critical for operational decision-making.

Efficiency Analysis

The efficiency of the models is evaluated in terms of memory and the number of parameters needed for training. Like TFT, the TCNA model captures the essential information in the time series data. It needs less training time than all other models except LSTM and TCN, whose predictions are considerably less accurate (see Figure 11). The remaining temporal models also produce less accurate predictions. Compared with TFT, TCNA reduces the model size by approximately 10%, the number of parameters by approximately 12%, and the training time per epoch by approximately 9%, which can be significant for longer sequence lengths LL.

Model Efficiency

Figure 11. Model Efficiency

Ablation Study Using Error Measurements

We performed an ablation study to understand the importance of each part of the TCNA architecture and its impact on the overall accuracy of the model. Experiments were conducted by removing individual components of the MiniFab TCNA model, as shown in Table 5. Removing the gating mechanism has the smallest adverse effect on accuracy, followed by removing the attention heads and then the initial TCN layer. Each component of the architecture therefore contributes to the overall accuracy and efficiency of the model. The gating mechanism contributes less to accuracy but is essential for maintaining the overall efficiency of the model.

Discussion

The experimental results across all seven temporal ML models demonstrate that lot size and factory configuration play a critical role in shaping the temporal dynamics of both throughput (THTH) and turnaround time (TATA), directly influencing prediction accuracy and stability across models. Smaller lot sizes produce shorter processing horizons and sharper transitions, particularly at the point where the first wafers exit the 8-stage MiniFab; this amplifies local temporal dependencies and exposes the limitations of models that rely primarily on autoregressive behavior or long-horizon averaging. In such cases, LSTM and TCN often lag behind the ground-truth PDEVS simulation trajectories, while transformer-based models without convolutional structure tend to exhibit overshoot or oscillatory behavior. This early-stage sensitivity is further reflected in large percentage differences at the initial prediction step and, in some cases, physically infeasible negative throughput values, highlighting the challenge of modeling the PDEVS dynamics. These inaccuracies likely arise from the difficulty of approximating instantaneous state transitions in machines that concurrently receive and send wafers through multiple input and output ports.

As the lot size increases, the throughput and turnaround time profiles become smoother and extend over longer horizons. Although this smoothing generally reduces short-term volatility and improves average error metrics for all models, it also introduces cumulative prediction bias, especially near saturation. This effect is evident in the large lot size configurations, where small temporal misalignments lead to noticeable drift in the TATA predictions. Incorporating short-horizon context via a look-back window reduces early-stage errors for most models; however, residual discrepancies remain for models with weaker short-term adaptability. TCNA mitigates many of these issues by combining convolutional feature extraction with attention-based contextual modeling, maintaining close alignment with the simulated PDEVS trajectories for turnaround time in both the early and late stages of processing, and for throughput once the initial transient has passed. Across small, medium, and large lot sizes, TCNA shows little variation in error behavior, indicating strong robustness to changes in batch configuration. TFT achieves competitive accuracy, and is more accurate than TCNA at several initial and final time steps, whereas TCNA attains comparable aggregate accuracy with a smaller model (§ 5.4). These findings suggest that explicitly modeling both local and global temporal dependencies is essential for better and reliable prediction in semiconductor fabrication manufacturing systems.

Conclusion and Future Work

Semiconductor factories involve complex processes and feedforward/feedback flows for fabricating many kinds of chips. In these factories, wafers undergo many incremental processing steps spanning hours to weeks in connected machines with re-entrant flows. Essential to the operation of such factories is accurately predicting throughput (THTH) and turnaround time (TATA) dynamics for individual and multiple machines, repeatedly and as fast as possible (e.g., every hour). This paper presented a Temporal Convolutional Network with Multi-head Attention (TCNA) to predict these essential dynamics. Simulated data sets of a benchmark discrete-event semiconductor manufacturing factory, processing wafer lots with uniform and sinusoidal wafer arrival profiles and subject to preventive maintenance, were used to develop TCNA and the comparison models. The THTH and TATA time series predicted by TCNA, evaluated with R2R^2, MSEMSE, and MAPEMAPE and in terms of their dynamics at the beginning and end of wafer processing, show satisfactory accuracy with comparable parameter counts and memory utilization relative to the other temporal ML models.

Future work will extend this study in two directions. First, the evaluation will move beyond the point-accuracy measures used here (MAPEMAPE, MSEMSE, and R2R^2) to a multi-metric assessment [34]: the mean absolute scaled error (MASEMASE), which scales errors by the typical one-step change in the training data and remains well defined; accumulated and position-weighted errors (IAE, ISE, and ITAE), which reveal persisting and accumulated errors in long horizon trajectories; dynamic time warping (DTW) and slope errors, which assess the shapes of trajectories; peak and transition errors, which assess the timing of events; and frequency-domain errors. Such measures can rank models differently than point accuracy does and are better suited to the piecewise-constant state transitions, event-driven I/O trajectories of discrete-event simulation. Second, the architecture will be extended to dynamic covariates. The present model conditions only on static covariates (lot size, wafer-arrival profile, and repair type). The future-time branch shown dashed in Figure 1 embeds known-future inputs, such as the event times of the steps to be predicted in a discrete-event trajectory, and joins them with the TCN summary and the static embedding in the output layer; time-varying inputs, such as the scaled simulation time, machine availability, or planned wafer releases, can likewise be added as further channels of the input window. Evaluating TCNA with these dynamic covariates and the multi-metric measures on discrete-event trajectories remains future work.

References

  1. [1]C. V. Goldman, M. Baltaxe, D. Chakraborty, and J. Arinez, “Explaining learning models in manufacturing processes,” Procedia Comput. Sci., vol. 180, pp. 259–268, 2021.DOI
  2. [2]L. Lingitz, V. Gallina, F. Ansari, D. Gyulai, A. Pfeiffer, W. Sihn, and L. Monostori, “Lead time prediction using machine learning algorithms: A case study by a semiconductor manufacturer,” Procedia CIRP, vol. 72, pp. 1051–1056, 2018.
  3. [3]M. Subramaniyan, A. Skoogh, H. Salomonsson, P. Bangalore, and J. Bokrantz, “A data-driven algorithm to predict throughput bottlenecks in a production system based on active periods of the machines,” Comput. Ind. Eng., vol. 125, pp. 533–544, 2018.DOI
  4. [4]J. Spier and K. Kempf, “Simulation of emergent behavior in manufacturing systems,” in Proc. ESMI Adv. Semiconductor Manuf. Conf. Workshop, Cambridge, MA, USA, Nov. 13–15, 1995, pp. 90–94.
  5. [5]H. S. Sarjoughian, F. Fallah, S. Saeidi, and E. J. Yellig, “Transforming discrete event models to machine learning models,” in Proc. 2023 Winter Simulation Conf. (WSC), IEEE, 2023, pp. 2662–2673.DOI
  6. [6]V. K. Pendyala, H. S. Sarjoughian, B. Potenini, and E. J. Yellig, “A benchmark time-series dataset for semiconductor fabrication manufacturing constructed using component-based discrete-event simulation models,” arXiv preprint arXiv:2408.09307, 2024.arxiv.org/abs/2408.09307
  7. [7]H. Wu, J. Xu, J. Wang, and M. Long, “Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting,” in Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 34, 2021, pp. 22419–22430.arxiv.org/abs/2106.13008
  8. [8]T. Zhou, Z. Ma, Q. Wen, X. Wang, L. Sun, and R. Jin, “FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting,” in Proc. 39th Int. Conf. Mach. Learn. (ICML), 2022, pp. 27268–27286.arxiv.org/abs/2201.12740
  9. [9]S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Comput., vol. 9, no. 8, pp. 1735–1780, Nov. 1997. DOI: 10.1162/neco.1997.9.8.1735.DOI
  10. [10]B. N. Oreshkin, D. Carpov, N. Chapados, and Y. Bengio, “N-BEATS: Neural basis expansion analysis for interpretable time series forecasting,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2020.arxiv.org/abs/1905.10437
  11. [11]S. Bai, J. Z. Kolter, and V. Koltun, “An empirical evaluation of generic convolutional and recurrent networks for sequence modeling,” arXiv preprint arXiv:1803.01271, 2018.arxiv.org/abs/1803.01271
  12. [12]B. Lim, S. Ö. Arık, N. Loeff, and T. Pfister, “Tempo-ral fusion transformers for interpretable multi-horizon time series forecasting,” Int. J. Forecasting, vol. 37, no. 4, pp. 1748–1764, 2021.arxiv.org/abs/1912.09363
  13. [13]A. C. H. Chow and B. P. Zeigler, “Parallel DEVS: A parallel, hierarchical, modular modeling formalism,” in Proc. Winter Simulation Conf., IEEE, 1994, pp. 716–722.
  14. [14]ACIMS, “DEVS-Suite simulator, version 7.0,” 2023. [Online]. Available: https://acims.asu.edu/devs-suite/
  15. [15]V. K. Pendyala, H. S. Sarjoughian, and E. J. Yellig, “Generating temporal convolutional network models from parallel DEVS models: Semiconductor manufacturing systems,” in Proc. 2024 Winter Simulation Conf. (WSC), IEEE, 2024, pp. 2265–2276.
  16. [16]K. C. Chan, M. Rabaev, and H. Pratama, “Generation of synthetic manufacturing datasets for machine learning using discrete-event simulation,” Prod. Manuf. Res., vol. 10, no. 1, pp. 337–353, 2022.DOI
  17. [17]R. Godahewa, C. Bergmeir, G. I. Webb, R. J. Hyndman, and P. Montero-Manso, “Monash time series forecasting archive,” arXiv preprint arXiv:2105.06643, 2021.arxiv.org/abs/2105.06643
  18. [18]A. Hussein and M. Awad, “Time series forecasting of electricity consumption using hybrid model of recurrent neural networks and genetic algorithms,” Measurement: Energy, vol. 2, 2024, Art. no. 100004. DOI: 10.1016/j.meaene.2024.100004.
  19. [19]J. Zheng and M. Huang, “Traffic flow forecast through time-series analysis based on deep learning,” IEEE Access, vol. 8, pp. 82562–82570, 2020. DOI: 10.1109/ACCESS.2020.2990738.DOI
  20. [20]J. Lorenz, F. Durst, and H. Ryssel, “Process simulation for the semiconductor industry,” in Numerical Simulation in Science and Engineering: Proc. FORTHWIR Symp. High Performance Scientific Computing, München, Germany, June 17–18, 1993. Wiesbaden, Germany: Vieweg+Teubner, 1994, pp. 115–129.DOI
  21. [21]D. Huang, H. S. Sarjoughian, W. Wang, G. Godding, D. E. Rivera, K. G. Kempf, and H. Mittelmann, “Simulation of semiconductor manufacturing supply-chain systems with DEVS, MPC, and KIB,” IEEE Trans. Semicond. Manuf., vol. 22, no. 1, pp. 164–174, 2009. DOI: 10.1109/TSM.2008.2011680.DOI
  22. [22]A. Zeng, M. Chen, L. Zhang, and Q. Xu, “Are transformers effective for time series forecasting?” in Proc. AAAI Conf. Artif. Intell., vol. 37, no. 9, 2023, pp. 11121–11128.arxiv.org/abs/2205.13504
  23. [23]Q. Wen, T. Zhou, C. Zhang, W. Chen, Z. Ma, J. Yan, and L. Sun, “Transformers in time series: A survey,” arXiv preprint arXiv:2202.07125, 2022.arxiv.org/abs/2202.07125
  24. [24]Q. Ren, Y. Li, and Y. Liu, “Transformer-enhanced periodic temporal convolution network for long short-term traffic flow forecasting,” Expert Syst. Appl., vol. 227, Art. no. 120203, 2023.DOI
  25. [25]S. Lin, W. Lin, W. Wu, H. Chen, and J. Yang, “SparseTSF: Modeling long-term time series forecasting with 1k parameters,” in Proc. 41st Int. Conf. Mach. Learn. (ICML), 2024. arXiv:2405.00946.arxiv.org/abs/2405.00946
  26. [26]R. Ilbert, A. Odonnat, V. Feofanov, A. Virmaux, G. Paolo, T. Palpanas, and I. Redko, “SAMFormer: Unlocking the potential of transformers in time series forecasting with sharpness-aware minimization and channel-wise attention,” in Proc. 41st Int. Conf. Mach. Learn. (ICML), 2024. arXiv:2402.10198.arxiv.org/abs/2402.10198
  27. [27]D. Luo and X. Wang, “ModernTCN: A modern pure convolution structure for general time series analysis,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2024.
  28. [28]Y. Liu, T. Hu, H. Zhang, H. Wu, S. Wang, L. Ma, and M. Long, “iTransformer: Inverted transformers are effective for time series forecasting,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2024. arXiv:2310.06625.arxiv.org/abs/2310.06625
  29. [29]Y. Zhen, J. Fang, X. Zhao, J. Ge, and Y. Xiao, “Temporal convolution network based on attention mechanism for well production prediction,” J. Petrol. Sci. Eng., vol. 218, Art. no. 111043, 2022. DOI: 10.1016/j.petrol.2022.111043.DOI
  30. [30]J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, “Gated feedback recurrent neural networks,” in Proc. 32nd Int. Conf. Mach. Learn. (ICML), 2015, pp. 2067–2075.arxiv.org/abs/1502.02367
  31. [31]A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 30, 2017, pp. 5998–6008.arxiv.org/abs/1706.03762
  32. [32]J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” arXiv preprint arXiv:1607.06450, 2016.arxiv.org/abs/1607.06450
  33. [33]Y. N. Dauphin, A. Fan, M. Auli, and D. Grangier, “Language modeling with gated convolutional networks,” in Proc. 34th Int. Conf. Mach. Learn. (ICML), 2017, pp. 933–941.
  34. [34]V. K. Pendyala, H. S. Sarjoughian, and E. J. Yellig, “A deep learning-based multi-model, multi-metric forecasting framework for parallel DEVS semiconductor manufacturing simulation,” in preparation, 2026.

Paper details

vlnc:W_b99778b7
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY-SA 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents