WorksComputer Science
A Temporal Convolutional Network with Multi-Head Attention Model: A Surrogate for Parallel DEVS Semiconductor Fab Simulation
Abstract
Semiconductor factories process many kinds of wafers organized into lots of various sizes and configurations. Different kinds of machines, each with its own operations, process wafers for hours to days, and the wafers undergo many incremental fabrication steps (with re-entry) and flows spanning days to weeks across connected machines. Fast and accurate predictions of throughput () and turnaround time () dynamics for individual machines and their combinations are essential for factory operations. To use machine learning methods for this purpose, this paper presents a Temporal Convolutional Network with Multi-head Attention (TCNA) architecture. It uses simulated datasets from a benchmark Parallel DEVS (PDEVS) semiconductor fabrication factory model. The factory processes configurable wafer lots with uniform and sinusoidal wafer arrival profiles and may undergo preventive maintenance. The wafer lot size, arrival profile, and factory maintenance policy are used as static covariates. TCNA is applied to a selected set of simulated discrete-event throughput and turnaround time trajectories. The and time series predicted by TCNA, evaluated with the common , , and measures, show comparable or higher accuracy relative to other model architectures, with similar parameter counts and epoch times; the TCNA model size and execution time are similar to those of other architectures such as the Temporal Fusion Transformer. Although time series predictions are less accurate than high-resolution simulations, they can be deployed at a much lower computational cost. The TCNA achieves orders-of-magnitude speedup over PDEVS simulation with an acceptable loss of accuracy, enabling frequent product scheduling, capacity planning, and maintenance planning.
Introduction
As stochastic, event-oriented systems, manufacturing factories involve many machines operating in mixed sequential and concurrent settings. In some factory segments, machines have only feedforward relationships, while in others they also have feedback relationships. Semiconductor factories have complicated dynamics and must operate under stringent time constraints on what machines do individually and collectively. Machines with different capabilities must accept a variety of wafer lots in different configurations and process them in different settings to produce different kinds of chips.
A key characteristic of semiconductor fabrication is the long duration from the time raw wafers enter the factory until chips are shipped. Even though manufacturing chips typically takes several weeks to a few months, individual machines are expected to operate efficiently, given their workload capacity and resource availability (wafer lots and machines), at time scales of hours to a few days. Therefore, every machine in the process must meet time constraints determined in part by its upstream and downstream machines. Two essential metrics for manufacturing are throughput (the rate at which wafers are processed) and turnaround time (the total time required for raw wafers to become chips).
Accurate and fast predictions of throughput and turnaround time for individual machines, and therefore for the whole factory, are of the utmost importance for efficient, cost-effective, and resilient factory operation. Over many years, models have been built using physics-based concepts, mathematical theories, methods, and tools. These models represent the structure and behavior of dynamical systems so that they can simulate the dynamics of components and their relationships with high accuracy and precision. Simulations of these models are integral to the development and operation of semiconductor manufacturing factories.
In recent years, researchers and practitioners have turned to machine learning (ML) as a means to understand factory dynamics by analyzing the large amounts of data routinely collected from individual machines. The high degree of automation used in smart manufacturing for individual machines and process flows allows high-resolution data collection at scale (e.g., [1, 2, 3]). However, such data sets are closely guarded and are therefore not available to other researchers. Instead, suitable simulation models can be developed and simulated to obtain synthetic data sets, as is done in this research. A benchmark model of a semiconductor fabrication factory [4] is used to develop discrete-event single-stage and multi-stage models. These models accept wafers of different types, configurations, and lot sizes, generated with uniform or sinusoidal arrival patterns. Machines can go offline (undergoing repair) for a finite period and then return to operation. The throughput and turnaround time trajectories are discrete-event and have a continuous time base. Wafers enter and exit machines at arbitrary time instants because the machines have stochastic dynamics. Such data sets can be converted to discrete-time trajectories for use in ML models [5].
This paper presents a Temporal Convolutional Network with Multi-head Attention (TCNA), a variant of the TCN architecture, for constructing time series models from simulation data sets obtained for a discrete-event semiconductor manufacturing factory model [5, 6]. A multi-head attention layer is used to obtain contextual information from historical values for predicting future values. The TCNA model predicts future values by taking into account the context of historical values along with the effects of static variables identified for the factory model, and it accurately reproduces the simulated throughput and turnaround time values.
The simulations are developed and evaluated under low, medium, and high wafer volumes and different operational settings (e.g., machines that are temporarily unavailable). The prediction accuracy of TCNA is compared with that of the Autoformer [7], FEDformer [8], LSTM [9], N-BEATS [10], TCN [11], and TFT [12] models. Owing to the simplicity of the TCNA architecture, our efficiency analysis shows that TCNA needs fewer parameters and less memory than the other attention- and transformer-based models. TCNA predicts the and time trajectories with comparable or better mean absolute percentage error (), mean squared error (), and score values.
Semiconductor Manufacturing Factory Simulation Models
The single-stage and multi-stage semiconductor factory models used in this research are developed using the Parallel Discrete Event System Specification (PDEVS) formalism [13] and simulated using the DEVS-Suite simulator [14]. An 8-stage cascade MiniFab factory model, composed of single-stage components, generates the simulation scenarios detailed in § 5. Each single-stage model has diffusion, implantation, and lithography segments, as defined for the benchmark Intel semiconductor factory called MiniFab [4]. The diffusion stage receives raw wafers from three generators and partially processed wafers from the lithography stage. The implantation stage receives partially processed wafers from the diffusion and lithography stages. The lithography stage receives partially processed wafers from the implantation and diffusion stages. In the last step, the lithography stage completes the processing of the wafers received from the diffusion stage. The interval between two (or more) wafers arriving at (or leaving) a stage may range from minutes to multiple hours, which is a distinct characteristic of semiconductor factories.
The diffusion model has a dispatcher with a queue and two machines that independently perform the same loading, processing, unloading, and transportation of wafer batches. Similarly, the implantation and lithography models consist of components that receive, process, and forward wafers to other machines, each operating under specific timing constraints. Single-stage factories are connected in a cascade fashion to form multi-stage semiconductor fabrication factories. Wafers are organized into different batch configurations and processed through a six-step feedforward and feedback process encompassing the diffusion, implantation, and lithography stages. Discrete-event data from machines and dispatchers are collected by transducers without disrupting their operations. The time resolution for loading, processing, unloading, and transporting wafers is in minutes, with 10% stochasticity. The 8-stage factory model is created by cascading eight single-stage factories, where the output of each upstream stage becomes the input of the downstream stage. The detailed design, simulation, data collection, and analysis of the PDEVS semiconductor fabrication factory models are described in [5]. These models are extended and simulated for multivariate time series analyses in [6, 15].
Related Work
Discrete-event simulations (DES) are widely used to generate synthetic data when real-world data are limited or inaccessible [16]. For example, the single-stage MiniFab model has more than 70 simple and compound variable types (e.g., queues) [5]. Compared with data sets such as pedestrian counts and wind farms [17], semiconductor factory data have unique characteristics. As discussed in [6], simulation models play a vital role in time series analysis for manufacturing. Such synthetic data are strongly event-driven, requiring temporal ML models to consider contextual information for effective predictions [5]. In the domain of time series prediction, [18] describes a hybrid model combining recurrent neural networks (RNNs) and genetic algorithms (GAs) to predict electricity consumption, and [19] applies deep learning to traffic flow forecasting. Compared with the electricity and traffic domains, semiconductor manufacturing involves intricate wafer fabrication processes [20, 21], and its time series capture complex process flows influenced by multiple feedforward and feedback loops. Transformer- and convolution-based architectures have gained prominence for time series forecasting [15, 22]. Temporal Fusion Transformers (TFT) [12] effectively model static covariates, such as repair modes in semiconductor processes, and use multi-head attention to capture contextual dependencies. Although transformer models excel at long-term forecasting [23], their computational efficiency can be improved with convolutional operations [24]. Lightweight models such as SparseTSF [25] and SAMformer [26] demonstrate efficient adaptations of transformers, and ModernTCN [27] and iTransformer [28] further optimize temporal feature extraction. For discrete-event time series, where both local transitions and global periodicity are critical, combining TCN and attention mechanisms can enhance temporal modeling. Unlike [29], which integrates TCN and attention in parallel, our architecture applies attention to the output of a TCN layer and feeds the result into a second TCN layer for richer feature extraction. In addition, gating mechanisms, inspired by [30], dynamically control the flow of information and enhance predictive accuracy.
Approach
As mentioned in § 2, the and series of a manufacturing factory are long (30,000 to 40,000 time steps), and forecasting them also depends on the static covariates associated with each series. Therefore, the problem can be formulated as follows:
Given historical observations , where is the number of variates and is the look-back window size, and a set of static covariates , , the aim is to predict future time steps .
Discrete-event-based forecasting: Figure 3 and Figure 4 highlight the importance of contextual dependencies in these time series. The and time series exhibit non-stationary, monotonically increasing trends, with sharp variations influenced by historical values. In semiconductor fabrication, the intervals during which a machine is unavailable depend on factors such as usage, maintenance, and repair time. This local context can be expressed as

Figure 3. Prediction Comparison for Medium Lot Size

Figure 4. Prediction Comparison for Medium Lot Size
where is the sequence of observations over the past time steps. In the same manufacturing system, the frequency of breakdowns may depend on seasonal trends, such as increased usage during peak production or reduced maintenance frequency over long intervals. This global context can be expressed as
where captures a long history of observations. Static factors such as machine type or operating policies significantly affect both contexts and are modeled as static covariates , . Events occur only at specific times (e.g., machine breakdowns and maintenance events); hence, observations are non-uniform over time, and their spacing is critical. Therefore, given historical observations , static covariates , and a target prediction horizon , the objective is to find a function that minimizes the error :
The loss function is the mean squared error:
where denotes the squared error between the predictions and the true values . A simple architecture that captures sharp changes in the time series while retaining historical context serves this goal. The throughput and turnaround time obtained from the manufacturing simulation can be modeled as
where is a noise term. A rolling forecast is used to predict future values iteratively, appending each prediction to the forecast sequence.
TCN with Attention
TCNA combines dilated causal convolutions [11], which capture local temporal patterns such as step-to-step increments, with multi-head self-attention [31], which relates all positions of the look-back window and thus captures the trend across time. The architecture is shown in Figure 1 and is organized in four stages: the input embedding layer, the input TCN layer, the attention and gating layer, and the output layer. In contrast to residual formulations, which predict the change of the target from one step to the next, TCNA predicts the target values directly.

Figure 1. TCN with Multi-head Attention (TCNA) Model Architecture. The dashed future-time branch is an optional input for known-future (dynamic) covariates; it is not used in the experiments of this paper (see § 7).
Input Embedding Layer: Let be the past window, which holds the dynamic input channels of the most recent observations; for the univariate or series considered here, these channels contain the scaled target values. Let be the static embedding width. The static covariates (lot size, wafer types, wafer-arrival profile, and repair type) are embedded by a dense layer of width , , repeated times, and concatenated with the past window:
where denotes concatenation and is a length- vector of ones. Every time step therefore carries the configuration of the factory that produced it.
Input TCN Layer: A TCN with filters, kernel size , and dilations or is applied to with causal padding, residual and skip connections, and returned sequences, so that the output keeps the sequence length . Let index positions in the input window, the output filters, a dilation factor, and the kernel offsets. With input vector and filter vector , each causal convolution computes
where is the output of filter at position and is its bias; causal padding supplies zeros for unavailable input positions, so each output uses only present and past inputs. Stacking dilations widens the receptive field of the layer. Writing for this transformation, the output is .
Attention and Gating Layer: The TCN output is layer-normalized [32] and passed through multi-head self-attention with heads. Let index the heads, be the key dimension, and , , and be the query, key, and value projections of the normalized sequence. Then
where the softmax normalizes each row of attention scores to sum to one. The concatenated heads are projected back to width and added to the TCN output through a residual connection,
where LN denotes layer normalization and MHA multi-head attention. A sigmoid gate then scales each element of this representation:
where is the logistic sigmoid and the element-wise (Hadamard) product. Each entry of is thus multiplied by its own gate value in , so the model can suppress positions and features that do not inform the forecast. This multiplicative gate is related to gated linear units [33] and to the gating of recurrent networks [30], but here the representation is scaled by a sigmoid projection of itself.
Output Layer: A second TCN with filters and dilations or summarizes the gated sequence into its last step, . This vector is concatenated with the static embedding and mapped by two dense layers to the predicted target values:
where is the number of forecast steps ( for the one-step-ahead forecasts reported in § 5). Concatenating again at the output lets the factory configuration act directly on the prediction as well as through the encoded sequence. For known-future inputs, the dashed branch of Figure 1 adds a future-time embedding of the times of the steps to be predicted, which is concatenated as ; this branch is not used in the present experiments.
Local and global temporal structure: The convolutional and attention stages capture the trajectory at different scales. A causal convolution combines each position with a fixed set of preceding positions through weights shared across the window, so it extracts local patterns of the target, such as step-to-step increments and the short bursts of completions of a batch, and stacking dilations widens this neighborhood. Self-attention instead weights every pair of positions in the window by the similarity of their representations, so each position can draw on any other position in a single layer, independently of their distance and of the receptive field. It can thereby relate the current state of the factory to the global course of the window, such as the start-up level or the rise towards a plateau. The residual connection in keeps the local convolutional features and adds this window-wide context, and the gate selects, per position and feature, how much of the combined representation reaches the output layer.
Computational Efficiency
To determine the efficiency of the model, we focus on the dominant terms in the computational cost of each component. For the input TCN layer with input , where is the look-back window and is the number of input features, the computation for stacked residual blocks with filters and kernel size can be approximated as . The time complexity of multi-head attention with heads of key dimension can be approximated as . The total time complexity of the architecture can therefore be approximated as
Here, the local context is handled by the TCN layers, which leverage dilated convolutions for efficient local feature extraction, and the global context is captured by multi-head attention, which models relationships among all time steps. TCN layers are memory-efficient owing to convolutional sparsity, whereas attention layers use memory; hence, the look-back window length strongly affects the overall efficiency of the model.
Experiments
We conducted experiments with the TCNA model and other temporal ML models (see Table 2) to predict (measured per minute) and (measured in minutes) time series for the benchmark MiniFab model, with time defined in minutes. The and time series are approximations of their simulated discrete-event trajectory counterparts. Note that the scales of throughput and turnaround time differ by a factor of . The , , and values of TCNA and the other models are computed against the ground-truth PDEVS simulated data sets. The experiments also evaluate the efficiency of the TCNA model (in terms of the number of parameters and maximum memory) relative to the other temporal ML models. An ablation study shows the contribution of each TCNA component (see Table 2 and Table 5).
| Dataset | Scenario | Metrics | LSTM | TCN | TFT | TCNA | FEDformer | Autoformer | N-BEATS |
| Throughput | A | Score | 0.6754 | 0.9833 | 0.9998 | 0.9999 | 0.9984 | 0.9997 | 0.9990 |
| Throughput | A | MSE | 1.22E-07 | 4.00E-08 | 5.67E-10 | 4.07E-10 | 5.22E-09 | 1.14E-09 | 3.32E-09 |
| Throughput | A | MAPE | 32.59% | 4.30% | 0.54% | 0.86% | 3.28% | 1.52% | 1.90% |
| Throughput | B | Score | 0.6810 | 0.9977 | 0.9998 | 0.9998 | 0.9983 | 0.9996 | 0.9989 |
| Throughput | B | MSE | 1.20E-06 | 6.07E-09 | 5.77E-10 | 6.43E-10 | 5.22E-09 | 1.15E-09 | 3.35E-09 |
| Throughput | B | MAPE | 32.56% | 2.05% | 0.57% | 0.95% | 3.36% | 1.54% | 1.98% |
| Throughput | C | Score | 0.6060 | 0.9990 | 0.9997 | 0.9999 | 0.9980 | 0.9996 | 0.9989 |
| Throughput | C | MSE | 1.21E-06 | 2.45E-09 | 5.30E-10 | 2.20E-10 | 5.21E-09 | 1.14E-09 | 2.92E-09 |
| Throughput | C | MAPE | 34.88% | 1.50% | 0.60% | 0.83% | 3.67% | 1.66% | 2.05% |
| Throughput | D | Score | 0.6320 | 0.9980 | 0.9998 | 0.9999 | 0.9981 | 0.9996 | 0.9989 |
| Throughput | D | MSE | 1.21E-06 | 4.24E-09 | 5.50E-10 | 2.33E-10 | 5.22E-09 | 1.13E-09 | 3.08E-09 |
| Throughput | D | MAPE | 38.69% | 1.97% | 0.68% | 0.96% | 3.82% | 1.72% | 2.29% |
| Throughput | E | Score | 0.6010 | 0.9990 | 0.9998 | 0.9999 | 0.9980 | 0.9996 | 0.9989 |
| Throughput | E | MSE | 1.22E-06 | 2.48E-09 | 5.37E-10 | 1.97E-10 | 5.21E-09 | 1.13E-09 | 2.91E-09 |
| Throughput | E | MAPE | 33.89% | 1.53% | 0.55% | 0.81% | 3.62% | 1.64% | 1.96% |
| Throughput | F | Score | 0.6230 | 0.9972 | 0.9998 | 0.9999 | 0.9981 | 0.9996 | 0.9989 |
| Throughput | F | MSE | 1.22E-06 | 6.22E-09 | 5.37E-10 | 2.13E-10 | 5.22E-09 | 1.14E-09 | 3.01E-09 |
| Throughput | F | MAPE | 37.92% | 2.18% | 0.56% | 0.83% | 3.56% | 1.62% | 1.99% |
| Turnaround Time | A | Score | 0.9144 | 0.9730 | 0.9996 | 0.9991 | 0.9977 | 0.9993 | 0.9983 |
| Turnaround Time | A | MSE | 3.67E+06 | 1.32E+06 | 2.11E+04 | 4.43E+04 | 9.98E+04 | 2.97E+04 | 7.26E+04 |
| Turnaround Time | A | MAPE | 10.72% | 6.51% | 1.10% | 0.89% | 2.20% | 0.92% | 1.34% |
| Turnaround Time | B | Score | 0.9133 | 0.9696 | 0.9995 | 0.9992 | 0.9977 | 0.9993 | 0.9983 |
| Turnaround Time | B | MSE | 3.67E+06 | 1.41E+06 | 1.99E+04 | 3.58E+04 | 9.93E+04 | 2.93E+04 | 7.20E+04 |
| Turnaround Time | B | MAPE | 10.88% | 7.09% | 1.10% | 0.86% | 2.21% | 0.92% | 1.36% |
| Turnaround Time | C | Score | 0.9121 | 0.9656 | 0.9995 | 0.9993 | 0.9976 | 0.9993 | 0.9982 |
| Turnaround Time | C | MSE | 3.47E+06 | 1.44E+06 | 1.88E+04 | 3.00E+04 | 9.74E+04 | 2.80E+04 | 7.07E+04 |
| Turnaround Time | C | MAPE | 10.32% | 7.43% | 1.07% | 0.90% | 2.29% | 0.95% | 1.46% |
| Turnaround Time | D | Score | 0.9137 | 0.9639 | 0.9996 | 0.9993 | 0.9976 | 0.9993 | 0.9982 |
| Turnaround Time | D | MSE | 3.50E+06 | 1.54E+06 | 1.91E+04 | 2.93E+04 | 9.84E+04 | 2.87E+04 | 7.16E+04 |
| Turnaround Time | D | MAPE | 10.07% | 7.66% | 1.08% | 0.81% | 2.25% | 0.93% | 1.41% |
| Turnaround Time | E | Score | 0.9138 | 0.9674 | 0.9996 | 0.9991 | 0.9977 | 0.9993 | 0.9983 |
| Turnaround Time | E | MSE | 3.57E+06 | 1.45E+06 | 1.93E+04 | 4.05E+04 | 9.81E+04 | 2.86E+04 | 7.08E+04 |
| Turnaround Time | E | MAPE | 10.15% | 7.22% | 1.07% | 0.97% | 2.24% | 0.93% | 1.45% |
| Turnaround Time | F | Score | 0.9145 | 0.9661 | 0.9995 | 0.9991 | 0.9976 | 0.9993 | 0.9985 |
| Turnaround Time | F | MSE | 3.50E+06 | 1.48E+06 | 1.92E+04 | 4.14E+04 | 9.78E+04 | 2.84E+04 | 6.90E+04 |
| Turnaround Time | F | MAPE | 9.91% | 7.35% | 1.06% | 0.92% | 2.25% | 0.94% | 1.35% |
Table 2. Model Performance Comparison: Average Values for Scenarios A–F
| Method | Score | ||
| TCNA | 2.60E-10 | 0.9995 | 0.86% |
| No Attention | 5.02E-09 | 0.9984 | 3.21% |
| No Initial TCN | 2.60E-07 | 0.9170 | 21.94% |
| No Gating Mechanism | 1.56E-09 | 0.9995 | 1.17% |
Table 5. Common Measurements for Ablation Study
Experimental Setup
The temporal ML models (Autoformer, FEDformer, LSTM, N-BEATS, TCN, TCNA, and TFT) were trained and tested using the MiniFab simulation data sets. The MiniFab model processes wafer lots containing three wafer types, called Pa, Pb, and Tw, which arrive at 8-, 12-, and 24-hour intervals, respectively. Each wafer type is produced by its own generator, and the generated wafers follow either uniform or sinusoidal arrival patterns. The wafers are combined into five different kinds of batches before going through the first processing step in the diffusion and implantation stages. In contrast, the machines belonging to the eight lithography stages can process only one wafer at a time. Every machine can be configured to undergo one or more repairs during its use. A repair (during which the machine is under maintenance or offline) is triggered either according to the mean time between failures (MTBF) or after a certain number of lots have been processed. The wafer types, repair type, and lot size are used as static covariates for training the ML models.
This configuration leads to the six scenarios in which the MiniFab model is simulated, as shown in Table 1. The lot size is the total number of Pa, Pb, and Tw wafers used in the manufacturing process. Lot sizes are divided into three groups: small (60 to 108), medium (120 to 156), and large (168 to 204). Nine lot size and wafer type configurations were used for training in each scenario, totaling 54 time series. Similarly, three different batch configurations were used for testing in each scenario, totaling 18 time series. The models are thus trained on one set of configurations and tested on unseen wafer batch configurations. Although the time needed for the MiniFab to process all wafers differs among the small, medium, and large configurations, each of the 72 throughput and turnaround time series is set to 30,000 values spanning simulated 21.83 days.
| Scenario | Repair Type | Wafer Generation |
| A | No Repair | Sinusoidal |
| B | No Repair | Uniform |
| C | MTBF | Uniform |
| D | Processing Steps | Uniform |
| E | MTBF | Sinusoidal |
| F | Processing Steps | Sinusoidal |
Table 1. Simulation Scenarios for Wafer Manufacturing
Prediction Measurements
The prediction accuracies of TCNA, TFT, TCN, Autoformer, FEDformer, N-BEATS, and LSTM are evaluated for throughput () and turnaround time () using input embeddings and static covariates. The , , and metrics are summarized in Table 2, and the values are visualized in Figure 2. For throughput, TFT achieves the lowest , followed by TCNA, whereas TCNA attains the best and , with TFT ranking second; for turnaround time, this trend is reversed. For the medium lot size configuration (Pa = 72, Pb = 45, Tw = 27), Figure 3 and Figure 4 show that TCNA and TFT most closely track the MiniFab simulation trajectories. In contrast, LSTM and TCN do not capture the correct directional changes in throughput, while FEDformer, Autoformer, and N-BEATS follow the trends with reduced accuracy. All models use a 10-minute look-back window to predict one-step-ahead values, a design choice aimed at achieving the high short-horizon accuracy needed in semiconductor fabrication factories.

Figure 2. MAPE Values for an 8-Stage Cascade Semiconductor MiniFab Factory
Further analysis highlights the impact of lot size on prediction behavior. For small lot sizes (Figure 5 and Figure 7), shorter processing horizons and abrupt transitions increase sensitivity to local temporal dynamics, leading to delayed or oscillatory predictions from LSTM, TCN, Autoformer, and FEDformer, particularly near the first non-zero event (Figure 9 and Figure 10). As lot size increases (Figure 6 and Figure 8), the and trajectories become smoother and extend over longer time horizons, reducing short-term volatility but introducing cumulative drift, especially for turnaround time. Across all lot size regimes, TCNA maintains stable alignment with the PDEVS trajectories by capturing both local transitions and long-range dependencies, resulting in more consistent performance and reduced sensitivity to changes in batch configuration.

Figure 5. Prediction Comparison for Small Lot Size

Figure 6. Prediction Comparison for Large Lot Size

Figure 7. Prediction Comparison for Small Lot Size

Figure 8. Prediction Comparison for Large Lot Size

Figure 9. Prediction Around the First Non-Zero Value for Medium Lot Size

Figure 10. Prediction Around the First Non-Zero Value for Medium Lot Size
Table 3 reports the paired -statistics and Kolmogorov–Smirnov (KS) statistics for the throughput () and turnaround time () predictions in Scenario E for the small (Pa = 3, Pb = 42, Tw = 15; lot size = 60), medium (Pa = 72, Pb = 45, Tw = 27; lot size = 144), and large (Pa , , ; lot size ) lot sizes. Scenario E (repair type MTBF, wafer generation sinusoidal) has the most complex dynamics of all the scenarios (see Table 1). The -statistics reveal substantial variability across models and configurations, particularly for throughput, where large magnitudes indicate statistically significant differences between the model predictions and the PDEVS reference. Models such as LSTM and FEDformer consistently exhibit large positive or negative -values across lot sizes, suggesting systematic bias in their predictions. In contrast, TCNA produces -statistics among the closest to zero in most configurations (e.g., 10.1 for medium-lot throughput, and 50.4 and 68.8 for small- and large-lot turnaround time), although not for large-lot throughput; this indicates a generally smaller mean deviation from the simulated trajectories. The KS statistics provide complementary insight into distributional similarity. For both throughput and turnaround time, TCNA, TFT, and Autoformer generally achieve lower KS values than LSTM and TCN, particularly in the medium and large lot size configurations. This indicates closer alignment between the predicted and simulated value distributions. Notably, the KS statistics are markedly lower for the medium and large lot sizes than for the small lot size, reflecting smoother system dynamics and improved distributional agreement for most models. Overall, the combined -test and KS-test results indicate that, while several models capture global trends, TCNA achieves consistently strong agreement with the PDEVS reference in both mean behavior and distributional characteristics across lot size regimes.
| Metrics | Lot Size | LSTM | TCN | TFT | TCNA | FEDformer | Autoformer | N-BEATS |
| t-statistic | ||||||||
| Throughput | Small | 3.37E+02 | 2.14E+02 | 1.84E+02 | 9.79E+01 | 1.02E+03 | -2.08E+02 | 8.96E+01 |
| Throughput | Medium | -1.68E+03 | 1.59E+02 | -2.12E+02 | 1.01E+01 | 8.02E+02 | -1.51E+02 | 1.73E+02 |
| Throughput | Large | -4.07E+02 | -1.20E+02 | 1.43E+02 | 1.76E+02 | 8.42E+02 | -1.57E+02 | 1.22E+02 |
| Turnaround Time | Small | 1.12E+03 | 1.75E+03 | -4.95E+02 | 5.04E+01 | 4.72E+02 | -1.42E+02 | 4.15E+02 |
| Turnaround Time | Medium | 4.79E+02 | -1.17E+00 | 2.96E+02 | 7.70E+01 | 4.42E+02 | -1.32E+02 | -1.79E+02 |
| Turnaround Time | Large | -6.78E+02 | -1.79E+03 | -2.52E+02 | 6.88E+01 | 4.58E+02 | -1.36E+02 | 3.45E+02 |
| KS-statistic | ||||||||
| Throughput | Small | 6.33E-01 | 5.94E-01 | 5.67E-01 | 5.67E-01 | 5.94E-01 | 5.68E-01 | 5.67E-01 |
| Throughput | Medium | 5.54E-01 | 1.09E-01 | 4.02E-02 | 4.05E-02 | 6.63E-02 | 3.98E-02 | 8.60E-02 |
| Throughput | Large | 1.27E-01 | 5.02E-02 | 5.25E-02 | 5.25E-02 | 6.45E-02 | 5.20E-02 | 7.89E-02 |
| Turnaround Time | Small | 7.12E-01 | 6.53E-01 | 5.96E-01 | 5.69E-01 | 5.98E-01 | 5.69E-01 | 6.07E-01 |
| Turnaround Time | Medium | 1.87E-01 | 1.07E-01 | 4.36E-02 | 1.12E-01 | 5.43E-02 | 5.01E-02 | 5.21E-02 |
| Turnaround Time | Large | 2.22E-01 | 3.23E-01 | 5.83E-02 | 6.40E-02 | 6.21E-02 | 5.46E-02 | 1.05E-01 |
Table 3. Statistical Analysis for Scenario E
Early- and Late-Stage Trajectory Predictions of Throughput and Turnaround Time
Although throughput () and turnaround time () exhibit similar trends at a macroscopic level, their early- and late-stage prediction behavior reveals important differences among the temporal ML models. Figures Figure 9 and Figure 10 illustrate model predictions around the first non-zero event, which corresponds to the time when the first wafer exits the MiniFab. This transition represents a sharp regime change that tests a model’s ability to capture local temporal dynamics near a zero-valued boundary. While most models achieve strong aggregate performance in terms of
, , and (see Table 2), the early-stage behavior exposes inaccuracies that are not reflected in global metrics. As shown in Table 4,1 Several models exhibit large percentage deviations at the initial prediction step, particularly for throughput. LSTM shows extremely high percentage differences (379% to 1932%), TFT, TCNA, and Autoformer deviate by approximately 87% to 155% for all lot sizes, and the transformer-based models without
| Timestamp | Lot Size | PDEVS Values | LSTM | TCN | TFT | TCNA | FEDformer | Autoformer | N-BEATS | |
| Initial | Throughput | Small | 1.14E-04 | 378.70% | 31.71% | 96.02% | 94.38% | 34.08% | 155.16% | 3.21% |
| Initial | Throughput | Medium | 1.10E-04 | 1931.99% | 28.55% | 87.50% | 97.23% | 32.03% | 100.16% | 23.99% |
| Initial | Throughput | Large | 1.12E-04 | 896.10% | 12.20% | 95.65% | 92.52% | 27.96% | 107.36% | 32.96% |
| Initial | Turnaround Time | Small | 8.75E+03 | 13.28% | 0.66% | 3.47% | 1.85% | 28.10% | 49.26% | 20.54% |
| Initial | Turnaround Time | Medium | 9.09E+03 | 8.06% | 11.29% | 0.92% | 0.49% | 28.05% | 47.30% | 16.87% |
| Initial | Turnaround Time | Large | 8.87E+03 | 23.57% | 36.75% | 3.10% | 2.62% | 3.23% | 1.29% | 2.09% |
| Early Horizon | Throughput | Small | 1.14E-04 | 315.60% | 13.63% | 97.48% | 90.70% | 62.52% | 24.59% | 51.14% |
| Early Horizon | Throughput | Medium | 1.10E-04 | 1906.65% | 10.21% | 85.49% | 96.10% | 26.74% | 140.99% | 43.12% |
| Early Horizon | Throughput | Large | 1.12E-04 | 874.67% | 1.60% | 99.34% | 91.60% | 63.40% | 24.94% | 52.07% |
| Early Horizon | Turnaround Time | Small | 8.75E+03 | 11.91% | 8.32% | 3.22% | 3.38% | 3.27% | 1.30% | 11.28% |
| Early Horizon | Turnaround Time | Medium | 9.09E+03 | 6.44% | 7.22% | 0.83% | 0.81% | 3.15% | 1.25% | 7.13% |
| Early Horizon | Turnaround Time | Large | 8.87E+03 | 22.39% | 38.94% | 2.52% | 1.64% | 3.23% | 1.29% | 11.34% |
| Final | Throughput | Small | 3.17E-03 | 10.19% | 3.28% | 1.76% | 1.49% | 6.20% | 0.45% | 2.71% |
| Final | Throughput | Medium | 4.31E-03 | 29.82% | 3.32% | 0.12% | 0.75% | 2.39% | 0.09% | 3.29% |
| Final | Throughput | Large | 4.60E-03 | 3.53% | 0.50% | 0.84% | 1.48% | 3.18% | 0.06% | 2.54% |
| Final | Turnaround Time | Small | 1.34E+04 | 11.88% | 7.47% | 1.13% | 1.43% | 3.21% | 1.04% | 3.79% |
| Final | Turnaround Time | Medium | 1.93E+04 | 9.07% | 3.29% | 1.64% | 5.58% | 2.19% | 0.36% | 0.14% |
| Final | Turnaround Time | Large | 1.89E+04 | 4.94% | 12.55% | 0.59% | 2.45% | 2.19% | 0.28% | 4.29% |
Table 4. Percentage Difference Between PDEVS and Model Predictions at Initial, Early-Horizon, and Final Time Steps for Throughput and Turnaround Time
convolutional structure (Autoformer and FEDformer) exhibit significant variability. Notably, some models predict negative initial throughput values, which are physically infeasible in a manufacturing context and arise from unconstrained regression near zero-valued targets.
To further examine short-horizon stabilization effects, Table 4 also reports prediction errors after a look-back window of 10 time steps, shown in the Early Horizon rows. These rows quantify the percentage difference between the model predictions and the PDEVS values after the first 10 minutes of production. For most models, and particularly for FEDformer and Autoformer on turnaround time, this short historical context reduces the error relative to the initial step alone. However, large discrepancies remain for the throughput predictions of LSTM (875% to 1907%) and, to a lesser extent, TFT and TCNA (approximately 85% to 99%), whereas TCN attains the lowest early-horizon throughput errors (1.6% to 13.6%). For turnaround time, TCNA exhibits low percentage differences at both the initial step and the early horizon (0.49% to 3.38%) across all lot sizes, demonstrating rapid stabilization. The final-value comparisons further show that most models converge toward the PDEVS reference by the end of processing, which can mask early-stage inaccuracies if only terminal performance is considered. Overall, these results highlight the importance of jointly evaluating initial, short-horizon, and final prediction behavior, especially in semiconductor manufacturing applications, where early prediction accuracy is critical for operational decision-making.
Efficiency Analysis
The efficiency of the models is evaluated in terms of memory and the number of parameters needed for training. Like TFT, the TCNA model captures the essential information in the time series data. It needs less training time than all other models except LSTM and TCN, whose predictions are considerably less accurate (see Figure 11). The remaining temporal models also produce less accurate predictions. Compared with TFT, TCNA reduces the model size by approximately 10%, the number of parameters by approximately 12%, and the training time per epoch by approximately 9%, which can be significant for longer sequence lengths .

Figure 11. Model Efficiency
Ablation Study Using Error Measurements
We performed an ablation study to understand the importance of each part of the TCNA architecture and its impact on the overall accuracy of the model. Experiments were conducted by removing individual components of the MiniFab TCNA model, as shown in Table 5. Removing the gating mechanism has the smallest adverse effect on accuracy, followed by removing the attention heads and then the initial TCN layer. Each component of the architecture therefore contributes to the overall accuracy and efficiency of the model. The gating mechanism contributes less to accuracy but is essential for maintaining the overall efficiency of the model.
Discussion
The experimental results across all seven temporal ML models demonstrate that lot size and factory configuration play a critical role in shaping the temporal dynamics of both throughput () and turnaround time (), directly influencing prediction accuracy and stability across models. Smaller lot sizes produce shorter processing horizons and sharper transitions, particularly at the point where the first wafers exit the 8-stage MiniFab; this amplifies local temporal dependencies and exposes the limitations of models that rely primarily on autoregressive behavior or long-horizon averaging. In such cases, LSTM and TCN often lag behind the ground-truth PDEVS simulation trajectories, while transformer-based models without convolutional structure tend to exhibit overshoot or oscillatory behavior. This early-stage sensitivity is further reflected in large percentage differences at the initial prediction step and, in some cases, physically infeasible negative throughput values, highlighting the challenge of modeling the PDEVS dynamics. These inaccuracies likely arise from the difficulty of approximating instantaneous state transitions in machines that concurrently receive and send wafers through multiple input and output ports.
As the lot size increases, the throughput and turnaround time profiles become smoother and extend over longer horizons. Although this smoothing generally reduces short-term volatility and improves average error metrics for all models, it also introduces cumulative prediction bias, especially near saturation. This effect is evident in the large lot size configurations, where small temporal misalignments lead to noticeable drift in the predictions. Incorporating short-horizon context via a look-back window reduces early-stage errors for most models; however, residual discrepancies remain for models with weaker short-term adaptability. TCNA mitigates many of these issues by combining convolutional feature extraction with attention-based contextual modeling, maintaining close alignment with the simulated PDEVS trajectories for turnaround time in both the early and late stages of processing, and for throughput once the initial transient has passed. Across small, medium, and large lot sizes, TCNA shows little variation in error behavior, indicating strong robustness to changes in batch configuration. TFT achieves competitive accuracy, and is more accurate than TCNA at several initial and final time steps, whereas TCNA attains comparable aggregate accuracy with a smaller model (§ 5.4). These findings suggest that explicitly modeling both local and global temporal dependencies is essential for better and reliable prediction in semiconductor fabrication manufacturing systems.
Conclusion and Future Work
Semiconductor factories involve complex processes and feedforward/feedback flows for fabricating many kinds of chips. In these factories, wafers undergo many incremental processing steps spanning hours to weeks in connected machines with re-entrant flows. Essential to the operation of such factories is accurately predicting throughput () and turnaround time () dynamics for individual and multiple machines, repeatedly and as fast as possible (e.g., every hour). This paper presented a Temporal Convolutional Network with Multi-head Attention (TCNA) to predict these essential dynamics. Simulated data sets of a benchmark discrete-event semiconductor manufacturing factory, processing wafer lots with uniform and sinusoidal wafer arrival profiles and subject to preventive maintenance, were used to develop TCNA and the comparison models. The and time series predicted by TCNA, evaluated with , , and and in terms of their dynamics at the beginning and end of wafer processing, show satisfactory accuracy with comparable parameter counts and memory utilization relative to the other temporal ML models.
Future work will extend this study in two directions. First, the evaluation will move beyond the point-accuracy measures used here (, , and ) to a multi-metric assessment [34]: the mean absolute scaled error (), which scales errors by the typical one-step change in the training data and remains well defined; accumulated and position-weighted errors (IAE, ISE, and ITAE), which reveal persisting and accumulated errors in long horizon trajectories; dynamic time warping (DTW) and slope errors, which assess the shapes of trajectories; peak and transition errors, which assess the timing of events; and frequency-domain errors. Such measures can rank models differently than point accuracy does and are better suited to the piecewise-constant state transitions, event-driven I/O trajectories of discrete-event simulation. Second, the architecture will be extended to dynamic covariates. The present model conditions only on static covariates (lot size, wafer-arrival profile, and repair type). The future-time branch shown dashed in Figure 1 embeds known-future inputs, such as the event times of the steps to be predicted in a discrete-event trajectory, and joins them with the TCN summary and the static embedding in the output layer; time-varying inputs, such as the scaled simulation time, machine availability, or planned wafer releases, can likewise be added as further channels of the input window. Evaluating TCNA with these dynamic covariates and the multi-metric measures on discrete-event trajectories remains future work.
References
- [1]C. V. Goldman, M. Baltaxe, D. Chakraborty, and J. Arinez, “Explaining learning models in manufacturing processes,” Procedia Comput. Sci., vol. 180, pp. 259–268, 2021.DOI
- [2]L. Lingitz, V. Gallina, F. Ansari, D. Gyulai, A. Pfeiffer, W. Sihn, and L. Monostori, “Lead time prediction using machine learning algorithms: A case study by a semiconductor manufacturer,” Procedia CIRP, vol. 72, pp. 1051–1056, 2018.
- [3]M. Subramaniyan, A. Skoogh, H. Salomonsson, P. Bangalore, and J. Bokrantz, “A data-driven algorithm to predict throughput bottlenecks in a production system based on active periods of the machines,” Comput. Ind. Eng., vol. 125, pp. 533–544, 2018.DOI
- [4]J. Spier and K. Kempf, “Simulation of emergent behavior in manufacturing systems,” in Proc. ESMI Adv. Semiconductor Manuf. Conf. Workshop, Cambridge, MA, USA, Nov. 13–15, 1995, pp. 90–94.
- [5]H. S. Sarjoughian, F. Fallah, S. Saeidi, and E. J. Yellig, “Transforming discrete event models to machine learning models,” in Proc. 2023 Winter Simulation Conf. (WSC), IEEE, 2023, pp. 2662–2673.DOI
- [6]V. K. Pendyala, H. S. Sarjoughian, B. Potenini, and E. J. Yellig, “A benchmark time-series dataset for semiconductor fabrication manufacturing constructed using component-based discrete-event simulation models,” arXiv preprint arXiv:2408.09307, 2024.arxiv.org/abs/2408.09307
- [7]H. Wu, J. Xu, J. Wang, and M. Long, “Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting,” in Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 34, 2021, pp. 22419–22430.arxiv.org/abs/2106.13008
- [8]T. Zhou, Z. Ma, Q. Wen, X. Wang, L. Sun, and R. Jin, “FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting,” in Proc. 39th Int. Conf. Mach. Learn. (ICML), 2022, pp. 27268–27286.arxiv.org/abs/2201.12740
- [9]S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Comput., vol. 9, no. 8, pp. 1735–1780, Nov. 1997. DOI: 10.1162/neco.1997.9.8.1735.DOI
- [10]B. N. Oreshkin, D. Carpov, N. Chapados, and Y. Bengio, “N-BEATS: Neural basis expansion analysis for interpretable time series forecasting,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2020.arxiv.org/abs/1905.10437
- [11]S. Bai, J. Z. Kolter, and V. Koltun, “An empirical evaluation of generic convolutional and recurrent networks for sequence modeling,” arXiv preprint arXiv:1803.01271, 2018.arxiv.org/abs/1803.01271
- [12]B. Lim, S. Ö. Arık, N. Loeff, and T. Pfister, “Tempo-ral fusion transformers for interpretable multi-horizon time series forecasting,” Int. J. Forecasting, vol. 37, no. 4, pp. 1748–1764, 2021.arxiv.org/abs/1912.09363
- [13]A. C. H. Chow and B. P. Zeigler, “Parallel DEVS: A parallel, hierarchical, modular modeling formalism,” in Proc. Winter Simulation Conf., IEEE, 1994, pp. 716–722.
- [14]ACIMS, “DEVS-Suite simulator, version 7.0,” 2023. [Online]. Available: https://acims.asu.edu/devs-suite/
- [15]V. K. Pendyala, H. S. Sarjoughian, and E. J. Yellig, “Generating temporal convolutional network models from parallel DEVS models: Semiconductor manufacturing systems,” in Proc. 2024 Winter Simulation Conf. (WSC), IEEE, 2024, pp. 2265–2276.
- [16]K. C. Chan, M. Rabaev, and H. Pratama, “Generation of synthetic manufacturing datasets for machine learning using discrete-event simulation,” Prod. Manuf. Res., vol. 10, no. 1, pp. 337–353, 2022.DOI
- [17]R. Godahewa, C. Bergmeir, G. I. Webb, R. J. Hyndman, and P. Montero-Manso, “Monash time series forecasting archive,” arXiv preprint arXiv:2105.06643, 2021.arxiv.org/abs/2105.06643
- [18]A. Hussein and M. Awad, “Time series forecasting of electricity consumption using hybrid model of recurrent neural networks and genetic algorithms,” Measurement: Energy, vol. 2, 2024, Art. no. 100004. DOI: 10.1016/j.meaene.2024.100004.
- [19]J. Zheng and M. Huang, “Traffic flow forecast through time-series analysis based on deep learning,” IEEE Access, vol. 8, pp. 82562–82570, 2020. DOI: 10.1109/ACCESS.2020.2990738.DOI
- [20]J. Lorenz, F. Durst, and H. Ryssel, “Process simulation for the semiconductor industry,” in Numerical Simulation in Science and Engineering: Proc. FORTHWIR Symp. High Performance Scientific Computing, München, Germany, June 17–18, 1993. Wiesbaden, Germany: Vieweg+Teubner, 1994, pp. 115–129.DOI
- [21]D. Huang, H. S. Sarjoughian, W. Wang, G. Godding, D. E. Rivera, K. G. Kempf, and H. Mittelmann, “Simulation of semiconductor manufacturing supply-chain systems with DEVS, MPC, and KIB,” IEEE Trans. Semicond. Manuf., vol. 22, no. 1, pp. 164–174, 2009. DOI: 10.1109/TSM.2008.2011680.DOI
- [22]A. Zeng, M. Chen, L. Zhang, and Q. Xu, “Are transformers effective for time series forecasting?” in Proc. AAAI Conf. Artif. Intell., vol. 37, no. 9, 2023, pp. 11121–11128.arxiv.org/abs/2205.13504
- [23]Q. Wen, T. Zhou, C. Zhang, W. Chen, Z. Ma, J. Yan, and L. Sun, “Transformers in time series: A survey,” arXiv preprint arXiv:2202.07125, 2022.arxiv.org/abs/2202.07125
- [24]Q. Ren, Y. Li, and Y. Liu, “Transformer-enhanced periodic temporal convolution network for long short-term traffic flow forecasting,” Expert Syst. Appl., vol. 227, Art. no. 120203, 2023.DOI
- [25]S. Lin, W. Lin, W. Wu, H. Chen, and J. Yang, “SparseTSF: Modeling long-term time series forecasting with 1k parameters,” in Proc. 41st Int. Conf. Mach. Learn. (ICML), 2024. arXiv:2405.00946.arxiv.org/abs/2405.00946
- [26]R. Ilbert, A. Odonnat, V. Feofanov, A. Virmaux, G. Paolo, T. Palpanas, and I. Redko, “SAMFormer: Unlocking the potential of transformers in time series forecasting with sharpness-aware minimization and channel-wise attention,” in Proc. 41st Int. Conf. Mach. Learn. (ICML), 2024. arXiv:2402.10198.arxiv.org/abs/2402.10198
- [27]D. Luo and X. Wang, “ModernTCN: A modern pure convolution structure for general time series analysis,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2024.
- [28]Y. Liu, T. Hu, H. Zhang, H. Wu, S. Wang, L. Ma, and M. Long, “iTransformer: Inverted transformers are effective for time series forecasting,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2024. arXiv:2310.06625.arxiv.org/abs/2310.06625
- [29]Y. Zhen, J. Fang, X. Zhao, J. Ge, and Y. Xiao, “Temporal convolution network based on attention mechanism for well production prediction,” J. Petrol. Sci. Eng., vol. 218, Art. no. 111043, 2022. DOI: 10.1016/j.petrol.2022.111043.DOI
- [30]J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, “Gated feedback recurrent neural networks,” in Proc. 32nd Int. Conf. Mach. Learn. (ICML), 2015, pp. 2067–2075.arxiv.org/abs/1502.02367
- [31]A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 30, 2017, pp. 5998–6008.arxiv.org/abs/1706.03762
- [32]J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” arXiv preprint arXiv:1607.06450, 2016.arxiv.org/abs/1607.06450
- [33]Y. N. Dauphin, A. Fan, M. Auli, and D. Grangier, “Language modeling with gated convolutional networks,” in Proc. 34th Int. Conf. Mach. Learn. (ICML), 2017, pp. 933–941.
- [34]V. K. Pendyala, H. S. Sarjoughian, and E. J. Yellig, “A deep learning-based multi-model, multi-metric forecasting framework for parallel DEVS semiconductor manufacturing simulation,” in preparation, 2026.