WorksComputer Science
Enhancing Edge AI System Security with a Stacked Hardware Security Controller for Robust and Energy-Efficient Anomaly Detectio
Abstract
In recent years, the growing adoption of Edge AI systems has been accompanied by a surge in security threats. Edge AI deployments face an rapidly evolving spectrum of security threats, from side-channel information leakage to ad- versarial model manipulation and physical device tampering. Conventional security mechanisms such as malware scanners and resource-intensive firewall solutions are unsuitable due to edge AIs typically limited computational resources, strict energy budgets and thermal constraints especially in battery-powered and field-deployed environments. This paper proposes a stacked, hardware-based security mechanism that enables low-overhead, continuous detection of anomalous execution patterns. Anomaly detection utilizing hard- ware performance counters and physical sensors outperforms software-based alternatives by providing key capabilities such as tamper-resistant root-of-trust isolation, negligible runtime overhead, and resilience against compromised operating systems. Experimental analysis shows that adversarial or malicious ap- plications produce observable deviations in sensor and thermal readings on Edge AI systems and hardware-based anomaly detection provides effective and low-overhead anomaly detection.
Introduction
Edge AI systems have witnessed substantial growth recently [20], [21],characterized by significant advancements in computational capabilities and an increasing number of deployments across diverse application domains [1], [5]. Their current deployments span numerous domains including autonomous vehicles, smart cameras, wearable medical devices, robotic platforms and smart city infrastructures. These cyber-physical systems utilize a wide range of hardware devices from smaller battery powered sensor devices to higher power processors and graphics processing units.
The rapid growth in the number of deployments has also fueled a large number of novel cyberattacks, ranging from physical device access-based attacks and side-channel attacks [26]to snooping attacks in multi-access edge devices [9], [11], [13]. Despite this increase in the number and diversity of cyberattacks, limitations in power, thermal, and computational budgets of edge AI devices challenge the feasibility of conventional software-based security solutions, posing a two-pronged challenge.
In this paper, we propose a stacked hardware security unit to continuously monitor and detect malicious and anomalous behavior with minimal overhead using hardware sensor infrastructure capabilities. Stacked hardware security unit provides fine-grain local access to the underlying hardware macros and sensors by sitting on top of the Edge AI device layer. Furthermore, unlike software-dependent methods, hardware-backed anomaly detection that leverages performance counters and physical sensors (i) achieves superior efficacy by establishing tamper-resistant hardware isolation, (ii) introduces minimal runtime overhead, and (iii) maintains resilience against compromised host edge AI operating systems. Beyond conventional micro-architectural monitoring, the proposed AI Guard-Layer architecture integrates advanced semantic activity tracking and multi-layered verification mechanisms, establishing a comprehensive defense framework for edge AI deployments [19].
Our experimental analysis shows that malicious applications generate distinct thermal profiles on edge AI architectures. Using underlying hardware and thermal sensors, the stacked security architecture discriminates between normal behavior and anomalous miner application behavior. The paper is organized as follows: Section II discusses the background and the growing cybersecurity challenges; Section III proposes a stacked hardware security controller approach; Section IV presents and discusses experimental analysis and results; and finally Section V provides concluding remarks.
Edge AI Security
Background and Security Challenges
With the rapid growth in edge AI deployments [5], [8], security challenges have proliferated [1], [3], [22]. A wide range of attack types have been reported from adversarial [23] and trojan attacks [4], malware attacks [9] to data poisoning [7], [10] and DDoS [6]. In addition, a large number of application specific cyberattacks have been targeting smart vehicles, drone systems [11], robotic systems, wearable medical devices, intelligent IoT devices and other edge AI systems.
Despite the exacerbating cybersecurity challenges, edge AI devices have serious resource limitations [14], [15] in contrast to the standard cloud-based AI systems:
(i) Energy and Battery Limitations: Edge AI devices frequently operate on battery power and have severe energy constraints, which limit their ability to run energy hungry security software and applications.
(ii) Communication Bandwidth: Similarly, communication bandwidth is constrained compared to the cloud-based AI systems, which limits the network-level security solutions.
(iii) Thermal Packaging Constraints: As edge AI devices are often embedded or wearable systems, they are often physically constrained to fully sealed, or passively cooled enclosures designed to survive harsh ambient conditions.
(iv) Computational Constraints: Edge AI devices have limited computational power to continuously monitor malicious activity, which becomes a bottleneck for software-based techniques.
(v) Physical Device Security: As edge AI devices are deployed in the field and often in adversarial environments, they are nearly impossible to fully physically secure, which makes it easier to install and run malicious code that conceals its activity patterns.
As a result, traditional software-based security solutions are not effective in edge AI systems. Their large overheads pose serious risks for full battery depletion and brownout, which causes degraded output quality due to resource constraints and trade offs. In addition, a large number of cyberattacks including DDoS, malware and trojan attacks throttle and strain the underlying hardware resources. This often reduces the bandwidth available

Figure 1. (i) 3D guard-layer configuration with one or more edge intelligence layers and stacked 3D guard-layer, (ii) 2.5D Silicon interposer configuration with multiple edge intelligence chips and a dedicated hardware security guard-layer interposer connected to all AI chips.
for detection and remediation, but it provides a unique opportunity to detect anomalous activity patterns.
Stacked Security Layer
Active 2.5D interposer and 3D integrated packaging, commonly used in embedded systems and edge AI solutions, provide a unique opportunity to address these challenges [2], [16], [17]. Stacking provides local and direct access to the underlying edge AI hardware through TSV or micro-bump level connections. This high-bandwidth, highly-localized access provides a chance to closely monitor the edge AI system at a fine-granularity (at the hardware macro level) through both hardware activity counters and physical (thermal and electromagnetic) monitoring perspective.
In this paper, we propose a stacked hardware security architecture (2.5D or 3D) vertically integrated with the primary edge AI chip. This stacked controller: (i) continuously monitors on-chip activity directly and indirectly through activity and hardware counters, thermal sensors, noise sensors and other sensors (ii) continuously learns from underlying hardware activity from the streaming data, (iii) detects anomalous activity by classifying normal and anomalous activity patterns locally with minimal performance overhead.
In addition, the stacked cybersecurity architecture provides additional capabilities such as (i) physical guarding against side-channel attacks through 3D stacking (ii) shadow processing of the streaming data for data poisoning prevention (iii) AI safety monitoring [12], [13] (iv) redundant processing capabilities for hardware resilience. Figure 1 shows the stacking configurations using 3D integration with the edge AI hardware security controller (i) and 2.5D interposer version (ii) to monitor multi-chip edge AI systems.
Experimental Analysis and Results
Experimental Setup
Hardware telemetry was collected on an NVIDIA L40S GPU (48 GB GDDR6, 350 W TDP) allocated via the NCSU HPC cluster (IBM Spectrum LSF scheduler, short_gpu queue). L40S was selected for its use in a range of edge AI applications. All jobs were submitted with mode=exclusive_process GPU allocation to prevent co-tenant interference, an important control given that prior runs using shared scheduling mode produced contaminated idle data with 99% spurious SM utilization from co-scheduled user jobs.
Data collection used a custom NVML-based collector (Python/pynvml) operating at 10 Hz, capturing 20 hardware telemetry metrics per polling cycle: GPU temperature, power draw, SM and memory utilization, SM and memory clock frequencies, PCIe RX/TX throughput, fan speed, ECC single-bit and double-bit volatile error counters, and supporting identifiers. All data was written directly to Parquet on cluster-attached storage without network dependency, ensuring no collection overhead on the GPU under test. The following workload conditions were collected on NVIDIA L40S GPUs of the same model and configuration:
Idle: No user processes attached. GPU in driver-managed power state. 300-second collection, 59,660 metric-level data points (20 channels, 2,983 polling cycles).
Training: Sustained FP32 matrix-multiply () via CuPy, 100% intensity with no sleep pacing. 300-second collection, 59,400 metric-level data points.
Inference: Batch inference simulation: two-layer MLP forward pass () with 40 ms inter-batch sleep gaps. Produces lower, burstier SM activity than training.
Proxy Miner: Workload structured to emulate cryptomining behavior. The compute pattern differs from the training workload in its SM scheduling behavior, as described in § 4-B. 300-second collection, 59,840 metric-level data points.
DDoS: Repeated 1 GB GPU→CPU DMA transfers using pinned memory. No compute kernels. Simulates covert model-weight exfiltration via CUDA DMA engine.
RowHammer: Custom CUDA kernel hammers aggressor DRAM rows using volatile loads. 192 MB buffer exceeds 48 MB L2 cache, forcing DRAM row activations. Simulates double-sided DRAM attack [24, 25].
| Workload | Class | –mode flag | Description |
| Idle | Normal | idle_0pct | GPU reserved (excl. proc.), no compute process. Minimum thermal and power floor. No gpu_stress.py — collector only. |
| Training (100%) | Normal | --mode train | Sustained FP32 matrix-multiply () via CuPy, 300 s. Saturates all 18,176 CUDA cores. Representative of DNN training. |
| Inference (75%) | Normal | --mode infer | Two-layer MLP forward pass (), 40 ms inter-batch sleep. Lower, burstier SM activity than training. |
| Proxy Miner | Anomaly | --mode train* | Same CUDA matrix-multiply kernel as Training, divergent SM scheduling behavior; labels Parquet as workload=miner. 0% SM util despite elevated power and pinned clocks. *No --mode miner in gpu_stress.py. |
| DDoS/Exfil | Anomaly | --mode ddos | Repeated 1 GB GPU→CPU DMA via pinned memory. No compute kernels. Simulates model-weight exfiltration via CUDA DMA engine. |
| RowHammer | Anomaly | --mode rowhammer | CUDA kernel hammers DRAM rows via volatile loads. 192 MB buffer exceeds 48 MB L2, forcing row activations. Double-sided DRAM attack [24, 25]. |
Table I. Descriptions for Idle, Training, Inference, Proxy Miner, DDoS and RowHammer workloads used in experimental analysis.
Data collection used 10 Hz polling. Temporal stability (T1) was verified via coefficient of variation (CoV = std/mean × 100%) for temperature across each run. All conditions except RowHammer (CoV = 11.46%) achieved CoV < 9%; note that for workloads with an initial thermal ramp (Training, RowHammer), CoV is computed over the full 300-second window. ECC volatile error counters were zero across all conditions. Hardware metrics were collected at user privilege via NVML without root access, consistent with the threat model in which a hardware security monitor operates within normal OS privilege boundaries.
Experimental Results
We collected hardware telemetry traces on an NVIDIA L40S GPU [18] across six workload conditions: idle, sustained matrix-multiply training at 100% intensity, inference, a proxy cryptominer workload, DDoS and RowHammer. All traces were collected at 10 Hz via NVML using exclusive GPU mode to eliminate co-tenant interference. Twenty hardware metrics were captured per polling cycle including temperature, power draw, SM and memory clock frequencies, SM utilization, PCIe throughput, and ECC error counters.
Idle Baseline: At idle the L40S maintained a stable thermal floor of 29.8°C (CoV = 2.4%) at 44.6 W mean power, with SM clock cycling dynamically between 0 and 1755 MHz. This establishes the reference distribution against which anomalous workloads are detected.
Legitimate Training Workload: Sustained matrix-multiply execution produced an immediate power step to 348.9 W and a characteristic thermal ramp from the idle floor to a steady-state equilibrium of 66.0°C over approximately 120 seconds. +36.2°C delta is shown above the idle baseline. SM utilization registered 99.7% throughout, SM clock stabilized at 1478 MHz, and PCIe RX throughput increased to 524 KB/s. These signatures collectively characterize a legitimate high-intensity compute workload.
Proxy Miner (Malicious Workload): The miner workload produced a qualitatively different profile: power stabilized at 87.2 W (CoV = 0.5%), double the idle baseline but one-quarter of the training load, while SM utilization reported 0% throughout the entire collection window. Despite zero reported SM utilization, the SM clock held at a mean of 2422 MHz (range 2370–2520 MHz) and memory clock locked at 9000 MHz, indicating that the GPU memory subsystem and clock infrastructure remained active. The divergence between reported SM utilization and pinned clock state is characteristic of cryptomining code paths that drive GPU power and memory interfaces while evading software-visible SM occupancy counters. Temperature rose only 5.4°C above idle, reaching a stable 35.2°C.
This combination — elevated power and maximum clock frequencies at reported zero SM utilization — constitutes a distinct anomalous signature uncharacteristic of any legitimate workload class.
DDoS: This mode contains no compute kernel. It allocates a 1 GB GPU buffer and a 1 GB pinned CPU buffer, then loops a single CUDA DMA transfer. Pinned (page-locked) memory is the key detail: it lets the PCIe DMA engine move data directly between VRAM and RAM without CPU or SM involvement. Because no CUDA kernels execute, SM utilization is 0% and the SM clock parks at its minimum (210 MHz). Power drops to 33.6 W (below the idle baseline of 44.6 W) because with nothing to compute, only the memory controller and PCIe interface are active. The data exfiltration workload is anomalously quiet in power and temperature. Detection requires combining sub-idle power with elevated NVML-reported PCIe TX (603 KB/s mean per NVML telemetry). Neither signal alone is sufficient; the combination is uniquely diagnostic.
RowHammer: The resulting telemetry signature is unique among all six classes: 100% SM utilization (unlike miner, inference, or DDoS) but only 237 W (versus training’s 349 W at equivalent SM saturation) because the workload is constrained by memory bandwidth rather than compute throughput. The GPU temperature ramps from 39°C to 65°C over 300 seconds (CoV 11.46%, highest of all classes), consistent with progressive thermal accumulation from sustained high-frequency memory access — noting that NVML temperature reflects the GPU die sensor rather than DRAM temperature directly.
Anomaly Detection Using Hardware Counters and Sensors
Hardware activity counters and thermal sensors provide an efficient way to track the underlying activity in the edge AI layers [27], [28]. Figure 2 plots all experimented workloads in two feature projections. Left panel (SM utilization vs. power): three clusters are immediately visible — Training (100% SM util, 349 W), RowHammer (100% SM util, 237 W), and a dense lower-left bundle containing the remaining four classes (Idle, Inference, Miner, DDoS) all at 0% reported SM utilization. Right panel (NVML PCIe TX vs. power) resolves the lower-left bundle: Miner is unique at 41 KB/s PCIe TX; Inference (627 KB/s), DDoS (603 KB/s), and RowHammer (752 KB/s) all show elevated PCIe TX, separating from Miner by over an order of magnitude. Combined across both panels, all six classes are discriminable.

Figure 2. Anomaly fingerprint. Left: SM utilization vs. power. Right: NVML PCIe TX vs. power. Six-class separation.
Figure 3 summarizes the key metrics across all workload classes (mean std). The classes occupy largely non-overlapping regions in this metric space. The SM Clock panel is particularly diagnostic: DDoS parks at 210 MHz (fully parked, DMA-only), while all compute-active workloads operate at 1478–2520 MHz.

Figure 3. Metric comparison of the normal and anomalous workloads.
Figure 4 shows the 300-second temperature and power traces for all six workload classes. Training ramps from 32°C to 70°C then stabilizes near 66°C. RowHammer (teal) shows a monotonic thermal ramp from 39°C to 65°C throughout the full 300 s (no plateau), consistent with progressive thermal accumulation from sustained memory-intensive operation. DDoS (orange) sits flat at 25°C, below the idle baseline of 29.8°C. In the power panel, three distinct tiers are immediately visible: Training at 349 W, RowHammer at 237 W, and the four remaining workloads clustered below 90 W.
Figure 5 shows the layered feature hierarchy across four projections. Layer 4 (Temperature vs. SM Clock, bottom-right) achieves near-complete separation of all six classes when using mean or steady-state values: DDoS isolated at 25°C / 210 MHz; Idle scattered across low-clock P-states; Inference and RowHammer lock at 2520 MHz while Miner clusters near 2422 MHz (range 2370–2520 MHz), all at distinct temperatures; Training at 1478 MHz mean / 50–70°C. It is important to note that training begins cold at 32°C with an initial SM clock near 2520 MHz, transiently overlapping the

Figure 4. Temperature (top) and power (bottom) time series for all six workload classes on NVIDIA L40S over 300 s.

Figure 5. Feature hierarchy: four projections showing layered discrimination from SM util → power → PCIe TX → temperature vs. SM clock.
Inference region during the first ∼30 s of ramp-up. Single-projection separation is therefore not perfect for raw per-sample point clouds; aggregate features or a classifier that handles startup transients achieves full separation.
While baseline hardware anomaly detection, using performance counters and physical sensors, may exhibit coverage gaps in detection for stealthy, low-footprint threats like passive data exfiltration or silent backdoors, the broader architecture of the proposed stacked hardware security controller [19] integrates semantic activity tracking and supplementary verification mechanisms designed to capture such subtle abnormalities. Although a comprehensive empirical evaluation of these advanced semantic modules falls outside the scope of the current study, their integration underscores the extensible nature of the proposed framework beyond conventional microarchitectural monitoring.
Conclusions
Edge AI systems face a growing number and diversity of cyberattacks in the field. Software-based malware detection in edge AI systems is considerably more challenging because of limited energy resources, battery capacity, thermal budgets, computational capabilities, and communication bandwidths. As a result, conventional software-based cybersecurity solutions are often impractical or cause serious issues during deployments.
In this paper, we propose a stacked cybersecurity controller unit that continuously monitors and analyzes the underlying activity from hardware counters and on-chip sensors. We show that this low-overhead hardware monitoring approach effectively discriminates between normal activity (including high-compute training periods) and anomalous activity, with distinct cluster separation across six workload classes that enables accurate anomaly classification.
References
- [1]Z. Huang et al., “Survey on edge computing security,” in Proc. Int. Conf. Big Data, Artif. Intell. Internet Things Eng. (ICBAIE), Jun. 2020, pp. 96–105, doi: 10.1109/ICBAIE49996.2020.00027.DOI
- [2]P. Garrou et al., Eds., Handbook of 3D Integration, Vol. 1: Technology and Applications of 3D Integrated Circuits. Weinheim, Germany: Wiley-VCH, 2008, doi: 10.1002/9783527623051.
- [3]L. Albshaier et al., “Federated learning for cloud and edge security: A systematic review of challenges and AI opportunities,” Electronics, vol. 14, no. 5, 2025, Art. no. 1019.DOI
- [4]S. P. Xu et al., “An interpretive perspective: Adversarial trojaning attack on neural-architecture-search enabled edge AI systems,” IEEE Trans. Ind. Informat., vol. 19, no. 1, pp. 503–510, 2023.DOI
- [5]S. S. Gill et al., “Edge AI: A taxonomy, systematic review and future directions,” Cluster Comput., vol. 28, no. 1, 2025, Art. no. 18, doi: 10.1007/s10586-024-04686-y.DOI
- [6]Q. He et al., “A game-theoretical approach for mitigating edge DDoS attack,” IEEE Trans. Dependable Secure Comput., vol. 19, no. 4, pp. 2333–2348, 2022.DOI
- [7]J. Zhang et al., “PoisonGAN: Generative poisoning attacks against federated learning in edge computing systems,” IEEE Internet Things J., vol. 8, no. 5, pp. 3310–3322, 2021.DOI
- [8]T. Meuser et al., “Revisiting edge AI: Opportunities and challenges,” IEEE Internet Comput., vol. 28, no. 4, pp. 49–59, 2024, doi: 10.1109/MIC.2024.3383758.DOI
- [9]C. Wang et al., “The security and privacy of mobile-edge computing: An artificial intelligence perspective,” IEEE Internet Things J., vol. 10, no. 24, pp. 22008–22032, 2023.arxiv.org/abs/2401.01589
- [10]Y. Chen et al., “Data poisoning attacks in internet-of-vehicle networks: Taxonomy, state-of-the-art, and future directions,” IEEE Trans. Ind. Informat., vol. 19, no. 1, pp. 20–28, 2023.DOI
- [11]J.-P. Yaacoub et al., “Security analysis of drones systems: Attacks, limitations, and recommendations,” Internet Things, vol. 11, 2020, Art. no. 100218.DOI
- [12]D. Amodei et al., “Concrete problems in AI safety,” arXiv preprint arXiv:1606.06565, 2016.arxiv.org/abs/1606.06565
- [13]Y. Hu et al., “Artificial intelligence security: Threats and countermeasures,” ACM Comput. Surv., vol. 55, no. 1, Art. no. 20, pp. 20:1–20:36, 2023, doi: 10.1145/3487890.
- [14]A. Sehgal et al., “Management of resource constrained devices in the internet of things,” IEEE Commun. Mag., vol. 50, no. 12, pp. 144–149, 2012.DOI
- [15]F. Pereira et al., “Challenges in resource-constrained IoT devices: Energy and communication as critical success factors for future IoT deployment,” Sensors, vol. 20, no. 22, 2020, Art. no. 6420.DOI
- [16]C. S. Tan, K.-N. Chen, and S. J. Koester, Eds., 3D Integration for VLSI Systems. Singapore: Pan Stanford, 2011.
- [17]X. Zhang et al., “Heterogeneous 2.5D integration on through silicon interposer,” Appl. Phys. Rev., vol. 2, no. 2, 2015, Art. no. 021308, doi: 10.1063/1.4921463.DOI
- [18]NVIDIA Corporation, “NVIDIA L40S GPU.” [Online]. Available: https://www.nvidia.com/en-us/data-center/l40s/. Accessed: Sep. 2026.
- [19]E. Kurshan and P. Franzon, “AI Guard-Layer: An Integrated AI Safety and Security Subsystem for Edge Artificial Intelligence,” in Proc. IEEE 3D Systems Conf., 2026.
- [20]W. Shi et al., “Edge computing: Vision and challenges,” IEEE Internet Things J., vol. 3, no. 5, pp. 637–646, Oct. 2016.DOI
- [21]Z. Zhou et al., “Edge intelligence: Paving the last mile of artificial intelligence with edge computing,” Proc. IEEE, vol. 107, no. 8, pp. 1738–1762, Aug. 2019.arxiv.org/abs/1905.10083
- [22]Y. Xiao et al., “Edge computing security: State of the art and challenges,” Proc. IEEE, vol. 107, no. 8, pp. 1608–1631, Aug. 2019.DOI
- [23]I. J. Goodfellow et al., “Explaining and harnessing adversarial examples,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2015.DOI
- [24]Y. Kim et al., “Flipping bits in memory without accessing them: An experimental study of DRAM disturbance errors,” in Proc. Int. Symp. Comput. Archit. (ISCA), 2014, pp. 361–372.DOI
- [25]P. Jattke et al., “GPUHammer: Rowhammer attacks on GPU memories are practical,” in Proc. USENIX Security Symp., 2025.
- [26]P. Kocher et al., “Differential power analysis,” in Proc. Adv. Cryptol. (CRYPTO), 1999, pp. 388–397.DOI
- [27]J. Demme et al., “On the feasibility of online malware detection with performance counters,” in Proc. Int. Symp. Comput. Archit. (ISCA), 2013, pp. 559–570.
- [28]A. Tang et al., “Unsupervised anomaly-based malware detection using hardware features,” in Proc. Int. Symp. Res. Attacks, Intrusions, Defenses (RAID), 2014, pp. 109–129.arxiv.org/abs/1403.1631