WorksEngineering
Pixel-KWS: Rendered-Glyph Representations for Orthographically Robust Open-Vocabulary Keyword Spotting
Abstract
Open-vocabulary keyword spotting (KWS) allows users to enroll arbitrary keywords at inference time. However, character and phoneme-based text encoders rely on fixed symbol vocabularies learned during training, limiting their ability to handle unseen or visually similar symbols such as homoglyphs, l33tspeak substitutions, and accented characters. We propose PIXEL-KWS, an open-vocabulary KWS framework that represents characters through rendered glyphs rather than learned embedding lookups. We render each character as a grayscale image and encode it with a lightweight CNN, enabling the text encoder to generalize from visual form rather than a fixed vocabulary. On LibriPhrase, PIXEL-KWS and its matched character-based counterpart achieve 9.57% and 9.55% EER on LibriPhrase-hard, 0.43% and 0.41% EER on LibriPhrase-easy, outperforming all published baselines without pre-training or synthetic data. PIXEL-KWS provides substantially greater robustness to both homoglyph and l33tspeak symbol-level perturbations. These results highlight glyph-based representations as a simple and effective approach for robust open-vocabulary KWS in practical settings.
Introduction
Open-vocabulary keyword spotting (KWS) matches speech with a keyword supplied by the user as text, allowing new keywords to be enrolled without additional speech examples [1, 2, 3, 4]. The text encoder operates directly on user-provided input, making its tokenizer is important design. Most existing approaches use either character-based or phoneme-based tokenizers. Character-based or grapheme-to-phoneme (G2P) tokenizers typically represent each symbol using a learnable embedding lookup table, resulting in a closed vocabulary. This limitation is particularly relevant to practical user input, where keywords may contain homoglyphs, l33tspeak substitutions, accented characters, or other unseen symbols as shown in Fig:Figure 2. Such symbols can naturally arise through copy-and-paste or other forms of user input, exposing a fundamental limitation of fixed-vocabulary.
Figure 2. Visualization of the enrolment string under character noise, homoglyph, and l33tspeak corruption.
To remove limitations on fixed vocabulary, NLP research has explored representing text through its visual form. Salesky et al. [5] render source text as images for machine translation and show improved robustness to orthographic noise. PIXEL [6] further pre-trains a language model directly on rendered text, enabling transfer across scripts without relying on a fixed textual vocabulary. Similar visual text representations have also been explored in speech synthesis as alternatives to conventional character representations [7, 8, 9]. While Pixel-TTS [9] uses image-based text rendering for speech synthesis, its robustness is evaluated using speech quality and intelligibility measures under textual noise. In contrast, open-vocabulary KWS requires keyword representations that remain matchable to audio under enrollment-time symbol corruption. These studies suggest that visual representations can improve robustness to textual variation and generalize beyond symbols seen during training [10].
To the best of our knowledge, no open-vocabulary KWS system directly represents user-provided keywords as rendered text images. We introduce PIXEL-KWS, an open-vocabulary KWS framework that renders each character of a keyword as a 16 × 16 grayscale image and encodes the resulting sequence with a lightweight CNN, replacing the conventional character-embedding lookup table. Our contributions are:
We introduce image-rendered glyph representations for open-vocabulary KWS, while retaining keyword enrollment.
Without any synthetic data or pre-training, both our systems outperform all published baselines (9.55% and 9.57% EER on LP), so the robustness comparison is made against a strong rather than a weak one.
We evaluate robustness to orthographic noise by introducing homoglyph and l33tspeak corruptions at varying corruption probabilities.
We show that the gains are particularly evident for visually similar substitutions: under full homoglyph corruption, PIXEL-KWS degrades by only 11.2% EER, compared with 31.3% for the character baseline.
Figure 1. Overview of the proposed PIXEL-KWS framework.
We systematically evaluate character image resolutions of , , and , showing that resolution affects KWS performance.
PIXEL-KWS METHODOLOGY
Character-to-Image representation
Let denote the sequence of characters in the enrolment keyword. Each character is rendered a grayscale character image called glyph using the bold monospaced DejaVu Sans Mono font. Each character is cached, and indexed by character ID to avoid repeated rendering. The word-break character (space) is represented by a white-space image and distinguished from padding using the sequence mask. This preserves word boundaries in multi-word keywords (phrases), preventing distinct keywords such as “a bridge” and “abridge” from becoming visually indistinguishable.
These glyph representations are processed by three convolutional layers with channel dimensions , with stride 2 in the first two layer. Each convolutional layer is followed by GroupNorm. The resulting feature map is flattened and linearly projected to a 64-dimensional character representation. The representations are then processed by a bidirectional GRU to capture contextual dependencies:
where denotes the representation of the -th character and its contextualized representation. We then aggregate these representations using masked attentive pooling:
where is the attention weight assigned to the -th character. The pooled representation is linearly projected into the shared -dimensional latent space to obtain the keyword embedding .
Audio Keyword representation
The audio encoder consists of a convolutional subsampler followed by a 4-layer Conformer [11], which takes log-Mel spectrograms as input and produces frame-level representations. These representations are aggregated using masked attentive pooling and linearly projected into the shared -dimensional latent space to obtain the embedding of audio keyword .
Training Objective
Given paired audio-text keyword embeddings, we compute their pairwise cosine similarities to form an similarity matrix, where matched pairs lie on the diagonal and all other pairs act as in-batch negatives. We train the model with a bidirectional InfoNCE objective at temperature , considering both audio-to-text and text-to-audio matching:
We add a CTC head to the Conformer output to jointly predict the phoneme sequence corresponding to the audio keyword. We further calibrate the cosine similarity using a learnable affine transformation followed by a sigmoid.
where is reparameterized using a softplus function to ensure a positive scale. The overall objective combines the contrastive, CTC, and binary cross-entropy losses:
where is computed using the pair label, with for matched audio–text pairs and for negative pairs.
EXPERIMENTAL SETUP
Data. We train our models using LibriPhrase dataset[1], derived from the LibriSpeech train-clean-100 and train-clean-360 subsets [12]. We evaluate the proposed models on official LibriPhrase test set constructed from train-other-500, under easy (LP) and hard (LP) conditions. We hold out 10% of the training non-overlapping keyword trials as validation data for model selection. We additionally mine hard negatives for each keyword and include them during training. Following the reference evaluation protocols [2, 13], each test pair is evaluated in both directions, with the anchor audio matched against comparison text and the anchor text matched against comparison audio. We report both Equal Error Rate (EER) and Area Under the Receiver Operating Characteristic (AU-ROC) for each evaluation condition. The EER is computed at the operating point where the false acceptance rate and false rejection rate are equal, while AUROC summarizes the system’s discrimination performance across all possible decision thresholds.
Model and training. The audio encoder uses with 4 Conformer layers, 4 attention heads, and an FFN dimension of 128. The text encoder uses a 2-layer BiGRU with hidden size 128, and both modalities are projected into a 64-dimensional matching space. The character baseline (Char-KWS) has 0.847M parameters, while PIXEL-KWS has 0.863M parameters. We intentionally designed Glyph CNN module to be lightweight, so that performance differences cannot be attributed simply to increased model capacity. All the models are trained for 30 epochs using AdamW with a learning rate of and 2000 warmup steps, with a batch size of 128 per GPU across 4 GPUs using bf16 precision.
Results and analysis
Performance Analysis
Table 1 shows PIXEL-KWS performance on LibriPhrase at three model sizes. The tiny variant (T, 0.86M parameters) achieves 12.10% EER and 94.65% AUROC on LPH, and 0.60% EER and 99.97% AUROC on LPE, the best result among the embedding-based systems despite its substantially smaller size and the absence of pre-training. Increasing the encoder dimension to 128 (S) and 256 (M), improves LPH EER to 10.22% and 9.57%. At the medium size both of our systems outperform every posterior-based, cross-attention-based and embedding-based baselines. Char-KWS-M reaches 9.55% EER and 96.27% AUROC on LPH and PIXEL-KWS-M
Comparison of open-vocabulary KWS methods on LibriPhrase. Char-KWS and PIXEL-KWS are both ours and differ only in the first stage of the text encoder; T/S/M denote 64/128/256. IS: Interspeech; ICP: ICASSP; {sup}* ASR-based pre-training, {sup}† phrase-based, {sup}‡` synthetic data.
| Method | Venue | #Params | AUROC (%) ↑ LPH | AUROC (%) ↑ LPE | EER (%) ↓ LPH | EER (%) ↓ LPE |
| Posterior-based | ||||||
| DONUT [14] | ICP 21 | — | 62.55 | 78.74 | 41.95 | 28.74 |
| CTCAT [15] | IS 24 | 0.2M | 77.10 | 98.32 | 29.63 | 6.06 |
| W-CTC [16] | IS 25 | 3.6M | 95.93 | 99.95 | 10.21 | 0.91 |
| Cross-attention-based | ||||||
| CMCD [1] | IS 22 | 0.7M | 73.58 | 96.70 | 32.90 | 8.42 |
| PhonMatchNet [2] | IS 23{sup}*` | 0.7M | 88.52 | 99.29 | 18.82 | 2.80 |
| CED [3] | ICP 24{sup}†` | 3.6M | 92.70 | 99.84 | 14.40 | 1.70 |
| MM-KWS@T [13] | IS 24{sup}‡` | 3.9M | 95.36 | 99.94 | 10.41 | 0.82 |
| SLiCK [17] | ICP 25{sup}†` | 0.6M | 94.90 | 99.82 | 11.10 | 1.78 |
| PLCL@T [4] | ICP 25{sup}†` | 40M | 95.56 | 99.95 | 9.96 | 1.21 |
| Embedding-based | ||||||
| Triplet [18] | IS 19 | — | 54.88 | 63.53 | 44.36 | 32.75 |
| CLAD [19] | ICP 24 | 3.6M | 76.15 | 97.03 | 30.30 | 8.65 |
| SynapSpot-TA [20] | ICP 26 | 0.9M | 79.15 | 97.34 | 27.29 | 5.77 |
| ADML [21] | IS 25{sup}*` | 1.8M | 88.71 | 99.86 | 20.09 | 1.33 |
| MATE [22] | ICP 26 | — | 88.70 | 99.86 | 20.06 | 1.38 |
| Char-KWS-T (this work) | — | 0.85M | 94.41 | 99.97 | 12.40 | 0.63 |
| PIXEL-KWS-T (this work) | — | 0.86M | 94.65 | 99.97 | 12.10 | 0.60 |
| Char-KWS-S (this work) | — | 2.9M | 95.96 | 99.98 | 10.11 | 0.39 |
| PIXEL-KWS-S (this work) | — | 3.0M | 95.91 | 99.98 | 10.22 | 0.41 |
| Char-KWS-M (this work) | — | 11.4M | 96.27 | 99.98 | 9.55 | 0.43 |
| PIXEL-KWS-M (this work) | — | 11.4M | 96.25 | 99.98 | 9.57 | 0.41 |
Table 1.
9.57% and 96.25%, against PLCL@T, which uses 680k h of pre-training. Whereas our systems do not use any pre-training and synthetic data. Char-KWS is not a prior system but our own matched baseline, identical to PIXEL-KWS except for the first stage of the text encoder. This controlled comparison isolates the effect of the glyph-based representation while keeping the architecture and training fixed. On clean trials the two front-ends are equivalent at every size (12.40 versus 12.10%, 10.11 versus 10.22%, and 9.55 versus 9.57%).
A further advantage of our embedding based model is the enrolment cost. Because the text side is reduced to a single embedding, a keyword is encoded once and scored by an inner product, which is per enrolled keyword and need not be recomputed. Cross-attention systems fuse the audio and text representations and therefore repeat the computation for every enrolled keyword, while posterior-based systems run a per-keyword decoding search. PIXEL-KWS provides substantially improved robustness to orthographic variations and character-level perturbations, as discussed in the following subsection 4.2.
Orthographic robustness
We evaluate robustness of PIXEL-KWS using character-level perturbations. This setting is particularly relevant because visual similarity between original and corrupted characters can preserve keyword representations under orthographic noise. In contrast, phoneme or BPE-based representations convert corrupted characters into different phoneme or subword sequences and do not directly preserve this visual relationship. We consider two perturbations following [5, 9]: Homoglyphs and l33tspeak. Homoglyphs are characters from different scripts that look nearly identical but have different Unicode values, whereas l33tspeak replaces standard letters with visually similar numbers.
Each eligible character of the enrolment keyword is substituted independently with probability , and the audio is left unchanged, so any change in EER is attributable to the text encoder alone. Ten homoglyph and eight l33tspeak substitutions are defined, so at the two maps affect 67% and 42% of all characters. Figure 2 shows both corruptions applied to “the bridge is closed”, with boxes marking the substituted positions: 13 positions in the homoglyph row and 9 in the l33tspeak row. The boxed homoglyphs are visually almost indistinguishable from the letters they replace, whereas the boxed l33tspeak characters are clearly digits. Figure 3 present EERs across the full range of , and Table 2 reports the degradation from to . Under homoglyph corruption Char-KWS-M degrades by 31.316% on LP and 25.246% points on LP. But PIXEL-KWS-M degrades only 11.236% and 2.246% on LP and LP respectively. Under l33tspeak the corresponding degradations are 26.46% and 12.70% EERs for Char-KWS-M, and 22.07% and 8.9% EERs for PIXEL-KWS-M. Across the sweep, PIXEL-KWS-M obtains a lower EER than Char-KWS-M at every non-zero corruption probability, for both perturbation types and on both LP and LP.
EER (%) under enrolment-string corruption at probability ; audio remains unchanged. denotes degradation from to . Best results are shown in bold.
| Cond. | System | homoglyph | homoglyph | homoglyph | l33tspeak | l33tspeak | l33tspeak |
| LPE | Char-KWS-M | 0.43 | 25.68 | +25.24 | 0.43 | 13.13 | +12.70 |
| PIXEL-KWS-M | 0.41 | 2.65 | +2.24 | 0.41 | 9.31 | +8.90 | |
| LPH | Char-KWS-M | 59.5 | 40.85 | +31.31 | 9.55 | 36.01 | +26.46 |
| PIXEL-KWS-M | 9.57 | 20.81 | +11.23 | 9.57 | 31.65 | +22.07 |
Table 2.
Figure 3. EER against enrolment-string corruption probability . Homoglyph and l33tspeak substitutions contain characters outside the training corpus.
Ablation: Character Image Resolutions
Table 3 shows that PIXEL-KWS is relatively insensitive to the character image resolution. The 16 × 16 resolution achieves the best performance on LP_H, while 8 × 8 and 24 × 24 yield comparable results. Lower resolutions may discard fine-grained character-shape information, whereas higher resolutions introduce additional pixels that primarily represent white space around the glyph, increasing the computational and parameter overhead without providing a consistent performance gain. We therefore use 16 × 16 character images in all experiments, as it provides a good balance between retaining character details and computational efficiency.
| Image | #Params | LP | LP |
| 8 × 8 | 0.85M | 0.58 | 12.28 |
| 16 × 16 | 0.86M | 0.60 | 12.10 |
| 24 × 24 | 0.89M | 0.58 | 12.25 |
Table 3. Performance with different character image resolutions
Visual Embeddings for Unseen Symbols
Figure 4 visualizes the t-SNE embeddings for characters. We extract the character embeddings of the original Latin characters and their homoglyph variants from the PIXEL-KWS model and project them into a 2-D t-SNE space. The resulting embeddings show that visually similar characters are mapped close to each other. Although the Unicode characters are different, PIXEL-KWS places the homoglyphs close to their corresponding Latin characters, with a mean cosine similarity of +0.785, compared with −0.002 between different Latin characters. The close correspondence arises because the representations are derived from the rendered glyphs, allowing visually similar characters to produce similar embeddings despite having different Unicode identities. In contrast, the character baseline has no embedding for these unseen symbols.
Figure 4. t-SNE visualisations of the character embeddings trained from PIXEL-KWS.
Conclusion
We presented PIXEL-KWS, an open-vocabulary keyword spotting framework that represents keyword characters through rendered glyphs rather than a fixed embedding vocabulary. This glyph representation allows the text encoder to process unseen symbols while preserving visual similarities between orthographically related characters. On LibriPhrase, PIXEL-KWS achieves performance comparable to the character-based baseline on clean keywords while providing substantially improved robustness to character-level perturbations. The glyph representation is script-independent by construction; extending PIXEL-KWS to multilingual settings is natural future work. Overall, the results demonstrate that visual character representations are a simple and effective approach for improving the robustness of open-vocabulary KWS to unseen and visually confusable symbols.
References
- [1]Hyeon-Kyeong Shin, Hyewon Han, Doyeon Kim, Soo-Whan Chung, and Hong-Goo Kang, “Learning Audio-Text Agreement for Open-vocabulary Keyword Spotting,” in Proceedings of Interspeech, 2022, pp. 1871–1875.arxiv.org/abs/2206.15400
- [2]Yong-Hyeok Lee and Namhyun Cho, “PhonMatchNet: Phoneme-Guided Zero-Shot Keyword Spotting for User-Defined Keywords,” in Proceedings of Interspeech, 2023, pp. 3964–3968.DOI
- [3]Kumari Nishu, Minsik Cho, Paul Dixon, and Devang Naik, “Flexible keyword spotting based on homogeneous audio-text embedding,” Proceedings of ICASSP, 2023, pp. 5050–5054.arxiv.org/abs/2308.06472
- [4]Kewei Li, Hengshun Zhou, Kai Shen, Yusheng Dai, and Jun Du, “Phoneme-level contrastive learning for user-defined keyword spotting with flexible enrollment,” in Proceedings of ICASSP, 2025, pp. 1–5.
- [5]Elizabeth Salesky, David Etter, and Matt Post, “Robust Open-Vocabulary Translation from Visual Text Representations,” in Proceedings of EMNLP, 2021, pp. 7235–7252.
- [6]Phillip Rust, Jonas F. Lotz, Emanuele Bugliarello, Elizabeth Salesky, Miryam de Lhoneux, and Desmond Elliott, “Language Modelling with Pixels,” in Proceedings of ICLR, 2023.arxiv.org/abs/2207.06991
- [7]Yoshifumi Nakano, Takaaki Saeki, Shinnosuke Takamichi, Katsuhito Sudoh, and Hiroshi Saruwatari, “Vtts: Visual-text to speech,” in IEEE Spoken Language Technology Workshop (SLT), 2022, pp. 936–942.arxiv.org/abs/2203.14725
- [8]Hien Ohnaka, Shinnosuke Takamichi, Keisuke Imoto, Yuki Okamoto, Kazuki Fujii, and Hiroshi Saruwatari, “Visual onoma-to-wave: Environmental sound synthesis from visual onomatopoeias and so,” in Proceedings of ICASSP, 2022, pp. 1–5.arxiv.org/abs/2210.09173
- [9]Adarsh Arigala, Arjun Gangwar, S. Umesh, and Yova Kementchedjhieva, “Pixel-TTS: Image based Text Rendering for Robust Text-to-Speech,” arXiv preprint arXiv:2606.14750, 2026.arxiv.org/abs/2606.14750
- [10]Jonas F. Lotz, Hendra Setiawan, Stephan Peitz, and Yova Kementchedjhieva, “Overcoming Vocabulary Constraints with Pixel-level Fallback,” in Proceedings of COLM, 2025.arxiv.org/abs/2504.02122
- [11]Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, and Ruoming Pang, “Conformer: Convolution-augmented Transformer for Speech Recognition,” in Proceedings of Interspeech, 2020, pp. 5036–5040.arxiv.org/abs/2005.08100
- [12]Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur, “Librispeech: An asr corpus based on public domain audio books,” in Proceedings of ICASSP, 2015, pp. 5206–5210.DOI
- [13]Zhiqi Ai, Zhiyong Chen, and Shugong Xu, “MM-KWS: Multi-modal Prompts for Multilingual User-defined Keyword Spotting,” in Proceedings of Interspeech 2024, 2024, pp. 2415–2419.arxiv.org/abs/2406.07310
- [14]Loren Lugosch, Samuel Myer, and Vikrant Singh Tomar, “Donut: Ctc-based query-by-example keyword spotting,” ArXiv, vol. abs/1811.10736, 2018.arxiv.org/abs/1811.10736
- [15]Sichen Jin, Youngmoon Jung, Seungjin Lee, Jaeyoung Roh, Changwoo Han, and Hoonyoung Cho, “CTC-aligned Audio-Text Embedding for Streaming Open-vocabulary Keyword Spotting,” in Proceedings of Interspeech, 2024, pp. 332–336.
- [16]Yu Nakagome and Michael Hentschel, “WCTC-Biasing: Retraining-free Contextual Biasing ASR with Wildcard CTC-based Keyword Spotting and Inter-layer Biasing,” in Proceedings of Interspeech, 2025, pp. 5178–5182.arxiv.org/abs/2506.01263
- [17]Kumari Nishu, Minsik Cho, and Devang Naik, “Slick: Exploiting subsequences for length-constrained keyword spotting,” in Proceedings of ICASSP, 2025, pp. 1–5.arxiv.org/abs/2409.09067
- [18]Niccolò Sacchi, Alexandre Nanchen, Martin Jaggi, and Milos Cernak, “Open-Vocabulary Keyword Spotting with Audio and Text Embeddings,” in Proceedings of Interspeech, 2019, pp. 3362–3366.DOI
- [19]Yu Xi, Baochen Yang, Hao Li, Jiaqi Guo, and Kai Yu, “Contrastive learning with audio discrimination for customizable keyword spotting in continuous speech,” in Proceedings of ICASSP, 2024, pp. 11666–11670.arxiv.org/abs/2401.06485
- [20]Kewei Li, Yinan Zhong, Xiaotao Liang, Tianchi Dai, and Shaofei Xue, “Synapto: A lightweight, streaming multi-modal framework for keyword spotting with audio-text synergy,” in Proceedings of ICASSP, 2026, pp. 17297–17301.arxiv.org/abs/2512.15124
- [21]Jaemin Jung, Youkyum Kim, Jihwan Park, Youshin Lim, Byeong-Yeol Kim, Youngjoon Jang, and Joon Son Chung, “Metric learning for user-defined keyword spotting,” in Proceedings of ICASSP, 2023, pp. 1–5.arxiv.org/abs/2211.00439
- [22]Youngmoon Jung, Myunghun Jung, Joon-Young Yang, Yong-Hyeok Lee, Jaeyoung Roh, and Hoon-Young Cho, “Mate: Matryoshka audio–text embeddings for open-vocabulary keyword spotting,” in Proceedings of ICASSP, 2026, pp. 19107–19111.



