betaFeedback

WorksComputer Science

BEACON: Distilled Begin/End-Aware Structured Decoding for Continuous Sign Language Segmentation

Abstract

In segmentation-based continuous sign language recognition, an untrimmed video is first partitioned into lexical units; segmentation is therefore a means to an end—downstream recognition. However, segmenters are usually judged by local boundary detection rather than by the quality of the lexical segments and the recognition they enable. We present BEACON, a begin/end-aware segmenter that combines four-class Outside/Begin/Inside/End (BIOE) labeling, constrained Viterbi decoding over eight legal transitions, and boundary knowledge distilled from a binary teacher into a shared student. Under strict no-cut evaluation with validation-based model selection, BEACON improves segment-F1 by 14.58/14.59 points and boundary-F1 by 2.17/1.22 points, and reduces the downstream word error rate (WER) by 9.10/4.15 points relative to the state-of-the-art in the PHOENIX-14/BSL Corpus. To examine which segmentation metric better reflects downstream recognition, we evaluated five distinct architecture families. Segment-F1 aligns more consistently with lower WER than boundary-F1 in both corpora (Spearman ρ=−1.0/−0.9ρ=-1.0/-0.9 versus −0.9/−0.8-0.9/-0.8). These results support evaluating sign language segmentation in terms of segment quality and downstream recognition utility rather than local boundary accuracy alone.

Introduction

In segmentation-based recognition pipelines, untrimmed sign language videos are first partitioned into lexical units that can be passed to an isolated sign language recognizer. Since signers produce fluid, co-articulated motions without explicit pauses, automatic segmentation remains an important bottleneck for pipelines that require localized lexical units; segmentation errors propagate into downstream recognition and contribute to a higher word error rate (WER) [14, 19]. Segmentation is therefore a means to an end, a perspective we make central both to how we evaluate

segmenters and to how we design them.

Renz et al. [19] formulate segmentation as binary boundary detection: each frame is labeled either “inside a sign” or “at a boundary”. A multi-stage temporal convolutional network (MS-TCN) [1] is trained with binary cross-entropy, followed by simple argmax decoding and an endpoint-cutting post-processing step. The released evaluation procedure uses ground-truth evaluation endpoints for this operation; we revisit it in § 4. This approach established a reproducible protocol on PHOENIX-14 [7] and the BSL Corpus [21] using boundary-F1 and segment-F1. Its binary representation, however, treats each frame only as either sign-internal or boundary, collapsing onset and offset into a single “boundary” class. It therefore does not explicitly represent how frames form a complete sign segment—where the segment begins, continues, and ends.

We make two departures. First, we recast segmentation as structured sequence labeling: every frame is tagged O/B/I/E (Outside/Begin/Inside/End), inspired by BIO chunk-tagging [17]. The scheme is binary-compatible—it maps exactly to Renz et al. [19]’s labels, as verified on both corpora—while explicitly distinguishing sign onset and offset. Figure 1 previews the central representational change from an undifferentiated boundary mask to a structurally valid Begin/Inside/End path. These explicit states define a small finite-state grammar; constrained Viterbi decoding [24] over eight legal BIOE transitions restricts predictions to structurally valid paths and returns a well-formed segment sequence by construction.

Figure 1. BEACON replaces Renz et al. [19]’s binary boundary mask—which collapses sign onset/offset—with a 4-class BIOE labeling (explicit Outside/Begin/Inside/End states) and a constrained Viterbi decoder whose eight legal transitions make every output a well-formed segment sequence by construction. This design improves segment quality while retaining boundary accuracy and lowers downstream recognition WER.

Global path validity does not by itself provide sharp local endpoints. A direct solution would be to pair the BIOE segmenter with a separate binary boundary model, but this would duplicate most of the temporal architecture. BEACON instead combines structured segment modeling and boundary modeling within a single student through knowledge distillation [6]. A trained binary boundary model is frozen as a teacher, and its boundary information is distilled into a lightweight IO branch built on the shared student encoder. The teacher is used only during training, leaving a single 5M-parameter student at inference for structured segment prediction and endpoint refinement. The two components serve complementary purposes: the fixed grammar guarantees structurally valid output sequences, while boundary distillation provides local information for more precise endpoint refinement. Matched ablations examine the contribution of each component.

Second, we ask a question the field has left implicit: Which metric should a segmenter optimize? Boundary-F1 (mF1B) is the de facto headline metric, but if segmentation serves downstream recognition, the more relevant target is the metric most closely aligned with WER. Under a matched frozen recognizer control, we compare five distinct architecture families on both PHOENIX-14 [7] and the BSL Corpus [21]. Among the tested architecture families, segment-F1 (mF1S) orders downstream WER more consistently than mF1B on both corpora. We assess this claim through each metric’s association with the independent downstream WER measure, rather than assume that segment-level metrics are inherently preferable. Accordingly, we evaluate BEACON not only by segmentation metrics, but also by whether its explicit segment structure and boundary-sensitive refinement translate into lower WER.

Concretely, our contributions are threefold:

  1. Single structured segmenter with distilled boundary knowledge. BEACON combines four-class BIOE prediction, constrained decoding over eight legal transitions, and training-only boundary distillation, while requiring only a single 5M-parameter model at inference. Under strict no-cut evaluation, it improves mF1S by 14.58/14.59 points and mF1B by 2.17/1.22 points over Renz et al. [19] on the PHOENIX-14/BSL Corpus.

  1. Recognition under strict no-cut protocol. With validation-based checkpoint and refinement-parameter selection, three seeds, and a frozen matched recognizer, BEACON without ground-truth endpoint cutting reduces WER by 9.10/4.15 points relative to Renz et al. [19] on the PHOENIX-14/BSL Corpus.

  1. Architecture-level analysis of segmentation metrics and downstream recognition. Across five architecture families, mF1S orders downstream WER more consistently than mF1B on both PHOENIX-14 and the BSL Corpus, providing an architecture-level test of which segmentation metric better reflects recognition utility.

Related Work

Sign Language Segmentation

Directly comparable baselines. Renz et al. [19] established the baseline protocol for this task using binary boundary masks, an MS-TCN [1] decoded by argmax, and an endpoint-cutting post-processing step that uses ground-truth endpoints. Performance is evaluated with mF1B/mF1S on PHOENIX-14 [7] and the BSL Corpus [21]. We use the same metrics, datasets, and I3D [3] representations, while adopting validation-based model selection and strict no-cut endpoint protocol (see § 4).

Hands-On [13] is the closest recent baseline. It also casts segmentation as sequence labeling, but uses three-class BIO with argmax decoding and stronger HaMeR [16] features on the BSL Corpus. In contrast, BEACON uses an explicit E (End) class (four-class BIOE), replaces argmax with constrained Viterbi decoding [24] (valid segments by construction), and adds soft B/E auxiliary heads, a boundary-aware temporal smoothness loss, and boundary knowledge distillation into a shared student. Since Hands-On uses HaMeR features, whereas our controlled setting uses the released I3D representations, we treat it as a feature-heterogeneous reference rather than a feature-matched comparison.

Related but not directly comparable. Other segmentation work differs from our setting in dataset, supervision, or evaluation protocol. CMPL [20] uses changepoint-modulated pseudo-labeling with additional unlabeled target data in a source-free domain adaptation setting. Since this differs from the fully supervised setting used by our method and primary baselines, we do not treat its reported results as a head-to-head comparison.

Belissen et al. [2] released Dicta-Sign-LSF-v2 with a sliding-window-F1 baseline rather than the mF1B/mF1S protocol. Moryossef et al. [14] studied linguistically motivated segmentation of German Sign Language using per-document frame-F1/IoU. Farfan et al. [4] applied MS-TCN and diffusion-based temporal segmentation to Peruvian Sign Language under a different dataset and evaluation setting. Sim et al. [22] performed unsupervised changepoint segmentation. MHB [26] incorporated handshape pre-training for boundary detection but evaluated on ASLLRP-S [15] rather than the PHOENIX-14/BSL Corpus.

Since these studies differ in datasets, frame rates, annotations, supervision, or evaluation metrics, we do not treat them as directly comparable baselines. Few prior methods are evaluated under the same mF1B/mF1S protocol.

The closest approaches are positioned in Table 1 by learning setting, sequence modeling, and evaluation.

PropertyRenz et al. [19]Hands-On [13]CMPL [20]BEACON
SettingSupervisedSupervisedSource-free DASupervised
InputI3DHaMeR + 3D anglesI3DI3D
Temporal modelMS-TCNTransformerMS-TCNTCN + Transformer
LabelsBinaryBIOBinaryBIOE + IO
ReadoutargmaxargmaxargmaxConstrained Viterbi
Boundary cueGT endpoint cut—CP-modulated PLDistilled IO
Matched WERNoNoNoYes

Table 1. Positioning of the closest sign-segmentation approaches. DA and PL denote domain adaptation and pseudo-labeling. Matched WER uses a fixed corpus-matched recognizer; § 5 reports our controlled re-evaluation.

Structured Prediction with Transition Graphs

Structured sequence labeling commonly combines explicit label schemes such as BIO [17] and BIOES [9, 18] with structured prediction models such as CRF [8]. Our BIOE scheme adapts BIOES by retaining explicit outside, begin, inside, and end states while omitting a separate singleton S state, yielding a compact four-state grammar with eight legal frame-to-frame transitions.

Because this label grammar is known a priori, we enforce these transitions directly with a non-learned constrained Viterbi decoder [24], rather than learning transition scores from data. This guarantees that the decoded BIOE path satisfies the declared segment grammar by construction. We compare this fixed transition graph with a learned-transition CRF control in § 5.3.

Downstream Impact on Recognition

Prior segmentation methods [2, 13, 19, 26] mostly stop at boundary- or segment-level scores; for example, MHB [26] reports boundary detection on ASLLRP-S but does not report downstream recognition WER under a matched recognizer. To our knowledge, none of these methods reports downstream recognition WER under matched recognizer control or examines which segmentation metric best aligns with it. We close the loop on both corpora with a frozen per-corpus recognizer: a partial continuous sign language recognition pipeline (segmenter followed by an isolated sign recognizer), not an end-to-end CTC [5] system.

Method

BIOE Labeling Scheme

We replace the binary boundary mask (yt∈{0,1}y_t \in\{0,1\}) proposed by Renz et al. [19] with a four-class sequence yt∈{O,B,I,E}y_t \in\{\mathrm{O}, \mathrm{B}, \mathrm{I}, \mathrm{E}\} (Outside/Begin/Inside/End). For multi-frame signs, the first and last frames are labeled B and E, with intervening frames labeled I; single-frame annotations (only 0.31%/0.35% of PHOENIX-14/BSL training frames) are assigned E. Because the eight-transition grammar has no singleton path, their evidence contributes to the highest-scoring valid path; their rarity motivates the compact four-state representation rather than a dedicated S class. The map O↦1\mathrm{O} \mapsto1, {B,I,E}↦0\{\mathrm{B},\mathrm{I},\mathrm{E}\} \mapsto0 recovers the binary labels exactly, so any binary boundary metric remains computable while providing explicit begin, inside, and end states for structured prediction. At inference, we restrict decoding to eight legal frame-to-frame transitions, and a constrained Viterbi decoder [24] (see § 3.5) keeps the highest emission path among them, so every decoded path satisfies the declared BIOE transition grammar by construction. Figure 2 illustrates the legal transition decoding, and Figure 3 summarizes the complete BEACON pipeline.

Constrained Viterbi decoding over the BIOE grammar

Figure 2. Constrained Viterbi decoding [24] over the BIOE grammar. A framewise argmax path may contain illegal transitions; the decoder instead returns the highest emission path restricted to the eight legal BIOE moves, so the decoded sequence satisfies the declared grammar. Emission totals are illustrative.

Figure 3. The BEACON pipeline. A frozen I3D [3] backbone supplies frame features to a shared temporal encoder (Stage 1, standard). BIOE and soft Begin/End predictions are fused and decoded over the eight legal BIOE transitions (inset), producing a preliminary segment path that satisfies the declared grammar (Stage 2, ours). A binary IO branch reads the same shared hidden states HH (no extra encoder) and refines endpoints within a validation-selected radius of at most ±3\pm3 frames (Stage 3, ours). During training only, a separately trained, frozen binary teacher (shown dashed, with its own encoder, and discarded after training) distills boundary evidence into this student IO branch via a KL loss; inference uses a single 5M-parameter model with one encoder. The lower strip visualizes the per-frame BIOE labels collapsing into the final segment spans.

Encoder

Throughout our main experiments, the frame features X∈RT×dfeatX \in\mathbb{R}^{T \times d_{\mathrm{feat}}} (dfeat=1024d_{\mathrm{feat}} = 1024) are extracted using I3D [3]. The features are projected by a linear layer to 256 dimensions, passed through a 5-block residual TCN [10] with dilations [1,2,4,8,16][1, 2, 4, 8, 16] (receptive field 63 frames ≈2.5\approx2.5 s at 25 fps), augmented with sinusoidal positional encoding, and finally Freezing the recognizer isolates the segmentation effect, and the five-architecture-family analysis tests whether mF1S orders WER more consistently than mF1B (see § 5.2). processed by a 4-layer pre-LN Transformer encoder [23] (8 heads, FFN 1024). The resulting hidden states H∈RT×256H \in\mathbb{R}^{T \times256} are shared by the BIOE and soft Begin/End heads and also serve as input to the distilled binary IO branch. The IO branch adds 3 residual temporal convolution blocks with dilations [1,2,4][1, 2, 4], layer normalization, and a binary sign/boundary classifier, but does not duplicate the input projection, 5-block TCN, or Transformer. The student has 5M trainable parameters; the frozen teacher is used only during training and discarded at inference.

Prediction Heads

The structured decoder uses three heads on the shared states HH. The fine BIOE head PBIOE=softmax⁡(WfH+bf)∈RT×4P_{\mathrm{BIOE}} = \operatorname{softmax}(W_f H + b_f) \in\mathbb{R}^{T \times4} is the primary output (class-weighted cross-entropy, label smoothing 0.1). Two soft B/E heads pB,pE=σ(⋅)∈RTp_B, p_E = \sigma(\cdot) \in\mathbb{R}^{T} are each trained by binary cross-entropy (λB=λE=0.4\lambda_B = \lambda_E = 0.4) against Gaussian-shaped targets centered on the ground-truth begin/end frames.

Loss Function

Supervised training combines the 4-class loss, binary IO loss, Begin/End auxiliaries, and temporal regularization:

Lsup=LBIOE+LIO+0.4(LB+LE)+0.05Lsmooth.L_{\mathrm{sup}} = L_{\mathrm{BIOE}} + L_{\mathrm{IO}} + 0.4(L_B + L_E) + 0.05L_{\mathrm{smooth}}.

Here, LIOL_{\mathrm{IO}} is a class-weighted 2-class cross-entropy loss using the binary boundary masks of Renz et al. [19] as targets. Its classes are sign/internal and boundary/outside: ytIO=1y_t^{\mathrm{IO}} = 1 for O frames and 0 for B/I/E frames, with respective class weights [1,4][1, 4]. LsmoothL_{\mathrm{smooth}} is a boundary-aware temporal MSE on BIOE log-probabilities,

∥log⁡PBIOE(t)−log⁡PBIOE(t−1)∥22,\lVert\log P_{\mathrm{BIOE}}(t) - \log P_{\mathrm{BIOE}}(t-1) \rVert_2^2,

masked to zero within ±2\pm2 frames of any ground-truth boundary, penalizing interior jitter while permitting legitimate transitions. For binary IO logits zkz_k define,

qk(Td)=softmax⁡(zk/Td),k∈{teacher,student}.q_k^{(T_d)} = \operatorname{softmax}(z_k/T_d), \qquad k \in\{\text{teacher},\text{student}\}.

The total distillation objective [6] is,

L=Lsup+λKDTd2KL⁡(qteacher(Td)∥qstudent(Td)),L = L_{\mathrm{sup}} + \lambda_{\mathrm{KD}}T_d^2 \operatorname{KL}\left(q_{\mathrm{teacher}}^{(T_d)} \mathbin{\|} q_{\mathrm{student}}^{(T_d)}\right),

where Td=2T_d = 2 and λKD=1\lambda_{\mathrm{KD}} = 1. Padding frames are excluded from all loss terms.

Constrained Viterbi Decoder with B/E Fusion

At inference, the three structured-decoding heads are combined in steps (see Figure 2):

  1. B/E fusion gate (α=0.5\alpha= 0.5, τ=0.4\tau= 0.4):

    For the B class, the fused probability is,

P~B(t)={αpB(t)+(1−α)PBIOE[B](t),pB(t)≥τPBIOE[B](t),otherwise,\widetilde{P}_B(t) = \begin{cases} \alpha p_B(t) + (1-\alpha)P_{\mathrm{BIOE}}[B](t), & p_B(t) \ge\tau\\ P_{\mathrm{BIOE}}[B](t), & \text{otherwise}, \end{cases}

with the same operation applied to the E class using pE(t)p_E(t) and PBIOE[E](t)P_{\mathrm{BIOE}}[E](t). The fused per-frame probabilities are re-normalized and log-transformed into the Viterbi emissions. In the full BEACON model, the distilled IO branch additionally contributes its boundary prediction to the O class emission before decoding, as described in § 3.6.

  1. Constrained Viterbi finds the highest emission path through the BIOE trellis subject to eight legal transitions:

O→{O,B},B→{I,E},\mathrm{O} \to\{\mathrm{O}, \mathrm{B}\}, \qquad\mathrm{B} \to\{\mathrm{I}, \mathrm{E}\},
I→{I,E},E→{O,B}.\mathrm{I} \to\{\mathrm{I}, \mathrm{E}\}, \qquad\mathrm{E} \to\{\mathrm{O}, \mathrm{B}\}.

We use a fixed O class emission bias bbias=−1.0b_{\mathrm{bias}}=-1.0 and transition bias btrans=0.0b_{\mathrm{trans}}=0.0.

  1. Endpoint policy: The primary strict no-cut protocol uses no ground-truth endpoint cleanup. The cutting rule used by Renz et al. [19] is evaluated only in a paired compatibility audit.

The final BIOE path is converted to segments by extracting every maximal BI∗EB I^{*} E run.

Distilled Endpoint Refinement

The constrained decoder produces structured preliminary segments, but their endpoint precision still depends on the boundary evidence available to the BIOE predictions. To improve boundary localization while restricting changes to local endpoint positions, the student uses a binary IO branch on the shared hidden states. For each corpus and seed, a 4.41M-parameter binary teacher is trained on the same training indices and selected by validation mF1B. During student training, this teacher remains frozen and supplies softened binary targets in addition to the ground-truth binary labels.

At inference, the student’s P(boundary)P(\text{boundary}) contributes to the O class emission before constrained decoding using the validation-selected αIO\alpha_{\mathrm{IO}}. A boundary run is a maximal contiguous run of boundary/outside frames in the decoded binary mask. After decoding, each run is shifted as a block toward the nearest sufficiently confident student boundary peak within the displacement radius, clipped to the video bounds, and combined with the other shifted runs; its width is preserved unless clipping or merging occurs. For each corpus and seed, after checkpoint selection we choose αIO∈{0,0.3,0.7}\alpha_{\mathrm{IO}} \in\{0, 0.3, 0.7\}, peak threshold τIO∈{0.3,0.4,0.5}\tau_{\mathrm{IO}} \in\{0.3, 0.4, 0.5\}, and maximum displacement r∈{1,2,3}r \in\{1, 2, 3\} using the held-out validation score. The selected setting is then applied once to the test set; all six selections are reported in Supplementary Tab. 7.

This prediction-only refinement adjusts endpoint locations locally; the strict BIOE validity guarantee applies to the Viterbi path before relocation. A 2-encoder implementation is evaluated only as a computational baseline.

Training Protocol

We train for 50 epochs using AdamW [12] (lr =3×10−4=3 \times10^{-4}, weight decay 0.01), with a cosine learning-rate schedule ending at 10−510^{-5}, batch size 8, and gradient clipping 1. We run three seeds (42, 43, 44) and report mean ±\pm SD. Student checkpoints are selected by the balanced validation score (mF1B+mF1S)/2(\mathrm{mF1B}+\mathrm{mF1S})/2 under a fixed checkpoint-selection configuration: B/E fusion α=0.5\alpha=0.5 and τ=0.4\tau=0.4, O class bias −1.0-1.0, transition bias 0, and IO refinement αIO=0.7\alpha_{\mathrm{IO}}=0.7, τIO=0.4\tau_{\mathrm{IO}}=0.4, and r=3r=3. After checkpoint selection, only the IO refinement parameters in § 3.6 are retuned on validation; test data are reserved for final reporting.

Experimental Setup

Datasets and features. We use the released PHOENIX-14 [7] package (4,556 train/1,115 test videos; German Sign Language) and BSL Corpus [21] package (5,413/702; British Sign Language). Both provide frame-level lexical annotations and the same 1024-D I3D [3] features used by Renz et al. [19], keeping the visual representation fixed so that the controlled comparison reflects changes in the segmentation model rather than a different visual backbone.

Neither package defines a validation split for the segmentation task. We therefore deterministically hold out 10% of the original training videos using split seed 100: 4,100/456 train/validation videos for PHOENIX-14 and 4,872/541 for the BSL Corpus. Every trained model—teachers, students, matched ablations, learned CRFs, and architecture controls—uses exactly these video indices. The train and validation partitions are video-disjoint and together cover the original training set, although recording groups can occur in both partitions (see § 6).

Segmentation metrics. Following the evaluation protocol of Renz et al. [19], mF1B averages boundary F1 at temporal tolerances of 1–4 frames, while mF1S averages segment-IoU F1 over thresholds 0.40–0.75. mF1B measures local boundary placement, whereas mF1S additionally penalizes fragmented, merged, or incorrectly extended segments. All primary predictions are evaluated without endpoint cutting. The post-processing step for endpoint cutting used by Renz et al. [19] reads ground-truth endpoints; we retain a paired cut/no-cut audit in the Supplementary only to quantify its effect, while all primary results use the no-cut protocol.

Baselines and controls. The primary baseline is the checkpoint linked by the official Renz et al. [19] repository, evaluated with the same strict no-cut protocol used throughout our experiments. Because the released files do not reconstruct the paper’s multi-seed aggregate, we report it as one public checkpoint rather than as the exact published mean. Supplementary Sec. A.5 separately reports the published aggregates and the cut protocol compatibility audit.

On the BSL Corpus, we additionally evaluate a three-seed full-resolution reproduction of Hands-On [13] with normalized HaMeR [16] features at the native frame rate of 25 fps, without label cleanup or endpoint cutting. Seed 42 uses the recovered checkpoint; seeds 43 and 44 follow the same recipe with fixed epoch-40 checkpoints. Their predicted segments are passed to the same frozen recognizer used for the other downstream comparisons. Because Hands-On uses HaMeR features while the controlled comparisons use I3D, it is treated as a feature-heterogeneous reference rather than a controlled I3D comparison.

Three additional architecture families use binary inside/boundary supervision: a local-attention Transformer with temporal refinement (2.51M), a BiLSTM-CRF (1.05M), and MS-TCN++ (0.89M) [11]. They share I3D inputs, train/validation indices, seeds, 50 epochs, AdamW training setup, balanced score checkpoint selection, endpoint policy, metrics, and downstream recognizer. Model depth and parameter count vary by architecture; the local Transformer is an ASFormer-inspired control, not an exact reproduction of its published configuration [25].

Downstream probe. Each predicted segment sequence is passed to one frozen recognizer per corpus: the released PHOENIX-14 classifier [19] and a 979-class BSL classifier shared across all BSL experiments. The classifier receives the frames selected by a segmenter, while its weights, vocabulary, and decoding remain unchanged. We report corpus WER = (S+D+I)/N(S + D + I)/N and its substitution, deletion, and insertion components, with fixed denominators NN: 10,987 (PHOENIX-14) and 4,214 (BSL Corpus). Test WER and recognition accuracy are never used for checkpoint or decoder selection.

Statistics. New models report three-seed mean and sample standard deviation. The BIO/BIOE comparison is paired by seed and emphasizes paired differences because three pairs cannot support a high-powered significance test. For the five family metric analysis, the unit is one architecture-family mean; with n=5n = 5, we use exact two-sided permutation pp-values for Spearman ρ\rho and report the small-sample limitation explicitly.

Results

No-Cut Segmentation and Recognition

Under the same strict no-cut endpoint protocol, BEACON improves over the public Renz et al. [19] checkpoint by +2.17/+14.58 mF1B/mF1S on PHOENIX-14 and +1.22/+14.59 on the BSL Corpus (see Table 2). On the BSL Corpus, it also exceeds our three-seed full-resolution reproduction of Hands-On [13] by +6.11/+11.36 mF1B/mF1S; because Hands-On uses HaMeR while the other rows use I3D, this is a feature-heterogeneous reference rather than a controlled I3D comparison.

CorpusSegmentermF1B↑mF1B^{\uparrow}mF1S↑mF1S^{\uparrow}
PHX [7]Renz et al. [19]69.7651.12
BEACON71.93±0.1965.70±0.15
BSL [21]Renz et al. [19]67.2843.83
Hands-On [13]†62.39±0.0647.06±0.41
BEACON68.50±0.6458.42±1.03

Table 2. Strict no-cut segmentation results. BEACON and Hands-On [13]† are three-seed mean ± SD; Renz et al. [19] is the public checkpoint. Hands-On† uses full-resolution HaMeR features; all other rows use I3D.

In downstream recognition, BEACON reduces WER relative to the public Renz et al. [19] checkpoint by 9.10 points (18.3% relative) on PHOENIX-14 and 4.15 points

(5.36% relative) on the BSL Corpus. Under the same frozen recognizer, the Hands-On [13] reproduction yields 77.52 ± 0.41 WER, whereas BEACON achieves 73.33 ± 0.31, a 4.19-point (5.40% relative) reduction in mean WER (see Table 3).

Table 3. Downstream recognition under a matched recognizer. WER and substitution/deletion/insertion rates are corpus-level percentages; lower is better. BEACON and Hands-On† use three-seed mean ± SD for WER and three-seed mean for S/D/I. Hands-On† uses HaMeR features; dashes denote an unavailable PHOENIX-14 model. None uses test labels for model/decoder selection or endpoint cutting.

SegmenterPHX [7] WER↓SDIBSL [21] WER↓SDI
Renz et al. [19]49.7332.035.5512.1577.4857.8512.137.50
Hands-On [13]†————77.52±0.4155.5516.555.42
BEACON40.63±0.2027.068.904.6773.33±0.3155.4312.685.22

Table 3.

Across three seeds, on PHOENIX-14 mF1S/WER spans 65.54–65.83/40.41–40.79, and on the BSL Corpus spans 57.39–59.45/73.02–73.64. On PHOENIX-14, insertion/substitution errors fall by 7.48/4.97 points relative to Renz et al. [19] while deletion rises by 3.35, consistent with fewer fragmented predictions and occasional missed short signs. On the BSL Corpus, insertion/substitution fall by 2.28/2.42 points while deletion rises by 0.55.

Figure 4 gives a frame-aligned view of two neighboring ground-truth signs. Renz et al. [19] divide each sign into two unmatched fragments (IoUs 0.45/0.18 and 0.23/0.45), whereas BEACON produces one matched segment per sign (IoUs 0.83/0.71).

PHOENIX-14 example showing frame-aligned ground-truth and predicted segments

Figure 4. PHOENIX-14 [7] example (no-cut; seed 42). Enlarged B/E frames anchor each clip to the frame-aligned GT and predictions. Numbers report segment IoU with the corresponding GT sign.

Endpoint cutting has little effect on BEACON: relative to no-cut, the paired cut evaluation changes WER by −0.05/−0.16 and mF1S by +0.08/+0.24 on the PHOENIX-14/BSL Corpus. This indicates that the primary results do not depend on ground-truth endpoint cleanup. For the public Renz et al. [19] checkpoint, cutting increases mF1S by +0.54/+1.17, showing that the cut and no-cut protocols are not equivalent for this baseline. Per-seed results and the full paired compatibility audit are provided in the Supplementary.

Architecture Families and Metric Associations

The association analysis (see Table 4) uses one representative point from each of five distinct architecture families (see Supplementary Tab. 15). All newly trained controls share features, split, training budget, selection rule, no-cut evaluation, and frozen recognizer.

Table 4. Architecture families comparison under strict no-cut evaluation (three-seed mean ± SD except Renz et al. [19]’s MS-TCN). Local Transformer is an architecture control, not an exact ASFormer [25] reproduction.

FamilymF1B↑mF1S↑WER↓
MS-TCN69.7651.1249.73
Local Transformer48.29±0.5437.03±0.0750.90±0.92
BiLSTM-CRF67.80±0.1058.08±0.2647.15±0.75
MS-TCN++70.60±0.6760.10±1.7142.54±0.29
BEACON71.93±0.1965.70±0.1540.63±0.20
FamilymF1B↑mF1S↑WER↓
MS-TCN67.2843.8377.48
Local Transformer49.97±1.1636.36±0.8479.41±0.49
BiLSTM-CRF62.59±0.2650.94±0.1576.15±0.13
MS-TCN++67.68±1.3355.78±0.9373.12±0.63
BEACON68.50±0.6458.42±1.0373.33±0.31

Table 4.

Across these architectures, mF1S has a stronger negative Spearman association with WER: ρ=−1.0\rho= -1.0 versus −0.9-0.9 on PHOENIX-14 and −0.9-0.9 versus −0.8-0.8 on BSL. The exact test reaches p<0.05p < 0.05 only for PHOENIX-14 mF1S; hence, the evidence supports only that mF1S orders WER more consistently among the tested families. Leave-one-family-out mF1S results are ρ=−1.0\rho= -1.0 throughout on PHOENIX-14 and [−1.00,−0.80][-1.00, -0.80] on BSL (see Supplementary). On BSL, MS-TCN++ has 0.21 lower mean WER than BEACON despite 2.64 lower mF1S, within either method’s reported seed variation.

Controlled Analyses

Matched BIO/BIOE. To isolate labels and decoding, we hold backbone, optimizer, split, loss, seeds, and 50-epoch budget fixed and cross BIO/BIOE labeling with argmax

or the corresponding grammar-constrained decoder (see Table 5). Under argmax, changing BIO to BIOE changes mF1S by −0.01-0.01 / +0.58+0.58 on PHOENIX-14/BSL; with each representation’s matching grammar, the changes are −0.04-0.04 / +0.92+0.92, and WER changes by −0.53-0.53 / 0.000.00. The isolated effect is modest and dataset-dependent. BIOE nevertheless represents a complete segment directly as a B I * E run. The full BEACON improvement therefore reflects the combined system rather than the label alphabet alone.

Table 5. Matched label/readout comparison, strict no-cut (three-seed mean). Other factors are fixed; each constrained row uses the matching grammar. WER is reported for constrained readout.

CorpusLabels/ReadoutmF1B↑mF1S↑ΔmF1SWER↓
PHX [7]BIO/argmax70.7862.26——
PHX [7]BIO/grammar70.6862.80—43.82
PHX [7]BIOE/argmax70.5662.25——
PHX [7]BIOE/8-edge70.3862.76−0.0443.29
BSL [21]BIO/argmax64.7954.85——
BSL [21]BIO/grammar63.9854.54—73.95
BSL [21]BIOE/argmax66.2755.43——
BSL [21]BIOE/8-edge65.4855.45+0.9273.95

Table 5.

Relative to argmax, matching-grammar decoding changes mF1S by +0.51+0.51 / +0.02+0.02 for BIOE and +0.54+0.54 / −0.31-0.31 for BIO. Its numerical effect is also modest and dataset-dependent; structurally, however, argmax can leave the declared grammar whereas constrained decoding cannot.

Relative to BIO/argmax in Table 5, the full model gains +1.15+1.15 / +3.71+3.71 mF1B and +3.44+3.44 / +3.57+3.57 mF1S on PHOENIX-14/BSL. Table 6 separates endpoint modeling from distillation. The supervised IO branch and refinement add +1.17+1.17 / +2.62+2.62 mF1B and +2.46+2.46 / +2.72+2.72 mF1S; KD then adds +0.38+0.38 / +0.40+0.40 and +0.48+0.48 / +0.25+0.25, respectively. The total gains over the structured base are +1.55+1.55 / +3.02+3.02 mF1B and +2.94+2.94 / +2.97+2.97 mF1S; the table also reports the corresponding WER.

SystemmF1B↑mF1B \uparrowmF1S↑mF1S \uparrowWER↓WER \downarrow
BIOE/8-edge base70.38/65.4862.76/55.4543.29/73.95
IO branch, no KD71.55/68.1065.22/58.1740.72/73.74
Full BEACON71.93/68.5065.70/58.4240.63/73.33

Table 6. Component comparison, strict no-cut (three-seed mean). Each entry reports PHOENIX-14/BSL. The last two rows have the same shared-student architecture and endpoint readout, isolating KD.

Fixed grammar and learned transitions. A same-backbone unrestricted CRF is numerically competitive (see Supplementary Tab. 14). It is close to fixed decoding on PHOENIX-14 and improves the matched non-distilled BIOE control on the BSL Corpus. However, 12.65%/12.63% of its decoded test sequences contain at least one transition outside the fixed grammar, whereas constrained decoding guarantees zero grammar-invalid sequences. Learned transitions are viable and can be numerically competitive; the fixed graph instead provides deterministic validity under the declared BIOE grammar.

Inference cost. Supplementary Table 16 compares a two-encoder implementation with the shared student. At the p50/p90 sequence lengths of both corpora, the student reduces neural-forward latency by 36.5%–36.9% and peak allocation by roughly 29%–33%. The teacher is never loaded at inference. Absolute timings are hardware-dependent, making the relative reductions more informative.

Table 16. Inference efficiency. Latency ranges cover both corpora and sequence lengths.

SystemParamsResident (MiB)Peak (MiB)Latency (ms)
Two encoders8.82M34.6545.25–47.711.41–1.43
Single distilled5.00M19.5930.19–32.650.89–0.90

Table 16. Inference efficiency. Latency ranges cover both corpora and sequence lengths.

Discussion

Segment Quality and Downstream Recognition

BEACON improves boundary and segment metrics under strict no-cut evaluation, with the larger gain at the segment level, and the same predictions lower WER under the frozen recognizers. The paired audit confirms that this pattern does not depend on ground-truth endpoint cleanup.

On PHOENIX-14, fewer insertions are consistent with fewer fragmented or spurious segments, while more deletions indicate occasional missed signs. On BSL, gains span substitutions and insertions. Together with § 5.2, this motivates reporting mF1BmF1B, mF1SmF1S, and downstream WER jointly.

Structure and Boundary Synergy

§ 5.3 shows modest standalone numerical effects for BIOE and constrained decoding. BIOE directly represents completion as a B I* E run; constrained decoding enforces the declared grammar. Learned CRF transitions can be competitive, whereas the fixed graph guarantees validity under that grammar.

Structural validity does not determine exact endpoints. The supervised IO branch and refinement provide most of the gain beyond the structured base; distillation adds a smaller, consistent gain without changing inference. BEACON thereby combines valid structure and precise endpoint localization in one model.

Limitations

Validation and generalization. Neither dataset defines an official segmentation validation split, so we construct a fixed video-disjoint partition from its training pool. It is not recording-group-disjoint; generalization to unseen recording groups remains untested.

Scope of the metric–WER evidence. The five-family analysis uses one frozen recognizer per corpus. This small sample is corpus- and recognizer-specific and cannot establish universal recognition optimality.

Dense supervision and training complexity. BEACON requires frame-level annotations and does not address corpora with only sentence- or gloss-level supervision. Inference uses one student, but training also requires a binary teacher and validation-selected refinement parameters.

Annotation scope. Frame endpoints are operational annotations, not unique linguistic boundaries: co-articulation and transitions may be ambiguous, and the corpora provide no inter-annotator analysis.

Conclusion

We presented BEACON, a single-encoder segmenter that combines four-class BIOE labeling, eight-transition constrained decoding, and distilled boundary knowledge. The design separates two complementary requirements: the fixed grammar restricts predictions to valid BIOE paths, while the shared IO branch supplies local evidence for endpoint refinement. Its binary teacher is used only during training, so inference retains one 5M-parameter student rather than a separate boundary encoder.

Under strict no-cut evaluation with validation-based selection, BEACON improves mF1S by 14.58/14.59 points and reduces downstream WER by 9.10/4.15 points relative to the state-of-the-art on PHOENIX-14/BSL. The matched ablations show that the full gain is a system-level result: the shared endpoint branch provides the largest improvement beyond the structured base, and distillation adds a smaller consistent gain without increasing inference cost. Across five architecture families, mF1S orders WER more consistently than mF1B. Although this evidence is limited to the tested corpora and recognizers, it supports reporting segment quality, boundary accuracy, and downstream WER together rather than selecting models by boundary precision alone.

Beyond sign segmentation, fixed-grammar decoding and training-only boundary distillation may be useful when temporal outputs must satisfy a known structure but local endpoints remain uncertain. Future work should model annotation ambiguity and transition regions, reduce dependence on dense frame-level supervision, broaden matched evaluation across datasets, recognizers, and vocabularies, and train segmenters more directly for downstream utility.

Additional Experimental Details

Canonical No-Cut Results by Seed

Tab. 7 gives every run behind the primary results. For each dataset, the binary teacher and distilled student use the same fixed 90/10 split of the original training set (split seed 100). The teacher and student checkpoints and IO refinement parameters are selected on validation segmentation metrics; the B/E fusion gate is fixed across runs. Seed-43 on PHOENIX-14 selects αIO=0\alpha_{\mathrm{IO}} = 0, yet still retains the IO/KD training losses. This suggests that distillation can regularize the shared encoder even when validation does not select IO fusion at inference.

Corpus/SeedEpochα\alpharrτ\taumF1B↑mF1S↑
PHX/4212.72.571.8065.54
PHX/436.01.371.8565.83
PHX/4410.73.372.1565.74
mean±SD————71.93±.1965.70±.15
BSL/4210.73.567.9257.39
BSL/436.71.369.1858.42
BSL/443.73.568.4159.45
mean±SD————68.50±.6458.42±1.03

Table 7. Distilled BEACON per-seed segmentation results and validation-selected IO refinement settings (αIO\alpha_{\mathrm{IO}}, rr, τIO\tau_{\mathrm{IO}}), all under strict no-cut evaluation.

Table 8. Hands-On† reproduction by seed using full-resolution HaMeR features under strict no-cut evaluation. Seed-42 uses the recovered checkpoint; seeds-43 and 44 use fixed epoch-40 checkpoints from the same recipe.

SeedmF1B↑mF1S↑
4262.4547.44
4362.3847.11
4462.3346.63
mean±SD62.39±.0647.06±.41

Table 8.

Per-Run Downstream Recognition

Tab. 9 separates the downstream error decomposition from the segmentation and decoder selection results above. Hands-On† uses seeds 42–44 at full 25 fps resolution with checkpoint-matched normalized HaMeR features, raw BIO argmax, no label cleanup, and no endpoint cutting. Seed-42 is the recovered checkpoint; seeds 43 and 44 follow the same recipe and use fixed epoch-40 checkpoints. Their segments index the same I3D stream consumed by the frozen recognizer. All BSL rows use N=4,214N = 4{,}214 and all PHX rows use N=10,987N = 10{,}987.

CorpusSegmenter/RunWERSDI
PHXRenz et al.49.7332.035.5512.15
BEACON/4240.7927.218.874.71
BEACON/4340.4126.6010.133.68
BEACON/4440.6927.377.715.62
BEACON mean±SD40.63±.2027.068.904.67
BSLRenz et al.77.4857.8512.137.50
Hands-On†/4277.9855.9315.956.10
Hands-On†/4377.4155.6716.944.79
Hands-On†/4477.1755.0516.755.36
Hands-On† mean±SD77.52±.4155.5516.555.42
BEACON/4273.3355.0813.504.75
BEACON/4373.6455.6511.966.03
BEACON/4473.0255.5512.584.89
BEACON mean±SD73.33±.3155.4312.685.22

Table 9. Per-run downstream decomposition under the same frozen recognizer per corpus. Rates are corpus-level percentages. Renz et al. is one public checkpoint; Hands-On† and BEACON report all three seeds, with mean ± SD for WER and mean S/D/I.

Selection and Test-Use Audit

Tab. 10 makes the selection boundary explicit. No test segmentation metric or downstream WER selects a checkpoint or decoder setting.

ItemChosen byTest Set Role
Teacher checkpointvalidation mF1Bfinal reporting only
Student checkpointvalidation balanced scorefinal reporting only
IO refinement (α\alpha, rr, τ\tau)validation gridfinal reporting only
B/E fusion and decoder biasesfixed across primary runsfinal reporting only
Architecture controlsvalidation balanced scorefinal reporting only
Renz et al. checkpointreleased artifactevaluation only
Hands-On checkpointsseed 42 recovered; seeds 43/44 fixed epoch 40evaluation only

Table 10. Training, selection, and test-use separation for the reported systems.

Distillation Component Diagnostic

The non-distilled and distilled students use the same 5M-parameter architecture and strict no-cut endpoint policy. Across three matched seeds, distillation changes mean mF1B/mF1S/WER from 71.55/65.22/40.72 to 71.93/65.70/40.63 on PHOENIX-14 and from 68.10/58.17/73.74 to 68.50/58.42/73.33 on the BSL Corpus. Tab. 11 reports every paired run.

CorpusTrainingSeedmF1B↑mF1S↑WER↓
PHXNo KD4271.7465.2640.57
No KD4371.2565.0540.68
No KD4471.6565.3440.92
KD4271.8065.5440.79
KD4371.8565.8340.41
KD4472.1565.7440.69
BSLNo KD4267.9757.8873.80
No KD4368.3558.3774.42
No KD4467.9958.2672.99
KD4267.9257.3973.33
KD4369.1858.4273.64
KD4468.4159.4573.02

Table 11. Matched distillation ablation under strict no-cut evaluation. The non-distilled and distilled rows use the same shared-student architecture and readout.

Endpoint-Cut Compatibility Check

The endpoint-cutting post-processing step in Renz et al. reads the ground-truth start/end extent of each evaluation video. It is therefore inappropriate for deployable headline inference, but paired evaluation quantifies its effect (see Table 12). The official paper reports multi-run cut aggregates of 71.50/52.78 on PHOENIX-14 and 68.68/47.71 on the BSL Corpus, but only one linked checkpoint per corpus was available to us. We therefore do not present our paired values as exact no-cut conversions of the published means.

CorpusSegmenter/ProtocolmF1B↑mF1S↑WER↓
PHXRenz et al./no-cut69.7651.1249.73
PHXRenz et al./cut70.2851.66—
PHXBEACON/no-cut71.9365.7040.63
PHXBEACON/cut72.0865.7840.58
BSLRenz et al./no-cut67.2843.8377.48
BSLRenz et al./cut68.3245.00—
BSLBEACON/no-cut68.5058.4273.33
BSLBEACON/cut68.8758.6673.17

Table 12.

Table 12. Paired no-cut/cut results from identical predictions. BEACON is a three-seed mean; Renz et al. is one checkpoint.

Fair Labeling and Decoder Factorial

The factorial comparison fixes the 1024-D input, 5-block TCN plus 4-layer Transformer, optimizer, losses, 50 epochs, split, paired seeds, selection rule, and no-cut evaluator. BIO and BIOE differ by 257 output-head parameters (0.0058%); no IO head, teacher, distillation, Begin/End auxiliary loss, or relocation is used. Each constrained row uses the grammar matching its label representation: the BIO grammar disallows an I start or O→I, while BIOE uses the declared eight-edge grammar. Paired BIOE-minus-BIO differences under these matching grammars are −0.30 ± 0.39/−0.04±0.62/−0.53±1.16 (mF1B/mF1S/WER) on PHOENIX-14 and +1.50±0.46/+0.92±0.77/0.00±0.88 on the BSL Corpus (see Table 13). With three paired seeds, exact sign-flip tests have little resolution; we therefore emphasize effect estimates and direction consistency rather than significance.

CorpusLabelsReadoutmF1B↑mF1S↑WER↓
PHXBIOargmax70.78±.1462.26±.19—
PHXBIOgrammar70.68±.1662.80±.3743.82±.15
PHXBIOEargmax70.56±.3462.25±.41—
PHXBIOE8-edge70.38±.5162.76±.2743.29±1.01
BSLBIOargmax64.79±.5854.85±.58—
BSLBIOgrammar63.98±.5554.54±.7573.95±.25
BSLBIOEargmax66.27±.2455.43±.29—
BSLBIOE8-edge65.48±.0955.45±.0373.95±.67

Table 13.

Table 13. Fair ablation, three-seed mean ± SD. WER is reported for grammar-constrained readout only.

Learned-Transition CRF

The learned transition control uses the same BIOE encoder and training split but an unrestricted linear-chain CRF. We count a decoded sequence as invalid when it contains any start, end, or adjacent-state transition outside the fixed eight-edge grammar. The CRF is competitive, but learned scores do not enforce the known grammar. Constrained decoding always returns a path in that grammar and adds no learned transition parameters.

Table 14. Learned transition CRF, strict no-cut, three-seed mean ± SD. Invalid is the mean percentage of test sequences containing at least one transition outside the fixed grammar.

CorpusmF1B↑mF1S↑WER↓Invalid (%)
PHX70.13±1.2962.78±2.0943.77±2.7512.65
BSL66.40±1.0656.53±1.4573.43±1.1512.63

Table 14.

Architecture Families Metric–WER Analysis

Tab. 15 retains sample-SD variation for three-seed models and reports no artificial variance for the public Renz et al. checkpoint. Exact permutation tests are used because n=5n = 5. The Spearman (p,ρ)(p,\rho) pairs for mF1B/mF1S are (−.90, .083)/(−1.00, .017) on PHOENIX-14 and (−.80, .133)/(−.90, .083) on the BSL Corpus. Leave-one-family-out ranges are [−1.00, −.80]/[−1.00, −1.00] and [−.80, −.60]/[−1.00, −.80], respectively. These sensitivity results reinforce the ordering observation but not a universal claim.

Table 15. Architecture families comparison (three-seed mean ± SD except the single public Renz et al. checkpoint). Local Transformer is an architecture control, not an exact ASFormer reproduction. Exact permutation pp-values accompany Spearman ρ\rho.

FamilymF1B↑mF1S↑WER↓
PHX
PHX / MS-TCN69.7651.1249.73
PHX / Local Transformer48.29±.5437.03±.0750.90±.92
PHX / BiLSTM-CRF67.80±.1058.08±.2647.15±.75
PHX / MS-TCN++70.60±.6760.10±1.7142.54±.29
PHX / BEACON71.93±.1965.70±.1540.63±.20
PHX / ρ\rho with WER (pp)−.90 (.083)−1.00 (.017)—
BSL
BSL / MS-TCN67.2843.8377.48
BSL / Local Transformer49.97±1.1636.36±.8479.41±.49
BSL / BiLSTM-CRF62.59±1.2650.94±.1576.15±.13
BSL / MS-TCN++67.68±1.3355.78±.9373.12±.63
BSL / BEACON68.50±.6458.42±.0373.33±.31
BSL / ρ\rho with WER (pp)−.80 (.133)−.90 (.083)—

Table 15.

Inference Efficiency

We measured the neural forward pass on an NVIDIA RTX 5090, FP32, batch size 1, at the p50/p90 test lengths of each corpus, with 20 warm-up and 100 CUDA-synchronized repetitions. CPU Viterbi and endpoint relocation are excluded from both systems. The shared student reduces parameters by 43.3%, resident allocation by 43.5%, peak allocation by approximately 29%–33%, and forward latency by 36.5%–36.9%.

Reproducibility Notes

All 12 fair ablation runs, 18 architecture control runs, 6 teachers, 6 non-distilled students, 6 distilled students, and 6 learned CRF runs completed without NaN, traceback, or out-of-memory failure. The fixed split has zero video overlap and covers the original training set exactly. Teacher and student use identical indices; teachers are frozen and excluded from the inference checkpoint. Corpus WER denominators are constant across segmenters (N=10,987N = 10{,}987 PHOENIX-14; N=4,214N = 4{,}214 BSL Corpus). The split is not recording group-disjoint because most available recording groups occur on both sides of the original training pool; this is disclosed as a limitation rather than treated as a cross-group validation result. Code and evaluation scripts are included in the Supplementary archive and will be released publicly upon acceptance.

Data and Ethical Scope

The experiments use previously released sign language research corpora and derived features under their existing access conditions. We do not infer signer identity or demographic attributes. Automatic lexical segmentation can support annotation and retrieval, but errors may disproportionately affect signing styles underrepresented in the training corpora; the present two-corpus evaluation does not quantify that disparity. Predicted boundaries and recognition outputs should therefore not be treated as authoritative linguistic annotation without human review.

References

  1. [1]Yazan Abu Farha and Juergen Gall. MS-TCN: Multi-stage temporal convolutional network for action segmentation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3575–3584, 2019.
  2. [2]Valentin Belissen, Annelies Braffort, and Michèle Gouiffès. Dicta-Sign-LSF-v2: Remake of a continuous French Sign Language dialogue corpus and a first baseline for automatic sign language processing. In Proceedings of the Twelfth Language Resources and Evaluation Conference (LREC), pages 6040–6048. European Language Resources Association, 2020.DOI
  3. [3]Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? A new model and the Kinetics dataset. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6299–6308, 2017.arxiv.org/abs/1705.07750
  4. [4]Sunny Farfan, Juan J. Choquehuanca-Zevallos, Ana Aguilera, Irvin Dongo, and Yudith Cardinale. Temporal video segmentation approach for peruvian sign language. Sensors, 25(17):5217, 2025.DOI
  5. [5]Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks. In International Conference on Machine Learning (ICML), 2005.
  6. [6]Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network, 2015.arxiv.org/abs/1503.02531
  7. [7]Oskar Koller, Jens Forster, and Hermann Ney. Continuous sign language recognition: Towards large vocabulary statistical recognition systems handling multiple signers. Computer Vision and Image Understanding, 141:108–125, 2015.DOI
  8. [8]John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In International Conference on Machine Learning (ICML), pages 282–289, 2001.
  9. [9]Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. Neural architectures for named entity recognition. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 260–270, 2016.arxiv.org/abs/1603.01360
  10. [10]Colin Lea, Michael D. Flynn, Rene Vidal, Austin Reiter, and Gregory D. Hager. Temporal convolutional networks for action segmentation and detection. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1003–1012, 2017.arxiv.org/abs/1611.05267
  11. [11]Shijie Li, Yazan Abu Farha, Yun Liu, Ming-Ming Cheng, and Juergen Gall. MS-TCN++: Multi-stage temporal convolutional network for action segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(6):6647–6658, 2023.
  12. [12]Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations (ICLR), 2019.
  13. [13]Jian He Low, Harry Walsh, Ozge Mercanoglu Sincan, and Richard Bowden. Hands-On: Segmenting individual signs from continuous sequences. In IEEE International Conference on Automatic Face and Gesture Recognition (FG), pages 1–5, 2025.DOI
  14. [14]Amit Moryossef, Zifan Jiang, Mathias Müller, Sarah Ebling, and Yoav Goldberg. Linguistically motivated sign language segmentation. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 12703–12724. Association for Computational Linguistics, 2023.arxiv.org/abs/2310.13960
  15. [15]Carol Neidle and Dimitris Metaxas. American Sign Language Linguistic Research Project (ASLLRP) Sign Bank: ASLLRP Continuous Signing Corpora — version 3, 2025. Videos provided by DawnSignPress (www.dawnsign.com).
  16. [16]Georgios Pavlakos, Dandan Shan, Ilija Radosavovic, Angjoo Kanazawa, David Fouhey, and Jitendra Malik. Reconstructing hands in 3D with transformers. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.arxiv.org/abs/2312.05251
  17. [17]Lance A. Ramshaw and Mitchell P. Marcus. Text chunking using transformation-based learning. In Third Workshop on Very Large Corpora, pages 82–94, 1995.arxiv.org/abs/cmp-lg/9505040
  18. [18]Lev Ratinov and Dan Roth. Design challenges and misconceptions in named entity recognition. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL), pages 147–155, 2009.DOI
  19. [19]Katrin Renz, Nicolaj C. Stache, Samuel Albanie, and Gül Varol. Sign language segmentation with temporal convolutional networks. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2135–2139, 2021.arxiv.org/abs/2011.12986
  20. [20]Katrin Renz, Nicolaj C. Stache, Neil Fox, Gül Varol, and Samuel Albanie. Sign segmentation with changepoint-modulated pseudo-labelling. In IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 3403–3412, 2021.arxiv.org/abs/2104.13817
  21. [21]Adam Schembri, Jordan Fenlon, Ramas Rentelis, Sally Reynolds, and Kearsy Cormier. Building the British Sign Language corpus. Language Documentation & Conservation, 7:136–154, 2013.
  22. [22]HoHyun Sim, Hyeonjoong Cho, and Hankyu Lee. Changepoint detection-assisted nonparametric clustering for unsupervised temporal sign segmentation. Engineering Applications of Artificial Intelligence, 127:107323, 2024.
  23. [23]Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), pages 5998–6008, 2017.
  24. [24]Andrew J. Viterbi. Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Transactions on Information Theory, 13(2):260–269, 1967.
  25. [25]Fangqiu Yi, Hongyu Wen, and Tingting Jiang. ASFormer: Transformer for action segmentation. In British Machine Vision Conference (BMVC), 2021.arxiv.org/abs/2110.08568
  26. [26]Mingyu Zhao, Zhanfu Yang, Yang Zhou, Zhaoyang Xia, Can Jin, Xiaoxiao He, Carol Neidle, and Dimitris N. Metaxas. MHB: Multi-modal handshape-aware boundary detection for continuous sign language recognition. arXiv preprint arXiv:2511.19907, 2025.arxiv.org/abs/2511.19907

Paper details

vlnc:W_mc39qgs5
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 10

downloads · 1

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents