betaFeedback

WorksComputer Science

Beyond a Single Sign: BiTE for Branch-Preserving Binary Threshold Expansion

Abstract

Binary neural networks (BNNs) reduce model storage and replace many arithmetic operations with efficient bitwise computations, but the single-sign activation limits their representational capacity. Each preactivation direction is compared with one threshold and reduced to one of two values, discarding variation within each sign region. We propose BiTE (Branch-Preserving Binary Threshold Expansion), which evaluates each feature direction against KK ordered thresholds and retains the resulting binary responses as separate inputs to the following learned operator. BiTE can distinguish up to K+1K+1 ordered activation states while allowing the following operator to learn a separate transformation for each threshold response. It also enables a controllable trade-off between the number of independently constructed feature directions and the number of ordered responses extracted from each direction. On BinaryViT, BiTE improves mean final-epoch Top-1 accuracy from 74.60±0.30%74.60\pm0.30\% to 75.83±0.15%75.83\pm0.15\% on CIFAR-10 and from 42.80±0.46%42.80\pm0.46\% to 44.07±0.32%44.07\pm0.32\% on CIFAR-100, with nearly unchanged trainable parameter counts. A matched uniform-collapse control reaches 75.33±0.15%75.33\pm0.15\% on CIFAR-10, supporting the value of retaining the threshold responses separately relative to the evaluated scalar collapse. Binary VGG-16 experiments further show a favorable observed parameter–accuracy trade-off, including a configuration that achieves 0.640.64 percentage points higher mean accuracy with 21.8%21.8\% fewer parameters than conventional width scaling. Overall, these results support branch-preserving threshold expansion as an effective way to improve binary representations and the observed parameter–accuracy trade-off.

Paper details

vlnc:W_e29ruw2p
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
arXiv nonexclusive license
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…