betaFeedback

WorksComputer Science

Where Mixture Recognition Fails: An Ingredient-Level Audit of Unseen Combinations in SmellNet

Abstract

An electronic nose must name every ingredient of a mixture, including combinations it has never measured. SmellNet, a public benchmark for this task, scores ingredients with a hit rate that is told how many each sample contains. We audit what this protocol hides. We hold out ingredient combinations from the SmellNet training data, evaluate four architectures (MLP, CNN, LSTM and a Transformer encoder) without revealing the ingredient count, and repeat the evaluation once on the official unseen split. Under the count-informed protocol, 90-96% of mixtures receive an ingredient that is absent, on our folds and on the official split alike. On our folds, every model identifies one ingredient reliably but the second rarely, and it completes the pair with a familiar training partner about twice as often as chance. On the official split, even the first-ranked ingredient is correct in only 29-46% of binary mixtures, often because a frequent training ingredient that is absent from every test pair is ranked first, and on its combinations that are truly absent from training no model names both ingredients in more than 1.2% of them. The failures fall on the same ingredients for every architecture, and a frequency correction only moves errors between ingredients. Mixture recognition is therefore limited by ingredients the models cannot separate and by a preference for frequent ingredients, not by the architecture, and it should be evaluated with set-level errors and without oracle cardinality.

Paper details

vlnc:W_zgwszdnr
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
arXiv nonexclusive license
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…