WorksComputer Science
Distilling Literary Sentiment from LLM Annotations into an Ordinal BERT Model
Abstract
The scarcity of human-annotated literary corpora limits the development of domain-specific sentiment analysis models. This study investigates whether LLM-generated annotations can supervise smaller models for literary sentiment analysis. We fine-tune five BERT models using ordinal regression on 10,889 literary sentences annotated by Gemma 4 31B on a ten-point sentiment scale. The models are evaluated against 1,923 human-annotated sentences from Hemingway's "The Old Man and the Sea", alongside the teacher model and conventional baselines. BERT-large achieves a Kendall's of 0.589, approaching the teacher's 0.604. Although a constant-neutral predictor achieves the lowest overall mean absolute error (MAE), BERT-base outperforms the teacher on non-neutral sentences, achieving an MAE of 0.726 versus 1.116. These findings demonstrate the potential of LLM-generated supervision for developing smaller literary sentiment models, while highlighting the limitations of aggregate error metrics on predominantly neutral datasets.
Introduction
Sentiment analysis (SA) has become an established method in Digital Humanities (DH), particularly for investigating emotional patterns and narrative sentiment arcs in literary texts (Öhman et al., 2024). However, most available sentiment classifiers were trained on non-literary domains, such as product reviews and social media [2], while human-annotated literary datasets remain scarce. Consequently, their application to literary texts presents challenges arising from implicit and context-dependent emotional expressions [5, 10].
Another limitation concerns sentiment representation. Most existing classifiers predict nominal categories, such as negative, neutral, and positive, which are often converted into continuous valence scores for literary analysis [5]. Ordinal regression provides an alternative that explicitly models the ordering of sentiment categories [6]. Meanwhile, LLMs offer opportunities for generating fine-grained sentiment annotations, although their repeated use for inference across large literary corpora can be computationally demanding [1].
This study investigates whether LLM-generated annotations can provide effective supervision for smaller, domain-specific sentiment models. Specifically, we address two research questions:
(RQ1) Can BERT models trained on LLM-generated ordinal annotations generalize to independently human-annotated literary text?
(RQ2) How does predictive performance vary across different BERT configurations and sentiment ranges?
To address these questions, we make three contributions: (1) we construct a corpus of 10,889 literary sentences annotated by a 31B-parameter LLM on a ten-point sentiment scale; (2) we fine-tune five BERT models using a shared ordinal regression objective; and (3) we evaluate their generalization against independent human annotations, comparing their performance with the teacher model, general-domain classifiers, and a constant-neutral baseline. Our results demonstrate that the fine-tuned models approach the teacher’s rank correlation with human judgments while achieving lower prediction errors on non-neutral literary sentences.
Related Work
SA has become an established method in computational literary studies, particularly for examining narrative sentiment arcs and emotional patterns in literary texts (Öhman et al., 2024). Transformer-based models, including BERT and RoBERTa, have been increasingly employed for these tasks. However, their effectiveness remains constrained by the complexity of literary language, where sentiment is frequently implicit, contextual, or conveyed through figurative expressions [5, 10].
Recent advances in LLMs have enabled their use both as sentiment classifiers [7] (Krugmann and Hartmann 2024; Zhang et al. 2024; Miah et al. 2024; Wang et al. 2024) and as annotators for constructing training datasets [12, 1] (Huang and Simpson 2026; Zhang et al. 2025; Xie et al. 2025; Shvets 2025; Pangakis and Wolken 2024). Previous experiments demonstrated that LLMs with approximately 8B parameters or more can achieve rank correlations comparable to or higher than conventional Transformer sentiment classifiers on literary texts (Suslov and Hashimoto, 2026). However, direct LLM inference is substantially more computationally demanding than using smaller encoder-based models.
In DH, custom SA models are being predominantly fine-tuned through nominal classification. A recently developed fiction-oriented RoBERTa model converts continuous sentiment annotations into three nominal classes and reports evaluations across several literary datasets (Pianzola, 2026). Ordinal regression offers an alternative by explicitly preserving the ordering of sentiment categories [6, 4]. Biasetton et al. demonstrated its application to BERT-based sentiment prediction using a five-point scale, although outside the literary domain. Building on these approaches, we investigate whether LLM-generated annotations can provide effective supervision for a ten-level ordinal BERT model trained specifically for literary SA and evaluated against independent human judgments.
Data: LLM-Annotated Literary Sentiment
The training corpus consists of 10,889 sentences extracted from literary works in the Project Gutenberg collection. To ensure diversity across literary works, the extraction procedure randomly sampled one sentence per novel. To reduce the inclusion of paratextual material, such as titles, tables of contents, and section headings, we excluded very short sentences and structural entries (e.g., “Chapter X”, “Section X”, and “Part X”), as well as the opening and closing sentences of each work.
The resulting corpus was annotated using Gemma 4 31B [11], which assigned each sentence a sentiment score on a ten-point scale, ranging from 1 (strongly negative) to 10 (strongly positive), with scores of 4–6 representing the near-neutral range. The annotation prompt, generation parameters, and post-processing procedures are described in Appendix § 9.
For independent evaluation, we employed the human-annotated corpus of Ernest Hemingway’s The Old Man and the Sea, introduced by [5]. This dataset comprises 1,923 sentences evaluated by two human annotators on a comparable ten-point valence scale. The evaluation corpus was excluded from model training.
Method: Ordinal BERT
We fine-tuned five BERT configurations using an ordinal regression architecture inspired by [6] (McCullagh 1980; ); Wang et al. 2026). Unlike conventional sentiment classifiers that treat sentiment categories as independent labels, ordinal regression explicitly preserves their inherent ordering. This approach is particularly suitable for literary SA, where valence is expressed on a graded scale rather than through discrete polarities.
Given an input sentence , BERT produces a contextual representation, which is projected onto a scalar latent sentiment score . For sentiment levels, the model learns nine monotonically ordered thresholds , enforced through softplus parameterization. Each threshold defines the cumulative probability:
where denotes the sigmoid function. Training labels are converted into cumulative binary targets and optimized using binary cross-entropy loss. The ordered thresholds ensure that predictions remain rank-consistent.
At inference, the expected sentiment score is calculated as:
This produces a continuous sentiment estimate on the original 1–10 scale while preserving the ordinal structure of the training labels, without relying on post-hoc transformations of nominal sentiment probabilities.
Experimental Setup
We fine-tuned five pretrained BERT models of different sizes: Large, Base, Medium, Small, and Mini, described in Appendix § 10. The LLM-annotated corpus was divided into training (), validation (), and test () sets. All models used identical optimization hyperparameters, detailed in Appendix § 11.
Baselines. We compared the models against Gemma 4 31B (teacher), two general-domain RoBERTa classifiers (twt_roberta and twt_roberta_xlm) [2, 3], and a constant-neutral baseline (score = 5). RoBERTa model scores were linearly scaled to 1-10 range. BERT-tiny also converged to constant predictions and is reported alongside this baseline.
Evaluation. Models were evaluated on 1,923 sentences from Hemingway’s The Old Man and the Sea, using the average scores of two human annotators. This corpus was excluded from training and validation. We calculated MAE, Kendall’s , and precision and accuracy within one sentiment level (±1). MAE and used original scores; tolerance-based precision was macro-averaged across observed predicted values.
To account for the predominance of neutral scores, we additionally evaluated 305 non-neutral sentences, defined by human scores below 4 or above 6.
Results
Non-Literary Domain SA
Table 1 presents the performance of the fine-tuned BERT models, the teacher LLM, general-domain RoBERTa classifiers, and the constant-neutral baseline against human sentiment annotations 1.
| Model | All sentences () MAE | Kendall’s | Prec. (±1) | Acc. (±1) | Non-neutral sentences () MAE | Prec. (±1) | Acc. (±1) |
| Gemma 4 31B (teacher) | 0.822 | 0.604 | 0.460 | 0.741 | 1.116 | 0.522 | 0.600 |
| twt_roberta | 0.972 | 0.497 | 0.176 | 0.695 | 1.656 | 0.306 | 0.223 |
| twt_roberta_xlm | 1.261 | 0.505 | 0.125 | 0.530 | 1.613 | 0.301 | 0.308 |
| BERT-large | 0.769 | 0.589 | 0.614 | 0.787 | 0.887 | 0.571 | 0.711 |
| BERT-base | 0.733 | 0.580 | 0.723 | 0.815 | 0.726 | 0.698 | 0.803 |
| BERT-medium | 0.752 | 0.489 | 0.659 | 0.808 | 0.998 | 0.669 | 0.675 |
| BERT-small | 0.717 | 0.468 | 0.708 | 0.837 | 0.982 | 0.668 | 0.698 |
| BERT-mini | 0.766 | 0.402 | 0.732 | 0.829 | 0.969 | 0.692 | 0.708 |
| BERT-tiny (constant-neutral) | 0.597 | – | 0.841 | 0.841 | 1.962 | 0.000 | 0.000 |
Table 1. Performance against human annotations () and on non-neutral sentences (; scores <4 or >6). MAE denotes mean absolute error; precision and accuracy use a tolerance of ±1. Bold indicates the best result per metric. Kendall’s is undefined for constant predictions.
Overall performance. All five fine-tuned BERT models achieved lower MAE than the general-domain RoBERTa baselines. Among the student models, BERT-small obtained the lowest MAE (0.717) and the highest accuracy within one sentiment level (0.837).
However, none of the evaluated models outperformed the constant-neutral baseline, which achieved an MAE of 0.597 and an accuracy of 0.841. This result reflects the predominance of near-neutral sentiment in the evaluation corpus, where 84.1% of human reference scores fell within the interval from 4 to 6.
Performance on non-neutral sentences. To examine predictive performance beyond the dominant neutral range, we separately evaluated 305 sentences with human sentiment scores below 4 or above 6.
All five fine-tuned BERT models outperformed both the teacher LLM and the general-domain RoBERTa classifiers in MAE and accuracy within one sentiment level on this subset.
BERT-base achieved the strongest performance, with an MAE of 0.726 and an accuracy of 0.803, compared with 1.116 and 0.600 for the teacher model, respectively. The constant-neutral baseline achieved an MAE of 1.962 and a zero accuracy on this subset.
These results demonstrate that the performance advantage of the constant-neutral baseline on the complete corpus does not extend to sentences expressing sentiment outside the near-neutral range.
Rank correlation and model size. Rank-based evaluation revealed a different pattern. The teacher achieved the highest Kendall’s (0.604), followed by BERT-large (0.589) and BERT-base (0.580), both exceeding the general-domain RoBERTa baselines (0.497 and 0.505). Smaller BERT configurations showed weaker agreement, with declining from 0.489 for BERT-medium to 0.402 for BERT-mini.
Literary Domain SA
Overall, performance depended on the evaluation metric and sentiment distribution. BERT-small achieved the lowest overall MAE among students, BERT-base performed best on non-neutral sentences, and BERT-large most closely approached the teacher’s rank correlation with human judgments.
Discussion
LLM Supervision and Ordinal Modeling
Trained on LLM-annotated sentences sampled across different literary works, the BERT models were evaluated on Hemingway’s stylistically distinctive The Old Man and the Sea. Despite this difference, BERT-large and BERT-base approached the teacher’s rank correlation with human judgments ( = 0.589 and 0.580 versus 0.604), while BERT-base outperformed the teacher on non-neutral MAE (0.726 versus 1.116). These findings suggest that LLM-generated supervision can transfer meaningful sentiment information from a stylistically diverse training corpus to a distinct literary work.
Nevertheless, evaluation on a single novel limits claims of broader stylistic generalization. Furthermore, the study did not examine alternative architectures, such as conventional regression or three-class nominal classification, or assess training variability across random seeds. Consequently, the specific contribution of ordinal modeling and the stability of the results remain to be established.
Sentiment Distribution and Evaluation
The results highlight a common challenge in SA: imbalanced sentiment distributions can favor models predicting the dominant neutral category (Jongeling et al., 2017). In our evaluation corpus, 84.1% of human annotations fell within the near-neutral range of 4–6. Consequently, a constant-neutral predictor achieved the lowest overall MAE, despite capturing no sentiment variation.
However, identifying non-neutral passages may have greater heuristic value for literary analysis than accurately predicting neutral sentences. On the non-neutral subset, all fine-tuned BERT models outperformed the teacher LLM and general-domain RoBERTa baselines. These findings suggest that aggregate error metrics may obscure models’ ability to estimate non-neutral sentiment, limiting their usefulness for analyzing emotional variation in literary narratives.
One possible improvement involves modifying the training objective to penalize errors on underrepresented sentiment categories more heavily. Class-balanced weighting [8] and focal loss (Lin et al., 2020) offer established approaches, while distribution-aware methods address imbalance in continuous prediction (Yang et al., 2021). Adapting these approaches to ordinal sentiment modeling remains a direction for future research.
Model Size and Computational Efficiency
Model size influenced predictive performance, with larger BERT variants generally achieving stronger rank correlations. BERT-base performed best on non-neutral sentences, while BERT-large most closely approached the teacher’s rank correlation. Thus, model selection should consider whether absolute sentiment estimation or relative ordering is more important.
Transferring LLM-generated annotations to smaller encoder models may also reduce computational requirements for large-scale literary analysis. However, inference efficiency was not measured directly, and these potential benefits require experimental verification.
Conclusion
This study demonstrates that LLM-generated annotations can provide effective supervision for literary sentiment analysis, at least within the examined configuration. Using 10,889 LLM-annotated sentences, we fine-tuned BERT models of different sizes and evaluated them against independent human judgments.
The larger models approached the teacher’s rank correlation (BERT-large: ; Gemma 4 31B: 0.604), while BERT-base outperformed the teacher on non-neutral sentences (MAE: 0.726 versus 1.116). Even compact BERT configurations demonstrated meaningful predictive performance, suggesting that smaller encoders can effectively learn from LLM-generated annotations. Nevertheless, the constant-neutral baseline achieved the lowest overall MAE, highlighting the influence of sentiment imbalance. Further research should examine alternative training objectives, prediction calibration, and generalization across literary genres.
Limitations
The model was trained on randomly sampled sentences from Project Gutenberg, annotated using an LLM. No additional filtering was applied to exclude potentially bias-inducing contexts. Consequently, the resulting dataset may inherit biases from the annotating LLM.
In particular, we observed that Gemma 4 31B tended to assign disproportionately negative sentiment scores to sentences describing sensitive topics, such as violence, death, or criminal acts, even when their narrative context conveyed positive or contrasting sentiment. This suggests that the model may associate certain semantic topics with negative valence regardless of their contextual meaning.
This was the only specific annotation bias identified in the present study; however, other systematic biases cannot be excluded.
Declaration of Generative AI Use
OpenAI’s ChatGPT 5.6 model was used for proofreading and minor stylistic refinements (paraphrasing and summarization under authors’ supervision) and assisting with LaTeX layout adjustment. Also, the same model was used for debugging assistance and code review.
Teacher Model Data Preparation
Annotation Prompt
The sentiment annotations were generated using Gemma 4 31B through Ollama. Each sentence was processed independently using the following prompt, where INPUT_SENTENCE was replaced with the corresponding sentence from the corpus:
You will perform the sentiment analysis of the sentence.
Sentiment is the
emotional tone, attitude, or opinion expressed in text.
Evaluate the sentence you will get as an input in the scale
from 1 to 10 where
1 = most negative and 10 = most positive sentiment.
Also, provide a brief explanation of your decision to score the
sentiment in the sentence.
Output the results as json in the following format:
[ { "Sentiment_score": your score from 1 to 10, "Explanation": your explanation for the score } ]
Here is the sentence: INPUT_SENTENCE
Generation Parameters and Post-processing
Inference was performed locally using Ollama with the model’s default generation parameters in instant mode. No additional modifications to the default inference settings were applied.
To ensure consistent annotation output, the processing pipeline implemented a JSON parsing and validation procedure. Each sentence was allowed up to three generation attempts if the model failed to produce a valid, parsable JSON response.
However, Gemma 4 31B successfully generated valid JSON responses containing sentiment scores on the first attempt for all 10,889 sentences. Consequently, no additional generation attempts were required.
The resulting sentiment scores were extracted from the Sentiment_score field and used as ordinal labels for subsequent BERT fine-tuning.
List of BERT Base Models for Fine-Tuning
Table Table 2 summarizes the six pretrained BERT configurations considered in this study. BERT-Large and BERT-Base follow the original architectures introduced by Devlin et al. [9], while the Medium, Small, Mini, and Tiny variants are compact models introduced by Turc et al. (2020).
| Model | Layers | Hidden | Heads | Params. |
| BERT-Large | 24 | 1024 | 16 | 340M |
| google-bert/bert-large-uncased | ||||
| BERT-Base | 12 | 768 | 12 | 110M |
| google-bert/bert-base-uncased | ||||
| BERT-Medium | 8 | 512 | 8 | 41.7M |
| google/bert_uncased_L-8_H-512_A-8 | ||||
| BERT-Small | 4 | 512 | 8 | 29.1M |
| google/bert_uncased_L-4_H-512_A-8 | ||||
| BERT-Mini | 4 | 256 | 4 | 11.3M |
| google/bert_uncased_L-4_H-256_A-4 | ||||
| BERT-Tiny | 2 | 128 | 2 | 4.4M |
| google/bert_uncased_L-2_H-128_A-2 |
Table 2. Architectural characteristics of the pretrained BERT models. Each Hugging Face model ID appears below its model. Parameter counts are approximate.
All models are English uncased BERT encoders. The configurations differ in the number of Transformer layers, hidden dimensions, attention heads, and trainable parameters. Parameter counts are approximate and refer to the pretrained architectures.
Fine-Tuning Training Parameters
All BERT variants were fine-tuned using the Hugging Face Trainer framework with the same optimization hyperparameters. Training was configured for up to three epochs, with a fixed random seed of 1111. BERT-large and BERT-base used a maximum sequence length of 256 tokens, while the remaining models used 128 tokens.
Model evaluation, checkpoint saving, and logging were scheduled every 100 training steps. The
The best checkpoint for each model was selected according to the lowest validation mean absolute error (MAE). At most six checkpoints were retained, and FP16 mixed precision was enabled when CUDA was available. The training configuration is summarized in Table 3.
| Hyperparameter | Value |
| Learning rate | |
| Weight decay | 0.01 |
| Warmup ratio | 0.10 |
| Maximum epochs | 3 |
| Training batch size | 16 |
| Evaluation batch size | 32 |
| Gradient accumulation | 1 |
| Evaluation interval | 100 steps |
| Checkpoint interval | 100 steps |
| Logging interval | 100 steps |
| Checkpoint selection | Min. val. MAE |
| Maximum checkpoints | 6 |
| Random seed | 1111 |
| Mixed precision | FP16 (CUDA only) |
| Max. length (Large, Base) | 256 |
| Max. length (other models) | 128 |
Table 3. Training hyperparameters shared across the ordinal BERT configurations.
LLM-Annotated Label Distribution
The distribution of LLM-generated sentiment labels across the complete corpus () is presented in Table 4. The annotations were unevenly distributed, with score 5 accounting for 41.5% of all sentences. Overall, 57.9% of annotations fell within the near-neutral range (scores 4–6).
| Score | Count | Score | Count |
| 1 | 239 | 6 | 960 |
| 2 | 800 | 7 | 637 |
| 3 | 1436 | 8 | 923 |
| 4 | 821 | 9 | 474 |
| 5 | 4519 | 10 | 80 |
Table 4. Distribution of LLM-generated sentiment labels across the complete annotated corpus (), before the training, validation, and test split.
Evaluation Metrics and Checkpoint Selection
During training, model performance was evaluated on the LLM-annotated validation set using MAE, root mean squared error (RMSE), accuracy, accuracy within one sentiment level, quadratic weighted Cohen’s kappa (QWK), and Spearman’s rank correlation ().
The checkpoint selection criterion was validation MAE, with lower values preferred. Following training, the models were also evaluated on the held-out test split of 1,089 LLM-annotated sentences. Table 5 reports the selected checkpoint steps, validation MAE, and test-set performance.
| Model | Step | Val. MAE | Test MAE | RMSE |
| BERT-large | 1600 | 0.847 | 0.838 | 1.196 |
| BERT-base | 600 | 0.971 | 0.959 | 1.312 |
| BERT-medium | 1100 | 1.012 | 1.078 | 1.505 |
| BERT-small | 1100 | 1.079 | 1.142 | 1.546 |
| BERT-mini | 1600 | 1.219 | 1.278 | 1.633 |
| BERT-tiny | 1600 | 1.354 | 1.353 | 1.895 |
| Model | Acc. | Acc. ±1 | QWK | |
| BERT-large | 0.461 | 0.814 | 0.746 | 0.763 |
| BERT-base | 0.401 | 0.771 | 0.680 | 0.711 |
| BERT-medium | 0.372 | 0.742 | 0.593 | 0.567 |
| BERT-small | 0.360 | 0.699 | 0.529 | 0.532 |
| BERT-mini | 0.264 | 0.661 | 0.445 | 0.479 |
| BERT-tiny | 0.415 | 0.579 | 0.000 | 0.226 |
Table 5. Validation and test performance on LLM-annotated sentences. Step identifies the selected checkpoint; denotes Spearman’s rank correlation.
The models were subsequently evaluated against the independent human annotated Hemingway corpus, as reported in § 6. These human annotations were not used for checkpoint selection.
References
- [1]Abubakar M. Ashir and Mohammed Abdulghani Taha. 2024. BERT with an augmented cross-attention decoder (BERT-ACD) for binary and fine-grained multi-band sentiment detection. Intelligenza Artificiale: The international journal of the AIxIA, 19(1):3–16.
- [2]Francesco Barbieri, Jose Camacho-Collados, Luis Espinosa Anke, and Leonardo Neves. 2020. TweetEval: Unified benchmark and comparative evaluation for tweet classification. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1644–1650, Online. Association for Computational Linguistics.
- [3]Francesco Barbieri, Luis Espinosa Anke, and Jose Camacho-Collados. 2022. XLM-T: Multilingual language models in twitter for sentiment analysis and beyond. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 258–266. European Language Resources Association.arxiv.org/abs/2104.12250
- [4]Nicolò Biasetton, Riccardo Ricciardi, Luigi Salmaso, and Paola Zuccolotto. 2026. How well do ratings reflect sentiment? Evidence from a large italian review corpus. Applied Stochastic Models in Business and Industry, 42(2):e70090.DOI
- [5]Yuri Bizzoni and Pascale Feldkamp. 2024. Sentiment analysis for literary texts: Hemingway as a case-study. Journal of Data Mining & Digital Humanities, NLP4DH:1–20.DOI
- [6]Wenzhi Cao, Vahid Mirjalili, and Sebastian Raschka. 2020. Rank consistent ordinal regression for neural networks with application to age estimation. Pattern Recognition Letters, 140:325–331.DOI
- [7]Daniel Coutinho, Breno Braga, Theo Canuto, Juliana Alves Pereira, Wesley K. G. Assunção, Igor Steinmacher, Marco Gerosa, and Alessandro Garcia. 2026. Leveraging large language models for sentiment analysis in GitHub pull request discussions. Empirical Software Engineering, 31(5):140.DOI
- [8]Yin Cui, Menglin Jia, Tsung-Yi Lin, Yang Song, and Serge Belongie. 2019. Class-Balanced Loss Based on Effective Number of Samples. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9260–9269, Los Alamitos, CA, USA. IEEE Computer Society.arxiv.org/abs/1901.05555
- [9]Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.arxiv.org/abs/1810.04805
- [10]Pascale Feldkamp, Jan Kostkan, Ea Overgaard, Mia Jacobsen, and Yuri Bizzoni. 2024. Comparing tools for sentiment analysis of Danish literature from hymns to fairy tales: Low-resource language and domain challenges. In Proceedings of the 14th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pages 186–199. Association for Computational Linguistics.
- [11]Gemma Team. 2026. Gemma 4 technical report. Preprint, arXiv:2607.02770.
- [12]Nils Constantin Hellwig, Jakob Fehle, Udo Kruschwitz, and Christian Wolff. 2026. LLM-as-an-annotator: Training lightweight models with LLM-annotated examples for aspect sentiment tuple prediction. In Proceedings of the Fifteenth Language Resources and Evaluation Conference, pages 7955–7972, Palma de Mallorca, Spain. ELRA Language Resource Association.