WorksComputer Science
Interpretable Text Embeddings Based on Hierarchical Questions
Abstract
QA-based text embeddings represent texts as vectors consisting of 1/0 answers to yes/no questions. Although these embeddings provide detailed information for interpretation and achieve high task performance, they often require humans to examine a large number of questions. Furthermore, the relationship between the number of questions and the appropriate level of question granularity has not been sufficiently studied. To address these challenges, we propose a method for constructing hierarchical questions for text embeddings. As the hierarchy deepens, questions become more specific and their number increases. During interpretation, we reduce the number of questions that humans need to examine by replacing questions at deeper levels with aligned questions at shallower levels. The question hierarchy is constructed through recursive clustering, contrastive prompting, and question alignment. Experimental results on general-purpose tasks show that our method improves interpretability while maintaining task performance. It is also found that, when only a limited number of questions are available, questions at shallower levels tend to achieve higher task performance than those at deeper levels, even with the same number of questions.
Introduction
Text embeddings, which represent sentences, paragraphs, and documents as vectors, are fundamental to a wide range of natural language processing (NLP) applications, such as information retrieval [40] and enhancing the generative capabilities of large language models (LLMs) [24]. For this reason, methods for obtaining high-quality text embeddings have been extensively studied [35, 17, 30]. However, these embeddings remain difficult to interpret, making it challenging for humans to
understand which semantic aspects contribute to the similarity or dissimilarity between two texts. This lack of interpretability is particularly critical in high-stakes domains, such as legal and medical applications, where transparent NLP systems are highly desirable [29].
To develop interpretable text embeddings, various methods have been proposed [45, 7, 36]. In particular, QA-based text embeddings [10, 37, 38] shown in Figure 1 (a) are highly interpretable because each dimension of an embedding corresponds to a yes/no question and represents its binary (1/0) answer. While conventional QA-based text embeddings represent texts using answers to relatively detailed questions, they tend to require a large number of questions for human interpretation. Furthermore, it remains unclear how many questions are needed and how specific they should be for general-purpose tasks.

Figure 1. Overview of QA-based text embeddings and interpreting each embedding.
This study organizes questions used for text embeddings into a hierarchical structure, where more specific questions are allocated at the lower levels. This approach has two advantages. First, it can flexibly change the number of dimensions of embeddings, i.e., the number of questions. It is preferable, since the optimal number of dimensions may differ across downstream tasks for which the text embeddings are applied. Second, it can alleviate the burden on humans to interpret text embeddings. Instead of reading all questions, a human can refer to a small number of general questions, as illustrated in Figure 1 (b), where the specific questions , , and are summarized by the general question .
To generate questions of varying granularity, as shown in Figure 2, we recursively apply the question-generation framework proposed by Sun et al. [37], which combines clustering of large-scale text corpora with contrastive prompting. Contrastive prompting generates highly discriminative questions, and recursive clustering is expected to result in questions at shallower levels summarizing questions at deeper levels. The alignment between a question at a given level and a question at the preceding level is determined based on their entailment relationship.

Figure 2. Overview of question generation.
Experiments on three tasks, semantic textual similarity (STS) [5], text clustering [1], and information retrieval [40] show that our method substantially improves the Pareto front between embedding performance and cognitive load. Our analysis also shows that questions become more specific at deeper levels of the hierarchy, and that aligned questions have an entailment relation with high probability. We further find that, when only a limited number of questions are available, questions at shallower levels produce better text embeddings than questions randomly selected at deeper levels.
QA-based Text Embeddings
QA-based text embeddings [10, 37, 38] represent a text through its responses to a set of natural-language questions. Each dimension of the embedding corresponds to a yes/no question, with a value of 1 or 0 indicating a “yes” or “no” response, respectively. For instance, the text “A study shows that exercise is good for health.” may be represented as a vector of for the questions “Is it about health?”, “Is it about politics?”, and “Does it describe re-
search?”. Thus, QA-based embeddings allow texts to be compared in semantic space while preserving the interpretability of individual dimensions.
Let be a set of yes/no questions. The QA-based embedding of an input text is expressed as follows:
where denotes the binary answer to question for . The model predicts the probability that the answer to question for sentence is “yes”, and is determined as follows,
where is a threshold. The model is trained using a collection of sample texts and LLM-generated answers to those samples.
Since questions directly affect the quality of text embeddings, the method to create questions is a critical element in QA-based text embedding. Benara et al. [10] proposed QA-Emb, a method uses LLMs to generate yes/no questions with task-specific prompts containing examples. Sun et al. [37] further proposed CQG-MBQA, a framework for generating more discriminative questions for QA-based text embeddings. It first groups texts into clusters based on the semantic similarity and then uses LLMs to generate yes/no questions that distinguish between the clusters. Since this framework can generate interpretable text embeddings comparable to those produced by black-box text embedding methods, we build our question generation method based on this approach.
Algorithm 1 Question Generation
1: # Step1. Recursive Clustering
2: $\mathcal{S} \leftarrow \{s_1, s_2, \ldots\}$
3: $\mathcal{C}^{1} \leftarrow \operatorname{kmeans}(\mathcal{S}, k_{\mathrm{first}})$
4: for $l \in \{2, 3, \ldots, l_{\max}\}$ do
5: $\mathcal{C}^{l} \leftarrow \bigcup_{\mathcal{C}_{i}^{l-1}\in\mathcal{C}^{l-1}} \operatorname{kmeans}(\mathcal{C}_{i}^{l-1}, k)$
6: end for
7: # Step2. Contrastive Question Generation
8: for $l \in \{1, 2, \ldots, l_{\max}\}$ do
9: $\widetilde{\mathcal{R}}^{l} \leftarrow \bigcup_{\mathcal{C}_{i}^{l}\in\mathcal{C}^{l}} \{\widetilde{\mathcal{R}}_{i}^{l} \mid \widetilde{\mathcal{R}}_{i}^{l} \mathrel{\mathop{\mathrm{def}}=} \operatorname{cqg}(\mathcal{C}_{i}^{l}, \mathcal{C}^{l})\}$
10: $\widetilde{\mathcal{Q}}^{l} \leftarrow \bigcup_{\widetilde{\mathcal{R}}_{i}^{l}\in\widetilde{\mathcal{R}}^{l}} \widetilde{\mathcal{R}}_{i}^{l}$
11: end for
12: # Step3. Question Filtering
13: for $l \in \{1, 2, \ldots, l_{\max}\}$ do
14: $\mathcal{R}^{l} \leftarrow \operatorname{filtering}(\widetilde{\mathcal{R}}^{l})$
15: $\mathcal{Q}^{l} \leftarrow \bigcup_{\mathcal{R}_{i}^{l}\in\mathcal{R}^{l}} \mathcal{R}_{i}^{l}$
16: end for
Proposed Method
Our proposed method consists of three processes: question generation, question alignment, and model training. In the question generation process, we combine recursive clustering with the question-generation method proposed by Sun et al. [37] to generate highly discriminative questions at various levels of granularity (§ 3.1). In question alignment, each question at a given hierarchical level is matched with a question at the superior level with which it has an entailment relationship (§ 3.2). In the model training process, for each generated question, we train a model that predicts the probability to answer “yes” for a given sentence (§ 3.3).
Question Generation
Question generation process consists of three steps: recursive clustering, contrastive question generation, and question filtering. A simplified overview is shown in Algorithm 1, and each step is explained in detail below.
Step1. Recursive Clustering We first use the MEDI2 dataset1 (Muennighoff et al. (2025)), which contains approximately 6.8 million texts, as a large-scale corpus . The recursive -means clustering is applied to , where each text is encoded using an existing black-box text embedding model. At the first level, is grouped into clusters . At subsequent levels, texts in each cluster from the previous level are grouped into clusters, and is obtained by merging them. Thus, the number of clusters increases by a factor of at each level, i.e., . The parameter is set larger than to make a relatively large number of clusters at the top level.
Step2. Contrastive Question Generation For each cluster at the level , questions are generated by the contrastive question generation (CQG) framework proposed by Sun et al. [37]. First, positive and negative samples are collected. The positive samples are texts randomly sampled from . The negative samples include hard negatives and easy negatives. For the hard negatives, texts are randomly sampled from the clusters in that are closest to . For the easy negatives, texts are randomly sampled from the remaining clusters. The sampled positive and negative texts are then included in a prompt that instructs the LLM to generate 10 questions that yield “yes” for the positive samples and “no” for the negative samples. The prompt is provided in Appendix § 8. As a result of this step, 10 times as many questions are generated as there are clusters. In Algorithm 1, is a collection of question sets, where each question set is generated from a single cluster, while is a set of all questions at level .
Step3. Filtering Questions Many of the generated questions are semantically similar, resulting in redundancy that increases the computational cost of constructing QA-based text embeddings. Therefore, we apply a filtering process to the generated questions following a probing mechanism proposed by Sun et al. [37]. First, we calculate the quality for each question at the level . As in CQG, we first randomly sample positive probes (samples) from the cluster where is generated, hard negative probes from clusters close to , and easy negative probes from other clusters. After that, the answers to are generated by a LLM for each probe, and the quality of is calculated as follows:
Afterward, all questions are encoded using a black-box text embedding method, and questions whose cosine similarity exceeds are removed. In this filtering process, it is ensured that higher-quality Algorithm 2 Question Alignment
: matrix
for do
for do
for do
index of in
index of in
end for
end for
end for
questions are prioritized and retained. The detailed algorithm for this filtering process is described in Appendix § 9.
Question Alignment
We align questions at level to the most related question at the superior level, called the “parent question” hereafter. The alignment table is defined as an index of the parent question of the -th question in , and computed by Algorithm 2. Let denote the precomputed alignment table that records an index of the parent cluster. For each question cluster at level , an index of the parent cluster () is obtained using . Next, the parent question is chosen from the questions in . Specifically, and are converted into declarative sentences and , respectively, and a natural language inference (NLI) model predicts the probability that entails . This score represents the validity of replacing with when a human interprets text embeddings. Then, the question with the highest entailment probability is selected, and its index in is recorded in .
Model Training
The QA model described in § 2 is trained based on the multi-task binary question answering (MBQA) framework proposed by Sun et al. [37]. For each question at the level , the training data for is created as the following procedure. First, 50 texts from the cluster where question is generated, 30 texts from the clusters closest to within , and 20 texts from all other clusters are randomly sampled. Next, the answers to a question for these texts are predicted by LLMs. After that, the pairs of the black-box embeddings from the sampled texts and their corresponding 1/0 answers are used as the training data to train a regression model. Training details are provided in Appendix § 10.
The QA-based text embedding for a given input text is obtained by feeding a black-box embedding into the regression models for all questions.
| Level | 1 | 2 | 3 | 4 | Baseline |
| # Clusters | 40 | 200 | 1,000 | 4,996 | 5,000 |
| # Questions | 213 | 861 | 3,462 | 13,914 | 16,419 |
caption: Number of clusters and questions.
:::
Experiments
This section demonstrates the effectiveness of our method by comparing it with the baseline. We first describe the experimental settings, such as the baseline and hyperparameters (§ 4.1). After that, we explain the metrics for measuring the quality and interpretability of QA-based text embeddings (§ 4.2). Finally, we evaluate the quality and interpretability of our method and the baseline on general-purpose tasks (§ 4.3).
Experimental Setup
We use CQG-MBQA (Sun et al., [37]) as the baseline. CQG-MBQA uses a single clustering step instead of the recursive clustering used in our method. For the recursive clustering described in § 3.1, we set the parameters and to 40 and 5, respectively. We use UAE-large-v1{2} for text embedding, gpt-4o-mini{3} for question generation, and mDeBERTa-base-v3{4} as the NLI model for question alignment. Table 1 shows the number of clusters and questions generated by our method and the baseline. The number of clusters at the fourth level is slightly smaller than 5,000 because some clusters at the third level contain fewer than five samples and therefore cannot be further divided into five clusters. The number of generated questions increases in proportion to the number of clusters. The distribution of the number of texts in each cluster is provided in Appendix § 11.
Table 2 shows the hyperparameters for question generation in our method and the baseline. For direct comparison, we use the same hyperparameter settings for the fourth level as for the baseline. As the hierarchical level becomes shallower, we increase by 2 and and by 6 each. This is because clusters at shallow levels contain more samples and therefore cover broader regions of the semantic space, but using too few contrastive samples for these clusters may cause the generated questions to focus on only a limited region of the semantic space.
| Level | ||||||||
| 1 | 12 | 36 | 36 | 3 | 5 | 3 | 2 | 0.8 |
| 2 | 10 | 30 | 30 | 3 | 5 | 3 | 2 | 0.8 |
| 3 | 8 | 24 | 24 | 3 | 5 | 3 | 2 | 0.8 |
| 4 | 6 | 18 | 18 | 3 | 5 | 3 | 2 | 0.8 |
| Baseline | 6 | 18 | 18 | 3 | 5 | 3 | 2 | 0.8 |
Table 2. Hyperparameters for question generation.
Our method and the baseline are evaluated on the three tasks, STS ([5]), text clustering ([1]), and information retrieval ([40]), using the datasets included in the massive text embedding benchmark (MTEB) ([32]). Details of the datasets are provided in Appendix § 12.
Metrics
Embedding Quality Based on the basic settings in MTEB ([32]), the STS task uses Spearman’s rank correlation coefficient, the text clustering task uses the V-Measure, and the information retrieval task uses nDCG@10 as evaluation metrics for the quality of text embeddings.
Interpretability As a metric for measuring interpretability, we define cognitive load as the number of questions a human must ask during interpretation. Specifically, it is defined for each of the three tasks as follows.
STS: Let be the indices of questions answered “yes” in both sentence vectors and . The cognitive load is defined as the size of .
Clustering: Let be the indices of questions answered “yes” in all text embeddings in a cluster , then .
Retrieval: Let be the indices of questions answered “yes” in both the query vector and the top-1 passage , then .
Here, the cognitive load for STS is the same as that defined by Sun et al. [37], while the cognitive loads for clustering and retrieval are newly added in this study.
Furthermore, we assume that question alignment reduces cognitive load by replacing the question with its aligned abstract question at the higher level, thereby reducing the number of questions a human must check. We define the cognitive load of embeddings with question alignment as
where and is the question level.
Result and Discussion
Performance vs. Cognitive Load Figure 3 shows the performance and cognitive load on the STS, text clustering, and information retrieval tasks when the threshold is 0.3. Several common trends can be observed across the three tasks. First, the performance and cognitive load of the fourth level are nearly identical to those of the baseline. This suggests that questions generated through recursive clustering provide similar embedding quality and cognitive load to those generated using a single clustering step. Second, there is a trade-off between task performance and cognitive load. In other words, obtaining higher-quality text embeddings requires more questions, increasing cognitive load. A user can choose the level of embeddings, considering a balance between the performance of the downstream task and the cognitive load of interpreting the model’s prediction. In this experiment, since cognitive load increases significantly from the third level to the fourth level while performance improves only slightly, the third level appears to offer the best balance between task performance and cognitive load. Finally, question alignment can reduce cognitive load while maintaining task performance. As a result, the Pareto front of task performance and cognitive load is significantly improved, outperforming the baseline. Results for other values of the threshold are given in Appendix § 14, indicating a similar trade-off between performance and cognitive load across all values.5
Figure 3. Task performance vs. cognitive load ( is 0.3). Q-Align indicates the results after question alignment, shown as white-filled shapes.
Impact of Thresholds on Performance and Cognitive Load
Figure 4 and Figure 5 show the task performance and cognitive load for the threshold , respectively. As shown in Figure 4, performance peaks around a threshold of 0.3 for all three tasks and gradually decreases thereafter. Furthermore, the difference in performance between the third and fourth level is not particularly significant, indicating that high-quality text embeddings can be generated for general-purpose tasks even with the number and granularity of questions at the third level. As shown in Figure 5, cognitive load decreases significantly as the threshold increases. Furthermore, the effect of reduction of cognitive load by question alignment is greater when the level is deep or the threshold is low. However, if the threshold is too low, the discrepancy between the model’s and human’s standards, such as a human interpreting a question the model answered “yes” to as “no”, may become greater, potentially reducing the reliability of the generated text embeddings.
Figure 4. Task performance for the different threshold .
Figure 5. Cognitive load for the different threshold . Q-Align indicates the results after question alignment, shown as white-filled shapes.
Analysis
This section analyzes our method from various perspectives: entailment relationships between aligned questions (§ 5.1), the appropriate question granularity (§ 5.2), and a case study (§ 5.3).
Entailment Relationships between Aligned Questions
To reduce cognitive load through question alignment, the target question needs to semantically summarize the source questions. To verify this, the entailment relationship between the aligned questions is estimated. Specifically, for the -th source question at the level and the -th target question at the level (where ), we convert them into declarative sentences and . After that, the probability that entails , and the probability that entails is estimated by an NLI model4. If is high (i.e., entails ) and is largely lower than (i.e., is more abstract than ), can be considered suitable for replacing . When the difference between and is 2 or greater, is determined by recursively consulting the alignment table . For example, if is 3 and is 1, is .
Figure 6 shows the proportion of combinations of questions across different levels for which the entailment probability exceeded 50%. For many level pairs, more than 67% of lower-level questions entail upper-level questions. On the other hand, the proportion of upper-level questions that entail lower-level questions is small. These results indicate that the granularity of the questions becomes finer as the level deepens, while aligned

Figure 6. Proportion of cases where the probability of “entailment” exceeded 50%.
questions preserve semantic coherence. Therefore, our method generates hierarchical questions in which upper-level questions summarize lower-level questions. The distribution of the probability of entailment is provided in Appendix § 13.
Impact of Granularity of Questions
We investigate how question granularity affects task performance and cognitive load. Figure 3 shows that embeddings encoded by the questions at the fourth level achieve the highest performance, but this may stem from using the largest number of questions. To evaluate the appropriateness of the questions themselves, questions from deeper levels are randomly chosen so that the number of sampled questions matches that of the upper levels, and the resulting text embeddings are evaluated using the same metrics as in § 4. Figure 7 shows the mean Spearman’s rank correlation coefficient and cognitive load for the STS task across five different seeds when the threshold is 0.3 and 0.7. When is 0.3, there is little difference in performance across question granularities. In contrast, when is 0.7, task performance at upper levels tends to be higher than that at lower levels with the same number of questions. This trend is particularly pronounced when the number of questions is small. When the threshold is high and “yes” answers are less likely, fine-grained questions are answered “no” in most cases, making the resulting text embeddings less informative, although cognitive cost is reduced. In such cases, abstract questions are more appropriate for QA-based text embeddings. Results for other values of and other tasks are provided in Appendix § 16.

Figure 7. Task performance vs. cognitive load when the number of questions is matched across levels by random sampling.
Case Study
As a case study, we present an example of interpreting text embeddings using our method. The two input texts are sampled from the MS MARCO dataset [8] for the retrieval task, and the threshold is set to 0.5. Questions at the fourth level for which the model answers “yes” for both texts are replaced with their aligned questions at the third level. The results shown in Table 3 illustrate the interpretability of our method. For questions related to concepts shared by two texts (e.g., education and medical care), the models answer “yes” for both. In contrast, for questions related to concepts that differ between the two texts (e.g., specific numerical thresholds and unsafe conditions), the models answer “yes” only for the corresponding text. Furthermore, the aligned questions summarize the questions for which the model answers “yes” for both texts. This makes it easier for humans to interpret the text embeddings.
| Text 1: | what level is blood pressure too low | ||
| Text 2: | As long as you are not feeling dizzy or passing out, and have a decent quality of life, it is totally okay to have that level of blood pressure. There are a lot of normal people walking around with blood pressures of 90/60 on no medication and it is just normal for them. The only time doctors treat low blood pressure is when there are any symptoms, such as dizziness, light-headedness, or the experience of passing out. Any of these could result in a potentially disastrous fall, so if you experience one of these symptoms then you do want to tell your doctor right away. | ||
| Source Question | A1 | A2 | Target Question |
| Is the article likely to be used in academic settings or coursework? | Yes | Yes | Is the article concerned with enhancing understanding or skills? |
| Is there a clear intent to explain a concept or process? | Yes | Yes | |
| Is the information presented in a clinical or educational context? | Yes | Yes | Does the article discuss physical or anatomical concepts? |
| Is the information provided related to health or medical conditions? | Yes | Yes | |
| Is there mention of commonly accepted blood pressure thresholds? | No | Yes | |
| Is there an absence of alarming or unsafe conditions in the description? | Yes | No |
Table 3. Case study of interpreting text embeddings using question alignment. A1 and A2 denote the answers of the model for Text 1 and Text 2, respectively.
Related Work
General Text Embeddings Text embeddings have been actively studied in the field of NLP [15, 35, 20]. Many recent text embedding models use Transformer encoders [16, 26, 41] trained using a contrastive learning framework [13, 17, 44]. More recently, methods that use the hidden states of LLMs as text embeddings have also been proposed. These include methods that make the attention in LLMs bidirectional and apply contrastive learning [9, 21] (Muennighoff et al. 2025), as well as methods that insert input text to specific prompt templates without updating the parameters of the LLMs [19, 23, 25].
Interpretable Text Embeddings Various methods have been proposed for interpretable text embeddings [18, 39, 7]. Many of these methods assign specific roles to each dimension of the text embeddings [29, 34, 36]. Opitz and Frank [33] mapped each dimension of the text embeddings to an abstract meaning representation metrics. Wang et al. [45] mapped each dimension of the text embeddings to an anchor text and defined its value as the similarity between the corresponding anchor text and the input sentence. However, because these methods produce dense vectors, their interpretations may vary across individuals. In contrast, QA-based text embeddings [10, 37, 38] are sparse, making such ambiguity less likely.
Conclusion
In this paper, we proposed an interpretable text embedding framework with hierarchical questions to improve the interpretability of QA-based text embeddings. Experiments across multiple tasks showed that our method improved the Pareto front derived from the trade-off between task performance and cognitive load. Our analysis further showed that the aligned questions frequently exhibited entailment relationships, and the questions in shallower levels can be more effective when only a limited number of questions are available.
Limitations
Although our method reduces cognitive load during interpretation, manually inspecting all generated questions remains difficult due to their large number. Therefore, our qualitative analysis is limited to sampled questions, and a more comprehensive human evaluation is needed to assess the quality and interpretability of the generated questions.
Another limitation is that our method focuses on improving the Pareto front derived from the trade-off between task performance and cognitive load rather than directly improving task performance. Future work could study how the question hierarchy can be used to improve embedding quality while keeping the embeddings interpretable.
In addition, we do not evaluate our method on multiple languages. The quality and granularity of generated questions may differ across languages, so it is still unclear whether our method works well in other languages. Evaluating our method in multilingual settings is therefore an important direction for future work.
Prompts
The prompt templates used in our experiments are shown in Figure 8–Figure 10. We use gpt-4o-mini3 for all LLM-based processing described in this section.

Evaluate the following text chunk based on
the yes/no questions provided.
**Text Chunk:**
{text chunk}
**Questions:**
{questions}
**Instruction for the model:** Please read
the provided text chunk and answer each of
the questions with either “yes” or “no”.
Format the responses as follows:
1. yes/no
2. yes/noFigure 8. Prompt template for QA-based text embedding.

Generate 10 simple yet insightful yes/no
questions that determine properties of an
article, where for all questions, the answer
will be “yes” for ALL the positive articles
and “no” for ALL the negative articles. Keep
questions concise and avoid using complex
sentence structures with “and” or “or” unless
necessary.
**Positive Articles:**
{positive articles}
**Negative Articles:**
{negative articles}
**Instruction:** Based on the excerpts
provided, generate 10 simple yet insightful
yes/no questions that can accurately
differentiate the positive articles from the
negative articles. Each question should be
concise and framed in such a way that it will
elicit a “yes” response for ALL positive
articles and a “no” response for ALL negative
articles. Avoid using complex sentence
structures with “and” or “or” unless
absolutely necessary. Format the questions
in a numbered list as shown below:
1. [First simple yes/no question]
2. [Second simple yes/no question]Figure 10. Prompt template for contrastive question generation.
Prompt: QA-based Text Embedding
Figure 8 shows the prompt template used to obtain QA-based text embeddings. The prompt takes two inputs: an input text and a set of yes/no questions. The LLM is instructed to answer each question with either “yes” or “no” in the same order as the questions. The resulting answers are used as labels for the corresponding text–question pairs.
Prompt: Question-to-Declarative Conversion
Figure 9 shows the prompt template used for question-to-declarative conversion. The prompt takes a yes/no question as input. The LLM is instructed to convert the question into a corresponding declarative sentence and to output only the converted sentence.

Convert the following question into a
declarative sentence.
**Question:**
{question}
**Instruction:** Please output only the
declarative sentence derived from the given
interrogative sentence.Figure 9. Prompt template for question-to-declarative conversion.
Prompt: Contrastive Question Generation
Figure 10 shows the prompt template used for contrastive question generation. The prompt takes two sets of articles as input: positive articles and negative articles. The LLM is instructed to generate 10 concise yes/no questions that yield “yes” for all positive articles and “no” for all negative articles. To encourage simple question structures, we additionally instruct the LLM to avoid using “and” or “or” unless necessary.
Details of Filtering
Algorithm 3 shows the detailed procedure for question filtering. It consists of two steps, and and are collections of sets of questions after filtering in Step 1 and Step 2, respectively. It is basically the same as the method proposed by Sun et al. [37]. However, in this study, the flag at line 14 is set to false to avoid selecting the top questions, because it may eliminate promising candidates.
Details of Training QA Models
The structure of the QA model is the same as that of Sun et al. [37]. The regression model is a multilayer perceptron that passes a 1024-dimensional vector through a (1024, 8) linear layer, a ReLU activation function, and an (8, 1) linear layer, in that order. The output logit is passed through a sigmoid function to obtain a probability of a “yes” answer. For training hyperparameters, we set the batch size to 20, the number of epochs to 10, and the learning rate to 1e-3. The model is trained using NVIDIA DGX Spark (128GB CPU).
Analysis of Clusters
The distribution of the number of texts in each cluster is shown in Figure 11.

Figure 11. Distribution of the number of texts in each cluster.
Evaluation Data
The evaluation data for each task is as follows:
STS Seven datasets: SemEval STS tasks 2012-2016 ([5, 6, 3, 2, 4]), STS Benchmark ([12]), and SICK-Relatedness ([28]).
Algorithm 3 Question Filtering
Step1. Filtering for each cluster
for do
for do
if then
if then
end if
end if
end for
if then
end if
end for 19.
Step2. Filtering across clusters
for do
for do
for do
if then
if then
end if
end if
end for
end for
end for
return
Clustering Seven datasets: TwentyNewsgroupsClustering6, StackExchangeClusteringP2P7, BiorxivClusteringP2P8, BiorxivClusteringS2S9, MedrxivClusteringP2P10, Medrxiv-ClusteringS2S11, and RedditClusteringP2P12
Retrieval Six datasets: MS MARCO ([8]), ArguAna ([42]), FiQA-2018 ([27]), NFCorpus ([11]), SCIDOCS ([14]), and SciFact ([43]).
Analysis of Questions
Length of Questions
The distribution of the number of characters in questions is shown in Figure 12. Even across different levels, the length of questions does not vary much.

Figure 12. Distribution of the number of characters in questions.
Concreteness of Questions
We calculate the concreteness of questions using wordtangible13. In wordtangible, a predefined dictionary, where the keys are words and the values are their concreteness scores, is used to compute the average concreteness of words as the concreteness of the sentence. In the original wordtangible setting, words not included in the dictionary are ignored. However, we believe that words not in the dictionary, such as proper nouns, should be considered concrete, so we assign them the maximum concreteness score (5 points). The distribution of the concreteness of questions is shown in Figure 13. As the level becomes deeper, the average concreteness tends to increase.

Figure 13. Distribution of the concreteness of questions.
Probability of Entailment
Figure 14 shows the distribution of entailment probabilities when the deeper-level sentences are the premise and the shallower-level sentences are the hypothesis, while Figure 15 shows the distribution of entailment probabilities when the premise and hypothesis are swapped.

Figure 14. Distribution of probabilities of entailment (deeper-level shallower-level).

Figure 15. Distribution of probabilities of entailment (shallower-level deeper-level).
Full Results of Performance vs. Cognitive Load
Figure 16–Figure 18 show the full results of the performance and cognitive load on the STS, text clustering, and information retrieval tasks, respectively.

(a) (a) is 0.1

(b) (b) is 0.2

(c) (c) is 0.3

(d) (d) is 0.4

(e) (e) is 0.5

(f) (f) is 0.6

(g) (g) is 0.7

(h) (h) is 0.8

(i) (i) is 0.9
Figure 16. Task performance vs. cognitive load on the STS task.

(a) is 0.1

(b) is 0.2

(c) is 0.3

(d) is 0.4

(e) is 0.5

(f) is 0.6

(g) is 0.7

(h) is 0.8

(i) is 0.9
Figure 18. Task performance vs. cognitive load on the information retrieval task.
Task Performance by Dataset
Table 4–Table 6 present the performance on individual datasets for the STS, text clustering, and information retrieval tasks. CQG-MBQA (Repro.) denotes
| Type | Model | STS12 | STS13 | STS14 | STS15 | STS16 | STS-B | SICK-R | Avg. |
| Black-box | BERT | 38.78 | 57.98 | 57.98 | 63.15 | 61.06 | 46.35 | 58.40 | 54.81 |
| Black-box | GloVe | 54.64 | 69.16 | 60.81 | 72.31 | 65.34 | 61.54 | 55.43 | 62.74 |
| Black-box | USE | 64.49 | 67.80 | 64.61 | 76.83 | 73.18 | 74.92 | 76.69 | 71.22 |
| Black-box | SimCSE (Unsup.) | 66.05 | 81.49 | 73.61 | 79.72 | 78.12 | 76.52 | 72.24 | 75.39 |
| Black-box | SBERT (Ori.) | 74.53 | 77.00 | 73.18 | 81.85 | 76.82 | 79.10 | 74.29 | 76.68 |
| Black-box | SimCSE (Sup.) | 75.30 | 84.67 | 80.19 | 85.40 | 80.82 | 84.25 | 68.38 | 79.86 |
| Black-box | WhitenedCSE | 74.65 | 85.79 | 77.49 | 84.71 | 80.33 | 81.48 | 75.34 | 79.97 |
| Black-box | SBERT (New) | 73.08 | 82.13 | 76.73 | 85.58 | 80.23 | 83.09 | 79.32 | 80.02 |
| Black-box | OpenAI | 72.84 | 86.1 | 81.15 | 88.49 | 85.08 | 83.56 | 79.00 | 82.31 |
| Black-box | AnglE | 79.09 | 89.62 | 85.02 | 89.51 | 86.61 | 89.06 | 82.62 | 85.93 |
| Interpretable | Bag-of-Tokens | 44.75 | 52.06 | 54.78 | 68.65 | 60.59 | 54.85 | 57.87 | 56.22 |
| Interpretable | QAEmb-MBQA | 59.40 | 63.19 | 57.68 | 69.29 | 63.18 | 71.33 | 72.33 | 65.20 |
| Interpretable | CQG-MBQA (Ori.) | 69.21 | 80.19 | 73.91 | 80.66 | 78.30 | 82.69 | 78.21 | 77.60 |
| Interpretable | CQG-MBQA (Repro.) | 74.95 | 79.96 | 75.66 | 82.47 | 80.47 | 84.26 | 77.16 | 79.28 |
| Interpretable | Ours (Level 4) | 75.58 | 80.23 | 75.63 | 82.75 | 80.27 | 83.73 | 76.74 | 79.27 |
| Interpretable | Ours (Level 3) | 74.13 | 81.32 | 75.46 | 82.02 | 79.26 | 83.11 | 76.07 | 78.77 |
| Interpretable | Ours (Level 2) | 67.28 | 75.52 | 70.83 | 77.50 | 74.77 | 79.60 | 74.47 | 74.28 |
| Interpretable | Ours (Level 1) | 58.28 | 69.82 | 59.68 | 72.43 | 67.36 | 72.05 | 68.96 | 66.94 |
Table 4. Task performance by dataset on the STS task.
| Type | Model | MS MARCO | ArguAna | FQA | NFC | SCIDOCS | SciFact | Avg. |
| Black-box | BERT | 16.86 | 28.29 | 2.19 | 4.30 | 2.82 | 13.34 | 11.30 |
| Black-box | SimCSE (Unsup.) | 44.63 | 38.34 | 9.84 | 9.88 | 5.50 | 25.72 | 22.32 |
| Black-box | GloVe | 44.27 | 36.30 | 10.09 | 13.87 | 8.04 | 29.58 | 23.69 |
| Black-box | SimCSE (Sup.) | 47.86 | 38.33 | 10.41 | 12.42 | 7.53 | 29.59 | 24.36 |
| Black-box | SBERT (New) | 88.74 | 47.13 | 37.27 | 32.25 | 21.82 | 62.64 | 48.31 |
| Black-box | AnglE | 90.43 | 66.15 | 44.84 | 38.65 | 22.98 | 74.07 | 56.19 |
| Black-box | OpenAI | 92.18 | 58.05 | 55.00 | 42.07 | 23.11 | 77.77 | 58.03 |
| Interpretable | Bag-of-Tokens | 29.79 | 34.25 | 3.99 | 21.51 | 6.79 | 47.36 | 23.95 |
| Interpretable | BM25 | 68.42 | 49.28 | 25.14 | 32.08 | 15.78 | 68.70 | 43.23 |
| Interpretable | QAEmb-MBQA | 40.51 | 34.75 | 8.23 | 3.87 | 3.74 | 12.01 | 17.18 |
| Interpretable | CQG-MBQA (Ori.) | 62.21 | 47.75 | 18.63 | 9.74 | 8.67 | 32.80 | 29.97 |
| Interpretable | CQG-MBQA (Repro.) | 67.79 | 55.57 | 16.64 | 7.54 | 12.87 | 25.16 | 30.93 |
| Interpretable | Ours (Level 4) | 69.66 | 55.04 | 18.63 | 8.36 | 12.65 | 25.40 | 31.62 |
| Interpretable | Ours (Level 3) | 61.80 | 49.00 | 15.12 | 7.07 | 10.18 | 20.21 | 27.29 |
| Interpretable | Ours (Level 2) | 38.29 | 35.64 | 5.62 | 5.47 | 5.69 | 9.54 | 16.71 |
| Interpretable | Ours (Level 1) | 12.16 | 14.54 | 2.29 | 3.11 | 2.12 | 4.11 | 6.39 |
Table 6. Task performance by dataset on the information retrieval task.
our reimplementation of CQG-MBQA, which corresponds to the baseline described in the main text. The results for all other methods, except CQG-MBQA (Repro.) and the proposed method, are taken from Sun et al. [37]. For CQG-MBQA (Ori.), CQG-MBQA (Repro.), and the proposed method, the threshold is set to 0.5.
Full Results of Analysis of Appropriate Granularity of Questions
Figure 19–Figure 21 show the full results of the performance and cognitive load on the STS, text clustering, and information retrieval tasks when the number of questions is matched across levels by random sampling, respectively.
Figure 19. Task performance vs. cognitive load on the STS task when the number of questions is matched across levels by random sampling.

(a) is 0.1

(b) is 0.2

(c) is 0.3

(d) is 0.4

(e) is 0.5

(f) is 0.6
(g) is 0.7
(h) is 0.8
(i) is 0.9
Figure 21. Task performance vs. cognitive load on the information retrieval task when the number of questions is matched across levels by random sampling.

(a) (a) is 0.1

(b) (b) is 0.2

(c) (c) is 0.3

(d) (d) is 0.4

(e) (e) is 0.5

(f) (f) is 0.6

(g) (g) is 0.7

(h) (h) is 0.8

(i) (i) is 0.9
Figure 17. Task performance vs. cognitive load on the text clustering task.
| Type | Model | TNG | SE-P2P | BR-P2 | BR-S2S | MR-P2P | MR-S2S | RD-P2P | Avg. |
| Black-box | SimCSE (Unsup.) | 23.21 | 28.50 | 24.90 | 19.55 | 23.60 | 21.97 | 45.14 | 26.70 |
| Black-box | GloVe | 25.83 | 28.51 | 29.27 | 19.18 | 26.12 | 20.38 | 35.82 | 26.44 |
| Black-box | BERT | 23.35 | 26.55 | 30.12 | 24.77 | 26.09 | 23.60 | 43.32 | 28.26 |
| Black-box | SimCSE (Sup.) | 34.86 | 29.45 | 30.15 | 24.67 | 26.25 | 24.12 | 47.74 | 31.03 |
| Black-box | SBERT (New) | 47.47 | 33.13 | 36.99 | 33.21 | 34.25 | 32.24 | 54.80 | 38.87 |
| Black-box | AnglE | 51.72 | 36.72 | 39.38 | 37.23 | 33.22 | 31.18 | 65.35 | 42.11 |
| Black-box | OpenAI | 58.14 | 36.88 | 38.03 | 36.53 | 32.70 | 31.27 | 67.96 | 43.07 |
| Interpretable | Bag-of-Tokens | 8.52 | 17.64 | 4.70 | 3.32 | 11.39 | 13.05 | 15.67 | 10.61 |
| Interpretable | QAEmb-MBQA | 36.72 | 25.68 | 24.66 | 21.16 | 25.53 | 22.85 | 46.57 | 29.02 |
| Interpretable | CQG-MBQA (Ori.) | 40.00 | 28.22 | 34.88 | 31.13 | 31.02 | 28.71 | 54.40 | 35.48 |
| Interpretable | CQG-MBQA (Repro.) | 43.95 | 34.50 | 36.78 | 33.21 | 31.45 | 29.29 | 57.93 | 38.16 |
| Interpretable | Ours (Level 4) | 42.78 | 34.50 | 36.93 | 32.65 | 31.74 | 29.41 | 58.24 | 38.04 |
| Interpretable | Ours (Level 3) | 41.01 | 33.64 | 35.68 | 31.80 | 31.02 | 28.71 | 56.61 | 36.93 |
| Interpretable | Ours (Level 2) | 34.32 | 32.36 | 29.87 | 27.31 | 27.87 | 25.74 | 50.78 | 32.61 |
| Interpretable | Ours (Level 1) | 28.75 | 30.21 | 23.58 | 18.36 | 23.80 | 20.60 | 38.63 | 26.27 |
Table 5. Task performance by dataset on the text clustering task.
(d) is 0.4 (e) is 0.5 (f) is 0.6
(g) is 0.7 (h) is 0.8 (i) is 0.9
Figure 20. Task performance vs. cognitive load on the text clustering task when the number of questions is matched across levels by random sampling.
References
- [1]Charu C. Aggarwal and ChengXiang Zhai. 2012. A survey of text clustering algorithms. In Mining Text Data, pages 77–128. Springer US, Boston, MA.DOI
- [2]Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Iñigo Lopez-Gazpio, Montse Maritxalar, Rada Mihalcea, German Rigau, Larraitz Uria, and Janyce Wiebe. 2015. SemEval-2015 task 2: Semantic textual similarity, English, Spanish and pilot on interpretability. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 252–263, Denver, Colorado. Association for Computational Linguistics.
- [3]Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2014. SemEval-2014 task 10: Multilingual semantic textual similarity. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 81–91, Dublin, Ireland. Association for Computational Linguistics.
- [4]Eneko Agirre, Carmen Banea, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2016. SemEval-2016 task 1: Semantic textual similarity, monolingual and cross-lingual evaluation. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 497–511, San Diego, California. Association for Computational Linguistics.
- [5]Eneko Agirre, Daniel Cer, Mona Diab, and Aitor Gonzalez-Agirre. 2012. SemEval-2012 task 6: A pilot on semantic textual similarity. In *SEM 2012: The First Joint Conference on Lexical and Computational Semantics – Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012), pages 385–393, Montréal, Canada. Association for Computational Linguistics.
- [7]Vishal Anand, Milad Alshomary, and Kathleen McKeown. 2026. iBERT: Interpretable embeddings via sense decomposition. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1413–1429, Rabat, Morocco. Association for Computational Linguistics.
- [8]Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. 2018. Ms marco: A human generated machine reading comprehension dataset. Preprint, arXiv:1611.09268.arxiv.org/abs/1611.09268
- [9]Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy. 2024. LLM2vec: Large language models are secretly powerful text encoders. In First Conference on Language Modeling.arxiv.org/abs/2404.05961
- [10]Vinamra Benara, Chandan Singh, John Xavier Morris, Richard Antonello, Ion Stoica, Alexander Huth, and Jianfeng Gao. 2024. Crafting interpretable embeddings for language neuroscience by asking LLMs questions. In The Thirty-eighth Annual Conference on Neural Information Processing Systems.pubmed.ncbi.nlm.nih.gov/40276238
- [11]Vera Boteva, Demian Gholipour, Artem Sokolov, and Stefan Riezler. 2016. A full-text learning to rank dataset for medical information retrieval. In Proceedings of the 38th European Conference on Information Retrieval. ECIR.DOI
- [12]Daniel Cer, Mona Diab, Eneko Agirre, Iñigo Lopez-Gazpio, and Lucia Specia. 2017. SemEval-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 1–14, Vancouver, Canada. Association for Computational Linguistics.DOI
- [13]Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 1597–1607. PMLR.
- [14]Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, and Daniel Weld. 2020. SPECTER: Document-level representation learning using citation-informed transformers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2270–2282. Online. Association for Computational Linguistics.arxiv.org/abs/2004.07180
- [15]Alexis Conneau, Douwe Kiela, Holger Schwenk, Loïc Barrault, and Antoine Bordes. 2017. Supervised learning of universal sentence representations from natural language inference data. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 670–680, Copenhagen, Denmark. Association for Computational Linguistics.arxiv.org/abs/1705.02364
- [16]Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.arxiv.org/abs/1810.04805
- [17]Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. SimCSE: Simple contrastive learning of sentence embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894–6910, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.arxiv.org/abs/2104.08821
- [18]John Hewitt, John Thickstun, Christopher Manning, and Percy Liang. 2023. Backpack language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9103–9125, Toronto, Canada. Association for Computational Linguistics.arxiv.org/abs/2305.16765
- [19]Ting Jiang, Shaohan Huang, Zhongzhi Luan, Deqing Wang, and Fuzhen Zhuang. 2024. Scaling sentence embeddings with large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 3182–3196, Miami, Florida, USA. Association for Computational Linguistics.arxiv.org/abs/2307.16645
- [20]Ting Jiang, Jian Jiao, Shaohan Huang, Zihan Zhang, Deqing Wang, Fuzhen Zhuang, Furu Wei, Haizhen Huang, Denvy Deng, and Qi Zhang. 2022. Prompt-BERT: Improving BERT sentence embeddings with prompts. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8826–8837, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.arxiv.org/abs/2201.04337
- [21]Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. 2025. NV-embed: Improved techniques for training LLMs as generalist embedding models. In The Thirteenth International Conference on Learning Representations.arxiv.org/abs/2405.17428
- [23]Yibin Lei, Di Wu, Tianyi Zhou, Tao Shen, Yu Cao, Chongyang Tao, and Andrew Yates. 2024. Meta-task prompting elicits embeddings from large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10141–10157, Bangkok, Thailand. Association for Computational Linguistics.
- [24]Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2021. Retrieval-augmented generation for knowledge-intensive nlp tasks. Preprint, arXiv:2005.11401.arxiv.org/abs/2005.11401
- [25]Ailiang Lin, Zhuoyun Li, Yusong Wang, Keyu Mao, Kotaro Funakoshi, and Manabu Okumura. 2026. Reverse-sequel: Improving training-free text embeddings via text reversal in decoder-only llms. Preprint, arXiv:2606.05858.arxiv.org/abs/2606.05858
- [26]Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized bert pretraining approach. Preprint, arXiv:1907.11692.arxiv.org/abs/1907.11692
- [27]Macedo Maia, Siegfried Handschuh, André Freitas, Brian Davis, Ross McDermott, Manel Zarrouk, and Alexandra Balahur. 2018. WWW ’18 open challenge: Financial opinion mining and question answering. In Companion Proceedings of the The Web Conference 2018, WWW ’18, page 1941–1942, Republic and Canton of Geneva, CHE. International World Wide Web Conferences Steering Committee.
- [28]Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, and Roberto Zamparelli. 2014. A SICK cure for the evaluation of compositional distributional semantic models. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 216–223, Reykjavik, Iceland. European Language Resources Association (ELRA).DOI
- [29]Denis McInerney, Geoffrey Young, Jan-Willem van de Meent, and Byron Wallace. 2023. CHiLL: Zero-shot custom interpretable feature extraction from clinical notes with large language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 8477–8494, Singapore. Association for Computational Linguistics.
- [30]John Xavier Morris and Alexander M Rush. 2025. Contextual document embeddings. In The Thirteenth International Conference on Learning Representations.arxiv.org/abs/2410.02525
- [32]Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023. MTEB: Massive text embedding benchmark. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2014–2037, Dubrovnik, Croatia. Association for Computational Linguistics.
- [33]Juri Opitz and Anette Frank. 2022. SBERT studies meaning representations: Decomposing sentence embeddings into explainable semantic features. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 625–638, Online only. Association for Computational Linguistics.DOI
- [34]Ajay Patel, Delip Rao, Ansh Kothary, Kathleen McKeown, and Chris Callison-Burch. 2023. Learning interpretable style embeddings via prompting LLMs. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 15270–15290, Singapore. Association for Computational Linguistics.arxiv.org/abs/2305.12696
- [35]Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992, Hong Kong, China. Association for Computational Linguistics.arxiv.org/abs/1908.10084
- [36]Wonseok Shin and Songkuk Kim. 2026. Aligning sentence embeddings to human concepts via sparse autoencoders. In ICLR 2026 Workshop on Representational Alignment (Re^4-Align).arxiv.org/abs/2607.00023
- [37]Yiqun Sun, Qiang Huang, Yixuan Tang, Anthony Kum Hoe Tung, and Jun Yu. 2025. A general framework for producing interpretable semantic text embeddings. In The Thirteenth International Conference on Learning Representations.arxiv.org/abs/2410.03435
- [38]Yixuan Tang, Zhenghong Lin, Yandong Sun, Wynne Hsu, Mong Li Lee, and Anthony K. H. Tung. 2026. Asking the right questions: Ontology-grounded interpretable embeddings for biomedical text. Preprint, arXiv:2603.01690.arxiv.org/abs/2603.01690
- [39]Matthieu Tehenan, Vikram Natarajan, Jonathan Michala, Milton Lin, and Juri Opitz. 2025. Mechanistic decomposition of sentence representations. Preprint, arXiv:2506.04373.arxiv.org/abs/2506.04373
- [40]Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).
- [41]Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2023. Attention is all you need. Preprint, arXiv:1706.03762.
- [42]Henning Wachsmuth, Shahbaz Syed, and Benno Stein. 2018. Retrieval of the best counterargument without prior topic knowledge. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 241–251, Melbourne, Australia. Association for Computational Linguistics.DOI
- [43]David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, and Hanneke Hajishirzi. 2020. Fact or fiction: Verifying scientific claims. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7534–7550, Online. Association for Computational Linguistics.arxiv.org/abs/2004.14974
- [44]Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2024. Text embeddings by weakly-supervised contrastive pre-training. Preprint, arXiv:2212.03533.arxiv.org/abs/2212.03533
- [45]Yile Wang, Zhanyu Shen, and Hui Huang. 2025. LDIR: Low-dimensional dense and interpretable text embeddings with relative representations. In Findings of the Association for Computational Linguistics: ACL 2025, pages 14397–14409, Vienna, Austria. Association for Computational Linguistics.arxiv.org/abs/2505.10354




