betaFeedback

WorksComputer Science

The Linguistic Structure of Dependency-Conditioned Embeddings

Abstract

Dependency-based distributional models capture syntactically informed lexical similarities, but the geometric organisation of dependency-conditioned lexical representations remains underexplored. We investigate the global and local geometry of lemma-dependency relation embeddings using four representations trained on WikiText-103: two count-based PPMI configurations and two Skip-Gram with Negative Sampling (SGNS) configurations. At the global level, we analyse dependency-relation centroids, control for lexical identity through within-lemma centring, and examine their alignment with part-of-speech (POS) distributions. At the local level, we evaluate cross-relation geometric consistency, cross-method agreement, lexical similarity, and polysemous usages. Results show that global geometry is structured strongly by context POS across all representations. Locally, broader lexical similarity structures are partially preserved across dependency relations, while immediate neighbourhoods vary substantially, highlighting different usages. PPMI-based representations exhibit greater cross-relation consistency than SGNS-based representations, with cross-method agreement varying by relation. These findings reveal systematic linguistic organisation at multiple geometric scales and its sensitivity to representation choices.

Introduction and Related Work

Distributional semantic models (DSMs) represent lexical meaning through patterns of word co-occurrence. The captured semantic information depends not only on the learning algorithm but also on the definition of the linguistic context. In particular, whereas linear-context embeddings exhibit stronger topical associations, dependency-based models, which use grammatical relations rather than linear proximity to define contexts, produce representations that emphasise functional similarities between words [7, 12, 2]. Beyond the broad distinction between linear and syntactic contexts, individual dependency relations may also provide complementary information about lexical meaning [4]. Mouton et al. [11] constructed multiple semantic spaces from distinct syntactic relations and provided preliminary observations suggesting that the nearest neighbours of a polysemous word can vary across these spaces, highlighting different usages.

Previous studies have predominantly investigated dependency-based semantic information through 2 complementary representational strategies. The first constructs separate semantic spaces for individual syntactic relations [11] using count-based methods, enabling relation-specific analyses but without a common coordinate system for directly examining cross-relation geometry. The second incorporates dependency relations into contextual features while learning a single target representation for each lexical item [7]. Although the latter aggregates information across dependency relations in its target embeddings, dependency-based SGNS also learns representations of lemma–relation pairs in its context embedding matrix. These context representations have mainly been used to interpret the learned target embeddings [8] or as features for downstream applications [5], rather than being systematically investigated as a semantic space in their own right. This motivates examining the geometric organisation of dependency-conditioned lexical units jointly represented within a shared space.

An additional question concerns the extent to which this geometric organisation depends on the representation method. Count-based and predictive distributional models have been extensively compared in terms of semantic performance and representation properties, including models employing dependency contexts [1, 3, 9, 14, 6]. However, these studies have primarily focused on lemma-level representations rather than dependency-conditioned lexical units themselves. The present study therefore examines whether the geometric properties of jointly represented dependency-conditioned lexical units are consistent across count-based and predictive methods.

We investigate 4 representations derived from dependency-based co-occurrence information:

  • count-based representations constructed using 5000 most frequent context features;

  • count-based representations constructed using all context features but reduced to 300 dimensions through Singular Value Decomposition (SVD);

  • target embeddings obtained from a reversed dependency SGNS configuration, in which lemma-relation pairs serve as prediction targets;

  • context embeddings obtained from original dependency SGNS.

All 4 represent lemma-relation pairs within a shared space, allowing their geometric organisation to be examined at 2 levels: global organisation across dependency relations and local lexical neighbourhoods within individual relations.

By examining comparable lexical units across these configurations, we aim to characterise their geometric organisation and assess its consistency across representation methods. Specifically, we address 3 research questions:

  • Global geometry: What geometric structures emerge when dependency-conditioned lexical units are jointly represented within a shared semantic space?

  • Local geometry: How are local semantic neighbourhoods organised within the shared semantic space, and to what extent do they reflect relation-specific semantic patterns?

  • Cross-method comparison: To what extent do count-based and predictive representations exhibit similar global and local geometric structures?

By characterising the geometry of dependency-conditioned lexical units and comparing its organisation across representation methods, this study extends dependency-based distributional analysis beyond conventional lexical target embeddings. The findings aim to inform the interpretation and selection of dependency-conditioned representations for lexical semantic research.

Data and Representations

Data

We used WikiText-103 [10], a large-scale English language modelling corpus comprising approximately 103 million tokens extracted from Wikipedia articles. Its size, linguistic diversity, and public availability make it suitable for training dependency embedding, which requires sufficient observations of lexical items across different syntactic contexts. We pre-processed the data (Appendix § 6.§ 1) then parsed the sentences using Stanza [13].

Types of representations

We construct 4 dependency distributional representations (Table 1) using 2 count-based and 2 SGNS configurations (original and reversed). In original SGNS, ordinary lemmas serve as targets and dependency-conditioned items as contexts. We additionally consider the reversed orientation, in which dependency-conditioned items become targets and ordinary lemmas become contexts. Since the present study focuses on the geometric organisation of dependency-conditioned lexical units, we examine the representations of these units across all 4 configurations. For count-based models and reversed SGNS, they correspond to target representations, whereas for original SGNS, they correspond to context representations.

ModelConstructionDim.Vectors
PPMI-5000Top 5,000 contexts + PPMI5,000Rows
PPMI-SVDFull contexts + PPMI + SVD300Rows
Orig. SGNSLemma → lemma/relation300Context
Rev. SGNSLemma/relation → lemma300Target

Table 1. Summary of the 4 dependency-conditioned representations. Orig. denotes original-context SGNS; Rev. denotes reversed SGNS.

Each dependency co-occurrence is represented as a tuple (A,B/r,f)(A, B/r, f), where AA is a lemma, B/rB/r is a dependency-conditioned lexical unit consisting of a lemma and a dependency relation, and ff is their co-occurrence frequency. To distinguish the direction of dependency relations, we use the prefixes chi_ and pa_. For example, if AA is the syntactic head of BB under relation rr, the corresponding co-occurrences are (A,B/chir,f)(A, B/chi_r, f) and (B,A/par,f)(B, A/pa_r, f). In the reversed training configuration, the order of these pairs is inverted, yielding (B/chir,A,f)(B/chi_r, A, f) and (A/par,B,f)(A/pa_r, B, f), respectively.

Count-based representations

We construct 2 count-based models using Positive Pointwise Mutual Information (PPMI), a commonly used association measure in distributional semantics [9]. Both models use the reversed orientation of the dependency co-occurrence data, such that matrix rows correspond to dependency-conditioned lexical units and columns correspond to ordinary lexical contexts. The constructing procedures are detailed in Appendix A.2.

PPMI-5000. The first configuration constructs an explicit sparse representation using the 5000 most frequent context features, ranked according to their marginal frequencies, with ties resolved alphabetically to ensure reproducibility.

PPMI-SVD. The second constructs a PPMI matrix using the full context vocabulary and applies Truncated Singular Value Decomposition (SVD) to obtain 300-dimensional vectors.

SGNS representations

We employ dependency SGNS [7], which extends the original SGNS to support arbitrary context features. We train 2 configurations that differ in the orientation of the target–context pairs. The training procedures are detailed in Appendix A.2.

Original-context SGNS. The first configuration follows the original orientation of dependency SGNS, in which ordinary lemmas serve as targets and dependency-conditioned lexical units serve as contexts. The model then learns separate target and context embedding matrices. Although previous dependency embedding studies have primarily examined the target matrix, our analysis focuses on the context matrix, where each dependency-conditioned lexical unit is represented by a 300-dimensional vector.

Reversed SGNS. The second configuration reverses the orientation of the training pairs, treating dependency-conditioned lexical units as targets and ordinary lemmas as contexts. Consequently, the dependency-conditioned units are learned in the target embedding matrix rather than the context matrix. The resulting 300-dimensional target vectors are extracted for geometric analysis. As reversing the target–context orientation changes the roles of the 2 vocabularies within the SGNS training procedure and negative samples are drawn from a smoothed unigram distribution with an exponent of 0.75, reversing the training orientation changes the negative sampling distribution, meaning that the 2 configurations do not necessarily yield equivalent embedding geometries.

Experiments and Results

We investigate the geometric and linguistic organisation of dependency-conditioned representations at 2 levels: global and local. Global geometry concerns the overall organisation of dependency relations within the shared semantic space, whereas local geometry concerns the relationships among lexical items within individual dependency relations. Unless otherwise specified, cosine similarity is used to measure relationships between embedding vectors. To examine the robustness of the findings, we additionally evaluate the effects of vocabulary size, frequency distribution, and representation method.

Global geometry

We first investigate how dependency relations are organised within the shared semantic space through 2 stages of analysis. To avoid confusion, target POS refers to the POS of the lemma in a dependency-conditioned item, whereas context POS refers to the POS of the lemma syntactically connected to that target.

Exploratory Analysis

We first explored the global organisation of dependency-conditioned embeddings by selecting the 10000 most frequent lemma–relation types and balancing their representation across dependency relations. We visualised the selected embeddings using t-SNE (Figure 1) and applied hierarchical agglomerative clustering (HAC) to relation centroids based on cosine distance (Figure 3). The resulting structures suggested systematic groupings of relations associated with similar contextual POS distributions.

t-SNE projection

Figure 1. t-SNE projection.

HAC dendrogram for relation centroid analysis

Figure 3. HAC dendrogram for relation centroid analysis

To examine whether these patterns were driven by differences in lexical composition, we conducted a second exploratory analysis using within-lemma centring. We subtracted each lemma’s mean vector across its available relation variants before computing residual relation centroids and applying HAC (Figure 2). Similar groupings still existed and persisted across all 4 representations, suggesting that the observed organisation was not solely attributable to lemma-specific embedding positions. Detailed sampling procedures, centring equations, and clustering settings are provided in Appendix § 7.§ 1.

HAC dendrogram for lexically centred relation analysis

Figure 2. HAC dendrogram for lexically centred relation analysis

POS distribution and embedding geometry

These observations motivated a quantitative analysis of the association between relation-level embedding geometry and POS distributions.

To quantify the patterns observed in the exploratory analyses, we examined whether dependency relations that are geometrically similar also have similar POS distributions. Specifically, we compared the distances between lexically centred relation centroids with the differences in the POS distributions of their represented lemmas (target POS) and syntactic counterparts (context POS).

For each relation, POS distributions were reconstructed from the parsed corpus, normalised for each lemma–relation type, and then averaged across types. We measured embedding distances using cosine distance and POS-distribution differences using Jensen–Shannon distance. Spearman’s rank correlation was used to assess the correspondence between these distances across relation pairs. Statistical significance was evaluated using 10,000 relation-label permutations. We additionally tested whether context-POS alignment was stronger than target-POS alignment (Δρ=ρcontext−ρtarget\Delta\rho= \rho_{\mathrm{context}} - \rho_{\mathrm{target}}).

Note: Ctx. = Context-POS; Tgt. = Target-POS; Orig. = Original-context; Rev. = Reversed. Δρ=ρctx−ρtgt\Delta\rho= \rho_{\mathrm{ctx}} - \rho_{\mathrm{tgt}}. pp < .01; *pp < .001 (10000 relation-label permutations). Correlations were tested two-sided; Δρ\Delta\rho was tested one-sided.

Context-POS alignment As shown in Table 2, all 4 representations exhibited significant positive correlations between relation-centroid distances and context-POS distances (ρ=.463\rho= .463–.603, p<.001p < .001). In contrast, correlations with target-POS distances were weakly negative (ρ=−.094\rho= -.094 to −.128). Direct permutation tests confirmed that context-POS alignment was significantly stronger than target-POS alignment in all 4 representations (Δρ=.557\Delta\rho= .557–.731, p<.001p < .001). These results support the hypothesis that the global geometry of lexically centred dependency-relation embeddings is more closely associated with the POS composition of context words than with that of the embedded lexical items.

ModelCtx. ρ\rhoTgt. ρ\rhoΔρ\Delta\rho
PPMI-SVD.603***−.128***.731***
Orig. SGNS.576***−.100***.676***
PPMI-5000.575***−.112***.687***
Rev. SGNS.463***−.094***.557***

Table 2. Correlations between lexically centred relation distances and POS-distribution distances at the top-10000 vocabulary cutoff.

Robustness and frequency sensitivity. To assess whether context-POS alignment depends on vocabulary selection, we repeated the analysis across 9 cumulative vocabulary cutoffs (top 200–20,000). At every cut-off, across all 4 representations, context-POS correlations remained positive and significant (p<.001p < .001), and direct context-minus-target comparisons were also significant (p<.001p < .001) (Table 6). This indicates that the main finding is not specific to the primary top-10,000 selection. However, because cumulative vocabulary expansion also increases the number of retained dependency relations, variation in correlation magnitude may reflect changes in both lexical coverage and relation composition. Detailed results are provided in Appendix § 7.§ 2.

RepresentationContext-POS ρ\rho
PPMI-SVD.577–.796
PPMI-5000.560–.651
Original-context SGNS.545–.749
Reversed SGNS.405–.494

Table 6. Context-POS alignment across 9 cumulative vocabulary cutoffs.

Note. All reported context-POS correlations and direct context-minus-target comparisons were significant at p<.001p < .001. The number of retained relations increased from 16 at the top 200 cutoff to 65 at the top 20000 cutoff.

To investigate whether context-POS alignment varied across lexical frequency ranges, we conducted a second sensitivity analysis using 6 non-overlapping frequency bands using a fixed set of 29 dependency relations shared by all bands. Context-POS alignment remained positive and significant across all bands and representations (p<.001p < .001; Table 7). However, different models exhibited different descriptive frequency profiles. In particular, the count-based representations exhibited stronger alignment among higher-frequency units, whereas reversed SGNS showed stronger alignment among lower-frequency units.

Freq. ranksRev.Orig.SVDPPMI
1–1000.420.621.738.605
1001–3000.447.612.714.647
3001–10000.450.562.632.617
10001–30000.479.546.539.563
30001–60000.520.527.438.458
60001–120000.609.539.368.390

Table 7. Context-POS correlations across 6 non-overlapping frequency bands, using a fixed set of 29 dependency relations. Note: Rev. = Reversed SGNS; Orig. = Original-context SGNS; SVD = PPMI-SVD (300d); PPMI = PPMI-5000. Values are Spearman correlations between lexically centred relation distances and context-POS distances. All correlations and direct context-minus-target comparisons were significant at p<.001p < .001 (10000 permutations).

These quantitative findings support the context-POS organisation suggested by the HAC analyses. It is robust across vocabulary selections, although its strength varies with lexical frequency and representation method.

Local Geometry

We investigate local lexical geometry from 3 complementary perspectives: its consistency across dependency relations, its agreement across embedding methods, and its linguistic interpretability. To ensure comparability across vocabulary sizes, we retained 112 dependency-relation pairs with at least 2,000 shared lemmas. The same relation pairs were used across all vocabulary cutoffs to prevent changes in relation-pair coverage from affecting the comparisons. Further details on shared-vocabulary coverage and pair selection are provided in Appendix § 8.§ 1.

Cross-Relation Geometric Consistency

Shared-vocabulary geometric consistency. To assess whether lexical similarity structures are preserved across dependency relations, we selected NN shared lemmas for each of the 112 eligible relation pairs. Within each relation, we calculated pairwise cosine similarities among the selected lemmas and compared their relative ordering across relations using Spearman’s rank correlation. The resulting correlations were macro-averaged across relation pairs. Higher values indicate greater consistency in lexical similarity rankings across dependency relations. The sampling procedures are provided in Appendix § 8.§ 2.

Interestingly, PPMI-based representations consistently exhibited higher cross-relation correlations than SGNS representations (Table 8). This pattern was observed across vocabulary sizes, indicating greater preservation of relative lexical similarity rankings across dependency relations in the PPMI-based representations.

NNRev.Orig.SVDPPMI
10.083.109.252.332
20.156.191.352.435
50.150.195.326.427
100.154.185.354.459
200.153.183.360.465
500.163.191.374.465
1000.165.195.376.450
1100.164.195.373.444
1200.164.195.370.439
1500.165.197.364.425
2000.174.206.357.408

Table 8. Macro-mean Spearman correlations across 112 dependency relation pairs.

Local neighbourhood consistency. While pairwise similarity correlations assess the preservation of lexical similarity rankings, they do not indicate whether individual lemmas retain the same nearest neighbours. We therefore compare the top-10 nearest-neighbour sets of each shared lemma across dependency relations using chance-corrected Jaccard overlap: Scores are averaged across lemmas within each relation pair and then macro-averaged across the same 112 relation pairs. Further implementation details and vocabulary sensitivity results are provided in Appendix § 8.§ 2.

Neighbourhood overlap remains low across all 4 representations (Table 3). Nevertheless, PPMI models consistently yield the higher Jaccard scores than SGNS models, further supporting the observation that PPMI-based representations exhibit greater cross-relation consistency in both lexical similarity rankings and immediate neighbourhoods.

NNRev.Orig.SVDPPMI
200.080.088.127.143
500.073.079.107.124
1000.065.070.095.112
1100.063.067.092.108
1200.061.065.090.106
1500.057.061.084.099
2000.053.056.078.091

Table 3. Macro-mean chance-corrected KNN Jaccard across 112 dependency relation pairs.

Note: Rev. = Reversed SGNS; Orig. = Original-context SGNS; SVD = PPMI-SVD (300d); PPMI = PPMI-5000.

Together, these findings suggest that dependency-conditioned embeddings preserve some broader lexical similarity regularities across dependency relations, while exhibiting substantial variation in their immediate lexical neighbourhoods. The extent of this cross-relation consistency also depends on the representation method, with PPMI-based representations showing greater consistency than SGNS-based representations.

Cross-Method Comparison

We further examine the lexical structures within individual dependency relations from 2 complementary perspectives: their correspondence with human semantic judgements and their consistency across representation methods.

Lexical Semantic Evaluation. We first evaluated all representations against the WordSim-353 Similarity and Relatedness subsets. Across 14 dependency relations with matched benchmark coverage, correlations with human judgements were consistently higher for Similarity than for Relatedness. PPMI-5000 achieved the highest correlations for 9 of 14 relations on Similarity and 7 of 14 on Relatedness. These results suggest that the representations capture lexical similarity information reflected in human judgements, with stronger alignment to similarity than to relatedness ratings. Moreover, the performance of PPMI-5000 indicates that predictive representations are not necessarily superior to count-based alternatives on this benchmark, consistent with previous comparative studies [6, 9]. A particularly notable result concerns the 2 SGNS representations. Original-context SGNS outperforms Reversed SGNS for 9/14 relations on Similarity and 10/14 relations on Relatedness, even though reversed SGNS is trained specifically to represent dependency-conditioned units as target vectors. Thus, the original SGNS context vectors, which are learned jointly with the target vectors but are often treated as a secondary training output and discarded, retain more benchmark-relevant semantic information for most dependency relations. Detailed evaluation procedures and results are provided in Appendix § 8.§ 3.

Cross-Method Representational Similarity Analysis (RSA). To examine whether different embedding methods preserve similar lexical structures within individual dependency relations, we compared pairwise cosine similarity rankings across the 4 representations. For each of 50 dependency relations, we selected 200 shared lexical units and calculated Spearman correlations between their similarity matrices.

Agreement was higher within methodological families (mean ρ=.774\rho= .774–.791) than between PPMI and SGNS (mean ρ=.581\rho= .581–.654). Notably, cross-family agreement varied substantially across dependency relations, from ρ=.911\rho= .911 for pa_conj to ρ=−.029\rho= -.029 for pa_det. These findings suggest that the extent to which lexical geometry is preserved across methods depends on the dependency relation. Full RSA results are provided in Appendix § 8.§ 3.

Overall, the findings of this section suggest that dependency-conditioned representations capture meaningful lexical semantic structures, while their geometric organisation varies across both dependency relations and representation methods.

Linguistic Interpretation

Within-relation lexical structure We qualitatively inspect the dendrograms of several relations (but use only pa_nsubj of Original-context SGNS for illustration) to examine whether lemma–relation embeddings belonging to the same dependency relation exhibit meaningful internal organisation. From the global top 2000 frequency-ranked lemma–relation types, 108 pa_nsubj types were retained. Figure 5 reveals several locally interpretable branches. Cognitive and attitudinal verbs form close groupings, including think–feel, consider–believe, and decide–want. Communication verbs also show coherent local structure, including describe–say and the broader claim–suggest–argue–tell branch. Process and event verbs produce additional pairings, such as begin–start, result–cause–occur, consist–contain–include, and release–announce. This therefore suggests that a single dependency-conditioned space can preserve fine-grained lexical structure rather than forming an undifferentiated cloud. Across the 4 models, several local lexical-semantic groupings remain relatively consistent, whereas their integration into larger hierarchical clusters varies across embedding methods.

HAC dendrogram for pa_nsubj of Original-context SGNS

Figure 5. HAC dendrogram for pa_nsubj of Original-context SGNS

Nearest-neighbour exploration To explore relation-specific semantic differentiation, we manually selected 30 English lemmas exhibiting within-POS polysemy or cross-POS ambiguity, each represented under multiple dependency relations. We compared their nearest-neighbour structures across relations to examine how different syntactic contexts highlight distinct meanings or usages. The full candidate list is provided in Table 13.

TypePOSLemmas
Within-POSNounbank, paper, channel, market, branch
Verbhold, charge, turn, see, find
Adjectivehard, heavy, certain, late, responsible
Cross-POSNounfile, match, face, head, chair
Verbrecord, plant, drive, call, watch
Adjectivefine, right, present, mean, last

Table 13. 30 selected lemmas for qualitative analysis.

For each query lemma/relation (e.g., bank/chi_compound, bank/chi_obl), we compute cosine similarity against all non-zero candidate vectors carrying the same relation suffix and retain the 5 candidates with the highest cosine similarity. Restricting the candidate set to the same relation ensures that a neighbourhood compares lexical items within 1 relation-conditioned space. Table 4 reports 6 Within-POS cases, in which bank, hold, and hard retain their broad Noun, Verb, and Adjective status, respectively, while different relations emphasise different readings within that category. bank provides the clearest nominal example. Under chi_compound, its neighbourhood is predominantly financial or institutional across all 4 spaces, with neighbours such as insurance, mortgage, bankruptcy, banking, service, company. Under chi_obl, all 4 spaces shift towards the geographical river-bank reading, retrieving tributary, estuary, valley, river, hill, shore. The precise neighbours vary, but the relation-conditioned sense contrast is stable across training algorithms. hold illustrates a frame-sensitive verbal contrast. Its chi_acl representation is surrounded by verbs associated with hosting or conducting events, including host, contest, preside, attend, organise. Its pa_ccomp representation instead lies near verbs of propositional stance and communication, including argue, maintain, conclude, believe, contend, suggest, claim, state. The dependency relation therefore changes which facet of the same lemma dominates its local geometry. hard shows particularly strong cross-model agreement. The chi_amod neighbourhood emphasises physical or material properties through soft, tough, solid, coarse, and related descriptors. In contrast, pa_ccomp consistently retrieves difficult, easy, impossible, highlighting an evaluative judgement about task difficulty or possibility. This contrast appears in both SGNS and PPMI spaces despite differences in the remaining neighbours. The selected Within-POS examples therefore exhibit broadly consistent semantic contrasts across the 4 representations, despite differences in individual nearest neighbours.

ReadingRelationReversed SGNSOriginal-context SGNSPPMI-SVDPPMI-5000
bank: financialchi_compoundinsurance, mortgage, booking, bankruptcymortgage, gate, booking, wire, KGBgovernment, service, property, estate, councilgovernment, banking, service, company, Bank
bank: river edgechi_obltributary, estuary, foothill, valley, ravinevalley, estuary, Plain, Firth, shorehill, river, shore, valley, RiverRiver, town, area, shore, south
hold: event hostingchi_acltake, contest, host, preside, benefitorganised, contest, host, preside, takehost, organised, attend, organize, celebratetake, host, attend, organised, call
hold: propositional stancepa_ccompargue, establish, conclude, maintain, believeargue, rule, maintain, conclude, assertrule, contend, maintain, determine, establishargue, believe, suggest, claim, state
hard: physical propertychi_amodsoft, process, tougher, shiny, toughtough, tougher, slippery, tinged, flavorsoft, solid, melt, loose, coarsesoft, tough, solid, more, color
hard: difficulty judgementpa_ccompdifficult, easy, impossible, affect, hopeeasy, impossible, difficult, discuss, affectdifficult, impossible, easy, help, teachdifficult, easy, impossible, state, say

Table 4. Within-POS examples. 5 neighbours are shown in decreasing cosine-similarity order.

Table 5 reports 8 Cross-POS cases. file and face contrast nominal and verbal readings, while fine and present contrast adjective-centred readings with nominal or verbal alternatives. Cross-POS differentiation is also consistent in several cases. file/chi_obj is organised around documents, data, photographs, and recorded information, whereas file/pa_obj is surrounded by submission verbs such as lodge, forward, submit, retract, withdraw; face/chi_obj retrieves body-part terms such as eye, head, nose, breast, throat, while face/pa_obj retrieves confrontational actions such as confront, overcome, defeat, beat, eliminate; fine/chi_amod expresses positive quality through superb, exquisite, elegant, brilliant, excellent, beautiful; fine/chi_obj instead emphasises penalties through tax, fee, sanction, ban, penalty. These contrasts are stable across all 4 models and are therefore interpreted once as shared cross-model patterns.

ReadingRelationReversed SGNSOriginal-context SGNSPPMI-SVDPPMI-5000
file: documentchi_obje-mail, diary, inventory, email, Spiritdiary, gem, software, poison, Spiritsample, fragment, photograph, tape, photodocument, data, information, map, tape
file: submit or lodgepa_objlodge, print, forward, invalidate, submitlodge, forward, retract, circulate, rescindissue, forward, submit, retract, appeallodge, submit, withdraw, dismiss, issue
face: body partchi_objbreast, sand, fence, cylinder, trunkbreast, throat, paint, rock, sandeye, hair, head, breast, taileye, head, nose, hand, image
face: confrontpa_objovercome, fear, eliminate, avoid, overover, defeat, eliminate, confront, overcomeeliminate, defeat, beat, pit, challengedefeat, beat, avoid, eliminate, overcome
fine: qualitychi_amodquality, coarse, superb, exquisite, elegantexquisite, elegant, quality, pleasing, colourfulpolished, brilliant, elegant, soft, staingood, famous, excellent, brilliant, beautiful
fine: financial penaltychi_objdividend, subsidy, Assent, penalty, taxdividend, Assent, condemnation, bribe, pensiontax, fee, sanction, suspension, finepenalty, tax, ban, fee, sanction
present: currentchi_amodcurrent, new, same, pre-existdifferent, current, serve, convenient, pre-exist, newcurrent, certain, typical, same, particularcurrent, new, different, certain, exist
present: model-dependent verbal neighbourhoodpa_objsee, compare, viewfind, tout, see, compare, address, find, acceptview, consider, compare, base, characterizeconsider, describe, introduce, show, discuss

Table 5. Cross-POS examples. 5 neighbours are shown in decreasing cosine-similarity order.

The Cross-POS cases also reveal a clearer division in model strengths. The PPMI spaces often provide the most transparent category separation: PPMI-SVD and PPMI-5000 produce especially coherent nominal and verbal sets for file, face, fine. The SGNS spaces are strong when the alternative reading is tied to a selective verbal frame, as in file/pa_obj, face/pa_obj, but their nominal neighbourhoods more often contain specialised or idiosyncratic items. present/pa_obj is the clearest cross-model disagreement and is therefore analysed separately. The difference is interpretive: the first 3 spaces foreground evaluation or perception, whereas PPMI-5000 foregrounds the act of presenting information.

In the selected Within-POS examples, the 4 models generally exhibit similar relation-conditioned semantic interpretations, with differences primarily concerning lexical specificity. The Cross-POS examples also show interpretable differentiation, but reveal greater variation in neighbourhood coherence and semantic emphasis across representations.

Discussion and Conclusion

Our study provides a systematic characterisation of the linguistic and geometric organisation of dependency-conditioned lexical representations within a shared semantic space. Three main findings emerge. First, at the global level, dependency-relation geometry is systematically associated with the POS composition of lexical contexts rather than that of the represented lemma. This pattern persists after within-lemma centring and is consistently observed across all representations, vocabulary selections, and frequency bands. Second, at the local level, dependency relations exhibit distinct but partially overlapping lexical similarity structures. Pairwise similarity correlations reveal some preservation of broader lexical relationships across relations, whereas relatively low nearest-neighbour overlap indicates substantial variation in immediate lexical neighbourhoods. These results suggest that dependency-conditioned embeddings preserve broader lexical regularities while allowing relation-specific differentiation in immediate lexical neighbourhoods, with the extent of this differentiation varying across representation methods. Qualitative analyses further demonstrate that these neighbourhoods can highlight different meanings or usages of polysemous words, consistent with earlier observations [11, 4]. Third, the geometric organisation of these representations is sensitive to the embedding method, particularly at the local level. PPMI-based representations exhibit greater cross-relation consistency in both pairwise similarity rankings and nearest-neighbour overlap than SGNS representations. Cross-method RSA further reveals stronger agreement within methodological families than between count-based and predictive models, with substantial variation across dependency relations. These findings indicate that method sensitivity is itself relation-dependent: some dependency relations exhibit similar lexical structures across representations, whereas others are considerably more sensitive to modelling choices. Taken together, these results highlight the value of investigating dependency-conditioned embeddings as structured semantic spaces in their own right, rather than treating lemma–relation vectors solely as contextual features. The shared representation enables the examination of global relationships among dependency relations while retaining interpretable local lexical structures. More broadly, the findings suggest that the geometric properties of dependency-conditioned representations should be considered at multiple levels and across modelling approaches. These findings provide a basis for evaluating and selecting dependency-conditioned representations in downstream lexical semantic tasks, particularly where relation-specific meaning differentiation is important.

Limitation

This study has 2 main limitations. The differences in local geometric consistency across embedding methods remain descriptive and may be beneficial from direct statistical significance testing. The qualitative polysemy analysis relies on manually selected examples, partly because conventional lexical similarity benchmarks do not directly evaluate relation-specific semantic distinctions.

Appendix A - Data Processing and Embedding Implementation Details

Data Processing

The WikiText-103 corpus was divided into articles, stripped of headings and section markup, normalised for Unicode and HTML entities, and corrected for WikiText-specific tokenisation artefacts. The text was detokenised while preserving case and punctuation, segmented into complete sentences, and processed with Stanza for tokenisation, lemmatisation, POS tagging, and dependency parsing.

Count-based Representation

Given a co-occurrence matrix MM, the association between a target ww and a context cc is calculated as:

PPMI⁡(w,c)=max⁡(0,log⁡P(w,c)P(w)P(c))(1)\operatorname{PPMI}(w,c) = \max\left(0,\log\frac{P(w,c)}{P(w)P(c)}\right) \tag*{(1)}

where the probabilities are estimated from the co-occurrence counts in the corresponding matrix. Negative PMI values are replaced with zero, producing non-negative association weights. Note that we only retain dependency-conditioned items. whose marginal frequency is at least 100 to ensure stability of the semantic space.

Both count-based configurations used dependency-conditioned units as rows. PPMI-5000 retained the 5000 context lemmas with the highest marginal frequencies (alphabetical tie-breaking); the PPMI probabilities were estimated after restricting the count matrix to those columns. PPMI-SVD instead computed PPMI over the full available context vocabulary before applying truncated SVD to 300 dimensions. Dependency-conditioned units with marginal frequency below 100 were excluded.

SGNS representations

Both SGNS configurations used word2vecf with 300 dimensions, 15 negative samples, 5 iterations, and 10 threads. We first restricted the reversed co-occurrence counts to dependency-conditioned units with marginal frequency at least 100, recalculated both vocabulary marginals on the retained pairs, and expanded counts into repeated training events. The same retained pairs were used in both configurations: original-context SGNS treated lemmas as targets and lemma–relation units as contexts, whereas reversed SGNS swapped these roles. Negative samples followed a smoothed unigram distribution with exponent 0.75.

Appendix B - Global Geometry Experiment

Exploratory Analysis for Global Structure

Relation centroid analysis

We ranked lemma–relation types by frequency and selected the top 10,000. These types spanned 73 of the 84 directed relation labels, giving a cap of ⌊10000/73⌋=136\lfloor10000/73 \rfloor= 136 types per relation. Relations with fewer than 10 units are excluded, while each remaining relation contributes up to its 136 highest-frequency types. This yielded 4,751 types, of which 4,702 remained after restricting lemmas to those represented in at least 2 relations. We used t-SNE for an illustrative projection of the selected vectors.

We then calculate the mean embedding vector for each dependency relation and examine the cosine similarities between the resulting relation centroids through HAC. In Figure 3, there is a clear region where the POS of the context is predominantly Noun-like and another region where the POS of the context is predominantly Verb-like. This pattern is consistent across different semantic spaces.

Lexically centred relation analysis

Starting from the lemma–relation types selected in section B.1.1, we retrieved all available relation variants for each lemma from the full embedding, retaining only lemmas represented in at least 2 relations as lemmas occurring in only 1 relation would yield 0 residual vectors after centring.

For a lemma ww, its baseline was calculated over all available relations DwD_w:

μw=1∣Dw∣∑d∈Dwvw,d\mu_w = \frac{1}{|D_w|} \sum_{d \in D_w} \mathbf{v}_{w,d}

We then subtracted this baseline from each selected lemma–relation vector to obtain a residual indicating its displacement from the lemma’s average position:

rw,d=vw,d−μw\mathbf{r}_{w,d} = \mathbf{v}_{w,d} - \mu_w

Finally, for each dependency relation dd, we averaged the residuals of its selected lemmas SdS_d to obtain a relation-level displacement vector:

cd=1∣Sd∣∑w∈Sdrw,d\mathbf{c}_d = \frac{1}{|S_d|} \sum_{w \in S_d} \mathbf{r}_{w,d}

Cosine distances between these centroids were used for HAC (Figure 2). Similar centroid directions indicate similar average within-lemma shifts across relations.

POS Distributions and Embedding Geometry

Appendix C - Local Geometry Experiment

Shared-Vocabulary Coverage and Relation-Pair Selection

The 84 directed dependency-relation labels yield (842)=3,486\binom{84}{2} = 3,486 possible unordered relation pairs. However, most relation pairs share too few usable lemmas to support comparisons over large vocabularies (Figure 4). We decided to retain relation pairs with at least 2000 shared lemmas, producing a fixed set of 112 relation pairs.

Distribution of shared-lemma counts and the percentage of relation pairs retained at each minimum vocabulary size

Figure 4. Distribution of shared-lemma counts and the percentage of relation pairs retained at each minimum vocabulary size.

Cross-Relation Geometric Consistency

Shared-vocabulary geometric consistency We identify lemmas shared by each eligible pair of relations, then rank the shared lemmas by aggregated corpus frequency and retain the requested number NN. Within each relation, pairwise cosine similarities are calculated for all selected lemmas. Spearman correlation is then used to compare the resulting similarity structures between relations. Higher correlations indicate greater consistency, whereas lower correlations suggest stronger geometric reorganisation across dependency relations. We repeat the calculation at NN = 10, 20, 50, 100, 200, 500, 1000, 1100, 1200, 1500, 2000, while retaining the same 112 relation pairs. The reported cross-space values in Table 8 are macro-averages of the 112 relation-pair correlations.

Note: Rev. = Reversed SGNS; Orig. = Original-context SGNS; SVD = PPMI-SVD (300d); PPMI = PPMI-5000.

Local neighbourhood consistency For each eligible relation pair, we select the same frequency-ranked shared vocabulary of NN lemmas. Each lemma is treated as a query, and its 10 nearest neighbours are retrieved independently under the 2 relations. We then calculate the Jaccard overlap between the resulting neighbour sets. Because some overlap can occur by chance, particularly when the candidate vocabulary is small, we estimate the expected overlap between 2 random sets of 10 neighbours drawn from the same N−1N - 1 candidates and apply the correction (Jobserved−Jexpected)/(1−Jexpected)(J_{\mathrm{observed}} - J_{\mathrm{expected}})/(1 - J_{\mathrm{expected}}). A corrected value of 0 indicates chance-level overlap, whereas a value of one indicates identical neighbour sets. The corrected scores are first averaged across query lemmas within each relation pair and subsequently macro-averaged across the fixed set of 112 relation pairs.

We repeat this procedure at NN = 200, 500, 1000, 1100, 1200, 1500, 2000. Smaller vocabulary sizes are excluded because a neighbour set of 10 represents a substantial proportion of the available candidates (for example, approximately 10% at NN = 100). This restricted candidate pool mechanically increases the probability of neighbour overlap and makes the Jaccard measure less discriminating, even after chance correction. Beginning at

NN = 200, the 10 neighbours constitute only about 5% of the candidate vocabulary, providing a more informative assessment of local-neighbourhood preservation. The macro-mean chance-corrected KNN Jaccard across 112 relation pairs are reported in table 3.

Cross-Method Comparison

Lexical semantic evaluation The WordSim-353 Evaluation is performed separately for each dependency relation. A word pair is included only when embeddings are available for both words under the same relation, and a relation–subset combination is reported only if at least 50% of the benchmark pairs are covered. Performance is measured using Spearman’s rank correlation between embedding-based cosine similarities and human judgements. For each correlation, we also report the number and proportion of matched pairs to make differences in vocabulary coverage explicit.

After applying the 50% coverage threshold, 14 dependency relations remain eligible for both subsets (Table 10). Coverage ranges from 53.2% to 87.3%. For every reported relation–subset combination, the number of matched pairs is identical across the 4 semantic spaces. Consequently, differences in performance across models cannot be attributed to differences in benchmark coverage.

Table 10: Coverage of the fourteen retained dependency relations in the WordSim subsets. Counts are identical across all 4 semantic spaces.

RelationSimilarityRelatedness
chi_compound115/203151/252
chi_conj131/203172/252
chi_nmod133/203188/252
chi_nsubj112/203148/252
chi_obj133/203178/252
chi_obl139/203192/252
pa_amod145/203195/252
pa_case166/203220/252
pa_cc108/203145/252
pa_compound119/203156/252
pa_conj121/203159/252
pa_det151/203197/252
pa_nmod120/203160/252
pa_punct159/203203/252

Table 10.

Cross-Method Representational Similarity Analysis (RSA) For each dependency relation, we select the 200 most frequent dependency-conditioned items shared across all 4 representations, using their relation-specific frequencies. Relations containing fewer than 200 shared items are excluded, resulting in 50 dependency relations. For each model mm and dependency relation rr, we construct a pairwise cosine-similarity matrix S(m,r)S^{(m,r)} over the 200 shared items. For each pair of representations (m1,m2)(m_1,m_2), RSA is calculated as the Spearman rank correlation between their corresponding similarity values:

RSA⁡r(m1,m2)=ρs(vec⁡△(S(m1,r)),vec⁡△(S(m2,r)))(2)\operatorname{RSA}_{r}(m_1,m_2) = \rho_s\left(\operatorname{vec}_{\triangle}\left(S^{(m_1,r)}\right), \operatorname{vec}_{\triangle}\left(S^{(m_2,r)}\right)\right) \tag*{(2)}

where vec⁡△\operatorname{vec}_{\triangle} extracts the upper-triangular elements of a matrix. Higher RSA values indicate greater agreement between models in the relative ordering of lexical similarities. 6 model-pair comparisons are performed for each of the 50 relations, yielding 300 relation-level RSA scores. These scores are subsequently summarised using their macro-mean, median, and standard deviation across relations.

Table 9: Spearman’s ρ\rho on the WordSim Similarity (Sim.) and Relatedness (Rel.) subsets across dependency relations. Bold values indicate the best-performing model for each relation and subset.

RelationSimilarityRelatedness
chi_compound.402.500.512.565.304.321.295.260
chi_conj.489.512.581.602.206.241.285.314
chi_nmod.422.445.452.502.084.168.187.204
chi_nsubj.489.527.500.468.192.285.222.249
chi_obj.502.497.551.611.179.188.155.251
chi_obl.445.536.495.520.218.274.312.252
pa_amod.505.473.445.480.107.152.126.224
pa_case.212.222.400.478.007.121-.033.149
pa_cc.085.161.122.045.007-.073.001-.052
pa_compound.387.454.494.525.180.239.380.278
pa_conj.502.522.540.566.324.311.271.296
pa_det.023-.082.175.222-.018-.133-.045.019
pa_nmod.485.479.481.413.168.185.293.244
pa_punct.107-.124.103.160-.020-.174.032.045

Table 9. Note: Rev. = Reversed SGNS; Orig. = Original-context SGNS; SVD = PPMI-SVD (300 dimensions); PPMI = PPMI-5000.

Table 11: Cross-method representational similarity across 50 dependency relations (N=200N = 200). Values are macro-level descriptive statistics of relation-specific Spearman correlations.

Model comparisonMean ρ\rhoMedian ρ\rhoSD
PPMI-SVD / PPMI-5000.791.802.122
Rev. / Orig. SGNS.774.820.140
Orig. SGNS / PPMI-SVD.654.721.245
Rev. SGNS / PPMI-SVD.610.682.244
Orig. SGNS / PPMI-5000.613.702.267
Rev. SGNS / PPMI-5000.581.668.260

Table 11.

Table 12: Dependency relations with the highest and lowest cross-family representational agreement. Values are mean Spearman correlations across 4 SGNS–PPMI comparisons.

Highest agreementLowest agreement
Relationρ\rhoRelationρ\rho
pa_conj.911pa_det-.029
chi_conj.909pa_mark.058
chi_nummod.890pa_cc.125
chi_nmod.856pa_aux.126
pa_xcomp.835pa_cop.135

Table 12.

Linguistic Interpretation

The following tables report 2 selected dependency relations for each of 30 lemmas. For each relation, the 5 nearest neighbours from each of the 4 models are reported in decreasing cosine-similarity order. Reading labels are qualitative interpretations rather than independently annotated gold senses; cases labelled weak do not show a clear semantic contrast in the selected relations.

ReadingRelationReversed SGNSOriginal-context SGNSPPMI-SVDPPMI-5000
bank: financialchi_compoundinsurance, mortgage, booking, bankruptcymortgage, gate, booking, wire, KGBgovernment, service, property, estate, councilgovernment, banking, service, company, Bank
bank: river edgechi_obltributary, estuary, foothill, valley, ravinevalley, estuary, Plain, Firth, shorehill, river, shore, valley, RiverRiver, town, area, shore, south
paper: materialchi_compoundflour, cardboard, coin, textile, chocolatecloth, flour, deposit, wool, cardboardcardboard, rubber, print, plastic, potterycloth, silk, plastic, textile, flour
paper: publicationchi_apposTimes, publication, News, Review, JournalTimes, Journal, publication, Review, journalnewspaper, publication, Times, Review, magazinenewspaper, pp., Times, News, Sun
channel: broadcastingchi_apposNBC, ABC, FM, f.m., RadioFM, f.m., NBC, ABC, TVnetwork, TV, ABC, f.m., RadioTV, ABC, NBC, affiliate, network
channel: passagechi_objcorridor, Channel, pipeline, log, roadwaypipeline, sand, frontier, waterway, runwaytrail, corridor, field, lane, canalroad, route, river, highway, line
market: economic sectorchi_compoundtourism, estate, sporting, commodity, lifestylemortgage, sector, resource, industry, tourismindustry, business, consumer, tourism, productindustry, retail, consumer, business, sale
market: trade/distributionpa_nmoddistributor, export, retailer, sell, utilityprice, sale, manufacturer, export, goodsgoods, manufacturer, seller, store, retailertrade, export, sale, city, port
branch: botanicalchi_conjline, trunk, shrub, stem, sectiontrunk, line, shrub, stem, basinline, route, Bridge, road, junctionCreek, trunk, root, line, Road
branch: corporatepa_nmod:possstock, merger, monopoly, subsidiary, franchisesubsidiary, stock, founder, network, franchisearchive, board, chairman, merger, assetprogram, headquarters, founder, department, subsidiary

Table 14. Within-POS nouns examples. 5 neighbours are shown in decreasing cosine-similarity order.

ReadingRelationReversed SGNSOriginal-context SGNSPPMI-SVDPPMI-5000
hold: host eventchi_acltake, contest, host, preside, benefitorganised, contest, host, preside, takehost, organised, attend, organize, celebratetake, host, attend, organised, call
hold: stancepa_ccompargue, establish, conclude, maintain, believeargue, rule, maintain, conclude, assertrule, contend, maintain, determine, establishargue, believe, suggest, claim, state
charge: accusationchi_objcomplaint, rumor, allegation, ultimatum, objectionbarrage, torpedo, warhead, allegation, complaintcase, call, complaint, order, demandclaim, allegation, accusation, attack, case
charge: financialpa_objfine, bribe, pay, station, alertdislodge, unload, bribe, outrun, slaughterpay, count, cost, fine, allocatepay, attack, cost, worth, collect
turn: sequencechi_oblstage, example, period, time, dayway, what, whole, process, relocationprocess, consequence, course, extent, effortwhich, time, result, end, period
turn: directionpa_advmodshift, accelerate, bend, angle, meandershift, bend, angle, tilt, swingpush, move, extend, travel, shiftpass, move, lead, run, make
see: propositional understandingpa_ccompknow, claim, believe, realize, addthink, tell, add, know, findknow, explain, sure, report, findfind, state, say, tell, believe
see: perceptionpa_obl:agentbelieve, view, watch, notice, thinkview, watch, believe, think, noticeview, regard, believe, admire, interpretview, watch, regard, receive, consider
find: discoverypa_nmodartefact, woodland, cache, artifact, topographyexcavation, artifact, dating, locality, dinosaurartefact, artifact, fragment, burial, excavationexcavation, discovery, remains, site, fossil
find: conclusionpa_ccompconclude, show, believe, suggest, statebelieve, prove, suggest, show, claimshow, suggest, claim, mean, concludeshow, believe, suggest, state, note

Table 15. Within-POS verbs examples. 5 neighbours are shown in decreasing cosine-similarity order.

ReadingRelationReversed SGNSOriginal-context SGNSPPMI-SVDPPMI-5000
hard: physicalchi_amodsoft, process, tougher, shiny, toughtough, tougher, slippery, tinged, flavorsoft, solid, melt, loose, coarsesoft, tough, solid, more, color
hard: difficultypa_ccompdifficult, easy, impossible, affect, hopeeasy, impossible, difficult, discuss, affectdifficult, impossible, easy, help, teachdifficult, easy, impossible, state, say
heavy: weightchi_amodlight, massed, torrential, cement, artillerylight, massed, torrential, cement, excesslight, explosive, less, minimal, morelight, more, additional, less, massive
heavy: intensity/mixedpa_oblstrong, severe, minor, cause, lightlight, minor, strong, wet, causestrong, severe, light, low, minimalcause, high, occur, produce, large
certain: specificitychi_amodvarious, such, many, different, samesuch, various, specific, particular, specifyparticular, various, same, different, suchspecific, various, different, such, many
certain: epistemicpa_ccompunlikely, question, claim, conclude, realizeallege, question, realize, observe, clearunlikely, likely, appear, uncertain, presumeknow, believe, suggest, claim, confirm
late: (weak)temporal chi_amodearly, previous, mid, recent, subsequentearly, mid, recent, subsequent, contemporaryearly, mid, mid-19th, mid-18th, mid-20thearly, subsequent, recent, previous, mid
responsible: causation/mixed (weak)pa_oblcontribute, involved, provide, result, requireneed, result, contribute, aid, essentialassist, contribute, aid, result, needresult, contribute, lead, involved, assist

Table 16. Within-POS adjectives examples. 5 neighbours are shown in decreasing cosine-similarity order.

ReadingRelationReversed SGNSOriginal-context SGNSPPMI-SVDPPMI-5000
file: documentchi_obje-mail, diary, inventory, email, Spiritdiary, gem, software, poison, Spiritssample, fragment, photograph, tape, photodocumentdata, information, map, tape
file: submit or lodgepa_objlodge, print, forward, invalidate, submitlodge, forward, retract, circulate, rescindissue, forward, submit, retract, appeallodge, submit, withdraw, dismiss, issue
match: sporting nounchi_nmodbout, game, fixture, final, tournamentgame, fixture, final, contest, boutround, final, win, tournament, contestgame, final, tournament, event, season
match: correspondpa_objdescribe, duplicate, sum, vary, fittingfit, project, duplicate, discern, fittingchange, alter, adjust, fit, displayhave, change, display, reflect, use
face: body partchi_objbreast, sand, fence, cylinder, trunkbreast, throat, paint, rock, sandeye, hair, head, breast, taileye, head, nose, hand, image
face: confrontpa_objovercome, fear, eliminate, avoid, overover, defeat, eliminate, confront, overcomeeliminate, defeat, beat, pit, challengedefeat, beat, avoid, eliminate, overcome
head: nounchi_compoundneck, assistant, thigh, toe, shoulderthigh, neck, wrist, rib, assistantshoulder, nose, tail, neck, faceassistant, back, knee, shoulder, body
head: leadpa_objchair, reorganize, curve, convene, centerreorganise, chair, reorganize, curve, centerchair, reorganize, restructure, reorganise, disbandchair, join, lead, organize, establish
chair: furniturechi_conjtable, treasurer, bench, lamp, laddertreasurer, bench, lamp, ladder, tablechairman, president, board, treasurer, tablepresident, professor, chairman, bench, leader
chair: presidechi_aclpreside, oversee, coordinate, prosecute, authorizepreside, oversee, appoint, coordinate, organisepreside, oversee, head, investigate, chargehead, task, appoint, preside, establish

Table 17. Cross-POS noun-centred examples. 5 neighbours are shown in decreasing cosine-similarity order.

ReadingRelationReversed SGNSOriginal-context SGNSPPMI-SVDPPMI-5000
record: nounchi_nmodtime, album, work, season, musicalbum, time, label, work, recordingrecording, label, album, title, workalbum, work, book, song, recording
record: verbpa_objrecording, remix, perform, co-produce, previewrecording, produce, perform, compare, co-producerecording, sang, perform, sing, sirecording, post, perform, report, produce
plant: organismchi_conjshrub, invertebrate, grasses, herb, fungiinvertebrate, grasses, shrub, fungi, vegetablegrasses, grass, tree, crop, flowerfungi, insect, tree, fruit, fish
plant: actionpa_objuproot, detonate, store, harvest, sprayharvest, smuggle, uproot, detonate, fabricateharvest, grow, chew, uproot, eatattach, store, detonate, destroy, burn
drive: machinerychi_compoundV8, horsepower, handheld, mw, plughorsepower, plug, mw, V8, axlegear, launch, plug, lift, turbinelaunch, propulsion, horsepower, engine, diesel
drive: propelpa_objpush, propel, dislodge, ambush, chokepropel, keep, stop, push, controlpush, force, ambush, keep, trapforce, push, stop, attack, keep
call: statementchi_objassertion, endorsement, signal, denial, pleamessage, edict, complaint, assertion, denialcomplaint, challenge, warning, suggestion, commentcomplaint, criticism, argument, request, signal
call: labelpa_xcompdub, term, label, consider, namedub, term, label, brand, namelabel, term, brand, consider, dubconsider, name, deem, title, find
watch: alertpa_compoundfirst, alert, advisory, forecasting, ninewarning, Seven, auction, Beta, Bootwarning, alert, relief, advisory, weekwarning, advisory, alert, forecasting, intensity
watch: observepa_objsee, tape, film, cheer, viewfilm, see, stop, cheer, filmingfilming, film, direct, air, broadcastsee, film, air, direct, call

Table 18. Cross-POS verb-centred examples. 5 neighbours are shown in decreasing cosine-similarity order.

ReadingRelationReversed SGNSOriginal-context SGNSPPMI-SVDPPMI-5000
fine: qualitychi_amodquality, coarse, exquisite, elegantsuperb, exquisite, elegant, quality, pleasing, colourfulpolished, brilliant, elegant, soft, staingood, famous, excellent, brilliant, beautiful
fine: penaltychi_objdividend, subsidy, Assent, penalty, taxdividend, Assent, condemnation, bribe, pensiontax, fee, sanction, suspension, writpenalty, tax, ban, fee, sanction
right: adjectivechi_amodleft, sore, leave, medial, wrongleft, leave, oblique, wrong, severleft, broken, sore, muscular, bruiseleft, leave, wrong, front, different
right: entitlementchi_objcopyright, liberty, privilege, dignity, citizenshipprivilege, dignity, liberty, patent, copyrightsovereignty, ownership, control, privilege, citizenshipprivilege, sovereignty, independence, freedom, autonomy
present: currentchi_amodcurrent, new, different, same, pre-existcurrent, serve, convenient, pre-exist, newcurrent, certain, typical, same, particularcurrent, new, different, certain, exist
present: verbalpa_objsee, compare, find, tout, viewsee, compare, address, find, acceptview, consider, compare, base, characterizeconsider, describe, introduce, show, discuss
mean: adjectivechi_amodblonde, average, superhuman, gentle, stereotypicalaverage, freezing, virtuous, gentle, unnaturalvary, measure, approximate, comparable, averageaverage, maximum, minimum, approximate, vary
mean: verbpa_ccompwrite, say, find, suggest, believeshow, find, determine, indicate, suggestshow, indicate, find, ensure, determineshow, suggest, believe, state, find
last: finalchi_amodfirst, second, previous, next, finalnext, final, previous, only, firstfirst, next, second, third, finalfirst, next, final, second, previous
last: endurepa_nsubjend, continue, begin, follow, proveend, begin, ensue, continue, commenceover, culminate, ensue, commence, progressend, result, begin, over, occur

Table 19. Cross-POS adjective-centred examples. 5 neighbours are shown in decreasing cosine-similarity order.

References

  1. [1]Marco Baroni, Georgiana Dinu, and Germán Kruszewski. 2014. Don’t count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 238–247, Baltimore, Maryland. Association for Computational Linguistics.DOI
  2. [2]Marco Baroni and Alessandro Lenci. 2010. Distributional Memory: A General Framework for Corpus-Based Semantics. Computational Linguistics, 36(4):673–721.DOI
  3. [3]Pablo Gamallo. 2017. Comparing explicit and predictive distributional semantic models endowed with syntactic contexts. Language Resources and Evaluation, 51(3):727–743.DOI
  4. [4]Gregory Grefenstette. 2007. Conquering Language: Using NLP on a Massive Scale to Build High Dimensional Language Models from the Web. In Computational Linguistics and Intelligent Text Processing, pages 35–49, Berlin, Heidelberg. Springer Berlin Heidelberg.DOI
  5. [5]Alexandros Komninos and Suresh Manandhar. 2016. Dependency Based Embeddings for Sentence Classification Tasks. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1490–1500, San Diego, California. Association for Computational Linguistics.DOI
  6. [6]Alessandro Lenci, Magnus Sahlgren, Patrick Jeuniaux, Amaru Cuba Gyllensten, and Martina Miliani. 2022. A comparative evaluation and analysis of three generations of Distributional Semantic Models. Language Resources and Evaluation, 56(4):1269–1313.arxiv.org/abs/2105.09825
  7. [7]Omer Levy and Yoav Goldberg. 2014a. Dependency-Based Word Embeddings. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 302–308, Baltimore, Maryland. Association for Computational Linguistics.DOI
  8. [8]Omer Levy and Yoav Goldberg. 2014b. Neural Word Embedding as Implicit Matrix Factorization. In Advances in Neural Information Processing Systems.
  9. [9]Omer Levy, Yoav Goldberg, and Ido Dagan. 2015. Improving Distributional Similarity with Lessons Learned from Word Embeddings. Transactions of the Association for Computational Linguistics, 3:211–225.DOI
  10. [10]Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. Pointer Sentinel Mixture Models. arXiv preprint. ArXiv:1609.07843 [cs.CL].arxiv.org/abs/1609.07843
  11. [11]Claire Mouton, Guillaume Pitel, Gaël de Chalendar, and Anne Vilnat. 2009. Unsupervised Word Sense Induction from Multiple Semantic Spaces with Locality Sensitive Hashing. In Proceedings of the International Conference RANLP-2009, pages 287–291, Borovets, Bulgaria. Association for Computational Linguistics.
  12. [12]Sebastian Pado and Mirella Lapata. 2007. Dependency-Based Construction of Semantic Space Models. Computational Linguistics, 33(2):161–199.DOI
  13. [13]Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020. Stanza: A python natural language processing toolkit for many human languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 101–108, Online. Association for Computational Linguistics.arxiv.org/abs/2003.07082
  14. [14]Martin Riedl and Chris Biemann. 2017. There’s no ‘Count or Predict’ but task-based selection for distributional models. In Proceedings of the 12th International Conference on Computational Semantics (IWCS) — Short papers.

Paper details

vlnc:W_w2z74wwv
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents